From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 02CC72119683F for ; Thu, 6 Dec 2018 04:32:55 -0800 (PST) Subject: Re: [RFC v3 11/19] kunit: add Python libraries for handing KUnit config and kernel References: <20181128193636.254378-1-brendanhiggins@google.com> <20181128193636.254378-12-brendanhiggins@google.com> <841cf4ae-501b-05ae-5863-a51010709b67@ideasonboard.com> <20181204204701.GT28501@garbanzo.do-not-panic.com> From: Kieran Bingham Message-ID: Date: Thu, 6 Dec 2018 12:32:47 +0000 MIME-Version: 1.0 In-Reply-To: <20181204204701.GT28501@garbanzo.do-not-panic.com> Content-Language: en-US List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: kieran.bingham@ideasonboard.com Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: linux-nvdimm-bounces@lists.01.org Sender: "Linux-nvdimm" To: Luis Chamberlain , Brendan Higgins , Kent Overstreet , Matthew Wilcox , Eryu Guan , Eric Sandeen , jeffm@suse.com, Sasha Levin Cc: brakmo@fb.com, dri-devel@lists.freedesktop.org, linux-kselftest@vger.kernel.org, shuah@kernel.org, Rob Herring , Frank Rowand , linux-nvdimm@lists.01.org, richard@nod.at, Knut Omang , Felix Guo , Joel Stanley , jdike@addtoit.com, Petr Mladek , Tim.Bird@sony.com, Kees Cook , linux-um@lists.infradead.org, rostedt@goodmis.org, Julia Lawall , kunit-dev@googlegroups.com, fsdevel@vger.kernel.org, Greg KH , Linux Kernel Mailing List , Daniel Vetter , mpe@ellerman.id.au, joe@perches.com, khilman@baylibre.com List-ID: Hi Luis, On 04/12/2018 20:47, Luis Chamberlain wrote: > On Mon, Dec 03, 2018 at 03:48:15PM -0800, Brendan Higgins wrote: >> On Thu, Nov 29, 2018 at 5:54 AM Kieran Bingham >> wrote: >>> >>> Hi Brendan, >>> >>> Thanks again for this series! >>> >>> On 28/11/2018 19:36, Brendan Higgins wrote: >>>> The ultimate goal is to create minimal isolated test binaries; in the >>>> meantime we are using UML to provide the infrastructure to run tests, so >>>> define an abstract way to configure and run tests that allow us to >>>> change the context in which tests are built without affecting the user. >>>> This also makes pretty and dynamic error reporting, and a lot of other >>>> nice features easier. >>> >>> >>> I wonder if we could somehow generate a shared library object >>> 'libkernel' or 'libumlinux' from a UM configured set of headers and >>> objects so that we could create binary targets directly ? >> >> That's an interesting idea. I think it would be difficult to figure >> out exactly where to draw the line of what goes in there and what >> needs to be built specific to a test a priori. Of course, that leads >> into the biggest problem in general, needed to know what I need to >> build to test the thing that I want to test. >> >> Nevertheless, I could definitely imagine that being useful in a lot of cases. > > Whether or not we can abstract away the kernel into such a mechanism > with uml libraries is a good question worth exploring. > > Developers working upstream do modify their kernels a lot, so we'd have > to update such libraries quite a bit, but I think that's fine too. The > *real* value I think from the above suggestion would be enterprise / > mobile distros or stable kernel maintainers which have a static kernel > they need to support for a relatively *long time*, consider a 10 year > time frame. Running unit tests without qemu with uml and libraries for > respective kernels seems real worthy. I think any such library might be something generated by the kernel build system, so if someone makes substantial changes to a core component provided by the library - it can be up to them to build a corresponding userspace library as well. We could also consider to only provide *static* libraries rather than dynamic. So any one building some userspace tool / test with this would be required to compile against (the version of) the kernel they expect perhaps... - much like we expect modules to be compiled currently. And then the userspace binary would be sufficiently able to live it's life on it's own :) > The overhead for testing a unit test for said targets, *ideally*, would > just be to to reboot into the system with such libraries available, a > unit test would just look for the respective uname -r library and mimic > that kernel, much the same way enterprise distributions today rely on > having debugging symbols available to run against crash / gdb. Having > debug modules / kernel for crash requires such effort already, so this > would just be an extra layer of other prospect tests. Oh - although, yes - there are some good concepts there - but I'm a bit weary of how easy it would be to 'run' the said test against multiple kernel version libraries... there would be a lot of possible ABI conflicts perhaps. My main initial idea for a libumlinux is to provide infrastructure such as our linked-lists and other kernel formatting so that we can take kernel code directly to userspace for test and debug (assuming that there are no hardware dependencies or things that we can't mock out) I think all of this could complement kunit of course - this isn't suggesting an alternative implementation :-) > All ideaware for now, but the roadmap seems to be paving itself. I guess all great ideas start as ideaware somehow ... Now we just have to start the race to see who can tweak the kernel build system to produce an output library first :) (I won't be upset if I don't win the race) -- Regards -- Kieran _______________________________________________ Linux-nvdimm mailing list Linux-nvdimm@lists.01.org https://lists.01.org/mailman/listinfo/linux-nvdimm From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.1 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_PASS, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 96B1AC04EB8 for ; Thu, 6 Dec 2018 12:32:57 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 4C83D214C1 for ; Thu, 6 Dec 2018 12:32:57 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="p2aXBVAh" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4C83D214C1 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=ideasonboard.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729558AbeLFMcz (ORCPT ); Thu, 6 Dec 2018 07:32:55 -0500 Received: from perceval.ideasonboard.com ([213.167.242.64]:41492 "EHLO perceval.ideasonboard.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729409AbeLFMcz (ORCPT ); Thu, 6 Dec 2018 07:32:55 -0500 Received: from [192.168.0.21] (cpc89242-aztw30-2-0-cust488.18-1.cable.virginm.net [86.31.129.233]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id E42CD595; Thu, 6 Dec 2018 13:32:49 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1544099571; bh=8EPqX/bcxJ4pUl68DoOSb6fQ2wMC+0TYgUztnGpilWk=; h=Reply-To:Subject:To:Cc:References:From:Date:In-Reply-To:From; b=p2aXBVAhoE5M73ougePAn37zXon5ZeSHqF/HwG/M52LX111eXvdbX9s/VCSPhh38N 6gVobRldQDfT4C7T5EKQdXV99YMAX0cew6J44Y/H326/lN1tMlHWWxzRpQ7IuehKRB 9jvcw8TXzUEhJpQ8VjCDR4bCF4nHUCYCfs66J01I= Reply-To: kieran.bingham@ideasonboard.com Subject: Re: [RFC v3 11/19] kunit: add Python libraries for handing KUnit config and kernel To: Luis Chamberlain , Brendan Higgins , Kent Overstreet , Matthew Wilcox , Eryu Guan , Eric Sandeen , jeffm@suse.com, Sasha Levin Cc: Greg KH , Kees Cook , shuah@kernel.org, Joel Stanley , mpe@ellerman.id.au, joe@perches.com, brakmo@fb.com, rostedt@goodmis.org, Tim.Bird@sony.com, khilman@baylibre.com, Julia Lawall , linux-kselftest@vger.kernel.org, kunit-dev@googlegroups.com, Linux Kernel Mailing List , jdike@addtoit.com, richard@nod.at, linux-um@lists.infradead.org, Daniel Vetter , dri-devel@lists.freedesktop.org, Rob Herring , dan.j.williams@intel.com, linux-nvdimm@lists.01.org, Frank Rowand , Knut Omang , Felix Guo , Petr Mladek , fsdevel@vger.kernel.org References: <20181128193636.254378-1-brendanhiggins@google.com> <20181128193636.254378-12-brendanhiggins@google.com> <841cf4ae-501b-05ae-5863-a51010709b67@ideasonboard.com> <20181204204701.GT28501@garbanzo.do-not-panic.com> From: Kieran Bingham Openpgp: preference=signencrypt Autocrypt: addr=kieran.bingham@ideasonboard.com; keydata= xsFNBFYE/WYBEACs1PwjMD9rgCu1hlIiUA1AXR4rv2v+BCLUq//vrX5S5bjzxKAryRf0uHat V/zwz6hiDrZuHUACDB7X8OaQcwhLaVlq6byfoBr25+hbZG7G3+5EUl9cQ7dQEdvNj6V6y/SC rRanWfelwQThCHckbobWiQJfK9n7rYNcPMq9B8e9F020LFH7Kj6YmO95ewJGgLm+idg1Kb3C potzWkXc1xmPzcQ1fvQMOfMwdS+4SNw4rY9f07Xb2K99rjMwZVDgESKIzhsDB5GY465sCsiQ cSAZRxqE49RTBq2+EQsbrQpIc8XiffAB8qexh5/QPzCmR4kJgCGeHIXBtgRj+nIkCJPZvZtf Kr2EAbc6tgg6DkAEHJb+1okosV09+0+TXywYvtEop/WUOWQ+zo+Y/OBd+8Ptgt1pDRyOBzL8 RXa8ZqRf0Mwg75D+dKntZeJHzPRJyrlfQokngAAs4PaFt6UfS+ypMAF37T6CeDArQC41V3ko lPn1yMsVD0p+6i3DPvA/GPIksDC4owjnzVX9kM8Zc5Cx+XoAN0w5Eqo4t6qEVbuettxx55gq 8K8FieAjgjMSxngo/HST8TpFeqI5nVeq0/lqtBRQKumuIqDg+Bkr4L1V/PSB6XgQcOdhtd36 Oe9X9dXB8YSNt7VjOcO7BTmFn/Z8r92mSAfHXpb07YJWJosQOQARAQABzTBLaWVyYW4gQmlu Z2hhbSA8a2llcmFuLmJpbmdoYW1AaWRlYXNvbmJvYXJkLmNvbT7CwYAEEwEKACoCGwMFCwkI BwIGFQgJCgsCBBYCAwECHgECF4ACGQEFAlnDk/gFCQeA/YsACgkQoR5GchCkYf3X5w/9EaZ7 cnUcT6dxjxrcmmMnfFPoQA1iQXr/MXQJBjFWfxRUWYzjvUJb2D/FpA8FY7y+vksoJP7pWDL7 QTbksdwzagUEk7CU45iLWL/CZ/knYhj1I/+5LSLFmvZ/5Gf5xn2ZCsmg7C0MdW/GbJ8IjWA8 /LKJSEYH8tefoiG6+9xSNp1p0Gesu3vhje/GdGX4wDsfAxx1rIYDYVoX4bDM+uBUQh7sQox/ R1bS0AaVJzPNcjeC14MS226mQRUaUPc9250aj44WmDfcg44/kMsoLFEmQo2II9aOlxUDJ+x1 xohGbh9mgBoVawMO3RMBihcEjo/8ytW6v7xSF+xP4Oc+HOn7qebAkxhSWcRxQVaQYw3S9iZz 2iA09AXAkbvPKuMSXi4uau5daXStfBnmOfalG0j+9Y6hOFjz5j0XzaoF6Pln0jisDtWltYhP X9LjFVhhLkTzPZB/xOeWGmsG4gv2V2ExbU3uAmb7t1VSD9+IO3Km4FtnYOKBWlxwEd8qOFpS jEqMXURKOiJvnw3OXe9MqG19XdeENA1KyhK5rqjpwdvPGfSn2V+SlsdJA0DFsobUScD9qXQw OvhapHe3XboK2+Rd7L+g/9Ud7ZKLQHAsMBXOVJbufA1AT+IaOt0ugMcFkAR5UbBg5+dZUYJj 1QbPQcGmM3wfvuaWV5+SlJ+WeKIb8tbOwU0EVgT9ZgEQAM4o5G/kmruIQJ3K9SYzmPishRHV DcUcvoakyXSX2mIoccmo9BHtD9MxIt+QmxOpYFNFM7YofX4lG0ld8H7FqoNVLd/+a0yru5Cx adeZBe3qr1eLns10Q90LuMo7/6zJhCW2w+HE7xgmCHejAwuNe3+7yt4QmwlSGUqdxl8cgtS1 PlEK93xXDsgsJj/bw1EfSVdAUqhx8UQ3aVFxNug5OpoX9FdWJLKROUrfNeBE16RLrNrq2ROc iSFETpVjyC/oZtzRFnwD9Or7EFMi76/xrWzk+/b15RJ9WrpXGMrttHUUcYZEOoiC2lEXMSAF SSSj4vHbKDJ0vKQdEFtdgB1roqzxdIOg4rlHz5qwOTynueiBpaZI3PHDudZSMR5Fk6QjFooE XTw3sSl/km/lvUFiv9CYyHOLdygWohvDuMkV/Jpdkfq8XwFSjOle+vT/4VqERnYFDIGBxaRx koBLfNDiiuR3lD8tnJ4A1F88K6ojOUs+jndKsOaQpDZV6iNFv8IaNIklTPvPkZsmNDhJMRHH Iu60S7BpzNeQeT4yyY4dX9lC2JL/LOEpw8DGf5BNOP1KgjCvyp1/KcFxDAo89IeqljaRsCdP 7WCIECWYem6pLwaw6IAL7oX+tEqIMPph/G/jwZcdS6Hkyt/esHPuHNwX4guqTbVEuRqbDzDI 2DJO5FbxABEBAAHCwWUEGAEKAA8CGwwFAlnDlGsFCQeA/gIACgkQoR5GchCkYf1yYRAAq+Yo nbf9DGdK1kTAm2RTFg+w9oOp2Xjqfhds2PAhFFvrHQg1XfQR/UF/SjeUmaOmLSczM0s6XMeO VcE77UFtJ/+hLo4PRFKm5X1Pcar6g5m4xGqa+Xfzi9tRkwC29KMCoQOag1BhHChgqYaUH3yo UzaPwT/fY75iVI+yD0ih/e6j8qYvP8pvGwMQfrmN9YB0zB39YzCSdaUaNrWGD3iCBxg6lwSO LKeRhxxfiXCIYEf3vwOsP3YMx2JkD5doseXmWBGW1U0T/oJF+DVfKB6mv5UfsTzpVhJRgee7 4jkjqFq4qsUGxcvF2xtRkfHFpZDbRgRlVmiWkqDkT4qMA+4q1y/dWwshSKi/uwVZNycuLsz+ +OD8xPNCsMTqeUkAKfbD8xW4LCay3r/dD2ckoxRxtMD9eOAyu5wYzo/ydIPTh1QEj9SYyvp8 O0g6CpxEwyHUQtF5oh15O018z3ZLztFJKR3RD42VKVsrnNDKnoY0f4U0z7eJv2NeF8xHMuiU RCIzqxX1GVYaNkKTnb/Qja8hnYnkUzY1Lc+OtwiGmXTwYsPZjjAaDX35J/RSKAoy5wGo/YFA JxB1gWThL4kOTbsqqXj9GLcyOImkW0lJGGR3o/fV91Zh63S5TKnf2YGGGzxki+ADdxVQAm+Q sbsRB8KNNvVXBOVNwko86rQqF9drZuw= Organization: Ideas on Board Message-ID: Date: Thu, 6 Dec 2018 12:32:47 +0000 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.2.1 MIME-Version: 1.0 In-Reply-To: <20181204204701.GT28501@garbanzo.do-not-panic.com> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Hi Luis, On 04/12/2018 20:47, Luis Chamberlain wrote: > On Mon, Dec 03, 2018 at 03:48:15PM -0800, Brendan Higgins wrote: >> On Thu, Nov 29, 2018 at 5:54 AM Kieran Bingham >> wrote: >>> >>> Hi Brendan, >>> >>> Thanks again for this series! >>> >>> On 28/11/2018 19:36, Brendan Higgins wrote: >>>> The ultimate goal is to create minimal isolated test binaries; in the >>>> meantime we are using UML to provide the infrastructure to run tests, so >>>> define an abstract way to configure and run tests that allow us to >>>> change the context in which tests are built without affecting the user. >>>> This also makes pretty and dynamic error reporting, and a lot of other >>>> nice features easier. >>> >>> >>> I wonder if we could somehow generate a shared library object >>> 'libkernel' or 'libumlinux' from a UM configured set of headers and >>> objects so that we could create binary targets directly ? >> >> That's an interesting idea. I think it would be difficult to figure >> out exactly where to draw the line of what goes in there and what >> needs to be built specific to a test a priori. Of course, that leads >> into the biggest problem in general, needed to know what I need to >> build to test the thing that I want to test. >> >> Nevertheless, I could definitely imagine that being useful in a lot of cases. > > Whether or not we can abstract away the kernel into such a mechanism > with uml libraries is a good question worth exploring. > > Developers working upstream do modify their kernels a lot, so we'd have > to update such libraries quite a bit, but I think that's fine too. The > *real* value I think from the above suggestion would be enterprise / > mobile distros or stable kernel maintainers which have a static kernel > they need to support for a relatively *long time*, consider a 10 year > time frame. Running unit tests without qemu with uml and libraries for > respective kernels seems real worthy. I think any such library might be something generated by the kernel build system, so if someone makes substantial changes to a core component provided by the library - it can be up to them to build a corresponding userspace library as well. We could also consider to only provide *static* libraries rather than dynamic. So any one building some userspace tool / test with this would be required to compile against (the version of) the kernel they expect perhaps... - much like we expect modules to be compiled currently. And then the userspace binary would be sufficiently able to live it's life on it's own :) > The overhead for testing a unit test for said targets, *ideally*, would > just be to to reboot into the system with such libraries available, a > unit test would just look for the respective uname -r library and mimic > that kernel, much the same way enterprise distributions today rely on > having debugging symbols available to run against crash / gdb. Having > debug modules / kernel for crash requires such effort already, so this > would just be an extra layer of other prospect tests. Oh - although, yes - there are some good concepts there - but I'm a bit weary of how easy it would be to 'run' the said test against multiple kernel version libraries... there would be a lot of possible ABI conflicts perhaps. My main initial idea for a libumlinux is to provide infrastructure such as our linked-lists and other kernel formatting so that we can take kernel code directly to userspace for test and debug (assuming that there are no hardware dependencies or things that we can't mock out) I think all of this could complement kunit of course - this isn't suggesting an alternative implementation :-) > All ideaware for now, but the roadmap seems to be paving itself. I guess all great ideas start as ideaware somehow ... Now we just have to start the race to see who can tweak the kernel build system to produce an output library first :) (I won't be upset if I don't win the race) -- Regards -- Kieran From mboxrd@z Thu Jan 1 00:00:00 1970 From: kieran.bingham at ideasonboard.com (Kieran Bingham) Date: Thu, 6 Dec 2018 12:32:47 +0000 Subject: [RFC v3 11/19] kunit: add Python libraries for handing KUnit config and kernel In-Reply-To: <20181204204701.GT28501@garbanzo.do-not-panic.com> References: <20181128193636.254378-1-brendanhiggins@google.com> <20181128193636.254378-12-brendanhiggins@google.com> <841cf4ae-501b-05ae-5863-a51010709b67@ideasonboard.com> <20181204204701.GT28501@garbanzo.do-not-panic.com> Message-ID: Hi Luis, On 04/12/2018 20:47, Luis Chamberlain wrote: > On Mon, Dec 03, 2018 at 03:48:15PM -0800, Brendan Higgins wrote: >> On Thu, Nov 29, 2018 at 5:54 AM Kieran Bingham >> wrote: >>> >>> Hi Brendan, >>> >>> Thanks again for this series! >>> >>> On 28/11/2018 19:36, Brendan Higgins wrote: >>>> The ultimate goal is to create minimal isolated test binaries; in the >>>> meantime we are using UML to provide the infrastructure to run tests, so >>>> define an abstract way to configure and run tests that allow us to >>>> change the context in which tests are built without affecting the user. >>>> This also makes pretty and dynamic error reporting, and a lot of other >>>> nice features easier. >>> >>> >>> I wonder if we could somehow generate a shared library object >>> 'libkernel' or 'libumlinux' from a UM configured set of headers and >>> objects so that we could create binary targets directly ? >> >> That's an interesting idea. I think it would be difficult to figure >> out exactly where to draw the line of what goes in there and what >> needs to be built specific to a test a priori. Of course, that leads >> into the biggest problem in general, needed to know what I need to >> build to test the thing that I want to test. >> >> Nevertheless, I could definitely imagine that being useful in a lot of cases. > > Whether or not we can abstract away the kernel into such a mechanism > with uml libraries is a good question worth exploring. > > Developers working upstream do modify their kernels a lot, so we'd have > to update such libraries quite a bit, but I think that's fine too. The > *real* value I think from the above suggestion would be enterprise / > mobile distros or stable kernel maintainers which have a static kernel > they need to support for a relatively *long time*, consider a 10 year > time frame. Running unit tests without qemu with uml and libraries for > respective kernels seems real worthy. I think any such library might be something generated by the kernel build system, so if someone makes substantial changes to a core component provided by the library - it can be up to them to build a corresponding userspace library as well. We could also consider to only provide *static* libraries rather than dynamic. So any one building some userspace tool / test with this would be required to compile against (the version of) the kernel they expect perhaps... - much like we expect modules to be compiled currently. And then the userspace binary would be sufficiently able to live it's life on it's own :) > The overhead for testing a unit test for said targets, *ideally*, would > just be to to reboot into the system with such libraries available, a > unit test would just look for the respective uname -r library and mimic > that kernel, much the same way enterprise distributions today rely on > having debugging symbols available to run against crash / gdb. Having > debug modules / kernel for crash requires such effort already, so this > would just be an extra layer of other prospect tests. Oh - although, yes - there are some good concepts there - but I'm a bit weary of how easy it would be to 'run' the said test against multiple kernel version libraries... there would be a lot of possible ABI conflicts perhaps. My main initial idea for a libumlinux is to provide infrastructure such as our linked-lists and other kernel formatting so that we can take kernel code directly to userspace for test and debug (assuming that there are no hardware dependencies or things that we can't mock out) I think all of this could complement kunit of course - this isn't suggesting an alternative implementation :-) > All ideaware for now, but the roadmap seems to be paving itself. I guess all great ideas start as ideaware somehow ... Now we just have to start the race to see who can tweak the kernel build system to produce an output library first :) (I won't be upset if I don't win the race) -- Regards -- Kieran From mboxrd@z Thu Jan 1 00:00:00 1970 From: kieran.bingham@ideasonboard.com (Kieran Bingham) Date: Thu, 6 Dec 2018 12:32:47 +0000 Subject: [RFC v3 11/19] kunit: add Python libraries for handing KUnit config and kernel In-Reply-To: <20181204204701.GT28501@garbanzo.do-not-panic.com> References: <20181128193636.254378-1-brendanhiggins@google.com> <20181128193636.254378-12-brendanhiggins@google.com> <841cf4ae-501b-05ae-5863-a51010709b67@ideasonboard.com> <20181204204701.GT28501@garbanzo.do-not-panic.com> Message-ID: Content-Type: text/plain; charset="UTF-8" Message-ID: <20181206123247.1s6iyriR_BXKPD1IgNwuWJLIZ58T7UfB38sQZXSeR8g@z> Hi Luis, On 04/12/2018 20:47, Luis Chamberlain wrote: > On Mon, Dec 03, 2018@03:48:15PM -0800, Brendan Higgins wrote: >> On Thu, Nov 29, 2018 at 5:54 AM Kieran Bingham >> wrote: >>> >>> Hi Brendan, >>> >>> Thanks again for this series! >>> >>> On 28/11/2018 19:36, Brendan Higgins wrote: >>>> The ultimate goal is to create minimal isolated test binaries; in the >>>> meantime we are using UML to provide the infrastructure to run tests, so >>>> define an abstract way to configure and run tests that allow us to >>>> change the context in which tests are built without affecting the user. >>>> This also makes pretty and dynamic error reporting, and a lot of other >>>> nice features easier. >>> >>> >>> I wonder if we could somehow generate a shared library object >>> 'libkernel' or 'libumlinux' from a UM configured set of headers and >>> objects so that we could create binary targets directly ? >> >> That's an interesting idea. I think it would be difficult to figure >> out exactly where to draw the line of what goes in there and what >> needs to be built specific to a test a priori. Of course, that leads >> into the biggest problem in general, needed to know what I need to >> build to test the thing that I want to test. >> >> Nevertheless, I could definitely imagine that being useful in a lot of cases. > > Whether or not we can abstract away the kernel into such a mechanism > with uml libraries is a good question worth exploring. > > Developers working upstream do modify their kernels a lot, so we'd have > to update such libraries quite a bit, but I think that's fine too. The > *real* value I think from the above suggestion would be enterprise / > mobile distros or stable kernel maintainers which have a static kernel > they need to support for a relatively *long time*, consider a 10 year > time frame. Running unit tests without qemu with uml and libraries for > respective kernels seems real worthy. I think any such library might be something generated by the kernel build system, so if someone makes substantial changes to a core component provided by the library - it can be up to them to build a corresponding userspace library as well. We could also consider to only provide *static* libraries rather than dynamic. So any one building some userspace tool / test with this would be required to compile against (the version of) the kernel they expect perhaps... - much like we expect modules to be compiled currently. And then the userspace binary would be sufficiently able to live it's life on it's own :) > The overhead for testing a unit test for said targets, *ideally*, would > just be to to reboot into the system with such libraries available, a > unit test would just look for the respective uname -r library and mimic > that kernel, much the same way enterprise distributions today rely on > having debugging symbols available to run against crash / gdb. Having > debug modules / kernel for crash requires such effort already, so this > would just be an extra layer of other prospect tests. Oh - although, yes - there are some good concepts there - but I'm a bit weary of how easy it would be to 'run' the said test against multiple kernel version libraries... there would be a lot of possible ABI conflicts perhaps. My main initial idea for a libumlinux is to provide infrastructure such as our linked-lists and other kernel formatting so that we can take kernel code directly to userspace for test and debug (assuming that there are no hardware dependencies or things that we can't mock out) I think all of this could complement kunit of course - this isn't suggesting an alternative implementation :-) > All ideaware for now, but the roadmap seems to be paving itself. I guess all great ideas start as ideaware somehow ... Now we just have to start the race to see who can tweak the kernel build system to produce an output library first :) (I won't be upset if I don't win the race) -- Regards -- Kieran From mboxrd@z Thu Jan 1 00:00:00 1970 From: Kieran Bingham Subject: Re: [RFC v3 11/19] kunit: add Python libraries for handing KUnit config and kernel Date: Thu, 6 Dec 2018 12:32:47 +0000 Message-ID: References: <20181128193636.254378-1-brendanhiggins@google.com> <20181128193636.254378-12-brendanhiggins@google.com> <841cf4ae-501b-05ae-5863-a51010709b67@ideasonboard.com> <20181204204701.GT28501@garbanzo.do-not-panic.com> Reply-To: kieran.bingham@ideasonboard.com Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Return-path: Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by gabe.freedesktop.org (Postfix) with ESMTPS id 3E8876E05A for ; Thu, 6 Dec 2018 12:32:54 +0000 (UTC) In-Reply-To: <20181204204701.GT28501@garbanzo.do-not-panic.com> Content-Language: en-US List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" To: Luis Chamberlain , Brendan Higgins , Kent Overstreet , Matthew Wilcox , Eryu Guan , Eric Sandeen , jeffm@suse.com, Sasha Levin Cc: brakmo@fb.com, dri-devel@lists.freedesktop.org, linux-kselftest@vger.kernel.org, shuah@kernel.org, Frank Rowand , linux-nvdimm@lists.01.org, richard@nod.at, Knut Omang , Felix Guo , Joel Stanley , jdike@addtoit.com, Petr Mladek , Tim.Bird@sony.com, Kees Cook , linux-um@lists.infradead.org, rostedt@goodmis.org, Julia Lawall , dan.j.williams@intel.com, kunit-dev@googlegroups.com, fsdevel@vger.kernel.org, Greg KH , Linux Kernel Mailing List , mpe@ellerman.id.au, joe@perches.com, khilman@baylibre.com List-Id: dri-devel@lists.freedesktop.org SGkgTHVpcywKCk9uIDA0LzEyLzIwMTggMjA6NDcsIEx1aXMgQ2hhbWJlcmxhaW4gd3JvdGU6Cj4g T24gTW9uLCBEZWMgMDMsIDIwMTggYXQgMDM6NDg6MTVQTSAtMDgwMCwgQnJlbmRhbiBIaWdnaW5z IHdyb3RlOgo+PiBPbiBUaHUsIE5vdiAyOSwgMjAxOCBhdCA1OjU0IEFNIEtpZXJhbiBCaW5naGFt Cj4+IDxraWVyYW4uYmluZ2hhbUBpZGVhc29uYm9hcmQuY29tPiB3cm90ZToKPj4+Cj4+PiBIaSBC cmVuZGFuLAo+Pj4KPj4+IFRoYW5rcyBhZ2FpbiBmb3IgdGhpcyBzZXJpZXMhCj4+Pgo+Pj4gT24g MjgvMTEvMjAxOCAxOTozNiwgQnJlbmRhbiBIaWdnaW5zIHdyb3RlOgo+Pj4+IFRoZSB1bHRpbWF0 ZSBnb2FsIGlzIHRvIGNyZWF0ZSBtaW5pbWFsIGlzb2xhdGVkIHRlc3QgYmluYXJpZXM7IGluIHRo ZQo+Pj4+IG1lYW50aW1lIHdlIGFyZSB1c2luZyBVTUwgdG8gcHJvdmlkZSB0aGUgaW5mcmFzdHJ1 Y3R1cmUgdG8gcnVuIHRlc3RzLCBzbwo+Pj4+IGRlZmluZSBhbiBhYnN0cmFjdCB3YXkgdG8gY29u ZmlndXJlIGFuZCBydW4gdGVzdHMgdGhhdCBhbGxvdyB1cyB0bwo+Pj4+IGNoYW5nZSB0aGUgY29u dGV4dCBpbiB3aGljaCB0ZXN0cyBhcmUgYnVpbHQgd2l0aG91dCBhZmZlY3RpbmcgdGhlIHVzZXIu Cj4+Pj4gVGhpcyBhbHNvIG1ha2VzIHByZXR0eSBhbmQgZHluYW1pYyBlcnJvciByZXBvcnRpbmcs IGFuZCBhIGxvdCBvZiBvdGhlcgo+Pj4+IG5pY2UgZmVhdHVyZXMgZWFzaWVyLgo+Pj4KPj4+Cj4+ PiBJIHdvbmRlciBpZiB3ZSBjb3VsZCBzb21laG93IGdlbmVyYXRlIGEgc2hhcmVkIGxpYnJhcnkg b2JqZWN0Cj4+PiAnbGlia2VybmVsJyBvciAnbGlidW1saW51eCcgZnJvbSBhIFVNIGNvbmZpZ3Vy ZWQgc2V0IG9mIGhlYWRlcnMgYW5kCj4+PiBvYmplY3RzIHNvIHRoYXQgd2UgY291bGQgY3JlYXRl IGJpbmFyeSB0YXJnZXRzIGRpcmVjdGx5ID8KPj4KPj4gVGhhdCdzIGFuIGludGVyZXN0aW5nIGlk ZWEuIEkgdGhpbmsgaXQgd291bGQgYmUgZGlmZmljdWx0IHRvIGZpZ3VyZQo+PiBvdXQgZXhhY3Rs eSB3aGVyZSB0byBkcmF3IHRoZSBsaW5lIG9mIHdoYXQgZ29lcyBpbiB0aGVyZSBhbmQgd2hhdAo+ PiBuZWVkcyB0byBiZSBidWlsdCBzcGVjaWZpYyB0byBhIHRlc3QgYSBwcmlvcmkuIE9mIGNvdXJz ZSwgdGhhdCBsZWFkcwo+PiBpbnRvIHRoZSBiaWdnZXN0IHByb2JsZW0gaW4gZ2VuZXJhbCwgbmVl ZGVkIHRvIGtub3cgd2hhdCBJIG5lZWQgdG8KPj4gYnVpbGQgdG8gdGVzdCB0aGUgdGhpbmcgdGhh dCBJIHdhbnQgdG8gdGVzdC4KPj4KPj4gTmV2ZXJ0aGVsZXNzLCBJIGNvdWxkIGRlZmluaXRlbHkg aW1hZ2luZSB0aGF0IGJlaW5nIHVzZWZ1bCBpbiBhIGxvdCBvZiBjYXNlcy4KPiAKPiBXaGV0aGVy IG9yIG5vdCB3ZSBjYW4gYWJzdHJhY3QgYXdheSB0aGUga2VybmVsIGludG8gc3VjaCBhIG1lY2hh bmlzbQo+IHdpdGggdW1sIGxpYnJhcmllcyBpcyBhIGdvb2QgcXVlc3Rpb24gd29ydGggZXhwbG9y aW5nLgo+IAo+IERldmVsb3BlcnMgd29ya2luZyB1cHN0cmVhbSBkbyBtb2RpZnkgdGhlaXIga2Vy bmVscyBhIGxvdCwgc28gd2UnZCBoYXZlCj4gdG8gdXBkYXRlIHN1Y2ggbGlicmFyaWVzIHF1aXRl IGEgYml0LCBidXQgSSB0aGluayB0aGF0J3MgZmluZSB0b28uIFRoZQo+ICpyZWFsKiB2YWx1ZSBJ IHRoaW5rIGZyb20gdGhlIGFib3ZlIHN1Z2dlc3Rpb24gd291bGQgYmUgZW50ZXJwcmlzZSAvCj4g bW9iaWxlIGRpc3Ryb3Mgb3Igc3RhYmxlIGtlcm5lbCBtYWludGFpbmVycyB3aGljaCBoYXZlIGEg c3RhdGljIGtlcm5lbAo+IHRoZXkgbmVlZCB0byBzdXBwb3J0IGZvciBhIHJlbGF0aXZlbHkgKmxv bmcgdGltZSosIGNvbnNpZGVyIGEgMTAgeWVhcgo+IHRpbWUgZnJhbWUuIFJ1bm5pbmcgdW5pdCB0 ZXN0cyB3aXRob3V0IHFlbXUgd2l0aCB1bWwgYW5kIGxpYnJhcmllcyBmb3IKPiByZXNwZWN0aXZl IGtlcm5lbHMgc2VlbXMgcmVhbCB3b3J0aHkuCgoKSSB0aGluayBhbnkgc3VjaCBsaWJyYXJ5IG1p Z2h0IGJlIHNvbWV0aGluZyBnZW5lcmF0ZWQgYnkgdGhlIGtlcm5lbApidWlsZCBzeXN0ZW0sIHNv IGlmIHNvbWVvbmUgbWFrZXMgc3Vic3RhbnRpYWwgY2hhbmdlcyB0byBhIGNvcmUKY29tcG9uZW50 IHByb3ZpZGVkIGJ5IHRoZSBsaWJyYXJ5IC0gaXQgY2FuIGJlIHVwIHRvIHRoZW0gdG8gYnVpbGQg YQpjb3JyZXNwb25kaW5nIHVzZXJzcGFjZSBsaWJyYXJ5IGFzIHdlbGwuCgpXZSBjb3VsZCBhbHNv IGNvbnNpZGVyIHRvIG9ubHkgcHJvdmlkZSAqc3RhdGljKiBsaWJyYXJpZXMgcmF0aGVyIHRoYW4K ZHluYW1pYy4gU28gYW55IG9uZSBidWlsZGluZyBzb21lIHVzZXJzcGFjZSB0b29sIC8gdGVzdCB3 aXRoIHRoaXMgd291bGQKYmUgcmVxdWlyZWQgdG8gY29tcGlsZSBhZ2FpbnN0ICh0aGUgdmVyc2lv biBvZikgdGhlIGtlcm5lbCB0aGV5IGV4cGVjdApwZXJoYXBzLi4uIC0gbXVjaCBsaWtlIHdlIGV4 cGVjdCBtb2R1bGVzIHRvIGJlIGNvbXBpbGVkIGN1cnJlbnRseS4KCkFuZCB0aGVuIHRoZSB1c2Vy c3BhY2UgYmluYXJ5IHdvdWxkIGJlIHN1ZmZpY2llbnRseSBhYmxlIHRvIGxpdmUgaXQncwpsaWZl IG9uIGl0J3Mgb3duIDopCgoKPiBUaGUgb3ZlcmhlYWQgZm9yIHRlc3RpbmcgYSB1bml0IHRlc3Qg Zm9yIHNhaWQgdGFyZ2V0cywgKmlkZWFsbHkqLCB3b3VsZAo+IGp1c3QgYmUgdG8gdG8gcmVib290 IGludG8gdGhlIHN5c3RlbSB3aXRoIHN1Y2ggbGlicmFyaWVzIGF2YWlsYWJsZSwgYQo+IHVuaXQg dGVzdCB3b3VsZCBqdXN0IGxvb2sgZm9yIHRoZSByZXNwZWN0aXZlIHVuYW1lIC1yIGxpYnJhcnkg YW5kIG1pbWljCj4gdGhhdCBrZXJuZWwsIG11Y2ggdGhlIHNhbWUgd2F5IGVudGVycHJpc2UgZGlz dHJpYnV0aW9ucyB0b2RheSByZWx5IG9uCj4gaGF2aW5nIGRlYnVnZ2luZyBzeW1ib2xzIGF2YWls YWJsZSB0byBydW4gYWdhaW5zdCBjcmFzaCAvIGdkYi4gSGF2aW5nCj4gZGVidWcgbW9kdWxlcyAv IGtlcm5lbCBmb3IgY3Jhc2ggcmVxdWlyZXMgc3VjaCBlZmZvcnQgYWxyZWFkeSwgc28gdGhpcwo+ IHdvdWxkIGp1c3QgYmUgYW4gZXh0cmEgbGF5ZXIgb2Ygb3RoZXIgcHJvc3BlY3QgdGVzdHMuCgpP aCAtIGFsdGhvdWdoLCB5ZXMgLSB0aGVyZSBhcmUgc29tZSBnb29kIGNvbmNlcHRzIHRoZXJlIC0g YnV0IEknbSBhIGJpdAp3ZWFyeSBvZiBob3cgZWFzeSBpdCB3b3VsZCBiZSB0byAncnVuJyB0aGUg c2FpZCB0ZXN0IGFnYWluc3QgbXVsdGlwbGUKa2VybmVsIHZlcnNpb24gbGlicmFyaWVzLi4uIHRo ZXJlIHdvdWxkIGJlIGEgbG90IG9mIHBvc3NpYmxlIEFCSQpjb25mbGljdHMgcGVyaGFwcy4KCk15 IG1haW4gaW5pdGlhbCBpZGVhIGZvciBhIGxpYnVtbGludXggaXMgdG8gcHJvdmlkZSBpbmZyYXN0 cnVjdHVyZSBzdWNoCmFzIG91ciBsaW5rZWQtbGlzdHMgYW5kIG90aGVyIGtlcm5lbCBmb3JtYXR0 aW5nIHNvIHRoYXQgd2UgY2FuIHRha2UKa2VybmVsIGNvZGUgZGlyZWN0bHkgdG8gdXNlcnNwYWNl IGZvciB0ZXN0IGFuZCBkZWJ1ZyAoYXNzdW1pbmcgdGhhdAp0aGVyZSBhcmUgbm8gaGFyZHdhcmUg ZGVwZW5kZW5jaWVzIG9yIHRoaW5ncyB0aGF0IHdlIGNhbid0IG1vY2sgb3V0KQoKCkkgdGhpbmsg YWxsIG9mIHRoaXMgY291bGQgY29tcGxlbWVudCBrdW5pdCBvZiBjb3Vyc2UgLSB0aGlzIGlzbid0 CnN1Z2dlc3RpbmcgYW4gYWx0ZXJuYXRpdmUgaW1wbGVtZW50YXRpb24gOi0pCgoKPiBBbGwgaWRl YXdhcmUgZm9yIG5vdywgYnV0IHRoZSByb2FkbWFwIHNlZW1zIHRvIGJlIHBhdmluZyBpdHNlbGYu CgpJIGd1ZXNzIGFsbCBncmVhdCBpZGVhcyBzdGFydCBhcyBpZGVhd2FyZSBzb21laG93IC4uLgoK Tm93IHdlIGp1c3QgaGF2ZSB0byBzdGFydCB0aGUgcmFjZSB0byBzZWUgd2hvIGNhbiB0d2VhayB0 aGUga2VybmVsIGJ1aWxkCnN5c3RlbSB0byBwcm9kdWNlIGFuIG91dHB1dCBsaWJyYXJ5IGZpcnN0 IDopCgogKEkgd29uJ3QgYmUgdXBzZXQgaWYgSSBkb24ndCB3aW4gdGhlIHJhY2UpCgotLSAKUmVn YXJkcwotLQpLaWVyYW4KX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX18KZHJpLWRldmVsIG1haWxpbmcgbGlzdApkcmktZGV2ZWxAbGlzdHMuZnJlZWRlc2t0b3Au b3JnCmh0dHBzOi8vbGlzdHMuZnJlZWRlc2t0b3Aub3JnL21haWxtYW4vbGlzdGluZm8vZHJpLWRl dmVsCg==