From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from phobos.denx.de (phobos.denx.de [85.214.62.61]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 5D1C2C43334 for ; Wed, 20 Jul 2022 17:16:08 +0000 (UTC) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id EC9568021D; Wed, 20 Jul 2022 19:16:05 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=konsulko.com Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=konsulko.com header.i=@konsulko.com header.b="psX8Jdht"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id AA2D380192; Wed, 20 Jul 2022 19:16:03 +0200 (CEST) Received: from mail-qk1-x72f.google.com (mail-qk1-x72f.google.com [IPv6:2607:f8b0:4864:20::72f]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id 4A5FF80192 for ; Wed, 20 Jul 2022 19:16:00 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=konsulko.com Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=trini@konsulko.com Received: by mail-qk1-x72f.google.com with SMTP id m16so9909648qka.12 for ; Wed, 20 Jul 2022 10:16:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=konsulko.com; s=google; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=ND3eB02LTAIsASBxGf9SNHDs4W2J4UY6/fKJF40WXmk=; b=psX8Jdht0JVgVfF4qezbTO4eev4LcCjVz7KemIPvj2hoNbUU91JYbSQRl5dI3ELicw /calht6UEkJDqiL0eIs4d1hfmtTb053S7oyJBk+IoS6omDyTRLA8NdvngbA7m95RCRzw XJsHb4cvVb6dcVeWt/6gH9FfiJvGexWssj6GE= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=ND3eB02LTAIsASBxGf9SNHDs4W2J4UY6/fKJF40WXmk=; b=3I7F04o6Kj9WBEydLZKnWADkZyWxUtx5vO0fvIRExjIRvjz5YsLBt3dNNYiIUDp0K8 x3pLWqSt8KAablThd2VognvQD3f0qYL86tZAmMbIXM5vlrgQgaiC3GWadVQyLF5lirkb i85iUO03gYpc95mX0VKTF34iYX8NkGT2K6JTKBRjiZ/AjTVrgEr1SoR0kP+ZnpsqDN7K rrMuLenJ9tNnbEPCf1GPGV5CMR4J0BLBuT6t5/Ywfxg1SQZhILq4vKjxnNNBwHTnZQb7 T+Uz1TeFQVokkrbpWxzRX+M/LwzWhlKPgj5wA3ajn6TSUt5cJbef2tnffku1F1gMNtLN RzDw== X-Gm-Message-State: AJIora+k2+9h2Y537j7yEELPecMJVIvTGYHKc7L02cZGuRm8fM0iobxl qZyAxXBi98qe5UOlRSsyyCX7bg== X-Google-Smtp-Source: AGRyM1vNH6pg6h1zW4yEPnkRSwS4CzBa+6dwew2SG/Sisqdg0yRuwn/Vpf1UvT6087j6lsvwPuCblw== X-Received: by 2002:a05:620a:d96:b0:6a7:9300:2ce1 with SMTP id q22-20020a05620a0d9600b006a793002ce1mr24879317qkl.661.1658337358817; Wed, 20 Jul 2022 10:15:58 -0700 (PDT) Received: from bill-the-cat (cpe-65-184-195-139.ec.res.rr.com. [65.184.195.139]) by smtp.gmail.com with ESMTPSA id z12-20020ac8710c000000b0031d3d0b2a04sm12936528qto.9.2022.07.20.10.15.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 20 Jul 2022 10:15:58 -0700 (PDT) Date: Wed, 20 Jul 2022 13:15:56 -0400 From: Tom Rini To: Simon Glass Cc: U-Boot Mailing List Subject: Re: [PATCH 10/19] buildman: Incorporate the genboardscfg.py tool Message-ID: <20220720171556.GU1146598@bill-the-cat> References: <20220712010413.331984-1-sjg@chromium.org> <20220712010413.331984-11-sjg@chromium.org> <20220712213851.GS1146598@bill-the-cat> <20220713182117.GQ1146598@bill-the-cat> <20220718121143.GU1146598@bill-the-cat> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="WTgYrpBYIvdcLv1L" Content-Disposition: inline In-Reply-To: X-Clacks-Overhead: GNU Terry Pratchett X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.6 at phobos.denx.de X-Virus-Status: Clean --WTgYrpBYIvdcLv1L Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Jul 20, 2022 at 09:01:04AM -0600, Simon Glass wrote: > Hi Tom, >=20 > On Mon, 18 Jul 2022 at 06:11, Tom Rini wrote: > > > > On Thu, Jul 14, 2022 at 04:21:57AM -0600, Simon Glass wrote: > > > Hi Tom, > > > > > > On Wed, 13 Jul 2022 at 12:21, Tom Rini wrote: > > > > > > > > On Wed, Jul 13, 2022 at 09:28:06AM -0600, Simon Glass wrote: > > > > > Hi Tom, > > > > > > > > > > On Tue, 12 Jul 2022 at 15:38, Tom Rini wrote: > > > > > > > > > > > > On Mon, Jul 11, 2022 at 07:04:04PM -0600, Simon Glass wrote: > > > > > > > Bring this tool into buildman, so we don't have to run it sep= arately. The > > > > > > > board.cfg file is still produced as part of the build, to sav= e time when > > > > > > > doing another build in the same working directory. If it is o= ut of date > > > > > > > with respect to the Kconfig, it is updated. > > > > > > > > > > > > > > Time to regenerate on a recent single-thread machine is 4.6s = (1.3s on a > > > > > > > 32-thread machine), so we do need some sort of cache if we wa= nt buildman > > > > > > > to be useful on incremental builds. We could use Python's pic= kle format > > > > > > > but: > > > > > > > > > > > > > > - it seems useful to allow boards.cfg to be regenerated, at l= east for a > > > > > > > while, in case other tools use it > > > > > > > - it is possible to grep the file easily, e.g. to find boards= which use > > > > > > > a particular SoC (similar to 'buildman -nv ' > > > > > > > > > > > > While I don't think other tools still use boards.cfg, this will= make it > > > > > > easier to find out that I'm wrong. Perhaps once the CONFIG to = Kconfig > > > > > > migration is done we can move to just pickle'ing the data or si= milar > > > > > > since I find the main use of what was in boards.cfg can be figu= red out > > > > > > with some other git grep'ing, and in turn that's mainly for me = when > > > > > > trying to convert stuff. Thanks for doing this. > > > > > > > > > > Yes. I'm excited to hear that Kconfig migration might be done - a= ny > > > > > forecast as to when? > > > > > > > > Not yet. I'm auditing CONFIG_SYS_* now, with a notion to move > > > > everything that's not really configurable just out of CONFIG namesp= ace > > > > as the starting point. That'll drop us down to ~500 to migrate, wh= ich > > > > feels a bit less daunting. > > > > > > > > > One thing we could to is provide an option for buildman to spit o= ut > > > > > the various fields that go into boards.cfg > > > > > > > > Right. So I might not have said this before, but one reason I want= ed > > > > buildman to natively know kconfiglib and have everything was that w= hile > > > > we can do a lot of good matching on what to build, it would be amaz= ingly > > > > good to be able to say "build every platform with NVME_PCI set" (an= d if > > > > it's not too hard hex/int options with a specific value). > > > > > > Ah OK. At present moveconfig has the functionality to list the boards > > > that have particular options (-b and -f). It is expensive to build the > > > database though - over a minute on a 32-thread machine. So we would > > > have to cache it. Also just about any change would invalidate the > > > cache and I'm not sure if it possible to detect which changes have no > > > effect on which cache entries... > > > > Ah, maybe it will take some more thinking about then. Maybe an > > "advanced" match option, and also seeing how to have Azure generate the= db > > in one job and pass it as an artifact to every other job in the world > > build stage. Not an immediate need. >=20 > Well I suppose having that logic in moveconfig doesn't make a lot of > sense. So we could move it to buildman and have a way of creating the > database, as you say. But bear in mind that every commit being built > has the potential to change the Kconfig. It may be possible to hash > the files or detect changes using timestamps. Yeah, we can revisit this later on down the line. I don't think (and we don't do it today either, fwiw) we need to ensure the list of boards to build is right for each step of the commit, and might even be counter-productive, at least for my use case. Build the list of boards on top of tree, tell me how it changed over N commits (which commit caused the size change) is how I use things. Then the CI case is only for top of tree anyhow. --=20 Tom --WTgYrpBYIvdcLv1L Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQGzBAABCgAdFiEEGjx/cOCPqxcHgJu/FHw5/5Y0tywFAmLYOEkACgkQFHw5/5Y0 tywDgwwAnzdTq30NB60EwIPTk06Pak8IlJL8O2P8tlpWANu4NLlT02JjZ0xx06st ekp+lQg6V9NuNYuj1GaH+8Bis8HyBIQWT7xEVICKFm/hXr+qooYYMNr5MvRi9ZXv 6HlYEdIU6KO4C5zrQKf6/V7ylfe81b1BHQlCy+9efaJV4dglRE5w7O7AR7V/wvK6 INIwUeeQ1MzrOC4Ps143KdaqK9oWJU9+4tLWl+/hLFMoXeEsCb18s8D386OWdH2J MjXRA+65peQYRYmX0QLiQdYnTYRV6NBT932wVXHrmX7Yzu6gpjiMdgSMuAcgDWR6 4WPVHCOxfJ7zp1t/pgZF6K+7ndvAi7JUqxAWgiTssNsF+QLZq1XWxxTh7CTEs7lI dwlA+IfIXyr/0COWhXV3hRBCdeX2Q0EojHTmxDuzcw9CPJMQAl5ZFGdFe0rUEBLN aytEpMtZvptDJG02Qof/VMqIBHytHkRZOfLuu5gMKA5XEN91oFF+x6bnw7EOxPuV /Y0+AM2O =cvOt -----END PGP SIGNATURE----- --WTgYrpBYIvdcLv1L--