Re: [OE-core] [PATCH] base/patch: Disable network for unpack/patch/configure/compile/install

[Konrad Weihmann <kweihmann@outlook.com>]

On 27.12.21 13:54, Stefan Herbrechtsmeier wrote:
> Hi Konrad,
> 
> Am 25.12.21 um 21:43 schrieb Konrad Weihmann:
>> What I so far don't really get is why increase in parsing time is such 
>> a big deal.
>> I admit when we're talking about npm it's some kind of a drastic 
>> increase in recipes one would have to maintain, just because some 
>> random project decides to use a trillion dependencies instead of 
>> writing two or three lines of code more.
>>
>> Still I come to think this might be actually beneficial, as it shows 
>> how broken npm is from a distribution perspective - as it may be that 
>> some users actually start to access the situation when they are 
>> actually aware what monstrosity of a dep tree they are inheriting by 
>> just a single npm module.
>>
>> So this is mind - and I don't want to sound radical - I would rather 
>> abandon npm and go support in core than to sacrifice closing the one 
>> main loophole in core that is preventing true accessibility at the 
>> moment.
>> Which is uncontrolled fetching outside of the fetching task, as this 
>> invalidates everything you will get in the end in terms of licensing, 
>> quality reports that haven't been done as part of the build and even 
>> CVE checking becomes pretty much worthless if one is allowed to inject 
>> random code on the fly into the build - and in the end everything that 
>> can't be assessed outside of the build is pretty much non-existing for 
>> most of the assessments I had to work with.
>>
>> In the past I showed that npm and go can be made to work with these 
>> principles, even if they would introduce a different way of working 
>> and maybe the need for better tooling - *but* they can work with how 
>> oe-core works at the moment - to the expense of parsing time - and 
>> that's pretty much it from my point of view.
>>
>> Still the gains outweigh it by far in my opinion, as it would make all 
>> of that accessible by common tooling already in place - including true 
>> reproducibility and builtin quality reports.
>>
>> BTW I don't think rust and therefore cargo is heavily affected by it, 
>> as the cargo to bitbake scripting works kind in the way I would 
>> imagine for npm and go too - so far just npm and go are really really 
>> bad to handle in this case.
>>
>> I'm happy to help on such scripting and tooling, but I don't see much 
>> worth in either accepting the fact that "modern languages" have to 
>> include some not validatable "magic" (which then would mean allowing 
>> uncontrolled network access) or working around the fact that a 
>> trillion dependencies is still a trillion dependencies no matter how 
>> you put it ( :) )
> 
> I fully agree with you and would be happy to add a 'recursive' option to 
> devtool / recipetool to create multiple recipes on the fly instead of 
> one big recipe with more than 100 different projects inclusive duplicate 
> and different compatible versions.

That, yes - and a split between sources and recipes, I guess, is needed.
Like creation of a sharable inc-file for definition of sources (SRC_URI 
and dependencies) in addition to the actual recipe itself.
(One could also think about versioned inc-files here)
Nevertheless this is needed to untangle go (luckily npm is much more 
straight forward in this sense).

Like one has a go module "a" that consists of an inc-file, which defines 
the needed sources for "a" and the needed dependencies in terms of other 
go modules.
And then you have the recipe "a" which just includes this inc file and 
go.bbclass (plus the usual recipe specific meta data).
Now I could build recipe a or I could build any other recipe that 
depends on this go module with a version I could define (or tune).

I know that this isn't something that is not very welcome, as projects 
tend to define their versioned dependencies with lock-files (and every 
language/eco system in their very own way) - still I think the "freedom 
of choice" for a version is something that is a superb feature when 
using yocto - so I personally would rather ignore the maybe 
outdated/wrong information of a project as to not have it (this choice) 
at all.

So if one could make that happen with devtool (maybe using templates or 
so) I think the project would be prepared to fully support these 
"modern" languages without giving up on some of the principles like 
reproducibility

-- to get back to the responses from you other mail --

 > Is it possible to create a recipe for the source and a recipe for the 
 > binaries which depends on the source recipe? In this case the DEPENDS 
 > is always the source package and the RDEPENDS the binary package.
 > A BBCLASSEXTEND could be used to create the source recipe automatically.

That could be worth a shot, as I think this could actually work - 
basically the same idea as above but with recipes instead of inc-files.