On Mon, 27 Dec 2021 at 14:38, Stefan Herbrechtsmeier <stefan@herbrechtsmeier.net> wrote:

> I tend to think that the best (and the hardest) option is to improve
> these tools so that they're usable inside do_fetch (e.g. fulfil the
> caching/reproducibility criteria for a bitbake fetcher), and the needed
> changes are acceptable to upstreams.

Is the fetcher really the problem? In all cases the input and output of
the package manager fetch task is well defined. In the npm case the
bitbake npmsw fetcher and my recipetool approach translate this
configuration into bitbake fetch and unpack commands.

The real problem is the different philosophy between OE and the package
manager. The package manager doesn't care about duplicate versions,
maintenance versions, version updates of indirect dependencies, license
compliance, CVE checks or dead code (examples, documentations, test,
...) and if they care every package manager have its own solution.

I feel that ship has sailed. The rest of the world has already chosen the self-contained bundling via custom language-specific package management. Keeping up with that via custom reimplementations of those tools will be a never ending maintenance burden; I'd rather directly use the tools that everyone else does. It's all open source and can be fixed.

Why C/C++ and Python doesn't fetch all its dependencies inside a single
recipe and why do we try to replace embedded dependencies? I think we
have good reasons for it and we shouldn't discard it for other languages.

C/C++ is kind of moving that way too. Meson already has support for it; so far it's optional, but it does exist. The reason you can still do manual dependency management there is that those environments are from a different era, and carry a lot of inertia.

Alex