[RFC-cargo-fetcher 0/3] Suggest a first step

From: "Andreas Müller" <schnitzeltony@gmail.com>
To: bitbake-devel@lists.openembedded.org
Cc: alex.kanavin@gmail.com, randy.macleod@windriver.com, ross.burton@arm.com
Subject: [RFC-cargo-fetcher 0/3] Suggest a first step
Date: Sun,  7 Feb 2021 16:01:44 +0100	[thread overview]
Message-ID: <20210207150147.31011-1-schnitzeltony@gmail.com> (raw)

This is a first version of a cargo fetcher. It mimics cargo fetch means
it creates cargo cache in ${DL_DIR}/cargo. As mentioned in comments the
proxy-thingy was stolen from npmsw for shrinkwrap files.

First approach was to parse Cargo.toml for projects that don't ship a
Cargo.lock (crates for rustlibs should do this) but that was skipped:
An unknown magic inside cargo detects which libs are avaiable: Blindly
followin -> download explodedc [1]

As proof of concept I did (no meta-rust in layers to avoid surprises):

* add a simple cargo-fetch.bbclass to oe-core [2]
* adapt a pure rust recipe 'spotifyd' to new fetcher [3-4]
* moved console to source of spotifyd and did
  > export CARGO_HOME="<DL_DIR>/cargo"
  > export CARGO_TARGET_DIR="<B>"
  > cargo build --frozen
  --> project was build without issues

Please review carefully because I am aware that:

* my python could be better
* the 'one cache for all in download area thingy' might cause issues I do not
  see yet

Am also aware that this is not more than a first step. Once this makes it in,
further TODOs are:

* Merge latest meta-rust -> oe-core (latest: There were some riscv adjustments
  recently - and I am not sure if these are in the poky-contrib of Alex/Ross)
* Add documentation / tests / devtool-support / ??

[1] https://github.com/schnitzeltony/mimic-cargo-fetcher/blob/master/test.py
[2] https://github.com/schnitzeltony/openembedded-core/blob/cargo-fetcher/meta/classes/cargo-fetch.bbclass
[3] https://github.com/schnitzeltony/meta-musicians/blob/cargo-fetcher/recipes-streaming/spotify/spotifyd.bb
[4] https://github.com/schnitzeltony/meta-musicians/blob/cargo-fetcher/recipes-streaming/spotify/spotifyd/Cargo.lock

Andreas Müller (3):
  Add a fetcher for cargo crate index
  Add parser for toml configuration files
  Add a fetcher for cargo crates

 lib/bb/fetch2/__init__.py   |    4 +
 lib/bb/fetch2/crateindex.py |  155 +++++
 lib/bb/fetch2/cratelock.py  |  268 +++++++++
 lib/toml/LICENSE            |   27 +
 lib/toml/README             |    8 +
 lib/toml/__init__.py        |   25 +
 lib/toml/__init__.pyi       |   15 +
 lib/toml/decoder.py         | 1057 +++++++++++++++++++++++++++++++++++
 lib/toml/decoder.pyi        |   52 ++
 lib/toml/encoder.py         |  304 ++++++++++
 lib/toml/encoder.pyi        |   34 ++
 lib/toml/ordered.py         |   15 +
 lib/toml/ordered.pyi        |    7 +
 lib/toml/source-origin      |    1 +
 lib/toml/tz.py              |   24 +
 lib/toml/tz.pyi             |    9 +
 16 files changed, 2005 insertions(+)
 create mode 100644 lib/bb/fetch2/crateindex.py
 create mode 100644 lib/bb/fetch2/cratelock.py
 create mode 100644 lib/toml/LICENSE
 create mode 100644 lib/toml/README
 create mode 100644 lib/toml/__init__.py
 create mode 100644 lib/toml/__init__.pyi
 create mode 100644 lib/toml/decoder.py
 create mode 100644 lib/toml/decoder.pyi
 create mode 100644 lib/toml/encoder.py
 create mode 100644 lib/toml/encoder.pyi
 create mode 100644 lib/toml/ordered.py
 create mode 100644 lib/toml/ordered.pyi
 create mode 100644 lib/toml/source-origin
 create mode 100644 lib/toml/tz.py
 create mode 100644 lib/toml/tz.pyi

-- 
2.26.2