* [Buildroot] [PATCH v3 01/11] support/download/dl-wrapper: add concept of download post-processing
2022-01-06 20:59 [Buildroot] [PATCH v3 00/11] Support for Cargo and Go vendoring Thomas Petazzoni
@ 2022-01-06 20:59 ` Thomas Petazzoni
2022-01-07 10:28 ` Yann E. MORIN
2022-01-06 20:59 ` [Buildroot] [PATCH v3 02/11] package/pkg-download.mk: add <pkg>_DOWNLOAD_POST_PROCESS variable Thomas Petazzoni
` (11 subsequent siblings)
12 siblings, 1 reply; 38+ messages in thread
From: Thomas Petazzoni @ 2022-01-06 20:59 UTC (permalink / raw)
To: James Hilliard, Christian Stewart, Buildroot List, Matt Weber,
Yann E. MORIN
Cc: Patrick Havelange, Thomas Petazzoni
In order to support package managers such as Cargo (Rust) or Go, we
want to run some custom logic after the main download, but before
packing the tarball and checking the hash.
To implement this, this commit introduces a concept of download
post-processing: if -p <something> is passed to the dl-wrapper, then
support/download/<something>-post-process will be called.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
---
support/download/dl-wrapper | 9 ++++++++-
1 file changed, 8 insertions(+), 1 deletion(-)
diff --git a/support/download/dl-wrapper b/support/download/dl-wrapper
index 3315bd410e..e6509e5671 100755
--- a/support/download/dl-wrapper
+++ b/support/download/dl-wrapper
@@ -25,7 +25,7 @@ main() {
local -a uris
# Parse our options; anything after '--' is for the backend
- while getopts ":c:d:D:o:n:N:H:rf:u:q" OPT; do
+ while getopts ":c:d:D:o:n:N:H:rf:u:qp:" OPT; do
case "${OPT}" in
c) cset="${OPTARG}";;
d) dl_dir="${OPTARG}";;
@@ -37,6 +37,7 @@ main() {
r) recurse="-r";;
f) filename="${OPTARG}";;
u) uris+=( "${OPTARG}" );;
+ p) post_process="${OPTARG}";;
q) quiet="-q";;
:) error "option '%s' expects a mandatory argument\n" "${OPTARG}";;
\?) error "unknown option '%s'\n" "${OPTARG}";;
@@ -135,6 +136,12 @@ main() {
continue
fi
+ if [ -n "${post_process}" ] ; then
+ ${OLDPWD}/support/download/${post_process}-post-process \
+ -o "${tmpf}" \
+ -n "${raw_base_name}"
+ fi
+
# cd back to free the temp-dir, so we can remove it later
cd "${OLDPWD}"
--
2.33.1
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot
^ permalink raw reply related [flat|nested] 38+ messages in thread* Re: [Buildroot] [PATCH v3 01/11] support/download/dl-wrapper: add concept of download post-processing
2022-01-06 20:59 ` [Buildroot] [PATCH v3 01/11] support/download/dl-wrapper: add concept of download post-processing Thomas Petazzoni
@ 2022-01-07 10:28 ` Yann E. MORIN
0 siblings, 0 replies; 38+ messages in thread
From: Yann E. MORIN @ 2022-01-07 10:28 UTC (permalink / raw)
To: Thomas Petazzoni
Cc: Matt Weber, Patrick Havelange, James Hilliard, Buildroot List
Thomas, All,
On 2022-01-06 21:59 +0100, Thomas Petazzoni spake thusly:
> In order to support package managers such as Cargo (Rust) or Go, we
> want to run some custom logic after the main download, but before
> packing the tarball and checking the hash.
>
> To implement this, this commit introduces a concept of download
> post-processing: if -p <something> is passed to the dl-wrapper, then
> support/download/<something>-post-process will be called.
>
> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
> ---
> support/download/dl-wrapper | 9 ++++++++-
> 1 file changed, 8 insertions(+), 1 deletion(-)
>
> diff --git a/support/download/dl-wrapper b/support/download/dl-wrapper
> index 3315bd410e..e6509e5671 100755
> --- a/support/download/dl-wrapper
> +++ b/support/download/dl-wrapper
> @@ -25,7 +25,7 @@ main() {
> local -a uris
>
> # Parse our options; anything after '--' is for the backend
> - while getopts ":c:d:D:o:n:N:H:rf:u:q" OPT; do
> + while getopts ":c:d:D:o:n:N:H:rf:u:qp:" OPT; do
> case "${OPT}" in
> c) cset="${OPTARG}";;
> d) dl_dir="${OPTARG}";;
> @@ -37,6 +37,7 @@ main() {
> r) recurse="-r";;
> f) filename="${OPTARG}";;
> u) uris+=( "${OPTARG}" );;
> + p) post_process="${OPTARG}";;
> q) quiet="-q";;
> :) error "option '%s' expects a mandatory argument\n" "${OPTARG}";;
> \?) error "unknown option '%s'\n" "${OPTARG}";;
> @@ -135,6 +136,12 @@ main() {
> continue
> fi
>
> + if [ -n "${post_process}" ] ; then
> + ${OLDPWD}/support/download/${post_process}-post-process \
I added double-quotes when expanding those variables.
Applied to master, thanks.
Regards,
Yann E. MORIN.
> + -o "${tmpf}" \
> + -n "${raw_base_name}"
> + fi
> +
> # cd back to free the temp-dir, so we can remove it later
> cd "${OLDPWD}"
>
> --
> 2.33.1
>
--
.-----------------.--------------------.------------------.--------------------.
| Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ |
| +33 561 099 427 `------------.-------: X AGAINST | \e/ There is no |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot
^ permalink raw reply [flat|nested] 38+ messages in thread
* [Buildroot] [PATCH v3 02/11] package/pkg-download.mk: add <pkg>_DOWNLOAD_POST_PROCESS variable
2022-01-06 20:59 [Buildroot] [PATCH v3 00/11] Support for Cargo and Go vendoring Thomas Petazzoni
2022-01-06 20:59 ` [Buildroot] [PATCH v3 01/11] support/download/dl-wrapper: add concept of download post-processing Thomas Petazzoni
@ 2022-01-06 20:59 ` Thomas Petazzoni
2022-01-07 10:35 ` Yann E. MORIN
2022-01-06 20:59 ` [Buildroot] [PATCH v3 03/11] support/download/post-process-helpers: add helper function for post process scripts Thomas Petazzoni
` (10 subsequent siblings)
12 siblings, 1 reply; 38+ messages in thread
From: Thomas Petazzoni @ 2022-01-06 20:59 UTC (permalink / raw)
To: James Hilliard, Christian Stewart, Buildroot List, Matt Weber,
Yann E. MORIN
Cc: Patrick Havelange, Thomas Petazzoni
This will allow packages to register than a download post-processing
is needed. Note that this variable is intentionally not documented: it
is an internal variable meant to be set by package infrastructures,
not directly by packages.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
---
package/pkg-download.mk | 1 +
1 file changed, 1 insertion(+)
diff --git a/package/pkg-download.mk b/package/pkg-download.mk
index 2527ba5c60..66a415cce0 100644
--- a/package/pkg-download.mk
+++ b/package/pkg-download.mk
@@ -117,6 +117,7 @@ define DOWNLOAD
-n '$($(2)_BASENAME_RAW)' \
-N '$($(2)_RAWNAME)' \
-o '$($(2)_DL_DIR)/$(notdir $(1))' \
+ $(if $($(2)_DOWNLOAD_POST_PROCESS),-p '$($(2)_DOWNLOAD_POST_PROCESS)') \
$(if $($(2)_GIT_SUBMODULES),-r) \
$(foreach uri,$(call DOWNLOAD_URIS,$(1),$(2)),-u $(uri)) \
$(QUIET) \
--
2.33.1
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot
^ permalink raw reply related [flat|nested] 38+ messages in thread* Re: [Buildroot] [PATCH v3 02/11] package/pkg-download.mk: add <pkg>_DOWNLOAD_POST_PROCESS variable
2022-01-06 20:59 ` [Buildroot] [PATCH v3 02/11] package/pkg-download.mk: add <pkg>_DOWNLOAD_POST_PROCESS variable Thomas Petazzoni
@ 2022-01-07 10:35 ` Yann E. MORIN
0 siblings, 0 replies; 38+ messages in thread
From: Yann E. MORIN @ 2022-01-07 10:35 UTC (permalink / raw)
To: Thomas Petazzoni
Cc: Matt Weber, Patrick Havelange, James Hilliard, Buildroot List
Thomas, All,
On 2022-01-06 21:59 +0100, Thomas Petazzoni spake thusly:
> This will allow packages to register than a download post-processing
> is needed. Note that this variable is intentionally not documented: it
> is an internal variable meant to be set by package infrastructures,
> not directly by packages.
>
> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Applied to master, thanks.
Regards,
Yann E. MORIN.
> ---
> package/pkg-download.mk | 1 +
> 1 file changed, 1 insertion(+)
>
> diff --git a/package/pkg-download.mk b/package/pkg-download.mk
> index 2527ba5c60..66a415cce0 100644
> --- a/package/pkg-download.mk
> +++ b/package/pkg-download.mk
> @@ -117,6 +117,7 @@ define DOWNLOAD
> -n '$($(2)_BASENAME_RAW)' \
> -N '$($(2)_RAWNAME)' \
> -o '$($(2)_DL_DIR)/$(notdir $(1))' \
> + $(if $($(2)_DOWNLOAD_POST_PROCESS),-p '$($(2)_DOWNLOAD_POST_PROCESS)') \
> $(if $($(2)_GIT_SUBMODULES),-r) \
> $(foreach uri,$(call DOWNLOAD_URIS,$(1),$(2)),-u $(uri)) \
> $(QUIET) \
> --
> 2.33.1
>
--
.-----------------.--------------------.------------------.--------------------.
| Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ |
| +33 561 099 427 `------------.-------: X AGAINST | \e/ There is no |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot
^ permalink raw reply [flat|nested] 38+ messages in thread
* [Buildroot] [PATCH v3 03/11] support/download/post-process-helpers: add helper function for post process scripts
2022-01-06 20:59 [Buildroot] [PATCH v3 00/11] Support for Cargo and Go vendoring Thomas Petazzoni
2022-01-06 20:59 ` [Buildroot] [PATCH v3 01/11] support/download/dl-wrapper: add concept of download post-processing Thomas Petazzoni
2022-01-06 20:59 ` [Buildroot] [PATCH v3 02/11] package/pkg-download.mk: add <pkg>_DOWNLOAD_POST_PROCESS variable Thomas Petazzoni
@ 2022-01-06 20:59 ` Thomas Petazzoni
2022-01-07 10:36 ` Yann E. MORIN
2022-01-06 20:59 ` [Buildroot] [PATCH v3 04/11] package/pkg-golang.mk: implement Go vendoring support Thomas Petazzoni
` (9 subsequent siblings)
12 siblings, 1 reply; 38+ messages in thread
From: Thomas Petazzoni @ 2022-01-06 20:59 UTC (permalink / raw)
To: James Hilliard, Christian Stewart, Buildroot List, Matt Weber,
Yann E. MORIN
Cc: Patrick Havelange, Thomas Petazzoni
download post process scripts will often need to unpack the source
code tarball, do some operation, and then repack it. In order to help
with this, post-process-helpers provide an unpack() function and a
repack() function.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
---
support/download/helpers | 22 ++++++++++++++++++++++
1 file changed, 22 insertions(+)
diff --git a/support/download/helpers b/support/download/helpers
index 40d5eea591..4c5eb466cc 100755
--- a/support/download/helpers
+++ b/support/download/helpers
@@ -72,5 +72,27 @@ mk_tar_gz() {
popd >/dev/null
}
+post_process_unpack() {
+ local dest="${1}"
+ local tarball="${2}"
+ local one_file
+
+ mkdir "${dest}"
+ tar -C "${dest}" --strip-components=1 -xf "${tarball}"
+ one_file="$(find "${dest}" -type f -print0 |LC_ALL=C sort -z |head -z -n1 |tr -d "\0")"
+ touch -r "${one_file}" "${dest}.timestamp"
+}
+
+post_process_repack() {
+ local in_dir="${1}"
+ local base_dir="${2}"
+ local out="${3}"
+ local date
+
+ date="@$(stat -c '%Y' "${in_dir}/${base_dir}.timestamp")"
+
+ mk_tar_gz "${in_dir}/${base_dir}" "${base_dir}" "${date}" "${out}"
+}
+
# Keep this line and the following as last lines in this file.
# vim: ft=bash
--
2.33.1
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot
^ permalink raw reply related [flat|nested] 38+ messages in thread* Re: [Buildroot] [PATCH v3 03/11] support/download/post-process-helpers: add helper function for post process scripts
2022-01-06 20:59 ` [Buildroot] [PATCH v3 03/11] support/download/post-process-helpers: add helper function for post process scripts Thomas Petazzoni
@ 2022-01-07 10:36 ` Yann E. MORIN
0 siblings, 0 replies; 38+ messages in thread
From: Yann E. MORIN @ 2022-01-07 10:36 UTC (permalink / raw)
To: Thomas Petazzoni
Cc: Matt Weber, Patrick Havelange, James Hilliard, Buildroot List
Thomas, All,
On 2022-01-06 21:59 +0100, Thomas Petazzoni spake thusly:
> download post process scripts will often need to unpack the source
> code tarball, do some operation, and then repack it. In order to help
> with this, post-process-helpers provide an unpack() function and a
> repack() function.
>
> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Applied to master, thanks.
Regards,
Yann E. MORIN.
> ---
> support/download/helpers | 22 ++++++++++++++++++++++
> 1 file changed, 22 insertions(+)
>
> diff --git a/support/download/helpers b/support/download/helpers
> index 40d5eea591..4c5eb466cc 100755
> --- a/support/download/helpers
> +++ b/support/download/helpers
> @@ -72,5 +72,27 @@ mk_tar_gz() {
> popd >/dev/null
> }
>
> +post_process_unpack() {
> + local dest="${1}"
> + local tarball="${2}"
> + local one_file
> +
> + mkdir "${dest}"
> + tar -C "${dest}" --strip-components=1 -xf "${tarball}"
> + one_file="$(find "${dest}" -type f -print0 |LC_ALL=C sort -z |head -z -n1 |tr -d "\0")"
> + touch -r "${one_file}" "${dest}.timestamp"
> +}
> +
> +post_process_repack() {
> + local in_dir="${1}"
> + local base_dir="${2}"
> + local out="${3}"
> + local date
> +
> + date="@$(stat -c '%Y' "${in_dir}/${base_dir}.timestamp")"
> +
> + mk_tar_gz "${in_dir}/${base_dir}" "${base_dir}" "${date}" "${out}"
> +}
> +
> # Keep this line and the following as last lines in this file.
> # vim: ft=bash
> --
> 2.33.1
>
--
.-----------------.--------------------.------------------.--------------------.
| Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ |
| +33 561 099 427 `------------.-------: X AGAINST | \e/ There is no |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot
^ permalink raw reply [flat|nested] 38+ messages in thread
* [Buildroot] [PATCH v3 04/11] package/pkg-golang.mk: implement Go vendoring support
2022-01-06 20:59 [Buildroot] [PATCH v3 00/11] Support for Cargo and Go vendoring Thomas Petazzoni
` (2 preceding siblings ...)
2022-01-06 20:59 ` [Buildroot] [PATCH v3 03/11] support/download/post-process-helpers: add helper function for post process scripts Thomas Petazzoni
@ 2022-01-06 20:59 ` Thomas Petazzoni
2022-01-09 11:49 ` Romain Naour
2022-01-06 20:59 ` [Buildroot] [PATCH v3 05/11] package/pkg-cargo.mk: introduce the cargo package infrastructure Thomas Petazzoni
` (8 subsequent siblings)
12 siblings, 1 reply; 38+ messages in thread
From: Thomas Petazzoni @ 2022-01-06 20:59 UTC (permalink / raw)
To: James Hilliard, Christian Stewart, Buildroot List, Matt Weber,
Yann E. MORIN
Cc: Patrick Havelange, Anisse Astier, Thomas Petazzoni
This commit introduces the download post-process script
support/download/go-post-process, and hooks it into the Go package
infrastructure.
The -modcacherw flag is added to ensure that the Go cache is
read/write, and can be deleted properly upon "make clean".
The <pkg>_LICENSE variable of golang packages is expanded with ",
vendored dependencies licenses probably not listed" as currently for
all packages, the licenses of the vendored dependencies are not taken
into account.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
---
package/pkg-golang.mk | 13 +++++++++++-
support/download/go-post-process | 35 ++++++++++++++++++++++++++++++++
2 files changed, 47 insertions(+), 1 deletion(-)
create mode 100755 support/download/go-post-process
diff --git a/package/pkg-golang.mk b/package/pkg-golang.mk
index d07242310d..35bcb1673b 100644
--- a/package/pkg-golang.mk
+++ b/package/pkg-golang.mk
@@ -42,12 +42,13 @@ define inner-golang-package
$(2)_BUILD_OPTS += \
-ldflags "$$($(2)_LDFLAGS)" \
+ -modcacherw \
-tags "$$($(2)_TAGS)" \
-trimpath \
-p $(PARALLEL_JOBS)
# Target packages need the Go compiler on the host.
-$(2)_DEPENDENCIES += host-go
+$(2)_DOWNLOAD_DEPENDENCIES += host-go
$(2)_BUILD_TARGETS ?= .
@@ -81,6 +82,16 @@ define $(2)_GEN_GOMOD
endef
$(2)_POST_PATCH_HOOKS += $(2)_GEN_GOMOD
+$(2)_DOWNLOAD_POST_PROCESS = go
+$(2)_DL_ENV = \
+ $(HOST_GO_COMMON_ENV) \
+ GOPROXY=direct \
+ BR_GOMOD=$$($(2)_GOMOD)
+
+# Due to vendoring, it is pretty likely that not all licenses are
+# listed in <pkg>_LICENSE.
+$(2)_LICENSE += , vendored dependencies licenses probably not listed
+
# Build step. Only define it if not already defined by the package .mk
# file.
ifndef $(2)_BUILD_CMDS
diff --git a/support/download/go-post-process b/support/download/go-post-process
new file mode 100755
index 0000000000..a6e74e6221
--- /dev/null
+++ b/support/download/go-post-process
@@ -0,0 +1,35 @@
+#!/usr/bin/env bash
+
+set -e
+
+. "${0%/*}/helpers"
+
+# Parse our options
+while getopts "n:o:" OPT; do
+ case "${OPT}" in
+ o) output="${OPTARG}";;
+ n) base_name="${OPTARG}";;
+ :) error "option '%s' expects a mandatory argument\n" "${OPTARG}";;
+ \?) error "unknown option '%s'\n" "${OPTARG}";;
+ esac
+done
+
+# Already vendored tarball, nothing to do
+if tar tf ${output} | grep -q "^[^/]*/vendor" ; then
+ exit 0
+fi
+
+post_process_unpack ${base_name} ${output}
+
+# Do the Go vendoring
+pushd ${base_name} > /dev/null
+
+# Generate go.mod if it doesn't exist
+if [ ! -f go.mod ] && [ -n "${BR_GOMOD}" ]; then
+ printf "module ${BR_GOMOD}\n" > go.mod
+fi
+
+go mod vendor -v -modcacherw
+popd > /dev/null
+
+post_process_repack $(pwd) ${base_name} ${output}
--
2.33.1
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot
^ permalink raw reply related [flat|nested] 38+ messages in thread* Re: [Buildroot] [PATCH v3 04/11] package/pkg-golang.mk: implement Go vendoring support
2022-01-06 20:59 ` [Buildroot] [PATCH v3 04/11] package/pkg-golang.mk: implement Go vendoring support Thomas Petazzoni
@ 2022-01-09 11:49 ` Romain Naour
0 siblings, 0 replies; 38+ messages in thread
From: Romain Naour @ 2022-01-09 11:49 UTC (permalink / raw)
To: Thomas Petazzoni, James Hilliard, Christian Stewart,
Buildroot List, Matt Weber, Yann E. MORIN
Cc: Patrick Havelange, Anisse Astier
Hello Thomas,
Le 06/01/2022 à 21:59, Thomas Petazzoni a écrit :
> This commit introduces the download post-process script
> support/download/go-post-process, and hooks it into the Go package
> infrastructure.
>
> The -modcacherw flag is added to ensure that the Go cache is
> read/write, and can be deleted properly upon "make clean".
>
> The <pkg>_LICENSE variable of golang packages is expanded with ",
> vendored dependencies licenses probably not listed" as currently for
> all packages, the licenses of the vendored dependencies are not taken
> into account.
>
> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
> ---
I'm testing the golang infra with telegraf package and I got an error while
downloading go modules:
"verifying collectd.org@v0.5.0: checksum mismatch
downloaded: h1:mRTLdljvxJNXPMMO9RSxf0PANDAqu/Tz+I6Dt6OjB28=
go.sum: h1:y4uFSAuOmeVhG3GCRa3/oH+ysePfO/+eGJNfd0Qa3d8=
SECURITY ERROR
This download does NOT match an earlier download recorded in go.sum.
The bits may have been replaced on the origin server, or an attacker may
have intercepted the download attempt.
For more information, see 'go help module-auth'."
It seems there is something wrong in go.sum file from telegraf... but we are in
download step not in patch step. It seems there is no way to fix locally (by
patching) this problem.
This is not the first time that infludb/telegraf releases are broken due to
change in git repo listed in go.sum.
Best regards,
Romain
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot
^ permalink raw reply [flat|nested] 38+ messages in thread
* [Buildroot] [PATCH v3 05/11] package/pkg-cargo.mk: introduce the cargo package infrastructure
2022-01-06 20:59 [Buildroot] [PATCH v3 00/11] Support for Cargo and Go vendoring Thomas Petazzoni
` (3 preceding siblings ...)
2022-01-06 20:59 ` [Buildroot] [PATCH v3 04/11] package/pkg-golang.mk: implement Go vendoring support Thomas Petazzoni
@ 2022-01-06 20:59 ` Thomas Petazzoni
2022-01-07 1:12 ` James Hilliard
` (3 more replies)
2022-01-06 20:59 ` [Buildroot] [PATCH v3 06/11] docs/manual/cargo: document the cargo-package infrastructure Thomas Petazzoni
` (7 subsequent siblings)
12 siblings, 4 replies; 38+ messages in thread
From: Thomas Petazzoni @ 2022-01-06 20:59 UTC (permalink / raw)
To: James Hilliard, Christian Stewart, Buildroot List, Matt Weber,
Yann E. MORIN
Cc: Patrick Havelange, Thomas Petazzoni
From: Patrick Havelange <patrick.havelange@essensium.com>
In order to be package agnostic, the install phase is now using cargo
instead of install. TARGET_CONFIGURE_OPTS is now also set when running
cargo in order to support cross compiling C code within cargo.
This commit also adds support/download/cargo-post-process to perform
the vendoring on Cargo packages.
The <pkg>_LICENSE variable of cargo packages is expanded with ",
vendored dependencies licenses probably not listed" as currently for
all packages, the licenses of the vendored dependencies are not taken
into account.
Signed-off-by: Patrick Havelange <patrick.havelange@essensium.com>
[Thomas: add support for host-cargo-package and vendoring]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
---
package/Makefile.in | 1 +
package/pkg-cargo.mk | 153 ++++++++++++++++++++++++++++
support/download/cargo-post-process | 38 +++++++
3 files changed, 192 insertions(+)
create mode 100644 package/pkg-cargo.mk
create mode 100755 support/download/cargo-post-process
diff --git a/package/Makefile.in b/package/Makefile.in
index dae7a859fd..0ca2a5844e 100644
--- a/package/Makefile.in
+++ b/package/Makefile.in
@@ -441,3 +441,4 @@ include package/pkg-waf.mk
include package/pkg-golang.mk
include package/pkg-meson.mk
include package/pkg-qmake.mk
+include package/pkg-cargo.mk
diff --git a/package/pkg-cargo.mk b/package/pkg-cargo.mk
new file mode 100644
index 0000000000..bf1436a86b
--- /dev/null
+++ b/package/pkg-cargo.mk
@@ -0,0 +1,153 @@
+################################################################################
+# Cargo package infrastructure
+#
+# This file implements an infrastructure that eases development of package
+# .mk files for Cargo packages. It should be used for all packages that use
+# Cargo as their build system.
+#
+# See the Buildroot documentation for details on the usage of this
+# infrastructure
+#
+# In terms of implementation, this Cargo infrastructure requires the .mk file
+# to only specify metadata information about the package: name, version,
+# download URL, etc.
+#
+# We still allow the package .mk file to override what the different steps
+# are doing, if needed. For example, if <PKG>_BUILD_CMDS is already defined,
+# it is used as the list of commands to perform to build the package,
+# instead of the default Cargo behaviour. The package can also define some
+# post operation hooks.
+#
+################################################################################
+
+################################################################################
+# inner-cargo-package -- defines how the configuration, compilation and
+# installation of a cargo package should be done, implements a few hooks
+# to tune the build process for cargo specifities and calls the generic
+# package infrastructure to generate the necessary make targets
+#
+# argument 1 is the lowercase package name
+# argument 2 is the uppercase package name, including a HOST_ prefix
+# for host packages
+# argument 3 is the uppercase package name, without the HOST_ prefix
+# for host packages
+# argument 4 is the type (target or host)
+################################################################################
+
+define inner-cargo-package
+
+# We need host-rustc to run cargo
+$(2)_DOWNLOAD_DEPENDENCIES += host-rustc
+
+$(2)_CARGO_ENV += \
+ CARGO_HOME=$$(HOST_DIR)/share/cargo \
+ __CARGO_TEST_CHANNEL_OVERRIDE_DO_NOT_USE_THIS="nightly" \
+ CARGO_TARGET_APPLIES_TO_HOST="false"
+
+$(2)_DOWNLOAD_POST_PROCESS = cargo
+$(2)_DL_ENV = CARGO_HOME=$$(HOST_DIR)/share/cargo
+
+# Due to vendoring, it is pretty likely that not all licenses are
+# listed in <pkg>_LICENSE.
+$(2)_LICENSE += , vendored dependencies licenses probably not listed
+
+# Note: in all the steps below, we "cd" into the build directory to
+# execute the "cargo" tool instead of passing $(@D)/Cargo.toml as the
+# manifest-path. Indeed while the latter seems to work, it in fact
+# breaks in subtle ways as the way cargo searches for its
+# configuration file is based (among other rules) on the current
+# directory. This means that if cargo is started outside of a package
+# directory, its configuration file will not be taken into account.
+#
+# Also, we pass:
+# * --offline to prevent cargo from downloading anything: all
+# dependencies should have been built by the download post
+# process logic
+# * --locked to force cargo to use the Cargo.lock file, which ensures
+# that a fixed set of dependency versions is used
+
+#
+# Build step. Only define it if not already defined by the package .mk
+# file.
+#
+ifndef $(2)_BUILD_CMDS
+ifeq ($(4),target)
+define $(2)_BUILD_CMDS
+ cd $$(@D) && \
+ $$(TARGET_MAKE_ENV) \
+ $$(TARGET_CONFIGURE_OPTS) \
+ $$($(2)_CARGO_ENV) \
+ cargo build \
+ --offline \
+ --target $$(RUSTC_TARGET_NAME) \
+ $$(if $$(BR2_ENABLE_DEBUG),--debug,--release) \
+ --manifest-path Cargo.toml \
+ --locked \
+ -Z target-applies-to-host \
+ $$($(2)_CARGO_BUILD_OPTS)
+endef
+else # ifeq ($(4),target)
+define $(2)_BUILD_CMDS
+ cd $$(@D) && \
+ $$(HOST_MAKE_ENV) \
+ RUSTFLAGS="$$(addprefix -C link-args=,$$(HOST_LDFLAGS))" \
+ $$($(2)_CARGO_ENV) \
+ cargo build \
+ --offline \
+ --release \
+ --manifest-path Cargo.toml \
+ --locked \
+ $$($(2)_CARGO_BUILD_OPTS)
+endef
+endif # ifeq ($(4),target)
+endif # ifndef $(2)_BUILD_CMDS
+
+#
+# Target installation step. Only define it if not already defined by
+# the package .mk file.
+#
+ifndef $(2)_INSTALL_TARGET_CMDS
+define $(2)_INSTALL_TARGET_CMDS
+ cd $$(@D) && \
+ $$(TARGET_MAKE_ENV) $$($(2)_CARGO_ENV) \
+ cargo install \
+ --target $$(RUSTC_TARGET_NAME) \
+ --offline \
+ --root $$(TARGET_DIR)/usr/ \
+ --bins \
+ --path ./ \
+ --force \
+ --locked \
+ $$($(2)_CARGO_INSTALL_OPTS)
+endef
+endif
+
+ifndef $(2)_INSTALL_CMDS
+define $(2)_INSTALL_CMDS
+ cd $$(@D) && \
+ $$(HOST_MAKE_ENV) \
+ RUSTFLAGS="$$(addprefix -C link-args=,$$(HOST_LDFLAGS))" \
+ $$($(2)_CARGO_ENV) \
+ cargo install \
+ --offline \
+ --root $$(HOST_DIR) \
+ --bins \
+ --path ./ \
+ --force \
+ --locked \
+ $$($(2)_CARGO_INSTALL_OPTS)
+endef
+endif
+
+# Call the generic package infrastructure to generate the necessary
+# make targets
+$(call inner-generic-package,$(1),$(2),$(3),$(4))
+
+endef
+
+################################################################################
+# cargo-package -- the target generator macro for Cargo packages
+################################################################################
+
+cargo-package = $(call inner-cargo-package,$(pkgname),$(call UPPERCASE,$(pkgname)),$(call UPPERCASE,$(pkgname)),target)
+host-cargo-package = $(call inner-cargo-package,host-$(pkgname),$(call UPPERCASE,host-$(pkgname)),$(call UPPERCASE,$(pkgname)),host)
diff --git a/support/download/cargo-post-process b/support/download/cargo-post-process
new file mode 100755
index 0000000000..25dbbedbfa
--- /dev/null
+++ b/support/download/cargo-post-process
@@ -0,0 +1,38 @@
+#!/usr/bin/env bash
+
+set -e
+
+. "${0%/*}/helpers"
+
+while getopts "n:o:" OPT; do
+ case "${OPT}" in
+ o) output="${OPTARG}";;
+ n) base_name="${OPTARG}";;
+ :) error "option '%s' expects a mandatory argument\n" "${OPTARG}";;
+ \?) error "unknown option '%s'\n" "${OPTARG}";;
+ esac
+done
+
+# Already vendored tarball, nothing to do
+if tar tf ${output} | grep -q "^[^/]*/VENDOR" ; then
+ exit 0
+fi
+
+post_process_unpack ${base_name} ${output}
+
+# Do the Cargo vendoring
+pushd ${base_name} > /dev/null
+cargo vendor --locked VENDOR
+
+# Create the local .cargo/config with vendor info
+mkdir -p .cargo/
+cat <<EOF >.cargo/config
+[source.crates-io]
+replace-with = "vendored-sources"
+
+[source.vendored-sources]
+directory = "VENDOR"
+EOF
+popd > /dev/null
+
+post_process_repack $(pwd) ${base_name} ${output}
--
2.33.1
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot
^ permalink raw reply related [flat|nested] 38+ messages in thread* Re: [Buildroot] [PATCH v3 05/11] package/pkg-cargo.mk: introduce the cargo package infrastructure
2022-01-06 20:59 ` [Buildroot] [PATCH v3 05/11] package/pkg-cargo.mk: introduce the cargo package infrastructure Thomas Petazzoni
@ 2022-01-07 1:12 ` James Hilliard
2022-01-07 9:52 ` Thomas Petazzoni
2022-01-07 10:05 ` Romain Naour
` (2 subsequent siblings)
3 siblings, 1 reply; 38+ messages in thread
From: James Hilliard @ 2022-01-07 1:12 UTC (permalink / raw)
To: Thomas Petazzoni
Cc: Matt Weber, Patrick Havelange, Yann E. MORIN, Buildroot List
On Thu, Jan 6, 2022 at 3:00 PM Thomas Petazzoni
<thomas.petazzoni@bootlin.com> wrote:
>
> From: Patrick Havelange <patrick.havelange@essensium.com>
>
> In order to be package agnostic, the install phase is now using cargo
> instead of install. TARGET_CONFIGURE_OPTS is now also set when running
> cargo in order to support cross compiling C code within cargo.
>
> This commit also adds support/download/cargo-post-process to perform
> the vendoring on Cargo packages.
>
> The <pkg>_LICENSE variable of cargo packages is expanded with ",
> vendored dependencies licenses probably not listed" as currently for
> all packages, the licenses of the vendored dependencies are not taken
> into account.
>
> Signed-off-by: Patrick Havelange <patrick.havelange@essensium.com>
> [Thomas: add support for host-cargo-package and vendoring]
> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
> ---
> package/Makefile.in | 1 +
> package/pkg-cargo.mk | 153 ++++++++++++++++++++++++++++
> support/download/cargo-post-process | 38 +++++++
> 3 files changed, 192 insertions(+)
> create mode 100644 package/pkg-cargo.mk
> create mode 100755 support/download/cargo-post-process
>
> diff --git a/package/Makefile.in b/package/Makefile.in
> index dae7a859fd..0ca2a5844e 100644
> --- a/package/Makefile.in
> +++ b/package/Makefile.in
> @@ -441,3 +441,4 @@ include package/pkg-waf.mk
> include package/pkg-golang.mk
> include package/pkg-meson.mk
> include package/pkg-qmake.mk
> +include package/pkg-cargo.mk
> diff --git a/package/pkg-cargo.mk b/package/pkg-cargo.mk
> new file mode 100644
> index 0000000000..bf1436a86b
> --- /dev/null
> +++ b/package/pkg-cargo.mk
> @@ -0,0 +1,153 @@
> +################################################################################
> +# Cargo package infrastructure
> +#
> +# This file implements an infrastructure that eases development of package
> +# .mk files for Cargo packages. It should be used for all packages that use
> +# Cargo as their build system.
> +#
> +# See the Buildroot documentation for details on the usage of this
> +# infrastructure
> +#
> +# In terms of implementation, this Cargo infrastructure requires the .mk file
> +# to only specify metadata information about the package: name, version,
> +# download URL, etc.
> +#
> +# We still allow the package .mk file to override what the different steps
> +# are doing, if needed. For example, if <PKG>_BUILD_CMDS is already defined,
> +# it is used as the list of commands to perform to build the package,
> +# instead of the default Cargo behaviour. The package can also define some
> +# post operation hooks.
> +#
> +################################################################################
> +
> +################################################################################
> +# inner-cargo-package -- defines how the configuration, compilation and
> +# installation of a cargo package should be done, implements a few hooks
> +# to tune the build process for cargo specifities and calls the generic
> +# package infrastructure to generate the necessary make targets
> +#
> +# argument 1 is the lowercase package name
> +# argument 2 is the uppercase package name, including a HOST_ prefix
> +# for host packages
> +# argument 3 is the uppercase package name, without the HOST_ prefix
> +# for host packages
> +# argument 4 is the type (target or host)
> +################################################################################
> +
> +define inner-cargo-package
> +
> +# We need host-rustc to run cargo
> +$(2)_DOWNLOAD_DEPENDENCIES += host-rustc
> +
> +$(2)_CARGO_ENV += \
> + CARGO_HOME=$$(HOST_DIR)/share/cargo \
> + __CARGO_TEST_CHANNEL_OVERRIDE_DO_NOT_USE_THIS="nightly" \
> + CARGO_TARGET_APPLIES_TO_HOST="false"
Maybe want to set these additional options as well so that packages
using cargo can
build with only the env variables set(in some cases at least I think),
still probably
need to pass some command line args but this should reduce the amount at least:
RUSTC_TARGET_TRIPLE = $(subst -,_,$(call UPPERCASE,$(RUSTC_TARGET_NAME)))
PKG_RUST_CARGO_ENV = \
CARGO_HOME=$(HOST_DIR)/share/cargo \
CARGO_BUILD_TARGET=$(RUSTC_TARGET_NAME) \
CARGO_INSTALL_ROOT=$(TARGET_DIR)/usr \
CARGO_TARGET_$(RUSTC_TARGET_TRIPLE)_LINKER=$(notdir $(TARGET_CROSS))gcc
HOST_PKG_RUST_CARGO_ENV = \
CARGO_HOME=$(HOST_DIR)/share/cargo \
CARGO_INSTALL_ROOT=$(HOST_DIR) \
RUSTFLAGS="$(addprefix -C link-args=,$(HOST_LDFLAGS))"
> +
> +$(2)_DOWNLOAD_POST_PROCESS = cargo
> +$(2)_DL_ENV = CARGO_HOME=$$(HOST_DIR)/share/cargo
> +
> +# Due to vendoring, it is pretty likely that not all licenses are
> +# listed in <pkg>_LICENSE.
> +$(2)_LICENSE += , vendored dependencies licenses probably not listed
> +
> +# Note: in all the steps below, we "cd" into the build directory to
> +# execute the "cargo" tool instead of passing $(@D)/Cargo.toml as the
> +# manifest-path. Indeed while the latter seems to work, it in fact
> +# breaks in subtle ways as the way cargo searches for its
> +# configuration file is based (among other rules) on the current
> +# directory. This means that if cargo is started outside of a package
> +# directory, its configuration file will not be taken into account.
> +#
> +# Also, we pass:
> +# * --offline to prevent cargo from downloading anything: all
> +# dependencies should have been built by the download post
> +# process logic
> +# * --locked to force cargo to use the Cargo.lock file, which ensures
> +# that a fixed set of dependency versions is used
> +
> +#
> +# Build step. Only define it if not already defined by the package .mk
> +# file.
> +#
> +ifndef $(2)_BUILD_CMDS
> +ifeq ($(4),target)
> +define $(2)_BUILD_CMDS
> + cd $$(@D) && \
> + $$(TARGET_MAKE_ENV) \
> + $$(TARGET_CONFIGURE_OPTS) \
> + $$($(2)_CARGO_ENV) \
> + cargo build \
> + --offline \
> + --target $$(RUSTC_TARGET_NAME) \
> + $$(if $$(BR2_ENABLE_DEBUG),--debug,--release) \
> + --manifest-path Cargo.toml \
> + --locked \
> + -Z target-applies-to-host \
> + $$($(2)_CARGO_BUILD_OPTS)
> +endef
> +else # ifeq ($(4),target)
> +define $(2)_BUILD_CMDS
> + cd $$(@D) && \
> + $$(HOST_MAKE_ENV) \
> + RUSTFLAGS="$$(addprefix -C link-args=,$$(HOST_LDFLAGS))" \
> + $$($(2)_CARGO_ENV) \
> + cargo build \
> + --offline \
> + --release \
> + --manifest-path Cargo.toml \
> + --locked \
> + $$($(2)_CARGO_BUILD_OPTS)
> +endef
> +endif # ifeq ($(4),target)
> +endif # ifndef $(2)_BUILD_CMDS
> +
> +#
> +# Target installation step. Only define it if not already defined by
> +# the package .mk file.
> +#
> +ifndef $(2)_INSTALL_TARGET_CMDS
> +define $(2)_INSTALL_TARGET_CMDS
> + cd $$(@D) && \
> + $$(TARGET_MAKE_ENV) $$($(2)_CARGO_ENV) \
> + cargo install \
> + --target $$(RUSTC_TARGET_NAME) \
> + --offline \
> + --root $$(TARGET_DIR)/usr/ \
> + --bins \
> + --path ./ \
> + --force \
> + --locked \
> + $$($(2)_CARGO_INSTALL_OPTS)
> +endef
> +endif
> +
> +ifndef $(2)_INSTALL_CMDS
> +define $(2)_INSTALL_CMDS
> + cd $$(@D) && \
> + $$(HOST_MAKE_ENV) \
> + RUSTFLAGS="$$(addprefix -C link-args=,$$(HOST_LDFLAGS))" \
> + $$($(2)_CARGO_ENV) \
> + cargo install \
> + --offline \
> + --root $$(HOST_DIR) \
> + --bins \
> + --path ./ \
> + --force \
> + --locked \
> + $$($(2)_CARGO_INSTALL_OPTS)
> +endef
> +endif
> +
> +# Call the generic package infrastructure to generate the necessary
> +# make targets
> +$(call inner-generic-package,$(1),$(2),$(3),$(4))
> +
> +endef
> +
> +################################################################################
> +# cargo-package -- the target generator macro for Cargo packages
> +################################################################################
> +
> +cargo-package = $(call inner-cargo-package,$(pkgname),$(call UPPERCASE,$(pkgname)),$(call UPPERCASE,$(pkgname)),target)
> +host-cargo-package = $(call inner-cargo-package,host-$(pkgname),$(call UPPERCASE,host-$(pkgname)),$(call UPPERCASE,$(pkgname)),host)
> diff --git a/support/download/cargo-post-process b/support/download/cargo-post-process
> new file mode 100755
> index 0000000000..25dbbedbfa
> --- /dev/null
> +++ b/support/download/cargo-post-process
> @@ -0,0 +1,38 @@
> +#!/usr/bin/env bash
> +
> +set -e
> +
> +. "${0%/*}/helpers"
> +
> +while getopts "n:o:" OPT; do
> + case "${OPT}" in
> + o) output="${OPTARG}";;
> + n) base_name="${OPTARG}";;
> + :) error "option '%s' expects a mandatory argument\n" "${OPTARG}";;
> + \?) error "unknown option '%s'\n" "${OPTARG}";;
> + esac
> +done
> +
> +# Already vendored tarball, nothing to do
> +if tar tf ${output} | grep -q "^[^/]*/VENDOR" ; then
> + exit 0
> +fi
> +
> +post_process_unpack ${base_name} ${output}
> +
> +# Do the Cargo vendoring
> +pushd ${base_name} > /dev/null
> +cargo vendor --locked VENDOR
> +
> +# Create the local .cargo/config with vendor info
> +mkdir -p .cargo/
> +cat <<EOF >.cargo/config
> +[source.crates-io]
> +replace-with = "vendored-sources"
> +
> +[source.vendored-sources]
> +directory = "VENDOR"
> +EOF
> +popd > /dev/null
> +
> +post_process_repack $(pwd) ${base_name} ${output}
> --
> 2.33.1
>
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot
^ permalink raw reply [flat|nested] 38+ messages in thread* Re: [Buildroot] [PATCH v3 05/11] package/pkg-cargo.mk: introduce the cargo package infrastructure
2022-01-07 1:12 ` James Hilliard
@ 2022-01-07 9:52 ` Thomas Petazzoni
2022-01-07 22:30 ` James Hilliard
0 siblings, 1 reply; 38+ messages in thread
From: Thomas Petazzoni @ 2022-01-07 9:52 UTC (permalink / raw)
To: James Hilliard
Cc: Matt Weber, Patrick Havelange, Yann E. MORIN, Buildroot List
Hello James,
On Thu, 6 Jan 2022 19:12:51 -0600
James Hilliard <james.hilliard1@gmail.com> wrote:
> > +$(2)_CARGO_ENV += \
> > + CARGO_HOME=$$(HOST_DIR)/share/cargo \
> > + __CARGO_TEST_CHANNEL_OVERRIDE_DO_NOT_USE_THIS="nightly" \
> > + CARGO_TARGET_APPLIES_TO_HOST="false"
>
> Maybe want to set these additional options as well so that packages
> using cargo can
> build with only the env variables set(in some cases at least I think),
Thanks for the suggestion. However, could you be more specific than
"maybe" and "I think" ? The way you put it seems like we don't really
know why those variables would be useful and what they do.
> still probably
> need to pass some command line args but this should reduce the amount at least:
> RUSTC_TARGET_TRIPLE = $(subst -,_,$(call UPPERCASE,$(RUSTC_TARGET_NAME)))
Which command line arguments would be replaced by those variables?
> PKG_RUST_CARGO_ENV = \
> CARGO_HOME=$(HOST_DIR)/share/cargo \
This one we already pass.
> CARGO_BUILD_TARGET=$(RUSTC_TARGET_NAME) \
This one would replace the --target argument? Just to understand, what
would be the advantage of the variable compared to the comand line
argument?
> CARGO_INSTALL_ROOT=$(TARGET_DIR)/usr \
This would replace --root $$(TARGET_DIR)/usr/ I suppose. Here as well,
any benefit/drawback of arguments vs. variables?
> CARGO_TARGET_$(RUSTC_TARGET_TRIPLE)_LINKER=$(notdir $(TARGET_CROSS))gcc
How come things are working today with this one?
>
> HOST_PKG_RUST_CARGO_ENV = \
> CARGO_HOME=$(HOST_DIR)/share/cargo \
This one already passed.
> CARGO_INSTALL_ROOT=$(HOST_DIR) \
This would replace --root $$(HOST_DIR) I suppose.
> RUSTFLAGS="$(addprefix -C link-args=,$(HOST_LDFLAGS))"
This is already passed, although replicated between the build and
install steps of the host variant.
So essentially the only questions is:
- Advantage of variables vs. command line options
- Relevance of CARGO_TARGET_$(RUSTC_TARGET_TRIPLE)_LINKER=$(notdir $(TARGET_CROSS))gcc
Thanks!
Thomas
--
Thomas Petazzoni, co-owner and CEO, Bootlin
Embedded Linux and Kernel engineering and training
https://bootlin.com
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [Buildroot] [PATCH v3 05/11] package/pkg-cargo.mk: introduce the cargo package infrastructure
2022-01-07 9:52 ` Thomas Petazzoni
@ 2022-01-07 22:30 ` James Hilliard
2022-01-07 22:59 ` Thomas Petazzoni
0 siblings, 1 reply; 38+ messages in thread
From: James Hilliard @ 2022-01-07 22:30 UTC (permalink / raw)
To: Thomas Petazzoni
Cc: Matt Weber, Patrick Havelange, Yann E. MORIN, Buildroot List
On Fri, Jan 7, 2022 at 3:52 AM Thomas Petazzoni
<thomas.petazzoni@bootlin.com> wrote:
>
> Hello James,
>
> On Thu, 6 Jan 2022 19:12:51 -0600
> James Hilliard <james.hilliard1@gmail.com> wrote:
>
> > > +$(2)_CARGO_ENV += \
> > > + CARGO_HOME=$$(HOST_DIR)/share/cargo \
> > > + __CARGO_TEST_CHANNEL_OVERRIDE_DO_NOT_USE_THIS="nightly" \
> > > + CARGO_TARGET_APPLIES_TO_HOST="false"
> >
> > Maybe want to set these additional options as well so that packages
> > using cargo can
> > build with only the env variables set(in some cases at least I think),
>
> Thanks for the suggestion. However, could you be more specific than
> "maybe" and "I think" ? The way you put it seems like we don't really
> know why those variables would be useful and what they do.
>
> > still probably
> > need to pass some command line args but this should reduce the amount at least:
> > RUSTC_TARGET_TRIPLE = $(subst -,_,$(call UPPERCASE,$(RUSTC_TARGET_NAME)))
>
> Which command line arguments would be replaced by those variables?
>
> > PKG_RUST_CARGO_ENV = \
> > CARGO_HOME=$(HOST_DIR)/share/cargo \
>
> This one we already pass.
>
> > CARGO_BUILD_TARGET=$(RUSTC_TARGET_NAME) \
>
> This one would replace the --target argument? Just to understand, what
> would be the advantage of the variable compared to the comand line
> argument?
Might be better for reusing this env for indirect cargo build
invocations like those
done by python-setuptools-rust.
>
> > CARGO_INSTALL_ROOT=$(TARGET_DIR)/usr \
>
> This would replace --root $$(TARGET_DIR)/usr/ I suppose. Here as well,
> any benefit/drawback of arguments vs. variables?
Yeah, mostly for indict invocations as well.
>
> > CARGO_TARGET_$(RUSTC_TARGET_TRIPLE)_LINKER=$(notdir $(TARGET_CROSS))gcc
>
> How come things are working today with this one?
I think this is currently set in the cargo config file.
>
> >
> > HOST_PKG_RUST_CARGO_ENV = \
> > CARGO_HOME=$(HOST_DIR)/share/cargo \
>
> This one already passed.
>
> > CARGO_INSTALL_ROOT=$(HOST_DIR) \
>
> This would replace --root $$(HOST_DIR) I suppose.
Yeah, sounds right.
>
> > RUSTFLAGS="$(addprefix -C link-args=,$(HOST_LDFLAGS))"
>
> This is already passed, although replicated between the build and
> install steps of the host variant.
>
> So essentially the only questions is:
>
> - Advantage of variables vs. command line options
Makes setting cargo variables when cargo is not directly invoked simpler.
>
> - Relevance of CARGO_TARGET_$(RUSTC_TARGET_TRIPLE)_LINKER=$(notdir $(TARGET_CROSS))gcc
Set via CARGO_HOME currently, but I think may allow the cargo config to not
be required.
>
> Thanks!
>
> Thomas
> --
> Thomas Petazzoni, co-owner and CEO, Bootlin
> Embedded Linux and Kernel engineering and training
> https://bootlin.com
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [Buildroot] [PATCH v3 05/11] package/pkg-cargo.mk: introduce the cargo package infrastructure
2022-01-07 22:30 ` James Hilliard
@ 2022-01-07 22:59 ` Thomas Petazzoni
2022-01-08 3:21 ` James Hilliard
2022-01-08 13:52 ` Juergen Stuber
0 siblings, 2 replies; 38+ messages in thread
From: Thomas Petazzoni @ 2022-01-07 22:59 UTC (permalink / raw)
To: James Hilliard
Cc: Matt Weber, Patrick Havelange, Yann E. MORIN, Buildroot List
Hello James,
On Fri, 7 Jan 2022 16:30:12 -0600
James Hilliard <james.hilliard1@gmail.com> wrote:
> > > CARGO_TARGET_$(RUSTC_TARGET_TRIPLE)_LINKER=$(notdir $(TARGET_CROSS))gcc
> >
> > How come things are working today with this one?
>
> I think this is currently set in the cargo config file.
Ah, right:
[target.@RUSTC_TARGET_NAME@]
linker = "@CROSS_PREFIX@gcc"
in package/rustc/cargo-config.in
> > - Relevance of CARGO_TARGET_$(RUSTC_TARGET_TRIPLE)_LINKER=$(notdir $(TARGET_CROSS))gcc
>
> Set via CARGO_HOME currently, but I think may allow the cargo config to not
> be required.
I'm no sure why you say this is set via CARGO_HOME? Because the
config file is stored in $(CARGO_HOME)/config ?
Thomas
--
Thomas Petazzoni, co-owner and CEO, Bootlin
Embedded Linux and Kernel engineering and training
https://bootlin.com
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [Buildroot] [PATCH v3 05/11] package/pkg-cargo.mk: introduce the cargo package infrastructure
2022-01-07 22:59 ` Thomas Petazzoni
@ 2022-01-08 3:21 ` James Hilliard
2022-01-08 13:52 ` Juergen Stuber
1 sibling, 0 replies; 38+ messages in thread
From: James Hilliard @ 2022-01-08 3:21 UTC (permalink / raw)
To: Thomas Petazzoni
Cc: Matt Weber, Patrick Havelange, Yann E. MORIN, Buildroot List
On Fri, Jan 7, 2022 at 5:00 PM Thomas Petazzoni
<thomas.petazzoni@bootlin.com> wrote:
>
> Hello James,
>
> On Fri, 7 Jan 2022 16:30:12 -0600
> James Hilliard <james.hilliard1@gmail.com> wrote:
>
> > > > CARGO_TARGET_$(RUSTC_TARGET_TRIPLE)_LINKER=$(notdir $(TARGET_CROSS))gcc
> > >
> > > How come things are working today with this one?
> >
> > I think this is currently set in the cargo config file.
>
> Ah, right:
>
> [target.@RUSTC_TARGET_NAME@]
> linker = "@CROSS_PREFIX@gcc"
>
> in package/rustc/cargo-config.in
>
> > > - Relevance of CARGO_TARGET_$(RUSTC_TARGET_TRIPLE)_LINKER=$(notdir $(TARGET_CROSS))gcc
> >
> > Set via CARGO_HOME currently, but I think may allow the cargo config to not
> > be required.
>
> I'm no sure why you say this is set via CARGO_HOME? Because the
> config file is stored in $(CARGO_HOME)/config ?
Yeah, that's what I mean.
>
> Thomas
> --
> Thomas Petazzoni, co-owner and CEO, Bootlin
> Embedded Linux and Kernel engineering and training
> https://bootlin.com
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [Buildroot] [PATCH v3 05/11] package/pkg-cargo.mk: introduce the cargo package infrastructure
2022-01-07 22:59 ` Thomas Petazzoni
2022-01-08 3:21 ` James Hilliard
@ 2022-01-08 13:52 ` Juergen Stuber
1 sibling, 0 replies; 38+ messages in thread
From: Juergen Stuber @ 2022-01-08 13:52 UTC (permalink / raw)
To: buildroot
Hi Thomas, James, all,
On Fri, 7 Jan 2022 23:59:59 +0100
Thomas Petazzoni <thomas.petazzoni@bootlin.com> wrote:
> On Fri, 7 Jan 2022 16:30:12 -0600
> James Hilliard <james.hilliard1@gmail.com> wrote:
>
> > > > CARGO_TARGET_$(RUSTC_TARGET_TRIPLE)_LINKER=$(notdir
> > > > $(TARGET_CROSS))gcc
> > >
> > > How come things are working today with this one?
> >
> > I think this is currently set in the cargo config file.
>
> Ah, right:
>
> [target.@RUSTC_TARGET_NAME@]
> linker = "@CROSS_PREFIX@gcc"
>
> in package/rustc/cargo-config.in
the config file has the problem that it is found via CARGO_HOME,
which comes last in the search path, after searching upward in the
directory hierarchy. When I used Rust under buildroot for the first
time, a broken .cargo/config in my home directory which I had
completely forgotten about caused the build to fail, and it took me a
while to figure it out.
Environment variables take precedence over config files, so IMHO they
are the better choice.
> > > - Relevance of
> > > CARGO_TARGET_$(RUSTC_TARGET_TRIPLE)_LINKER=$(notdir
> > > $(TARGET_CROSS))gcc
> >
> > Set via CARGO_HOME currently, but I think may allow the cargo
> > config to not be required.
>
> I'm no sure why you say this is set via CARGO_HOME? Because the
> config file is stored in $(CARGO_HOME)/config ?
Greetings
Jürgen
--
Jürgen Stuber <juergen@jstuber.net>
http://www.jstuber.net/
Tel: +49-208-304 20 50
Mobil: +49-178-39 39 628
1B78 A579 E159 2A85 67BB 1314 C083 224B 0F9C DA21
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [Buildroot] [PATCH v3 05/11] package/pkg-cargo.mk: introduce the cargo package infrastructure
2022-01-06 20:59 ` [Buildroot] [PATCH v3 05/11] package/pkg-cargo.mk: introduce the cargo package infrastructure Thomas Petazzoni
2022-01-07 1:12 ` James Hilliard
@ 2022-01-07 10:05 ` Romain Naour
2022-01-07 10:26 ` Romain Naour
2022-01-07 21:07 ` Romain Naour
3 siblings, 0 replies; 38+ messages in thread
From: Romain Naour @ 2022-01-07 10:05 UTC (permalink / raw)
To: Thomas Petazzoni, James Hilliard, Christian Stewart,
Buildroot List, Matt Weber, Yann E. MORIN
Cc: Patrick Havelange
Hello Thomas, All,
Le 06/01/2022 à 21:59, Thomas Petazzoni a écrit :
> From: Patrick Havelange <patrick.havelange@essensium.com>
>
> In order to be package agnostic, the install phase is now using cargo
> instead of install. TARGET_CONFIGURE_OPTS is now also set when running
> cargo in order to support cross compiling C code within cargo.
>
> This commit also adds support/download/cargo-post-process to perform
> the vendoring on Cargo packages.
>
> The <pkg>_LICENSE variable of cargo packages is expanded with ",
> vendored dependencies licenses probably not listed" as currently for
> all packages, the licenses of the vendored dependencies are not taken
> into account.
>
> Signed-off-by: Patrick Havelange <patrick.havelange@essensium.com>
> [Thomas: add support for host-cargo-package and vendoring]
> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
> ---
> +#
> +# Build step. Only define it if not already defined by the package .mk
> +# file.
> +#
> +ifndef $(2)_BUILD_CMDS
> +ifeq ($(4),target)
> +define $(2)_BUILD_CMDS
> + cd $$(@D) && \
> + $$(TARGET_MAKE_ENV) \
> + $$(TARGET_CONFIGURE_OPTS) \
> + $$($(2)_CARGO_ENV) \
> + cargo build \
> + --offline \
> + --target $$(RUSTC_TARGET_NAME) \
> + $$(if $$(BR2_ENABLE_DEBUG),--debug,--release) \
> + --manifest-path Cargo.toml \
> + --locked \
> + -Z target-applies-to-host \
> + $$($(2)_CARGO_BUILD_OPTS)
> +endef
> +else # ifeq ($(4),target)
> +define $(2)_BUILD_CMDS
> + cd $$(@D) && \
> + $$(HOST_MAKE_ENV) \
> + RUSTFLAGS="$$(addprefix -C link-args=,$$(HOST_LDFLAGS))" \
> + $$($(2)_CARGO_ENV) \
> + cargo build \
> + --offline \
> + --release \
> + --manifest-path Cargo.toml \
> + --locked \
> + $$($(2)_CARGO_BUILD_OPTS)
> +endef
> +endif # ifeq ($(4),target)
> +endif # ifndef $(2)_BUILD_CMDS
> +
> +#
> +# Target installation step. Only define it if not already defined by
> +# the package .mk file.
> +#
> +ifndef $(2)_INSTALL_TARGET_CMDS
> +define $(2)_INSTALL_TARGET_CMDS
> + cd $$(@D) && \
> + $$(TARGET_MAKE_ENV) $$($(2)_CARGO_ENV) \
Removing "CC_$$(subst -,_,$$(RUSTC_TARGET_NAME))=$$(TARGET_CC) " [1]
and adding $$(TARGET_CONFIGURE_OPTS) fixes the build of bat package.
The package is still rebuild during the install step but at least it build...
[1] http://lists.busybox.net/pipermail/buildroot/2022-January/632610.html
Best regards,
Romain
> + cargo install \
> + --target $$(RUSTC_TARGET_NAME) \
> + --offline \
> + --root $$(TARGET_DIR)/usr/ \
> + --bins \
> + --path ./ \
> + --force \
> + --locked \
> + $$($(2)_CARGO_INSTALL_OPTS)
> +endef
> +endif
> +
> +ifndef $(2)_INSTALL_CMDS
> +define $(2)_INSTALL_CMDS
> + cd $$(@D) && \
> + $$(HOST_MAKE_ENV) \
> + RUSTFLAGS="$$(addprefix -C link-args=,$$(HOST_LDFLAGS))" \
> + $$($(2)_CARGO_ENV) \
> + cargo install \
> + --offline \
> + --root $$(HOST_DIR) \
> + --bins \
> + --path ./ \
> + --force \
> + --locked \
> + $$($(2)_CARGO_INSTALL_OPTS)
> +endef
> +endif
> +
>
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [Buildroot] [PATCH v3 05/11] package/pkg-cargo.mk: introduce the cargo package infrastructure
2022-01-06 20:59 ` [Buildroot] [PATCH v3 05/11] package/pkg-cargo.mk: introduce the cargo package infrastructure Thomas Petazzoni
2022-01-07 1:12 ` James Hilliard
2022-01-07 10:05 ` Romain Naour
@ 2022-01-07 10:26 ` Romain Naour
2022-01-07 10:44 ` Yann E. MORIN
2022-01-07 21:07 ` Romain Naour
3 siblings, 1 reply; 38+ messages in thread
From: Romain Naour @ 2022-01-07 10:26 UTC (permalink / raw)
To: Thomas Petazzoni, James Hilliard, Christian Stewart,
Buildroot List, Matt Weber, Yann E. MORIN
Cc: Patrick Havelange
Le 06/01/2022 à 21:59, Thomas Petazzoni a écrit :
> From: Patrick Havelange <patrick.havelange@essensium.com>
>
> In order to be package agnostic, the install phase is now using cargo
> instead of install. TARGET_CONFIGURE_OPTS is now also set when running
> cargo in order to support cross compiling C code within cargo.
>
> This commit also adds support/download/cargo-post-process to perform
> the vendoring on Cargo packages.
Download post-processing does't work when we use _OVERRIDE_SRCDIR.
Maybe we have to make sure that all download of vendor modules are already
present in the source directory used by _OVERRIDE_SRCDIR.
Best regards,
Romain
>
> The <pkg>_LICENSE variable of cargo packages is expanded with ",
> vendored dependencies licenses probably not listed" as currently for
> all packages, the licenses of the vendored dependencies are not taken
> into account.
>
> Signed-off-by: Patrick Havelange <patrick.havelange@essensium.com>
> [Thomas: add support for host-cargo-package and vendoring]
> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [Buildroot] [PATCH v3 05/11] package/pkg-cargo.mk: introduce the cargo package infrastructure
2022-01-07 10:26 ` Romain Naour
@ 2022-01-07 10:44 ` Yann E. MORIN
2022-01-07 11:03 ` Romain Naour
0 siblings, 1 reply; 38+ messages in thread
From: Yann E. MORIN @ 2022-01-07 10:44 UTC (permalink / raw)
To: Romain Naour
Cc: Matt Weber, Patrick Havelange, James Hilliard, Thomas Petazzoni,
Buildroot List
Romain, All,
On 2022-01-07 11:26 +0100, Romain Naour spake thusly:
> Le 06/01/2022 à 21:59, Thomas Petazzoni a écrit :
> > From: Patrick Havelange <patrick.havelange@essensium.com>
> > In order to be package agnostic, the install phase is now using cargo
> > instead of install. TARGET_CONFIGURE_OPTS is now also set when running
> > cargo in order to support cross compiling C code within cargo.
> >
> > This commit also adds support/download/cargo-post-process to perform
> > the vendoring on Cargo packages.
>
> Download post-processing does't work when we use _OVERRIDE_SRCDIR.
This is indeed expected: _OVERRIDE_SRCDIR is skipping all the download,
extract and patch steps.
Developers that are using an _OVERRIDE_SRCDIR are responsible for
providing an source tree that is ready to be configured.
So, this is probably something that should be added to the manual, if it
is not already there, or to emphasise it if already there:
When you are using OVERRIDE_SRCDIR, you are responsible for
providing a source tree that is ready to be configured.
Care to send a patch, please?
Regards,
Yann E. MORIN.
> Maybe we have to make sure that all download of vendor modules are already
> present in the source directory used by _OVERRIDE_SRCDIR.
>
> Best regards,
> Romain
>
> >
> > The <pkg>_LICENSE variable of cargo packages is expanded with ",
> > vendored dependencies licenses probably not listed" as currently for
> > all packages, the licenses of the vendored dependencies are not taken
> > into account.
> >
> > Signed-off-by: Patrick Havelange <patrick.havelange@essensium.com>
> > [Thomas: add support for host-cargo-package and vendoring]
> > Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
--
.-----------------.--------------------.------------------.--------------------.
| Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ |
| +33 561 099 427 `------------.-------: X AGAINST | \e/ There is no |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot
^ permalink raw reply [flat|nested] 38+ messages in thread* Re: [Buildroot] [PATCH v3 05/11] package/pkg-cargo.mk: introduce the cargo package infrastructure
2022-01-07 10:44 ` Yann E. MORIN
@ 2022-01-07 11:03 ` Romain Naour
0 siblings, 0 replies; 38+ messages in thread
From: Romain Naour @ 2022-01-07 11:03 UTC (permalink / raw)
To: Yann E. MORIN
Cc: Matt Weber, Patrick Havelange, James Hilliard, Thomas Petazzoni,
Buildroot List
Le 07/01/2022 à 11:44, Yann E. MORIN a écrit :
> Romain, All,
>
> On 2022-01-07 11:26 +0100, Romain Naour spake thusly:
>> Le 06/01/2022 à 21:59, Thomas Petazzoni a écrit :
>>> From: Patrick Havelange <patrick.havelange@essensium.com>
>>> In order to be package agnostic, the install phase is now using cargo
>>> instead of install. TARGET_CONFIGURE_OPTS is now also set when running
>>> cargo in order to support cross compiling C code within cargo.
>>>
>>> This commit also adds support/download/cargo-post-process to perform
>>> the vendoring on Cargo packages.
>>
>> Download post-processing does't work when we use _OVERRIDE_SRCDIR.
>
> This is indeed expected: _OVERRIDE_SRCDIR is skipping all the download,
> extract and patch steps.
>
> Developers that are using an _OVERRIDE_SRCDIR are responsible for
> providing an source tree that is ready to be configured.
>
> So, this is probably something that should be added to the manual, if it
> is not already there, or to emphasise it if already there:
>
> When you are using OVERRIDE_SRCDIR, you are responsible for
> providing a source tree that is ready to be configured.
I did a try, It's not just adding vendor sources into package sources (in my
case "bat" package).
We have to execute "cargo vendor" from "bat" sources and use CARGO_HOME used by
Buildroot:
CARGO_HOME=$HOST_DIR/share/cargo cargo vendor
Do we want to document this method ?
Best regards,
Romain
>
> Care to send a patch, please?
>
> Regards,
> Yann E. MORIN.
>
>> Maybe we have to make sure that all download of vendor modules are already
>> present in the source directory used by _OVERRIDE_SRCDIR.
>>
>> Best regards,
>> Romain
>>
>>>
>>> The <pkg>_LICENSE variable of cargo packages is expanded with ",
>>> vendored dependencies licenses probably not listed" as currently for
>>> all packages, the licenses of the vendored dependencies are not taken
>>> into account.
>>>
>>> Signed-off-by: Patrick Havelange <patrick.havelange@essensium.com>
>>> [Thomas: add support for host-cargo-package and vendoring]
>>> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
>
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [Buildroot] [PATCH v3 05/11] package/pkg-cargo.mk: introduce the cargo package infrastructure
2022-01-06 20:59 ` [Buildroot] [PATCH v3 05/11] package/pkg-cargo.mk: introduce the cargo package infrastructure Thomas Petazzoni
` (2 preceding siblings ...)
2022-01-07 10:26 ` Romain Naour
@ 2022-01-07 21:07 ` Romain Naour
2022-01-07 21:53 ` Thomas Petazzoni
2022-01-09 10:04 ` Thomas Petazzoni
3 siblings, 2 replies; 38+ messages in thread
From: Romain Naour @ 2022-01-07 21:07 UTC (permalink / raw)
To: Thomas Petazzoni, James Hilliard, Christian Stewart,
Buildroot List, Matt Weber, Yann E. MORIN
Cc: Patrick Havelange
Hello,
Le 06/01/2022 à 21:59, Thomas Petazzoni a écrit :
> From: Patrick Havelange <patrick.havelange@essensium.com>
>
> In order to be package agnostic, the install phase is now using cargo
> instead of install. TARGET_CONFIGURE_OPTS is now also set when running
> cargo in order to support cross compiling C code within cargo.
>
> This commit also adds support/download/cargo-post-process to perform
> the vendoring on Cargo packages.
>
> The <pkg>_LICENSE variable of cargo packages is expanded with ",
> vendored dependencies licenses probably not listed" as currently for
> all packages, the licenses of the vendored dependencies are not taken
> into account.
>
> Signed-off-by: Patrick Havelange <patrick.havelange@essensium.com>
> [Thomas: add support for host-cargo-package and vendoring]
> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
> ---
> package/Makefile.in | 1 +
> package/pkg-cargo.mk | 153 ++++++++++++++++++++++++++++
> support/download/cargo-post-process | 38 +++++++
> 3 files changed, 192 insertions(+)
> create mode 100644 package/pkg-cargo.mk
> create mode 100755 support/download/cargo-post-process
>
> diff --git a/package/Makefile.in b/package/Makefile.in
> index dae7a859fd..0ca2a5844e 100644
> --- a/package/Makefile.in
> +++ b/package/Makefile.in
> @@ -441,3 +441,4 @@ include package/pkg-waf.mk
> include package/pkg-golang.mk
> include package/pkg-meson.mk
> include package/pkg-qmake.mk
> +include package/pkg-cargo.mk
> diff --git a/package/pkg-cargo.mk b/package/pkg-cargo.mk
> new file mode 100644
> index 0000000000..bf1436a86b
> --- /dev/null
> +++ b/package/pkg-cargo.mk
> @@ -0,0 +1,153 @@
> +################################################################################
> +# Cargo package infrastructure
> +#
> +# This file implements an infrastructure that eases development of package
> +# .mk files for Cargo packages. It should be used for all packages that use
> +# Cargo as their build system.
> +#
> +# See the Buildroot documentation for details on the usage of this
> +# infrastructure
> +#
> +# In terms of implementation, this Cargo infrastructure requires the .mk file
> +# to only specify metadata information about the package: name, version,
> +# download URL, etc.
> +#
> +# We still allow the package .mk file to override what the different steps
> +# are doing, if needed. For example, if <PKG>_BUILD_CMDS is already defined,
> +# it is used as the list of commands to perform to build the package,
> +# instead of the default Cargo behaviour. The package can also define some
> +# post operation hooks.
> +#
> +################################################################################
> +
> +################################################################################
> +# inner-cargo-package -- defines how the configuration, compilation and
> +# installation of a cargo package should be done, implements a few hooks
> +# to tune the build process for cargo specifities and calls the generic
> +# package infrastructure to generate the necessary make targets
> +#
> +# argument 1 is the lowercase package name
> +# argument 2 is the uppercase package name, including a HOST_ prefix
> +# for host packages
> +# argument 3 is the uppercase package name, without the HOST_ prefix
> +# for host packages
> +# argument 4 is the type (target or host)
> +################################################################################
> +
> +define inner-cargo-package
> +
> +# We need host-rustc to run cargo
> +$(2)_DOWNLOAD_DEPENDENCIES += host-rustc
> +
> +$(2)_CARGO_ENV += \
> + CARGO_HOME=$$(HOST_DIR)/share/cargo \
> + __CARGO_TEST_CHANNEL_OVERRIDE_DO_NOT_USE_THIS="nightly" \
> + CARGO_TARGET_APPLIES_TO_HOST="false"
> +
> +$(2)_DOWNLOAD_POST_PROCESS = cargo
> +$(2)_DL_ENV = CARGO_HOME=$$(HOST_DIR)/share/cargo
> +
> +# Due to vendoring, it is pretty likely that not all licenses are
> +# listed in <pkg>_LICENSE.
> +$(2)_LICENSE += , vendored dependencies licenses probably not listed
> +
> +# Note: in all the steps below, we "cd" into the build directory to
> +# execute the "cargo" tool instead of passing $(@D)/Cargo.toml as the
> +# manifest-path. Indeed while the latter seems to work, it in fact
> +# breaks in subtle ways as the way cargo searches for its
> +# configuration file is based (among other rules) on the current
> +# directory. This means that if cargo is started outside of a package
> +# directory, its configuration file will not be taken into account.
> +#
> +# Also, we pass:
> +# * --offline to prevent cargo from downloading anything: all
> +# dependencies should have been built by the download post
> +# process logic
> +# * --locked to force cargo to use the Cargo.lock file, which ensures
> +# that a fixed set of dependency versions is used
> +
> +#
> +# Build step. Only define it if not already defined by the package .mk
> +# file.
> +#
> +ifndef $(2)_BUILD_CMDS
> +ifeq ($(4),target)
> +define $(2)_BUILD_CMDS
> + cd $$(@D) && \
> + $$(TARGET_MAKE_ENV) \
> + $$(TARGET_CONFIGURE_OPTS) \
> + $$($(2)_CARGO_ENV) \
> + cargo build \
> + --offline \
> + --target $$(RUSTC_TARGET_NAME) \
> + $$(if $$(BR2_ENABLE_DEBUG),--debug,--release) \
> + --manifest-path Cargo.toml \
> + --locked \
> + -Z target-applies-to-host \
> + $$($(2)_CARGO_BUILD_OPTS)
> +endef
> +else # ifeq ($(4),target)
> +define $(2)_BUILD_CMDS
> + cd $$(@D) && \
> + $$(HOST_MAKE_ENV) \
> + RUSTFLAGS="$$(addprefix -C link-args=,$$(HOST_LDFLAGS))" \
> + $$($(2)_CARGO_ENV) \
> + cargo build \
> + --offline \
> + --release \
> + --manifest-path Cargo.toml \
> + --locked \
> + $$($(2)_CARGO_BUILD_OPTS)
> +endef
> +endif # ifeq ($(4),target)
> +endif # ifndef $(2)_BUILD_CMDS
> +
is _INSTALL_STAGING_CMDS is missing here ?
Rust package may wand to install *.rlib files [1] (similar to .a)
> +#
> +# Target installation step. Only define it if not already defined by
> +# the package .mk file.
> +#
> +ifndef $(2)_INSTALL_TARGET_CMDS
> +define $(2)_INSTALL_TARGET_CMDS
> + cd $$(@D) && \
> + $$(TARGET_MAKE_ENV) $$($(2)_CARGO_ENV) \
> + cargo install \
cargo install seems overingeneered and produce unwanted rebuild when packages
are using cargo features [2].
meta-rust doesn't use cago install at all but use an cargo_do_install() function
that use manual install commandes [3].
[1] https://doc.rust-lang.org/reference/linkage.html
[2] https://github.com/rust-lang/cargo/issues/8703
[3] https://github.com/meta-rust/meta-rust/blob/master/classes/cargo.bbclass#L55
> + --target $$(RUSTC_TARGET_NAME) \
> + --offline \
> + --root $$(TARGET_DIR)/usr/ \
> + --bins \
> + --path ./ \
> + --force \
> + --locked \
> + $$($(2)_CARGO_INSTALL_OPTS)
> +endef
> +endif
> +
> +ifndef $(2)_INSTALL_CMDS
> +define $(2)_INSTALL_CMDS
> + cd $$(@D) && \
> + $$(HOST_MAKE_ENV) \
> + RUSTFLAGS="$$(addprefix -C link-args=,$$(HOST_LDFLAGS))" \
> + $$($(2)_CARGO_ENV) \
> + cargo install \
> + --offline \
> + --root $$(HOST_DIR) \
> + --bins \
> + --path ./ \
> + --force \
> + --locked \
> + $$($(2)_CARGO_INSTALL_OPTS)
> +endef
> +endif
> +
> +# Call the generic package infrastructure to generate the necessary
> +# make targets
> +$(call inner-generic-package,$(1),$(2),$(3),$(4))
> +
> +endef
> +
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot
^ permalink raw reply [flat|nested] 38+ messages in thread* Re: [Buildroot] [PATCH v3 05/11] package/pkg-cargo.mk: introduce the cargo package infrastructure
2022-01-07 21:07 ` Romain Naour
@ 2022-01-07 21:53 ` Thomas Petazzoni
2022-01-09 10:04 ` Thomas Petazzoni
1 sibling, 0 replies; 38+ messages in thread
From: Thomas Petazzoni @ 2022-01-07 21:53 UTC (permalink / raw)
To: Romain Naour
Cc: Matt Weber, Patrick Havelange, James Hilliard, Buildroot List,
Yann E. MORIN
Hello,
On Fri, 7 Jan 2022 22:07:26 +0100
Romain Naour <romain.naour@smile.fr> wrote:
> > +endef
> > +endif # ifeq ($(4),target)
> > +endif # ifndef $(2)_BUILD_CMDS
> > +
>
> is _INSTALL_STAGING_CMDS is missing here ?
>
> Rust package may wand to install *.rlib files [1] (similar to .a)
So far we haven't seen any package that needed staging installation, so
I did not implement it (Yann asked a similar question today, live).
> cargo install seems overingeneered and produce unwanted rebuild when packages
> are using cargo features [2].
Why do you say "over engineered" ? Buggy perhaps, over-engineered, I'm not sure.
> meta-rust doesn't use cago install at all but use an cargo_do_install() function
> that use manual install commandes [3].
>
> [1] https://doc.rust-lang.org/reference/linkage.html
> [2] https://github.com/rust-lang/cargo/issues/8703
> [3] https://github.com/meta-rust/meta-rust/blob/master/classes/cargo.bbclass#L55
I was wondering how they knew what to install and how/where, but that
link explains it.
It would a lot simpler to be able to use "cargo install", but I agree
those rebuilds during the install step are not very nice (and they are
time consuming).
Thomas
--
Thomas Petazzoni, co-owner and CEO, Bootlin
Embedded Linux and Kernel engineering and training
https://bootlin.com
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [Buildroot] [PATCH v3 05/11] package/pkg-cargo.mk: introduce the cargo package infrastructure
2022-01-07 21:07 ` Romain Naour
2022-01-07 21:53 ` Thomas Petazzoni
@ 2022-01-09 10:04 ` Thomas Petazzoni
1 sibling, 0 replies; 38+ messages in thread
From: Thomas Petazzoni @ 2022-01-09 10:04 UTC (permalink / raw)
To: Romain Naour
Cc: Matt Weber, Patrick Havelange, James Hilliard, Buildroot List,
Yann E. MORIN
On Fri, 7 Jan 2022 22:07:26 +0100
Romain Naour <romain.naour@smile.fr> wrote:
> > +endif # ifeq ($(4),target)
> > +endif # ifndef $(2)_BUILD_CMDS
> > +
>
> is _INSTALL_STAGING_CMDS is missing here ?
>
> Rust package may wand to install *.rlib files [1] (similar to .a)
But will Cargo pick up .rlib files from STAGING_DIR ? I don't see how
Cargo/Rust knows about STAGING_DIR. Aren't those .rlib files anyway
installed in CARGO_HOME, and Cargo picks them up from here, or
something like that?
Installing to STAGING_DIR is one thing, but what's important is to
figure out if Cargo is really going to use what we install in
STAGING_DIR.
Do we have a specific example where one package installs a .rlib and
another package uses it?
Thomas
--
Thomas Petazzoni, co-owner and CEO, Bootlin
Embedded Linux and Kernel engineering and training
https://bootlin.com
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot
^ permalink raw reply [flat|nested] 38+ messages in thread
* [Buildroot] [PATCH v3 06/11] docs/manual/cargo: document the cargo-package infrastructure
2022-01-06 20:59 [Buildroot] [PATCH v3 00/11] Support for Cargo and Go vendoring Thomas Petazzoni
` (4 preceding siblings ...)
2022-01-06 20:59 ` [Buildroot] [PATCH v3 05/11] package/pkg-cargo.mk: introduce the cargo package infrastructure Thomas Petazzoni
@ 2022-01-06 20:59 ` Thomas Petazzoni
2022-01-06 20:59 ` [Buildroot] [PATCH v3 07/11] package/ripgrep: convert to cargo infrastructure Thomas Petazzoni
` (6 subsequent siblings)
12 siblings, 0 replies; 38+ messages in thread
From: Thomas Petazzoni @ 2022-01-06 20:59 UTC (permalink / raw)
To: James Hilliard, Christian Stewart, Buildroot List, Matt Weber,
Yann E. MORIN
Cc: Patrick Havelange, Thomas De Schampheleire, Thomas Petazzoni
From: Patrick Havelange <patrick.havelange@essensium.com>
The Buildroot manual was already providing some details on how to
integrate Cargo packages, and those details now need to be updated
with a proper documentation for the cargo-package infrastructure, as
well as updates related to vendoring support, which affects how
dependencies are handled.
Signed-off-by: Patrick Havelange <patrick.havelange@essensium.com>
[Thomas: numerous updates and extensions.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
docs/manual/adding-packages-cargo.txt: rewrite explanation about dependency management
Now that we have vendoring support for Cargo packages, let's rewrite
the dependency management section in a more accurate way.
We drop the part about the local cache of the registry, because
+CARGO_HOME+ in Buildroot points to $(HOST_DIR)/share/cargo, which is
not shared between builds nor preserved accross builds, so its effect
as a cache is limited.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
---
docs/manual/adding-packages-cargo.txt | 106 ++++++++++++--------------
1 file changed, 47 insertions(+), 59 deletions(-)
diff --git a/docs/manual/adding-packages-cargo.txt b/docs/manual/adding-packages-cargo.txt
index 8fcc80bcc6..fb3e7d0780 100644
--- a/docs/manual/adding-packages-cargo.txt
+++ b/docs/manual/adding-packages-cargo.txt
@@ -1,7 +1,7 @@
// -*- mode:doc; -*-
// vim: set syntax=asciidoc:
-=== Integration of Cargo-based packages
+=== Infrastructure for Cargo-based packages
Cargo is the package manager for the Rust programming language. It allows the
user to build programs or libraries written in Rust, but it also downloads and
@@ -10,7 +10,7 @@ called "crates".
[[cargo-package-tutorial]]
-==== Cargo-based package's +Config.in+ file
+==== +cargo-package+ tutorial
The +Config.in+ file of Cargo-based package 'foo' should contain:
@@ -25,11 +25,7 @@ The +Config.in+ file of Cargo-based package 'foo' should contain:
08: http://foosoftware.org/foo/
---------------------------
-==== Cargo-based package's +.mk+ file
-
-Buildroot does not (yet) provide a dedicated package infrastructure for
-Cargo-based packages. So, we will explain how to write a +.mk+ file for such a
-package. Let's start with an example:
+And the +.mk+ file for this package should contain:
------------------------------
01: ################################################################################
@@ -44,66 +40,58 @@ package. Let's start with an example:
10: FOO_LICENSE = GPL-3.0+
11: FOO_LICENSE_FILES = COPYING
12:
-13: FOO_DEPENDENCIES = host-rustc
-14:
-15: FOO_CARGO_ENV = CARGO_HOME=$(HOST_DIR)/share/cargo
-16:
-17: FOO_BIN_DIR = target/$(RUSTC_TARGET_NAME)/$(FOO_CARGO_MODE)
-18:
-19: FOO_CARGO_OPTS = \
-20: $(if $(BR2_ENABLE_DEBUG),,--release) \
-21: --target=$(RUSTC_TARGET_NAME) \
-22: --manifest-path=$(@D)/Cargo.toml
-23:
-24: define FOO_BUILD_CMDS
-25: $(TARGET_MAKE_ENV) $(FOO_CARGO_ENV) \
-26: cargo build $(FOO_CARGO_OPTS)
-27: endef
-28:
-29: define FOO_INSTALL_TARGET_CMDS
-30: $(INSTALL) -D -m 0755 $(@D)/$(FOO_BIN_DIR)/foo \
-31: $(TARGET_DIR)/usr/bin/foo
-32: endef
-33:
-34: $(eval $(generic-package))
+13: $(eval $(cargo-package))
--------------------------------
-The Makefile starts with the definition of the standard variables for package
-declaration (lines 7 to 11).
+The Makefile starts with the definition of the standard variables for
+package declaration (lines 7 to 11).
-As seen in line 34, it is based on the
-xref:generic-package-tutorial[+generic-package+ infrastructure]. So, it defines
-the variables required by this particular infrastructure, where Cargo is
-invoked:
+As seen in line 13, it is based on the +cargo-package+
+infrastructure. Cargo will be invoked automatically by this
+infrastructure to build and install the package.
-* +FOO_BUILD_CMDS+: Cargo is invoked to perform the build. The options required
- to configure the cross-compilation of the package are passed via
- +FOO_CONF_OPTS+.
+It is still possible to define custom build commands or install
+commands (i.e. with FOO_BUILD_CMDS and FOO_INSTALL_TARGET_CMDS).
+Those will then replace the commands from the cargo infrastructure.
-* +FOO_INSTALL_TARGET_CMDS+: The binary executable generated is installed on
- the target.
+==== +cargo-package+ reference
-In order to have Cargo available for the build, +FOO_DEPENDENCIES+ needs to
-contain +host-cargo+.
+The main macros for the Cargo package infrastructure are
++cargo-package+ for target packages and +host-cargo-package+ for host
+packages.
-To sum it up, to add a new Cargo-based package, the Makefile example can be
-copied verbatim then edited to replace all occurences of +FOO+ with the
-uppercase name of the new package and update the values of the standard
-variables.
+Just like the generic infrastructure, the Cargo infrastructure works
+by defining a number of variables before calling the +cargo-package+
+or +host-cargo-package+ macros.
-==== About Dependencies Management
+First, all the package metadata information variables that exist in
+the generic infrastructure also exist in the Cargo infrastructure:
++FOO_VERSION+, +FOO_SOURCE+, +FOO_PATCH+, +FOO_SITE+,
++FOO_DEPENDENCIES+, +FOO_LICENSE+, +FOO_LICENSE_FILES+, etc.
+
+A few additional variables, specific to the Cargo infrastructure, can
+also be defined. Many of them are only useful in very specific cases,
+typical packages will therefore only use a few of them.
+
+* +FOO_CARGO_ENV+ can be used to pass additional variables in the
+ environment of +cargo+ invocations. It used at both build and
+ installation time
-A crate can depend on other libraries from crates.io or git repositories, listed
-in its Cargo.toml file. Before starting a build, Cargo usually downloads
-automatically them. This step can also be performed independently, via the
-+cargo fetch+ command.
+* +FOO_CARGO_BUILD_OPTS+ can be used to pass additional options to
+ +cargo+ at build time.
+
+* +FOO_CARGO_INSTALL_OPTS+ can be used to pass additional options to
+ +cargo+ at install time.
+
+==== About Dependencies Management
-Cargo maintains a local cache of the registry index and of git checkouts of the
-crates, whose location is given by +$CARGO_HOME+. As seen in the package
-Makefile example at line 15, this environment variable is set to
-+$(HOST_DIR)/share/cargo+.
+A crate can depend on other libraries from crates.io or git
+repositories, listed in its Cargo.toml file. Buildroot automatically
+takes care of downloading such dependencies as part of the download
+step of packages that use the +cargo-package+ infrastructure. Such
+dependencies are then kept together with the package source code in
+the tarball cached in Buildroot's +DL_DIR+, and therefore the hash of
+the package's tarball includes such dependencies.
-This dependency download mechanism is not convenient when performing an offline
-build, as Cargo will fail to fetch the dependencies. In that case, it is advised
-to generate a tarball of the dependencies using the +cargo vendor+ and add it to
-+FOO_EXTRA_DOWNLOADS+.
+This mechanism ensures that any change in the dependencies will be
+detected, and allows the build to be performed completely offline.
--
2.33.1
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot
^ permalink raw reply related [flat|nested] 38+ messages in thread* [Buildroot] [PATCH v3 07/11] package/ripgrep: convert to cargo infrastructure
2022-01-06 20:59 [Buildroot] [PATCH v3 00/11] Support for Cargo and Go vendoring Thomas Petazzoni
` (5 preceding siblings ...)
2022-01-06 20:59 ` [Buildroot] [PATCH v3 06/11] docs/manual/cargo: document the cargo-package infrastructure Thomas Petazzoni
@ 2022-01-06 20:59 ` Thomas Petazzoni
2022-01-06 20:59 ` [Buildroot] [PATCH v3 08/11] package/sentry-cli: re-add package Thomas Petazzoni
` (5 subsequent siblings)
12 siblings, 0 replies; 38+ messages in thread
From: Thomas Petazzoni @ 2022-01-06 20:59 UTC (permalink / raw)
To: James Hilliard, Christian Stewart, Buildroot List, Matt Weber,
Yann E. MORIN
Cc: Patrick Havelange, Sam Voss, Thomas Petazzoni
From: Patrick Havelange <patrick.havelange@essensium.com>
This commit converts the ripgrep package to the cargo
infrastructure. This causes the vendoring mechanism to kick in, which
changes the contents of the tarball. To avoid a clash on the hash, we
change the version of ripgrep to the Git commit hash matching the
13.0.0 tag, so there are no functional changes, but the tarball name
is different.
Signed-off-by: Patrick Havelange <patrick.havelange@essensium.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
---
package/ripgrep/ripgrep.hash | 2 +-
package/ripgrep/ripgrep.mk | 37 ++++++------------------------------
2 files changed, 7 insertions(+), 32 deletions(-)
diff --git a/package/ripgrep/ripgrep.hash b/package/ripgrep/ripgrep.hash
index b2f03a95e6..cd2dbedfb4 100644
--- a/package/ripgrep/ripgrep.hash
+++ b/package/ripgrep/ripgrep.hash
@@ -1,3 +1,3 @@
# Locally calculated
-sha256 0fb17aaf285b3eee8ddab17b833af1e190d73de317ff9648751ab0660d763ed2 ripgrep-13.0.0.tar.gz
+sha256 6f1d4a8b653ce48d59ad777288b1257cbda607db29db19d031b7e622c60526f8 ripgrep-af6b6c543b224d348a8876f0c06245d9ea7929c5.tar.gz
sha256 0f96a83840e146e43c0ec96a22ec1f392e0680e6c1226e6f3ba87e0740af850f LICENSE-MIT
diff --git a/package/ripgrep/ripgrep.mk b/package/ripgrep/ripgrep.mk
index 7c2b79a4a7..d587441cfa 100644
--- a/package/ripgrep/ripgrep.mk
+++ b/package/ripgrep/ripgrep.mk
@@ -4,7 +4,11 @@
#
################################################################################
-RIPGREP_VERSION = 13.0.0
+# Same as 13.0.0, we use a Git commit hash because the hash of this
+# tarball changed when moving to the cargo-package infrastructure, and
+# we can't change the hash of existing tarball. Please switch back to
+# a Git tag at the next version bump.
+RIPGREP_VERSION = af6b6c543b224d348a8876f0c06245d9ea7929c5
RIPGREP_SITE = $(call github,burntsushi,ripgrep,$(RIPGREP_VERSION))
RIPGREP_LICENSE = MIT
RIPGREP_LICENSE_FILES = LICENSE-MIT
@@ -13,33 +17,4 @@ RIPGREP_CPE_ID_VENDOR = ripgrep_project
# CVE only impacts ripgrep on Windows
RIPGREP_IGNORE_CVES += CVE-2021-3013
-RIPGREP_DEPENDENCIES = host-rustc
-RIPGREP_CARGO_ENV = CARGO_HOME=$(HOST_DIR)/share/cargo \
- __CARGO_TEST_CHANNEL_OVERRIDE_DO_NOT_USE_THIS="nightly" \
- CARGO_TARGET_APPLIES_TO_HOST="false"
-
-RIPGREP_BIN_DIR = target/$(RUSTC_TARGET_NAME)/$(RIPGREP_CARGO_BIN_SUBDIR)
-
-RIPGREP_CARGO_OPTS = \
- -Z target-applies-to-host \
- --target=$(RUSTC_TARGET_NAME) \
- --manifest-path=$(@D)/Cargo.toml
-
-ifeq ($(BR2_ENABLE_RUNTIME_DEBUG),y)
-RIPGREP_CARGO_BIN_SUBDIR = debug
-else
-RIPGREP_CARGO_OPTS += --release
-RIPGREP_CARGO_BIN_SUBDIR = release
-endif
-
-define RIPGREP_BUILD_CMDS
- $(TARGET_MAKE_ENV) $(RIPGREP_CARGO_ENV) \
- cargo build $(RIPGREP_CARGO_OPTS)
-endef
-
-define RIPGREP_INSTALL_TARGET_CMDS
- $(INSTALL) -D -m 0755 $(@D)/$(RIPGREP_BIN_DIR)/rg \
- $(TARGET_DIR)/usr/bin/rg
-endef
-
-$(eval $(generic-package))
+$(eval $(cargo-package))
--
2.33.1
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot
^ permalink raw reply related [flat|nested] 38+ messages in thread* [Buildroot] [PATCH v3 08/11] package/sentry-cli: re-add package
2022-01-06 20:59 [Buildroot] [PATCH v3 00/11] Support for Cargo and Go vendoring Thomas Petazzoni
` (6 preceding siblings ...)
2022-01-06 20:59 ` [Buildroot] [PATCH v3 07/11] package/ripgrep: convert to cargo infrastructure Thomas Petazzoni
@ 2022-01-06 20:59 ` Thomas Petazzoni
2022-01-06 22:04 ` Christian Stewart via buildroot
2022-01-06 20:59 ` [Buildroot] [PATCH v3 09/11] package/tinifier: new package Thomas Petazzoni
` (4 subsequent siblings)
12 siblings, 1 reply; 38+ messages in thread
From: Thomas Petazzoni @ 2022-01-06 20:59 UTC (permalink / raw)
To: James Hilliard, Christian Stewart, Buildroot List, Matt Weber,
Yann E. MORIN
Cc: Joseph Kogut, Patrick Havelange, Thomas Petazzoni
This reverts commit d19077677fc71fc8c8cd73945606f773d907a99a, but the
package now uses the cargo-package infrastructure with its vendoring
mechanism, which allows us to avoid the problems that caused the
package to be removed in d19077677fc71fc8c8cd73945606f773d907a99a.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
---
DEVELOPERS | 1 +
package/Config.in.host | 1 +
package/sentry-cli/Config.in.host | 10 ++++++++++
package/sentry-cli/sentry-cli.hash | 3 +++
package/sentry-cli/sentry-cli.mk | 14 ++++++++++++++
5 files changed, 29 insertions(+)
create mode 100644 package/sentry-cli/Config.in.host
create mode 100644 package/sentry-cli/sentry-cli.hash
create mode 100644 package/sentry-cli/sentry-cli.mk
diff --git a/DEVELOPERS b/DEVELOPERS
index a52a8cfa38..55ea7805f7 100644
--- a/DEVELOPERS
+++ b/DEVELOPERS
@@ -1508,6 +1508,7 @@ F: package/python-schedule/
F: package/python-sentry-sdk/
F: package/python-websockets/
F: package/python-xlib/
+F: package/sentry-cli/
F: package/sentry-native/
F: package/unclutter-xfixes/
diff --git a/package/Config.in.host b/package/Config.in.host
index 7400da5894..b3e00bb482 100644
--- a/package/Config.in.host
+++ b/package/Config.in.host
@@ -81,6 +81,7 @@ menu "Host utilities"
source "package/s6-rc/Config.in.host"
source "package/sam-ba/Config.in.host"
source "package/sdbusplus/Config.in.host"
+ source "package/sentry-cli/Config.in.host"
source "package/sloci-image/Config.in.host"
source "package/squashfs/Config.in.host"
source "package/sunxi-tools/Config.in.host"
diff --git a/package/sentry-cli/Config.in.host b/package/sentry-cli/Config.in.host
new file mode 100644
index 0000000000..3124015367
--- /dev/null
+++ b/package/sentry-cli/Config.in.host
@@ -0,0 +1,10 @@
+config BR2_PACKAGE_HOST_SENTRY_CLI
+ bool "host sentry-cli"
+ depends on BR2_PACKAGE_HOST_RUSTC_ARCH_SUPPORTS
+ select BR2_PACKAGE_HOST_RUSTC
+ help
+ Official Sentry command line interface
+
+ For managing debug information files and source maps.
+
+ https://docs.sentry.io/cli/
diff --git a/package/sentry-cli/sentry-cli.hash b/package/sentry-cli/sentry-cli.hash
new file mode 100644
index 0000000000..73f818125f
--- /dev/null
+++ b/package/sentry-cli/sentry-cli.hash
@@ -0,0 +1,3 @@
+# locally calculated
+sha256 d687336379fd96c4ee73f24bd062d31672bd7453af53311b72977be3c8c78fef sentry-cli-1.59.0.tar.gz
+sha256 9503def7b54ceb6e3cd182fd59bc05d3a30d7eae481e65aaba4b495133c83c14 LICENSE
diff --git a/package/sentry-cli/sentry-cli.mk b/package/sentry-cli/sentry-cli.mk
new file mode 100644
index 0000000000..58f5f1e325
--- /dev/null
+++ b/package/sentry-cli/sentry-cli.mk
@@ -0,0 +1,14 @@
+################################################################################
+#
+# sentry-cli
+#
+################################################################################
+
+SENTRY_CLI_VERSION = 1.59.0
+SENTRY_CLI_SITE = $(call github,getsentry,sentry-cli,$(SENTRY_CLI_VERSION))
+SENTRY_CLI_LICENSE = BSD-3-clause
+SENTRY_CLI_LICENSE_FILES = LICENSE
+
+HOST_SENTRY_CLI_DEPENDENCIES = host-zlib
+
+$(eval $(host-cargo-package))
--
2.33.1
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot
^ permalink raw reply related [flat|nested] 38+ messages in thread* Re: [Buildroot] [PATCH v3 08/11] package/sentry-cli: re-add package
2022-01-06 20:59 ` [Buildroot] [PATCH v3 08/11] package/sentry-cli: re-add package Thomas Petazzoni
@ 2022-01-06 22:04 ` Christian Stewart via buildroot
2022-01-06 22:29 ` Thomas Petazzoni
0 siblings, 1 reply; 38+ messages in thread
From: Christian Stewart via buildroot @ 2022-01-06 22:04 UTC (permalink / raw)
To: Thomas Petazzoni
Cc: Joseph Kogut, Christian Stewart, Matt Weber, Patrick Havelange,
James Hilliard, Buildroot List, Yann E. MORIN
Hi Thomas, all,
I get a build error with "make host-sentry-cli":
error: failed to run custom build command for `libc v0.2.71`
Caused by:
process didn't exit successfully:
`/build/host-sentry-cli-1.59.0/target/release/build/libc-a39b7215be30da0c/build-script-build`
(exit status: 1)
--- stderr
/build/host-sentry-cli-1.59.0/target/release/build/libc-a39b7215be30da0c/build-script-build:
/lib64/libc.so.6: version `GLIBC_2.34' not found (required by
/build/host-sentry-cli-1.59.0/target/release/build/libc-a39b7215be30da0c/build-script-build)
I guess this is a rust-related issue? I think it's complaining about
the glibc version of my build system.
On Thu, Jan 6, 2022 at 1:00 PM Thomas Petazzoni
<thomas.petazzoni@bootlin.com> wrote:
###############################################
> +SENTRY_CLI_VERSION = 1.59.0
> +SENTRY_CLI_SITE = $(call github,getsentry,sentry-cli,$(SENTRY_CLI_VERSION))
Thanks & best regards,
Christian
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [Buildroot] [PATCH v3 08/11] package/sentry-cli: re-add package
2022-01-06 22:04 ` Christian Stewart via buildroot
@ 2022-01-06 22:29 ` Thomas Petazzoni
2022-01-06 22:38 ` Christian Stewart via buildroot
0 siblings, 1 reply; 38+ messages in thread
From: Thomas Petazzoni @ 2022-01-06 22:29 UTC (permalink / raw)
To: Christian Stewart
Cc: Joseph Kogut, Matt Weber, Patrick Havelange, James Hilliard,
Buildroot List, Yann E. MORIN
Hello Christian,
On Thu, 6 Jan 2022 14:04:56 -0800
Christian Stewart <christian@paral.in> wrote:
> I get a build error with "make host-sentry-cli":
>
> error: failed to run custom build command for `libc v0.2.71`
>
> Caused by:
> process didn't exit successfully:
> `/build/host-sentry-cli-1.59.0/target/release/build/libc-a39b7215be30da0c/build-script-build`
> (exit status: 1)
> --- stderr
> /build/host-sentry-cli-1.59.0/target/release/build/libc-a39b7215be30da0c/build-script-build:
> /lib64/libc.so.6: version `GLIBC_2.34' not found (required by
> /build/host-sentry-cli-1.59.0/target/release/build/libc-a39b7215be30da0c/build-script-build)
>
> I guess this is a rust-related issue? I think it's complaining about
> the glibc version of my build system.
It build fine on two machines here. Could you provide more details
about your host distribution? Perhaps provide a docker container that
reproduces this?
Thomas
--
Thomas Petazzoni, co-owner and CEO, Bootlin
Embedded Linux and Kernel engineering and training
https://bootlin.com
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [Buildroot] [PATCH v3 08/11] package/sentry-cli: re-add package
2022-01-06 22:29 ` Thomas Petazzoni
@ 2022-01-06 22:38 ` Christian Stewart via buildroot
2022-01-06 22:59 ` Thomas Petazzoni
0 siblings, 1 reply; 38+ messages in thread
From: Christian Stewart via buildroot @ 2022-01-06 22:38 UTC (permalink / raw)
To: Thomas Petazzoni
Cc: Joseph Kogut, Christian Stewart, Matt Weber, Patrick Havelange,
James Hilliard, Buildroot List, Yann E. MORIN
Hi Thomas,
On Thu, Jan 6, 2022 at 2:29 PM Thomas Petazzoni
<thomas.petazzoni@bootlin.com> wrote:
> On Thu, 6 Jan 2022 14:04:56 -0800
> Christian Stewart <christian@paral.in> wrote:
>
> > I get a build error with "make host-sentry-cli":
> >
> > error: failed to run custom build command for `libc v0.2.71`
> > /lib64/libc.so.6: version `GLIBC_2.34' not found (required by
> It build fine on two machines here. Could you provide more details
> about your host distribution? Perhaps provide a docker container that
> reproduces this?
I'm running an up-to-date Gentoo system at version 2.33.
Currently 2.34 is masked:
https://packages.gentoo.org/packages/sys-libs/glibc
I guess the host-rust packages expect at least glibc 2.34.
Best regards,
Christian Stewart
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [Buildroot] [PATCH v3 08/11] package/sentry-cli: re-add package
2022-01-06 22:38 ` Christian Stewart via buildroot
@ 2022-01-06 22:59 ` Thomas Petazzoni
0 siblings, 0 replies; 38+ messages in thread
From: Thomas Petazzoni @ 2022-01-06 22:59 UTC (permalink / raw)
To: Christian Stewart
Cc: Joseph Kogut, Matt Weber, Patrick Havelange, James Hilliard,
Buildroot List, Yann E. MORIN
Hello,
On Thu, 6 Jan 2022 14:38:06 -0800
Christian Stewart <christian@paral.in> wrote:
> I'm running an up-to-date Gentoo system at version 2.33.
>
> Currently 2.34 is masked:
>
> https://packages.gentoo.org/packages/sys-libs/glibc
>
> I guess the host-rust packages expect at least glibc 2.34.
Seems unlikely, because I just built the following configuration in the
buildroot/base:20200814.2228 container, which is very old Debian with
glibc 2.24 (yes 2.24).
BR2_aarch64=y
BR2_TOOLCHAIN_EXTERNAL=y
BR2_TOOLCHAIN_EXTERNAL_BOOTLIN=y
BR2_TOOLCHAIN_EXTERNAL_BOOTLIN_AARCH64_GLIBC_STABLE=y
BR2_INIT_NONE=y
BR2_SYSTEM_BIN_SH_NONE=y
# BR2_PACKAGE_BUSYBOX is not set
# BR2_TARGET_ROOTFS_TAR is not set
BR2_PACKAGE_HOST_SENTRY_CLI=y
$ dpkg -l | grep libc6
ii libc6:amd64 2.24-11+deb9u1 amd64 GNU C Library: Shared libraries
ii libc6:i386 2.24-11+deb9u1 i386 GNU C Library: Shared libraries
Best regards,
Thomas
--
Thomas Petazzoni, co-owner and CEO, Bootlin
Embedded Linux and Kernel engineering and training
https://bootlin.com
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot
^ permalink raw reply [flat|nested] 38+ messages in thread
* [Buildroot] [PATCH v3 09/11] package/tinifier: new package
2022-01-06 20:59 [Buildroot] [PATCH v3 00/11] Support for Cargo and Go vendoring Thomas Petazzoni
` (7 preceding siblings ...)
2022-01-06 20:59 ` [Buildroot] [PATCH v3 08/11] package/sentry-cli: re-add package Thomas Petazzoni
@ 2022-01-06 20:59 ` Thomas Petazzoni
2022-01-06 22:30 ` Christian Stewart via buildroot
2022-01-06 20:59 ` [Buildroot] [PATCH v3 10/11] package/embiggen-disk: " Thomas Petazzoni
` (3 subsequent siblings)
12 siblings, 1 reply; 38+ messages in thread
From: Thomas Petazzoni @ 2022-01-06 20:59 UTC (permalink / raw)
To: James Hilliard, Christian Stewart, Buildroot List, Matt Weber,
Yann E. MORIN
Cc: Patrick Havelange, Anisse Astier, Thomas Petazzoni
This is a Go package that needs vendor modules to be downloaded at
build time.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
---
DEVELOPERS | 1 +
package/Config.in | 1 +
package/tinifier/Config.in | 10 ++++++++++
package/tinifier/tinifier.hash | 3 +++
package/tinifier/tinifier.mk | 13 +++++++++++++
5 files changed, 28 insertions(+)
create mode 100644 package/tinifier/Config.in
create mode 100644 package/tinifier/tinifier.hash
create mode 100644 package/tinifier/tinifier.mk
diff --git a/DEVELOPERS b/DEVELOPERS
index 55ea7805f7..774deb012f 100644
--- a/DEVELOPERS
+++ b/DEVELOPERS
@@ -2765,6 +2765,7 @@ F: package/rtc-tools/
F: package/sam-ba/
F: package/scons/
F: package/squashfs/
+F: package/tinifier/
F: package/wayland/
F: package/weston/
F: support/testing/tests/boot/test_syslinux.py
diff --git a/package/Config.in b/package/Config.in
index cb15574746..e47c49ade5 100644
--- a/package/Config.in
+++ b/package/Config.in
@@ -314,6 +314,7 @@ comment "Graphic applications"
source "package/rrdtool/Config.in"
source "package/stellarium/Config.in"
source "package/tesseract-ocr/Config.in"
+ source "package/tinifier/Config.in"
comment "Graphic libraries"
source "package/cegui/Config.in"
diff --git a/package/tinifier/Config.in b/package/tinifier/Config.in
new file mode 100644
index 0000000000..fbadfe6bd9
--- /dev/null
+++ b/package/tinifier/Config.in
@@ -0,0 +1,10 @@
+config BR2_PACKAGE_TINIFIER
+ bool "tinifier"
+ depends on BR2_PACKAGE_HOST_GO_TARGET_ARCH_SUPPORTS
+ help
+ CLI tool for images compressing
+
+ This tool uses tinypng.com API endpoint for compressing your
+ local jpg/png images (it supports parallel jobs).
+
+ https://github.com/tarampampam/tinifier
diff --git a/package/tinifier/tinifier.hash b/package/tinifier/tinifier.hash
new file mode 100644
index 0000000000..5681ed6a55
--- /dev/null
+++ b/package/tinifier/tinifier.hash
@@ -0,0 +1,3 @@
+# Locally calculated
+sha256 927e966f9d23ea7cc1942967a8fc16ef4c55c51bd096d44f54b418844a51aa12 tinifier-3.4.0.tar.gz
+sha256 791d8fd993ace44d4d83e2f4820a64d5ad3e37412f029afad46d17a9259de2b6 LICENSE
diff --git a/package/tinifier/tinifier.mk b/package/tinifier/tinifier.mk
new file mode 100644
index 0000000000..d7ed6f7c93
--- /dev/null
+++ b/package/tinifier/tinifier.mk
@@ -0,0 +1,13 @@
+################################################################################
+#
+# tinifier
+#
+################################################################################
+
+TINIFIER_VERSION = 3.4.0
+TINIFIER_SITE = $(call github,tarampampam,tinifier,v$(TINIFIER_VERSION))
+TINIFIER_LICENSE = MIT
+TINIFIER_LICENSE_FILES = LICENSE
+TINIFIER_GOMOD = ./cmd/tinifier
+
+$(eval $(golang-package))
--
2.33.1
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot
^ permalink raw reply related [flat|nested] 38+ messages in thread* Re: [Buildroot] [PATCH v3 09/11] package/tinifier: new package
2022-01-06 20:59 ` [Buildroot] [PATCH v3 09/11] package/tinifier: new package Thomas Petazzoni
@ 2022-01-06 22:30 ` Christian Stewart via buildroot
0 siblings, 0 replies; 38+ messages in thread
From: Christian Stewart via buildroot @ 2022-01-06 22:30 UTC (permalink / raw)
To: Thomas Petazzoni
Cc: Christian Stewart, Matt Weber, Patrick Havelange, Anisse Astier,
James Hilliard, Buildroot List, Yann E. MORIN
Hi all,
On Thu, Jan 6, 2022 at 1:00 PM Thomas Petazzoni
<thomas.petazzoni@bootlin.com> wrote:
>
> This is a Go package that needs vendor modules to be downloaded at
> build time.
>
> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
> ---
> +TINIFIER_VERSION = 3.4.0
> +TINIFIER_SITE = $(call github,tarampampam,tinifier,v$(TINIFIER_VERSION))
Build tested.
Reviewed-by: Christian Stewart <christian@paral.in>
Thanks,
Christian Stewart
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot
^ permalink raw reply [flat|nested] 38+ messages in thread
* [Buildroot] [PATCH v3 10/11] package/embiggen-disk: new package
2022-01-06 20:59 [Buildroot] [PATCH v3 00/11] Support for Cargo and Go vendoring Thomas Petazzoni
` (8 preceding siblings ...)
2022-01-06 20:59 ` [Buildroot] [PATCH v3 09/11] package/tinifier: new package Thomas Petazzoni
@ 2022-01-06 20:59 ` Thomas Petazzoni
2022-01-06 22:11 ` Christian Stewart via buildroot
2022-01-06 20:59 ` [Buildroot] [PATCH v3 11/11] package/gocryptfs: " Thomas Petazzoni
` (2 subsequent siblings)
12 siblings, 1 reply; 38+ messages in thread
From: Thomas Petazzoni @ 2022-01-06 20:59 UTC (permalink / raw)
To: James Hilliard, Christian Stewart, Buildroot List, Matt Weber,
Yann E. MORIN
Cc: Patrick Havelange, Anisse Astier, Thomas Petazzoni
From: Christian Stewart via buildroot <buildroot@buildroot.org>
embiggen-disk is a tool to automatically resize disks to fill available space.
Patch submitted upstream: https://github.com/bradfitz/embiggen-disk/pull/13
Adds support for /dev/mmcblk0pN type paths.
Signed-off-by: Christian Stewart <christian@paral.in>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
---
DEVELOPERS | 1 +
package/Config.in | 1 +
...-Fix-resizing-of-dev-mmcblk0pN-paths.patch | 45 +++++++++++++++++++
package/embiggen-disk/Config.in | 19 ++++++++
package/embiggen-disk/embiggen-disk.hash | 3 ++
package/embiggen-disk/embiggen-disk.mk | 12 +++++
6 files changed, 81 insertions(+)
create mode 100644 package/embiggen-disk/0001-Fix-resizing-of-dev-mmcblk0pN-paths.patch
create mode 100644 package/embiggen-disk/Config.in
create mode 100644 package/embiggen-disk/embiggen-disk.hash
create mode 100644 package/embiggen-disk/embiggen-disk.mk
diff --git a/DEVELOPERS b/DEVELOPERS
index 774deb012f..2649cf4ac4 100644
--- a/DEVELOPERS
+++ b/DEVELOPERS
@@ -518,6 +518,7 @@ F: package/delve/
F: package/docker-cli/
F: package/docker-engine/
F: package/docker-proxy/
+F: package/embiggen-disk/
F: package/fuse-overlayfs/
F: package/go/
F: package/mbpfan/
diff --git a/package/Config.in b/package/Config.in
index e47c49ade5..34a36279ea 100644
--- a/package/Config.in
+++ b/package/Config.in
@@ -2516,6 +2516,7 @@ menu "System tools"
source "package/earlyoom/Config.in"
source "package/efibootmgr/Config.in"
source "package/efivar/Config.in"
+ source "package/embiggen-disk/Config.in"
source "package/emlog/Config.in"
source "package/ftop/Config.in"
source "package/getent/Config.in"
diff --git a/package/embiggen-disk/0001-Fix-resizing-of-dev-mmcblk0pN-paths.patch b/package/embiggen-disk/0001-Fix-resizing-of-dev-mmcblk0pN-paths.patch
new file mode 100644
index 0000000000..31cfea9dde
--- /dev/null
+++ b/package/embiggen-disk/0001-Fix-resizing-of-dev-mmcblk0pN-paths.patch
@@ -0,0 +1,45 @@
+From b628e4c561cb4d3ffc92a5d7aac8f4967e44977e Mon Sep 17 00:00:00 2001
+From: Christian Stewart <christian@paral.in>
+Date: Thu, 27 May 2021 20:07:07 -0700
+Subject: [PATCH] Fix resizing of /dev/mmcblk0pN paths
+
+Signed-off-by: Christian Stewart <christian@paral.in>
+---
+ fs.go | 4 +++-
+ part.go | 5 +++++
+ 2 files changed, 8 insertions(+), 1 deletion(-)
+
+diff --git a/fs.go b/fs.go
+index 0b83e24..8ab3690 100644
+--- a/fs.go
++++ b/fs.go
+@@ -67,7 +67,9 @@ func (e fsResizer) DepResizer() (Resizer, error) {
+ if dev == "/dev/root" {
+ return nil, errors.New("unexpected device /dev/root from statFS")
+ }
+- if (strings.HasPrefix(dev, "/dev/sd") || strings.HasPrefix(dev, "/dev/nvme")) &&
++ if (strings.HasPrefix(dev, "/dev/sd") ||
++ strings.HasPrefix(dev, "/dev/mmcblk") ||
++ strings.HasPrefix(dev, "/dev/nvme")) &&
+ devEndsInNumber(dev) {
+ vlogf("fsResizer.DepResizer: returning partitionResizer(%q)", dev)
+ return partitionResizer(dev), nil
+diff --git a/part.go b/part.go
+index 0315a6e..f3e280f 100644
+--- a/part.go
++++ b/part.go
+@@ -51,6 +51,11 @@ func diskDev(partDev string) string {
+ if strings.HasPrefix(partDev, "/dev/sd") {
+ return strings.TrimRight(partDev, "0123456789")
+ }
++ if strings.HasPrefix(partDev, "/dev/mmcblk") {
++ v := strings.TrimRight(partDev, "0123456789")
++ v = strings.TrimSuffix(v, "p")
++ return v
++ }
+ if strings.HasPrefix(partDev, "/dev/nvme") {
+ chopP := regexp.MustCompile(`p\d+$`)
+ if !chopP.MatchString(partDev) {
+--
+2.31.1
+
diff --git a/package/embiggen-disk/Config.in b/package/embiggen-disk/Config.in
new file mode 100644
index 0000000000..667d71c86e
--- /dev/null
+++ b/package/embiggen-disk/Config.in
@@ -0,0 +1,19 @@
+config BR2_PACKAGE_EMBIGGEN_DISK
+ bool "embiggen-disk"
+ depends on BR2_PACKAGE_HOST_GO_TARGET_ARCH_SUPPORTS
+ depends on BR2_PACKAGE_HOST_GO_TARGET_CGO_LINKING_SUPPORTS
+ depends on BR2_TOOLCHAIN_HAS_THREADS
+ depends on !BR2_TOOLCHAIN_USES_UCLIBC # no fexecve
+ depends on BR2_USE_MMU # util-linux
+ select BR2_PACKAGE_UTIL_LINUX # sfdisk
+ select BR2_PACKAGE_UTIL_LINUX_BINARIES # sfdisk
+ help
+ embiggen-disk is a tool to resize disk partitions at runtime.
+
+ https://github.com/bradfitz/embiggen-disk
+
+comment "embiggen-disk needs a glibc or musl toolchain w/ threads"
+ depends on BR2_USE_MMU
+ depends on BR2_PACKAGE_HOST_GO_TARGET_ARCH_SUPPORTS && \
+ BR2_PACKAGE_HOST_GO_TARGET_CGO_LINKING_SUPPORTS
+ depends on !BR2_TOOLCHAIN_HAS_THREADS || BR2_TOOLCHAIN_USES_UCLIBC
diff --git a/package/embiggen-disk/embiggen-disk.hash b/package/embiggen-disk/embiggen-disk.hash
new file mode 100644
index 0000000000..274958edc4
--- /dev/null
+++ b/package/embiggen-disk/embiggen-disk.hash
@@ -0,0 +1,3 @@
+# Locally computed
+sha256 5ef943b1f6468cf0eaf9d7e6ca22d8b5692cbb5b80f19d95468fb41b675b986c embiggen-disk-c554fc1c93a4004ce0b6a3f69b0dabe0481a2308.tar.gz
+sha256 063aedec1652c5a05c2d04c40e032b932453142ee8ef7fd53c04a9acc127fc95 LICENSE
diff --git a/package/embiggen-disk/embiggen-disk.mk b/package/embiggen-disk/embiggen-disk.mk
new file mode 100644
index 0000000000..5d5ac15ae8
--- /dev/null
+++ b/package/embiggen-disk/embiggen-disk.mk
@@ -0,0 +1,12 @@
+################################################################################
+#
+# embiggen-disk
+#
+################################################################################
+
+EMBIGGEN_DISK_VERSION = c554fc1c93a4004ce0b6a3f69b0dabe0481a2308
+EMBIGGEN_DISK_SITE = $(call github,bradfitz,embiggen-disk,$(EMBIGGEN_DISK_VERSION))
+EMBIGGEN_DISK_LICENSE = Apache-2.0
+EMBIGGEN_DISK_LICENSE_FILES = LICENSE
+
+$(eval $(golang-package))
--
2.33.1
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot
^ permalink raw reply related [flat|nested] 38+ messages in thread* Re: [Buildroot] [PATCH v3 10/11] package/embiggen-disk: new package
2022-01-06 20:59 ` [Buildroot] [PATCH v3 10/11] package/embiggen-disk: " Thomas Petazzoni
@ 2022-01-06 22:11 ` Christian Stewart via buildroot
0 siblings, 0 replies; 38+ messages in thread
From: Christian Stewart via buildroot @ 2022-01-06 22:11 UTC (permalink / raw)
To: Thomas Petazzoni
Cc: Christian Stewart, Matt Weber, Patrick Havelange, Anisse Astier,
James Hilliard, Buildroot List, Yann E. MORIN
Hi all,
On Thu, Jan 6, 2022 at 1:00 PM Thomas Petazzoni
<thomas.petazzoni@bootlin.com> wrote:
>
> From: Christian Stewart via buildroot <buildroot@buildroot.org>
>
> embiggen-disk is a tool to automatically resize disks to fill available space.
>
> +EMBIGGEN_DISK_VERSION = c554fc1c93a4004ce0b6a3f69b0dabe0481a2308
> +EMBIGGEN_DISK_SITE = $(call github,bradfitz,embiggen-disk,$(EMBIGGEN_DISK_VERSION))
The PR to fix mmcblk resizing was just merged (thanks bradfitz):
https://github.com/bradfitz/embiggen-disk/pull/13#event-5854783562
So the patch can be dropped and the VERSION changed to:
9e7b2fc7b99c4dece41a805489a6ca377ce55a62
Thanks & best,
Christian Stewart
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot
^ permalink raw reply [flat|nested] 38+ messages in thread
* [Buildroot] [PATCH v3 11/11] package/gocryptfs: new package
2022-01-06 20:59 [Buildroot] [PATCH v3 00/11] Support for Cargo and Go vendoring Thomas Petazzoni
` (9 preceding siblings ...)
2022-01-06 20:59 ` [Buildroot] [PATCH v3 10/11] package/embiggen-disk: " Thomas Petazzoni
@ 2022-01-06 20:59 ` Thomas Petazzoni
2022-01-06 22:26 ` [Buildroot] [PATCH v3 00/11] Support for Cargo and Go vendoring Christian Stewart via buildroot
2022-01-08 22:39 ` Thomas Petazzoni
12 siblings, 0 replies; 38+ messages in thread
From: Thomas Petazzoni @ 2022-01-06 20:59 UTC (permalink / raw)
To: James Hilliard, Christian Stewart, Buildroot List, Matt Weber,
Yann E. MORIN
Cc: Patrick Havelange, Anisse Astier, Thomas Petazzoni
From: Christian Stewart via buildroot <buildroot@buildroot.org>
Adds the gocryptfs encrypted FUSE filesystem.
Currently uses without_openssl build tag, to use the native Go cryptography.
However, the package could be improved by conditionally enabling openssl if it
is also configured to be built by Buildroot.
Signed-off-by: Christian Stewart <christian@paral.in>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
---
DEVELOPERS | 1 +
package/Config.in | 1 +
package/gocryptfs/Config.in | 12 ++++++++++++
package/gocryptfs/gocryptfs.hash | 3 +++
package/gocryptfs/gocryptfs.mk | 19 +++++++++++++++++++
5 files changed, 36 insertions(+)
create mode 100644 package/gocryptfs/Config.in
create mode 100644 package/gocryptfs/gocryptfs.hash
create mode 100644 package/gocryptfs/gocryptfs.mk
diff --git a/DEVELOPERS b/DEVELOPERS
index 2649cf4ac4..bf58a20fc4 100644
--- a/DEVELOPERS
+++ b/DEVELOPERS
@@ -521,6 +521,7 @@ F: package/docker-proxy/
F: package/embiggen-disk/
F: package/fuse-overlayfs/
F: package/go/
+F: package/gocryptfs/
F: package/mbpfan/
F: package/mosh/
F: package/pkg-golang.mk
diff --git a/package/Config.in b/package/Config.in
index 34a36279ea..2946ce7a4b 100644
--- a/package/Config.in
+++ b/package/Config.in
@@ -221,6 +221,7 @@ menu "Filesystem and flash utilities"
source "package/genext2fs/Config.in"
source "package/genpart/Config.in"
source "package/genromfs/Config.in"
+ source "package/gocryptfs/Config.in"
source "package/imx-usb-loader/Config.in"
source "package/mmc-utils/Config.in"
source "package/mtd/Config.in"
diff --git a/package/gocryptfs/Config.in b/package/gocryptfs/Config.in
new file mode 100644
index 0000000000..2630cafdcf
--- /dev/null
+++ b/package/gocryptfs/Config.in
@@ -0,0 +1,12 @@
+config BR2_PACKAGE_GOCRYPTFS
+ bool "gocryptfs"
+ depends on BR2_PACKAGE_HOST_GO_TARGET_ARCH_SUPPORTS
+ depends on BR2_TOOLCHAIN_HAS_THREADS
+ help
+ gocryptfs is an encrypted FUSE overlay filesystem.
+
+ https://github.com/rfjakob/gocryptfs
+
+comment "gocryptfs needs a toolchain w/ threads"
+ depends on BR2_PACKAGE_HOST_GO_TARGET_ARCH_SUPPORTS
+ depends on !BR2_TOOLCHAIN_HAS_THREADS
diff --git a/package/gocryptfs/gocryptfs.hash b/package/gocryptfs/gocryptfs.hash
new file mode 100644
index 0000000000..94487815f5
--- /dev/null
+++ b/package/gocryptfs/gocryptfs.hash
@@ -0,0 +1,3 @@
+# Locally computed
+sha256 4d89b76fe0b0c7218099f6f0ea6c1c87efe13927d95579a6ede82f20b6dbe48a gocryptfs-2.2.0.tar.gz
+sha256 322a7e3b02cf18e38b7e6b18cafefb773df8676c65634b34e8a2beb931294a4b LICENSE
diff --git a/package/gocryptfs/gocryptfs.mk b/package/gocryptfs/gocryptfs.mk
new file mode 100644
index 0000000000..2a00e9a959
--- /dev/null
+++ b/package/gocryptfs/gocryptfs.mk
@@ -0,0 +1,19 @@
+################################################################################
+#
+# gocryptfs
+#
+################################################################################
+
+GOCRYPTFS_VERSION = 2.2.0
+GOCRYPTFS_SITE = $(call github,rfjakob,gocryptfs,v$(GOCRYPTFS_VERSION))
+GOCRYPTFS_LICENSE = MIT
+GOCRYPTFS_LICENSE_FILES = LICENSE
+
+GOCRYPTFS_GOMOD = github.com/rfjakob/gocryptfs/v2
+
+GOCRYPTFS_LDFLAGS = \
+ -X main.GitVersion=$(GOCRYPTFS_VERSION) \
+ -X main.GitVersionFuse=[vendored]
+GOCRYPTFS_TAGS = without_openssl
+
+$(eval $(golang-package))
--
2.33.1
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot
^ permalink raw reply related [flat|nested] 38+ messages in thread* Re: [Buildroot] [PATCH v3 00/11] Support for Cargo and Go vendoring
2022-01-06 20:59 [Buildroot] [PATCH v3 00/11] Support for Cargo and Go vendoring Thomas Petazzoni
` (10 preceding siblings ...)
2022-01-06 20:59 ` [Buildroot] [PATCH v3 11/11] package/gocryptfs: " Thomas Petazzoni
@ 2022-01-06 22:26 ` Christian Stewart via buildroot
2022-01-06 22:29 ` Thomas Petazzoni
2022-01-08 22:39 ` Thomas Petazzoni
12 siblings, 1 reply; 38+ messages in thread
From: Christian Stewart via buildroot @ 2022-01-06 22:26 UTC (permalink / raw)
To: Thomas Petazzoni
Cc: Matt Weber, Patrick Havelange, James Hilliard, Christian Stewart,
Buildroot List, Yann E. MORIN
Hi Thomas,
Your time spent on this series is much appreciated.
On Thu, Jan 6, 2022 at 1:00 PM Thomas Petazzoni
<thomas.petazzoni@bootlin.com> wrote:
> Christian Stewart via buildroot (2):
> package/embiggen-disk: new package
> package/gocryptfs: new package
It'd be nice if the author on these commits could be edited to remove
the "via buildroot"
... this was added by a Patchwork bug a while back.
Best regards,
Christian
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot
^ permalink raw reply [flat|nested] 38+ messages in thread* Re: [Buildroot] [PATCH v3 00/11] Support for Cargo and Go vendoring
2022-01-06 22:26 ` [Buildroot] [PATCH v3 00/11] Support for Cargo and Go vendoring Christian Stewart via buildroot
@ 2022-01-06 22:29 ` Thomas Petazzoni
0 siblings, 0 replies; 38+ messages in thread
From: Thomas Petazzoni @ 2022-01-06 22:29 UTC (permalink / raw)
To: Christian Stewart
Cc: Matt Weber, Patrick Havelange, James Hilliard, Christian Stewart,
Buildroot List, Yann E. MORIN
On Thu, 6 Jan 2022 14:26:13 -0800
Christian Stewart <christian@paral.in> wrote:
> It'd be nice if the author on these commits could be edited to remove
> the "via buildroot"
>
> ... this was added by a Patchwork bug a while back.
ACK, will fix!
Thomas
--
Thomas Petazzoni, co-owner and CEO, Bootlin
Embedded Linux and Kernel engineering and training
https://bootlin.com
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [Buildroot] [PATCH v3 00/11] Support for Cargo and Go vendoring
2022-01-06 20:59 [Buildroot] [PATCH v3 00/11] Support for Cargo and Go vendoring Thomas Petazzoni
` (11 preceding siblings ...)
2022-01-06 22:26 ` [Buildroot] [PATCH v3 00/11] Support for Cargo and Go vendoring Christian Stewart via buildroot
@ 2022-01-08 22:39 ` Thomas Petazzoni
12 siblings, 0 replies; 38+ messages in thread
From: Thomas Petazzoni @ 2022-01-08 22:39 UTC (permalink / raw)
To: James Hilliard, Christian Stewart, Buildroot List, Matt Weber,
Yann E. MORIN
Cc: Patrick Havelange
Hello,
On Thu, 6 Jan 2022 21:59:48 +0100
Thomas Petazzoni <thomas.petazzoni@bootlin.com> wrote:
> Christian Stewart via buildroot (2):
> package/embiggen-disk: new package
> package/gocryptfs: new package
>
> Patrick Havelange (3):
> package/pkg-cargo.mk: introduce the cargo package infrastructure
> docs/manual/cargo: document the cargo-package infrastructure
> package/ripgrep: convert to cargo infrastructure
>
> Thomas Petazzoni (6):
> support/download/dl-wrapper: add concept of download post-processing
> package/pkg-download.mk: add <pkg>_DOWNLOAD_POST_PROCESS variable
> support/download/post-process-helpers: add helper function for post
> process scripts
> package/pkg-golang.mk: implement Go vendoring support
> package/sentry-cli: re-add package
> package/tinifier: new package
So, this series has been applied, with some additional tweaks based on
your feedback + review from Yann and Arnout.
Here are the things that have *not* yet been taken into account, and
left for future work:
* We still use "cargo install" even though that causes rebuilds with
some packages. Either we switch to a manual installation, or we
"wait" for some better solution from cargo maintainers.
* We don't use environment variables for everything, and some flags
are passed through a cargo config file, which is not ideal.
* We only support cargo/go packages that use .tar.gz tarballs, because
the repacking with mk_tar_gz assumes .tar.gz is used.
These 3 limitations have been captured in the TODO list at
https://elinux.org/Buildroot#Core_Buildroot_infrastructure. Patches
welcome to implement these things of course, but we wanted to get the
initial infrastructure in place and merged, so that we can build on top
of it.
Best regards,
Thomas
--
Thomas Petazzoni, co-owner and CEO, Bootlin
Embedded Linux and Kernel engineering and training
https://bootlin.com
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot
^ permalink raw reply [flat|nested] 38+ messages in thread