* [Buildroot] [PATCH v4 1/1] package/pkg-generic.mk: fix absolute paths in per-package config files
@ 2022-01-06 17:17 Adam Duskett
2022-01-08 18:22 ` Yann E. MORIN
0 siblings, 1 reply; 2+ messages in thread
From: Adam Duskett @ 2022-01-06 17:17 UTC (permalink / raw)
To: buildroot; +Cc: Adam Duskett, Thomas Petazzoni
Some packages (notably qmake packages) generate .cmake, .pc, and .pri files
with absolute paths. Unfortunately, doing so breaks per-package builds because
the paths in those files point outside the per-package sysroot for packages
that have rsynced those same files.
- Rename FIXUP_PYTHON_SYSCONFIGDATA to FIXUP_PER_PACKAGE_FILES in pkg-generic.mk
to avoid more calls to ifeq ($(BR2_PER_PACKAGE_DIRECTORIES),y)
- Add .cmake, .pc, and .pri files to the files of which to find to sed the
per-package paths.
This simple fix allows qt5 submodule packages to build with per-package enabled.
Based off of Louis-Paul's earlier work found here:
https://lore.kernel.org/buildroot/a339f273-33f3-f232-eac4-6e50427abf6d@cordier.org/
And Yann Morin's feedback found here:
https://patchwork.ozlabs.org/project/buildroot/patch/20200217212350.29750-21-anaumann@ultratronik.de/
Signed-off-by: Adam Duskett <aduskett@gmail.com>
---
changes v3 -> v4:
- Further compress the method by simply adding the files to the already
existing find command. (Yann)
changes v2 -> v3:
- Rename FIXUP_PYTHON_SYSCONFIGDATA to FIXUP_PER_PACKAGE_FILES in
pkg-generic.mk (Yann)
- Combine FIXUP_CONFIG_FILES_ABSOLUTE_PATHS with FIXUP_PER_PACKAGE_FILES (Yann)
changes v1 -> v2
- Drop patches 2 and 3 and combine them into a single macro in pkg-generic.mk
- Move the PRE_CONFIGURE_HOOK to inner-generic-package so it actually runs
- Rename FIXUP_PC_FILES_ABSOLUTE_PATHS to FIXUP_CONFIG_FILES_ABSOLUTE_PATHS
- Use $(HOST_DIR) instead of $(STAGING_DIR) to cover both stagin and host
directories in a single step.
package/pkg-generic.mk | 15 +++++++++++++--
1 file changed, 13 insertions(+), 2 deletions(-)
diff --git a/package/pkg-generic.mk b/package/pkg-generic.mk
index 6a5fe5507b..9952ff1963 100644
--- a/package/pkg-generic.mk
+++ b/package/pkg-generic.mk
@@ -113,8 +113,16 @@ endif
# fail.
# So we just use HOST_DIR as a starting point, and filter on the two directories
# of interest.
+#
+# When per-package is enabled, packages may generate files with absolute paths
+# that point to the packages per-package sysroot. However, this breaks
+# per-package isolation when the files are rsynced to a new package that depends
+# on the first package. because the hard-coded paths point to directories
+# outside the packages sysroot. Run sed on all .cmake, .pc, and .pri files in a
+# given packages sysroot containing the word "per-package" and replace the path
+# with the appropriate per-package directory.
ifeq ($(BR2_PER_PACKAGE_DIRECTORIES),y)
-define FIXUP_PYTHON_SYSCONFIGDATA
+define FIXUP_PER_PACKAGE_FILES
$(Q)find $(HOST_DIR) \
\( -path '$(HOST_DIR)/lib/python*' \
-o -path '$(STAGING_DIR)/usr/lib/python*' \
@@ -122,6 +130,9 @@ define FIXUP_PYTHON_SYSCONFIGDATA
\( \( -name "_sysconfigdata*.pyc" -delete \) \
-o \( -name "_sysconfigdata*.py" -print0 \) \
\) \
+ -o -name "*.cmake" -print0 \
+ -o -name "*.pc" -print0 \
+ -o -name "*.pri" -print0 \
| xargs -0 --no-run-if-empty \
$(SED) 's:$(PER_PACKAGE_DIR)/[^/]\+/:$(PER_PACKAGE_DIR)/$($(PKG)_NAME)/:g'
endef
@@ -836,7 +847,7 @@ $(2)_EXTRACT_CMDS ?= \
$$(TAR_OPTIONS) -)
# pre/post-steps hooks
-$(2)_POST_PREPARE_HOOKS += FIXUP_PYTHON_SYSCONFIGDATA
+$(2)_POST_PREPARE_HOOKS += FIXUP_PER_PACKAGE_FILES
ifeq ($$($(2)_TYPE),target)
ifneq ($$(HOST_$(2)_KCONFIG_VAR),)
--
2.33.1
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot
^ permalink raw reply related [flat|nested] 2+ messages in thread* Re: [Buildroot] [PATCH v4 1/1] package/pkg-generic.mk: fix absolute paths in per-package config files
2022-01-06 17:17 [Buildroot] [PATCH v4 1/1] package/pkg-generic.mk: fix absolute paths in per-package config files Adam Duskett
@ 2022-01-08 18:22 ` Yann E. MORIN
0 siblings, 0 replies; 2+ messages in thread
From: Yann E. MORIN @ 2022-01-08 18:22 UTC (permalink / raw)
To: Adam Duskett; +Cc: Thomas Petazzoni, buildroot
Adam, All,
On 2022-01-06 09:17 -0800, Adam Duskett spake thusly:
> Some packages (notably qmake packages) generate .cmake, .pc, and .pri files
> with absolute paths. Unfortunately, doing so breaks per-package builds because
> the paths in those files point outside the per-package sysroot for packages
> that have rsynced those same files.
>
> - Rename FIXUP_PYTHON_SYSCONFIGDATA to FIXUP_PER_PACKAGE_FILES in pkg-generic.mk
> to avoid more calls to ifeq ($(BR2_PER_PACKAGE_DIRECTORIES),y)
>
> - Add .cmake, .pc, and .pri files to the files of which to find to sed the
> per-package paths.
>
> This simple fix allows qt5 submodule packages to build with per-package enabled.
>
> Based off of Louis-Paul's earlier work found here:
> https://lore.kernel.org/buildroot/a339f273-33f3-f232-eac4-6e50427abf6d@cordier.org/
>
> And Yann Morin's feedback found here:
> https://patchwork.ozlabs.org/project/buildroot/patch/20200217212350.29750-21-anaumann@ultratronik.de/
>
> Signed-off-by: Adam Duskett <aduskett@gmail.com>
Thanks for this iteration!
While discussing with the otehr maintainers int he room, we concluded
that we did want an even more generic soltuion, so I've sent a further
series;
https://lore.kernel.org/buildroot/cover.1641663315.git.yann.morin.1998@free.fr/T/#t
Thanks for your persistence! ;-)
Regards,
Yann E. MORIN.
> ---
> changes v3 -> v4:
> - Further compress the method by simply adding the files to the already
> existing find command. (Yann)
>
> changes v2 -> v3:
> - Rename FIXUP_PYTHON_SYSCONFIGDATA to FIXUP_PER_PACKAGE_FILES in
> pkg-generic.mk (Yann)
>
> - Combine FIXUP_CONFIG_FILES_ABSOLUTE_PATHS with FIXUP_PER_PACKAGE_FILES (Yann)
>
> changes v1 -> v2
> - Drop patches 2 and 3 and combine them into a single macro in pkg-generic.mk
> - Move the PRE_CONFIGURE_HOOK to inner-generic-package so it actually runs
> - Rename FIXUP_PC_FILES_ABSOLUTE_PATHS to FIXUP_CONFIG_FILES_ABSOLUTE_PATHS
> - Use $(HOST_DIR) instead of $(STAGING_DIR) to cover both stagin and host
> directories in a single step.
>
> package/pkg-generic.mk | 15 +++++++++++++--
> 1 file changed, 13 insertions(+), 2 deletions(-)
>
> diff --git a/package/pkg-generic.mk b/package/pkg-generic.mk
> index 6a5fe5507b..9952ff1963 100644
> --- a/package/pkg-generic.mk
> +++ b/package/pkg-generic.mk
> @@ -113,8 +113,16 @@ endif
> # fail.
> # So we just use HOST_DIR as a starting point, and filter on the two directories
> # of interest.
> +#
> +# When per-package is enabled, packages may generate files with absolute paths
> +# that point to the packages per-package sysroot. However, this breaks
> +# per-package isolation when the files are rsynced to a new package that depends
> +# on the first package. because the hard-coded paths point to directories
> +# outside the packages sysroot. Run sed on all .cmake, .pc, and .pri files in a
> +# given packages sysroot containing the word "per-package" and replace the path
> +# with the appropriate per-package directory.
> ifeq ($(BR2_PER_PACKAGE_DIRECTORIES),y)
> -define FIXUP_PYTHON_SYSCONFIGDATA
> +define FIXUP_PER_PACKAGE_FILES
> $(Q)find $(HOST_DIR) \
> \( -path '$(HOST_DIR)/lib/python*' \
> -o -path '$(STAGING_DIR)/usr/lib/python*' \
> @@ -122,6 +130,9 @@ define FIXUP_PYTHON_SYSCONFIGDATA
> \( \( -name "_sysconfigdata*.pyc" -delete \) \
> -o \( -name "_sysconfigdata*.py" -print0 \) \
> \) \
> + -o -name "*.cmake" -print0 \
> + -o -name "*.pc" -print0 \
> + -o -name "*.pri" -print0 \
> | xargs -0 --no-run-if-empty \
> $(SED) 's:$(PER_PACKAGE_DIR)/[^/]\+/:$(PER_PACKAGE_DIR)/$($(PKG)_NAME)/:g'
> endef
> @@ -836,7 +847,7 @@ $(2)_EXTRACT_CMDS ?= \
> $$(TAR_OPTIONS) -)
>
> # pre/post-steps hooks
> -$(2)_POST_PREPARE_HOOKS += FIXUP_PYTHON_SYSCONFIGDATA
> +$(2)_POST_PREPARE_HOOKS += FIXUP_PER_PACKAGE_FILES
>
> ifeq ($$($(2)_TYPE),target)
> ifneq ($$(HOST_$(2)_KCONFIG_VAR),)
> --
> 2.33.1
>
> _______________________________________________
> buildroot mailing list
> buildroot@buildroot.org
> https://lists.buildroot.org/mailman/listinfo/buildroot
--
.-----------------.--------------------.------------------.--------------------.
| Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ |
| +33 561 099 427 `------------.-------: X AGAINST | \e/ There is no |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot
^ permalink raw reply [flat|nested] 2+ messages in thread
end of thread, other threads:[~2022-01-08 18:22 UTC | newest]
Thread overview: 2+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-01-06 17:17 [Buildroot] [PATCH v4 1/1] package/pkg-generic.mk: fix absolute paths in per-package config files Adam Duskett
2022-01-08 18:22 ` Yann E. MORIN
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).