From: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
To: buildroot@busybox.net
Subject: [Buildroot] [PATCH next v6 00/10] Per-package host/target directory support
Date: Fri, 23 Nov 2018 15:58:05 +0100 [thread overview]
Message-ID: <20181123145815.13008-1-thomas.petazzoni@bootlin.com> (raw)
Hello,
Here is a fifth iteration of the per-package SDK and target directory
implementation.
If you're interested in testing it, you can find the patch series at:
http://git.bootlin.com/users/thomas-petazzoni/buildroot/log/?h=ppsh-v6
However, if you want the full set of PPSH related patches, including
the package-specific fixes, and work on the Luarocks and Meson
infrastructure, you can pick up the following branch:
http://git.bootlin.com/users/thomas-petazzoni/buildroot/log/?h=ppsh-v6-work
I.e the ppsh-v6 branch is more for reviewing, while ppsh-v6-work is
more for testing the whole PPSH functionality.
Changes since v6
================
- Add Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr> on patch
"Makefile: move .NOTPARALLEL statement after including .config
file"
- Add Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr> on patch
Makefile: define TARGET_DIR_WARNING_FILE relative to TARGET_DIR
- Add Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr> on patch
"package/pkg-generic: adjust config scripts tweaks for per-package
directories"
- Move the change that really allows top-level parallel build (by
making .NOTPARALLEL conditional) into a separate commit, as
suggested by Yann E. Morin.
- Sort $(PACKAGES) in host-finalize and target-finalize when creating
the global HOST_DIR and global TARGET_DIR. Suggested by Yann
E. Morin.
- Split too long line in the main Makefile listing all the top-level
directories as targets. Suggested by Yann E. Morin.
- Drop change in fs/common.mk adding ROOTFS_COMMON_DEPENDENCIES to
PACKAGES, since a similar change has already been merged in
master. Noticed by Yann E. Morin.
- Use "directory/directories" instead of "folder/folders", as
suggested by Arnout Vandecappelle.
- Move prepare-per-package-folder (now named
prepare-per-package-directory) to pkg-utils.mk. Suggested by both
Yann E. Morin and Arnout Vandecappelle.
- Add a comment above prepare-per-package-directory, as suggested by
Yann E. Morin.
- Update the commit log of "core: implement per-package SDK and
target" to explain why we are modifying the check-host-rpath
script. Suggested by Arnout Vandecappelle.
Changes since v5
================
- Add patch "package/pkg-generic: adjust config scripts tweaks for
per-package folders" that adjusts how pkg-generic.mk fixes *-config
scripts, to use relative paths instead of absolute paths.
- Add patch "package/pkg-generic: make libtool .la files compatible
with per-package folders" that fixes libtool.la files for
per-package folders.
- Add patch "package/pkg-kconfig: handle KCONFIG_DEPENDENCIES with
per-package folders" that properly handles KCONFIG_DEPENDENCIES in
the pkg-kconfig infrastructure for per-package folders.
- Added Acked-by from Yann E. Morin on the following patches:
support/scripts/check-host-rpath: split condition on two statements
Makefile: evaluate CCACHE and HOST{CC,CXX} at time of use
- It is not included in this series, but since the v4 was sent, the
case of the Luarocks and Meson based packages have been fixed, and
I have sent separate series for these, because they can be merged
separately. Many thanks to Fran?ois Perrad and Peter Seiderer for
providing very useful insights on Luarocks and Meson.
Changes since v4
================
- Dropped patches that have been merged upstream, mainly the "extract
dependency" patches, but also a few other preparation patches.
- Dropped the change of the RPATH handling. As discussed during
previous Buildroot Developers meeting, it is not a big problem if
during the build, the RPATH of a binary points to the library
folder of another package per-package folder. Therefore, instead of
fixing up RPATH at the end of each package installation, we keep
what Buildroot does today: fix them at the very end of the build.
- Made the support for per-package folders optional. Indeed, we
realized there was no way to make sure it would be perfect from day
1. Even if the principle works, there are lots of package-specific
details to solve, and solving all of them before merging the base
per-package folder support is not reasonable. So for now, an option
BR2_PER_PACKAGE_FOLDERS is introduced, which defaults to off. When
this option is enabled, the .NOTPARALLEL statement of the main
Makefile goes away, which allows to do top-level parallel build.
- Addition of a commit that reworks how the top-level directories are
created.
- Introduction of a prepare-per-package-folder function in
pkg-generic.mk to encapsulate the logic to create the per-package
folders from the dependencies of a package.
- Rebased on top of the latest master.
Changes since v3
================
- Dropped patches that have been merged upstream:
pkgconf: use relative path to STAGING_DIR instead of absolute path
toolchain: post-pone evaluation of TOOLCHAIN_EXTERNAL_BIN
- For now, removed the most controversial/complicated patches
(changing the rpath strategy, and switching to per-package
host/target directories). The goal for now is to merge only the
preparation patches.
Changes since v2
================
- Solved the problem of all DEPENDENCIES_HOST_PREREQ targets:
host-ccache, host-tar, host-lzip, host-xz, host-fakedate.
To achieve this, introduced <pkg>_EXTRACT_DEPENDENCIES and used
that to handle the dependencies on host-tar, host-lzip and host-xz.
Used regular dependencies for host-ccache and host-fakedate.
See below for more details.
Changes since v1
================
- Rebased on the latest Buildroot next branch
- The pkg-config changes are reworked according to Arnout's comments:
only @STAGING_SUBDIR@ is replaced in pkg-config.in to generate
pkg-config, the rest of the logic is internal to the script.
- New patch to move the "host skeleton" logic into a proper
host-skeleton package.
- New patch to have the CMake related files generated as part of the
toolchain package.
- New patch to add a new "install" step that depends on the different
install steps (target, staging, images, host). This is needed to
later add some logic that is executed once all install steps of a
package have been done.
- Fix the approach to solve the RPATH logic: instead of using
LD_LIBRARY_PATH, we actually fix with patchelf the RPATH of host
binaries right after their installation.
- Misc other improvements in the per-package SDK implementation.
What is this all about ?
========================
See the cover letter of v3 for details:
http://lists.busybox.net/pipermail/buildroot/2017-December/208384.html
What remains to be done?
========================
Known issues:
- The qt5 stack is entirely broken, because qmake hardcodes the
installation path to be $(STAGING_DIR), so all qt5 packages install
to the per-package $(STAGING_DIR) of qt5base.
See
http://lists.busybox.net/pipermail/buildroot/2018-November/235942.html
- The Python external modules that need native libraries currently do
not build. This is due to the Python module building logic reading
a Makefile that hardcodes the path to the compiler. Investigation
is underway.
- The host-cargo package doesn't build, it fails to find the
http_parser library at link time.
Best regards,
Thomas
Thomas Petazzoni (10):
Makefile: evaluate CCACHE and HOST{CC,CXX} at time of use
support/scripts/check-host-rpath: split condition on two statements
Makefile: rework main directory creation logic
Makefile: move .NOTPARALLEL statement after including .config file
Makefile: define TARGET_DIR_WARNING_FILE relative to TARGET_DIR
package/pkg-generic: adjust config scripts tweaks for per-package
directories
core: implement per-package SDK and target
Makefile: allow top-level parallel build with
BR2_PER_PACKAGE_DIRECTORIES=y
package/pkg-generic: make libtool .la files compatible with
per-package directories
package/pkg-kconfig: handle KCONFIG_DEPENDENCIES with per-package
directories
Config.in | 18 ++++++
Makefile | 95 ++++++++++++++++++++------------
fs/common.mk | 1 +
package/pkg-generic.mk | 26 +++++++--
package/pkg-kconfig.mk | 1 +
package/pkg-utils.mk | 21 +++++++
support/scripts/check-host-rpath | 8 ++-
7 files changed, 128 insertions(+), 42 deletions(-)
--
2.19.1
next reply other threads:[~2018-11-23 14:58 UTC|newest]
Thread overview: 54+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-11-23 14:58 Thomas Petazzoni [this message]
2018-11-23 14:58 ` [Buildroot] [PATCH next v6 01/10] Makefile: evaluate CCACHE and HOST{CC, CXX} at time of use Thomas Petazzoni
2018-11-26 18:14 ` Peter Korsgaard
2018-11-23 14:58 ` [Buildroot] [PATCH next v6 02/10] support/scripts/check-host-rpath: split condition on two statements Thomas Petazzoni
2018-11-26 18:14 ` Peter Korsgaard
2018-11-23 14:58 ` [Buildroot] [PATCH next v6 03/10] Makefile: rework main directory creation logic Thomas Petazzoni
2018-11-23 18:07 ` Yann E. MORIN
2018-11-26 18:14 ` Peter Korsgaard
2018-11-23 14:58 ` [Buildroot] [PATCH next v6 04/10] Makefile: move .NOTPARALLEL statement after including .config file Thomas Petazzoni
2018-11-26 18:15 ` Peter Korsgaard
2018-11-23 14:58 ` [Buildroot] [PATCH next v6 05/10] Makefile: define TARGET_DIR_WARNING_FILE relative to TARGET_DIR Thomas Petazzoni
2018-11-26 18:15 ` Peter Korsgaard
2018-11-23 14:58 ` [Buildroot] [PATCH next v6 06/10] package/pkg-generic: adjust config scripts tweaks for per-package directories Thomas Petazzoni
2018-11-23 18:09 ` Yann E. MORIN
2018-11-26 18:15 ` Peter Korsgaard
2018-11-23 14:58 ` [Buildroot] [PATCH next v6 07/10] core: implement per-package SDK and target Thomas Petazzoni
2018-11-23 21:50 ` Yann E. MORIN
2018-12-04 22:24 ` Arnout Vandecappelle
2018-12-05 8:04 ` Thomas Petazzoni
2018-12-05 9:18 ` Arnout Vandecappelle
2018-12-05 9:44 ` Thomas Petazzoni
2018-12-05 10:41 ` Arnout Vandecappelle
2018-12-05 16:08 ` Andreas Naumann
2018-12-05 16:31 ` Thomas Petazzoni
2018-12-05 16:52 ` Andreas Naumann
2018-12-06 10:21 ` Andreas Naumann
2018-12-06 10:28 ` Thomas Petazzoni
2018-12-06 10:42 ` Andreas Naumann
2018-12-06 10:58 ` Thomas Petazzoni
2018-12-06 13:31 ` Andreas Naumann
2018-12-26 17:34 ` Thomas Petazzoni
2018-12-26 18:33 ` Thomas De Schampheleire
2019-01-10 21:25 ` Arnout Vandecappelle
2019-01-11 20:09 ` Thomas De Schampheleire
2019-01-13 22:10 ` Arnout Vandecappelle
2019-01-14 16:01 ` Thomas De Schampheleire
2019-01-14 16:33 ` Thomas Petazzoni
2019-01-14 20:19 ` Thomas De Schampheleire
2019-01-14 20:43 ` Thomas De Schampheleire
2018-12-26 18:58 ` Yann E. MORIN
2018-12-27 16:17 ` Thomas Petazzoni
2019-01-10 21:14 ` Arnout Vandecappelle
2018-11-23 14:58 ` [Buildroot] [PATCH next v6 08/10] Makefile: allow top-level parallel build with BR2_PER_PACKAGE_DIRECTORIES=y Thomas Petazzoni
2018-11-23 18:11 ` Yann E. MORIN
2018-11-23 14:58 ` [Buildroot] [PATCH next v6 09/10] package/pkg-generic: make libtool .la files compatible with per-package directories Thomas Petazzoni
2018-11-25 17:57 ` Yann E. MORIN
2018-11-30 10:20 ` Thomas Petazzoni
2018-12-01 10:59 ` Yann E. MORIN
2018-11-23 14:58 ` [Buildroot] [PATCH next v6 10/10] package/pkg-kconfig: handle KCONFIG_DEPENDENCIES " Thomas Petazzoni
2018-11-25 17:57 ` Yann E. MORIN
2018-12-05 15:23 ` Andreas Naumann
2018-12-06 14:07 ` Andreas Naumann
2018-12-06 14:25 ` Thomas Petazzoni
2018-12-26 16:40 ` Thomas Petazzoni
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20181123145815.13008-1-thomas.petazzoni@bootlin.com \
--to=thomas.petazzoni@bootlin.com \
--cc=buildroot@busybox.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.