[Buildroot] [PATCH 1/1] Config.in: add option to only use ccache for BR

[Buildroot] [PATCH 1/1] Config.in: add option to only use ccache for BR

[Paul Cercueil]

Add an option, disabled by default, to only use ccache for Buildroot
packages.

When enabled, ccache will be used when building Buildroot itself, but
the toolchain exported as SDK will not make use of ccache to build
external programs, unless the "BR_USE_CCACHE" environment variable is
set.

The motivation behind this change is to be able to speed up iterative
building of a Buildroot based distribution, without forcing the users of
the SDK to use ccache, which has the problem of clobbering the
filesystem in a hidden directory if the cache directory is left as
default, or trying to access a cache folder that existed on the machine
that built the SDK but which may not exist on the machine on which it
was later installed.

Signed-off-by: Paul Cercueil <paul@crapouillou.net>
---
 Config.in                      | 9 +++++++++
 linux/linux.mk                 | 4 ++++
 package/Makefile.in            | 4 ++++
 toolchain/toolchain-wrapper.c  | 5 +++++
 toolchain/toolchain-wrapper.mk | 4 ++++
 5 files changed, 26 insertions(+)

diff --git a/Config.in b/Config.in
index f0cd6f48ed..fec2332077 100644
--- a/Config.in
+++ b/Config.in
@@ -374,6 +374,15 @@ config BR2_CCACHE_USE_BASEDIR
 	  the section "COMPILING IN DIFFERENT DIRECTORIES" in the ccache
 	  manual for more information.
 
+config BR2_CCACHE_BR_ONLY
+	bool "Only use ccache for building Buildroot packages"
+	default n
+	help
+	  When enabled, ccache will be used when building Buildroot
+	  itself, but the toolchain exported as SDK will not make use of
+	  ccache to build external programs, unless the "BR_USE_CCACHE"
+	  environment variable is set.
+
 endif
 
 config BR2_ENABLE_DEBUG
diff --git a/linux/linux.mk b/linux/linux.mk
index 61fdc0c76c..339848e29b 100644
--- a/linux/linux.mk
+++ b/linux/linux.mk
@@ -69,6 +69,10 @@ LINUX_MAKE_ENV = \
 	$(HOST_MAKE_ENV) \
 	BR_BINARIES_DIR=$(BINARIES_DIR)
 
+ifeq ($(BR2_CCACHE_BR_ONLY),y)
+LINUX_MAKE_ENV += BR_USE_CCACHE=1
+endif
+
 LINUX_INSTALL_IMAGES = YES
 LINUX_DEPENDENCIES = host-kmod \
 	$(if $(BR2_PACKAGE_INTEL_MICROCODE),intel-microcode) \
diff --git a/package/Makefile.in b/package/Makefile.in
index 508ea7c366..34556fa7b0 100644
--- a/package/Makefile.in
+++ b/package/Makefile.in
@@ -263,6 +263,10 @@ export PERL5LIB=$(HOST_DIR)/lib/perl
 
 TARGET_MAKE_ENV = PATH=$(BR_PATH)
 
+ifeq ($(BR2_CCACHE_BR_ONLY),y)
+	TARGET_MAKE_ENV += BR_USE_CCACHE=1
+endif
+
 TARGET_CONFIGURE_OPTS = \
 	$(TARGET_MAKE_ENV) \
 	AR="$(TARGET_AR)" \
diff --git a/toolchain/toolchain-wrapper.c b/toolchain/toolchain-wrapper.c
index 0fb6064b1c..bf0194c8fc 100644
--- a/toolchain/toolchain-wrapper.c
+++ b/toolchain/toolchain-wrapper.c
@@ -510,6 +510,11 @@ int main(int argc, char **argv)
 	if (getenv("BR_NO_CCACHE"))
 		/* Skip the ccache call */
 		exec_args++;
+#ifdef BR_CCACHE_BR_ONLY
+	else if (!getenv("BR_USE_CCACHE"))
+		/* Skip the ccache call */
+		exec_args++;
+#endif
 #endif
 
 	/* Debug the wrapper to see final arguments passed to the real compiler. */
diff --git a/toolchain/toolchain-wrapper.mk b/toolchain/toolchain-wrapper.mk
index cbf46f15fa..cda9eaf342 100644
--- a/toolchain/toolchain-wrapper.mk
+++ b/toolchain/toolchain-wrapper.mk
@@ -73,6 +73,10 @@ ifeq ($(BR2_CCACHE_USE_BASEDIR),y)
 TOOLCHAIN_WRAPPER_ARGS += -DBR_CCACHE_BASEDIR='"$(BASE_DIR)"'
 endif
 
+ifeq ($(BR2_CCACHE_BR_ONLY),y)
+TOOLCHAIN_WRAPPER_ARGS += -DBR_CCACHE_BR_ONLY
+endif
+
 ifeq ($(BR2_PIC_PIE),y)
 TOOLCHAIN_WRAPPER_ARGS += -DBR2_PIC_PIE
 endif
-- 
2.34.1

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH 1/1] Config.in: add option to only use ccache for BR

[Paul Cercueil]

Le jeu., janv. 13 2022 at 21:04:31 +0000, Paul Cercueil 
<paul@crapouillou.net> a écrit :
> Add an option, disabled by default, to only use ccache for Buildroot
> packages.
> 
> When enabled, ccache will be used when building Buildroot itself, but
> the toolchain exported as SDK will not make use of ccache to build
> external programs, unless the "BR_USE_CCACHE" environment variable is
> set.
> 
> The motivation behind this change is to be able to speed up iterative
> building of a Buildroot based distribution, without forcing the users 
> of
> the SDK to use ccache, which has the problem of clobbering the

s/clobbering/cluttering/

-Paul

> filesystem in a hidden directory if the cache directory is left as
> default, or trying to access a cache folder that existed on the 
> machine
> that built the SDK but which may not exist on the machine on which it
> was later installed.
> 
> Signed-off-by: Paul Cercueil <paul@crapouillou.net>
> ---
>  Config.in                      | 9 +++++++++
>  linux/linux.mk                 | 4 ++++
>  package/Makefile.in            | 4 ++++
>  toolchain/toolchain-wrapper.c  | 5 +++++
>  toolchain/toolchain-wrapper.mk | 4 ++++
>  5 files changed, 26 insertions(+)
> 
> diff --git a/Config.in b/Config.in
> index f0cd6f48ed..fec2332077 100644
> --- a/Config.in
> +++ b/Config.in
> @@ -374,6 +374,15 @@ config BR2_CCACHE_USE_BASEDIR
>  	  the section "COMPILING IN DIFFERENT DIRECTORIES" in the ccache
>  	  manual for more information.
> 
> +config BR2_CCACHE_BR_ONLY
> +	bool "Only use ccache for building Buildroot packages"
> +	default n
> +	help
> +	  When enabled, ccache will be used when building Buildroot
> +	  itself, but the toolchain exported as SDK will not make use of
> +	  ccache to build external programs, unless the "BR_USE_CCACHE"
> +	  environment variable is set.
> +
>  endif
> 
>  config BR2_ENABLE_DEBUG
> diff --git a/linux/linux.mk b/linux/linux.mk
> index 61fdc0c76c..339848e29b 100644
> --- a/linux/linux.mk
> +++ b/linux/linux.mk
> @@ -69,6 +69,10 @@ LINUX_MAKE_ENV = \
>  	$(HOST_MAKE_ENV) \
>  	BR_BINARIES_DIR=$(BINARIES_DIR)
> 
> +ifeq ($(BR2_CCACHE_BR_ONLY),y)
> +LINUX_MAKE_ENV += BR_USE_CCACHE=1
> +endif
> +
>  LINUX_INSTALL_IMAGES = YES
>  LINUX_DEPENDENCIES = host-kmod \
>  	$(if $(BR2_PACKAGE_INTEL_MICROCODE),intel-microcode) \
> diff --git a/package/Makefile.in b/package/Makefile.in
> index 508ea7c366..34556fa7b0 100644
> --- a/package/Makefile.in
> +++ b/package/Makefile.in
> @@ -263,6 +263,10 @@ export PERL5LIB=$(HOST_DIR)/lib/perl
> 
>  TARGET_MAKE_ENV = PATH=$(BR_PATH)
> 
> +ifeq ($(BR2_CCACHE_BR_ONLY),y)
> +	TARGET_MAKE_ENV += BR_USE_CCACHE=1
> +endif
> +
>  TARGET_CONFIGURE_OPTS = \
>  	$(TARGET_MAKE_ENV) \
>  	AR="$(TARGET_AR)" \
> diff --git a/toolchain/toolchain-wrapper.c 
> b/toolchain/toolchain-wrapper.c
> index 0fb6064b1c..bf0194c8fc 100644
> --- a/toolchain/toolchain-wrapper.c
> +++ b/toolchain/toolchain-wrapper.c
> @@ -510,6 +510,11 @@ int main(int argc, char **argv)
>  	if (getenv("BR_NO_CCACHE"))
>  		/* Skip the ccache call */
>  		exec_args++;
> +#ifdef BR_CCACHE_BR_ONLY
> +	else if (!getenv("BR_USE_CCACHE"))
> +		/* Skip the ccache call */
> +		exec_args++;
> +#endif
>  #endif
> 
>  	/* Debug the wrapper to see final arguments passed to the real 
> compiler. */
> diff --git a/toolchain/toolchain-wrapper.mk 
> b/toolchain/toolchain-wrapper.mk
> index cbf46f15fa..cda9eaf342 100644
> --- a/toolchain/toolchain-wrapper.mk
> +++ b/toolchain/toolchain-wrapper.mk
> @@ -73,6 +73,10 @@ ifeq ($(BR2_CCACHE_USE_BASEDIR),y)
>  TOOLCHAIN_WRAPPER_ARGS += -DBR_CCACHE_BASEDIR='"$(BASE_DIR)"'
>  endif
> 
> +ifeq ($(BR2_CCACHE_BR_ONLY),y)
> +TOOLCHAIN_WRAPPER_ARGS += -DBR_CCACHE_BR_ONLY
> +endif
> +
>  ifeq ($(BR2_PIC_PIE),y)
>  TOOLCHAIN_WRAPPER_ARGS += -DBR2_PIC_PIE
>  endif
> --
> 2.34.1
> 


_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH 1/1] Config.in: add option to only use ccache for BR

[Thomas Petazzoni via buildroot]

Hello Paul,

On Thu, 13 Jan 2022 21:04:31 +0000
Paul Cercueil <paul@crapouillou.net> wrote:

> Add an option, disabled by default, to only use ccache for Buildroot
> packages.
> 
> When enabled, ccache will be used when building Buildroot itself, but
> the toolchain exported as SDK will not make use of ccache to build
> external programs, unless the "BR_USE_CCACHE" environment variable is
> set.
> 
> The motivation behind this change is to be able to speed up iterative
> building of a Buildroot based distribution, without forcing the users of
> the SDK to use ccache, which has the problem of clobbering the
> filesystem in a hidden directory if the cache directory is left as
> default, or trying to access a cache folder that existed on the machine
> that built the SDK but which may not exist on the machine on which it
> was later installed.
> 
> Signed-off-by: Paul Cercueil <paul@crapouillou.net>
> ---
>  Config.in                      | 9 +++++++++
>  linux/linux.mk                 | 4 ++++
>  package/Makefile.in            | 4 ++++
>  toolchain/toolchain-wrapper.c  | 5 +++++
>  toolchain/toolchain-wrapper.mk | 4 ++++
>  5 files changed, 26 insertions(+)

We have applied your patch, but with some significant change.

We have dropped the Config.in option, as we believe the SDK should not
use ccache in general. We have introduce a BR2_USE_CCACHE variable (not
a Config.in option!) which when set to 1 tells the wrapper to use
ccache. Buildroot defines/exports this variable globally in its
Makefile, so that during the Buildroot build ccache is used (if
enabled, of course). ccache is not used in the SDK situation, unless
explicitly overridden with BR2_USE_CCACHE=1.

Thanks to that, we could also remove the BR_NO_CCACHE variable, so that
we don't have two confusing variables doing almost the same thing.

See
https://git.buildroot.org/buildroot/commit/?id=d5c0eaef1f7c3b705cc1cf3087bd83ad9098aa2f
for the final implementation.

Thanks a lot!

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot