Backports Archive on lore.kernel.org
 help / color / Atom feed
* [PATCH] backports: speed up building with bash
@ 2019-12-03 15:06 Johannes Berg
  2019-12-03 15:15 ` Johannes Berg
  2019-12-03 15:18 ` Johannes Berg
  0 siblings, 2 replies; 4+ messages in thread
From: Johannes Berg @ 2019-12-03 15:06 UTC (permalink / raw)
  To: backports; +Cc: Johannes Berg

From: Johannes Berg <johannes.berg@intel.com>

When building with /bin/sh -> bash, things are SUPER slow (at least
for me), because bash takes a LOOONG time to look at the environment
variables (and we typically have around 6k), adding ~300ms to each
bash invocation for me.

The reason we export them is that we need them in all of the sub-
makes, and those only read auto.conf, which we can't change.

Work around this by overriding 'make' itself, and using --eval to
read *our* .config file into each make that gets called. This way,
the variables are present in all make invocations in the same way
as they would be through the environment, but don't get passed to
shell invocations.

Signed-off-by: Johannes Berg <johannes.berg@intel.com>
---
 backport/Makefile.build | 3 ++-
 backport/scripts/make   | 3 +++
 2 files changed, 5 insertions(+), 1 deletion(-)
 create mode 100755 backport/scripts/make

diff --git a/backport/Makefile.build b/backport/Makefile.build
index a848b37ed64b..7c479ace9f4a 100644
--- a/backport/Makefile.build
+++ b/backport/Makefile.build
@@ -1,6 +1,7 @@
--include .config
 export
 
+export MAKE=$(BACKPORT_DIR)/scripts/make
+
 .PHONY: modules
 modules:
 	@$(MAKE) -C $(KLIB_BUILD) M=$(BACKPORT_DIR) modules
diff --git a/backport/scripts/make b/backport/scripts/make
new file mode 100755
index 000000000000..f5330a3730c8
--- /dev/null
+++ b/backport/scripts/make
@@ -0,0 +1,3 @@
+#!/bin/sh
+
+exec make --eval '-include $(BACKPORT_DIR)/.config' "$@"
-- 
2.23.0

--
To unsubscribe from this list: send the line "unsubscribe backports" in

^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: [PATCH] backports: speed up building with bash
  2019-12-03 15:06 [PATCH] backports: speed up building with bash Johannes Berg
@ 2019-12-03 15:15 ` Johannes Berg
  2019-12-03 15:18 ` Johannes Berg
  1 sibling, 0 replies; 4+ messages in thread
From: Johannes Berg @ 2019-12-03 15:15 UTC (permalink / raw)
  To: backports

On Tue, 2019-12-03 at 16:06 +0100, Johannes Berg wrote:
> From: Johannes Berg <johannes.berg@intel.com>
> 
> When building with /bin/sh -> bash, things are SUPER slow (at least
> for me), because bash takes a LOOONG time to look at the environment
> variables (and we typically have around 6k), adding ~300ms to each
> bash invocation for me.

I think this was under-estimating it a bit - it's more like 1.3 seconds?

Anyway ...

johannes

--
To unsubscribe from this list: send the line "unsubscribe backports" in

^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: [PATCH] backports: speed up building with bash
  2019-12-03 15:06 [PATCH] backports: speed up building with bash Johannes Berg
  2019-12-03 15:15 ` Johannes Berg
@ 2019-12-03 15:18 ` Johannes Berg
  2019-12-03 15:21   ` Johannes Berg
  1 sibling, 1 reply; 4+ messages in thread
From: Johannes Berg @ 2019-12-03 15:18 UTC (permalink / raw)
  To: backports

On Tue, 2019-12-03 at 16:06 +0100, Johannes Berg wrote:
> From: Johannes Berg <johannes.berg@intel.com>
> 
> When building with /bin/sh -> bash, things are SUPER slow (at least
> for me), because bash takes a LOOONG time to look at the environment
> variables (and we typically have around 6k), adding ~300ms to each
> bash invocation for me.

The real results are like this:

Completely clean build, after git clean -fdxq, but I have ccache primed
so all the stuff is build overhead - gcc is ccache and just returns
directly from the cache.

without the patch:
real	9m17.180s
user	24m38.063s
sys	0m32.138s


with the patch:
real	0m18.157s
user	0m24.553s
sys	0m13.342s


I'll let those results speak for themselves ...

johannes

--
To unsubscribe from this list: send the line "unsubscribe backports" in

^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: [PATCH] backports: speed up building with bash
  2019-12-03 15:18 ` Johannes Berg
@ 2019-12-03 15:21   ` Johannes Berg
  0 siblings, 0 replies; 4+ messages in thread
From: Johannes Berg @ 2019-12-03 15:21 UTC (permalink / raw)
  To: backports

On Tue, 2019-12-03 at 16:18 +0100, Johannes Berg wrote:
> On Tue, 2019-12-03 at 16:06 +0100, Johannes Berg wrote:
> > From: Johannes Berg <johannes.berg@intel.com>
> > 
> > When building with /bin/sh -> bash, things are SUPER slow (at least
> > for me), because bash takes a LOOONG time to look at the environment
> > variables (and we typically have around 6k), adding ~300ms to each
> > bash invocation for me.
> 
> The real results are like this:
> 
> Completely clean build, after git clean -fdxq, but I have ccache primed
> so all the stuff is build overhead - gcc is ccache and just returns
> directly from the cache.
> 
> without the patch:
> real	9m17.180s
> user	24m38.063s
> sys	0m32.138s
> 
> 
> with the patch:
> real	0m18.157s
> user	0m24.553s
> sys	0m13.342s

That was with bash, with dash:

without the patch:
real	0m33.018s
user	0m56.248s
sys	0m22.898s

with the patch:
real	0m18.113s
user	0m18.036s
sys	0m12.903s

johannes

--
To unsubscribe from this list: send the line "unsubscribe backports" in

^ permalink raw reply	[flat|nested] 4+ messages in thread

end of thread, back to index

Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-12-03 15:06 [PATCH] backports: speed up building with bash Johannes Berg
2019-12-03 15:15 ` Johannes Berg
2019-12-03 15:18 ` Johannes Berg
2019-12-03 15:21   ` Johannes Berg

Backports Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/backports/0 backports/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 backports backports/ https://lore.kernel.org/backports \
		backports@vger.kernel.org
	public-inbox-index backports

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.backports


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git