* [PATCH v3 0/1] kbuild: deb-pkg: Build parallely with current dpkg-buildpackage
@ 2023-03-15 17:35 Bastian Germann
2023-03-15 17:35 ` [PATCH v3 1/1] " Bastian Germann
0 siblings, 1 reply; 5+ messages in thread
From: Bastian Germann @ 2023-03-15 17:35 UTC (permalink / raw)
To: Masahiro Yamada
Cc: Bastian Germann, Nathan Chancellor, Nick Desaulniers,
linux-kbuild, linux-kernel
Make use of DEB_BUILD_OPTIONS' parallel option because that is the only
parallel build option that is set by default in current dpkg-buildpackage
versions.
v2:
* Clarify that this is for current dpkg-buildpackage versions
* Evaluate DEB_BUILD_OPTIONS in debian/rules.
v3:
* Use the Policy snippet which sets MAKEFLAGS and uses patsubst over subst
* Mention the dpkg commit that changed the -j behaviour.
Bastian Germann (1):
kbuild: deb-pkg: Build parallely with current dpkg-buildpackage
scripts/package/mkdebian | 5 +++++
1 file changed, 5 insertions(+)
--
2.39.2
^ permalink raw reply [flat|nested] 5+ messages in thread
* [PATCH v3 1/1] kbuild: deb-pkg: Build parallely with current dpkg-buildpackage
2023-03-15 17:35 [PATCH v3 0/1] kbuild: deb-pkg: Build parallely with current dpkg-buildpackage Bastian Germann
@ 2023-03-15 17:35 ` Bastian Germann
2023-03-21 10:18 ` Masahiro Yamada
0 siblings, 1 reply; 5+ messages in thread
From: Bastian Germann @ 2023-03-15 17:35 UTC (permalink / raw)
To: Masahiro Yamada
Cc: Bastian Germann, Nathan Chancellor, Nick Desaulniers,
linux-kbuild, linux-kernel
With dpkg-buildpackage v1.21.10 or later, the only way to build the
deb-pkg generated package parallely is adding -j<N> to the MAKEFLAGS
environment variable or using the --jobs-force option, see dpkg commit
1d0ea9b2ba3f ("dpkg-buildpackage: Change -j, --jobs semantics to
non-force mode"). The package ignores the usual parallel build option
that is described in Debian Policy.
Derive make's -j parameter from the DEB_BUILD_OPTIONS environment variable
that ends up being set by dpkg-buildpackage -j<N>. The snippet is copied
from Debian Policy.
Link: https://www.debian.org/doc/debian-policy/ch-source.html#debian-rules-and-deb-build-options
Signed-off-by: Bastian Germann <bage@linutronix.de>
---
scripts/package/mkdebian | 5 +++++
1 file changed, 5 insertions(+)
diff --git a/scripts/package/mkdebian b/scripts/package/mkdebian
index f74380036bb5..ed5c2b65798b 100755
--- a/scripts/package/mkdebian
+++ b/scripts/package/mkdebian
@@ -238,6 +238,11 @@ fi
cat <<EOF > debian/rules
#!$(command -v $MAKE) -f
+ifneq (,\$(filter parallel=%,\$(DEB_BUILD_OPTIONS)))
+ NUMJOBS = \$(patsubst parallel=%,%,\$(filter parallel=%,\$(DEB_BUILD_OPTIONS)))
+ MAKEFLAGS += -j\$(NUMJOBS)
+endif
+
srctree ?= .
build-indep:
--
2.39.2
^ permalink raw reply related [flat|nested] 5+ messages in thread
* Re: [PATCH v3 1/1] kbuild: deb-pkg: Build parallely with current dpkg-buildpackage
2023-03-15 17:35 ` [PATCH v3 1/1] " Bastian Germann
@ 2023-03-21 10:18 ` Masahiro Yamada
2023-03-21 10:28 ` Bastian Germann
0 siblings, 1 reply; 5+ messages in thread
From: Masahiro Yamada @ 2023-03-21 10:18 UTC (permalink / raw)
To: Bastian Germann
Cc: Nathan Chancellor, Nick Desaulniers, linux-kbuild, linux-kernel
On Thu, Mar 16, 2023 at 2:35 AM Bastian Germann <bage@linutronix.de> wrote:
>
> With dpkg-buildpackage v1.21.10 or later, the only way to build the
> deb-pkg generated package parallely is adding -j<N> to the MAKEFLAGS
> environment variable or using the --jobs-force option, see dpkg commit
> 1d0ea9b2ba3f ("dpkg-buildpackage: Change -j, --jobs semantics to
> non-force mode"). The package ignores the usual parallel build option
> that is described in Debian Policy.
>
> Derive make's -j parameter from the DEB_BUILD_OPTIONS environment variable
> that ends up being set by dpkg-buildpackage -j<N>. The snippet is copied
> from Debian Policy.
>
> Link: https://www.debian.org/doc/debian-policy/ch-source.html#debian-rules-and-deb-build-options
> Signed-off-by: Bastian Germann <bage@linutronix.de>
> ---
I was about to pick this up, then I stopped it.
If I apply this patch,
"make -j2 bindeb-pkg" shows a new warning.
make[3]: warning: -j16 forced in makefile: resetting jobserver mode.
Please note `nproc` returns 16 on my machine.
"make bindeb-pkg" also runs in 16 threads because
$(DEB_BUILD_OPTIONS) always contains "parallel=16"
even if -j<N> option is given at all.
In other words, this patch will take away user's freedom
to choose the number of threads.
> scripts/package/mkdebian | 5 +++++
> 1 file changed, 5 insertions(+)
>
> diff --git a/scripts/package/mkdebian b/scripts/package/mkdebian
> index f74380036bb5..ed5c2b65798b 100755
> --- a/scripts/package/mkdebian
> +++ b/scripts/package/mkdebian
> @@ -238,6 +238,11 @@ fi
> cat <<EOF > debian/rules
> #!$(command -v $MAKE) -f
>
> +ifneq (,\$(filter parallel=%,\$(DEB_BUILD_OPTIONS)))
> + NUMJOBS = \$(patsubst parallel=%,%,\$(filter parallel=%,\$(DEB_BUILD_OPTIONS)))
> + MAKEFLAGS += -j\$(NUMJOBS)
> +endif
> +
> srctree ?= .
>
> build-indep:
> --
> 2.39.2
>
--
Best Regards
Masahiro Yamada
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH v3 1/1] kbuild: deb-pkg: Build parallely with current dpkg-buildpackage
2023-03-21 10:18 ` Masahiro Yamada
@ 2023-03-21 10:28 ` Bastian Germann
2023-04-10 12:26 ` Masahiro Yamada
0 siblings, 1 reply; 5+ messages in thread
From: Bastian Germann @ 2023-03-21 10:28 UTC (permalink / raw)
To: Masahiro Yamada
Cc: Nathan Chancellor, Nick Desaulniers, linux-kbuild, linux-kernel
Am 21.03.23 um 11:18 schrieb Masahiro Yamada:
> "make bindeb-pkg" also runs in 16 threads because
> $(DEB_BUILD_OPTIONS) always contains "parallel=16"
> even if -j<N> option is given at all.
>
> In other words, this patch will take away user's freedom
> to choose the number of threads.
Okay. To solve this, I would forward make's -j option to the dpkg-buildpackage calls,
which is now called with the defaults.
I think you should see the warning with older dpkg-buildpackage versions without the
patch applied as well because their default is dpkg-buildpackage -j`nproc` and is in
effect currently.
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH v3 1/1] kbuild: deb-pkg: Build parallely with current dpkg-buildpackage
2023-03-21 10:28 ` Bastian Germann
@ 2023-04-10 12:26 ` Masahiro Yamada
0 siblings, 0 replies; 5+ messages in thread
From: Masahiro Yamada @ 2023-04-10 12:26 UTC (permalink / raw)
To: Bastian Germann
Cc: Nathan Chancellor, Nick Desaulniers, linux-kbuild, linux-kernel
On Tue, Mar 21, 2023 at 7:28 PM Bastian Germann <bage@linutronix.de> wrote:
>
> Am 21.03.23 um 11:18 schrieb Masahiro Yamada:
> > "make bindeb-pkg" also runs in 16 threads because
> > $(DEB_BUILD_OPTIONS) always contains "parallel=16"
> > even if -j<N> option is given at all.
> >
> > In other words, this patch will take away user's freedom
> > to choose the number of threads.
>
> Okay. To solve this, I would forward make's -j option to the dpkg-buildpackage calls,
> which is now called with the defaults.
Yes. Agreed.
> I think you should see the warning with older dpkg-buildpackage versions without the
> patch applied as well because their default is dpkg-buildpackage -j`nproc` and is in
> effect currently.
--
Best Regards
Masahiro Yamada
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2023-04-10 12:27 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-03-15 17:35 [PATCH v3 0/1] kbuild: deb-pkg: Build parallely with current dpkg-buildpackage Bastian Germann
2023-03-15 17:35 ` [PATCH v3 1/1] " Bastian Germann
2023-03-21 10:18 ` Masahiro Yamada
2023-03-21 10:28 ` Bastian Germann
2023-04-10 12:26 ` Masahiro Yamada
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.