* [PATCH v3 0/1] kbuild: deb-pkg: Build parallely with current dpkg-buildpackage @ 2023-03-15 17:35 Bastian Germann 2023-03-15 17:35 ` [PATCH v3 1/1] " Bastian Germann 0 siblings, 1 reply; 5+ messages in thread From: Bastian Germann @ 2023-03-15 17:35 UTC (permalink / raw) To: Masahiro Yamada Cc: Bastian Germann, Nathan Chancellor, Nick Desaulniers, linux-kbuild, linux-kernel Make use of DEB_BUILD_OPTIONS' parallel option because that is the only parallel build option that is set by default in current dpkg-buildpackage versions. v2: * Clarify that this is for current dpkg-buildpackage versions * Evaluate DEB_BUILD_OPTIONS in debian/rules. v3: * Use the Policy snippet which sets MAKEFLAGS and uses patsubst over subst * Mention the dpkg commit that changed the -j behaviour. Bastian Germann (1): kbuild: deb-pkg: Build parallely with current dpkg-buildpackage scripts/package/mkdebian | 5 +++++ 1 file changed, 5 insertions(+) -- 2.39.2 ^ permalink raw reply [flat|nested] 5+ messages in thread
* [PATCH v3 1/1] kbuild: deb-pkg: Build parallely with current dpkg-buildpackage 2023-03-15 17:35 [PATCH v3 0/1] kbuild: deb-pkg: Build parallely with current dpkg-buildpackage Bastian Germann @ 2023-03-15 17:35 ` Bastian Germann 2023-03-21 10:18 ` Masahiro Yamada 0 siblings, 1 reply; 5+ messages in thread From: Bastian Germann @ 2023-03-15 17:35 UTC (permalink / raw) To: Masahiro Yamada Cc: Bastian Germann, Nathan Chancellor, Nick Desaulniers, linux-kbuild, linux-kernel With dpkg-buildpackage v1.21.10 or later, the only way to build the deb-pkg generated package parallely is adding -j<N> to the MAKEFLAGS environment variable or using the --jobs-force option, see dpkg commit 1d0ea9b2ba3f ("dpkg-buildpackage: Change -j, --jobs semantics to non-force mode"). The package ignores the usual parallel build option that is described in Debian Policy. Derive make's -j parameter from the DEB_BUILD_OPTIONS environment variable that ends up being set by dpkg-buildpackage -j<N>. The snippet is copied from Debian Policy. Link: https://www.debian.org/doc/debian-policy/ch-source.html#debian-rules-and-deb-build-options Signed-off-by: Bastian Germann <bage@linutronix.de> --- scripts/package/mkdebian | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/scripts/package/mkdebian b/scripts/package/mkdebian index f74380036bb5..ed5c2b65798b 100755 --- a/scripts/package/mkdebian +++ b/scripts/package/mkdebian @@ -238,6 +238,11 @@ fi cat <<EOF > debian/rules #!$(command -v $MAKE) -f +ifneq (,\$(filter parallel=%,\$(DEB_BUILD_OPTIONS))) + NUMJOBS = \$(patsubst parallel=%,%,\$(filter parallel=%,\$(DEB_BUILD_OPTIONS))) + MAKEFLAGS += -j\$(NUMJOBS) +endif + srctree ?= . build-indep: -- 2.39.2 ^ permalink raw reply related [flat|nested] 5+ messages in thread
* Re: [PATCH v3 1/1] kbuild: deb-pkg: Build parallely with current dpkg-buildpackage 2023-03-15 17:35 ` [PATCH v3 1/1] " Bastian Germann @ 2023-03-21 10:18 ` Masahiro Yamada 2023-03-21 10:28 ` Bastian Germann 0 siblings, 1 reply; 5+ messages in thread From: Masahiro Yamada @ 2023-03-21 10:18 UTC (permalink / raw) To: Bastian Germann Cc: Nathan Chancellor, Nick Desaulniers, linux-kbuild, linux-kernel On Thu, Mar 16, 2023 at 2:35 AM Bastian Germann <bage@linutronix.de> wrote: > > With dpkg-buildpackage v1.21.10 or later, the only way to build the > deb-pkg generated package parallely is adding -j<N> to the MAKEFLAGS > environment variable or using the --jobs-force option, see dpkg commit > 1d0ea9b2ba3f ("dpkg-buildpackage: Change -j, --jobs semantics to > non-force mode"). The package ignores the usual parallel build option > that is described in Debian Policy. > > Derive make's -j parameter from the DEB_BUILD_OPTIONS environment variable > that ends up being set by dpkg-buildpackage -j<N>. The snippet is copied > from Debian Policy. > > Link: https://www.debian.org/doc/debian-policy/ch-source.html#debian-rules-and-deb-build-options > Signed-off-by: Bastian Germann <bage@linutronix.de> > --- I was about to pick this up, then I stopped it. If I apply this patch, "make -j2 bindeb-pkg" shows a new warning. make[3]: warning: -j16 forced in makefile: resetting jobserver mode. Please note `nproc` returns 16 on my machine. "make bindeb-pkg" also runs in 16 threads because $(DEB_BUILD_OPTIONS) always contains "parallel=16" even if -j<N> option is given at all. In other words, this patch will take away user's freedom to choose the number of threads. > scripts/package/mkdebian | 5 +++++ > 1 file changed, 5 insertions(+) > > diff --git a/scripts/package/mkdebian b/scripts/package/mkdebian > index f74380036bb5..ed5c2b65798b 100755 > --- a/scripts/package/mkdebian > +++ b/scripts/package/mkdebian > @@ -238,6 +238,11 @@ fi > cat <<EOF > debian/rules > #!$(command -v $MAKE) -f > > +ifneq (,\$(filter parallel=%,\$(DEB_BUILD_OPTIONS))) > + NUMJOBS = \$(patsubst parallel=%,%,\$(filter parallel=%,\$(DEB_BUILD_OPTIONS))) > + MAKEFLAGS += -j\$(NUMJOBS) > +endif > + > srctree ?= . > > build-indep: > -- > 2.39.2 > -- Best Regards Masahiro Yamada ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH v3 1/1] kbuild: deb-pkg: Build parallely with current dpkg-buildpackage 2023-03-21 10:18 ` Masahiro Yamada @ 2023-03-21 10:28 ` Bastian Germann 2023-04-10 12:26 ` Masahiro Yamada 0 siblings, 1 reply; 5+ messages in thread From: Bastian Germann @ 2023-03-21 10:28 UTC (permalink / raw) To: Masahiro Yamada Cc: Nathan Chancellor, Nick Desaulniers, linux-kbuild, linux-kernel Am 21.03.23 um 11:18 schrieb Masahiro Yamada: > "make bindeb-pkg" also runs in 16 threads because > $(DEB_BUILD_OPTIONS) always contains "parallel=16" > even if -j<N> option is given at all. > > In other words, this patch will take away user's freedom > to choose the number of threads. Okay. To solve this, I would forward make's -j option to the dpkg-buildpackage calls, which is now called with the defaults. I think you should see the warning with older dpkg-buildpackage versions without the patch applied as well because their default is dpkg-buildpackage -j`nproc` and is in effect currently. ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH v3 1/1] kbuild: deb-pkg: Build parallely with current dpkg-buildpackage 2023-03-21 10:28 ` Bastian Germann @ 2023-04-10 12:26 ` Masahiro Yamada 0 siblings, 0 replies; 5+ messages in thread From: Masahiro Yamada @ 2023-04-10 12:26 UTC (permalink / raw) To: Bastian Germann Cc: Nathan Chancellor, Nick Desaulniers, linux-kbuild, linux-kernel On Tue, Mar 21, 2023 at 7:28 PM Bastian Germann <bage@linutronix.de> wrote: > > Am 21.03.23 um 11:18 schrieb Masahiro Yamada: > > "make bindeb-pkg" also runs in 16 threads because > > $(DEB_BUILD_OPTIONS) always contains "parallel=16" > > even if -j<N> option is given at all. > > > > In other words, this patch will take away user's freedom > > to choose the number of threads. > > Okay. To solve this, I would forward make's -j option to the dpkg-buildpackage calls, > which is now called with the defaults. Yes. Agreed. > I think you should see the warning with older dpkg-buildpackage versions without the > patch applied as well because their default is dpkg-buildpackage -j`nproc` and is in > effect currently. -- Best Regards Masahiro Yamada ^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2023-04-10 12:27 UTC | newest] Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2023-03-15 17:35 [PATCH v3 0/1] kbuild: deb-pkg: Build parallely with current dpkg-buildpackage Bastian Germann 2023-03-15 17:35 ` [PATCH v3 1/1] " Bastian Germann 2023-03-21 10:18 ` Masahiro Yamada 2023-03-21 10:28 ` Bastian Germann 2023-04-10 12:26 ` Masahiro Yamada
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.