[PATCH 0/6] builddeb: make deb building more flexible

[PATCH 0/6] builddeb: make deb building more flexible

[bage]

From: Bastian Germann <bage@linutronix.de>

Building Debian packages via the [bin]deb-pkg make targets has some
shortcomings. These targets do not allow for packing a Debian source
package independently from actually building the binary package.

This series improves that process by making it possible to rebuild
the packages without errors via dpkg-buildpackage, by separating
source package creation from binary package building, and by making
the dbg package build optional.

Bastian Germann (6):
  builddeb: diff-ignore unexported top-level files
  builddeb: set CC on cross build to prefixed gcc
  builddeb: clean generated package content
  builddeb: use standard format for copyright file
  builddeb: introduce profile excluding the dbg pkg
  kbuild: introduce srcdeb-pkg target

 scripts/Makefile.package |  8 ++++++--
 scripts/package/mkdebian | 40 ++++++++++++++++++++++++++--------------
 2 files changed, 32 insertions(+), 16 deletions(-)

-- 
2.30.2

[PATCH 1/6] builddeb: diff-ignore unexported top-level files

[bage]

From: Bastian Germann <bage@linutronix.de>

scripts/Makefile.package's TAR_CONTENT lists the files to include in orig
tarballs while the deb-pkg make target only ignores .git. This results in
the other top-level files ending up in the .diff.gz.

Let dpkg-source ignore .git with the default diff-ignore and list the other
top-level files in extend-diff-ignore. Use the debian/source/options to
always have those available on building the package.

Signed-off-by: Bastian Germann <bage@linutronix.de>
---
 scripts/Makefile.package | 2 +-
 scripts/package/mkdebian | 2 ++
 2 files changed, 3 insertions(+), 1 deletion(-)

diff --git a/scripts/Makefile.package b/scripts/Makefile.package
index f952fb64789d..280f3a2fa334 100644
--- a/scripts/Makefile.package
+++ b/scripts/Makefile.package
@@ -75,7 +75,7 @@ deb-pkg:
 	$(call cmd,src_tar,$(KDEB_SOURCENAME))
 	origversion=$$(dpkg-parsechangelog -SVersion |sed 's/-[^-]*$$//');\
 		mv $(KDEB_SOURCENAME).tar.gz ../$(KDEB_SOURCENAME)_$${origversion}.orig.tar.gz
-	+dpkg-buildpackage -r$(KBUILD_PKG_ROOTCMD) -a$$(cat debian/arch) $(DPKG_FLAGS) -i.git -us -uc
+	+dpkg-buildpackage -r$(KBUILD_PKG_ROOTCMD) -a$$(cat debian/arch) $(DPKG_FLAGS) -us -uc
 
 PHONY += bindeb-pkg
 bindeb-pkg:
diff --git a/scripts/package/mkdebian b/scripts/package/mkdebian
index 60a2a63a5e90..3d2d4b033e44 100755
--- a/scripts/package/mkdebian
+++ b/scripts/package/mkdebian
@@ -134,6 +134,8 @@ fi
 
 mkdir -p debian/source/
 echo "1.0" > debian/source/format
+echo diff-ignore > debian/source/options
+echo 'extend-diff-ignore = ".clang-format|.cocciconfig|.config.old|.*ignore|.mailmap|.version|CREDITS|MAINTAINERS|README"' >> debian/source/options
 
 echo $debarch > debian/arch
 extra_build_depends=", $(if_enabled_echo CONFIG_UNWINDER_ORC libelf-dev:native)"
-- 
2.30.2

Re: [PATCH 1/6] builddeb: diff-ignore unexported top-level files

[Masahiro Yamada]

On Sun, Apr 11, 2021 at 7:14 PM <bage@linutronix.de> wrote:
>
> From: Bastian Germann <bage@linutronix.de>
>
> scripts/Makefile.package's TAR_CONTENT lists the files to include in orig
> tarballs while the deb-pkg make target only ignores .git. This results in
> the other top-level files ending up in the .diff.gz.
>
> Let dpkg-source ignore .git with the default diff-ignore and list the other
> top-level files in extend-diff-ignore. Use the debian/source/options to
> always have those available on building the package.
>
> Signed-off-by: Bastian Germann <bage@linutronix.de>
> ---
>  scripts/Makefile.package | 2 +-
>  scripts/package/mkdebian | 2 ++
>  2 files changed, 3 insertions(+), 1 deletion(-)
>
> diff --git a/scripts/Makefile.package b/scripts/Makefile.package
> index f952fb64789d..280f3a2fa334 100644
> --- a/scripts/Makefile.package
> +++ b/scripts/Makefile.package
> @@ -75,7 +75,7 @@ deb-pkg:
>         $(call cmd,src_tar,$(KDEB_SOURCENAME))
>         origversion=$$(dpkg-parsechangelog -SVersion |sed 's/-[^-]*$$//');\
>                 mv $(KDEB_SOURCENAME).tar.gz ../$(KDEB_SOURCENAME)_$${origversion}.orig.tar.gz
> -       +dpkg-buildpackage -r$(KBUILD_PKG_ROOTCMD) -a$$(cat debian/arch) $(DPKG_FLAGS) -i.git -us -uc
> +       +dpkg-buildpackage -r$(KBUILD_PKG_ROOTCMD) -a$$(cat debian/arch) $(DPKG_FLAGS) -us -uc
>
>  PHONY += bindeb-pkg
>  bindeb-pkg:
> diff --git a/scripts/package/mkdebian b/scripts/package/mkdebian
> index 60a2a63a5e90..3d2d4b033e44 100755
> --- a/scripts/package/mkdebian
> +++ b/scripts/package/mkdebian
> @@ -134,6 +134,8 @@ fi
>
>  mkdir -p debian/source/
>  echo "1.0" > debian/source/format
> +echo diff-ignore > debian/source/options
> +echo 'extend-diff-ignore = ".clang-format|.cocciconfig|.config.old|.*ignore|.mailmap|.version|CREDITS|MAINTAINERS|README"' >> debian/source/options

Please add
.clang-format, .cocciconfig, .mailmap, CREDITS, MAINTAINERS, README
to TAR_CONTENTS.

These are real source files.




>
>  echo $debarch > debian/arch
>  extra_build_depends=", $(if_enabled_echo CONFIG_UNWINDER_ORC libelf-dev:native)"
> --
> 2.30.2
>


-- 
Best Regards
Masahiro Yamada

Re: [PATCH 1/6] builddeb: diff-ignore unexported top-level files

[Masahiro Yamada]

On Sat, Apr 17, 2021 at 5:29 PM Masahiro Yamada <masahiroy@kernel.org> wrote:
>
> On Sun, Apr 11, 2021 at 7:14 PM <bage@linutronix.de> wrote:
> >
> > From: Bastian Germann <bage@linutronix.de>
> >
> > scripts/Makefile.package's TAR_CONTENT lists the files to include in orig
> > tarballs while the deb-pkg make target only ignores .git. This results in
> > the other top-level files ending up in the .diff.gz.
> >
> > Let dpkg-source ignore .git with the default diff-ignore and list the other
> > top-level files in extend-diff-ignore. Use the debian/source/options to
> > always have those available on building the package.
> >
> > Signed-off-by: Bastian Germann <bage@linutronix.de>
> > ---
> >  scripts/Makefile.package | 2 +-
> >  scripts/package/mkdebian | 2 ++
> >  2 files changed, 3 insertions(+), 1 deletion(-)
> >
> > diff --git a/scripts/Makefile.package b/scripts/Makefile.package
> > index f952fb64789d..280f3a2fa334 100644
> > --- a/scripts/Makefile.package
> > +++ b/scripts/Makefile.package
> > @@ -75,7 +75,7 @@ deb-pkg:
> >         $(call cmd,src_tar,$(KDEB_SOURCENAME))
> >         origversion=$$(dpkg-parsechangelog -SVersion |sed 's/-[^-]*$$//');\
> >                 mv $(KDEB_SOURCENAME).tar.gz ../$(KDEB_SOURCENAME)_$${origversion}.orig.tar.gz
> > -       +dpkg-buildpackage -r$(KBUILD_PKG_ROOTCMD) -a$$(cat debian/arch) $(DPKG_FLAGS) -i.git -us -uc
> > +       +dpkg-buildpackage -r$(KBUILD_PKG_ROOTCMD) -a$$(cat debian/arch) $(DPKG_FLAGS) -us -uc
> >
> >  PHONY += bindeb-pkg
> >  bindeb-pkg:
> > diff --git a/scripts/package/mkdebian b/scripts/package/mkdebian
> > index 60a2a63a5e90..3d2d4b033e44 100755
> > --- a/scripts/package/mkdebian
> > +++ b/scripts/package/mkdebian
> > @@ -134,6 +134,8 @@ fi
> >
> >  mkdir -p debian/source/
> >  echo "1.0" > debian/source/format
> > +echo diff-ignore > debian/source/options
> > +echo 'extend-diff-ignore = ".clang-format|.cocciconfig|.config.old|.*ignore|.mailmap|.version|CREDITS|MAINTAINERS|README"' >> debian/source/options
>
> Please add
> .clang-format, .cocciconfig, .mailmap, CREDITS, MAINTAINERS, README
> to TAR_CONTENTS.
>
> These are real source files.
>



.*ignore

is difficult to understand.

Please add  .gitignore to TAR_CONTENTS
and .git to diff-ignore.





-- 
Best Regards
Masahiro Yamada

[PATCH 2/6] builddeb: set CC on cross build to prefixed gcc

[bage]

From: Bastian Germann <bage@linutronix.de>

Building the generated package with CC unset will fail for cross
compilation. Detect that and set CC to a sane default prefixed by
dpkg-architecture's DEB_HOST_GNU_TYPE variable.

Signed-off-by: Bastian Germann <bage@linutronix.de>
---
 scripts/package/mkdebian | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/scripts/package/mkdebian b/scripts/package/mkdebian
index 3d2d4b033e44..2fa149796791 100755
--- a/scripts/package/mkdebian
+++ b/scripts/package/mkdebian
@@ -223,6 +223,14 @@ fi
 cat <<EOF > debian/rules
 #!$(command -v $MAKE) -f
 
+include /usr/share/dpkg/architecture.mk
+
+ifneq (\$(DEB_BUILD_GNU_TYPE),\$(DEB_HOST_GNU_TYPE))
+ifeq (\$(origin CC),default)
+CC := \$(DEB_HOST_GNU_TYPE)-gcc
+endif
+endif
+
 srctree ?= .
 
 build-indep:
-- 
2.30.2

[PATCH 3/6] builddeb: clean generated package content

[bage]

From: Bastian Germann <bage@linutronix.de>

For each binary Debian package, a directory with the package name is
created in the debian directory. Include the generated files in the
package's clean target.

Signed-off-by: Bastian Germann <bage@linutronix.de>
---
 scripts/package/mkdebian | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/package/mkdebian b/scripts/package/mkdebian
index 2fa149796791..717223ef6b03 100755
--- a/scripts/package/mkdebian
+++ b/scripts/package/mkdebian
@@ -246,7 +246,7 @@ binary-arch: build-arch
 	KBUILD_BUILD_VERSION=${revision} -f \$(srctree)/Makefile intdeb-pkg
 
 clean:
-	rm -rf debian/*tmp debian/files
+	rm -rf debian/*tmp debian/files debian/linux-*
 	\$(MAKE) clean
 
 binary: binary-arch
-- 
2.30.2

Re: [PATCH 3/6] builddeb: clean generated package content

[Masahiro Yamada]

On Sun, Apr 11, 2021 at 7:14 PM <bage@linutronix.de> wrote:
>
> From: Bastian Germann <bage@linutronix.de>
>
> For each binary Debian package, a directory with the package name is
> created in the debian directory. Include the generated files in the
> package's clean target.
>
> Signed-off-by: Bastian Germann <bage@linutronix.de>
> ---
>  scripts/package/mkdebian | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/scripts/package/mkdebian b/scripts/package/mkdebian
> index 2fa149796791..717223ef6b03 100755
> --- a/scripts/package/mkdebian
> +++ b/scripts/package/mkdebian
> @@ -246,7 +246,7 @@ binary-arch: build-arch
>         KBUILD_BUILD_VERSION=${revision} -f \$(srctree)/Makefile intdeb-pkg
>
>  clean:
> -       rm -rf debian/*tmp debian/files
> +       rm -rf debian/*tmp debian/files debian/linux-*
>         \$(MAKE) clean
>
>  binary: binary-arch
> --
> 2.30.2
>


Please also remove debian/*tmp.


Add this tag:

Fixes: 1694e94e4f46 ("builddeb: match temporary directory name to the
package name")


-- 
Best Regards
Masahiro Yamada

[PATCH 4/6] builddeb: use standard format for copyright file

[bage]

From: Bastian Germann <bage@linutronix.de>

Convert the minimal copyright file to the DEP-5 standard format that is
commonly used in Debian.

Link: https://trends.debian.net/#copyright-format-machine-readable-dep-5-vs-old-format
Signed-off-by: Bastian Germann <bage@linutronix.de>
---
 scripts/package/mkdebian | 27 ++++++++++++++-------------
 1 file changed, 14 insertions(+), 13 deletions(-)

diff --git a/scripts/package/mkdebian b/scripts/package/mkdebian
index 717223ef6b03..bb5c19735345 100755
--- a/scripts/package/mkdebian
+++ b/scripts/package/mkdebian
@@ -152,22 +152,23 @@ EOF
 
 # Generate copyright file
 cat <<EOF > debian/copyright
-This is a packacked upstream version of the Linux kernel.
-
-The sources may be found at most Linux archive sites, including:
-https://www.kernel.org/pub/linux/kernel
-
-Copyright: 1991 - 2018 Linus Torvalds and others.
-
-The git repository for mainline kernel development is at:
-git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
-
+Format: https://www.debian.org/doc/packaging-manuals/copyright-format/1.0/
+Upstream-Name: Linux
+Source: https://www.kernel.org/pub/linux/kernel
+Comment: This is a packaged upstream version of the Linux kernel.
+
+Files: *
+Copyright: 1991 - 2021 Linus Torvalds and others.
+License: GPL-2
+ The git repository for mainline kernel development is at:
+ git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
+ .
     This program is free software; you can redistribute it and/or modify
     it under the terms of the GNU General Public License as published by
     the Free Software Foundation; version 2 dated June, 1991.
-
-On Debian GNU/Linux systems, the complete text of the GNU General Public
-License version 2 can be found in \`/usr/share/common-licenses/GPL-2'.
+ .
+ On Debian GNU/Linux systems, the complete text of the GNU General Public
+ License version 2 can be found in \`/usr/share/common-licenses/GPL-2'.
 EOF
 
 # Generate a control file
-- 
2.30.2

Re: [PATCH 4/6] builddeb: use standard format for copyright file

[Masahiro Yamada]

On Sun, Apr 11, 2021 at 7:14 PM <bage@linutronix.de> wrote:
>
> From: Bastian Germann <bage@linutronix.de>
>
> Convert the minimal copyright file to the DEP-5 standard format that is
> commonly used in Debian.
>
> Link: https://trends.debian.net/#copyright-format-machine-readable-dep-5-vs-old-format
> Signed-off-by: Bastian Germann <bage@linutronix.de>

This is equivalent to this patch:
https://lore.kernel.org/linux-kbuild/20200920222556.10002-4-guillem@hadrons.org/


> ---
>  scripts/package/mkdebian | 27 ++++++++++++++-------------
>  1 file changed, 14 insertions(+), 13 deletions(-)
>
> diff --git a/scripts/package/mkdebian b/scripts/package/mkdebian
> index 717223ef6b03..bb5c19735345 100755
> --- a/scripts/package/mkdebian
> +++ b/scripts/package/mkdebian
> @@ -152,22 +152,23 @@ EOF
>
>  # Generate copyright file
>  cat <<EOF > debian/copyright
> -This is a packacked upstream version of the Linux kernel.
> -
> -The sources may be found at most Linux archive sites, including:
> -https://www.kernel.org/pub/linux/kernel
> -
> -Copyright: 1991 - 2018 Linus Torvalds and others.
> -
> -The git repository for mainline kernel development is at:
> -git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
> -
> +Format: https://www.debian.org/doc/packaging-manuals/copyright-format/1.0/
> +Upstream-Name: Linux
> +Source: https://www.kernel.org/pub/linux/kernel
> +Comment: This is a packaged upstream version of the Linux kernel.
> +
> +Files: *
> +Copyright: 1991 - 2021 Linus Torvalds and others.
> +License: GPL-2

My concern is this license info is wrong
since not all files are GPL-2.

Is it better to make apparently wrong license info
machine-interpretable?

That is a question.



-- 
Best Regards
Masahiro Yamada

[PATCH 5/6] builddeb: introduce profile excluding the dbg pkg

[bage]

From: Bastian Germann <bage@linutronix.de>

Enabling CONFIG_DEBUG_INFO implies building the binary linux-image-*-dbg.
As this increases package build time significantly, one might want to
exclude it from being built.

Add a pkg.$sourcename.nodbg build profile for that package so it can be
excluded via e.g.: `make DPKG_FLAGS="-P=pkg.linux-5.11.0.nodbg" deb-pkg`

Link: https://wiki.debian.org/BuildProfileSpec
Signed-off-by: Bastian Germann <bage@linutronix.de>
---
 scripts/package/mkdebian | 1 +
 1 file changed, 1 insertion(+)

diff --git a/scripts/package/mkdebian b/scripts/package/mkdebian
index bb5c19735345..369a989499ef 100755
--- a/scripts/package/mkdebian
+++ b/scripts/package/mkdebian
@@ -213,6 +213,7 @@ if is_enabled CONFIG_DEBUG_INFO; then
 cat <<EOF >> debian/control
 
 Package: linux-image-$version-dbg
+Build-Profiles: <!pkg.$sourcename.nodbg>
 Section: debug
 Architecture: $debarch
 Description: Linux kernel debugging symbols for $version
-- 
2.30.2

[PATCH 6/6] kbuild: introduce srcdeb-pkg target

[bage]

From: Bastian Germann <bage@linutronix.de>

A Debian source package can be generated only in combination with building
it afterwards. Introduce a target srcdeb-pkg that generates the source
package without building it (adding dpkg-buildpackage's -S flag).

Make the former deb-pkg depend on both srcdeb-pkg and bindeb-pkg to retain
its behavior.

Signed-off-by: Bastian Germann <bage@linutronix.de>
---
 scripts/Makefile.package | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/scripts/Makefile.package b/scripts/Makefile.package
index 280f3a2fa334..78a363623c3a 100644
--- a/scripts/Makefile.package
+++ b/scripts/Makefile.package
@@ -69,13 +69,16 @@ binrpm-pkg:
 		$(UTS_MACHINE) -bb $(objtree)/binkernel.spec
 
 PHONY += deb-pkg
-deb-pkg:
+deb-pkg: srcdeb-pkg bindeb-pkg
+
+PHONY += srcdeb-pkg
+srcdeb-pkg:
 	$(MAKE) clean
 	$(CONFIG_SHELL) $(srctree)/scripts/package/mkdebian
 	$(call cmd,src_tar,$(KDEB_SOURCENAME))
 	origversion=$$(dpkg-parsechangelog -SVersion |sed 's/-[^-]*$$//');\
 		mv $(KDEB_SOURCENAME).tar.gz ../$(KDEB_SOURCENAME)_$${origversion}.orig.tar.gz
-	+dpkg-buildpackage -r$(KBUILD_PKG_ROOTCMD) -a$$(cat debian/arch) $(DPKG_FLAGS) -us -uc
+	+dpkg-buildpackage -r$(KBUILD_PKG_ROOTCMD) -a$$(cat debian/arch) $(DPKG_FLAGS) -S -us -uc
 
 PHONY += bindeb-pkg
 bindeb-pkg:
@@ -145,6 +148,7 @@ help:
 	@echo '  rpm-pkg             - Build both source and binary RPM kernel packages'
 	@echo '  binrpm-pkg          - Build only the binary kernel RPM package'
 	@echo '  deb-pkg             - Build both source and binary deb kernel packages'
+	@echo '  srcdeb-pkg          - Build only the source kernel deb package'
 	@echo '  bindeb-pkg          - Build only the binary kernel deb package'
 	@echo '  snap-pkg            - Build only the binary kernel snap package'
 	@echo '                        (will connect to external hosts)'
-- 
2.30.2

Re: [PATCH 6/6] kbuild: introduce srcdeb-pkg target

[Masahiro Yamada]

On Sun, Apr 11, 2021 at 7:14 PM <bage@linutronix.de> wrote:
>
> From: Bastian Germann <bage@linutronix.de>
>
> A Debian source package can be generated only in combination with building
> it afterwards. Introduce a target srcdeb-pkg that generates the source
> package without building it (adding dpkg-buildpackage's -S flag).
>
> Make the former deb-pkg depend on both srcdeb-pkg and bindeb-pkg to retain
> its behavior.
>
> Signed-off-by: Bastian Germann <bage@linutronix.de>
> ---
>  scripts/Makefile.package | 8 ++++++--
>  1 file changed, 6 insertions(+), 2 deletions(-)
>
> diff --git a/scripts/Makefile.package b/scripts/Makefile.package
> index 280f3a2fa334..78a363623c3a 100644
> --- a/scripts/Makefile.package
> +++ b/scripts/Makefile.package
> @@ -69,13 +69,16 @@ binrpm-pkg:
>                 $(UTS_MACHINE) -bb $(objtree)/binkernel.spec
>
>  PHONY += deb-pkg
> -deb-pkg:
> +deb-pkg: srcdeb-pkg bindeb-pkg


Does this work reliably with the parallel build option?


While srcdeb-pkg is cleaning the tree,
bindeb-pkg simultaneously builds objects in the tree.





> +
> +PHONY += srcdeb-pkg
> +srcdeb-pkg:
>         $(MAKE) clean
>         $(CONFIG_SHELL) $(srctree)/scripts/package/mkdebian
>         $(call cmd,src_tar,$(KDEB_SOURCENAME))
>         origversion=$$(dpkg-parsechangelog -SVersion |sed 's/-[^-]*$$//');\
>                 mv $(KDEB_SOURCENAME).tar.gz ../$(KDEB_SOURCENAME)_$${origversion}.orig.tar.gz
> -       +dpkg-buildpackage -r$(KBUILD_PKG_ROOTCMD) -a$$(cat debian/arch) $(DPKG_FLAGS) -us -uc
> +       +dpkg-buildpackage -r$(KBUILD_PKG_ROOTCMD) -a$$(cat debian/arch) $(DPKG_FLAGS) -S -us -uc
>
>  PHONY += bindeb-pkg
>  bindeb-pkg:
> @@ -145,6 +148,7 @@ help:
>         @echo '  rpm-pkg             - Build both source and binary RPM kernel packages'
>         @echo '  binrpm-pkg          - Build only the binary kernel RPM package'
>         @echo '  deb-pkg             - Build both source and binary deb kernel packages'
> +       @echo '  srcdeb-pkg          - Build only the source kernel deb package'
>         @echo '  bindeb-pkg          - Build only the binary kernel deb package'
>         @echo '  snap-pkg            - Build only the binary kernel snap package'
>         @echo '                        (will connect to external hosts)'
> --
> 2.30.2
>


-- 
Best Regards
Masahiro Yamada