[PATCH 0/4] xfsprogs: packaging cleanups

[PATCH 0/4] xfsprogs: packaging cleanups

[Darrick J. Wong]

Hi all,

Introduce flatpak build system support, forward-port debian packages to
compat 11 to take advantage of systemd helper scripts.

If you're going to start using this mess, you probably ought to just
pull from my git trees, which are linked below.

This is an extraordinary way to destroy everything.  Enjoy!
Comments and questions are, as always, welcome.

--D

xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=packaging-cleanups

[PATCH 1/4] misc: support building flatpak images

[Darrick J. Wong]

From: Darrick J. Wong <darrick.wong@oracle.com>

Poke the system until it supports building flatpaks.  Maybe.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
 bootstrap  |   12 ++++++++++++
 flatpak.sh |   55 +++++++++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 67 insertions(+)
 create mode 100755 bootstrap
 create mode 100755 flatpak.sh


diff --git a/bootstrap b/bootstrap
new file mode 100755
index 00000000..d403cf58
--- /dev/null
+++ b/bootstrap
@@ -0,0 +1,12 @@
+#!/bin/bash -x
+
+libtoolize
+automake --add-missing
+aclocal -I m4
+autoconf
+make configure
+libtoolize
+automake --add-missing
+aclocal -I m4
+autoconf
+make configure
diff --git a/flatpak.sh b/flatpak.sh
new file mode 100755
index 00000000..a1ec916e
--- /dev/null
+++ b/flatpak.sh
@@ -0,0 +1,55 @@
+#!/bin/bash -xe
+
+if [ "$1" = "--help" ]; then
+	echo "Usage: $0 [branch] [build]"
+	echo "Build xfsprogs flatpak from the given branch."
+fi
+
+branch="$1"
+test -z "${branch}" && branch=master
+build="$2"
+test -z "${build}" && build=release
+
+if [ "${build}" = "debug" ]; then
+	config_opts=', "--disable-lto"'
+fi
+
+cat > flatpak.ini << ENDL
+[Application]
+name=org.xfs.progs
+runtime=org.freedesktop.Platform/$(uname -m)/18.08
+
+[Context]
+devices=all;
+ENDL
+
+cat > org.xfs.progs.json << ENDL
+{
+    "app-id": "org.xfs.progs",
+    "runtime": "org.freedesktop.Platform",
+    "runtime-version": "18.08",
+    "sdk": "org.freedesktop.Sdk",
+    "command": "/app/sbin/xfs_repair",
+    "metadata": "flatpak.ini",
+    "modules": [
+        {
+            "name": "xfsprogs",
+            "buildsystem": "autotools",
+            "config-opts": ["--enable-shared=no" ${config_opts} ],
+            "sources": [
+                {
+                    "type": "git",
+                    "path": "${PWD}/",
+                    "branch": "${branch}"
+                }
+            ]
+        }
+    ]
+}
+ENDL
+
+flatpak-builder --repo=repo build-dir org.xfs.progs.json --force-clean
+flatpak build-bundle repo xfsprogs.flatpak org.xfs.progs
+echo build done, run:
+echo flatpak install xfsprogs.flatpak
+echo flatpak run org.xfs.progs

[PATCH 2/4] debian: turn debhelper compat level up to 11

[Darrick J. Wong]

From: Darrick J. Wong <darrick.wong@oracle.com>

Upgrade to debhelper level 11 to take advantage of dh_installsystemd,
which greatly simplifies the installation and activation of the scrub
systemd services.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
 debian/compat |    2 +-
 debian/rules  |    1 +
 2 files changed, 2 insertions(+), 1 deletion(-)


diff --git a/debian/compat b/debian/compat
index ec635144..b4de3947 100644
--- a/debian/compat
+++ b/debian/compat
@@ -1 +1 @@
-9
+11
diff --git a/debian/rules b/debian/rules
index e8509fb3..a79db896 100755
--- a/debian/rules
+++ b/debian/rules
@@ -93,6 +93,7 @@ binary-arch: checkroot built
 	dh_compress
 	dh_fixperms
 	dh_makeshlibs
+	dh_installsystemd -p xfsprogs --no-enable --no-start --no-restart-after-upgrade --no-stop-on-upgrade
 	dh_installdeb
 	dh_shlibdeps
 	dh_gencontrol

[PATCH 3/4] debian: permit compat level 9 dh builds

[Darrick J. Wong]

From: Darrick J. Wong <darrick.wong@oracle.com>

Retain compatibility with existing dh level 9 build systems.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
 debian/rules |   16 +++++++++++++++-
 1 file changed, 15 insertions(+), 1 deletion(-)


diff --git a/debian/rules b/debian/rules
index a79db896..1dd1f524 100755
--- a/debian/rules
+++ b/debian/rules
@@ -1,5 +1,7 @@
 #!/usr/bin/make -f
 
+.PHONY: dpkg_config
+
 export DH_VERBOSE=1
 
 ifneq (,$(filter parallel=%,$(DEB_BUILD_OPTIONS)))
@@ -7,6 +9,9 @@ ifneq (,$(filter parallel=%,$(DEB_BUILD_OPTIONS)))
     PMAKEFLAGS += -j$(NUMJOBS)
 endif
 
+DH_VERSION := $(shell dpkg-query -W -f '$${Version}' debhelper)
+USE_DH9 ?= $(shell if dpkg --compare-versions $(DH_VERSION) lt 11 ; then echo yes ; fi)
+
 package = xfsprogs
 develop = xfslibs-dev
 bootpkg = xfsprogs-udeb
@@ -33,11 +38,18 @@ checkdir = test -f debian/rules
 build: build-arch build-indep
 build-arch: built
 build-indep: built
-built: dibuild config
+built: dpkg_config dibuild config
 	@echo "== dpkg-buildpackage: build" 1>&2
 	$(MAKE) $(PMAKEFLAGS) default
 	touch built
 
+dpkg_config:
+ifeq ($(USE_DH9),yes)
+	mv debian/compat debian/compat.save
+	echo 9 > debian/compat
+endif
+	@true
+
 config: .census
 .census:
 	@echo "== dpkg-buildpackage: configure" 1>&2
@@ -93,7 +105,9 @@ binary-arch: checkroot built
 	dh_compress
 	dh_fixperms
 	dh_makeshlibs
+ifneq ($(USE_DH9),yes)
 	dh_installsystemd -p xfsprogs --no-enable --no-start --no-restart-after-upgrade --no-stop-on-upgrade
+endif
 	dh_installdeb
 	dh_shlibdeps
 	dh_gencontrol

[PATCH 4/4] xfs_scrub_all: failure reporting for the xfs_scrub_all job

[Darrick J. Wong]

From: Darrick J. Wong <darrick.wong@oracle.com>

Create a failure reporting service for when xfs_scrub_all fails.  This
is probably a debug-only patch.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
 scrub/Makefile                      |    3 ++-
 scrub/xfs_scrub_all_fail            |   24 ++++++++++++++++++++++++
 scrub/xfs_scrub_all_fail.service.in |   11 +++++++++++
 3 files changed, 37 insertions(+), 1 deletion(-)
 create mode 100755 scrub/xfs_scrub_all_fail
 create mode 100644 scrub/xfs_scrub_all_fail.service.in


diff --git a/scrub/Makefile b/scrub/Makefile
index 47c887eb..50d16121 100644
--- a/scrub/Makefile
+++ b/scrub/Makefile
@@ -18,7 +18,7 @@ XFS_SCRUB_ALL_PROG = xfs_scrub_all
 XFS_SCRUB_ARGS = -b -n
 ifeq ($(HAVE_SYSTEMD),yes)
 INSTALL_SCRUB += install-systemd
-SYSTEMD_SERVICES = xfs_scrub@.service xfs_scrub_all.service xfs_scrub_all.timer xfs_scrub_fail@.service
+SYSTEMD_SERVICES = xfs_scrub@.service xfs_scrub_all.service xfs_scrub_all.timer xfs_scrub_fail@.service xfs_scrub_all_fail.service
 OPTIONAL_TARGETS += $(SYSTEMD_SERVICES)
 endif
 ifeq ($(HAVE_CROND),yes)
@@ -133,6 +133,7 @@ install-systemd: default $(SYSTEMD_SERVICES)
 	$(INSTALL) -m 644 $(SYSTEMD_SERVICES) $(SYSTEMD_SYSTEM_UNIT_DIR)
 	$(INSTALL) -m 755 -d $(PKG_LIB_DIR)/$(PKG_NAME)
 	$(INSTALL) -m 755 xfs_scrub_fail $(PKG_LIB_DIR)/$(PKG_NAME)
+	$(INSTALL) -m 755 xfs_scrub_all_fail $(PKG_LIB_DIR)/$(PKG_NAME)
 
 install-crond: default $(CRONTABS)
 	$(INSTALL) -m 755 -d $(CROND_DIR)
diff --git a/scrub/xfs_scrub_all_fail b/scrub/xfs_scrub_all_fail
new file mode 100755
index 00000000..8391abc8
--- /dev/null
+++ b/scrub/xfs_scrub_all_fail
@@ -0,0 +1,24 @@
+#!/bin/bash
+
+# Email logs of failed xfs_scrub_all unit runs
+
+mailer=/usr/sbin/sendmail
+recipient="$1"
+test -z "${recipient}" && exit 0
+hostname="$(hostname -f 2>/dev/null)"
+test -z "${hostname}" && hostname="${HOSTNAME}"
+if [ ! -x "${mailer}" ]; then
+	echo "${mailer}: Mailer program not found."
+	exit 1
+fi
+
+(cat << ENDL
+To: $1
+From: <xfs_scrub_all@${hostname}>
+Subject: xfs_scrub_all failure on ${mntpoint}
+
+So sorry, the automatic xfs_scrub_all on ${hostname} failed.
+
+A log of what happened follows:
+ENDL
+systemctl status --full --lines 4294967295 "xfs_scrub_all") | "${mailer}" -t -i
diff --git a/scrub/xfs_scrub_all_fail.service.in b/scrub/xfs_scrub_all_fail.service.in
new file mode 100644
index 00000000..2d85db28
--- /dev/null
+++ b/scrub/xfs_scrub_all_fail.service.in
@@ -0,0 +1,11 @@
+[Unit]
+Description=Online XFS Metadata Check for All Filesystems Failure Reporting for %I
+Documentation=man:xfs_scrub_all(8)
+
+[Service]
+Type=oneshot
+Environment=EMAIL_ADDR=root
+ExecStart=@pkg_lib_dir@/@pkg_name@/xfs_scrub_all_fail "${EMAIL_ADDR}"
+User=mail
+Group=mail
+SupplementaryGroups=systemd-journal

Re: [PATCH 1/4] misc: support building flatpak images

[Christoph Hellwig]

On Tue, Dec 31, 2019 at 05:22:47PM -0800, Darrick J. Wong wrote:
> From: Darrick J. Wong <darrick.wong@oracle.com>
> 
> Poke the system until it supports building flatpaks.  Maybe.

Why would we want to support such a fucked up package delivery
mechanism?

Re: [PATCH 2/4] debian: turn debhelper compat level up to 11

[Christoph Hellwig]

On Tue, Dec 31, 2019 at 05:22:53PM -0800, Darrick J. Wong wrote:
> From: Darrick J. Wong <darrick.wong@oracle.com>
> 
> Upgrade to debhelper level 11 to take advantage of dh_installsystemd,
> which greatly simplifies the installation and activation of the scrub
> systemd services.

The subject and description seem upside down.  You want to use
dh_installsystemd, and to do so you'll have to bump the level..

Re: [PATCH 2/4] debian: turn debhelper compat level up to 11

[Darrick J. Wong]

On Tue, Jan 07, 2020 at 06:21:23AM -0800, Christoph Hellwig wrote:
> On Tue, Dec 31, 2019 at 05:22:53PM -0800, Darrick J. Wong wrote:
> > From: Darrick J. Wong <darrick.wong@oracle.com>
> > 
> > Upgrade to debhelper level 11 to take advantage of dh_installsystemd,
> > which greatly simplifies the installation and activation of the scrub
> > systemd services.
> 
> The subject and description seem upside down.  You want to use
> dh_installsystemd, and to do so you'll have to bump the level..

Heh, ok.

"debian: install scrub services with dh_installsystemd

"Use dh_installsystemd to handle the installation and activation of the
scrub systemd services.  This requires bumping the compat version to
11."

(Admittedly the subject was a riff on the old 'turn it up to 11' joke)

Related question: Are Debian oldoldstable and Ubuntu 16.04LTS old enough
to drop the "debian: permit compat level 9 dh builds" patch?  On systems
with software that old the kernel isn't going to support scrub, and who's
going to build xfsprogs 5.xx for those old systems?

--D

Re: [PATCH 1/4] misc: support building flatpak images

[Darrick J. Wong]

On Tue, Jan 07, 2020 at 06:20:35AM -0800, Christoph Hellwig wrote:
> On Tue, Dec 31, 2019 at 05:22:47PM -0800, Darrick J. Wong wrote:
> > From: Darrick J. Wong <darrick.wong@oracle.com>
> > 
> > Poke the system until it supports building flatpaks.  Maybe.
> 
> Why would we want to support such a fucked up package delivery
> mechanism?

In theory we could use it as a means to distribute uptodate xfs_repair
so that anyone with a problem that their distro's copy of xfs_repair
can't or won't fix can try upstream without having to build xfsprogs
themselves.

Though I wonder how many people in that situation are running some
colorful linux distribution but are too stingy to pay for a support
contract.... :)

Also, can you elaborate on 'fucked up package delivery mechanism'?

Admittedly I'm /not/ volunteering to maintain a xfs_repair flatpak, so
perhaps I should just drop this whole thing.

--D

Re: [PATCH 2/4] debian: turn debhelper compat level up to 11

[Christoph Hellwig]

On Tue, Jan 07, 2020 at 10:51:51AM -0800, Darrick J. Wong wrote:
> Related question: Are Debian oldoldstable and Ubuntu 16.04LTS old enough
> to drop the "debian: permit compat level 9 dh builds" patch?  On systems
> with software that old the kernel isn't going to support scrub, and who's
> going to build xfsprogs 5.xx for those old systems?

This sounds old enough to not bother.  And if someone screams loud
enough we could still add it back.

Re: [PATCH 1/4] misc: support building flatpak images

[Christoph Hellwig]

On Tue, Jan 07, 2020 at 11:35:37AM -0800, Darrick J. Wong wrote:
> In theory we could use it as a means to distribute uptodate xfs_repair
> so that anyone with a problem that their distro's copy of xfs_repair
> can't or won't fix can try upstream without having to build xfsprogs
> themselves.

And someone would allow a random weirdo package format access to their
disks?

> Also, can you elaborate on 'fucked up package delivery mechanism'?

It completely messes up how the system is supposed to work, and not your
are in a maze of unssuportable package versions just like npm.