[Buildroot] package/libfuse3: new package [V2]

[Buildroot] package/libfuse3: new package [V2]

[Norbert Lange]

New since V1:
-   be more verbose both in commit msg as in comments
    about sharing files between libfuse3 and libfuse
-   add *_DEVICES for static device table
-   add myself to DEVELOPERS
-   rework target installation for libfuse.
    Now installs to temp dir first, to not overwrite files
    from libfuse3
-   adopt patch that fixes installation
    now allows some failures instead removing the operation

[Buildroot] [PATCH v2 1/4] package/libfuse: Install udev rules and set permissions

[Norbert Lange]

This fixes some omissions from the installation.

Install the udev rules.

Tell buildroot about the fuse device.

Apply setuid permissions on the fusermount tool.

Signed-off-by: Norbert Lange <norbert.lange@andritz.com>
---
 package/libfuse/libfuse.mk | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)

diff --git a/package/libfuse/libfuse.mk b/package/libfuse/libfuse.mk
index e8a79a3166..84027ebedb 100644
--- a/package/libfuse/libfuse.mk
+++ b/package/libfuse/libfuse.mk
@@ -14,11 +14,22 @@ LIBFUSE_DEPENDENCIES = $(if $(BR2_PACKAGE_LIBICONV),libiconv)
 LIBFUSE_CONF_OPTS = \
 	--disable-example \
 	--enable-lib \
-	--enable-util
+	--enable-util \
+	UDEV_RULES_PATH=/lib/udev/rules.d
 
 define LIBFUSE_INSTALL_TARGET_CMDS
 	cp -dpf $(STAGING_DIR)/usr/bin/fusermount $(TARGET_DIR)/usr/bin/
 	cp -dpf $(STAGING_DIR)/usr/lib/libfuse.so* $(TARGET_DIR)/usr/lib/
+	mkdir -p $(TARGET_DIR)/lib/udev/rules.d
+	cp $(STAGING_DIR)/lib/udev/rules.d/*-fuse.rules $(TARGET_DIR)/lib/udev/rules.d
+endef
+
+define LIBFUSE_DEVICES
+	/dev/fuse  c  666  0  0  10  229  0  0  -
+endef
+
+define LIBFUSE_PERMISSIONS
+	/usr/bin/fusermount f 4755 0 0 - - - - -
 endef
 
 $(eval $(autotools-package))
-- 
2.20.1

[Buildroot] [PATCH v2 1/4] package/libfuse: Install udev rules and set permissions

[Arnout Vandecappelle]

On 15/03/2019 14:30, Norbert Lange wrote:
> This fixes some omissions from the installation.
> 
> Install the udev rules.
> 
> Tell buildroot about the fuse device.
> 
> Apply setuid permissions on the fusermount tool.
> 
> Signed-off-by: Norbert Lange <norbert.lange@andritz.com>

 Applied to master, thanks.

 Regards,
 Arnout

[Buildroot] [PATCH v2 1/4] package/libfuse: Install udev rules and set permissions

[Peter Korsgaard]

>>>>> "Norbert" == Norbert Lange <nolange79@gmail.com> writes:

 > This fixes some omissions from the installation.
 > Install the udev rules.

 > Tell buildroot about the fuse device.

 > Apply setuid permissions on the fusermount tool.

 > Signed-off-by: Norbert Lange <norbert.lange@andritz.com>
 > ---
 >  package/libfuse/libfuse.mk | 13 ++++++++++++-
 >  1 file changed, 12 insertions(+), 1 deletion(-)

 > diff --git a/package/libfuse/libfuse.mk b/package/libfuse/libfuse.mk
 > index e8a79a3166..84027ebedb 100644
 > --- a/package/libfuse/libfuse.mk
 > +++ b/package/libfuse/libfuse.mk
 > @@ -14,11 +14,22 @@ LIBFUSE_DEPENDENCIES = $(if $(BR2_PACKAGE_LIBICONV),libiconv)
 >  LIBFUSE_CONF_OPTS = \
 >  	--disable-example \
 >  	--enable-lib \
 > -	--enable-util
 > +	--enable-util \
 > +	UDEV_RULES_PATH=/lib/udev/rules.d
 
 >  define LIBFUSE_INSTALL_TARGET_CMDS
 >  	cp -dpf $(STAGING_DIR)/usr/bin/fusermount $(TARGET_DIR)/usr/bin/
 >  	cp -dpf $(STAGING_DIR)/usr/lib/libfuse.so* $(TARGET_DIR)/usr/lib/
 > +	mkdir -p $(TARGET_DIR)/lib/udev/rules.d
 > +	cp $(STAGING_DIR)/lib/udev/rules.d/*-fuse.rules $(TARGET_DIR)/lib/udev/rules.d

It would have been nicer to only do this if (e)udev was enabled. I've
sent a patch to do so:

https://patchwork.ozlabs.org/patch/1074963/

-- 
Bye, Peter Korsgaard

[Buildroot] [PATCH v2 1/4] package/libfuse: Install udev rules and set permissions

[Peter Korsgaard]

>>>>> "Norbert" == Norbert Lange <nolange79@gmail.com> writes:
 >> This fixes some omissions from the installation.
 >> Install the udev rules.

 >> Tell buildroot about the fuse device.

 >> Apply setuid permissions on the fusermount tool.

 >> Signed-off-by: Norbert Lange <norbert.lange@andritz.com>

Committed to 2019.02.x, thanks.

-- 
Bye, Peter Korsgaard

[Buildroot] [PATCH v2 2/4] package/libfuse3: new package

[Norbert Lange]

Signed-off-by: Norbert Lange <norbert.lange@andritz.com>
---
 package/Config.in                      |  1 +
 package/libfuse3/Config.in             | 12 ++++++++
 package/libfuse3/disable_fileops.patch | 23 +++++++++++++++
 package/libfuse3/libfuse3.hash         |  8 ++++++
 package/libfuse3/libfuse3.mk           | 40 ++++++++++++++++++++++++++
 5 files changed, 84 insertions(+)
 create mode 100644 package/libfuse3/Config.in
 create mode 100644 package/libfuse3/disable_fileops.patch
 create mode 100644 package/libfuse3/libfuse3.hash
 create mode 100644 package/libfuse3/libfuse3.mk

diff --git a/package/Config.in b/package/Config.in
index 3231aada31..bcd1c6297f 100644
--- a/package/Config.in
+++ b/package/Config.in
@@ -1231,6 +1231,7 @@ menu "Filesystem"
 	source "package/libconfig/Config.in"
 	source "package/libconfuse/Config.in"
 	source "package/libfuse/Config.in"
+	source "package/libfuse3/Config.in"
 	source "package/liblockfile/Config.in"
 	source "package/libnfs/Config.in"
 	source "package/libsysfs/Config.in"
diff --git a/package/libfuse3/Config.in b/package/libfuse3/Config.in
new file mode 100644
index 0000000000..d349fcd67f
--- /dev/null
+++ b/package/libfuse3/Config.in
@@ -0,0 +1,12 @@
+config BR2_PACKAGE_LIBFUSE3
+	bool "libfuse3"
+	depends on BR2_TOOLCHAIN_HAS_THREADS
+	depends on BR2_USE_MMU # fork()
+	help
+	  FUSE 3 (Filesystem in UserSpacE)
+
+	  https://github.com/libfuse/libfuse
+
+comment "libfuse3 needs a toolchain w/ threads"
+	depends on BR2_USE_MMU
+	depends on !BR2_TOOLCHAIN_HAS_THREADS
diff --git a/package/libfuse3/disable_fileops.patch b/package/libfuse3/disable_fileops.patch
new file mode 100644
index 0000000000..bd3a8e2be3
--- /dev/null
+++ b/package/libfuse3/disable_fileops.patch
@@ -0,0 +1,23 @@
+diff -burN fuse-3.4.1.org/util/install_helper.sh fuse-3.4.1/util/install_helper.sh
+--- fuse-3.4.1.org/util/install_helper.sh	2019-02-12 14:33:18.549507335 +0100
++++ fuse-3.4.1/util/install_helper.sh	2019-02-12 14:35:20.726772913 +0100
+@@ -22,8 +22,8 @@
+     DESTDIR="${DESTDIR%/}"
+ fi
+
+-chown root:root "${DESTDIR}${bindir}/fusermount3"
+-chmod u+s "${DESTDIR}${bindir}/fusermount3"
++chown root:root "${DESTDIR}${bindir}/fusermount3" || :
++chmod u+s "${DESTDIR}${bindir}/fusermount3" || :
+
+ install -D -m 644 "${MESON_SOURCE_ROOT}/util/fuse.conf" \
+ 	"${DESTDIR}${sysconfdir}/fuse.conf"
+@@ -31,7 +31,7 @@
+
+ if test ! -e "${DESTDIR}/dev/fuse"; then
+     mkdir -p "${DESTDIR}/dev"
+-    mknod "${DESTDIR}/dev/fuse" -m 0666 c 10 229
++    mknod "${DESTDIR}/dev/fuse" -m 0666 c 10 229 || :
+ fi
+
+ install -D -m 644 "${MESON_SOURCE_ROOT}/util/udev.rules" \
diff --git a/package/libfuse3/libfuse3.hash b/package/libfuse3/libfuse3.hash
new file mode 100644
index 0000000000..e9b9f8384c
--- /dev/null
+++ b/package/libfuse3/libfuse3.hash
@@ -0,0 +1,8 @@
+# Locally calculated after checking pgp signature
+sha256  224dd4a598e23e114395a9717bc79638ab2b1e42c82ae8210aed9365aff325a3  fuse-3.4.2.tar.xz
+
+
+# Hash for license files:
+sha256  8177f97513213526df2cf6184d8ff986c675afb514d4e68a404010521b880643  GPL2.txt
+sha256  dc626520dcd53a22f727af3ee42c770e56c97a64fe3adb063799d8ab032fe551  LGPL2.txt
+sha256  b8832d9caaa075bbbd2aef24efa09f8b7ab66a832812d88c602da0c7b4397fad  LICENSE
diff --git a/package/libfuse3/libfuse3.mk b/package/libfuse3/libfuse3.mk
new file mode 100644
index 0000000000..ff8d251a96
--- /dev/null
+++ b/package/libfuse3/libfuse3.mk
@@ -0,0 +1,40 @@
+################################################################################
+#
+# libfuse
+#
+################################################################################
+
+LIBFUSE3_VERSION = 3.4.2
+LIBFUSE3_SOURCE = fuse-$(LIBFUSE3_VERSION).tar.xz
+LIBFUSE3_SITE = https://github.com/libfuse/libfuse/releases/download/fuse-$(LIBFUSE3_VERSION)
+LIBFUSE3_LICENSE = LGPL-2.1 (library), GPL-2.0 (rest)
+LIBFUSE3_LICENSE_FILES = GPL2.txt LGPL2.txt LICENSE
+LIBFUSE3_INSTALL_STAGING = YES
+LIBFUSE3_DEPENDENCIES = $(if $(BR2_PACKAGE_LIBICONV),libiconv)
+LIBFUSE3_CONF_OPTS = \
+	-Dexamples=false \
+	-Dutils=true \
+	-Dudevrulesdir=/lib/udev/rules.d
+
+define LIBFUSE3_INSTALL_TARGET_CMDS
+	$(INSTALL) -D -m 0755 $(STAGING_DIR)/usr/bin/fusermount3 $(TARGET_DIR)/usr/bin/
+	$(if $(filter y,$(BR2_STATIC_LIBS)),,cp -dpf $(STAGING_DIR)/usr/lib/libfuse3.so* $(TARGET_DIR)/usr/lib/)
+	mkdir -p $(TARGET_DIR)/lib/udev/rules.d
+	$(INSTALL) -m 0644 $(STAGING_DIR)/lib/udev/rules.d/*-fuse3.rules $(TARGET_DIR)/lib/udev/rules.d
+endef
+
+define LIBFUSE3_INSTALL_TARGET_POST
+	ln -sf fusermount3 $(TARGET_DIR)/usr/bin/fusermount
+endef
+
+LIBFUSE3_POST_INSTALL_TARGET_HOOKS += LIBFUSE3_INSTALL_TARGET_POST
+
+define LIBFUSE3_DEVICES
+	/dev/fuse  c  666  0  0  10  229  0  0  -
+endef
+
+define LIBFUSE3_PERMISSIONS
+	/usr/bin/fusermount3 f 4755 0 0 - - - - -
+endef
+
+$(eval $(meson-package))
-- 
2.20.1

[Buildroot] [PATCH v2 2/4] package/libfuse3: new package

[Arnout Vandecappelle]

On 15/03/2019 14:30, Norbert Lange wrote:
> Signed-off-by: Norbert Lange <norbert.lange@andritz.com>
> ---
>  package/Config.in                      |  1 +
>  package/libfuse3/Config.in             | 12 ++++++++
>  package/libfuse3/disable_fileops.patch | 23 +++++++++++++++
>  package/libfuse3/libfuse3.hash         |  8 ++++++
>  package/libfuse3/libfuse3.mk           | 40 ++++++++++++++++++++++++++
>  5 files changed, 84 insertions(+)
>  create mode 100644 package/libfuse3/Config.in
>  create mode 100644 package/libfuse3/disable_fileops.patch
>  create mode 100644 package/libfuse3/libfuse3.hash
>  create mode 100644 package/libfuse3/libfuse3.mk

[snip]
> diff --git a/package/libfuse3/disable_fileops.patch b/package/libfuse3/disable_fileops.patch
> new file mode 100644
> index 0000000000..bd3a8e2be3
> --- /dev/null
> +++ b/package/libfuse3/disable_fileops.patch
> @@ -0,0 +1,23 @@
> +diff -burN fuse-3.4.1.org/util/install_helper.sh fuse-3.4.1/util/install_helper.sh

 The patch should have a commit message and a signed-off-by.

 Also, since upstream is git, it should be git-formatted.

 Also, please send it upstream. Ideally, send it upstream first, wait a little
bit for feedback, and then when submitting to Buildroot, include the upstream
reference. If it is not yet merged, add something like

Upstream: https://github.com/libfuse/libfuse/pulls/385

If it was already accepted upstream, just do a git-format-patch -N of the
upstream commit (the commit hash will be the same as the upstream one).

> +--- fuse-3.4.1.org/util/install_helper.sh	2019-02-12 14:33:18.549507335 +0100
> ++++ fuse-3.4.1/util/install_helper.sh	2019-02-12 14:35:20.726772913 +0100
> +@@ -22,8 +22,8 @@
> +     DESTDIR="${DESTDIR%/}"
> + fi
> +
> +-chown root:root "${DESTDIR}${bindir}/fusermount3"
> +-chmod u+s "${DESTDIR}${bindir}/fusermount3"
> ++chown root:root "${DESTDIR}${bindir}/fusermount3" || :
> ++chmod u+s "${DESTDIR}${bindir}/fusermount3" || :
> +
> + install -D -m 644 "${MESON_SOURCE_ROOT}/util/fuse.conf" \
> + 	"${DESTDIR}${sysconfdir}/fuse.conf"
> +@@ -31,7 +31,7 @@
> +
> + if test ! -e "${DESTDIR}/dev/fuse"; then
> +     mkdir -p "${DESTDIR}/dev"
> +-    mknod "${DESTDIR}/dev/fuse" -m 0666 c 10 229
> ++    mknod "${DESTDIR}/dev/fuse" -m 0666 c 10 229 || :
> + fi
> +
> + install -D -m 644 "${MESON_SOURCE_ROOT}/util/udev.rules" \
> diff --git a/package/libfuse3/libfuse3.hash b/package/libfuse3/libfuse3.hash
> new file mode 100644
> index 0000000000..e9b9f8384c
> --- /dev/null
> +++ b/package/libfuse3/libfuse3.hash
> @@ -0,0 +1,8 @@
> +# Locally calculated after checking pgp signature
> +sha256  224dd4a598e23e114395a9717bc79638ab2b1e42c82ae8210aed9365aff325a3  fuse-3.4.2.tar.xz
> +
> +

 Just one empty line.

> +# Hash for license files:
> +sha256  8177f97513213526df2cf6184d8ff986c675afb514d4e68a404010521b880643  GPL2.txt
> +sha256  dc626520dcd53a22f727af3ee42c770e56c97a64fe3adb063799d8ab032fe551  LGPL2.txt
> +sha256  b8832d9caaa075bbbd2aef24efa09f8b7ab66a832812d88c602da0c7b4397fad  LICENSE
> diff --git a/package/libfuse3/libfuse3.mk b/package/libfuse3/libfuse3.mk
> new file mode 100644
> index 0000000000..ff8d251a96
> --- /dev/null
> +++ b/package/libfuse3/libfuse3.mk
> @@ -0,0 +1,40 @@
> +################################################################################
> +#
> +# libfuse
> +#
> +################################################################################
> +
> +LIBFUSE3_VERSION = 3.4.2
> +LIBFUSE3_SOURCE = fuse-$(LIBFUSE3_VERSION).tar.xz
> +LIBFUSE3_SITE = https://github.com/libfuse/libfuse/releases/download/fuse-$(LIBFUSE3_VERSION)
> +LIBFUSE3_LICENSE = LGPL-2.1 (library), GPL-2.0 (rest)
> +LIBFUSE3_LICENSE_FILES = GPL2.txt LGPL2.txt LICENSE

 PLease keep the order of the license files the same as the licenses. I would
put LICENSE first, though.

> +LIBFUSE3_INSTALL_STAGING = YES
> +LIBFUSE3_DEPENDENCIES = $(if $(BR2_PACKAGE_LIBICONV),libiconv)
> +LIBFUSE3_CONF_OPTS = \
> +	-Dexamples=false \
> +	-Dutils=true \
> +	-Dudevrulesdir=/lib/udev/rules.d
> +
> +define LIBFUSE3_INSTALL_TARGET_CMDS
> +	$(INSTALL) -D -m 0755 $(STAGING_DIR)/usr/bin/fusermount3 $(TARGET_DIR)/usr/bin/
> +	$(if $(filter y,$(BR2_STATIC_LIBS)),,cp -dpf $(STAGING_DIR)/usr/lib/libfuse3.so* $(TARGET_DIR)/usr/lib/)

 This looks ugly.

 Why doesn't the default make install not work? I think this was discussed
before, but the commit message gives me no enlightenment. If I already forget it
after a week, imagine what will happen in a few years when it becomes relevant
again...

> +	mkdir -p $(TARGET_DIR)/lib/udev/rules.d
> +	$(INSTALL) -m 0644 $(STAGING_DIR)/lib/udev/rules.d/*-fuse3.rules $(TARGET_DIR)/lib/udev/rules.d
> +endef
> +
> +define LIBFUSE3_INSTALL_TARGET_POST

 Give a better name, e.g. LIBFUSE3_INSTALL_TARGET_FUSERMOUNT_SYMLINK.

> +	ln -sf fusermount3 $(TARGET_DIR)/usr/bin/fusermount
> +endef
> +
> +LIBFUSE3_POST_INSTALL_TARGET_HOOKS += LIBFUSE3_INSTALL_TARGET_POST

 The reason to make this a post-install-hook is that you'd be able to use the
default install commands. If that is not possible, you can do it directly in the
install commands.

 Regards,
 Arnout

> +
> +define LIBFUSE3_DEVICES
> +	/dev/fuse  c  666  0  0  10  229  0  0  -
> +endef
> +
> +define LIBFUSE3_PERMISSIONS
> +	/usr/bin/fusermount3 f 4755 0 0 - - - - -
> +endef
> +
> +$(eval $(meson-package))
> 

[Buildroot] [PATCH v2 3/4] add myself to DEVELOPERS

[Norbert Lange]

Signed-off-by: Norbert Lange <norbert.lange@andritz.com>
---
 DEVELOPERS | 1 +
 1 file changed, 1 insertion(+)

diff --git a/DEVELOPERS b/DEVELOPERS
index 8a57cb2e23..ce0e9234d8 100644
--- a/DEVELOPERS
+++ b/DEVELOPERS
@@ -1628,6 +1628,7 @@ F:	package/tpm-tools/
 F:	package/trousers/
 
 N:	Norbert Lange <nolange79@gmail.com>
+F:  package/libfuse3/
 F:	package/tcf-agent/
 
 N:	Olaf Rempel <razzor@kopf-tisch.de>
-- 
2.20.1

[Buildroot] [PATCH v2 3/4] add myself to DEVELOPERS

[Arnout Vandecappelle]

On 15/03/2019 14:30, Norbert Lange wrote:
> Signed-off-by: Norbert Lange <norbert.lange@andritz.com>
> ---
>  DEVELOPERS | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/DEVELOPERS b/DEVELOPERS
> index 8a57cb2e23..ce0e9234d8 100644
> --- a/DEVELOPERS
> +++ b/DEVELOPERS
> @@ -1628,6 +1628,7 @@ F:	package/tpm-tools/
>  F:	package/trousers/
>  
>  N:	Norbert Lange <nolange79@gmail.com>
> +F:  package/libfuse3/

 Indent with tab.

 Also, this should be squashed with the previous one (when a package is added,
to the DEVELOPERS entry right away).

 Regards,
 Arnout

>  F:	package/tcf-agent/
>  
>  N:	Olaf Rempel <razzor@kopf-tisch.de>
> 

[Buildroot] [PATCH v2 4/4] package/libfuse: common files from libfuse3 are prefered

[Norbert Lange]

if both libfuse and libfuse3 are co-installed,
make sure libfuse3 will be build first and
only install the files unique to libfuse.

The fusermount tool is a simlink to fusermount3,
mimicing debians approach. fuse device and udev rule
is identicall between packages and only one definition
is needed

see the libfuse3 Changelog for 3.0.0 for details.

Signed-off-by: Norbert Lange <norbert.lange@andritz.com>
---
 package/libfuse/libfuse.mk | 35 ++++++++++++++++++++++++++++++-----
 1 file changed, 30 insertions(+), 5 deletions(-)

diff --git a/package/libfuse/libfuse.mk b/package/libfuse/libfuse.mk
index 84027ebedb..a0a00a6532 100644
--- a/package/libfuse/libfuse.mk
+++ b/package/libfuse/libfuse.mk
@@ -10,20 +10,44 @@ LIBFUSE_SITE = https://github.com/libfuse/libfuse/releases/download/fuse-$(LIBFU
 LIBFUSE_LICENSE = GPL-2.0, LGPL-2.1
 LIBFUSE_LICENSE_FILES = COPYING COPYING.LIB
 LIBFUSE_INSTALL_STAGING = YES
-LIBFUSE_DEPENDENCIES = $(if $(BR2_PACKAGE_LIBICONV),libiconv)
+LIBFUSE_DEPENDENCIES = $(if $(BR2_PACKAGE_LIBICONV),libiconv) $(if $(BR2_PACKAGE_LIBFUSE3),libfuse3)
 LIBFUSE_CONF_OPTS = \
 	--disable-example \
 	--enable-lib \
 	--enable-util \
 	UDEV_RULES_PATH=/lib/udev/rules.d
 
-define LIBFUSE_INSTALL_TARGET_CMDS
-	cp -dpf $(STAGING_DIR)/usr/bin/fusermount $(TARGET_DIR)/usr/bin/
-	cp -dpf $(STAGING_DIR)/usr/lib/libfuse.so* $(TARGET_DIR)/usr/lib/
+# From libfuse3 README:
+# libfuse 3 is designed to be co-installable with libfuse 2. However, some
+# files will be installed by both libfuse 2 and libfuse 3
+# (e.g. /etc/fuse.conf, the udev and init scripts, and the mount.fuse(8) manpage).
+# These files should be taken from libfuse 3. The format/content is guaranteed
+# to remain backwards compatible with libfuse 2.
+#
+# The way we handle this is to let libfuse3 install as usual,
+# but libfuse has to be carefull to not overwrite any common files.
+# Also some files are named diferently (udev-rules),
+# but only the newer is needed.
+# To ensure this, we install in a temporary directory and
+# hand-pick the few unique files
+
+LIBFUSE_INSTALL_TARGET_OPTS = install DESTDIR=$(@D)/tmpinstall
+
+define LIBFUSE_INSTALL_TARGET_POST
+	cp -dpf $(@D)/tmpinstall/usr/lib/libfuse.so* $(TARGET_DIR)/usr/lib/
+endef
+
+LIBFUSE_POST_INSTALL_TARGET_HOOKS += LIBFUSE_INSTALL_TARGET_POST
+
+ifeq ($(BR2_PACKAGE_LIBFUSE3),)
+define LIBFUSE_INSTALL_TARGET_POST_COMMON
+	$(INSTALL) -D -m 0755 $(@D)/tmpinstall/usr/bin/fusermount $(TARGET_DIR)/usr/bin/fusermount
 	mkdir -p $(TARGET_DIR)/lib/udev/rules.d
-	cp $(STAGING_DIR)/lib/udev/rules.d/*-fuse.rules $(TARGET_DIR)/lib/udev/rules.d
+	cp $(@D)/tmpinstall/lib/udev/rules.d/*-fuse.rules $(TARGET_DIR)/lib/udev/rules.d
 endef
 
+LIBFUSE_POST_INSTALL_TARGET_HOOKS += LIBFUSE_INSTALL_TARGET_POST_COMMON
+
 define LIBFUSE_DEVICES
 	/dev/fuse  c  666  0  0  10  229  0  0  -
 endef
@@ -31,5 +55,6 @@ endef
 define LIBFUSE_PERMISSIONS
 	/usr/bin/fusermount f 4755 0 0 - - - - -
 endef
+endif
 
 $(eval $(autotools-package))
-- 
2.20.1

[Buildroot] [PATCH v2 4/4] package/libfuse: common files from libfuse3 are prefered

[Arnout Vandecappelle]

On 15/03/2019 14:30, Norbert Lange wrote:
> if both libfuse and libfuse3 are co-installed,
> make sure libfuse3 will be build first and

 Why does libfuse3 have to be installed first? Are there any files that are
installed by both after this patch? If so, please list them here. But then the
sentence below is not true :-)

> only install the files unique to libfuse.
> 
> The fusermount tool is a simlink to fusermount3,

 symlink

> mimicing debians approach. fuse device and udev rule
> is identicall between packages and only one definition

 identical

> is needed

 Missing . at the end of the sentence.

> 
> see the libfuse3 Changelog for 3.0.0 for details.

 See

 However, this patch should also be squashed with the one adding libfuse3. Or,
if you want to separate things a little (possibly a good idea), then originally
libfuse3 should depend on !libfuse, and this patch would lift the dependency again.

> 
> Signed-off-by: Norbert Lange <norbert.lange@andritz.com>
> ---
>  package/libfuse/libfuse.mk | 35 ++++++++++++++++++++++++++++++-----
>  1 file changed, 30 insertions(+), 5 deletions(-)
> 
> diff --git a/package/libfuse/libfuse.mk b/package/libfuse/libfuse.mk
> index 84027ebedb..a0a00a6532 100644
> --- a/package/libfuse/libfuse.mk
> +++ b/package/libfuse/libfuse.mk
> @@ -10,20 +10,44 @@ LIBFUSE_SITE = https://github.com/libfuse/libfuse/releases/download/fuse-$(LIBFU
>  LIBFUSE_LICENSE = GPL-2.0, LGPL-2.1
>  LIBFUSE_LICENSE_FILES = COPYING COPYING.LIB
>  LIBFUSE_INSTALL_STAGING = YES
> -LIBFUSE_DEPENDENCIES = $(if $(BR2_PACKAGE_LIBICONV),libiconv)
> +LIBFUSE_DEPENDENCIES = $(if $(BR2_PACKAGE_LIBICONV),libiconv) $(if $(BR2_PACKAGE_LIBFUSE3),libfuse3)
>  LIBFUSE_CONF_OPTS = \
>  	--disable-example \
>  	--enable-lib \
>  	--enable-util \
>  	UDEV_RULES_PATH=/lib/udev/rules.d

 Why does this have to be set explicitly? Isn't it the default?

>  
> -define LIBFUSE_INSTALL_TARGET_CMDS
> -	cp -dpf $(STAGING_DIR)/usr/bin/fusermount $(TARGET_DIR)/usr/bin/
> -	cp -dpf $(STAGING_DIR)/usr/lib/libfuse.so* $(TARGET_DIR)/usr/lib/
> +# From libfuse3 README:
> +# libfuse 3 is designed to be co-installable with libfuse 2. However, some
> +# files will be installed by both libfuse 2 and libfuse 3
> +# (e.g. /etc/fuse.conf, the udev and init scripts, and the mount.fuse(8) manpage).
> +# These files should be taken from libfuse 3. The format/content is guaranteed
> +# to remain backwards compatible with libfuse 2.
> +#
> +# The way we handle this is to let libfuse3 install as usual,
> +# but libfuse has to be carefull to not overwrite any common files.
> +# Also some files are named diferently (udev-rules),
> +# but only the newer is needed.
> +# To ensure this, we install in a temporary directory and
> +# hand-pick the few unique files
> +
> +LIBFUSE_INSTALL_TARGET_OPTS = install DESTDIR=$(@D)/tmpinstall
> +
> +define LIBFUSE_INSTALL_TARGET_POST
> +	cp -dpf $(@D)/tmpinstall/usr/lib/libfuse.so* $(TARGET_DIR)/usr/lib/
> +endef

 This is way too complicated.

 Since we anyway have custom commands for target install, you can do it like this:

ifeq ($(BR2_PACKAGE_LIBFUSE3),)
define LIBFUSE_INSTALL_SUPPORT_FILES
        cp -dpf $(STAGING_DIR)/usr/bin/fusermount $(TARGET_DIR)/usr/bin/
        mkdir -p $(TARGET_DIR)/lib/udev/rules.d
        cp $(STAGING_DIR)/lib/udev/rules.d/*-fuse.rules \
		$(TARGET_DIR)/lib/udev/rules.d
endef
endif

define LIBFUSE_INSTALL_TARGET_CMDS
        cp -dpf $(STAGING_DIR)/usr/lib/libfuse.so* $(TARGET_DIR)/usr/lib/
	$(LIBFUSE_INSTALL_SUPPORT_FILES)
endef

 Regards,
 Arnout

> +
> +LIBFUSE_POST_INSTALL_TARGET_HOOKS += LIBFUSE_INSTALL_TARGET_POST
> +
> +ifeq ($(BR2_PACKAGE_LIBFUSE3),)
> +define LIBFUSE_INSTALL_TARGET_POST_COMMON
> +	$(INSTALL) -D -m 0755 $(@D)/tmpinstall/usr/bin/fusermount $(TARGET_DIR)/usr/bin/fusermount
>  	mkdir -p $(TARGET_DIR)/lib/udev/rules.d
> -	cp $(STAGING_DIR)/lib/udev/rules.d/*-fuse.rules $(TARGET_DIR)/lib/udev/rules.d
> +	cp $(@D)/tmpinstall/lib/udev/rules.d/*-fuse.rules $(TARGET_DIR)/lib/udev/rules.d
>  endef
>  
> +LIBFUSE_POST_INSTALL_TARGET_HOOKS += LIBFUSE_INSTALL_TARGET_POST_COMMON
> +
>  define LIBFUSE_DEVICES
>  	/dev/fuse  c  666  0  0  10  229  0  0  -
>  endef
> @@ -31,5 +55,6 @@ endef
>  define LIBFUSE_PERMISSIONS
>  	/usr/bin/fusermount f 4755 0 0 - - - - -
>  endef
> +endif
>  
>  $(eval $(autotools-package))
> 

[Buildroot] [PATCH v2 4/4] package/libfuse: common files from libfuse3 are prefered

[Norbert Lange]

Am Mi., 20. M?rz 2019 um 23:55 Uhr schrieb Arnout Vandecappelle
<arnout@mind.be>:
>
>
>
> On 15/03/2019 14:30, Norbert Lange wrote:
> > if both libfuse and libfuse3 are co-installed,
> > make sure libfuse3 will be build first and
>
>  Why does libfuse3 have to be installed first? Are there any files that are
> installed by both after this patch? If so, please list them here. But then the
> sentence below is not true :-)

I consider the common files part of libfuse3, so if you
'make libfuse' it should be ready-to-use.
The common files are just the udev-rules (which currently are named differently)

>
> > only install the files unique to libfuse.
> >
> > The fusermount tool is a simlink to fusermount3,
>
>  symlink
>
> > mimicing debians approach. fuse device and udev rule
> > is identicall between packages and only one definition
>
>  identical
>
> > is needed
>
>  Missing . at the end of the sentence.
>
> >
> > see the libfuse3 Changelog for 3.0.0 for details.
>
>  See

Tough crowd. Fixed.


>  However, this patch should also be squashed with the one adding libfuse3. Or,
> if you want to separate things a little (possibly a good idea), then originally
> libfuse3 should depend on !libfuse, and this patch would lift the dependency again.

K, primary though was that the first patch would be faster to accept.


>
> >
> > Signed-off-by: Norbert Lange <norbert.lange@andritz.com>
> > ---
> >  package/libfuse/libfuse.mk | 35 ++++++++++++++++++++++++++++++-----
> >  1 file changed, 30 insertions(+), 5 deletions(-)
> >
> > diff --git a/package/libfuse/libfuse.mk b/package/libfuse/libfuse.mk
> > index 84027ebedb..a0a00a6532 100644
> > --- a/package/libfuse/libfuse.mk
> > +++ b/package/libfuse/libfuse.mk
> > @@ -10,20 +10,44 @@ LIBFUSE_SITE = https://github.com/libfuse/libfuse/releases/download/fuse-$(LIBFU
> >  LIBFUSE_LICENSE = GPL-2.0, LGPL-2.1
> >  LIBFUSE_LICENSE_FILES = COPYING COPYING.LIB
> >  LIBFUSE_INSTALL_STAGING = YES
> > -LIBFUSE_DEPENDENCIES = $(if $(BR2_PACKAGE_LIBICONV),libiconv)
> > +LIBFUSE_DEPENDENCIES = $(if $(BR2_PACKAGE_LIBICONV),libiconv) $(if $(BR2_PACKAGE_LIBFUSE3),libfuse3)
> >  LIBFUSE_CONF_OPTS = \
> >       --disable-example \
> >       --enable-lib \
> >       --enable-util \
> >       UDEV_RULES_PATH=/lib/udev/rules.d
>
>  Why does this have to be set explicitly? Isn't it the default?

no, its '/etc/udev/rules.d'

>
> >
> > -define LIBFUSE_INSTALL_TARGET_CMDS
> > -     cp -dpf $(STAGING_DIR)/usr/bin/fusermount $(TARGET_DIR)/usr/bin/
> > -     cp -dpf $(STAGING_DIR)/usr/lib/libfuse.so* $(TARGET_DIR)/usr/lib/
> > +# From libfuse3 README:
> > +# libfuse 3 is designed to be co-installable with libfuse 2. However, some
> > +# files will be installed by both libfuse 2 and libfuse 3
> > +# (e.g. /etc/fuse.conf, the udev and init scripts, and the mount.fuse(8) manpage).
> > +# These files should be taken from libfuse 3. The format/content is guaranteed
> > +# to remain backwards compatible with libfuse 2.
> > +#
> > +# The way we handle this is to let libfuse3 install as usual,
> > +# but libfuse has to be carefull to not overwrite any common files.
> > +# Also some files are named diferently (udev-rules),
> > +# but only the newer is needed.
> > +# To ensure this, we install in a temporary directory and
> > +# hand-pick the few unique files
> > +
> > +LIBFUSE_INSTALL_TARGET_OPTS = install DESTDIR=$(@D)/tmpinstall
> > +
> > +define LIBFUSE_INSTALL_TARGET_POST
> > +     cp -dpf $(@D)/tmpinstall/usr/lib/libfuse.so* $(TARGET_DIR)/usr/lib/
> > +endef
>
>  This is way too complicated.

But more robust, specifically if you rebuild single packages (I know
this is probably not officially supported).

>  Since we anyway have custom commands for target install, you can do it like this:
>
> ifeq ($(BR2_PACKAGE_LIBFUSE3),)
> define LIBFUSE_INSTALL_SUPPORT_FILES
>         cp -dpf $(STAGING_DIR)/usr/bin/fusermount $(TARGET_DIR)/usr/bin/
>         mkdir -p $(TARGET_DIR)/lib/udev/rules.d
>         cp $(STAGING_DIR)/lib/udev/rules.d/*-fuse.rules \
>                 $(TARGET_DIR)/lib/udev/rules.d
> endef
> endif
>
> define LIBFUSE_INSTALL_TARGET_CMDS
>         cp -dpf $(STAGING_DIR)/usr/lib/libfuse.so* $(TARGET_DIR)/usr/lib/
>         $(LIBFUSE_INSTALL_SUPPORT_FILES)
> endef

This only shifts the issue to the staging directory. installing both might
cause conflicts (like having 99-fuse.rules and 90-fuse-rules) in the future.
Using a temporary dir for atleast one of the packages cleanly goes around any
potential issues.
Its easier to maintain (especially per-package), if a file is renamed the build
will break vs. just causing issues when either both libfuse* packages
are installed or not.

(TBH I think a common infrastructure to install in a temp dir would be ideal)

Norbert

[Buildroot] [PATCH v2 4/4] package/libfuse: common files from libfuse3 are prefered

[Arnout Vandecappelle]

On 25/03/2019 22:12, Norbert Lange wrote:
> Am Mi., 20. M?rz 2019 um 23:55 Uhr schrieb Arnout Vandecappelle
> <arnout@mind.be>:
>>
>>
>>
>> On 15/03/2019 14:30, Norbert Lange wrote:
>>> if both libfuse and libfuse3 are co-installed,
>>> make sure libfuse3 will be build first and
>>
>>  Why does libfuse3 have to be installed first? Are there any files that are
>> installed by both after this patch? If so, please list them here. But then the
>> sentence below is not true :-)
> 
> I consider the common files part of libfuse3, so if you
> 'make libfuse' it should be ready-to-use.

 You mean when you do 'make libfuse' without selecting libfuse in .config?
That's a use case we definitely don't support (it works now, but it's not really
meant to work).

 Your patch makes sure that libfuse and libfuse3 install non-overlapping files,
so there is no need to add it to _DEPENDENCIES.

> The common files are just the udev-rules (which currently are named differently
[snip]
>>  However, this patch should also be squashed with the one adding libfuse3. Or,
>> if you want to separate things a little (possibly a good idea), then originally
>> libfuse3 should depend on !libfuse, and this patch would lift the dependency again.
> 
> K, primary though was that the first patch would be faster to accept.

 Well, the libfuse3 patch can't be applied as is because it conflicts with
libfuse... So to make it faster to accept, the patch adding libfuse3 should make
it depend on !libfuse.

[snip]
>>> -define LIBFUSE_INSTALL_TARGET_CMDS
>>> -     cp -dpf $(STAGING_DIR)/usr/bin/fusermount $(TARGET_DIR)/usr/bin/
>>> -     cp -dpf $(STAGING_DIR)/usr/lib/libfuse.so* $(TARGET_DIR)/usr/lib/
>>> +# From libfuse3 README:
>>> +# libfuse 3 is designed to be co-installable with libfuse 2. However, some
>>> +# files will be installed by both libfuse 2 and libfuse 3
>>> +# (e.g. /etc/fuse.conf, the udev and init scripts, and the mount.fuse(8) manpage).
>>> +# These files should be taken from libfuse 3. The format/content is guaranteed
>>> +# to remain backwards compatible with libfuse 2.
>>> +#
>>> +# The way we handle this is to let libfuse3 install as usual,
>>> +# but libfuse has to be carefull to not overwrite any common files.
>>> +# Also some files are named diferently (udev-rules),
>>> +# but only the newer is needed.
>>> +# To ensure this, we install in a temporary directory and
>>> +# hand-pick the few unique files
>>> +
>>> +LIBFUSE_INSTALL_TARGET_OPTS = install DESTDIR=$(@D)/tmpinstall
>>> +
>>> +define LIBFUSE_INSTALL_TARGET_POST
>>> +     cp -dpf $(@D)/tmpinstall/usr/lib/libfuse.so* $(TARGET_DIR)/usr/lib/
>>> +endef
>>
>>  This is way too complicated.
> 
> But more robust, specifically if you rebuild single packages (I know
> this is probably not officially supported).

 It is supported, but I don't see how it creates a problem. There could be a
problem if libfuse3 would call it something that matches libfuse.so*, but it
doesn't (and it never will).

>>  Since we anyway have custom commands for target install, you can do it like this:
>>
>> ifeq ($(BR2_PACKAGE_LIBFUSE3),)
>> define LIBFUSE_INSTALL_SUPPORT_FILES
>>         cp -dpf $(STAGING_DIR)/usr/bin/fusermount $(TARGET_DIR)/usr/bin/
>>         mkdir -p $(TARGET_DIR)/lib/udev/rules.d
>>         cp $(STAGING_DIR)/lib/udev/rules.d/*-fuse.rules \
>>                 $(TARGET_DIR)/lib/udev/rules.d
>> endef
>> endif
>>
>> define LIBFUSE_INSTALL_TARGET_CMDS
>>         cp -dpf $(STAGING_DIR)/usr/lib/libfuse.so* $(TARGET_DIR)/usr/lib/
>>         $(LIBFUSE_INSTALL_SUPPORT_FILES)
>> endef
> 
> This only shifts the issue to the staging directory. installing both might
> cause conflicts (like having 99-fuse.rules and 90-fuse-rules) in the future.

 We don't care about staging, in fact I think we would really prefer to have
everything there in case both packages are installed.

> Using a temporary dir for atleast one of the packages cleanly goes around any
> potential issues.
> Its easier to maintain (especially per-package), if a file is renamed the build
> will break vs. just causing issues when either both libfuse* packages
> are installed or not.

 I don't see how this installating in tmpinstall handles that.


> (TBH I think a common infrastructure to install in a temp dir would be ideal)

 You do have a point there. I think it even was one of the options we considered
in our extensive discussion of PPD in Prague back in 2017 [1]. I don't remember
why we didn't stick to that possibility. Perhaps it was because of all the
packages with custom staging/target install commands that would need to be
adapted. But possibly that was before we thought of the trick to rewrite
STAGING_DIR and TARGET_DIR during the installation steps.

 Maybe indeed this approach would be a good follow-up of PPD, and it would be a
giant step in the direction of the target-from-staging approach that Yann would
like to see.

 That said, for the time being, I'd prefer new stuff to stick to our existing
approach of copying from staging to target, except for cases where it is really
difficult to do differently (e.g. the qt5 stuff). And we can use that as a
learning path to generalise to all packages.

 Regards,
 Arnout

[1] https://elinux.org/Buildroot:DeveloperDaysELCE2017

[Buildroot] [PATCH v2 4/4] package/libfuse: common files from libfuse3 are prefered

[Norbert Lange]

Am Di., 26. M?rz 2019 um 10:50 Uhr schrieb Arnout Vandecappelle
<arnout@mind.be>:
>
>
>
> On 25/03/2019 22:12, Norbert Lange wrote:
> > Am Mi., 20. M?rz 2019 um 23:55 Uhr schrieb Arnout Vandecappelle
> > <arnout@mind.be>:
> >>
> >>
> >>
> >> On 15/03/2019 14:30, Norbert Lange wrote:
> >>> if both libfuse and libfuse3 are co-installed,
> >>> make sure libfuse3 will be build first and
> >>
> >>  Why does libfuse3 have to be installed first? Are there any files that are
> >> installed by both after this patch? If so, please list them here. But then the
> >> sentence below is not true :-)
> >
> > I consider the common files part of libfuse3, so if you
> > 'make libfuse' it should be ready-to-use.
>
>  You mean when you do 'make libfuse' without selecting libfuse in .config?
> That's a use case we definitely don't support (it works now, but it's not really
> meant to work).

well, I do that sometimes (running make without config'ed package),
but also 'make libfuse-dirclean; make libfuse' is interesting, for example
after updating the package version (full BR rebuild takes ages).

What happens is that staging/target directory might get littered with
multiple versions of files.

>  Your patch makes sure that libfuse and libfuse3 install non-overlapping files,
> so there is no need to add it to _DEPENDENCIES.

well, if both are configured, then libfuse will need files that are
installed from libfuse3
(udev rule, and the helper executable).
if I dirclean libfuse* and run 'make libfuse' I would like those files
to be installed aswell

>
> > The common files are just the udev-rules (which currently are named differently
> [snip]
> >>  However, this patch should also be squashed with the one adding libfuse3. Or,
> >> if you want to separate things a little (possibly a good idea), then originally
> >> libfuse3 should depend on !libfuse, and this patch would lift the dependency again.
> >
> > K, primary though was that the first patch would be faster to accept.
>
>  Well, the libfuse3 patch can't be applied as is because it conflicts with
> libfuse... So to make it faster to accept, the patch adding libfuse3 should make
> it depend on !libfuse.

Think I got that, yes.


>
> [snip]
> >>> -define LIBFUSE_INSTALL_TARGET_CMDS
> >>> -     cp -dpf $(STAGING_DIR)/usr/bin/fusermount $(TARGET_DIR)/usr/bin/
> >>> -     cp -dpf $(STAGING_DIR)/usr/lib/libfuse.so* $(TARGET_DIR)/usr/lib/
> >>> +# From libfuse3 README:
> >>> +# libfuse 3 is designed to be co-installable with libfuse 2. However, some
> >>> +# files will be installed by both libfuse 2 and libfuse 3
> >>> +# (e.g. /etc/fuse.conf, the udev and init scripts, and the mount.fuse(8) manpage).
> >>> +# These files should be taken from libfuse 3. The format/content is guaranteed
> >>> +# to remain backwards compatible with libfuse 2.
> >>> +#
> >>> +# The way we handle this is to let libfuse3 install as usual,
> >>> +# but libfuse has to be carefull to not overwrite any common files.
> >>> +# Also some files are named diferently (udev-rules),
> >>> +# but only the newer is needed.
> >>> +# To ensure this, we install in a temporary directory and
> >>> +# hand-pick the few unique files
> >>> +
> >>> +LIBFUSE_INSTALL_TARGET_OPTS = install DESTDIR=$(@D)/tmpinstall
> >>> +
> >>> +define LIBFUSE_INSTALL_TARGET_POST
> >>> +     cp -dpf $(@D)/tmpinstall/usr/lib/libfuse.so* $(TARGET_DIR)/usr/lib/
> >>> +endef
> >>
> >>  This is way too complicated.
> >
> > But more robust, specifically if you rebuild single packages (I know
> > this is probably not officially supported).
>
>  It is supported, but I don't see how it creates a problem. There could be a
> problem if libfuse3 would call it something that matches libfuse.so*, but it
> doesn't (and it never will).

Aslong we dont install /etc/fuse.conf =)

>
> >>  Since we anyway have custom commands for target install, you can do it like this:
> >>
> >> ifeq ($(BR2_PACKAGE_LIBFUSE3),)
> >> define LIBFUSE_INSTALL_SUPPORT_FILES
> >>         cp -dpf $(STAGING_DIR)/usr/bin/fusermount $(TARGET_DIR)/usr/bin/
> >>         mkdir -p $(TARGET_DIR)/lib/udev/rules.d
> >>         cp $(STAGING_DIR)/lib/udev/rules.d/*-fuse.rules \
> >>                 $(TARGET_DIR)/lib/udev/rules.d
> >> endef
> >> endif
> >>
> >> define LIBFUSE_INSTALL_TARGET_CMDS
> >>         cp -dpf $(STAGING_DIR)/usr/lib/libfuse.so* $(TARGET_DIR)/usr/lib/
> >>         $(LIBFUSE_INSTALL_SUPPORT_FILES)
> >> endef
> >
> > This only shifts the issue to the staging directory. installing both might
> > cause conflicts (like having 99-fuse.rules and 90-fuse-rules) in the future.
>
>  We don't care about staging, in fact I think we would really prefer to have
> everything there in case both packages are installed.

And you have everything there, but the target should not contain duplcate files
(even semantically duplicates like fuse.udev and fuse3.udev).

>
> > Using a temporary dir for atleast one of the packages cleanly goes around any
> > potential issues.
> > Its easier to maintain (especially per-package), if a file is renamed the build
> > will break vs. just causing issues when either both libfuse* packages
> > are installed or not.
>
>  I don't see how this installating in tmpinstall handles that.

if you install into a shared staging directory, your glob patterns
might match more than they should (old versions) and (in general,
unless you make sure this does not happen) packages could override
other packages files

tmpinstall would isolate that (I should make sure to delete the dir
before installation)

>
>
> > (TBH I think a common infrastructure to install in a temp dir would be ideal)
>
>  You do have a point there. I think it even was one of the options we considered
> in our extensive discussion of PPD in Prague back in 2017 [1]. I don't remember
> why we didn't stick to that possibility. Perhaps it was because of all the
> packages with custom staging/target install commands that would need to be
> adapted. But possibly that was before we thought of the trick to rewrite
> STAGING_DIR and TARGET_DIR during the installation steps.

Could start with a whitelist-flag for testing.

>  Maybe indeed this approach would be a good follow-up of PPD, and it would be a
> giant step in the direction of the target-from-staging approach that Yann would
> like to see.

Not sure if target-from-staging aint the opposite of what I mean...

>  That said, for the time being, I'd prefer new stuff to stick to our existing
> approach of copying from staging to target, except for cases where it is really
> difficult to do differently (e.g. the qt5 stuff). And we can use that as a
> learning path to generalise to all packages.
>
>  Regards,
>  Arnout
>
> [1] https://elinux.org/Buildroot:DeveloperDaysELCE2017

Norbert

[Buildroot] [PATCH v2 4/4] package/libfuse: common files from libfuse3 are prefered

[Arnout Vandecappelle]

On 26/03/2019 18:45, Norbert Lange wrote:
> Am Di., 26. M?rz 2019 um 10:50 Uhr schrieb Arnout Vandecappelle
> <arnout@mind.be>:
>>
>>
>>
>> On 25/03/2019 22:12, Norbert Lange wrote:
>>> Am Mi., 20. M?rz 2019 um 23:55 Uhr schrieb Arnout Vandecappelle
>>> <arnout@mind.be>:
>>>>
>>>>
>>>>
>>>> On 15/03/2019 14:30, Norbert Lange wrote:
>>>>> if both libfuse and libfuse3 are co-installed,
>>>>> make sure libfuse3 will be build first and
>>>>
>>>>  Why does libfuse3 have to be installed first? Are there any files that are
>>>> installed by both after this patch? If so, please list them here. But then the
>>>> sentence below is not true :-)
>>>
>>> I consider the common files part of libfuse3, so if you
>>> 'make libfuse' it should be ready-to-use.
>>
>>  You mean when you do 'make libfuse' without selecting libfuse in .config?
>> That's a use case we definitely don't support (it works now, but it's not really
>> meant to work).
> 
> well, I do that sometimes (running make without config'ed package),

 So do I. And it works most of the time. But not always, so it's not "officially
supported".

> but also 'make libfuse-dirclean; make libfuse' is interesting, for example
> after updating the package version (full BR rebuild takes ages).

 Yes, but that will not rebuild libfuse3 anyway, so whether or not it's a
dependency doesn't make a difference.


> What happens is that staging/target directory might get littered with
> multiple versions of files.

 Indeed, if you update the version and there is some file that is no longer
there, the old file doesn't get removed. Similarly, if you first build with only
libfuse enabled, then enable libfuse3 and rebuild, the libfuse udev stuff will
still be there in target. But neither of these issues is affected by making
libfuse3 a dependency of libfuse.


>>  Your patch makes sure that libfuse and libfuse3 install non-overlapping files,
>> so there is no need to add it to _DEPENDENCIES.
> 
> well, if both are configured, then libfuse will need files that are
> installed from libfuse3
> (udev rule, and the helper executable).
> if I dirclean libfuse* and run 'make libfuse' I would like those files
> to be installed aswell

 I see, you have a good point there. However, that classifies under "runtime
dependency" and traditionally we don't add make dependencies for runtime
dependencies. There is no clear reason for that. It has one advantage: running
'make libfuse' goes a little bit faster because libfuse3 doesn't need to be
built. It may also break a circular dependency (though I doubt there are many
instances of that). But neither of these arguments is very strong.

 Still, adding runtime dependencies to _DEPENDENCIES is a change in Buildroot
philosophy so you can't sneak it in in a random patch :-)


[snip]
>>> But more robust, specifically if you rebuild single packages (I know
>>> this is probably not officially supported).
>>
>>  It is supported, but I don't see how it creates a problem. There could be a
>> problem if libfuse3 would call it something that matches libfuse.so*, but it
>> doesn't (and it never will).
> 
> Aslong we dont install /etc/fuse.conf =)

 Indeed. But then, IIUC, the fuse.conf of libfuse3 should be used. So libfuse3
should depend on libfuse, no? That was actually my original thought when I saw
this dependency on libfuse3, but then I found that it was in fact entirely
redundant.


>>>>  Since we anyway have custom commands for target install, you can do it like this:
>>>>
>>>> ifeq ($(BR2_PACKAGE_LIBFUSE3),)
>>>> define LIBFUSE_INSTALL_SUPPORT_FILES
>>>>         cp -dpf $(STAGING_DIR)/usr/bin/fusermount $(TARGET_DIR)/usr/bin/
>>>>         mkdir -p $(TARGET_DIR)/lib/udev/rules.d
>>>>         cp $(STAGING_DIR)/lib/udev/rules.d/*-fuse.rules \
>>>>                 $(TARGET_DIR)/lib/udev/rules.d
>>>> endef
>>>> endif
>>>>
>>>> define LIBFUSE_INSTALL_TARGET_CMDS
>>>>         cp -dpf $(STAGING_DIR)/usr/lib/libfuse.so* $(TARGET_DIR)/usr/lib/
>>>>         $(LIBFUSE_INSTALL_SUPPORT_FILES)
>>>> endef
>>>
>>> This only shifts the issue to the staging directory. installing both might
>>> cause conflicts (like having 99-fuse.rules and 90-fuse-rules) in the future.
>>
>>  We don't care about staging, in fact I think we would really prefer to have
>> everything there in case both packages are installed.
> 
> And you have everything there, but the target should not contain duplcate files
> (even semantically duplicates like fuse.udev and fuse3.udev).

 Target, yes - but that is covered by my proposal as well. Staging, not so much.
I wouldn't mind having both fuse.udev and fuse3.udev in staging.


>>> Using a temporary dir for atleast one of the packages cleanly goes around any
>>> potential issues.
>>> Its easier to maintain (especially per-package), if a file is renamed the build
>>> will break vs. just causing issues when either both libfuse* packages
>>> are installed or not.
>>
>>  I don't see how this installating in tmpinstall handles that.
> 
> if you install into a shared staging directory, your glob patterns
> might match more than they should (old versions) and (in general,
> unless you make sure this does not happen) packages could override
> other packages files
> 
> tmpinstall would isolate that

 Indeed. Which is in fact another argument why this kind of tmpinstall may be a
good idea in general. (The fact that all package managers I know do this might
be a hint :-).

> (I should make sure to delete the dir
> before installation)

 That doesn't make any difference: whatever is in the tmpinstall has already
been copied to staging/target in the previous run, so if you remove tmpinstall,
it's still there in staging/target.


>>> (TBH I think a common infrastructure to install in a temp dir would be ideal)
>>
>>  You do have a point there. I think it even was one of the options we considered
>> in our extensive discussion of PPD in Prague back in 2017 [1]. I don't remember
>> why we didn't stick to that possibility. Perhaps it was because of all the
>> packages with custom staging/target install commands that would need to be
>> adapted. But possibly that was before we thought of the trick to rewrite
>> STAGING_DIR and TARGET_DIR during the installation steps.
> 
> Could start with a whitelist-flag for testing.

 Or with just a few packages. libfuse, perhaps :-)


>>  Maybe indeed this approach would be a good follow-up of PPD, and it would be a
>> giant step in the direction of the target-from-staging approach that Yann would
>> like to see.
> 
> Not sure if target-from-staging aint the opposite of what I mean...

 The target-from-staging concept is neatly served by doing
install-into-tmpinstall and then rsync-tmpinstall-to-staging and
selective-rsync-tmpinstall-to-target. The key idea of target-from-staging is
that everything in target is also in staging (which is not currently the case),
with ancillary that package install should be done only once instead of twice.


>>  That said, for the time being, I'd prefer new stuff to stick to our existing
>> approach of copying from staging to target, except for cases where it is really
>> difficult to do differently (e.g. the qt5 stuff). And we can use that as a
>> learning path to generalise to all packages.

 So that's what I said this morning, but now I'm not so sure :-)

 If we have two fronts that would like to use the tmpinstall concept, then maybe
we should do it in both at the same time, to maximize our learning. And since
this would be a kind of prototype of the eventual tmpinstall for all packages,
we don't need to bikeshed too much on the exact details (tmpinstall?
tmp-target-install? In $($(PKG)_BUILDDIR) or in $(BUILDDIR)/tmpinstall/$(pkg)? etc.)

 So, if you stick with the tmpinstall approach, I'm not going to oppose it any more.

 Regards,
 Arnout

[Buildroot] [PATCH v2 4/4] package/libfuse: common files from libfuse3 are prefered

[Norbert Lange]

Am Di., 26. M?rz 2019 um 21:19 Uhr schrieb Arnout Vandecappelle
<arnout@mind.be>:
>
>
>
> On 26/03/2019 18:45, Norbert Lange wrote:
> > Am Di., 26. M?rz 2019 um 10:50 Uhr schrieb Arnout Vandecappelle
> > <arnout@mind.be>:
> >>
> >>
> >>
> >> On 25/03/2019 22:12, Norbert Lange wrote:
> >>> Am Mi., 20. M?rz 2019 um 23:55 Uhr schrieb Arnout Vandecappelle
> >>> <arnout@mind.be>:
> >>>>
> >>>>
> >>>>
> >>>> On 15/03/2019 14:30, Norbert Lange wrote:
> >>>>> if both libfuse and libfuse3 are co-installed,
> >>>>> make sure libfuse3 will be build first and
> >>>>
> >>>>  Why does libfuse3 have to be installed first? Are there any files that are
> >>>> installed by both after this patch? If so, please list them here. But then the
> >>>> sentence below is not true :-)
> >>>
> >>> I consider the common files part of libfuse3, so if you
> >>> 'make libfuse' it should be ready-to-use.
> >>
> >>  You mean when you do 'make libfuse' without selecting libfuse in .config?
> >> That's a use case we definitely don't support (it works now, but it's not really
> >> meant to work).
> >
> > well, I do that sometimes (running make without config'ed package),
>
>  So do I. And it works most of the time. But not always, so it's not "officially
> supported".
>
> > but also 'make libfuse-dirclean; make libfuse' is interesting, for example
> > after updating the package version (full BR rebuild takes ages).
>
>  Yes, but that will not rebuild libfuse3 anyway, so whether or not it's a
> dependency doesn't make a difference.
>
>
> > What happens is that staging/target directory might get littered with
> > multiple versions of files.
>
>  Indeed, if you update the version and there is some file that is no longer
> there, the old file doesn't get removed. Similarly, if you first build with only
> libfuse enabled, then enable libfuse3 and rebuild, the libfuse udev stuff will
> still be there in target. But neither of these issues is affected by making
> libfuse3 a dependency of libfuse.
>
>
> >>  Your patch makes sure that libfuse and libfuse3 install non-overlapping files,
> >> so there is no need to add it to _DEPENDENCIES.
> >
> > well, if both are configured, then libfuse will need files that are
> > installed from libfuse3
> > (udev rule, and the helper executable).
> > if I dirclean libfuse* and run 'make libfuse' I would like those files
> > to be installed aswell
>
>  I see, you have a good point there. However, that classifies under "runtime
> dependency" and traditionally we don't add make dependencies for runtime
> dependencies. There is no clear reason for that. It has one advantage: running
> 'make libfuse' goes a little bit faster because libfuse3 doesn't need to be
> built. It may also break a circular dependency (though I doubt there are many
> instances of that). But neither of these arguments is very strong.
>
>  Still, adding runtime dependencies to _DEPENDENCIES is a change in Buildroot
> philosophy so you can't sneak it in in a random patch :-)

Yes, its not needed for the build. granted, its easy to get that mixed
up expecially
if you dont have a way to specify runtime dependencies.

> [snip]
> >>> But more robust, specifically if you rebuild single packages (I know
> >>> this is probably not officially supported).
> >>
> >>  It is supported, but I don't see how it creates a problem. There could be a
> >> problem if libfuse3 would call it something that matches libfuse.so*, but it
> >> doesn't (and it never will).
> >
> > Aslong we dont install /etc/fuse.conf =)
>
>  Indeed. But then, IIUC, the fuse.conf of libfuse3 should be used. So libfuse3
> should depend on libfuse, no? That was actually my original thought when I saw
> this dependency on libfuse3, but then I found that it was in fact entirely
> redundant.

err. why would libfuse3 depend on libfuse, so that the files get overwritten
in the right order? To me the latter is a (crude) implementation detail.

>
>
> >>>>  Since we anyway have custom commands for target install, you can do it like this:
> >>>>
> >>>> ifeq ($(BR2_PACKAGE_LIBFUSE3),)
> >>>> define LIBFUSE_INSTALL_SUPPORT_FILES
> >>>>         cp -dpf $(STAGING_DIR)/usr/bin/fusermount $(TARGET_DIR)/usr/bin/
> >>>>         mkdir -p $(TARGET_DIR)/lib/udev/rules.d
> >>>>         cp $(STAGING_DIR)/lib/udev/rules.d/*-fuse.rules \
> >>>>                 $(TARGET_DIR)/lib/udev/rules.d
> >>>> endef
> >>>> endif
> >>>>
> >>>> define LIBFUSE_INSTALL_TARGET_CMDS
> >>>>         cp -dpf $(STAGING_DIR)/usr/lib/libfuse.so* $(TARGET_DIR)/usr/lib/
> >>>>         $(LIBFUSE_INSTALL_SUPPORT_FILES)
> >>>> endef
> >>>
> >>> This only shifts the issue to the staging directory. installing both might
> >>> cause conflicts (like having 99-fuse.rules and 90-fuse-rules) in the future.
> >>
> >>  We don't care about staging, in fact I think we would really prefer to have
> >> everything there in case both packages are installed.
> >
> > And you have everything there, but the target should not contain duplcate files
> > (even semantically duplicates like fuse.udev and fuse3.udev).
>
>  Target, yes - but that is covered by my proposal as well. Staging, not so much.
> I wouldn't mind having both fuse.udev and fuse3.udev in staging.

Me neither.

>
>
> >>> Using a temporary dir for atleast one of the packages cleanly goes around any
> >>> potential issues.
> >>> Its easier to maintain (especially per-package), if a file is renamed the build
> >>> will break vs. just causing issues when either both libfuse* packages
> >>> are installed or not.
> >>
> >>  I don't see how this installating in tmpinstall handles that.
> >
> > if you install into a shared staging directory, your glob patterns
> > might match more than they should (old versions) and (in general,
> > unless you make sure this does not happen) packages could override
> > other packages files
> >
> > tmpinstall would isolate that
>
>  Indeed. Which is in fact another argument why this kind of tmpinstall may be a
> good idea in general. (The fact that all package managers I know do this might
> be a hint :-).
>
> > (I should make sure to delete the dir
> > before installation)
>
>  That doesn't make any difference: whatever is in the tmpinstall has already
> been copied to staging/target in the previous run, so if you remove tmpinstall,
> it's still there in staging/target.

Recreating the target installation cleanly would require that, that
this is not possible
for other reasons so far is another thing.

> >>> (TBH I think a common infrastructure to install in a temp dir would be ideal)
> >>
> >>  You do have a point there. I think it even was one of the options we considered
> >> in our extensive discussion of PPD in Prague back in 2017 [1]. I don't remember
> >> why we didn't stick to that possibility. Perhaps it was because of all the
> >> packages with custom staging/target install commands that would need to be
> >> adapted. But possibly that was before we thought of the trick to rewrite
> >> STAGING_DIR and TARGET_DIR during the installation steps.
> >
> > Could start with a whitelist-flag for testing.
>
>  Or with just a few packages. libfuse, perhaps :-)

The idea I have in mind would need some touching of files
outside the package. But yeah, same difference.

> >>  Maybe indeed this approach would be a good follow-up of PPD, and it would be a
> >> giant step in the direction of the target-from-staging approach that Yann would
> >> like to see.
> >
> > Not sure if target-from-staging aint the opposite of what I mean...
>
>  The target-from-staging concept is neatly served by doing
> install-into-tmpinstall and then rsync-tmpinstall-to-staging and
> selective-rsync-tmpinstall-to-target. The key idea of target-from-staging is
> that everything in target is also in staging (which is not currently the case),
> with ancillary that package install should be done only once instead of twice.
>
>
> >>  That said, for the time being, I'd prefer new stuff to stick to our existing
> >> approach of copying from staging to target, except for cases where it is really
> >> difficult to do differently (e.g. the qt5 stuff). And we can use that as a
> >> learning path to generalise to all packages.
>
>  So that's what I said this morning, but now I'm not so sure :-)
>
>  If we have two fronts that would like to use the tmpinstall concept, then maybe
> we should do it in both at the same time, to maximize our learning. And since
> this would be a kind of prototype of the eventual tmpinstall for all packages,
> we don't need to bikeshed too much on the exact details (tmpinstall?
> tmp-target-install? In $($(PKG)_BUILDDIR) or in $(BUILDDIR)/tmpinstall/$(pkg)? etc.)

You are giving me just more ideas.
What if after building you end up with those
$(BUILDDIR)/tmpinstall/$(pkg) directories,
be free to delete all space gobbling  $($(PKG)_BUILDDIR) dirs, and be
able to reinstall target
or even staging purely from these "deployment packages"...

if you are familiar with debhelper (with multiple binary packages resulting),
the install step installs into debian/tmp then uses textfile to define
what-goes-where.
You could drop some files in the package directory, like an
{target,staging}.install file,
having relative paths/globs to mark the stuff that should be
installed [1].  Or the same stuff for links [2].

The upside here is, you could copy those files into/nextto the
"deployment packages",
so those could be complete and self-sufficient for later Installation.
(of course there might be always outliers that are too complex...)

[1] https://manpages.debian.org/testing/debhelper/dh_install.1.en.html
[2] https://manpages.debian.org/testing/debhelper/dh_link.1.en.html

>  So, if you stick with the tmpinstall approach, I'm not going to oppose it any more.

Id like to finish the package (after addressing the open points),
then I may try to brew up something.

Norbert

PS. I glanced at the qt thread (quite hard to keep track of the
discussion entering in the middle),
but it seems that is completely orthogonal and more about hacking into
the qmake environment,
not a post-processing step.

[Buildroot] [PATCH v2 4/4] package/libfuse: common files from libfuse3 are prefered

[Arnout Vandecappelle]

On 28/03/2019 21:28, Norbert Lange wrote:
> Am Di., 26. M?rz 2019 um 21:19 Uhr schrieb Arnout Vandecappelle
> <arnout@mind.be>:
[snip]
>>>>> But more robust, specifically if you rebuild single packages (I know
>>>>> this is probably not officially supported).
>>>>
>>>>  It is supported, but I don't see how it creates a problem. There could be a
>>>> problem if libfuse3 would call it something that matches libfuse.so*, but it
>>>> doesn't (and it never will).
>>>
>>> Aslong we dont install /etc/fuse.conf =)
>>
>>  Indeed. But then, IIUC, the fuse.conf of libfuse3 should be used. So libfuse3
>> should depend on libfuse, no? That was actually my original thought when I saw
>> this dependency on libfuse3, but then I found that it was in fact entirely
>> redundant.
> 
> err. why would libfuse3 depend on libfuse, so that the files get overwritten
> in the right order? To me the latter is a (crude) implementation detail.

 When there are two packages that create the same file, we need some kind of
dependency to make sure the right version ends up in the target rootfs. This can
be a Config.in dependency (e.g. for python/python3: only one of them can be
selected), or a .mk adaptation that filters the to-be-installed files based on
whether the other package is selected (= what you've done here), or adding it to
_DEPENDENCIES in some order. In the latter case, usually the one that is
supposed to "win" will add the one that is supposed to "loose" to its
_DEPENDENCIES. The (only?) exception is busybox, because the busybox install
script will avoid overwriting files that already exist.

[snip]
>>  If we have two fronts that would like to use the tmpinstall concept, then maybe
>> we should do it in both at the same time, to maximize our learning. And since
>> this would be a kind of prototype of the eventual tmpinstall for all packages,
>> we don't need to bikeshed too much on the exact details (tmpinstall?
>> tmp-target-install? In $($(PKG)_BUILDDIR) or in $(BUILDDIR)/tmpinstall/$(pkg)? etc.)
> 
> You are giving me just more ideas.
> What if after building you end up with those
> $(BUILDDIR)/tmpinstall/$(pkg) directories,
> be free to delete all space gobbling  $($(PKG)_BUILDDIR) dirs, and be
> able to reinstall target
> or even staging purely from these "deployment packages"...

 Yes, that would be nice. It would also require moving the stamp files to the
tmpinstall directory, though. But then, it is no longer to delete the build dir
to force a clean build. So, more discussion needed...


> if you are familiar with debhelper (with multiple binary packages resulting),
> the install step installs into debian/tmp then uses textfile to define
> what-goes-where.

 Yes, that's what all distro packagers that I know are doing. Including
OpenEmbedded and OpenWrt.

> You could drop some files in the package directory, like an
> {target,staging}.install file,
> having relative paths/globs to mark the stuff that should be
> installed [1].  Or the same stuff for links [2].
> 
> The upside here is, you could copy those files into/nextto the
> "deployment packages",
> so those could be complete and self-sufficient for later Installation.
> (of course there might be always outliers that are too complex...)
> 
> [1] https://manpages.debian.org/testing/debhelper/dh_install.1.en.html
> [2] https://manpages.debian.org/testing/debhelper/dh_link.1.en.html
> 
>>  So, if you stick with the tmpinstall approach, I'm not going to oppose it any more.
> 
> Id like to finish the package (after addressing the open points),
> then I may try to brew up something.
> 
> Norbert
> 
> PS. I glanced at the qt thread (quite hard to keep track of the
> discussion entering in the middle),
> but it seems that is completely orthogonal and more about hacking into
> the qmake environment,
> not a post-processing step.

 The similarity is that the qt packages also can't do a target install with a
simple 'make -C $(@D) install DESTDIR=$(TARGET_DIR)'. And the solution chosen is
to install into a temporary install directory and from there copy to the real
target. Which is exactly what you want to do for libfuse/libfuse3.

 Regards,
 Arnout