[Buildroot] [PATCH 0/8] package/opkg-utils: add target build

[Buildroot] [PATCH 0/8] package/opkg-utils: add target build

[Ryan Barnett]

A first revision of adding opkg-utils as a target package was proposed
in the following patch:

  https://patchwork.ozlabs.org/project/buildroot/patch/20210126192605.20002-1-matthew.weber at rockwellcollins.com/

Based on feedback received from Yann, it uncovered some issues with the
current host package build for opkg-utils.

This patch series addresses the following items:

 * Remove build step since all it does is generate man pages for a
   single command (Patch 1)
 * Incorporated accepted patch for splitting off the installation of
   docs and utils (Patch 2)
 * Add missing host dependencies for host-opkg-utils which are not
   manditory packages of for buildroot (Patches 3-6)
 * Add target package for opkg-utils (Patch 7)

Thanks,
-Ryan

CC: Yann E. MORIN <yann.morin.1998@free.fr>

Ryan Barnett (8):
  package/opkg-utils: remove build step
  package/opkg-utils: install only utility scripts
  package/diffutils: add host package
  package/findutils: add host package
  package/grep: add host package
  package/opkg-utils: add missing dependencies for host build
  package/opkg-utils: add opkg-utils as target pkg
  DEVELOPERS: Add Ryan Barnett for opkg and opkg-utils

 DEVELOPERS                                    |  2 +
 package/Config.in                             |  1 +
 package/diffutils/diffutils.mk                |  1 +
 package/findutils/findutils.mk                |  1 +
 package/grep/grep.mk                          |  1 +
 ...-seperate-manpages-and-utils-install.patch | 50 +++++++++++++++++++
 package/opkg-utils/Config.in                  | 33 ++++++++++++
 package/opkg-utils/opkg-utils.mk              | 31 ++++++++++--
 8 files changed, 116 insertions(+), 4 deletions(-)
 create mode 100644 package/opkg-utils/0001-Makefile-seperate-manpages-and-utils-install.patch
 create mode 100644 package/opkg-utils/Config.in

--
2.17.1

[Buildroot] [PATCH 1/8] package/opkg-utils: remove build step

[Ryan Barnett]

opkg-utils is a package that only provides bash and python scripts.
Upon further inspection of the Makefile for the package, invoking
`make` is equivalent to `make install`. The makes the build step
unnecessary so remove it.

Signed-off-by: Ryan Barnett <ryan.barnett@rockwellcollins.com>
---
 package/opkg-utils/opkg-utils.mk | 4 ----
 1 file changed, 4 deletions(-)

diff --git a/package/opkg-utils/opkg-utils.mk b/package/opkg-utils/opkg-utils.mk
index a94a4cd5cd..47fc4c63b4 100644
--- a/package/opkg-utils/opkg-utils.mk
+++ b/package/opkg-utils/opkg-utils.mk
@@ -12,10 +12,6 @@ OPKG_UTILS_LICENSE_FILES = COPYING
 
 HOST_OPKG_UTILS_DEPENDENCIES += $(BR2_PYTHON3_HOST_DEPENDENCY)
 
-define HOST_OPKG_UTILS_BUILD_CMDS
-	$(MAKE) -C $(@D) $(HOST_CONFIGURE_OPTS)
-endef
-
 define HOST_OPKG_UTILS_INSTALL_CMDS
 	$(MAKE) -C $(@D) PREFIX=$(HOST_DIR) install
 endef
-- 
2.17.1

[Buildroot] [PATCH 2/8] package/opkg-utils: install only utility scripts

[Ryan Barnett]

When `make install` is run to install the opkg-utils scripts, it also
invokes building of the man page for opkg-build. The generation of the
man page requires `pod2man` executable which is a part of perl.

Since buildroot does not support man pages in the host directory,
patch the opkg-utils Makefile to separate the installation of man
pages and utility scripts.

With the options to install man pages and utils separately, only
install the opkg-utils scripts.

Signed-off-by: Ryan Barnett <ryan.barnett@rockwellcollins.com>
---
 ...-seperate-manpages-and-utils-install.patch | 50 +++++++++++++++++++
 package/opkg-utils/opkg-utils.mk              |  2 +-
 2 files changed, 51 insertions(+), 1 deletion(-)
 create mode 100644 package/opkg-utils/0001-Makefile-seperate-manpages-and-utils-install.patch

diff --git a/package/opkg-utils/0001-Makefile-seperate-manpages-and-utils-install.patch b/package/opkg-utils/0001-Makefile-seperate-manpages-and-utils-install.patch
new file mode 100644
index 0000000000..140187ca3a
--- /dev/null
+++ b/package/opkg-utils/0001-Makefile-seperate-manpages-and-utils-install.patch
@@ -0,0 +1,50 @@
+From 74ccbee0f798822041dba5c6564df62a0c60d86b Mon Sep 17 00:00:00 2001
+From: Ryan Barnett <ryanbarnett3@gmail.com>
+Date: Mon, 22 Feb 2021 07:09:34 -0600
+Subject: Makefile: separate manpages and utils install
+
+The installation of opkg-build man page introduces a host dependency
+on perl for the pod2man package to generate the man page.
+
+To allow the opkg-utils scripts to be installed separately from the
+manpages, break apart the install step into two install steps:
+install-utils and install-docs
+
+Backported from: 74ccbee0f798822041dba5c6564df62a0c60d86b
+
+CC: Christian Hermann <mail@hermannch.dev>
+Signed-off-by: Ryan Barnett <ryanbarnett3@gmail.com>
+Signed-off-by: Alex Stewart <alex.stewart@ni.com>
+Signed-off-by: Ryan Barnett <ryan.barnett@rockwellcollins.com>
+---
+ Makefile | 8 ++++++--
+ 1 file changed, 6 insertions(+), 2 deletions(-)
+
+diff --git a/Makefile b/Makefile
+index 4049654..fe96d5a 100644
+--- a/Makefile
++++ b/Makefile
+@@ -27,9 +27,11 @@ mandir ?= $(PREFIX)/man
+ 
+ all: $(UTILS) $(MANPAGES)
+ 
+-install: all
++install-utils: $(UTILS)
+ 	install -d $(DESTDIR)$(bindir)
+ 	install -m 755 $(UTILS) $(DESTDIR)$(bindir)
++
++install-docs: $(MANPAGES)
+ 	install -d $(DESTDIR)$(mandir)
+ 	for m in $(MANPAGES); \
+ 	do \
+@@ -37,4 +39,6 @@ install: all
+ 		install -m 644 "$$m" $(DESTDIR)$(mandir)/man$${m##*.}; \
+ 	done
+ 
+-.PHONY: install all
++install: install-utils install-docs
++
++.PHONY: install install-utils install-docs all
+-- 
+cgit v1.2.2-1-g5e49
+
diff --git a/package/opkg-utils/opkg-utils.mk b/package/opkg-utils/opkg-utils.mk
index 47fc4c63b4..a250b4ef9f 100644
--- a/package/opkg-utils/opkg-utils.mk
+++ b/package/opkg-utils/opkg-utils.mk
@@ -13,7 +13,7 @@ OPKG_UTILS_LICENSE_FILES = COPYING
 HOST_OPKG_UTILS_DEPENDENCIES += $(BR2_PYTHON3_HOST_DEPENDENCY)
 
 define HOST_OPKG_UTILS_INSTALL_CMDS
-	$(MAKE) -C $(@D) PREFIX=$(HOST_DIR) install
+	$(MAKE) -C $(@D) PREFIX=$(HOST_DIR) install-utils
 endef
 
 $(eval $(host-generic-package))
-- 
2.17.1

[Buildroot] [PATCH 3/8] package/diffutils: add host package

[Ryan Barnett]

Signed-off-by: Ryan Barnett <ryan.barnett@rockwellcollins.com>
---
 package/diffutils/diffutils.mk | 1 +
 1 file changed, 1 insertion(+)

diff --git a/package/diffutils/diffutils.mk b/package/diffutils/diffutils.mk
index 586b81ace8..107de79383 100644
--- a/package/diffutils/diffutils.mk
+++ b/package/diffutils/diffutils.mk
@@ -18,3 +18,4 @@ DIFFUTILS_CONF_ENV += gl_cv_func_getopt_gnu=yes
 endif
 
 $(eval $(autotools-package))
+$(eval $(host-autotools-package))
-- 
2.17.1

[Buildroot] [PATCH 4/8] package/findutils: add host package

[Ryan Barnett]

Signed-off-by: Ryan Barnett <ryan.barnett@rockwellcollins.com>
---
 package/findutils/findutils.mk | 1 +
 1 file changed, 1 insertion(+)

diff --git a/package/findutils/findutils.mk b/package/findutils/findutils.mk
index 13d8271e18..278eeec9b9 100644
--- a/package/findutils/findutils.mk
+++ b/package/findutils/findutils.mk
@@ -15,3 +15,4 @@ FINDUTILS_CONF_ENV = \
 	gl_cv_func_wcwidth_works=yes
 
 $(eval $(autotools-package))
+$(eval $(host-autotools-package))
-- 
2.17.1

[Buildroot] [PATCH 5/8] package/grep: add host package

[Ryan Barnett]

Signed-off-by: Ryan Barnett <ryan.barnett@rockwellcollins.com>
---
 package/grep/grep.mk | 1 +
 1 file changed, 1 insertion(+)

diff --git a/package/grep/grep.mk b/package/grep/grep.mk
index 27d204d4ce..332f9509ee 100644
--- a/package/grep/grep.mk
+++ b/package/grep/grep.mk
@@ -46,3 +46,4 @@ GREP_CONF_OPTS += --disable-perl-regexp
 endif
 
 $(eval $(autotools-package))
+$(eval $(host-autotools-package))
-- 
2.17.1

[Buildroot] [PATCH 6/8] package/opkg-utils: add missing dependencies for host build

[Ryan Barnett]

opkg-utils is a collection of bash and python scripts which require
additional commands/tools be available for the bash scripts. The full
list of dependencies that the opkg-util scripts require is:

  bash
  binutils
  bzip2
  coreutils
  diffutils
  findutils
  grep
  gzip
  lz4
  python3
  sed
  tar
  xz

Since buildroot requires that bash, binutils, bzip2, gzip, sed and tar
be installed on the host system, only add dependencies on the
remaining host tools.

Signed-off-by: Ryan Barnett <ryan.barnett@rockwellcollins.com>
---
 package/opkg-utils/opkg-utils.mk | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/package/opkg-utils/opkg-utils.mk b/package/opkg-utils/opkg-utils.mk
index a250b4ef9f..d7f68b8555 100644
--- a/package/opkg-utils/opkg-utils.mk
+++ b/package/opkg-utils/opkg-utils.mk
@@ -10,7 +10,14 @@ OPKG_UTILS_SITE_METHOD = git
 OPKG_UTILS_LICENSE = GPL-2.0+
 OPKG_UTILS_LICENSE_FILES = COPYING
 
-HOST_OPKG_UTILS_DEPENDENCIES += $(BR2_PYTHON3_HOST_DEPENDENCY)
+HOST_OPKG_UTILS_DEPENDENCIES += \
+	host-coreutils \
+	host-diffutils \
+	host-findutils \
+	host-grep \
+	host-lz4 \
+	$(BR2_PYTHON3_HOST_DEPENDENCY) \
+	host-xz
 
 define HOST_OPKG_UTILS_INSTALL_CMDS
 	$(MAKE) -C $(@D) PREFIX=$(HOST_DIR) install-utils
-- 
2.17.1

[Buildroot] [PATCH 7/8] package/opkg-utils: add opkg-utils as target pkg

[Ryan Barnett]

Supports a use case of building container rootfs images where a matching
target version of the tools is required for repackaging of a installer
 archive.

Signed-off-by: Ryan Barnett <ryan.barnett@rockwellcollins.com>
---
 package/Config.in                |  1 +
 package/opkg-utils/Config.in     | 33 ++++++++++++++++++++++++++++++++
 package/opkg-utils/opkg-utils.mk | 20 +++++++++++++++++++
 3 files changed, 54 insertions(+)
 create mode 100644 package/opkg-utils/Config.in

diff --git a/package/Config.in b/package/Config.in
index 992e46a319..fb77ba134e 100644
--- a/package/Config.in
+++ b/package/Config.in
@@ -2337,6 +2337,7 @@ comment "See the manual:                                        "
 comment "http://buildroot.org/manual.html#faq-no-binary-packages"
 comment "-------------------------------------------------------"
 	source "package/opkg/Config.in"
+	source "package/opkg-utils/Config.in"
 	source "package/rpm/Config.in"
 endmenu
 
diff --git a/package/opkg-utils/Config.in b/package/opkg-utils/Config.in
new file mode 100644
index 0000000000..b60a48c19d
--- /dev/null
+++ b/package/opkg-utils/Config.in
@@ -0,0 +1,33 @@
+config BR2_PACKAGE_OPKG_UTILS
+	bool "opkg-utils"
+	depends on BR2_USE_MMU # bash, coreutils, findutils, tar
+	depends on !BR2_nios2 # binutils
+	# binutils, coreutils, diffutils, findutils, grep, gzip, python3, sed,
+	# tar
+	depends on BR2_USE_WCHAR
+	depends on BR2_PACKAGE_BUSYBOX_SHOW_OTHERS
+	depends on BR2_TOOLCHAIN_HAS_THREADS # libffi
+	depends on !BR2_STATIC_LIBS # python3
+	depends on !BR2_PACKAGE_PYTHON # python3
+	select BR2_PACKAGE_BASH
+	select BR2_PACKAGE_BINUTILS
+	select BR2_PACKAGE_BZIP2
+	select BR2_PACKAGE_COREUTILS
+	select BR2_PACKAGE_DIFFUTILS
+	select BR2_PACKAGE_FINDUTILS
+	select BR2_PACKAGE_GREP
+	select BR2_PACKAGE_GZIP
+	select BR2_PACKAGE_LZ4
+	select BR2_PACKAGE_PYTHON3
+	select BR2_PACKAGE_SED
+	select BR2_PACKAGE_TAR
+	select BR2_PACKAGE_XZ
+	help
+	  Helper scripts for use with the opkg package manager.
+
+	  https://git.yoctoproject.org/cgit/cgit.cgi/opkg-utils/
+
+comment "opkg-utils needs a toolchain w/ wchar, threads, dynamic library"
+	depends on BR2_USE_MMU
+	depends on !BR2_nios2
+	depends on !BR2_USE_WCHAR || !BR2_TOOLCHAIN_HAS_THREADS || BR2_STATIC_LIBS
diff --git a/package/opkg-utils/opkg-utils.mk b/package/opkg-utils/opkg-utils.mk
index d7f68b8555..a445c9f49e 100644
--- a/package/opkg-utils/opkg-utils.mk
+++ b/package/opkg-utils/opkg-utils.mk
@@ -10,6 +10,21 @@ OPKG_UTILS_SITE_METHOD = git
 OPKG_UTILS_LICENSE = GPL-2.0+
 OPKG_UTILS_LICENSE_FILES = COPYING
 
+OPKG_UTILS_DEPENDENCIES += \
+	bash \
+	binutils \
+	bzip2 \
+	coreutils \
+	diffutils \
+	findutils \
+	grep \
+	gzip \
+	lz4 \
+	python3 \
+	sed \
+	tar \
+	xz
+
 HOST_OPKG_UTILS_DEPENDENCIES += \
 	host-coreutils \
 	host-diffutils \
@@ -19,8 +34,13 @@ HOST_OPKG_UTILS_DEPENDENCIES += \
 	$(BR2_PYTHON3_HOST_DEPENDENCY) \
 	host-xz
 
+define OPKG_UTILS_INSTALL_CMDS
+	$(MAKE) -C $(@D) PREFIX=$(TARGET_DIR) install-utils
+endef
+
 define HOST_OPKG_UTILS_INSTALL_CMDS
 	$(MAKE) -C $(@D) PREFIX=$(HOST_DIR) install-utils
 endef
 
+$(eval $(generic-package))
 $(eval $(host-generic-package))
-- 
2.17.1

[Buildroot] [PATCH 8/8] DEVELOPERS: Add Ryan Barnett for opkg and opkg-utils

[Ryan Barnett]

Signed-off-by: Ryan Barnett <ryan.barnett@rockwellcollins.com>
---
 DEVELOPERS | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/DEVELOPERS b/DEVELOPERS
index f6d6e4aa06..412cf827c2 100644
--- a/DEVELOPERS
+++ b/DEVELOPERS
@@ -2313,6 +2313,8 @@ N:	Ryan Barnett <ryan.barnett@rockwellcollins.com>
 F:	package/atftp/
 F:	package/c-periphery/
 F:	package/miraclecast/
+F:	package/opkg/
+F:	package/opkg-utils/
 F:	package/python-pysnmp/
 F:	package/python-pysnmp-mibs/
 F:	package/python-tornado/
-- 
2.17.1

[Buildroot] [PATCH 6/8] package/opkg-utils: add missing dependencies for host build

[Yann E. MORIN]

Ryan, All,

On 2021-03-12 14:05 -0600, Ryan Barnett spake thusly:
> opkg-utils is a collection of bash and python scripts which require
> additional commands/tools be available for the bash scripts. The full
> list of dependencies that the opkg-util scripts require is:
> 
>   bash
>   binutils
>   bzip2
>   coreutils
>   diffutils
>   findutils
>   grep
>   gzip
>   lz4
>   python3
>   sed
>   tar
>   xz
> 
> Since buildroot requires that bash, binutils, bzip2, gzip, sed and tar
> be installed on the host system, only add dependencies on the
> remaining host tools.
> 
> Signed-off-by: Ryan Barnett <ryan.barnett@rockwellcollins.com>
> ---
>  package/opkg-utils/opkg-utils.mk | 9 ++++++++-
>  1 file changed, 8 insertions(+), 1 deletion(-)
> 
> diff --git a/package/opkg-utils/opkg-utils.mk b/package/opkg-utils/opkg-utils.mk
> index a250b4ef9f..d7f68b8555 100644
> --- a/package/opkg-utils/opkg-utils.mk
> +++ b/package/opkg-utils/opkg-utils.mk
> @@ -10,7 +10,14 @@ OPKG_UTILS_SITE_METHOD = git
>  OPKG_UTILS_LICENSE = GPL-2.0+
>  OPKG_UTILS_LICENSE_FILES = COPYING
>  
> -HOST_OPKG_UTILS_DEPENDENCIES += $(BR2_PYTHON3_HOST_DEPENDENCY)
> +HOST_OPKG_UTILS_DEPENDENCIES += \
> +	host-coreutils \
> +	host-diffutils \
> +	host-findutils \
> +	host-grep \

We already implicitely require grep: we're calling it so many times in
the existing code. I am of the opinion that grep is a required package.

That it is inot in the manual, is probably because it is so wildely
installed by default in all distributions that we never had an issue
with it missing so far, so it was not added to the list of required
packages.

Yes, that's basically how that list came to be: someone reported an
issue about a tool missing, and it got added to the list (or we
instroduced a host version).

In the case of grep, I would say we should just require it on the host.

find is also on the same page: it is a required package.

As for coreutils, I would tend to be on the same page: it is a required
package.

However, we do have a host-coreutils package, but we only use when a
package needs either or both of 'ln --relative' or 'realpath'. See:
support/dependencies/check-host-coreutils.sh

If that is the case for opkg-utils, then it should add
$(BR2_COREUTILS_HOST_DEPENDENCY) to its _DEPENDENCIES. See:
    package/libselinux/libselinux.mk
    package/libsepol/libsepol.mk
    package/systemd/systemd.mk

Regards,
Yann E. MORIN.

> +	host-lz4 \
> +	$(BR2_PYTHON3_HOST_DEPENDENCY) \
> +	host-xz
>  
>  define HOST_OPKG_UTILS_INSTALL_CMDS
>  	$(MAKE) -C $(@D) PREFIX=$(HOST_DIR) install-utils
> -- 
> 2.17.1
> 
> _______________________________________________
> buildroot mailing list
> buildroot at busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCH 7/8] package/opkg-utils: add opkg-utils as target pkg

[Yann E. MORIN]

Ryan, All,

On 2021-03-12 14:05 -0600, Ryan Barnett spake thusly:
> Supports a use case of building container rootfs images where a matching
> target version of the tools is required for repackaging of a installer
>  archive.
> 
> Signed-off-by: Ryan Barnett <ryan.barnett@rockwellcollins.com>
> ---
[--SNIP--]
> diff --git a/package/opkg-utils/opkg-utils.mk b/package/opkg-utils/opkg-utils.mk
> index d7f68b8555..a445c9f49e 100644
> --- a/package/opkg-utils/opkg-utils.mk
> +++ b/package/opkg-utils/opkg-utils.mk
> @@ -10,6 +10,21 @@ OPKG_UTILS_SITE_METHOD = git
>  OPKG_UTILS_LICENSE = GPL-2.0+
>  OPKG_UTILS_LICENSE_FILES = COPYING
>  
> +OPKG_UTILS_DEPENDENCIES += \
> +	bash \
> +	binutils \
> +	bzip2 \
> +	coreutils \
> +	diffutils \
> +	findutils \
> +	grep \
> +	gzip \
> +	lz4 \
> +	python3 \
> +	sed \
> +	tar \
> +	xz

As you explained in a previous commit, opkg-utils only contains scripts
(in various languages, but still scripts), so the dependencies are not
needed at build time.

Regards,
Yann E. MORIN.

>  HOST_OPKG_UTILS_DEPENDENCIES += \
>  	host-coreutils \
>  	host-diffutils \
> @@ -19,8 +34,13 @@ HOST_OPKG_UTILS_DEPENDENCIES += \
>  	$(BR2_PYTHON3_HOST_DEPENDENCY) \
>  	host-xz
>  
> +define OPKG_UTILS_INSTALL_CMDS
> +	$(MAKE) -C $(@D) PREFIX=$(TARGET_DIR) install-utils
> +endef
> +
>  define HOST_OPKG_UTILS_INSTALL_CMDS
>  	$(MAKE) -C $(@D) PREFIX=$(HOST_DIR) install-utils
>  endef
>  
> +$(eval $(generic-package))
>  $(eval $(host-generic-package))
> -- 
> 2.17.1
> 
> _______________________________________________
> buildroot mailing list
> buildroot at busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCH 0/8] package/opkg-utils: add target build

[Yann E. MORIN]

Ryan, All,

On 2021-03-12 14:05 -0600, Ryan Barnett spake thusly:
> A first revision of adding opkg-utils as a target package was proposed
> in the following patch:
> 
>   https://patchwork.ozlabs.org/project/buildroot/patch/20210126192605.20002-1-matthew.weber at rockwellcollins.com/
> 
> Based on feedback received from Yann, it uncovered some issues with the
> current host package build for opkg-utils.
> 
> This patch series addresses the following items:
> 
>  * Remove build step since all it does is generate man pages for a
>    single command (Patch 1)
>  * Incorporated accepted patch for splitting off the installation of
>    docs and utils (Patch 2)
>  * Add missing host dependencies for host-opkg-utils which are not
>    manditory packages of for buildroot (Patches 3-6)
>  * Add target package for opkg-utils (Patch 7)
> 
> Thanks,
> -Ryan
> 
> CC: Yann E. MORIN <yann.morin.1998@free.fr>
> 
> Ryan Barnett (8):
>   package/opkg-utils: remove build step
>   package/opkg-utils: install only utility scripts

I've inverted the order of those two first patches; I reworded the
commit log of the now-second patch (I'll reply further in that patch).

Applied to master.

>   package/diffutils: add host package

Applied to master.

>   package/findutils: add host package
>   package/grep: add host package

I've dropped those two, as I explained in the review of...

>   package/opkg-utils: add missing dependencies for host build

... this patch: grep and find are already required (but not on the list
in the manual).

Applied to master with less dependencies and a commit log reworded
accodingly.

I got short on time, so I stopped here. I'll resume the remaining two
later.

Thanks!

Regards,
Yann E. MORIN.

>   package/opkg-utils: add opkg-utils as target pkg
>   DEVELOPERS: Add Ryan Barnett for opkg and opkg-utils
> 
>  DEVELOPERS                                    |  2 +
>  package/Config.in                             |  1 +
>  package/diffutils/diffutils.mk                |  1 +
>  package/findutils/findutils.mk                |  1 +
>  package/grep/grep.mk                          |  1 +
>  ...-seperate-manpages-and-utils-install.patch | 50 +++++++++++++++++++
>  package/opkg-utils/Config.in                  | 33 ++++++++++++
>  package/opkg-utils/opkg-utils.mk              | 31 ++++++++++--
>  8 files changed, 116 insertions(+), 4 deletions(-)
>  create mode 100644 package/opkg-utils/0001-Makefile-seperate-manpages-and-utils-install.patch
>  create mode 100644 package/opkg-utils/Config.in
> 
> --
> 2.17.1
> 
> _______________________________________________
> buildroot mailing list
> buildroot at busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCH 1/8] package/opkg-utils: remove build step

[Yann E. MORIN]

On 2021-03-12 14:05 -0600, Ryan Barnett spake thusly:
> opkg-utils is a package that only provides bash and python scripts.
> Upon further inspection of the Makefile for the package, invoking
> `make` is equivalent to `make install`.

That is not really true: 'make' does not install, which 'make isntall'
does; to they are not equivalent.

In fact, 'make' only buiold the manpage, and as you explained in the
following patch, we don't want to build+install it.

So by inverting the order f patches 2 and 1, building the manpage is no
longer ncessary as it is no longer installed.

(if we'd keep your initial order, the build of the manpage would happen
durign installation, which is not nice)

I've added a small comment to explain why we only have an install step.

Applied to master, thanks.

Regards,
Yann E. MORIN.

> The makes the build step
> unnecessary so remove it.
> 
> Signed-off-by: Ryan Barnett <ryan.barnett@rockwellcollins.com>
> ---
>  package/opkg-utils/opkg-utils.mk | 4 ----
>  1 file changed, 4 deletions(-)
> 
> diff --git a/package/opkg-utils/opkg-utils.mk b/package/opkg-utils/opkg-utils.mk
> index a94a4cd5cd..47fc4c63b4 100644
> --- a/package/opkg-utils/opkg-utils.mk
> +++ b/package/opkg-utils/opkg-utils.mk
> @@ -12,10 +12,6 @@ OPKG_UTILS_LICENSE_FILES = COPYING
>  
>  HOST_OPKG_UTILS_DEPENDENCIES += $(BR2_PYTHON3_HOST_DEPENDENCY)
>  
> -define HOST_OPKG_UTILS_BUILD_CMDS
> -	$(MAKE) -C $(@D) $(HOST_CONFIGURE_OPTS)
> -endef
> -
>  define HOST_OPKG_UTILS_INSTALL_CMDS
>  	$(MAKE) -C $(@D) PREFIX=$(HOST_DIR) install
>  endef
> -- 
> 2.17.1
> 
> _______________________________________________
> buildroot mailing list
> buildroot at busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCH 7/8] package/opkg-utils: add opkg-utils as target pkg

[Yann E. MORIN]

Ryan, All,

On 2021-03-12 14:05 -0600, Ryan Barnett spake thusly:
> Supports a use case of building container rootfs images where a matching
> target version of the tools is required for repackaging of a installer
>  archive.
> 
> Signed-off-by: Ryan Barnett <ryan.barnett@rockwellcollins.com>

Applied to master, with the following changes:

  - only select full-blown packages if busybox is not enabled
  - select BR2_PACKAGE_BUSYBOX_SHOW_OTHERS, instead of 'depends on'
  - allow on nios2 when busybox is enabled
  - add binutils binaries on target (for 'ar')
  - drop _DEPENDENCIES: they all are only runtime-dpeendencies
  - add comment when python(2) is enabled

Thanks!

Regards,
Yann E. MORIN.

> ---
>  package/Config.in                |  1 +
>  package/opkg-utils/Config.in     | 33 ++++++++++++++++++++++++++++++++
>  package/opkg-utils/opkg-utils.mk | 20 +++++++++++++++++++
>  3 files changed, 54 insertions(+)
>  create mode 100644 package/opkg-utils/Config.in
> 
> diff --git a/package/Config.in b/package/Config.in
> index 992e46a319..fb77ba134e 100644
> --- a/package/Config.in
> +++ b/package/Config.in
> @@ -2337,6 +2337,7 @@ comment "See the manual:                                        "
>  comment "http://buildroot.org/manual.html#faq-no-binary-packages"
>  comment "-------------------------------------------------------"
>  	source "package/opkg/Config.in"
> +	source "package/opkg-utils/Config.in"
>  	source "package/rpm/Config.in"
>  endmenu
>  
> diff --git a/package/opkg-utils/Config.in b/package/opkg-utils/Config.in
> new file mode 100644
> index 0000000000..b60a48c19d
> --- /dev/null
> +++ b/package/opkg-utils/Config.in
> @@ -0,0 +1,33 @@
> +config BR2_PACKAGE_OPKG_UTILS
> +	bool "opkg-utils"
> +	depends on BR2_USE_MMU # bash, coreutils, findutils, tar
> +	depends on !BR2_nios2 # binutils
> +	# binutils, coreutils, diffutils, findutils, grep, gzip, python3, sed,
> +	# tar
> +	depends on BR2_USE_WCHAR
> +	depends on BR2_PACKAGE_BUSYBOX_SHOW_OTHERS
> +	depends on BR2_TOOLCHAIN_HAS_THREADS # libffi
> +	depends on !BR2_STATIC_LIBS # python3
> +	depends on !BR2_PACKAGE_PYTHON # python3
> +	select BR2_PACKAGE_BASH
> +	select BR2_PACKAGE_BINUTILS
> +	select BR2_PACKAGE_BZIP2
> +	select BR2_PACKAGE_COREUTILS
> +	select BR2_PACKAGE_DIFFUTILS
> +	select BR2_PACKAGE_FINDUTILS
> +	select BR2_PACKAGE_GREP
> +	select BR2_PACKAGE_GZIP
> +	select BR2_PACKAGE_LZ4
> +	select BR2_PACKAGE_PYTHON3
> +	select BR2_PACKAGE_SED
> +	select BR2_PACKAGE_TAR
> +	select BR2_PACKAGE_XZ
> +	help
> +	  Helper scripts for use with the opkg package manager.
> +
> +	  https://git.yoctoproject.org/cgit/cgit.cgi/opkg-utils/
> +
> +comment "opkg-utils needs a toolchain w/ wchar, threads, dynamic library"
> +	depends on BR2_USE_MMU
> +	depends on !BR2_nios2
> +	depends on !BR2_USE_WCHAR || !BR2_TOOLCHAIN_HAS_THREADS || BR2_STATIC_LIBS
> diff --git a/package/opkg-utils/opkg-utils.mk b/package/opkg-utils/opkg-utils.mk
> index d7f68b8555..a445c9f49e 100644
> --- a/package/opkg-utils/opkg-utils.mk
> +++ b/package/opkg-utils/opkg-utils.mk
> @@ -10,6 +10,21 @@ OPKG_UTILS_SITE_METHOD = git
>  OPKG_UTILS_LICENSE = GPL-2.0+
>  OPKG_UTILS_LICENSE_FILES = COPYING
>  
> +OPKG_UTILS_DEPENDENCIES += \
> +	bash \
> +	binutils \
> +	bzip2 \
> +	coreutils \
> +	diffutils \
> +	findutils \
> +	grep \
> +	gzip \
> +	lz4 \
> +	python3 \
> +	sed \
> +	tar \
> +	xz
> +
>  HOST_OPKG_UTILS_DEPENDENCIES += \
>  	host-coreutils \
>  	host-diffutils \
> @@ -19,8 +34,13 @@ HOST_OPKG_UTILS_DEPENDENCIES += \
>  	$(BR2_PYTHON3_HOST_DEPENDENCY) \
>  	host-xz
>  
> +define OPKG_UTILS_INSTALL_CMDS
> +	$(MAKE) -C $(@D) PREFIX=$(TARGET_DIR) install-utils
> +endef
> +
>  define HOST_OPKG_UTILS_INSTALL_CMDS
>  	$(MAKE) -C $(@D) PREFIX=$(HOST_DIR) install-utils
>  endef
>  
> +$(eval $(generic-package))
>  $(eval $(host-generic-package))
> -- 
> 2.17.1
> 
> _______________________________________________
> buildroot mailing list
> buildroot at busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCH 8/8] DEVELOPERS: Add Ryan Barnett for opkg and opkg-utils

[Yann E. MORIN]

Ryan, All,

On 2021-03-12 14:05 -0600, Ryan Barnett spake thusly:
> Signed-off-by: Ryan Barnett <ryan.barnett@rockwellcollins.com>

Applied to master, thanks for adopting those! :-)

Regards,
Yann E. MORIN.

> ---
>  DEVELOPERS | 2 ++
>  1 file changed, 2 insertions(+)
> 
> diff --git a/DEVELOPERS b/DEVELOPERS
> index f6d6e4aa06..412cf827c2 100644
> --- a/DEVELOPERS
> +++ b/DEVELOPERS
> @@ -2313,6 +2313,8 @@ N:	Ryan Barnett <ryan.barnett@rockwellcollins.com>
>  F:	package/atftp/
>  F:	package/c-periphery/
>  F:	package/miraclecast/
> +F:	package/opkg/
> +F:	package/opkg-utils/
>  F:	package/python-pysnmp/
>  F:	package/python-pysnmp-mibs/
>  F:	package/python-tornado/
> -- 
> 2.17.1
> 
> _______________________________________________
> buildroot mailing list
> buildroot at busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCH 6/8] package/opkg-utils: add missing dependencies for host build

[Ryan Barnett]

Yann,


On Fri, Mar 12, 2021 at 4:52 PM Yann E. MORIN <yann.morin.1998@free.fr> wrote:
>
> Ryan, All,
>
> On 2021-03-12 14:05 -0600, Ryan Barnett spake thusly:
> > opkg-utils is a collection of bash and python scripts which require
> > additional commands/tools be available for the bash scripts. The full
> > list of dependencies that the opkg-util scripts require is:
> >
> >   bash
> >   binutils
> >   bzip2
> >   coreutils
> >   diffutils
> >   findutils
> >   grep
> >   gzip
> >   lz4
> >   python3
> >   sed
> >   tar
> >   xz
> >
> > Since buildroot requires that bash, binutils, bzip2, gzip, sed and tar
> > be installed on the host system, only add dependencies on the
> > remaining host tools.
> >
> > Signed-off-by: Ryan Barnett <ryan.barnett@rockwellcollins.com>
> > ---
> >  package/opkg-utils/opkg-utils.mk | 9 ++++++++-
> >  1 file changed, 8 insertions(+), 1 deletion(-)
> >
> > diff --git a/package/opkg-utils/opkg-utils.mk b/package/opkg-utils/opkg-utils.mk
> > index a250b4ef9f..d7f68b8555 100644
> > --- a/package/opkg-utils/opkg-utils.mk
> > +++ b/package/opkg-utils/opkg-utils.mk
> > @@ -10,7 +10,14 @@ OPKG_UTILS_SITE_METHOD = git
> >  OPKG_UTILS_LICENSE = GPL-2.0+
> >  OPKG_UTILS_LICENSE_FILES = COPYING
> >
> > -HOST_OPKG_UTILS_DEPENDENCIES += $(BR2_PYTHON3_HOST_DEPENDENCY)
> > +HOST_OPKG_UTILS_DEPENDENCIES += \
> > +     host-coreutils \
> > +     host-diffutils \
> > +     host-findutils \
> > +     host-grep \
>
> We already implicitely require grep: we're calling it so many times in
> the existing code. I am of the opinion that grep is a required package.
>
> That it is inot in the manual, is probably because it is so wildely
> installed by default in all distributions that we never had an issue
> with it missing so far, so it was not added to the list of required
> packages.
>
> Yes, that's basically how that list came to be: someone reported an
> issue about a tool missing, and it got added to the list (or we
> instroduced a host version).
>
> In the case of grep, I would say we should just require it on the host.
>
> find is also on the same page: it is a required package.
>
> As for coreutils, I would tend to be on the same page: it is a required
> package.

Thanks for the feedback and applying the series!

I've been looking at getting buildroot to work on a minimal Alpine
Docker container and I've found that these packages are required. I
will look to submit a patch which includes that so we can capture
this.

Thanks,
-Ryan

> However, we do have a host-coreutils package, but we only use when a
> package needs either or both of 'ln --relative' or 'realpath'. See:
> support/dependencies/check-host-coreutils.sh
>
> If that is the case for opkg-utils, then it should add
> $(BR2_COREUTILS_HOST_DEPENDENCY) to its _DEPENDENCIES. See:
>     package/libselinux/libselinux.mk
>     package/libsepol/libsepol.mk
>     package/systemd/systemd.mk
>
> Regards,
> Yann E. MORIN.
>
> > +     host-lz4 \
> > +     $(BR2_PYTHON3_HOST_DEPENDENCY) \
> > +     host-xz
> >
> >  define HOST_OPKG_UTILS_INSTALL_CMDS
> >       $(MAKE) -C $(@D) PREFIX=$(HOST_DIR) install-utils
> > --
> > 2.17.1

[Buildroot] [PATCH 1/8] package/opkg-utils: remove build step

[Peter Korsgaard]

>>>>> "Ryan" == Ryan Barnett <ryan.barnett@rockwellcollins.com> writes:

 > opkg-utils is a package that only provides bash and python scripts.
 > Upon further inspection of the Makefile for the package, invoking
 > `make` is equivalent to `make install`. The makes the build step
 > unnecessary so remove it.

 > Signed-off-by: Ryan Barnett <ryan.barnett@rockwellcollins.com>

Committed to 2020.02.x, 2020.11.x and 2021.02.x, thanks.

-- 
Bye, Peter Korsgaard

[Buildroot] [PATCH 2/8] package/opkg-utils: install only utility scripts

[Peter Korsgaard]

>>>>> "Ryan" == Ryan Barnett <ryan.barnett@rockwellcollins.com> writes:

 > When `make install` is run to install the opkg-utils scripts, it also
 > invokes building of the man page for opkg-build. The generation of the
 > man page requires `pod2man` executable which is a part of perl.

 > Since buildroot does not support man pages in the host directory,
 > patch the opkg-utils Makefile to separate the installation of man
 > pages and utility scripts.

 > With the options to install man pages and utils separately, only
 > install the opkg-utils scripts.

 > Signed-off-by: Ryan Barnett <ryan.barnett@rockwellcollins.com>

Committed to 2020.02.x, 2020.11.x and 2021.02.x, thanks.

-- 
Bye, Peter Korsgaard

[Buildroot] [PATCH 3/8] package/diffutils: add host package

[Peter Korsgaard]

>>>>> "Ryan" == Ryan Barnett <ryan.barnett@rockwellcollins.com> writes:

 > Signed-off-by: Ryan Barnett <ryan.barnett@rockwellcollins.com>

Committed to 2020.02.x, 2020.11.x and 2021.02.x, thanks.

-- 
Bye, Peter Korsgaard

[Buildroot] [PATCH 6/8] package/opkg-utils: add missing dependencies for host build

[Peter Korsgaard]

>>>>> "Ryan" == Ryan Barnett <ryan.barnett@rockwellcollins.com> writes:

 > opkg-utils is a collection of bash and python scripts which require
 > additional commands/tools be available for the bash scripts. The full
 > list of dependencies that the opkg-util scripts require is:

 >   bash
 >   binutils
 >   bzip2
 >   coreutils
 >   diffutils
 >   findutils
 >   grep
 >   gzip
 >   lz4
 >   python3
 >   sed
 >   tar
 >   xz

 > Since buildroot requires that bash, binutils, bzip2, gzip, sed and tar
 > be installed on the host system, only add dependencies on the
 > remaining host tools.

 > Signed-off-by: Ryan Barnett <ryan.barnett@rockwellcollins.com>

Committed to 2020.02.x, 2020.11.x and 2021.02.x, thanks.

-- 
Bye, Peter Korsgaard