[thud] [PATCH 0/2] opkg and BAD_RECOMMENDATIONS in thud 2.6.2

[thud] [PATCH 0/2] opkg and BAD_RECOMMENDATIONS in thud 2.6.2

[Quentin Schulz]

Hi all,

I've been hit with a rather unsettling bug when building an image with
PACKAGE_CLASSES = "ipk" in Thud 2.6.2.

The error message is the following:
ERROR: meh-1.0-r0 do_rootfs: Unable to install packages. Command
'/tmp/yocto/temp/work/machine-poky-linux/meh/1.0-r0/recipe-sysroot-native/usr/bin/opkg --volatile-cache -f /tmp/yocto/temp/work/machine-poky-linux/meh/1.0-r0/opkg.conf -t /tmp/yocto/temp/work/machine-poky-linux/meh/1.0-r0/temp/ipktemp/ -o /tmp/yocto/temp/work/machine-poky-linux/meh/1.0-r0/rootfs  --force_postinstall --prefer-arch-to-version   --force-maintainer --force-overwrite install packagegroup-base-extended packagegroup-core-boot plop run-postinsts' returned 2:
Collected errors:
 * Solver encountered 2 problem(s):
 * Problem 1/2:
 *   - nothing provides bluez5 needed by plop-1.0-r0.aarch64
 * 
 * Solution 1:
 *   - do not ask to install a package providing plop

 * Problem 2/2:
 *   - nothing provides bluez5 needed by packagegroup-base-bluetooth-1.0-r83.stream1832prime
 * 
 * Solution 1:
 *   - do not ask to install a package providing packagegroup-base-extended




ERROR: meh-1.0-r0 do_rootfs: Function failed: do_rootfs
ERROR: Logfile of failure stored in: /tmp/yocto/temp/work/machine-poky-linux/meh/1.0-r0/temp/log.do_rootfs.30869
ERROR: Task (/media/qschulz/yocto/build/oe/recipes-images/nsdk/meh.bb:do_rootfs) failed with exit code '1'
NOTE: Tasks Summary: Attempted 3059 tasks of which 3046 didn't need to be rerun and 1 failed.

Summary: 1 task failed:
  /media/qschulz/yocto/build/oe/recipes-images/nsdk/meh.bb:do_rootfs
Summary: There were 7 WARNING messages shown.
Summary: There were 2 ERROR messages shown, returning a non-zero exit code.


I could reproduce it with an image meh.bb:
inherit core-image

CORE_IMAGE_EXTRA_INSTALL += "plop"

BAD_RECOMMENDATIONS += "bluez5"

with plop.bb being:
DESCRIPTION = "Plap"
LICENSE = "CLOSED"

RDEPENDS_${PN} += "bluez5"

do_install() {
        install -d ${D}${bindir}
        echo "plip" > ${D}${bindir}/plep
}

I took me a while to find out that it's an issue with opkg itself and
nothing could be done in Yocto to fix it but I found the three following
patches in yocto-2.7:

https://git.yoctoproject.org/cgit.cgi/poky/commit/?id=d8e12f2bfc4deaa0dc7530cf3af905e18e8037f6
https://git.yoctoproject.org/cgit.cgi/poky/commit/?id=c76c50df6498d85e7a6254002ff8f5e0ef3e67f7
https://git.yoctoproject.org/cgit.cgi/poky/commit/?id=5bcb626cf49d2295ccca45d6fc7e1617cca932d6

With those three applied on top of thud 2.6.2, my image is built without
complaints.

Please apply those backported patches for next Thud releases.


Thanks,
Quentin

Alejandro del Castillo (2):
  opkg: add --ignore-recommends flag
  OpkgPM: use --add-ignore-recommends to process BAD_RECOMMENDATIONS

 meta/lib/oe/package_manager.py                |  41 +--
 meta/lib/oe/rootfs.py                         |   2 -
 ...pkg-add-add-ignore-recommends-option.patch | 260 ++++++++++++++++++
 meta/recipes-devtools/opkg/opkg_0.3.6.bb      |   1 +
 4 files changed, 263 insertions(+), 41 deletions(-)
 create mode 100644 meta/recipes-devtools/opkg/opkg/0001-libopkg-add-add-ignore-recommends-option.patch

-- 
2.17.1

[thud] [PATCH 1/2] opkg: add --ignore-recommends flag

[Quentin Schulz]

From: Alejandro del Castillo <alejandro.delcastillo@ni.com>

To be used for BAD_RECOMMENDATIONS feature.

(From OE-Core rev: 788d97b4f8e4452cef1ba6bb3e565e1b52dbb7de)

Signed-off-by: Alejandro del Castillo <alejandro.delcastillo@ni.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
[Backport from opkg_0.4.0.bb]
Signed-off-by: Quentin Schulz <quentin.schulz@streamunlimited.com>
---
 ...pkg-add-add-ignore-recommends-option.patch | 260 ++++++++++++++++++
 meta/recipes-devtools/opkg/opkg_0.3.6.bb      |   1 +
 2 files changed, 261 insertions(+)
 create mode 100644 meta/recipes-devtools/opkg/opkg/0001-libopkg-add-add-ignore-recommends-option.patch

diff --git a/meta/recipes-devtools/opkg/opkg/0001-libopkg-add-add-ignore-recommends-option.patch b/meta/recipes-devtools/opkg/opkg/0001-libopkg-add-add-ignore-recommends-option.patch
new file mode 100644
index 0000000000..11954e9032
--- /dev/null
+++ b/meta/recipes-devtools/opkg/opkg/0001-libopkg-add-add-ignore-recommends-option.patch
@@ -0,0 +1,260 @@
+From 2eca28b6a37be92e4e835c51872c7df34ec6dedd Mon Sep 17 00:00:00 2001
+From: Quentin Schulz <quentin.schulz@streamunlimited.com>
+Date: Fri, 31 May 2019 17:30:40 +0200
+Subject: [PATCH] [PATCH] libopkg: add --add-ignore-recommends option
+
+Add option to ignore specific recommended packages. On the libsolv
+backed, this feature will only work on libsolv version > 0.7.2 [1].
+
+[1] https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_openSUSE_libsolv_issues_254&d=DwIBaQ&c=I_0YwoKy7z5LMTVdyO6YCiE2uzI1jjZZuIPelcSjixA&r=wNcrL2akRn6jfxhHaKavUrJB_C9JAMXtynjLd8ZzgXQ&m=GObNHzFJpWpf_PripIrf-K2RhsktYdAUEieAJexXOKw&s=3G-meChUqClFggFPqsrAxIZBfLnRKIHm62Uuy1X6nQQ&e=
+
+Signed-off-by: Alejandro del Castillo <alejandro.delcastillo@ni.com>
+Signed-off-by: Quentin Schulz <quentin.schulz@streamunlimited.com>
+
+Upstream-Status: Backport
+---
+ libopkg/opkg_conf.c                           |  2 +
+ libopkg/opkg_conf.h                           |  1 +
+ .../solvers/internal/pkg_depends_internal.c   |  3 +-
+ libopkg/solvers/libsolv/opkg_solver_libsolv.c | 21 ++++++-
+ man/opkg.1.in                                 |  3 +
+ src/opkg.c                                    |  6 ++
+ tests/Makefile                                |  1 +
+ tests/core/43_add_ignore_recommends.py        | 62 +++++++++++++++++++
+ 8 files changed, 97 insertions(+), 2 deletions(-)
+ create mode 100644 tests/core/43_add_ignore_recommends.py
+
+diff --git a/libopkg/opkg_conf.c b/libopkg/opkg_conf.c
+index 06880a1..f2330cd 100644
+--- a/libopkg/opkg_conf.c
++++ b/libopkg/opkg_conf.c
+@@ -597,6 +597,7 @@ int opkg_conf_init(void)
+     pkg_dest_list_init(&opkg_config->tmp_dest_list);
+     nv_pair_list_init(&opkg_config->arch_list);
+     str_list_init(&opkg_config->exclude_list);
++    str_list_init(&opkg_config->ignore_recommends_list);
+ 
+     return 0;
+ }
+@@ -938,6 +939,7 @@ void opkg_conf_deinit(void)
+     pkg_dest_list_deinit(&opkg_config->pkg_dest_list);
+     nv_pair_list_deinit(&opkg_config->arch_list);
+     str_list_deinit(&opkg_config->exclude_list);
++    str_list_deinit(&opkg_config->ignore_recommends_list);
+ 
+     if (opkg_config->verbosity >= DEBUG) {
+         hash_print_stats(&opkg_config->pkg_hash);
+diff --git a/libopkg/opkg_conf.h b/libopkg/opkg_conf.h
+index dc11516..fc42de3 100644
+--- a/libopkg/opkg_conf.h
++++ b/libopkg/opkg_conf.h
+@@ -61,6 +61,7 @@ typedef struct opkg_conf {
+     pkg_dest_list_t tmp_dest_list;
+     nv_pair_list_t arch_list;
+     str_list_t exclude_list;
++    str_list_t ignore_recommends_list;
+ 
+     int restrict_to_default_dest;
+     pkg_dest_t *default_dest;
+diff --git a/libopkg/solvers/internal/pkg_depends_internal.c b/libopkg/solvers/internal/pkg_depends_internal.c
+index cd56d84..5deee70 100644
+--- a/libopkg/solvers/internal/pkg_depends_internal.c
++++ b/libopkg/solvers/internal/pkg_depends_internal.c
+@@ -228,7 +228,8 @@ int pkg_hash_fetch_unsatisfied_dependencies(pkg_t *pkg,
+                             || compound_depend->type == SUGGEST)
+                         && (satisfying_pkg->state_want == SW_DEINSTALL
+                             || satisfying_pkg->state_want == SW_PURGE
+-                            || opkg_config->no_install_recommends);
++                            || opkg_config->no_install_recommends
++                            || str_list_contains(&opkg_config->ignore_recommends_list, satisfying_pkg->name));
+                 if (ignore) {
+                     opkg_msg(NOTICE,
+                              "%s: ignoring recommendation for "
+diff --git a/libopkg/solvers/libsolv/opkg_solver_libsolv.c b/libopkg/solvers/libsolv/opkg_solver_libsolv.c
+index bf0b72c..ea00a37 100644
+--- a/libopkg/solvers/libsolv/opkg_solver_libsolv.c
++++ b/libopkg/solvers/libsolv/opkg_solver_libsolv.c
+@@ -484,6 +484,7 @@ static void pkg2solvable(pkg_t *pkg, Solvable *solvable_out)
+ static void populate_installed_repo(libsolv_solver_t *libsolv_solver)
+ {
+     int i;
++    Id what;
+ 
+     pkg_vec_t *installed_pkgs = pkg_vec_alloc();
+ 
+@@ -507,6 +508,15 @@ static void populate_installed_repo(libsolv_solver_t *libsolv_solver)
+         /* set solvable attributes */
+         pkg2solvable(pkg, solvable);
+ 
++        /* if the package is in ignore-recommends-list, disfavor installation */
++        if (str_list_contains(&opkg_config->ignore_recommends_list, pkg->name)) {
++            opkg_message(NOTICE, "Disfavor package: %s\n",
++                         pkg->name);
++            what = pool_str2id(libsolv_solver->pool, pkg->name, 1);
++            queue_push2(&libsolv_solver->solver_jobs, SOLVER_SOLVABLE_NAME
++                        | SOLVER_DISFAVOR, what);
++        }
++
+         /* if the package is not autoinstalled, mark it as user installed */
+         if (!pkg->auto_installed)
+             queue_push2(&libsolv_solver->solver_jobs, SOLVER_SOLVABLE
+@@ -533,7 +543,7 @@ static void populate_available_repos(libsolv_solver_t *libsolv_solver)
+ {
+     int i;
+     Solvable *solvable;
+-    Id solvable_id;
++    Id solvable_id, what;
+ 
+     pkg_vec_t *available_pkgs = pkg_vec_alloc();
+ 
+@@ -602,6 +612,15 @@ static void populate_available_repos(libsolv_solver_t *libsolv_solver)
+         solvable = pool_id2solvable(libsolv_solver->pool, solvable_id);
+         pkg2solvable(pkg, solvable);
+ 
++        /* if the package is in ignore-recommends-list, disfavor installation */
++        if (str_list_contains(&opkg_config->ignore_recommends_list, pkg->name)) {
++            opkg_message(NOTICE, "Disfavor package: %s\n",
++                         pkg->name);
++            what = pool_str2id(libsolv_solver->pool, pkg->name, 1);
++            queue_push2(&libsolv_solver->solver_jobs, SOLVER_SOLVABLE_NAME
++                        | SOLVER_DISFAVOR, what);
++        }
++
+         /* if the --force-depends option is specified make dependencies weak */
+         if (opkg_config->force_depends)
+             queue_push2(&libsolv_solver->solver_jobs, SOLVER_SOLVABLE
+diff --git a/man/opkg.1.in b/man/opkg.1.in
+index 026fb15..c0d4bf3 100644
+--- a/man/opkg.1.in
++++ b/man/opkg.1.in
+@@ -143,6 +143,9 @@ conjunction with \fB\--dest\fP
+ \fB\--add-arch <\fIarch\fP>:<\fIprio\fP>\fR
+ Register the package architecture \fIarch\fP with the numeric
+ priority \fIprio\fP. Lower priorities take precedence.
++.TP
++\fB\--add-ignore-recommends <\fIname\fP>\fR
++Register package to be ignored as a recomendee
+ .SS FORCE OPTIONS
+ .TP
+ \fB\--force-depends \fR
+diff --git a/src/opkg.c b/src/opkg.c
+index f23467d..5181065 100644
+--- a/src/opkg.c
++++ b/src/opkg.c
+@@ -51,6 +51,7 @@ enum {
+     ARGS_OPT_ADD_DEST,
+     ARGS_OPT_SIZE,
+     ARGS_OPT_ADD_EXCLUDE,
++    ARGS_OPT_ADD_IGNORE_RECOMMENDS,
+     ARGS_OPT_NOACTION,
+     ARGS_OPT_DOWNLOAD_ONLY,
+     ARGS_OPT_NODEPS,
+@@ -110,6 +111,7 @@ static struct option long_options[] = {
+     {"add-dest", 1, 0, ARGS_OPT_ADD_DEST},
+     {"size", 0, 0, ARGS_OPT_SIZE},
+     {"add-exclude", 1, 0, ARGS_OPT_ADD_EXCLUDE},
++    {"add-ignore-recommends", 1, 0, ARGS_OPT_ADD_IGNORE_RECOMMENDS},
+     {"test", 0, 0, ARGS_OPT_NOACTION},
+     {"tmp-dir", 1, 0, 't'},
+     {"tmp_dir", 1, 0, 't'},
+@@ -235,6 +237,9 @@ static int args_parse(int argc, char *argv[])
+         case ARGS_OPT_ADD_EXCLUDE:
+             str_list_append(&opkg_config->exclude_list, optarg);
+             break;
++        case ARGS_OPT_ADD_IGNORE_RECOMMENDS:
++            str_list_append(&opkg_config->ignore_recommends_list, optarg);
++            break;
+         case ARGS_OPT_SIZE:
+             opkg_config->size = 1;
+             break;
+@@ -335,6 +340,7 @@ static void usage()
+     printf("\t--add-arch <arch>:<prio>        Register architecture with given priority\n");
+     printf("\t--add-dest <name>:<path>        Register destination with given path\n");
+     printf("\t--add-exclude <name>            Register package to be excluded from install\n");
++    printf("\t--add-ignore-recommends <name>  Register package to be ignored as a recomendee\n");
+     printf("\t--prefer-arch-to-version        Use the architecture priority package rather\n");
+     printf("\t                                than the higher version one if more\n");
+     printf("\t                                than one candidate is found.\n");
+diff --git a/tests/Makefile b/tests/Makefile
+index 148c844..ddf027f 100644
+--- a/tests/Makefile
++++ b/tests/Makefile
+@@ -38,6 +38,7 @@ REGRESSION_TESTS := core/01_install.py \
+ 		    core/37_globs.py \
+ 		    core/38_install_constrained_version.py \
+ 		    core/39_distupgrade.py \
++		    core/43_add_ignore_recommends.py \
+ 		    regress/issue26.py \
+ 		    regress/issue31.py \
+ 		    regress/issue32.py \
+diff --git a/tests/core/43_add_ignore_recommends.py b/tests/core/43_add_ignore_recommends.py
+new file mode 100644
+index 0000000..7da0096
+--- /dev/null
++++ b/tests/core/43_add_ignore_recommends.py
+@@ -0,0 +1,62 @@
++#! /usr/bin/env python3
++#
++# Create package 'a' (1.0) which Recommends 'c'.
++# Install 'a' with --add-ignore-recommends 'c'.
++# Check that only 'a' (1.0) is installed.
++# Create package 'b' which Depends on 'c'.
++# Install 'a' & 'b', with --add-ignore-recommends 'c'.
++# Verify that 'a','b' & 'c' are installed.
++# Uninstall 'b' & 'c'.
++# Create package 'a' (2.0), which Recommends 'c'.
++# Upgrade 'a' with --add-ignore-recommends 'c'
++# Verify that only 'a' (2.0) is installed
++#
++
++import os
++import opk, cfg, opkgcl
++
++opk.regress_init()
++o = opk.OpkGroup()
++
++o.add(Package='a', Recommends='c', Version='1.0')
++o.add(Package='b', Depends='c')
++o.add(Package='c')
++o.write_opk()
++o.write_list()
++
++opkgcl.update()
++
++opkgcl.install('a', '--add-ignore-recommends c')
++
++if not opkgcl.is_installed('a'):
++	opk.fail("Package 'a' installed but reports as not installed.")
++
++if opkgcl.is_installed('c'):
++	opk.xfail("[libsolv<0.7.3] Package 'c' should not have been installed since it was in --add-ignore-recommends.")
++
++opkgcl.remove('a')
++opkgcl.install('a b', '--add-ignore-recommends c')
++
++if not opkgcl.is_installed('a'):
++	opk.fail("Package 'a' installed but reports as not installed.")
++
++if not opkgcl.is_installed('b'):
++	opk.fail("Package 'b' installed but reports as not installed.")
++
++if not opkgcl.is_installed('c'):
++	opk.fail("Package 'c' should have been installed since 'b' depends on it.")
++
++opkgcl.remove('b c', '--force-depends')
++o.add(Package='a', Recommends='c', Version='2.0')
++o.write_opk()
++o.write_list()
++
++opkgcl.update()
++
++opkgcl.upgrade('a', '--add-ignore-recommends c')
++
++if not opkgcl.is_installed('a', '2.0'):
++	opk.fail("Package 'a (2.0)' installed but reports as not installed.")
++
++if opkgcl.is_installed('c'):
++	opk.fail("Package 'c' should not have been installed since it was in --add-ignore-recommends.")
+-- 
+2.17.1
+
diff --git a/meta/recipes-devtools/opkg/opkg_0.3.6.bb b/meta/recipes-devtools/opkg/opkg_0.3.6.bb
index 6ebd58b967..b26d30b571 100644
--- a/meta/recipes-devtools/opkg/opkg_0.3.6.bb
+++ b/meta/recipes-devtools/opkg/opkg_0.3.6.bb
@@ -15,6 +15,7 @@ SRC_URI = "http://downloads.yoctoproject.org/releases/${BPN}/${BPN}-${PV}.tar.gz
            file://opkg.conf \
            file://0001-opkg_conf-create-opkg.lock-in-run-instead-of-var-run.patch \
            file://0001-remove_maintainer_scripts-use-strict-matching.patch \
+           file://0001-libopkg-add-add-ignore-recommends-option.patch \
 "
 
 SRC_URI[md5sum] = "79e04307f6f54db431c251772d7d987c"
-- 
2.17.1

[thud] [PATCH 2/2] OpkgPM: use --add-ignore-recommends to process BAD_RECOMMENDATIONS

[Quentin Schulz]

From: Alejandro del Castillo <alejandro.delcastillo@ni.com>

Currently, BAD_RECOMMENDATIONS on the opkg backed relies on editing the
opkg status file (it sets BAD_RECOMMENDATIONS pkg want state to
deinstalled and pinned). This is brittle, and not consistent across the
different solver backends. Use new --add-ignore-recommends flag instead.

(From OE-Core rev: 0d11e813ba9b4e8de9e6e5099ff85f5d914243bc)

Signed-off-by: Alejandro del Castillo <alejandro.delcastillo@ni.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
 meta/lib/oe/package_manager.py | 41 ++--------------------------------
 meta/lib/oe/rootfs.py          |  2 --
 2 files changed, 2 insertions(+), 41 deletions(-)

diff --git a/meta/lib/oe/package_manager.py b/meta/lib/oe/package_manager.py
index 882e7c429f..7d8804811c 100644
--- a/meta/lib/oe/package_manager.py
+++ b/meta/lib/oe/package_manager.py
@@ -1329,6 +1329,8 @@ class OpkgPM(OpkgDpkgPM):
         cmd = "%s %s" % (self.opkg_cmd, self.opkg_args)
         for exclude in (self.d.getVar("PACKAGE_EXCLUDE") or "").split():
             cmd += " --add-exclude %s" % exclude
+        for bad_recommendation in (self.d.getVar("BAD_RECOMMENDATIONS") or "").split():
+            cmd += " --add-ignore-recommends %s" % bad_recommendation
         cmd += " install "
         cmd += " ".join(pkgs)
 
@@ -1397,45 +1399,6 @@ class OpkgPM(OpkgDpkgPM):
     def list_installed(self):
         return OpkgPkgsList(self.d, self.target_rootfs, self.config_file).list_pkgs()
 
-    def handle_bad_recommendations(self):
-        bad_recommendations = self.d.getVar("BAD_RECOMMENDATIONS") or ""
-        if bad_recommendations.strip() == "":
-            return
-
-        status_file = os.path.join(self.opkg_dir, "status")
-
-        # If status file existed, it means the bad recommendations has already
-        # been handled
-        if os.path.exists(status_file):
-            return
-
-        cmd = "%s %s info " % (self.opkg_cmd, self.opkg_args)
-
-        with open(status_file, "w+") as status:
-            for pkg in bad_recommendations.split():
-                pkg_info = cmd + pkg
-
-                try:
-                    output = subprocess.check_output(pkg_info.split(), stderr=subprocess.STDOUT).strip().decode("utf-8")
-                except subprocess.CalledProcessError as e:
-                    bb.fatal("Cannot get package info. Command '%s' "
-                             "returned %d:\n%s" % (pkg_info, e.returncode, e.output.decode("utf-8")))
-
-                if output == "":
-                    bb.note("Ignored bad recommendation: '%s' is "
-                            "not a package" % pkg)
-                    continue
-
-                for line in output.split('\n'):
-                    if line.startswith("Status:"):
-                        status.write("Status: deinstall hold not-installed\n")
-                    else:
-                        status.write(line + "\n")
-
-                # Append a blank line after each package entry to ensure that it
-                # is separated from the following entry
-                status.write("\n")
-
     def dummy_install(self, pkgs):
         """
         The following function dummy installs pkgs and returns the log of output.
diff --git a/meta/lib/oe/rootfs.py b/meta/lib/oe/rootfs.py
index e5512d09ef..aa9fb2e0b5 100644
--- a/meta/lib/oe/rootfs.py
+++ b/meta/lib/oe/rootfs.py
@@ -879,8 +879,6 @@ class OpkgRootfs(DpkgOpkgRootfs):
 
         self.pm.update()
 
-        self.pm.handle_bad_recommendations()
-
         if self.progress_reporter:
             self.progress_reporter.next_stage()
 
-- 
2.17.1

Re: [thud] [PATCH 0/2] opkg and BAD_RECOMMENDATIONS in thud 2.6.2

[akuster808]

On 6/11/19 7:46 AM, Quentin Schulz wrote:
> Hi all,
>
> I've been hit with a rather unsettling bug when building an image with
> PACKAGE_CLASSES = "ipk" in Thud 2.6.2.
>
> The error message is the following:
> ERROR: meh-1.0-r0 do_rootfs: Unable to install packages. Command
> '/tmp/yocto/temp/work/machine-poky-linux/meh/1.0-r0/recipe-sysroot-native/usr/bin/opkg --volatile-cache -f /tmp/yocto/temp/work/machine-poky-linux/meh/1.0-r0/opkg.conf -t /tmp/yocto/temp/work/machine-poky-linux/meh/1.0-r0/temp/ipktemp/ -o /tmp/yocto/temp/work/machine-poky-linux/meh/1.0-r0/rootfs  --force_postinstall --prefer-arch-to-version   --force-maintainer --force-overwrite install packagegroup-base-extended packagegroup-core-boot plop run-postinsts' returned 2:
> Collected errors:
>  * Solver encountered 2 problem(s):
>  * Problem 1/2:
>  *   - nothing provides bluez5 needed by plop-1.0-r0.aarch64
>  * 
>  * Solution 1:
>  *   - do not ask to install a package providing plop
>
>  * Problem 2/2:
>  *   - nothing provides bluez5 needed by packagegroup-base-bluetooth-1.0-r83.stream1832prime
>  * 
>  * Solution 1:
>  *   - do not ask to install a package providing packagegroup-base-extended
>
>
>
>
> ERROR: meh-1.0-r0 do_rootfs: Function failed: do_rootfs
> ERROR: Logfile of failure stored in: /tmp/yocto/temp/work/machine-poky-linux/meh/1.0-r0/temp/log.do_rootfs.30869
> ERROR: Task (/media/qschulz/yocto/build/oe/recipes-images/nsdk/meh.bb:do_rootfs) failed with exit code '1'
> NOTE: Tasks Summary: Attempted 3059 tasks of which 3046 didn't need to be rerun and 1 failed.
>
> Summary: 1 task failed:
>   /media/qschulz/yocto/build/oe/recipes-images/nsdk/meh.bb:do_rootfs
> Summary: There were 7 WARNING messages shown.
> Summary: There were 2 ERROR messages shown, returning a non-zero exit code.
>
>
> I could reproduce it with an image meh.bb:
> inherit core-image
>
> CORE_IMAGE_EXTRA_INSTALL += "plop"
>
> BAD_RECOMMENDATIONS += "bluez5"
>
> with plop.bb being:
> DESCRIPTION = "Plap"
> LICENSE = "CLOSED"
>
> RDEPENDS_${PN} += "bluez5"
>
> do_install() {
>         install -d ${D}${bindir}
>         echo "plip" > ${D}${bindir}/plep
> }
>
> I took me a while to find out that it's an issue with opkg itself and
> nothing could be done in Yocto to fix it but I found the three following
> patches in yocto-2.7:
>
> https://git.yoctoproject.org/cgit.cgi/poky/commit/?id=d8e12f2bfc4deaa0dc7530cf3af905e18e8037f6
> https://git.yoctoproject.org/cgit.cgi/poky/commit/?id=c76c50df6498d85e7a6254002ff8f5e0ef3e67f7
> https://git.yoctoproject.org/cgit.cgi/poky/commit/?id=5bcb626cf49d2295ccca45d6fc7e1617cca932d6

You mentioned 3 git commits but i only see 2 patches on the list.

please clarify.

And Thanks for the fixes.

- Armin
>
> With those three applied on top of thud 2.6.2, my image is built without
> complaints.
>
> Please apply those backported patches for next Thud releases.
>
>
> Thanks,
> Quentin
>
> Alejandro del Castillo (2):
>   opkg: add --ignore-recommends flag
>   OpkgPM: use --add-ignore-recommends to process BAD_RECOMMENDATIONS
>
>  meta/lib/oe/package_manager.py                |  41 +--
>  meta/lib/oe/rootfs.py                         |   2 -
>  ...pkg-add-add-ignore-recommends-option.patch | 260 ++++++++++++++++++
>  meta/recipes-devtools/opkg/opkg_0.3.6.bb      |   1 +
>  4 files changed, 263 insertions(+), 41 deletions(-)
>  create mode 100644 meta/recipes-devtools/opkg/opkg/0001-libopkg-add-add-ignore-recommends-option.patch
>

Re: [thud] [PATCH 0/2] opkg and BAD_RECOMMENDATIONS in thud 2.6.2

[Quentin Schulz]

Hi Armin,

On Tue, Jun 11, 2019 at 10:24:20AM -0700, akuster808 wrote:
> 
> 
> On 6/11/19 7:46 AM, Quentin Schulz wrote:
[...]
> > I took me a while to find out that it's an issue with opkg itself and
> > nothing could be done in Yocto to fix it but I found the three following
> > patches in yocto-2.7:
> >
> > https://git.yoctoproject.org/cgit.cgi/poky/commit/?id=d8e12f2bfc4deaa0dc7530cf3af905e18e8037f6
> > https://git.yoctoproject.org/cgit.cgi/poky/commit/?id=c76c50df6498d85e7a6254002ff8f5e0ef3e67f7
> > https://git.yoctoproject.org/cgit.cgi/poky/commit/?id=5bcb626cf49d2295ccca45d6fc7e1617cca932d6
> 
> You mentioned 3 git commits but i only see 2 patches on the list.
> 
> please clarify.
> 

Ugh, yes. Sorry :)

These patches are the outcome of a discussion with Richard Purdie[1]
where he suggested that the upgrade commit is probably not a backport
candidate so I just went by it and removed it.

Quentin

[1] https://lists.yoctoproject.org/pipermail/poky/2019-May/011766.html
-- 
StreamUnlimited Engineering GmbH
High Tech Campus Vienna, Gutheil-Schoder-Gasse 10, 1100 Vienna, Austria
quentin.schulz@streamunlimited.com, www.streamunlimited.com

Re: [thud] [PATCH 0/2] opkg and BAD_RECOMMENDATIONS in thud 2.6.2

[Quentin Schulz]

Hi all,

On Wed, Jun 12, 2019 at 09:25:45AM +0200, Quentin Schulz wrote:
> Hi Armin,
> 
> On Tue, Jun 11, 2019 at 10:24:20AM -0700, akuster808 wrote:
> > 
> > 
> > On 6/11/19 7:46 AM, Quentin Schulz wrote:
> [...]
> > > I took me a while to find out that it's an issue with opkg itself and
> > > nothing could be done in Yocto to fix it but I found the three following
> > > patches in yocto-2.7:
> > >
> > > https://git.yoctoproject.org/cgit.cgi/poky/commit/?id=d8e12f2bfc4deaa0dc7530cf3af905e18e8037f6
> > > https://git.yoctoproject.org/cgit.cgi/poky/commit/?id=c76c50df6498d85e7a6254002ff8f5e0ef3e67f7
> > > https://git.yoctoproject.org/cgit.cgi/poky/commit/?id=5bcb626cf49d2295ccca45d6fc7e1617cca932d6
> > 
> > You mentioned 3 git commits but i only see 2 patches on the list.
> > 
> > please clarify.
> > 
> 
> Ugh, yes. Sorry :)
> 
> These patches are the outcome of a discussion with Richard Purdie[1]
> where he suggested that the upgrade commit is probably not a backport
> candidate so I just went by it and removed it.
> 

Kindly pinging so that this patch does not get lost at the bottom of the
inbox :)

Let me know if there's anything I can do,

Quentin
-- 
StreamUnlimited Engineering GmbH
High Tech Campus Vienna, Gutheil-Schoder-Gasse 10, 1100 Vienna, Austria
quentin.schulz@streamunlimited.com, www.streamunlimited.com

Re: [thud] [PATCH 0/2] opkg and BAD_RECOMMENDATIONS in thud 2.6.2

[Quentin Schulz]

Hi all,
   
On Mon, Jul 01, 2019 at 05:35:45PM +0200, Quentin Schulz wrote:
> Hi all,
> 
> On Wed, Jun 12, 2019 at 09:25:45AM +0200, Quentin Schulz wrote:
> > Hi Armin,
> > 
> > On Tue, Jun 11, 2019 at 10:24:20AM -0700, akuster808 wrote:
> > > 
> > > 
> > > On 6/11/19 7:46 AM, Quentin Schulz wrote:
> > [...]
> > > > I took me a while to find out that it's an issue with opkg itself and
> > > > nothing could be done in Yocto to fix it but I found the three following
> > > > patches in yocto-2.7:
> > > >
> > > > https://git.yoctoproject.org/cgit.cgi/poky/commit/?id=d8e12f2bfc4deaa0dc7530cf3af905e18e8037f6
> > > > https://git.yoctoproject.org/cgit.cgi/poky/commit/?id=c76c50df6498d85e7a6254002ff8f5e0ef3e67f7
> > > > https://git.yoctoproject.org/cgit.cgi/poky/commit/?id=5bcb626cf49d2295ccca45d6fc7e1617cca932d6
> > > 
> > > You mentioned 3 git commits but i only see 2 patches on the list.
> > > 
> > > please clarify.
> > > 
> > 
> > Ugh, yes. Sorry :)
> > 
> > These patches are the outcome of a discussion with Richard Purdie[1]
> > where he suggested that the upgrade commit is probably not a backport
> > candidate so I just went by it and removed it.
> > 
> 
> Kindly pinging so that this patch does not get lost at the bottom of the
> inbox :)
> 

I haven't seen the patches in thud or thud-next nor in the latest series
of patches to be reviewed for thud and there isn't an ongoing discussion
on those two patches, so kindly pinging.

Let me know if there's anything I can do,
Quentin
-- 
StreamUnlimited Engineering GmbH
High Tech Campus Vienna, Gutheil-Schoder-Gasse 10, 1100 Vienna, Austria
quentin.schulz@streamunlimited.com, www.streamunlimited.com

Re: [thud] [PATCH 0/2] opkg and BAD_RECOMMENDATIONS in thud 2.6.2

[Quentin Schulz]

Hi,

I can see both patches in yocto-2.6.3. Thanks for backporting.

Have a good day,
Quentin
-- 
StreamUnlimited Engineering GmbH
High Tech Campus Vienna, Gutheil-Schoder-Gasse 10, 1100 Vienna, Austria
quentin.schulz@streamunlimited.com, www.streamunlimited.com

Re: [thud] [PATCH 0/2] opkg and BAD_RECOMMENDATIONS in thud 2.6.2

[Sean Nyekjaer]

On 11/06/2019 16.46, Quentin Schulz wrote:
> Hi all,
> 
> I've been hit with a rather unsettling bug when building an image with
> PACKAGE_CLASSES = "ipk" in Thud 2.6.2.
> 
> The error message is the following:
> ERROR: meh-1.0-r0 do_rootfs: Unable to install packages. Command
> '/tmp/yocto/temp/work/machine-poky-linux/meh/1.0-r0/recipe-sysroot-native/usr/bin/opkg --volatile-cache -f /tmp/yocto/temp/work/machine-poky-linux/meh/1.0-r0/opkg.conf -t /tmp/yocto/temp/work/machine-poky-linux/meh/1.0-r0/temp/ipktemp/ -o /tmp/yocto/temp/work/machine-poky-linux/meh/1.0-r0/rootfs  --force_postinstall --prefer-arch-to-version   --force-maintainer --force-overwrite install packagegroup-base-extended packagegroup-core-boot plop run-postinsts' returned 2:
> Collected errors:
>   * Solver encountered 2 problem(s):
>   * Problem 1/2:
>   *   - nothing provides bluez5 needed by plop-1.0-r0.aarch64
>   *
>   * Solution 1:
>   *   - do not ask to install a package providing plop
> 
>   * Problem 2/2:
>   *   - nothing provides bluez5 needed by packagegroup-base-bluetooth-1.0-r83.stream1832prime
>   *
>   * Solution 1:
>   *   - do not ask to install a package providing packagegroup-base-extended

Hi all,

After this series was applied to thud, our BAD_RECOMMENDATIONS is 
included in our rootfs :(

We have in our distro conf:
BAD_RECOMMENDATIONS_append = " udev-hwdb"

If I revert this series on current thud HEAD, we are back to what is 
expected :) An image without udev-hwdb

-- 

Best regards,

Sean Nyekjær

Embedded Linux Consultant



+45 42427326

sean@geanix.com



Geanix ApS

https://geanix.com

DK39600706

Re: [thud] [PATCH 0/2] opkg and BAD_RECOMMENDATIONS in thud 2.6.2

[Quentin Schulz]

Hi Sean,

On Wed, Oct 16, 2019 at 10:27:23AM +0200, Sean Nyekjaer wrote:
> 
> 
> On 11/06/2019 16.46, Quentin Schulz wrote:
> > Hi all,
> > 
> > I've been hit with a rather unsettling bug when building an image with
> > PACKAGE_CLASSES = "ipk" in Thud 2.6.2.
> > 
> > The error message is the following:
> > ERROR: meh-1.0-r0 do_rootfs: Unable to install packages. Command
> > '/tmp/yocto/temp/work/machine-poky-linux/meh/1.0-r0/recipe-sysroot-native/usr/bin/opkg --volatile-cache -f /tmp/yocto/temp/work/machine-poky-linux/meh/1.0-r0/opkg.conf -t /tmp/yocto/temp/work/machine-poky-linux/meh/1.0-r0/temp/ipktemp/ -o /tmp/yocto/temp/work/machine-poky-linux/meh/1.0-r0/rootfs  --force_postinstall --prefer-arch-to-version   --force-maintainer --force-overwrite install packagegroup-base-extended packagegroup-core-boot plop run-postinsts' returned 2:
> > Collected errors:
> >   * Solver encountered 2 problem(s):
> >   * Problem 1/2:
> >   *   - nothing provides bluez5 needed by plop-1.0-r0.aarch64
> >   *
> >   * Solution 1:
> >   *   - do not ask to install a package providing plop
> > 
> >   * Problem 2/2:
> >   *   - nothing provides bluez5 needed by packagegroup-base-bluetooth-1.0-r83.stream1832prime
> >   *
> >   * Solution 1:
> >   *   - do not ask to install a package providing packagegroup-base-extended
> 
> Hi all,
> 
> After this series was applied to thud, our BAD_RECOMMENDATIONS is included
> in our rootfs :(
> 
> We have in our distro conf:
> BAD_RECOMMENDATIONS_append = " udev-hwdb"
> 
> If I revert this series on current thud HEAD, we are back to what is
> expected :) An image without udev-hwdb
> 

Thanks for letting the community know. I can confirm we're having the
same issue here though I haven't taken the time to check whether it's
because of this series. I believe you though :)

I think there isn't much more than a week left before the last dot
version of thud is out making me say that we should probably revert this
series.

The reasoning is the following:
Situation before the patch series, as explained in the cover letter:
in some specific cases, a few packages in BAD_RECOMMENDATIONS trigger a
build error (see above) but overall the mechanism works.
Situation after the patch series (2.6.3): BAD_RECOMMENDATIONS is not
taken into account at all (but build is working, but not a surprise per
se since BAD_RECOMMENDATIONS was the culprit).

Now it's down to maintainers but I'd advocate for reverting it if we
can't fix it in the meantime because I personnally would rather have the
same buggy behavior (but working BAD_RECOMMENDATIONS in 99% of the time)
from 2.6.0, 2.6.1 and 2.6.2 than having a BAD_RECOMMENDATIONS not
working at all as it is for 2.6.3.

Thanks for reporting.
Sorry for messing up.

Quentin
-- 
StreamUnlimited Engineering GmbH
High Tech Campus Vienna, Gutheil-Schoder-Gasse 10, 1100 Vienna, Austria
Fax: +43 1 667 20 02 4401
quentin.schulz@streamunlimited.com, www.streamunlimited.com

Re: [thud] [PATCH 0/2] opkg and BAD_RECOMMENDATIONS in thud 2.6.2

[richard.purdie]

On Wed, 2019-10-16 at 10:27 +0200, Sean Nyekjaer wrote:
> 
> On 11/06/2019 16.46, Quentin Schulz wrote:
> > Hi all,
> > 
> > I've been hit with a rather unsettling bug when building an image
> > with
> > PACKAGE_CLASSES = "ipk" in Thud 2.6.2.
> > 
> > The error message is the following:
> > ERROR: meh-1.0-r0 do_rootfs: Unable to install packages. Command
> > '/tmp/yocto/temp/work/machine-poky-linux/meh/1.0-r0/recipe-sysroot-
> > native/usr/bin/opkg --volatile-cache -f
> > /tmp/yocto/temp/work/machine-poky-linux/meh/1.0-r0/opkg.conf -t
> > /tmp/yocto/temp/work/machine-poky-linux/meh/1.0-r0/temp/ipktemp/ -o
> > /tmp/yocto/temp/work/machine-poky-linux/meh/1.0-r0/rootfs  
> > --force_postinstall --prefer-arch-to-version   --force-maintainer
> > --force-overwrite install packagegroup-base-extended packagegroup-
> > core-boot plop run-postinsts' returned 2:
> > Collected errors:
> >   * Solver encountered 2 problem(s):
> >   * Problem 1/2:
> >   *   - nothing provides bluez5 needed by plop-1.0-r0.aarch64
> >   *
> >   * Solution 1:
> >   *   - do not ask to install a package providing plop
> > 
> >   * Problem 2/2:
> >   *   - nothing provides bluez5 needed by packagegroup-base-
> > bluetooth-1.0-r83.stream1832prime
> >   *
> >   * Solution 1:
> >   *   - do not ask to install a package providing packagegroup-
> > base-extended
> 
> Hi all,
> 
> After this series was applied to thud, our BAD_RECOMMENDATIONS is 
> included in our rootfs :(
> 
> We have in our distro conf:
> BAD_RECOMMENDATIONS_append = " udev-hwdb"
> 
> If I revert this series on current thud HEAD, we are back to what is 
> expected :) An image without udev-hwdb

Would someone be able to confirm which patches we need to revert
please? It may also help narrow down the problem if we could figure out
which patches specifically need to be reverted to fix it.

It'd be easier if someone who can reproduce the issue easily could do
that...

Thanks,

Richard

Re: [thud] [PATCH 0/2] opkg and BAD_RECOMMENDATIONS in thud 2.6.2

[Quentin Schulz]

Hi Richard,

On Thu, Oct 17, 2019 at 09:23:18AM +0100, richard.purdie@linuxfoundation.org wrote:
> On Wed, 2019-10-16 at 10:27 +0200, Sean Nyekjaer wrote:
> > 
> > On 11/06/2019 16.46, Quentin Schulz wrote:
> > > Hi all,
> > > 
> > > I've been hit with a rather unsettling bug when building an image
> > > with
> > > PACKAGE_CLASSES = "ipk" in Thud 2.6.2.
> > > 
> > > The error message is the following:
> > > ERROR: meh-1.0-r0 do_rootfs: Unable to install packages. Command
> > > '/tmp/yocto/temp/work/machine-poky-linux/meh/1.0-r0/recipe-sysroot-
> > > native/usr/bin/opkg --volatile-cache -f
> > > /tmp/yocto/temp/work/machine-poky-linux/meh/1.0-r0/opkg.conf -t
> > > /tmp/yocto/temp/work/machine-poky-linux/meh/1.0-r0/temp/ipktemp/ -o
> > > /tmp/yocto/temp/work/machine-poky-linux/meh/1.0-r0/rootfs  
> > > --force_postinstall --prefer-arch-to-version   --force-maintainer
> > > --force-overwrite install packagegroup-base-extended packagegroup-
> > > core-boot plop run-postinsts' returned 2:
> > > Collected errors:
> > >   * Solver encountered 2 problem(s):
> > >   * Problem 1/2:
> > >   *   - nothing provides bluez5 needed by plop-1.0-r0.aarch64
> > >   *
> > >   * Solution 1:
> > >   *   - do not ask to install a package providing plop
> > > 
> > >   * Problem 2/2:
> > >   *   - nothing provides bluez5 needed by packagegroup-base-
> > > bluetooth-1.0-r83.stream1832prime
> > >   *
> > >   * Solution 1:
> > >   *   - do not ask to install a package providing packagegroup-
> > > base-extended
> > 
> > Hi all,
> > 
> > After this series was applied to thud, our BAD_RECOMMENDATIONS is 
> > included in our rootfs :(
> > 
> > We have in our distro conf:
> > BAD_RECOMMENDATIONS_append = " udev-hwdb"
> > 
> > If I revert this series on current thud HEAD, we are back to what is 
> > expected :) An image without udev-hwdb
> 
> Would someone be able to confirm which patches we need to revert
> please? It may also help narrow down the problem if we could figure out
> which patches specifically need to be reverted to fix it.
> 

@Sean, please confirm so I don't mess up twice in a row :)

e8cd30ba6cec854d85c7ad47edc208107858a5d7 ("OpkgPM: use --add-ignore-recommends to process BAD_RECOMMENDATIONS")
is the one which should be reverted. This commit cannot work without the
one mentioned below because it relies on opkg supporting
--add-ignore-recommends on the command line.

Without this commit,
eecc4121ad3efac4242b023245bccc21439ecc1f ("opkg: add --ignore-recommends flag")
is the equivalent of no-op because nothing will call
--add-ignore-recommends (and anyway, we couldn't use it because, as stated in
the commit log of the patch in that commit, it is compatible only with
libsolv > 0.7.2 which isn't the case in thud).

So first one for sure, the second for cleaning up/not confusing people
if need be.

Quentin
-- 
StreamUnlimited Engineering GmbH
High Tech Campus Vienna, Gutheil-Schoder-Gasse 10, 1100 Vienna, Austria
Fax: +43 1 667 20 02 4401
quentin.schulz@streamunlimited.com, www.streamunlimited.com

Re: [thud] [PATCH 0/2] opkg and BAD_RECOMMENDATIONS in thud 2.6.2

[Sean Nyekjaer]

On 17/10/2019 10.54, Quentin Schulz wrote:

..

>> Would someone be able to confirm which patches we need to revert
>> please? It may also help narrow down the problem if we could figure out
>> which patches specifically need to be reverted to fix it.
>>
> 
> @Sean, please confirm so I don't mess up twice in a row :)
> 
> e8cd30ba6cec854d85c7ad47edc208107858a5d7 ("OpkgPM: use --add-ignore-recommends to process BAD_RECOMMENDATIONS")
> is the one which should be reverted. This commit cannot work without the
> one mentioned below because it relies on opkg supporting
> --add-ignore-recommends on the command line.

I can comfirm it works as expected when:

e8cd30ba6cec854d85c7ad47edc208107858a5d7 ("OpkgPM: use 
--add-ignore-recommends to process BAD_RECOMMENDATIONS")

is reverted :-)

But had to wipe my `tmp` to prove it. Seems like some changes are not 
picked up properly.

/Sean