* [PATCH] package management: Allow dynamic loading of PM
@ 2020-09-08 10:53 Fredrik Gustafsson
2020-09-08 11:25 ` [OE-core] " Richard Purdie
0 siblings, 1 reply; 3+ messages in thread
From: Fredrik Gustafsson @ 2020-09-08 10:53 UTC (permalink / raw)
To: fredrigu; +Cc: openembedded-core, tools-cfpbuild-internal
Dynamic loading of package managers will allow other layers to simply
add their package manager code in package_manager/ and have bitbake find
it according to the package manager configuration. This is useful for
adding new (faster) package managers to Open Embedded while not increasing the
test scope or require Open Embedded to support more package managers.
How this is tested:
* Build core-image-minimal with all three package managers
* Build the sdk with all three package managers. dpkg fails, but
it fails on master as well.
* Run the complete test suite, all tests passed except 16
* Run those 16 tests on master and verify that they fail there as well
* Fix errors making tests works on master but not with this patch.
Signed-off-by: Fredrik Gustafsson <fredrigu@axis.com>
---
meta/lib/oe/manifest.py | 10 ++----
meta/lib/oe/package_manager/deb/__init__.py | 4 +--
meta/lib/oe/package_manager/deb/manifest.py | 2 +-
meta/lib/oe/package_manager/deb/rootfs.py | 8 ++---
meta/lib/oe/package_manager/deb/sdk.py | 10 +++---
meta/lib/oe/package_manager/ipk/__init__.py | 9 +++---
meta/lib/oe/package_manager/ipk/manifest.py | 2 +-
meta/lib/oe/package_manager/ipk/rootfs.py | 8 ++---
meta/lib/oe/package_manager/ipk/sdk.py | 10 +++---
meta/lib/oe/package_manager/rpm/__init__.py | 2 +-
meta/lib/oe/package_manager/rpm/manifest.py | 2 +-
meta/lib/oe/package_manager/rpm/rootfs.py | 8 ++---
meta/lib/oe/package_manager/rpm/sdk.py | 10 +++---
meta/lib/oe/rootfs.py | 36 ++++++---------------
meta/lib/oe/sdk.py | 27 ++++------------
15 files changed, 55 insertions(+), 93 deletions(-)
diff --git a/meta/lib/oe/manifest.py b/meta/lib/oe/manifest.py
index 47bd622412..1a058dcd73 100644
--- a/meta/lib/oe/manifest.py
+++ b/meta/lib/oe/manifest.py
@@ -191,14 +191,8 @@ class Manifest(object, metaclass=ABCMeta):
def create_manifest(d, final_manifest=False, manifest_dir=None,
manifest_type=Manifest.MANIFEST_TYPE_IMAGE):
- from oe.package_manager.rpm.manifest import RpmManifest
- from oe.package_manager.ipk.manifest import OpkgManifest
- from oe.package_manager.deb.manifest import DpkgManifest
- manifest_map = {'rpm': RpmManifest,
- 'ipk': OpkgManifest,
- 'deb': DpkgManifest}
-
- manifest = manifest_map[d.getVar('IMAGE_PKGTYPE')](d, manifest_dir, manifest_type)
+ import importlib
+ manifest = importlib.import_module('oe.package_manager.' + d.getVar('IMAGE_PKGTYPE') + '.manifest').PkgManifest(d, manifest_dir, manifest_type)
if final_manifest:
manifest.create_final()
diff --git a/meta/lib/oe/package_manager/deb/__init__.py b/meta/lib/oe/package_manager/deb/__init__.py
index 72155b178c..67c241a6f6 100644
--- a/meta/lib/oe/package_manager/deb/__init__.py
+++ b/meta/lib/oe/package_manager/deb/__init__.py
@@ -79,7 +79,7 @@ class DpkgIndexer(Indexer):
if self.d.getVar('PACKAGE_FEED_SIGN') == '1':
raise NotImplementedError('Package feed signing not implementd for dpkg')
-class DpkgPkgsList(PkgsList):
+class PMPkgsList(PkgsList):
def list_pkgs(self):
cmd = [bb.utils.which(os.getenv('PATH'), "dpkg-query"),
@@ -459,7 +459,7 @@ class DpkgPM(OpkgDpkgPM):
"returned %d:\n%s" % (cmd, e.returncode, e.output.decode("utf-8")))
def list_installed(self):
- return DpkgPkgsList(self.d, self.target_rootfs).list_pkgs()
+ return PMPkgsList(self.d, self.target_rootfs).list_pkgs()
def package_info(self, pkg):
"""
diff --git a/meta/lib/oe/package_manager/deb/manifest.py b/meta/lib/oe/package_manager/deb/manifest.py
index 0b12036644..d8eab24a06 100644
--- a/meta/lib/oe/package_manager/deb/manifest.py
+++ b/meta/lib/oe/package_manager/deb/manifest.py
@@ -4,7 +4,7 @@
from oe.manifest import Manifest
-class DpkgManifest(Manifest):
+class PkgManifest(Manifest):
def create_initial(self):
with open(self.initial_manifest, "w+") as manifest:
manifest.write(self.initial_manifest_file_header)
diff --git a/meta/lib/oe/package_manager/deb/rootfs.py b/meta/lib/oe/package_manager/deb/rootfs.py
index 819f67eda5..8fbaca11d6 100644
--- a/meta/lib/oe/package_manager/deb/rootfs.py
+++ b/meta/lib/oe/package_manager/deb/rootfs.py
@@ -7,7 +7,7 @@ import shutil
from oe.rootfs import Rootfs
from oe.manifest import Manifest
from oe.utils import execute_pre_post_process
-from oe.package_manager.deb.manifest import DpkgManifest
+from oe.package_manager.deb.manifest import PkgManifest
from oe.package_manager.deb import DpkgPM
class DpkgOpkgRootfs(Rootfs):
@@ -120,9 +120,9 @@ class DpkgOpkgRootfs(Rootfs):
num += 1
-class DpkgRootfs(DpkgOpkgRootfs):
+class PkgRootfs(DpkgOpkgRootfs):
def __init__(self, d, manifest_dir, progress_reporter=None, logcatcher=None):
- super(DpkgRootfs, self).__init__(d, progress_reporter, logcatcher)
+ super(PkgRootfs, self).__init__(d, progress_reporter, logcatcher)
self.log_check_regex = '^E:'
self.log_check_expected_regexes = \
[
@@ -131,7 +131,7 @@ class DpkgRootfs(DpkgOpkgRootfs):
bb.utils.remove(self.image_rootfs, True)
bb.utils.remove(self.d.getVar('MULTILIB_TEMP_ROOTFS'), True)
- self.manifest = DpkgManifest(d, manifest_dir)
+ self.manifest = PkgManifest(d, manifest_dir)
self.pm = DpkgPM(d, d.getVar('IMAGE_ROOTFS'),
d.getVar('PACKAGE_ARCHS'),
d.getVar('DPKG_ARCH'))
diff --git a/meta/lib/oe/package_manager/deb/sdk.py b/meta/lib/oe/package_manager/deb/sdk.py
index b25eb70b00..9859d8f32d 100644
--- a/meta/lib/oe/package_manager/deb/sdk.py
+++ b/meta/lib/oe/package_manager/deb/sdk.py
@@ -8,19 +8,19 @@ from oe.utils import execute_pre_post_process
from oe.sdk import Sdk
from oe.manifest import Manifest
from oe.package_manager.deb import DpkgPM
+from oe.package_manager.deb.manifest import PkgManifest
-class DpkgSdk(Sdk):
+class PkgSdk(Sdk):
def __init__(self, d, manifest_dir=None):
- super(DpkgSdk, self).__init__(d, manifest_dir)
+ super(PkgSdk, self).__init__(d, manifest_dir)
self.target_conf_dir = os.path.join(self.d.getVar("APTCONF_TARGET"), "apt")
self.host_conf_dir = os.path.join(self.d.getVar("APTCONF_TARGET"), "apt-sdk")
- from oe.package_manager.deb.manifest import DpkgManifest
- self.target_manifest = DpkgManifest(d, self.manifest_dir,
+ self.target_manifest = PkgManifest(d, self.manifest_dir,
Manifest.MANIFEST_TYPE_SDK_TARGET)
- self.host_manifest = DpkgManifest(d, self.manifest_dir,
+ self.host_manifest = PkgManifest(d, self.manifest_dir,
Manifest.MANIFEST_TYPE_SDK_HOST)
deb_repo_workdir = "oe-sdk-repo"
diff --git a/meta/lib/oe/package_manager/ipk/__init__.py b/meta/lib/oe/package_manager/ipk/__init__.py
index 9603993a59..416ed23d47 100644
--- a/meta/lib/oe/package_manager/ipk/__init__.py
+++ b/meta/lib/oe/package_manager/ipk/__init__.py
@@ -59,9 +59,10 @@ class OpkgIndexer(Indexer):
self.d.getVar('PACKAGE_FEED_GPG_PASSPHRASE_FILE'),
armor=is_ascii_sig)
-class OpkgPkgsList(PkgsList):
- def __init__(self, d, rootfs_dir, config_file):
- super(OpkgPkgsList, self).__init__(d, rootfs_dir)
+class PMPkgsList(PkgsList):
+ def __init__(self, d, rootfs_dir):
+ super(PMPkgsList, self).__init__(d, rootfs_dir)
+ config_file = d.getVar("IPKGCONF_TARGET")
self.opkg_cmd = bb.utils.which(os.getenv('PATH'), "opkg")
self.opkg_args = "-f %s -o %s " % (config_file, rootfs_dir)
@@ -416,7 +417,7 @@ class OpkgPM(OpkgDpkgPM):
bb.utils.remove(os.path.join(self.opkg_dir, "lists"), True)
def list_installed(self):
- return OpkgPkgsList(self.d, self.target_rootfs, self.config_file).list_pkgs()
+ return PMPkgsList(self.d, self.target_rootfs).list_pkgs()
def dummy_install(self, pkgs):
"""
diff --git a/meta/lib/oe/package_manager/ipk/manifest.py b/meta/lib/oe/package_manager/ipk/manifest.py
index 69676903ab..ee4b57bcb0 100644
--- a/meta/lib/oe/package_manager/ipk/manifest.py
+++ b/meta/lib/oe/package_manager/ipk/manifest.py
@@ -4,7 +4,7 @@
from oe.manifest import Manifest
-class OpkgManifest(Manifest):
+class PkgManifest(Manifest):
"""
Returns a dictionary object with mip and mlp packages.
"""
diff --git a/meta/lib/oe/package_manager/ipk/rootfs.py b/meta/lib/oe/package_manager/ipk/rootfs.py
index 63b4a59c40..26dbee6f6a 100644
--- a/meta/lib/oe/package_manager/ipk/rootfs.py
+++ b/meta/lib/oe/package_manager/ipk/rootfs.py
@@ -8,7 +8,7 @@ import shutil
from oe.rootfs import Rootfs
from oe.manifest import Manifest
from oe.utils import execute_pre_post_process
-from oe.package_manager.ipk.manifest import OpkgManifest
+from oe.package_manager.ipk.manifest import PkgManifest
from oe.package_manager.ipk import OpkgPM
class DpkgOpkgRootfs(Rootfs):
@@ -121,12 +121,12 @@ class DpkgOpkgRootfs(Rootfs):
num += 1
-class OpkgRootfs(DpkgOpkgRootfs):
+class PkgRootfs(DpkgOpkgRootfs):
def __init__(self, d, manifest_dir, progress_reporter=None, logcatcher=None):
- super(OpkgRootfs, self).__init__(d, progress_reporter, logcatcher)
+ super(PkgRootfs, self).__init__(d, progress_reporter, logcatcher)
self.log_check_regex = '(exit 1|Collected errors)'
- self.manifest = OpkgManifest(d, manifest_dir)
+ self.manifest = PkgManifest(d, manifest_dir)
self.opkg_conf = self.d.getVar("IPKGCONF_TARGET")
self.pkg_archs = self.d.getVar("ALL_MULTILIB_PACKAGE_ARCHS")
diff --git a/meta/lib/oe/package_manager/ipk/sdk.py b/meta/lib/oe/package_manager/ipk/sdk.py
index 47c0a92c1b..37af0344eb 100644
--- a/meta/lib/oe/package_manager/ipk/sdk.py
+++ b/meta/lib/oe/package_manager/ipk/sdk.py
@@ -6,20 +6,20 @@ import glob
import shutil
from oe.utils import execute_pre_post_process
from oe.sdk import Sdk
+from oe.package_manager.ipk.manifest import PkgManifest
from oe.manifest import Manifest
from oe.package_manager.ipk import OpkgPM
-class OpkgSdk(Sdk):
+class PkgSdk(Sdk):
def __init__(self, d, manifest_dir=None):
- super(OpkgSdk, self).__init__(d, manifest_dir)
+ super(PkgSdk, self).__init__(d, manifest_dir)
self.target_conf = self.d.getVar("IPKGCONF_TARGET")
self.host_conf = self.d.getVar("IPKGCONF_SDK")
- from oe.package_manager.ipk.manifest import OpkgManifest
- self.target_manifest = OpkgManifest(d, self.manifest_dir,
+ self.target_manifest = PkgManifest(d, self.manifest_dir,
Manifest.MANIFEST_TYPE_SDK_TARGET)
- self.host_manifest = OpkgManifest(d, self.manifest_dir,
+ self.host_manifest = PkgManifest(d, self.manifest_dir,
Manifest.MANIFEST_TYPE_SDK_HOST)
ipk_repo_workdir = "oe-sdk-repo"
diff --git a/meta/lib/oe/package_manager/rpm/__init__.py b/meta/lib/oe/package_manager/rpm/__init__.py
index c91f61ae5c..898184442f 100644
--- a/meta/lib/oe/package_manager/rpm/__init__.py
+++ b/meta/lib/oe/package_manager/rpm/__init__.py
@@ -43,7 +43,7 @@ class RpmSubdirIndexer(RpmIndexer):
self.do_write_index(dir_path)
-class RpmPkgsList(PkgsList):
+class PMPkgsList(PkgsList):
def list_pkgs(self):
return RpmPM(self.d, self.rootfs_dir, self.d.getVar('TARGET_VENDOR'), needfeed=False).list_installed()
diff --git a/meta/lib/oe/package_manager/rpm/manifest.py b/meta/lib/oe/package_manager/rpm/manifest.py
index a75f6bdabf..e6604b301f 100644
--- a/meta/lib/oe/package_manager/rpm/manifest.py
+++ b/meta/lib/oe/package_manager/rpm/manifest.py
@@ -4,7 +4,7 @@
from oe.manifest import Manifest
-class RpmManifest(Manifest):
+class PkgManifest(Manifest):
"""
Returns a dictionary object with mip and mlp packages.
"""
diff --git a/meta/lib/oe/package_manager/rpm/rootfs.py b/meta/lib/oe/package_manager/rpm/rootfs.py
index 2de5752b91..00d07cd9cc 100644
--- a/meta/lib/oe/package_manager/rpm/rootfs.py
+++ b/meta/lib/oe/package_manager/rpm/rootfs.py
@@ -5,17 +5,17 @@
from oe.rootfs import Rootfs
from oe.manifest import Manifest
from oe.utils import execute_pre_post_process
-from oe.package_manager.rpm.manifest import RpmManifest
+from oe.package_manager.rpm.manifest import PkgManifest
from oe.package_manager.rpm import RpmPM
-class RpmRootfs(Rootfs):
+class PkgRootfs(Rootfs):
def __init__(self, d, manifest_dir, progress_reporter=None, logcatcher=None):
- super(RpmRootfs, self).__init__(d, progress_reporter, logcatcher)
+ super(PkgRootfs, self).__init__(d, progress_reporter, logcatcher)
self.log_check_regex = r'(unpacking of archive failed|Cannot find package'\
r'|exit 1|ERROR: |Error: |Error |ERROR '\
r'|Failed |Failed: |Failed$|Failed\(\d+\):)'
- self.manifest = RpmManifest(d, manifest_dir)
+ self.manifest = PkgManifest(d, manifest_dir)
self.pm = RpmPM(d,
d.getVar('IMAGE_ROOTFS'),
diff --git a/meta/lib/oe/package_manager/rpm/sdk.py b/meta/lib/oe/package_manager/rpm/sdk.py
index b14b155a85..c5f232431f 100644
--- a/meta/lib/oe/package_manager/rpm/sdk.py
+++ b/meta/lib/oe/package_manager/rpm/sdk.py
@@ -6,16 +6,16 @@ import glob
from oe.utils import execute_pre_post_process
from oe.sdk import Sdk
from oe.manifest import Manifest
+from oe.package_manager.rpm.manifest import PkgManifest
from oe.package_manager.rpm import RpmPM
-class RpmSdk(Sdk):
+class PkgSdk(Sdk):
def __init__(self, d, manifest_dir=None, rpm_workdir="oe-sdk-repo"):
- super(RpmSdk, self).__init__(d, manifest_dir)
+ super(PkgSdk, self).__init__(d, manifest_dir)
- from oe.package_manager.rpm.manifest import RpmManifest
- self.target_manifest = RpmManifest(d, self.manifest_dir,
+ self.target_manifest = PkgManifest(d, self.manifest_dir,
Manifest.MANIFEST_TYPE_SDK_TARGET)
- self.host_manifest = RpmManifest(d, self.manifest_dir,
+ self.host_manifest = PkgManifest(d, self.manifest_dir,
Manifest.MANIFEST_TYPE_SDK_HOST)
rpm_repo_workdir = "oe-sdk-repo"
diff --git a/meta/lib/oe/rootfs.py b/meta/lib/oe/rootfs.py
index 3813f68e8b..ac300828a1 100644
--- a/meta/lib/oe/rootfs.py
+++ b/meta/lib/oe/rootfs.py
@@ -10,12 +10,6 @@ import shutil
import os
import subprocess
import re
-from oe.package_manager.rpm.manifest import RpmManifest
-from oe.package_manager.ipk.manifest import OpkgManifest
-from oe.package_manager.deb.manifest import DpkgManifest
-from oe.package_manager.rpm import RpmPkgsList
-from oe.package_manager.ipk import OpkgPkgsList
-from oe.package_manager.deb import DpkgPkgsList
class Rootfs(object, metaclass=ABCMeta):
"""
@@ -358,12 +352,9 @@ class Rootfs(object, metaclass=ABCMeta):
def get_class_for_type(imgtype):
- from oe.package_manager.rpm.rootfs import RpmRootfs
- from oe.package_manager.ipk.rootfs import OpkgRootfs
- from oe.package_manager.deb.rootfs import DpkgRootfs
- return {"rpm": RpmRootfs,
- "ipk": OpkgRootfs,
- "deb": DpkgRootfs}[imgtype]
+ import importlib
+ mod = importlib.import_module('oe.package_manager.' + imgtype + '.rootfs')
+ return mod.PkgRootfs
def variable_depends(d, manifest_dir=None):
img_type = d.getVar('IMAGE_PKGTYPE')
@@ -373,17 +364,10 @@ def variable_depends(d, manifest_dir=None):
def create_rootfs(d, manifest_dir=None, progress_reporter=None, logcatcher=None):
env_bkp = os.environ.copy()
- from oe.package_manager.rpm.rootfs import RpmRootfs
- from oe.package_manager.ipk.rootfs import OpkgRootfs
- from oe.package_manager.deb.rootfs import DpkgRootfs
img_type = d.getVar('IMAGE_PKGTYPE')
- if img_type == "rpm":
- RpmRootfs(d, manifest_dir, progress_reporter, logcatcher).create()
- elif img_type == "ipk":
- OpkgRootfs(d, manifest_dir, progress_reporter, logcatcher).create()
- elif img_type == "deb":
- DpkgRootfs(d, manifest_dir, progress_reporter, logcatcher).create()
+ cls = get_class_for_type(img_type)
+ cls(d, manifest_dir, progress_reporter, logcatcher).create()
os.environ.clear()
os.environ.update(env_bkp)
@@ -393,12 +377,10 @@ def image_list_installed_packages(d, rootfs_dir=None):
rootfs_dir = d.getVar('IMAGE_ROOTFS')
img_type = d.getVar('IMAGE_PKGTYPE')
- if img_type == "rpm":
- return RpmPkgsList(d, rootfs_dir).list_pkgs()
- elif img_type == "ipk":
- return OpkgPkgsList(d, rootfs_dir, d.getVar("IPKGCONF_TARGET")).list_pkgs()
- elif img_type == "deb":
- return DpkgPkgsList(d, rootfs_dir).list_pkgs()
+
+ import importlib
+ cls = importlib.import_module('oe.package_manager.' + img_type)
+ return cls.PMPkgsList(d, rootfs_dir).list_pkgs()
if __name__ == "__main__":
"""
diff --git a/meta/lib/oe/sdk.py b/meta/lib/oe/sdk.py
index fdcadcb8de..37b59afd1a 100644
--- a/meta/lib/oe/sdk.py
+++ b/meta/lib/oe/sdk.py
@@ -115,33 +115,18 @@ def sdk_list_installed_packages(d, target, rootfs_dir=None):
rootfs_dir = [sdk_output, os.path.join(sdk_output, target_path)][target is True]
- from oe.package_manager.rpm import RpmPkgsList
- from oe.package_manager.ipk import OpkgPkgsList
- from oe.package_manager.deb import DpkgPkgsList
img_type = d.getVar('IMAGE_PKGTYPE')
- if img_type == "rpm":
- arch_var = ["SDK_PACKAGE_ARCHS", None][target is True]
- os_var = ["SDK_OS", None][target is True]
- return RpmPkgsList(d, rootfs_dir).list_pkgs()
- elif img_type == "ipk":
- conf_file_var = ["IPKGCONF_SDK", "IPKGCONF_TARGET"][target is True]
- return OpkgPkgsList(d, rootfs_dir, d.getVar(conf_file_var)).list_pkgs()
- elif img_type == "deb":
- return DpkgPkgsList(d, rootfs_dir).list_pkgs()
+ import importlib
+ cls = importlib.import_module('oe.package_manager.' + img_type)
+ return cls.PMPkgsList(d, rootfs_dir).list_pkgs()
def populate_sdk(d, manifest_dir=None):
env_bkp = os.environ.copy()
img_type = d.getVar('IMAGE_PKGTYPE')
- from oe.package_manager.rpm.sdk import RpmSdk
- from oe.package_manager.ipk.sdk import OpkgSdk
- from oe.package_manager.deb.sdk import DpkgSdk
- if img_type == "rpm":
- RpmSdk(d, manifest_dir).populate()
- elif img_type == "ipk":
- OpkgSdk(d, manifest_dir).populate()
- elif img_type == "deb":
- DpkgSdk(d, manifest_dir).populate()
+ import importlib
+ cls = importlib.import_module('oe.package_manager.' + img_type + '.sdk')
+ cls.PkgSdk(d, manifest_dir).populate()
os.environ.clear()
os.environ.update(env_bkp)
--
2.20.1
^ permalink raw reply related [flat|nested] 3+ messages in thread
* Re: [OE-core] [PATCH] package management: Allow dynamic loading of PM
2020-09-08 10:53 [PATCH] package management: Allow dynamic loading of PM Fredrik Gustafsson
@ 2020-09-08 11:25 ` Richard Purdie
2020-09-08 20:30 ` Fredrik Gustafsson
0 siblings, 1 reply; 3+ messages in thread
From: Richard Purdie @ 2020-09-08 11:25 UTC (permalink / raw)
To: Fredrik Gustafsson, fredrigu; +Cc: openembedded-core, tools-cfpbuild-internal
On Tue, 2020-09-08 at 12:53 +0200, Fredrik Gustafsson wrote:
> Dynamic loading of package managers will allow other layers to simply
> add their package manager code in package_manager/ and have bitbake
> find
> it according to the package manager configuration. This is useful for
> adding new (faster) package managers to Open Embedded while not
> increasing the
> test scope or require Open Embedded to support more package managers.
>
> How this is tested:
> * Build core-image-minimal with all three package managers
> * Build the sdk with all three package managers. dpkg fails, but
> it fails on master as well.
> * Run the complete test suite, all tests passed except 16
> * Run those 16 tests on master and verify that they fail there as
> well
> * Fix errors making tests works on master but not with this patch.
Did you test both the sdk and esdk? Which tests are failing? Are these
oe-selftests? They all should pass and do on the autobuilder?
We're over a week past feature freeze and this series has a troubled
history of testing so realistically this needs to be 3.3 material at
this point (just to ensure we have the same expectations).
Cheers,
Richard
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: [OE-core] [PATCH] package management: Allow dynamic loading of PM
2020-09-08 11:25 ` [OE-core] " Richard Purdie
@ 2020-09-08 20:30 ` Fredrik Gustafsson
0 siblings, 0 replies; 3+ messages in thread
From: Fredrik Gustafsson @ 2020-09-08 20:30 UTC (permalink / raw)
To: Richard Purdie; +Cc: openembedded-core, tools-cfpbuild-internal
________________________________________
From: Richard Purdie <richard.purdie@linuxfoundation.org>
Sent: Tuesday, September 8, 2020 1:25 PM
To: Fredrik Gustafsson; Fredrik Gustafsson
Cc: openembedded-core@lists.openembedded.org; tools-cfpbuild-internal
Subject: Re: [OE-core] [PATCH] package management: Allow dynamic loading of PM
On Tue, 2020-09-08 at 12:53 +0200, Fredrik Gustafsson wrote:
> Dynamic loading of package managers will allow other layers to simply
> add their package manager code in package_manager/ and have bitbake
> find
> it according to the package manager configuration. This is useful for
> adding new (faster) package managers to Open Embedded while not
> increasing the
> test scope or require Open Embedded to support more package managers.
>
> How this is tested:
> * Build core-image-minimal with all three package managers
> * Build the sdk with all three package managers. dpkg fails, but
> it fails on master as well.
> * Run the complete test suite, all tests passed except 16
> * Run those 16 tests on master and verify that they fail there as
> well
> * Fix errors making tests works on master but not with this patch.
Did you test both the sdk and esdk? Which tests are failing? Are these
oe-selftests? They all should pass and do on the autobuilder?
We're over a week past feature freeze and this series has a troubled
history of testing so realistically this needs to be 3.3 material at
this point (just to ensure we have the same expectations).
Cheers,
Richard
Hi Richard,
I'm sorry that you experienced trouble with this patch series. That's
why I've tried to be really clear on what's tested and not tested. I don't
mind running tests, but of course I might miss which tests that should be run.
I did the following test:
set PACKAGE_CLASSES = rpm
bitbake core-image-minimal # OK
bitbake core-image-minimal -c populate_sdk # OK
bitbake core-image-minimal -c populate_sdk_ext # fail
# Failed to generate filtered task list for extensible SDK
# Fails on master as well
oe-selftest -a
set PACKAGE_CLASSES = ipk
bitbake core-image-minimal
bitbake core-image-minimal -c populate_sdk # OK
bitbake core-image-minimal -c populate_sdk_ext # fail
# Failed to generate filtered task list for extensible SDK
# Fails on master as well
set PACKAGE_CLASSES = deb
bitbake core-image-minimal
bitbake core-image-minimal -c populate_sdk # fail
# dpkg: nativesdk-qemu-helper: dependency problems, but configuring anyway as your requested
# Fails on master as well
And also ran oe-selftest -a with the follwing erros:
2020-08-28 21:39:27,092 - oe-selftest - INFO - RESULTS - imagefeatures.ImageFeatures.test_all_users_can_connect_via_ssh_without_password: ERROR (
26.84s)
2020-08-28 21:39:27,092 - oe-selftest - INFO - RESULTS - imagefeatures.ImageFeatures.test_non_root_user_can_connect_via_ssh_without_password: ERR
OR (26.86s)
2020-08-28 21:39:27,092 - oe-selftest - INFO - RESULTS - package.PackageTests.test_gdb_hardlink_debug: ERROR (165.96s)
2020-08-28 21:39:27,092 - oe-selftest - INFO - RESULTS - package.PackageTests.test_preserve_ownership: ERROR (30.97s)
2020-08-28 21:39:27,093 - oe-selftest - INFO - RESULTS - runtime_test.SystemTap.test_crosstap_helloworld: ERROR (768.02s)
2020-08-28 21:39:27,093 - oe-selftest - INFO - RESULTS - runtime_test.SystemTap.test_crosstap_pstree: ERROR (12.23s)
2020-08-28 21:39:27,093 - oe-selftest - INFO - RESULTS - runtime_test.SystemTap.test_crosstap_syscalls_by_pid: ERROR (11.61s)
2020-08-28 21:39:27,093 - oe-selftest - INFO - RESULTS - runtime_test.SystemTap.test_crosstap_syscalls_by_proc: ERROR (11.61s)
2020-08-28 21:39:27,093 - oe-selftest - INFO - RESULTS - runtime_test.TestExport.test_testexport_basic: ERROR (33.40s)
2020-08-28 21:39:27,093 - oe-selftest - INFO - RESULTS - wic.Wic2.test_biosplusefi_plugin_qemu: ERROR (7.40s)
2020-08-28 21:39:27,093 - oe-selftest - INFO - RESULTS - wic.Wic2.test_expand_mbr_image: ERROR (32.09s)
2020-08-28 21:39:27,093 - oe-selftest - INFO - RESULTS - wic.Wic2.test_qemu: ERROR (34.10s)
2020-08-28 21:39:27,093 - oe-selftest - INFO - RESULTS - wic.Wic2.test_qemu_efi: ERROR (29.09s)
2020-08-28 21:39:27,093 - oe-selftest - INFO - RESULTS - wic.Wic2.test_rawcopy_plugin_qemu: ERROR (53.20s)
2020-08-28 21:39:27,093 - oe-selftest - INFO - RESULTS - buildoptions.ArchiverTest.test_arch_work_dir_and_export_source: FAILED (14.04s)
2020-08-28 21:39:27,093 - oe-selftest - INFO - RESULTS - containerimage.ContainerImageTests.test_expected_files: FAILED (18.34s)
2020-08-28 21:39:27,093 - oe-selftest - INFO - RESULTS - devtool.DevtoolAddTests.test_devtool_add_fetch_git: FAILED (7.82s)
2020-08-28 21:39:27,093 - oe-selftest - INFO - RESULTS - distrodata.Distrodata.test_checkpkg: FAILED (100.84s)
2020-08-28 21:39:27,093 - oe-selftest - INFO - RESULTS - fetch.Fetch.test_git_mirrors: FAILED (5.81s)
2020-08-28 21:39:27,093 - oe-selftest - INFO - RESULTS - incompatible_lic.NoGPL3InImagesTests.test_core_image_full_cmdline: FAILED (74.05s)
2020-08-28 21:39:27,093 - oe-selftest - INFO - RESULTS - recipetool.RecipetoolTests.test_recipetool_create_git_http: FAILED (1.60s)
2020-08-28 21:39:27,093 - oe-selftest - INFO - RESULTS - recipetool.RecipetoolTests.test_recipetool_create_github: FAILED (3.59s)
2020-08-28 21:39:27,093 - oe-selftest - INFO - RESULTS - recipetool.RecipetoolTests.test_recipetool_create_npm: FAILED (1.70s)
2020-08-28 21:39:27,093 - oe-selftest - INFO - RESULTS - runtime_test.TestImage.test_testimage_dnf: FAILED (144.74s)
2020-08-28 21:39:27,093 - oe-selftest - INFO - RESULTS - runtime_test.TestImage.test_testimage_install: FAILED (50.83s)
2020-08-28 21:39:27,093 - oe-selftest - INFO - RESULTS - runtime_test.TestImage.test_testimage_virgl_gtk_sdl: FAILED (108.25s)
2020-08-28 21:39:27,093 - oe-selftest - INFO - RESULTS - runtime_test.Postinst.test_failing_postinst: UNKNOWN (61.95s)
2020-08-28 21:39:27,093 - oe-selftest - INFO - RESULTS - runtime_test.Postinst.test_postinst_rootfs_and_boot_systemd: UNKNOWN (1375.15s)
2020-08-28 21:39:27,093 - oe-selftest - INFO - RESULTS - runtime_test.Postinst.test_postinst_rootfs_and_boot_sysvinit: UNKNOWN (383.32s)
Here I know that our corporate network might be a culprint. Some tests also worked with I ran only that test. Is there anyway I can trigger a test with
this patch on the autobuilder?
I was hoping to get this into 3.2, but I appreciate that you're clear. Is there something I can do to lower the risk of this patch enough to make it suitable for 3.2?
BR
Fredrik
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2020-09-08 20:30 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-09-08 10:53 [PATCH] package management: Allow dynamic loading of PM Fredrik Gustafsson
2020-09-08 11:25 ` [OE-core] " Richard Purdie
2020-09-08 20:30 ` Fredrik Gustafsson
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.