* [PATCH v2 0/3] *** SUBJECT HERE ***
2022-04-03 19:50 package_manager: support for signed DEB package feeds Ferry Toth
@ 2022-04-03 19:50 ` Ferry Toth
2022-04-03 19:50 ` [PATCH v2 1/3] gpg-sign: Add parameters to gpg signature function Ferry Toth
` (2 subsequent siblings)
3 siblings, 0 replies; 17+ messages in thread
From: Ferry Toth @ 2022-04-03 19:50 UTC (permalink / raw)
To: openembedded-core
Cc: Richard Purdie, Xavier Berger, Alexander Kanavin, Ferry Toth
From: Ferry Toth <ftoth@exalondelft.nl>
*** BLURB HERE ***
Ferry Toth (2):
package_manager: sign DEB package feeds
apt: add apt selftest to test signed package feeds
Xavier Berger (1):
gpg-sign: Add parameters to gpg signature function
meta/lib/oe/gpg_sign.py | 6 +++-
meta/lib/oe/package_manager/deb/__init__.py | 19 ++++++++--
meta/lib/oeqa/runtime/cases/apt.py | 16 ++++++---
meta/lib/oeqa/selftest/cases/runtime_test.py | 38 ++++++++++++++++++++
4 files changed, 70 insertions(+), 9 deletions(-)
--
2.32.0
^ permalink raw reply [flat|nested] 17+ messages in thread
* [PATCH v2 1/3] gpg-sign: Add parameters to gpg signature function
2022-04-03 19:50 package_manager: support for signed DEB package feeds Ferry Toth
2022-04-03 19:50 ` [PATCH v2 0/3] *** SUBJECT HERE *** Ferry Toth
@ 2022-04-03 19:50 ` Ferry Toth
2022-04-03 19:50 ` [PATCH v2 2/3] package_manager: sign DEB package feeds Ferry Toth
2022-04-03 19:50 ` [PATCH v2 3/3] apt: add apt selftest to test signed " Ferry Toth
3 siblings, 0 replies; 17+ messages in thread
From: Ferry Toth @ 2022-04-03 19:50 UTC (permalink / raw)
To: openembedded-core
Cc: Richard Purdie, Xavier Berger, Alexander Kanavin, Ferry Toth
From: Xavier Berger <xavier.berger@bio-logic.net>
output_suffix: If defined, add output_suffix as file name extension.
use_sha256: If True, use sha256 for gpg as digest algorithm
Signed-off-by: Xavier Berger <xavier.berger@bio-logic.net>
Signed-off-by: Ferry Toth <ftoth@exalondelft.nl>
---
meta/lib/oe/gpg_sign.py | 6 +++++-
1 file changed, 5 insertions(+), 1 deletion(-)
diff --git a/meta/lib/oe/gpg_sign.py b/meta/lib/oe/gpg_sign.py
index 1bce6cb792..aa9bb49f2c 100644
--- a/meta/lib/oe/gpg_sign.py
+++ b/meta/lib/oe/gpg_sign.py
@@ -58,7 +58,7 @@ class LocalSigner(object):
for i in range(0, len(files), sign_chunk):
subprocess.check_output(shlex.split(cmd + ' '.join(files[i:i+sign_chunk])), stderr=subprocess.STDOUT)
- def detach_sign(self, input_file, keyid, passphrase_file, passphrase=None, armor=True):
+ def detach_sign(self, input_file, keyid, passphrase_file, passphrase=None, armor=True, output_suffix=None, use_sha256=False):
"""Create a detached signature of a file"""
if passphrase_file and passphrase:
@@ -71,6 +71,10 @@ class LocalSigner(object):
cmd += ['--homedir', self.gpg_path]
if armor:
cmd += ['--armor']
+ if output_suffix:
+ cmd += ['-o', input_file + "." + output_suffix]
+ if use_sha256:
+ cmd += ['--digest-algo', "SHA256"]
#gpg > 2.1 supports password pipes only through the loopback interface
#gpg < 2.1 errors out if given unknown parameters
--
2.32.0
^ permalink raw reply related [flat|nested] 17+ messages in thread
* [PATCH v2 2/3] package_manager: sign DEB package feeds
2022-04-03 19:50 package_manager: support for signed DEB package feeds Ferry Toth
2022-04-03 19:50 ` [PATCH v2 0/3] *** SUBJECT HERE *** Ferry Toth
2022-04-03 19:50 ` [PATCH v2 1/3] gpg-sign: Add parameters to gpg signature function Ferry Toth
@ 2022-04-03 19:50 ` Ferry Toth
2022-04-03 19:50 ` [PATCH v2 3/3] apt: add apt selftest to test signed " Ferry Toth
3 siblings, 0 replies; 17+ messages in thread
From: Ferry Toth @ 2022-04-03 19:50 UTC (permalink / raw)
To: openembedded-core
Cc: Richard Purdie, Xavier Berger, Alexander Kanavin, Ferry Toth
From: Ferry Toth <ftoth@exalondelft.nl>
Implement debian package repository signature.
For each Release file created in repository subdirectory, a signature
Release.gpg is created.
Signature is performed using gpg backend when the following variables
are set in local.conf:
PACKAGE_CLASSES += "sign_package_feed"
PACKAGE_FEED_GPG_NAME = "<Id of GPG key>"
PACKAGE_FEED_GPG_PASSPHRASE_FILE="<path to password file>"
Signed-off-by: Xavier Berger <xavier.berger@bio-logic.net>
Signed-off-by: Ferry Toth <ftoth@exalondelft.nl>
---
meta/lib/oe/package_manager/deb/__init__.py | 19 ++++++++++++++++---
1 file changed, 16 insertions(+), 3 deletions(-)
diff --git a/meta/lib/oe/package_manager/deb/__init__.py b/meta/lib/oe/package_manager/deb/__init__.py
index 9f112ae25b..86ddb130ad 100644
--- a/meta/lib/oe/package_manager/deb/__init__.py
+++ b/meta/lib/oe/package_manager/deb/__init__.py
@@ -53,6 +53,7 @@ class DpkgIndexer(Indexer):
index_cmds = []
deb_dirs_found = False
+ index_sign_files = set()
for arch in arch_list:
arch_dir = os.path.join(self.deploy_dir, arch)
if not os.path.isdir(arch_dir):
@@ -62,7 +63,10 @@ class DpkgIndexer(Indexer):
cmd += "%s -fcn Packages > Packages.gz;" % gzip
- with open(os.path.join(arch_dir, "Release"), "w+") as release:
+ release_file = os.path.join(arch_dir, "Release")
+ index_sign_files.add(release_file)
+
+ with open(release_file, "w+") as release:
release.write("Label: %s\n" % arch)
cmd += "PSEUDO_UNLOAD=1 %s release . >> Release" % apt_ftparchive
@@ -76,8 +80,17 @@ class DpkgIndexer(Indexer):
return
oe.utils.multiprocess_launch(create_index, index_cmds, self.d)
- if self.d.getVar('PACKAGE_FEED_SIGN') == '1':
- raise NotImplementedError('Package feed signing not implementd for dpkg')
+ if self.d.getVar('PACKAGE_FEED_SIGN', True) == '1':
+ signer = get_signer(self.d, self.d.getVar('PACKAGE_FEED_GPG_BACKEND', True))
+ else:
+ signer = None
+ if signer:
+ for f in index_sign_files:
+ signer.detach_sign(f,
+ self.d.getVar('PACKAGE_FEED_GPG_NAME', True),
+ self.d.getVar('PACKAGE_FEED_GPG_PASSPHRASE_FILE', True),
+ output_suffix="gpg",
+ use_sha256=True)
class PMPkgsList(PkgsList):
--
2.32.0
^ permalink raw reply related [flat|nested] 17+ messages in thread
* [PATCH v2 3/3] apt: add apt selftest to test signed package feeds
2022-04-03 19:50 package_manager: support for signed DEB package feeds Ferry Toth
` (2 preceding siblings ...)
2022-04-03 19:50 ` [PATCH v2 2/3] package_manager: sign DEB package feeds Ferry Toth
@ 2022-04-03 19:50 ` Ferry Toth
2022-04-04 13:58 ` Richard Purdie
3 siblings, 1 reply; 17+ messages in thread
From: Ferry Toth @ 2022-04-03 19:50 UTC (permalink / raw)
To: openembedded-core
Cc: Richard Purdie, Xavier Berger, Alexander Kanavin, Ferry Toth
From: Ferry Toth <ftoth@exalondelft.nl>
Since Gatesgarth apt (1.8.2) has become more strict and doesn’t allow unsigned repositories by default.
Currently when building images this requirement is worked around by using [allow-insecure=yes] and
equivalently when performing selftest.
Patches "gpg-sign: Add parameters to gpg signature function" and "package_manager: sign DEB package feeds"
enable signed DEB package feeds. This patch adds a runtime test for apt derived from the test_testimage_dnf
test. It creates a signed deb package feed, runs a qemu image to install the key and performs some package
management. To be able to install the key the gnupg package is added to the testimage.
Signed-off-by: Ferry Toth <ftoth@exalondelft.nl>
---
meta/lib/oeqa/runtime/cases/apt.py | 16 ++++++---
meta/lib/oeqa/selftest/cases/runtime_test.py | 38 ++++++++++++++++++++
2 files changed, 49 insertions(+), 5 deletions(-)
diff --git a/meta/lib/oeqa/runtime/cases/apt.py b/meta/lib/oeqa/runtime/cases/apt.py
index 53745df93f..49f8714730 100644
--- a/meta/lib/oeqa/runtime/cases/apt.py
+++ b/meta/lib/oeqa/runtime/cases/apt.py
@@ -21,7 +21,7 @@ class AptRepoTest(AptTest):
@classmethod
def setUpClass(cls):
- service_repo = os.path.join(cls.tc.td['DEPLOY_DIR_DEB'], 'all')
+ service_repo = os.path.join(cls.tc.td['DEPLOY_DIR_DEB'], '')
cls.repo_server = HTTPService(service_repo,
'0.0.0.0', port=cls.tc.target.server_port,
logger=cls.tc.logger)
@@ -32,13 +32,18 @@ class AptRepoTest(AptTest):
cls.repo_server.stop()
def setup_source_config_for_package_install(self):
- apt_get_source_server = 'http://%s:%s/' % (self.tc.target.server_ip, self.repo_server.port)
+ apt_get_source_server = 'http:\/\/%s:%s' % (self.tc.target.server_ip, self.repo_server.port)
apt_get_sourceslist_dir = '/etc/apt/'
- self.target.run('cd %s; echo deb [ allow-insecure=yes ] %s ./ > sources.list' % (apt_get_sourceslist_dir, apt_get_source_server))
+ self.target.run("cd %s; cp sources.list sources.list.bak; sed -i 's/\[trusted=yes\] http:\/\/bogus_ip:bogus_port/%s/g' sources.list" % (apt_get_sourceslist_dir, apt_get_source_server))
def cleanup_source_config_for_package_install(self):
apt_get_sourceslist_dir = '/etc/apt/'
- self.target.run('cd %s; rm sources.list' % (apt_get_sourceslist_dir))
+ self.target.run('cd %s; mv sources.list.bak sources.list' % (apt_get_sourceslist_dir))
+
+ def setup_key(self):
+ # the key is found on the target /etc/pki/packagefeed-gpg/
+ # named PACKAGEFEED-GPG-KEY-poky-branch
+ self.target.run('cd %s; apt-key add P*' % ('/etc/pki/packagefeed-gpg'))
@skipIfNotFeature('package-management',
'Test requires package-management to be in IMAGE_FEATURES')
@@ -47,7 +52,8 @@ class AptRepoTest(AptTest):
@OEHasPackage(['apt'])
def test_apt_install_from_repo(self):
self.setup_source_config_for_package_install()
+ self.setup_key()
self.pkg('update')
self.pkg('remove --yes run-postinsts-dev')
- self.pkg('install --yes --allow-unauthenticated run-postinsts-dev')
+ self.pkg('install --yes run-postinsts-dev')
self.cleanup_source_config_for_package_install()
diff --git a/meta/lib/oeqa/selftest/cases/runtime_test.py b/meta/lib/oeqa/selftest/cases/runtime_test.py
index 642f0eb637..7a75b95a99 100644
--- a/meta/lib/oeqa/selftest/cases/runtime_test.py
+++ b/meta/lib/oeqa/selftest/cases/runtime_test.py
@@ -162,6 +162,44 @@ class TestImage(OESelftestTestCase):
bitbake('core-image-full-cmdline socat')
bitbake('-c testimage core-image-full-cmdline')
+ def test_testimage_apt(self):
+ """
+ Summary: Check package feeds functionality for apt
+ Expected: 1. Check that remote package feeds can be accessed
+ Product: oe-core
+ Author: Ferry Toth <fntoth@gmail.com>
+ """
+ if get_bb_var('DISTRO') == 'poky-tiny':
+ self.skipTest('core-image-full-cmdline not buildable for poky-tiny')
+
+ features = 'INHERIT += "testimage"\n'
+ features += 'TEST_SUITES = "ping ssh apt.AptRepoTest.test_apt_install_from_repo"\n'
+ # We don't yet know what the server ip and port will be - they will be patched
+ # in at the start of the on-image test
+ features += 'PACKAGE_FEED_URIS = "http://bogus_ip:bogus_port"\n'
+ features += 'EXTRA_IMAGE_FEATURES += "package-management"\n'
+ features += 'PACKAGE_CLASSES = "package_deb"\n'
+ # We need gnupg on the target to install keys
+ features += 'IMAGE_INSTALL:append:pn-core-image-full-cmdline = " gnupg"\n'
+
+ bitbake('gnupg-native -c addto_recipe_sysroot')
+
+ # Enable package feed signing
+ self.gpg_home = tempfile.mkdtemp(prefix="oeqa-feed-sign-")
+ self.track_for_cleanup(self.gpg_home)
+ signing_key_dir = os.path.join(self.testlayer_path, 'files', 'signing')
+ runCmd('gpgconf --list-dirs --homedir %s; gpg -v --batch --homedir %s --import %s' % (self.gpg_home, self.gpg_home, os.path.join(signing_key_dir, 'key.secret')), native_sysroot=get_bb_var("RECIPE_SYSROOT_NATIVE", "gnupg-native"), shell=True)
+ features += 'INHERIT += "sign_package_feed"\n'
+ features += 'PACKAGE_FEED_GPG_NAME = "testuser"\n'
+ features += 'PACKAGE_FEED_GPG_PASSPHRASE_FILE = "%s"\n' % os.path.join(signing_key_dir, 'key.passphrase')
+ features += 'GPG_PATH = "%s"\n' % self.gpg_home
+ features += 'PSEUDO_IGNORE_PATHS .= ",%s"\n' % self.gpg_home
+ self.write_config(features)
+
+ # Build core-image-sato and testimage
+ bitbake('core-image-full-cmdline socat')
+ bitbake('-c testimage core-image-full-cmdline')
+
def test_testimage_virgl_gtk_sdl(self):
"""
Summary: Check host-assisted accelerate OpenGL functionality in qemu with gtk and SDL frontends
--
2.32.0
^ permalink raw reply related [flat|nested] 17+ messages in thread
* Re: [PATCH v2 3/3] apt: add apt selftest to test signed package feeds
2022-04-03 19:50 ` [PATCH v2 3/3] apt: add apt selftest to test signed " Ferry Toth
@ 2022-04-04 13:58 ` Richard Purdie
2022-04-04 17:35 ` Ferry Toth
2022-04-06 10:10 ` [OE-core] " Alexandre Belloni
0 siblings, 2 replies; 17+ messages in thread
From: Richard Purdie @ 2022-04-04 13:58 UTC (permalink / raw)
To: Ferry Toth, openembedded-core
Cc: Xavier Berger, Alexander Kanavin, Ferry Toth
On Sun, 2022-04-03 at 21:50 +0200, Ferry Toth wrote:
> From: Ferry Toth <ftoth@exalondelft.nl>
>
> Since Gatesgarth apt (1.8.2) has become more strict and doesn’t allow unsigned repositories by default.
> Currently when building images this requirement is worked around by using [allow-insecure=yes] and
> equivalently when performing selftest.
>
> Patches "gpg-sign: Add parameters to gpg signature function" and "package_manager: sign DEB package feeds"
> enable signed DEB package feeds. This patch adds a runtime test for apt derived from the test_testimage_dnf
> test. It creates a signed deb package feed, runs a qemu image to install the key and performs some package
> management. To be able to install the key the gnupg package is added to the testimage.
>
> Signed-off-by: Ferry Toth <ftoth@exalondelft.nl>
> ---
> meta/lib/oeqa/runtime/cases/apt.py | 16 ++++++---
> meta/lib/oeqa/selftest/cases/runtime_test.py | 38 ++++++++++++++++++++
> 2 files changed, 49 insertions(+), 5 deletions(-)
>
> diff --git a/meta/lib/oeqa/runtime/cases/apt.py b/meta/lib/oeqa/runtime/cases/apt.py
> index 53745df93f..49f8714730 100644
> --- a/meta/lib/oeqa/runtime/cases/apt.py
> +++ b/meta/lib/oeqa/runtime/cases/apt.py
> @@ -21,7 +21,7 @@ class AptRepoTest(AptTest):
>
> @classmethod
> def setUpClass(cls):
> - service_repo = os.path.join(cls.tc.td['DEPLOY_DIR_DEB'], 'all')
> + service_repo = os.path.join(cls.tc.td['DEPLOY_DIR_DEB'], '')
> cls.repo_server = HTTPService(service_repo,
> '0.0.0.0', port=cls.tc.target.server_port,
> logger=cls.tc.logger)
> @@ -32,13 +32,18 @@ class AptRepoTest(AptTest):
> cls.repo_server.stop()
>
> def setup_source_config_for_package_install(self):
> - apt_get_source_server = 'http://%s:%s/' % (self.tc.target.server_ip, self.repo_server.port)
> + apt_get_source_server = 'http:\/\/%s:%s' % (self.tc.target.server_ip, self.repo_server.port)
> apt_get_sourceslist_dir = '/etc/apt/'
> - self.target.run('cd %s; echo deb [ allow-insecure=yes ] %s ./ > sources.list' % (apt_get_sourceslist_dir, apt_get_source_server))
> + self.target.run("cd %s; cp sources.list sources.list.bak; sed -i 's/\[trusted=yes\] http:\/\/bogus_ip:bogus_port/%s/g' sources.list" % (apt_get_sourceslist_dir, apt_get_source_server))
>
> def cleanup_source_config_for_package_install(self):
> apt_get_sourceslist_dir = '/etc/apt/'
> - self.target.run('cd %s; rm sources.list' % (apt_get_sourceslist_dir))
> + self.target.run('cd %s; mv sources.list.bak sources.list' % (apt_get_sourceslist_dir))
> +
> + def setup_key(self):
> + # the key is found on the target /etc/pki/packagefeed-gpg/
> + # named PACKAGEFEED-GPG-KEY-poky-branch
> + self.target.run('cd %s; apt-key add P*' % ('/etc/pki/packagefeed-gpg'))
>
> @skipIfNotFeature('package-management',
> 'Test requires package-management to be in IMAGE_FEATURES')
> @@ -47,7 +52,8 @@ class AptRepoTest(AptTest):
> @OEHasPackage(['apt'])
> def test_apt_install_from_repo(self):
> self.setup_source_config_for_package_install()
> + self.setup_key()
> self.pkg('update')
> self.pkg('remove --yes run-postinsts-dev')
> - self.pkg('install --yes --allow-unauthenticated run-postinsts-dev')
> + self.pkg('install --yes run-postinsts-dev')
> self.cleanup_source_config_for_package_install()
> diff --git a/meta/lib/oeqa/selftest/cases/runtime_test.py b/meta/lib/oeqa/selftest/cases/runtime_test.py
> index 642f0eb637..7a75b95a99 100644
> --- a/meta/lib/oeqa/selftest/cases/runtime_test.py
> +++ b/meta/lib/oeqa/selftest/cases/runtime_test.py
> @@ -162,6 +162,44 @@ class TestImage(OESelftestTestCase):
> bitbake('core-image-full-cmdline socat')
> bitbake('-c testimage core-image-full-cmdline')
>
> + def test_testimage_apt(self):
> + """
> + Summary: Check package feeds functionality for apt
> + Expected: 1. Check that remote package feeds can be accessed
> + Product: oe-core
> + Author: Ferry Toth <fntoth@gmail.com>
> + """
> + if get_bb_var('DISTRO') == 'poky-tiny':
> + self.skipTest('core-image-full-cmdline not buildable for poky-tiny')
> +
> + features = 'INHERIT += "testimage"\n'
> + features += 'TEST_SUITES = "ping ssh apt.AptRepoTest.test_apt_install_from_repo"\n'
> + # We don't yet know what the server ip and port will be - they will be patched
> + # in at the start of the on-image test
> + features += 'PACKAGE_FEED_URIS = "http://bogus_ip:bogus_port"\n'
> + features += 'EXTRA_IMAGE_FEATURES += "package-management"\n'
> + features += 'PACKAGE_CLASSES = "package_deb"\n'
> + # We need gnupg on the target to install keys
> + features += 'IMAGE_INSTALL:append:pn-core-image-full-cmdline = " gnupg"\n'
> +
> + bitbake('gnupg-native -c addto_recipe_sysroot')
> +
> + # Enable package feed signing
> + self.gpg_home = tempfile.mkdtemp(prefix="oeqa-feed-sign-")
> + self.track_for_cleanup(self.gpg_home)
> + signing_key_dir = os.path.join(self.testlayer_path, 'files', 'signing')
> + runCmd('gpgconf --list-dirs --homedir %s; gpg -v --batch --homedir %s --import %s' % (self.gpg_home, self.gpg_home, os.path.join(signing_key_dir, 'key.secret')), native_sysroot=get_bb_var("RECIPE_SYSROOT_NATIVE", "gnupg-native"), shell=True)
> + features += 'INHERIT += "sign_package_feed"\n'
> + features += 'PACKAGE_FEED_GPG_NAME = "testuser"\n'
> + features += 'PACKAGE_FEED_GPG_PASSPHRASE_FILE = "%s"\n' % os.path.join(signing_key_dir, 'key.passphrase')
> + features += 'GPG_PATH = "%s"\n' % self.gpg_home
> + features += 'PSEUDO_IGNORE_PATHS .= ",%s"\n' % self.gpg_home
> + self.write_config(features)
> +
> + # Build core-image-sato and testimage
> + bitbake('core-image-full-cmdline socat')
> + bitbake('-c testimage core-image-full-cmdline')
> +
> def test_testimage_virgl_gtk_sdl(self):
> """
> Summary: Check host-assisted accelerate OpenGL functionality in qemu with gtk and SDL frontends
Thanks for working on this!
Looking at the patches I wondered if this would break testimage and
unfortunately it does:
https://autobuilder.yoctoproject.org/typhoon/#/builders/50/builds/5013/steps/12/logs/stdio
https://autobuilder.yoctoproject.org/typhoon/#/builders/76/builds/4975
however hopefully these shouldn't be too hard to fix?
The rest of the build is still running.
Cheers,
Richard
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH v2 3/3] apt: add apt selftest to test signed package feeds
2022-04-04 13:58 ` Richard Purdie
@ 2022-04-04 17:35 ` Ferry Toth
2022-04-04 20:39 ` Richard Purdie
2022-04-06 10:10 ` [OE-core] " Alexandre Belloni
1 sibling, 1 reply; 17+ messages in thread
From: Ferry Toth @ 2022-04-04 17:35 UTC (permalink / raw)
To: Richard Purdie, openembedded-core
Cc: Xavier Berger, Alexander Kanavin, Ferry Toth
Hi,
Op 04-04-2022 om 15:58 schreef Richard Purdie:
> On Sun, 2022-04-03 at 21:50 +0200, Ferry Toth wrote:
>> From: Ferry Toth <ftoth@exalondelft.nl>
>>
>> Since Gatesgarth apt (1.8.2) has become more strict and doesn’t allow unsigned repositories by default.
>> Currently when building images this requirement is worked around by using [allow-insecure=yes] and
>> equivalently when performing selftest.
>>
>> Patches "gpg-sign: Add parameters to gpg signature function" and "package_manager: sign DEB package feeds"
>> enable signed DEB package feeds. This patch adds a runtime test for apt derived from the test_testimage_dnf
>> test. It creates a signed deb package feed, runs a qemu image to install the key and performs some package
>> management. To be able to install the key the gnupg package is added to the testimage.
>>
>> Signed-off-by: Ferry Toth <ftoth@exalondelft.nl>
>> ---
>> meta/lib/oeqa/runtime/cases/apt.py | 16 ++++++---
>> meta/lib/oeqa/selftest/cases/runtime_test.py | 38 ++++++++++++++++++++
>> 2 files changed, 49 insertions(+), 5 deletions(-)
>>
>> diff --git a/meta/lib/oeqa/runtime/cases/apt.py b/meta/lib/oeqa/runtime/cases/apt.py
>> index 53745df93f..49f8714730 100644
>> --- a/meta/lib/oeqa/runtime/cases/apt.py
>> +++ b/meta/lib/oeqa/runtime/cases/apt.py
>> @@ -21,7 +21,7 @@ class AptRepoTest(AptTest):
>>
>> @classmethod
>> def setUpClass(cls):
>> - service_repo = os.path.join(cls.tc.td['DEPLOY_DIR_DEB'], 'all')
>> + service_repo = os.path.join(cls.tc.td['DEPLOY_DIR_DEB'], '')
>> cls.repo_server = HTTPService(service_repo,
>> '0.0.0.0', port=cls.tc.target.server_port,
>> logger=cls.tc.logger)
>> @@ -32,13 +32,18 @@ class AptRepoTest(AptTest):
>> cls.repo_server.stop()
>>
>> def setup_source_config_for_package_install(self):
>> - apt_get_source_server = 'http://%s:%s/' % (self.tc.target.server_ip, self.repo_server.port)
>> + apt_get_source_server = 'http:\/\/%s:%s' % (self.tc.target.server_ip, self.repo_server.port)
>> apt_get_sourceslist_dir = '/etc/apt/'
>> - self.target.run('cd %s; echo deb [ allow-insecure=yes ] %s ./ > sources.list' % (apt_get_sourceslist_dir, apt_get_source_server))
>> + self.target.run("cd %s; cp sources.list sources.list.bak; sed -i 's/\[trusted=yes\] http:\/\/bogus_ip:bogus_port/%s/g' sources.list" % (apt_get_sourceslist_dir, apt_get_source_server))
>>
>> def cleanup_source_config_for_package_install(self):
>> apt_get_sourceslist_dir = '/etc/apt/'
>> - self.target.run('cd %s; rm sources.list' % (apt_get_sourceslist_dir))
>> + self.target.run('cd %s; mv sources.list.bak sources.list' % (apt_get_sourceslist_dir))
>> +
>> + def setup_key(self):
>> + # the key is found on the target /etc/pki/packagefeed-gpg/
>> + # named PACKAGEFEED-GPG-KEY-poky-branch
>> + self.target.run('cd %s; apt-key add P*' % ('/etc/pki/packagefeed-gpg'))
>>
>> @skipIfNotFeature('package-management',
>> 'Test requires package-management to be in IMAGE_FEATURES')
>> @@ -47,7 +52,8 @@ class AptRepoTest(AptTest):
>> @OEHasPackage(['apt'])
>> def test_apt_install_from_repo(self):
>> self.setup_source_config_for_package_install()
>> + self.setup_key()
>> self.pkg('update')
>> self.pkg('remove --yes run-postinsts-dev')
>> - self.pkg('install --yes --allow-unauthenticated run-postinsts-dev')
>> + self.pkg('install --yes run-postinsts-dev')
>> self.cleanup_source_config_for_package_install()
>> diff --git a/meta/lib/oeqa/selftest/cases/runtime_test.py b/meta/lib/oeqa/selftest/cases/runtime_test.py
>> index 642f0eb637..7a75b95a99 100644
>> --- a/meta/lib/oeqa/selftest/cases/runtime_test.py
>> +++ b/meta/lib/oeqa/selftest/cases/runtime_test.py
>> @@ -162,6 +162,44 @@ class TestImage(OESelftestTestCase):
>> bitbake('core-image-full-cmdline socat')
>> bitbake('-c testimage core-image-full-cmdline')
>>
>> + def test_testimage_apt(self):
>> + """
>> + Summary: Check package feeds functionality for apt
>> + Expected: 1. Check that remote package feeds can be accessed
>> + Product: oe-core
>> + Author: Ferry Toth <fntoth@gmail.com>
>> + """
>> + if get_bb_var('DISTRO') == 'poky-tiny':
>> + self.skipTest('core-image-full-cmdline not buildable for poky-tiny')
>> +
>> + features = 'INHERIT += "testimage"\n'
>> + features += 'TEST_SUITES = "ping ssh apt.AptRepoTest.test_apt_install_from_repo"\n'
>> + # We don't yet know what the server ip and port will be - they will be patched
>> + # in at the start of the on-image test
>> + features += 'PACKAGE_FEED_URIS = "http://bogus_ip:bogus_port"\n'
>> + features += 'EXTRA_IMAGE_FEATURES += "package-management"\n'
>> + features += 'PACKAGE_CLASSES = "package_deb"\n'
>> + # We need gnupg on the target to install keys
>> + features += 'IMAGE_INSTALL:append:pn-core-image-full-cmdline = " gnupg"\n'
>> +
>> + bitbake('gnupg-native -c addto_recipe_sysroot')
>> +
>> + # Enable package feed signing
>> + self.gpg_home = tempfile.mkdtemp(prefix="oeqa-feed-sign-")
>> + self.track_for_cleanup(self.gpg_home)
>> + signing_key_dir = os.path.join(self.testlayer_path, 'files', 'signing')
>> + runCmd('gpgconf --list-dirs --homedir %s; gpg -v --batch --homedir %s --import %s' % (self.gpg_home, self.gpg_home, os.path.join(signing_key_dir, 'key.secret')), native_sysroot=get_bb_var("RECIPE_SYSROOT_NATIVE", "gnupg-native"), shell=True)
>> + features += 'INHERIT += "sign_package_feed"\n'
>> + features += 'PACKAGE_FEED_GPG_NAME = "testuser"\n'
>> + features += 'PACKAGE_FEED_GPG_PASSPHRASE_FILE = "%s"\n' % os.path.join(signing_key_dir, 'key.passphrase')
>> + features += 'GPG_PATH = "%s"\n' % self.gpg_home
>> + features += 'PSEUDO_IGNORE_PATHS .= ",%s"\n' % self.gpg_home
>> + self.write_config(features)
>> +
>> + # Build core-image-sato and testimage
>> + bitbake('core-image-full-cmdline socat')
>> + bitbake('-c testimage core-image-full-cmdline')
>> +
>> def test_testimage_virgl_gtk_sdl(self):
>> """
>> Summary: Check host-assisted accelerate OpenGL functionality in qemu with gtk and SDL frontends
>
> Thanks for working on this!
>
> Looking at the patches I wondered if this would break testimage and
> unfortunately it does:
>
> https://autobuilder.yoctoproject.org/typhoon/#/builders/50/builds/5013/steps/12/logs/stdio
> https://autobuilder.yoctoproject.org/typhoon/#/builders/76/builds/4975
That is weird, do I understand correctly that it fails on:
apt-get remove --yes run-postinsts-dev
Reading package lists...
Building dependency tree...
E: Unable to locate package run-postinsts-dev
That is actually *) one line I didn't touch. I did note while testing
that I saw this exact message, however that was not counted as a fail.
What could cause this? Because the complaint is it can't remove the
package because it was not installed.
It would be trivial to remove the line
*) self.pkg('remove --yes run-postinsts-dev')
but how could it have passed the test before?
> however hopefully these shouldn't be too hard to fix?
>
> The rest of the build is still running.
>
> Cheers,
>
> Richard
>
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH v2 3/3] apt: add apt selftest to test signed package feeds
2022-04-04 17:35 ` Ferry Toth
@ 2022-04-04 20:39 ` Richard Purdie
2022-04-05 15:23 ` Ferry Toth
0 siblings, 1 reply; 17+ messages in thread
From: Richard Purdie @ 2022-04-04 20:39 UTC (permalink / raw)
To: Ferry Toth, openembedded-core
Cc: Xavier Berger, Alexander Kanavin, Ferry Toth
On Mon, 2022-04-04 at 19:35 +0200, Ferry Toth wrote:
> Hi,
>
> Op 04-04-2022 om 15:58 schreef Richard Purdie:
> > On Sun, 2022-04-03 at 21:50 +0200, Ferry Toth wrote:
> > > From: Ferry Toth <ftoth@exalondelft.nl>
> > >
> > > Since Gatesgarth apt (1.8.2) has become more strict and doesn’t allow unsigned repositories by default.
> > > Currently when building images this requirement is worked around by using [allow-insecure=yes] and
> > > equivalently when performing selftest.
> > >
> > > Patches "gpg-sign: Add parameters to gpg signature function" and "package_manager: sign DEB package feeds"
> > > enable signed DEB package feeds. This patch adds a runtime test for apt derived from the test_testimage_dnf
> > > test. It creates a signed deb package feed, runs a qemu image to install the key and performs some package
> > > management. To be able to install the key the gnupg package is added to the testimage.
> > >
> > > Signed-off-by: Ferry Toth <ftoth@exalondelft.nl>
> > > ---
> > > meta/lib/oeqa/runtime/cases/apt.py | 16 ++++++---
> > > meta/lib/oeqa/selftest/cases/runtime_test.py | 38 ++++++++++++++++++++
> > > 2 files changed, 49 insertions(+), 5 deletions(-)
> > >
> > > diff --git a/meta/lib/oeqa/runtime/cases/apt.py b/meta/lib/oeqa/runtime/cases/apt.py
> > > index 53745df93f..49f8714730 100644
> > > --- a/meta/lib/oeqa/runtime/cases/apt.py
> > > +++ b/meta/lib/oeqa/runtime/cases/apt.py
> > > @@ -21,7 +21,7 @@ class AptRepoTest(AptTest):
> > >
> > > @classmethod
> > > def setUpClass(cls):
> > > - service_repo = os.path.join(cls.tc.td['DEPLOY_DIR_DEB'], 'all')
> > > + service_repo = os.path.join(cls.tc.td['DEPLOY_DIR_DEB'], '')
> > > cls.repo_server = HTTPService(service_repo,
> > > '0.0.0.0', port=cls.tc.target.server_port,
> > > logger=cls.tc.logger)
> > > @@ -32,13 +32,18 @@ class AptRepoTest(AptTest):
> > > cls.repo_server.stop()
> > >
> > > def setup_source_config_for_package_install(self):
> > > - apt_get_source_server = 'http://%s:%s/' % (self.tc.target.server_ip, self.repo_server.port)
> > > + apt_get_source_server = 'http:\/\/%s:%s' % (self.tc.target.server_ip, self.repo_server.port)
> > > apt_get_sourceslist_dir = '/etc/apt/'
> > > - self.target.run('cd %s; echo deb [ allow-insecure=yes ] %s ./ > sources.list' % (apt_get_sourceslist_dir, apt_get_source_server))
> > > + self.target.run("cd %s; cp sources.list sources.list.bak; sed -i 's/\[trusted=yes\] http:\/\/bogus_ip:bogus_port/%s/g' sources.list" % (apt_get_sourceslist_dir, apt_get_source_server))
> > >
> > > def cleanup_source_config_for_package_install(self):
> > > apt_get_sourceslist_dir = '/etc/apt/'
> > > - self.target.run('cd %s; rm sources.list' % (apt_get_sourceslist_dir))
> > > + self.target.run('cd %s; mv sources.list.bak sources.list' % (apt_get_sourceslist_dir))
> > > +
> > > + def setup_key(self):
> > > + # the key is found on the target /etc/pki/packagefeed-gpg/
> > > + # named PACKAGEFEED-GPG-KEY-poky-branch
> > > + self.target.run('cd %s; apt-key add P*' % ('/etc/pki/packagefeed-gpg'))
> > >
> > > @skipIfNotFeature('package-management',
> > > 'Test requires package-management to be in IMAGE_FEATURES')
> > > @@ -47,7 +52,8 @@ class AptRepoTest(AptTest):
> > > @OEHasPackage(['apt'])
> > > def test_apt_install_from_repo(self):
> > > self.setup_source_config_for_package_install()
> > > + self.setup_key()
> > > self.pkg('update')
> > > self.pkg('remove --yes run-postinsts-dev')
> > > - self.pkg('install --yes --allow-unauthenticated run-postinsts-dev')
> > > + self.pkg('install --yes run-postinsts-dev')
> > > self.cleanup_source_config_for_package_install()
> > > diff --git a/meta/lib/oeqa/selftest/cases/runtime_test.py b/meta/lib/oeqa/selftest/cases/runtime_test.py
> > > index 642f0eb637..7a75b95a99 100644
> > > --- a/meta/lib/oeqa/selftest/cases/runtime_test.py
> > > +++ b/meta/lib/oeqa/selftest/cases/runtime_test.py
> > > @@ -162,6 +162,44 @@ class TestImage(OESelftestTestCase):
> > > bitbake('core-image-full-cmdline socat')
> > > bitbake('-c testimage core-image-full-cmdline')
> > >
> > > + def test_testimage_apt(self):
> > > + """
> > > + Summary: Check package feeds functionality for apt
> > > + Expected: 1. Check that remote package feeds can be accessed
> > > + Product: oe-core
> > > + Author: Ferry Toth <fntoth@gmail.com>
> > > + """
> > > + if get_bb_var('DISTRO') == 'poky-tiny':
> > > + self.skipTest('core-image-full-cmdline not buildable for poky-tiny')
> > > +
> > > + features = 'INHERIT += "testimage"\n'
> > > + features += 'TEST_SUITES = "ping ssh apt.AptRepoTest.test_apt_install_from_repo"\n'
> > > + # We don't yet know what the server ip and port will be - they will be patched
> > > + # in at the start of the on-image test
> > > + features += 'PACKAGE_FEED_URIS = "http://bogus_ip:bogus_port"\n'
> > > + features += 'EXTRA_IMAGE_FEATURES += "package-management"\n'
> > > + features += 'PACKAGE_CLASSES = "package_deb"\n'
> > > + # We need gnupg on the target to install keys
> > > + features += 'IMAGE_INSTALL:append:pn-core-image-full-cmdline = " gnupg"\n'
> > > +
> > > + bitbake('gnupg-native -c addto_recipe_sysroot')
> > > +
> > > + # Enable package feed signing
> > > + self.gpg_home = tempfile.mkdtemp(prefix="oeqa-feed-sign-")
> > > + self.track_for_cleanup(self.gpg_home)
> > > + signing_key_dir = os.path.join(self.testlayer_path, 'files', 'signing')
> > > + runCmd('gpgconf --list-dirs --homedir %s; gpg -v --batch --homedir %s --import %s' % (self.gpg_home, self.gpg_home, os.path.join(signing_key_dir, 'key.secret')), native_sysroot=get_bb_var("RECIPE_SYSROOT_NATIVE", "gnupg-native"), shell=True)
> > > + features += 'INHERIT += "sign_package_feed"\n'
> > > + features += 'PACKAGE_FEED_GPG_NAME = "testuser"\n'
> > > + features += 'PACKAGE_FEED_GPG_PASSPHRASE_FILE = "%s"\n' % os.path.join(signing_key_dir, 'key.passphrase')
> > > + features += 'GPG_PATH = "%s"\n' % self.gpg_home
> > > + features += 'PSEUDO_IGNORE_PATHS .= ",%s"\n' % self.gpg_home
> > > + self.write_config(features)
> > > +
> > > + # Build core-image-sato and testimage
> > > + bitbake('core-image-full-cmdline socat')
> > > + bitbake('-c testimage core-image-full-cmdline')
> > > +
> > > def test_testimage_virgl_gtk_sdl(self):
> > > """
> > > Summary: Check host-assisted accelerate OpenGL functionality in qemu with gtk and SDL frontends
> >
> > Thanks for working on this!
> >
> > Looking at the patches I wondered if this would break testimage and
> > unfortunately it does:
> >
> > https://autobuilder.yoctoproject.org/typhoon/#/builders/50/builds/5013/steps/12/logs/stdio
> > https://autobuilder.yoctoproject.org/typhoon/#/builders/76/builds/4975
>
> That is weird, do I understand correctly that it fails on:
> apt-get remove --yes run-postinsts-dev
> Reading package lists...
> Building dependency tree...
> E: Unable to locate package run-postinsts-dev
>
> That is actually *) one line I didn't touch. I did note while testing
> that I saw this exact message, however that was not counted as a fail.
>
> What could cause this? Because the complaint is it can't remove the
> package because it was not installed.
>
> It would be trivial to remove the line
>
> *) self.pkg('remove --yes run-postinsts-dev')
>
> but how could it have passed the test before?
I think the issue is you edited testimage which is a different set of tests
which aren't just called by oe-selftest but by things like
"bitbake core-image-sato -c testimage"
as well. I'd suggest making the changes in testimage conditional on signing
being configured.
Cheers,
Richard
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH v2 3/3] apt: add apt selftest to test signed package feeds
2022-04-04 20:39 ` Richard Purdie
@ 2022-04-05 15:23 ` Ferry Toth
2022-04-06 11:40 ` Richard Purdie
0 siblings, 1 reply; 17+ messages in thread
From: Ferry Toth @ 2022-04-05 15:23 UTC (permalink / raw)
To: Richard Purdie, openembedded-core
Cc: Xavier Berger, Alexander Kanavin, Ferry Toth
[-- Attachment #1: Type: text/plain, Size: 9640 bytes --]
Hi,
Op 04-04-2022 om 22:39 schreef Richard Purdie:
> On Mon, 2022-04-04 at 19:35 +0200, Ferry Toth wrote:
>> Hi,
>>
>> Op 04-04-2022 om 15:58 schreef Richard Purdie:
>>> On Sun, 2022-04-03 at 21:50 +0200, Ferry Toth wrote:
>>>> From: Ferry Toth<ftoth@exalondelft.nl>
>>>>
>>>> Since Gatesgarth apt (1.8.2) has become more strict and doesn’t allow unsigned repositories by default.
>>>> Currently when building images this requirement is worked around by using [allow-insecure=yes] and
>>>> equivalently when performing selftest.
>>>>
>>>> Patches "gpg-sign: Add parameters to gpg signature function" and "package_manager: sign DEB package feeds"
>>>> enable signed DEB package feeds. This patch adds a runtime test for apt derived from the test_testimage_dnf
>>>> test. It creates a signed deb package feed, runs a qemu image to install the key and performs some package
>>>> management. To be able to install the key the gnupg package is added to the testimage.
>>>>
>>>> Signed-off-by: Ferry Toth<ftoth@exalondelft.nl>
>>>> ---
>>>> meta/lib/oeqa/runtime/cases/apt.py | 16 ++++++---
>>>> meta/lib/oeqa/selftest/cases/runtime_test.py | 38 ++++++++++++++++++++
>>>> 2 files changed, 49 insertions(+), 5 deletions(-)
>>>>
>>>> diff --git a/meta/lib/oeqa/runtime/cases/apt.py b/meta/lib/oeqa/runtime/cases/apt.py
>>>> index 53745df93f..49f8714730 100644
>>>> --- a/meta/lib/oeqa/runtime/cases/apt.py
>>>> +++ b/meta/lib/oeqa/runtime/cases/apt.py
>>>> @@ -21,7 +21,7 @@ class AptRepoTest(AptTest):
>>>>
>>>> @classmethod
>>>> def setUpClass(cls):
>>>> - service_repo = os.path.join(cls.tc.td['DEPLOY_DIR_DEB'], 'all')
>>>> + service_repo = os.path.join(cls.tc.td['DEPLOY_DIR_DEB'], '')
>>>> cls.repo_server = HTTPService(service_repo,
>>>> '0.0.0.0', port=cls.tc.target.server_port,
>>>> logger=cls.tc.logger)
>>>> @@ -32,13 +32,18 @@ class AptRepoTest(AptTest):
>>>> cls.repo_server.stop()
>>>>
>>>> def setup_source_config_for_package_install(self):
>>>> - apt_get_source_server = 'http://%s:%s/' % (self.tc.target.server_ip, self.repo_server.port)
>>>> + apt_get_source_server = 'http:\/\/%s:%s' % (self.tc.target.server_ip, self.repo_server.port)
>>>> apt_get_sourceslist_dir = '/etc/apt/'
>>>> - self.target.run('cd %s; echo deb [ allow-insecure=yes ] %s ./ > sources.list' % (apt_get_sourceslist_dir, apt_get_source_server))
>>>> + self.target.run("cd %s; cp sources.list sources.list.bak; sed -i 's/\[trusted=yes\] http:\/\/bogus_ip:bogus_port/%s/g' sources.list" % (apt_get_sourceslist_dir, apt_get_source_server))
>>>>
>>>> def cleanup_source_config_for_package_install(self):
>>>> apt_get_sourceslist_dir = '/etc/apt/'
>>>> - self.target.run('cd %s; rm sources.list' % (apt_get_sourceslist_dir))
>>>> + self.target.run('cd %s; mv sources.list.bak sources.list' % (apt_get_sourceslist_dir))
>>>> +
>>>> + def setup_key(self):
>>>> + # the key is found on the target /etc/pki/packagefeed-gpg/
>>>> + # named PACKAGEFEED-GPG-KEY-poky-branch
>>>> + self.target.run('cd %s; apt-key add P*' % ('/etc/pki/packagefeed-gpg'))
>>>>
>>>> @skipIfNotFeature('package-management',
>>>> 'Test requires package-management to be in IMAGE_FEATURES')
>>>> @@ -47,7 +52,8 @@ class AptRepoTest(AptTest):
>>>> @OEHasPackage(['apt'])
>>>> def test_apt_install_from_repo(self):
>>>> self.setup_source_config_for_package_install()
>>>> + self.setup_key()
>>>> self.pkg('update')
>>>> self.pkg('remove --yes run-postinsts-dev')
>>>> - self.pkg('install --yes --allow-unauthenticated run-postinsts-dev')
>>>> + self.pkg('install --yes run-postinsts-dev')
>>>> self.cleanup_source_config_for_package_install()
>>>> diff --git a/meta/lib/oeqa/selftest/cases/runtime_test.py b/meta/lib/oeqa/selftest/cases/runtime_test.py
>>>> index 642f0eb637..7a75b95a99 100644
>>>> --- a/meta/lib/oeqa/selftest/cases/runtime_test.py
>>>> +++ b/meta/lib/oeqa/selftest/cases/runtime_test.py
>>>> @@ -162,6 +162,44 @@ class TestImage(OESelftestTestCase):
>>>> bitbake('core-image-full-cmdline socat')
>>>> bitbake('-c testimage core-image-full-cmdline')
>>>>
>>>> + def test_testimage_apt(self):
>>>> + """
>>>> + Summary: Check package feeds functionality for apt
>>>> + Expected: 1. Check that remote package feeds can be accessed
>>>> + Product: oe-core
>>>> + Author: Ferry Toth<fntoth@gmail.com>
>>>> + """
>>>> + if get_bb_var('DISTRO') == 'poky-tiny':
>>>> + self.skipTest('core-image-full-cmdline not buildable for poky-tiny')
>>>> +
>>>> + features = 'INHERIT += "testimage"\n'
>>>> + features += 'TEST_SUITES = "ping ssh apt.AptRepoTest.test_apt_install_from_repo"\n'
>>>> + # We don't yet know what the server ip and port will be - they will be patched
>>>> + # in at the start of the on-image test
>>>> + features += 'PACKAGE_FEED_URIS = "http://bogus_ip:bogus_port"\n'
>>>> + features += 'EXTRA_IMAGE_FEATURES += "package-management"\n'
>>>> + features += 'PACKAGE_CLASSES = "package_deb"\n'
>>>> + # We need gnupg on the target to install keys
>>>> + features += 'IMAGE_INSTALL:append:pn-core-image-full-cmdline = " gnupg"\n'
>>>> +
>>>> + bitbake('gnupg-native -c addto_recipe_sysroot')
>>>> +
>>>> + # Enable package feed signing
>>>> + self.gpg_home = tempfile.mkdtemp(prefix="oeqa-feed-sign-")
>>>> + self.track_for_cleanup(self.gpg_home)
>>>> + signing_key_dir = os.path.join(self.testlayer_path, 'files', 'signing')
>>>> + runCmd('gpgconf --list-dirs --homedir %s; gpg -v --batch --homedir %s --import %s' % (self.gpg_home, self.gpg_home, os.path.join(signing_key_dir, 'key.secret')), native_sysroot=get_bb_var("RECIPE_SYSROOT_NATIVE", "gnupg-native"), shell=True)
>>>> + features += 'INHERIT += "sign_package_feed"\n'
>>>> + features += 'PACKAGE_FEED_GPG_NAME = "testuser"\n'
>>>> + features += 'PACKAGE_FEED_GPG_PASSPHRASE_FILE = "%s"\n' % os.path.join(signing_key_dir, 'key.passphrase')
>>>> + features += 'GPG_PATH = "%s"\n' % self.gpg_home
>>>> + features += 'PSEUDO_IGNORE_PATHS .= ",%s"\n' % self.gpg_home
>>>> + self.write_config(features)
>>>> +
>>>> + # Build core-image-sato and testimage
>>>> + bitbake('core-image-full-cmdline socat')
>>>> + bitbake('-c testimage core-image-full-cmdline')
>>>> +
>>>> def test_testimage_virgl_gtk_sdl(self):
>>>> """
>>>> Summary: Check host-assisted accelerate OpenGL functionality in qemu with gtk and SDL frontends
>>> Thanks for working on this!
>>>
>>> Looking at the patches I wondered if this would break testimage and
>>> unfortunately it does:
>>>
>>> https://autobuilder.yoctoproject.org/typhoon/#/builders/50/builds/5013/steps/12/logs/stdio
>>> https://autobuilder.yoctoproject.org/typhoon/#/builders/76/builds/4975
>> That is weird, do I understand correctly that it fails on:
>> apt-get remove --yes run-postinsts-dev
>> Reading package lists...
>> Building dependency tree...
>> E: Unable to locate package run-postinsts-dev
>>
>> That is actually *) one line I didn't touch. I did note while testing
>> that I saw this exact message, however that was not counted as a fail.
>>
>> What could cause this? Because the complaint is it can't remove the
>> package because it was not installed.
>>
>> It would be trivial to remove the line
>>
>> *) self.pkg('remove --yes run-postinsts-dev')
>>
>> but how could it have passed the test before?
>
> I think the issue is you edited testimage which is a different set of tests
> which aren't just called by oe-selftest but by things like
That would be my first thought too, but...
because the failure seems to be on the line self.pkg('remove --yes
run-postinsts-dev'), that would mean the line self.pkg('update') passed.
And that should only pass if it finds a signed repository and has the
key installed (and believe me, I saw a log of that in the last week).
So, there may be a second thing wrong?
Do you know where I can find the log files referred to:
<..>tmp/work/qemux86-poky-linux/core-image-sato/1.0-r0/temp/log.do_testimage.35553
<..>tmp/work/qemux86-poky-linux/core-image-sato-sdk/1.0-r0/temp/log.do_testimage.35362
or could we do a 'quick' check by changing
self.pkg('update')
self.pkg('remove --yes run-postinsts-dev')
self.pkg('install --yes run-postinsts-dev')
to
self.pkg('update')
self.pkg('install --yes run-postinsts-dev')
self.pkg('remove --yes run-postinsts-dev')
?
>
> "bitbake core-image-sato -c testimage"
>
> as well. I'd suggest making the changes in testimage conditional on signing
> being configured.
Yes, regardless the above, we need to either make signing always enabled
in all test cases or detect whether signing is used.
Do you have a hint if there is a variable to test in class AptRepoTest
if PACKAGE_FEED_GPG_NAME has been set?
Otherwise I could just duplicate code and create
apt.AptRepoTest.test_apt_install_from_repo_signed.
What would you prefer?
> Cheers,
>
> Richard
>
>
[-- Attachment #2: Type: text/html, Size: 10907 bytes --]
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH v2 3/3] apt: add apt selftest to test signed package feeds
2022-04-05 15:23 ` Ferry Toth
@ 2022-04-06 11:40 ` Richard Purdie
2022-04-06 14:43 ` Ferry Toth
0 siblings, 1 reply; 17+ messages in thread
From: Richard Purdie @ 2022-04-06 11:40 UTC (permalink / raw)
To: Ferry Toth, openembedded-core
Cc: Xavier Berger, Alexander Kanavin, Ferry Toth
On Tue, 2022-04-05 at 17:23 +0200, Ferry Toth wrote:
> Op 04-04-2022 om 22:39 schreef Richard Purdie:
> On Mon, 2022-04-04 at 19:35 +0200, Ferry Toth wrote:
> > Op 04-04-2022 om 15:58 schreef Richard Purdie:
> > >
> > > > On Sun, 2022-04-03 at 21:50 +0200, Ferry Toth wrote:
> > > > Looking at the patches I wondered if this would break testimage and
> > > > unfortunately it does:
> > > >
> > > > https://autobuilder.yoctoproject.org/typhoon/#/builders/50/builds/5013/s
> > > > teps/12/logs/stdio
> > > > https://autobuilder.yoctoproject.org/typhoon/#/builders/76/builds/4975
> > > That is weird, do I understand correctly that it fails on:
> > > apt-get remove --yes run-postinsts-dev
> > > Reading package lists...
> > > Building dependency tree...
> > > E: Unable to locate package run-postinsts-dev
> > >
> > > That is actually *) one line I didn't touch. I did note while testing
> > > that I saw this exact message, however that was not counted as a fail.
> > >
> > > What could cause this? Because the complaint is it can't remove the
> > > package because it was not installed.
> > >
> > > It would be trivial to remove the line
> > >
> > > *) self.pkg('remove --yes run-postinsts-dev')
> > >
> > > but how could it have passed the test before?
> >
> > I think the issue is you edited testimage which is a different set of tests
> > which aren't just called by oe-selftest but by things like
> That would be my first thought too, but...
> because the failure seems to be on the line self.pkg('remove --yes run-
> postinsts-dev'), that would mean the line self.pkg('update') passed.
> And that should only pass if it finds a signed repository and has the key
> installed (and believe me, I saw a log of that in the last week).
> So, there may be a second thing wrong?
I was easily able to reproduce this locally and it shows the
setup_source_config_for_package_install() step fails and hence the sources
aren't setup correctly, hence the update probably works.
> Do you know where I can find the log files referred to:
> <..>tmp/work/qemux86-poky-linux/core-image-sato/1.0-
> r0/temp/log.do_testimage.35553
> <..>tmp/work/qemux86-poky-linux/core-image-sato-sdk/1.0-
> r0/temp/log.do_testimage.35362
We can get them off the autobuilder if needed but someone would have to manually
go in and find/share them. The issue does locally reproduce for me with a
"bitbake core-image-sato -c testimage" with package_deb set as the backend.
> or could we do a 'quick' check by changing
> self.pkg('update')
> self.pkg('remove --yes run-postinsts-dev')
> self.pkg('install --yes run-postinsts-dev')
> to
> self.pkg('update')
> self.pkg('install --yes run-postinsts-dev')
> self.pkg('remove --yes run-postinsts-dev')
> ?
I'm not convinced that would help us...
>
> >
> > "bitbake core-image-sato -c testimage"
> >
> > as well. I'd suggest making the changes in testimage conditional on signing
> > being configured.
> Yes, regardless the above, we need to either make signing always enabled in
> all test cases or detect whether signing is used.
> Do you have a hint if there is a variable to test in class AptRepoTest if
> PACKAGE_FEED_GPG_NAME has been set?
> Otherwise I could just duplicate code and create
> apt.AptRepoTest.test_apt_install_from_repo_signed.
> What would you prefer?
>
We should be able to test self.tc.td.get('PACKAGE_FEED_GPG_NAME') in the test
and handle accordingly?
I did merge the base changes into the release since I thought it was fair to get
the fixes in before it was built. We just need to get the test sorted now, I
think it is close.
Cheers,
Richard
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH v2 3/3] apt: add apt selftest to test signed package feeds
2022-04-06 11:40 ` Richard Purdie
@ 2022-04-06 14:43 ` Ferry Toth
2022-04-06 15:23 ` Richard Purdie
0 siblings, 1 reply; 17+ messages in thread
From: Ferry Toth @ 2022-04-06 14:43 UTC (permalink / raw)
To: Richard Purdie, openembedded-core
Cc: Xavier Berger, Alexander Kanavin, Ferry Toth
[-- Attachment #1: Type: text/plain, Size: 4362 bytes --]
Hi,
Op 06-04-2022 om 13:40 schreef Richard Purdie:
> On Tue, 2022-04-05 at 17:23 +0200, Ferry Toth wrote:
>> Op 04-04-2022 om 22:39 schreef Richard Purdie:
>> On Mon, 2022-04-04 at 19:35 +0200, Ferry Toth wrote:
>>> Op 04-04-2022 om 15:58 schreef Richard Purdie:
>>>>> On Sun, 2022-04-03 at 21:50 +0200, Ferry Toth wrote:
>>>>> Looking at the patches I wondered if this would break testimage and
>>>>> unfortunately it does:
>>>>>
>>>>> https://autobuilder.yoctoproject.org/typhoon/#/builders/50/builds/5013/s
>>>>> teps/12/logs/stdio
>>>>> https://autobuilder.yoctoproject.org/typhoon/#/builders/76/builds/4975
>>>> That is weird, do I understand correctly that it fails on:
>>>> apt-get remove --yes run-postinsts-dev
>>>> Reading package lists...
>>>> Building dependency tree...
>>>> E: Unable to locate package run-postinsts-dev
>>>>
>>>> That is actually *) one line I didn't touch. I did note while testing
>>>> that I saw this exact message, however that was not counted as a fail.
>>>>
>>>> What could cause this? Because the complaint is it can't remove the
>>>> package because it was not installed.
>>>>
>>>> It would be trivial to remove the line
>>>>
>>>> *) self.pkg('remove --yes run-postinsts-dev')
>>>>
>>>> but how could it have passed the test before?
>>> I think the issue is you edited testimage which is a different set of tests
>>> which aren't just called by oe-selftest but by things like
>> That would be my first thought too, but...
>> because the failure seems to be on the line self.pkg('remove --yes run-
>> postinsts-dev'), that would mean the line self.pkg('update') passed.
>> And that should only pass if it finds a signed repository and has the key
>> installed (and believe me, I saw a log of that in the last week).
>> So, there may be a second thing wrong?
> I was easily able to reproduce this locally and it shows the
> setup_source_config_for_package_install() step fails and hence the sources
> aren't setup correctly, hence the update probably works.
not correct, hence works. You lost me here, but I'll try to reproduce.
>> Do you know where I can find the log files referred to:
>> <..>tmp/work/qemux86-poky-linux/core-image-sato/1.0-
>> r0/temp/log.do_testimage.35553
>> <..>tmp/work/qemux86-poky-linux/core-image-sato-sdk/1.0-
>> r0/temp/log.do_testimage.35362
> We can get them off the autobuilder if needed but someone would have to manually
No, that would be too much work. I'll try to reproduce myself.
> go in and find/share them. The issue does locally reproduce for me with a
> "bitbake core-image-sato -c testimage" with package_deb set as the backend.
..in conf. But without PACKAGE_CLASSES, PACKAGE_FEED_GPG_NAME,
PACKAGE_FEED_GPG_PASSPHRASE_FILE?
>> or could we do a 'quick' check by changing
>> self.pkg('update')
>> self.pkg('remove --yes run-postinsts-dev')
>> self.pkg('install --yes run-postinsts-dev')
>> to
>> self.pkg('update')
>> self.pkg('install --yes run-postinsts-dev')
>> self.pkg('remove --yes run-postinsts-dev')
>> ?
> I'm not convinced that would help us...
I'll try locally.
>>
>>> "bitbake core-image-sato -c testimage"
>>>
>>> as well. I'd suggest making the changes in testimage conditional on signing
>>> being configured.
>> Yes, regardless the above, we need to either make signing always enabled in
>> all test cases or detect whether signing is used.
>> Do you have a hint if there is a variable to test in class AptRepoTest if
>> PACKAGE_FEED_GPG_NAME has been set?
>> Otherwise I could just duplicate code and create
>> apt.AptRepoTest.test_apt_install_from_repo_signed.
>> What would you prefer?
>>
> We should be able to test self.tc.td.get('PACKAGE_FEED_GPG_NAME') in the test
> and handle accordingly?
>
> I did merge the base changes into the release since I thought it was fair to get
> the fixes in before it was built. We just need to get the test sorted now, I
> think it is close.
Thanks for merging.
I'll fix the test, that's only fair.
One thing, the test "test_testimage_apt" is new. It needs to be
scheduled somewhere (where "test_testimage_dnf" is called i guess), I
didn't add that. Is that correct?
>
> Cheers,
>
> Richard
>
>
>
[-- Attachment #2: Type: text/html, Size: 7024 bytes --]
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH v2 3/3] apt: add apt selftest to test signed package feeds
2022-04-06 14:43 ` Ferry Toth
@ 2022-04-06 15:23 ` Richard Purdie
2022-04-06 19:44 ` Ferry Toth
0 siblings, 1 reply; 17+ messages in thread
From: Richard Purdie @ 2022-04-06 15:23 UTC (permalink / raw)
To: Ferry Toth, openembedded-core
Cc: Xavier Berger, Alexander Kanavin, Ferry Toth
On Wed, 2022-04-06 at 16:43 +0200, Ferry Toth wrote:
> Op 06-04-2022 om 13:40 schreef Richard Purdie:
> > On Tue, 2022-04-05 at 17:23 +0200, Ferry Toth wrote:
> > > Op 04-04-2022 om 22:39 schreef Richard Purdie:
> > > On Mon, 2022-04-04 at 19:35 +0200, Ferry Toth wrote:
> > > > Op 04-04-2022 om 15:58 schreef Richard Purdie:
> > > > > > On Sun, 2022-04-03 at 21:50 +0200, Ferry Toth wrote:
> > > > > > Looking at the patches I wondered if this would break testimage and
> > > > > > unfortunately it does:
> > > > > >
> > > > > > https://autobuilder.yoctoproject.org/typhoon/#/builders/50/builds/50
> > > > > > 13/s
> > > > > > teps/12/logs/stdio
> > > > > > https://autobuilder.yoctoproject.org/typhoon/#/builders/76/builds/49
> > > > > > 75
> > > > > That is weird, do I understand correctly that it fails on:
> > > > > apt-get remove --yes run-postinsts-dev
> > > > > Reading package lists...
> > > > > Building dependency tree...
> > > > > E: Unable to locate package run-postinsts-dev
> > > > >
> > > > > That is actually *) one line I didn't touch. I did note while testing
> > > > > that I saw this exact message, however that was not counted as a fail.
> > > > >
> > > > > What could cause this? Because the complaint is it can't remove the
> > > > > package because it was not installed.
> > > > >
> > > > > It would be trivial to remove the line
> > > > >
> > > > > *) self.pkg('remove --yes run-postinsts-dev')
> > > > >
> > > > > but how could it have passed the test before?
> > > > I think the issue is you edited testimage which is a different set of
> > > > tests
> > > > which aren't just called by oe-selftest but by things like
> > > That would be my first thought too, but...
> > > because the failure seems to be on the line self.pkg('remove --yes run-
> > > postinsts-dev'), that would mean the line self.pkg('update') passed.
> > > And that should only pass if it finds a signed repository and has the key
> > > installed (and believe me, I saw a log of that in the last week).
> > > So, there may be a second thing wrong?
> > I was easily able to reproduce this locally and it shows the
> > setup_source_config_for_package_install() step fails and hence the sources
> > aren't setup correctly, hence the update probably works.
> not correct, hence works. You lost me here, but I'll try to reproduce.
I mean the command doesn't work correctly. In my local logs I see:
DEBUG: Command: cd /etc/apt/; cp sources.list sources.list.bak; sed -i 's/\[trusted=yes\] http:\/\/bogus_ip:bogus_port/http:\/\/192.168.7.1:46599/g' sources.list
Status: 1 Output: cp: can't stat 'sources.list': No such file or directory
sed: sources.list: No such file or directory
>
>
> > go in and find/share them. The issue does locally reproduce for me with a
> > "bitbake core-image-sato -c testimage" with package_deb set as the backend.
> ..in conf. But without PACKAGE_CLASSES, PACKAGE_FEED_GPG_NAME,
> PACKAGE_FEED_GPG_PASSPHRASE_FILE?
Yes.
> >
> > > Yes, regardless the above, we need to either make signing always enabled
> > > in
> > > all test cases or detect whether signing is used.
> > > Do you have a hint if there is a variable to test in class AptRepoTest if
> > > PACKAGE_FEED_GPG_NAME has been set?
> > > Otherwise I could just duplicate code and create
> > > apt.AptRepoTest.test_apt_install_from_repo_signed.
> > > What would you prefer?
> > >
> > We should be able to test self.tc.td.get('PACKAGE_FEED_GPG_NAME') in the
> > test
> > and handle accordingly?
> >
> > I did merge the base changes into the release since I thought it was fair to
> > get
> > the fixes in before it was built. We just need to get the test sorted now, I
> > think it is close.
> Thanks for merging.
> I'll fix the test, that's only fair.
> One thing, the test "test_testimage_apt" is new. It needs to be scheduled
> somewhere (where "test_testimage_dnf" is called i guess), I didn't add that.
> Is that correct?
>
No, the autobuilder runs all the tests in oe-selftest so it should be covered
(and is why we saw the failures on the autobuilder).
Cheers,
Richard
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH v2 3/3] apt: add apt selftest to test signed package feeds
2022-04-06 15:23 ` Richard Purdie
@ 2022-04-06 19:44 ` Ferry Toth
2022-04-06 21:05 ` Richard Purdie
0 siblings, 1 reply; 17+ messages in thread
From: Ferry Toth @ 2022-04-06 19:44 UTC (permalink / raw)
To: Richard Purdie, openembedded-core
Cc: Xavier Berger, Alexander Kanavin, Alexandre Belloni
Hi
Op 06-04-2022 om 17:23 schreef Richard Purdie:
> On Wed, 2022-04-06 at 16:43 +0200, Ferry Toth wrote:
>> Op 06-04-2022 om 13:40 schreef Richard Purdie:
>>> On Tue, 2022-04-05 at 17:23 +0200, Ferry Toth wrote:
>>>> Op 04-04-2022 om 22:39 schreef Richard Purdie:
>>>> On Mon, 2022-04-04 at 19:35 +0200, Ferry Toth wrote:
>>>>> Op 04-04-2022 om 15:58 schreef Richard Purdie:
>>>>>>> On Sun, 2022-04-03 at 21:50 +0200, Ferry Toth wrote:
>>>>>>> Looking at the patches I wondered if this would break testimage and
>>>>>>> unfortunately it does:
>>>>>>>
>>>>>>> https://autobuilder.yoctoproject.org/typhoon/#/builders/50/builds/50
>>>>>>> 13/s
>>>>>>> teps/12/logs/stdio
>>>>>>> https://autobuilder.yoctoproject.org/typhoon/#/builders/76/builds/49
>>>>>>> 75
>>>>>> That is weird, do I understand correctly that it fails on:
>>>>>> apt-get remove --yes run-postinsts-dev
>>>>>> Reading package lists...
>>>>>> Building dependency tree...
>>>>>> E: Unable to locate package run-postinsts-dev
>>>>>>
>>>>>> That is actually *) one line I didn't touch. I did note while testing
>>>>>> that I saw this exact message, however that was not counted as a fail.
>>>>>>
>>>>>> What could cause this? Because the complaint is it can't remove the
>>>>>> package because it was not installed.
>>>>>>
>>>>>> It would be trivial to remove the line
>>>>>>
>>>>>> *) self.pkg('remove --yes run-postinsts-dev')
>>>>>>
>>>>>> but how could it have passed the test before?
>>>>> I think the issue is you edited testimage which is a different set of
>>>>> tests
>>>>> which aren't just called by oe-selftest but by things like
>>>> That would be my first thought too, but...
>>>> because the failure seems to be on the line self.pkg('remove --yes run-
>>>> postinsts-dev'), that would mean the line self.pkg('update') passed.
>>>> And that should only pass if it finds a signed repository and has the key
>>>> installed (and believe me, I saw a log of that in the last week).
>>>> So, there may be a second thing wrong?
>>> I was easily able to reproduce this locally and it shows the
>>> setup_source_config_for_package_install() step fails and hence the sources
>>> aren't setup correctly, hence the update probably works.
>> not correct, hence works. You lost me here, but I'll try to reproduce.
> I mean the command doesn't work correctly. In my local logs I see:
>
> DEBUG: Command: cd /etc/apt/; cp sources.list sources.list.bak; sed -i 's/\[trusted=yes\] http:\/\/bogus_ip:bogus_port/http:\/\/192.168.7.1:46599/g' sources.list
> Status: 1 Output: cp: can't stat 'sources.list': No such file or directory
> sed: sources.list: No such file or directory
>
>>
>>> go in and find/share them. The issue does locally reproduce for me with a
>>> "bitbake core-image-sato -c testimage" with package_deb set as the backend.
>> ..in conf. But without PACKAGE_CLASSES, PACKAGE_FEED_GPG_NAME,
>> PACKAGE_FEED_GPG_PASSPHRASE_FILE?
> Yes.
>
>>>> Yes, regardless the above, we need to either make signing always enabled
>>>> in
>>>> all test cases or detect whether signing is used.
>>>> Do you have a hint if there is a variable to test in class AptRepoTest if
>>>> PACKAGE_FEED_GPG_NAME has been set?
>>>> Otherwise I could just duplicate code and create
>>>> apt.AptRepoTest.test_apt_install_from_repo_signed.
>>>> What would you prefer?
>>>>
>>> We should be able to test self.tc.td.get('PACKAGE_FEED_GPG_NAME') in the
>>> test
>>> and handle accordingly?
>>>
>>> I did merge the base changes into the release since I thought it was fair to
>>> get
>>> the fixes in before it was built. We just need to get the test sorted now, I
>>> think it is close.
>> Thanks for merging.
>> I'll fix the test, that's only fair.
>> One thing, the test "test_testimage_apt" is new. It needs to be scheduled
>> somewhere (where "test_testimage_dnf" is called i guess), I didn't add that.
>> Is that correct?
>>
> No, the autobuilder runs all the tests in oe-selftest so it should be covered
> (and is why we saw the failures on the autobuilder).
I was running 'oe-selftest -K -r
runtime_test.TestImage.test_testimage_apt' whereas buildbot seems to be
running 'apt.AptRepoTest.test_apt_install_from_repo' directly.
However, test_testimage_apt is where keys are setup (keys found in
meta-selftest). So, where/when is test_testimage_apt called (or
test_testimage_dnf for that matter)?
> Cheers,
>
> Richard
>
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH v2 3/3] apt: add apt selftest to test signed package feeds
2022-04-06 19:44 ` Ferry Toth
@ 2022-04-06 21:05 ` Richard Purdie
2022-04-07 9:59 ` Ferry Toth
0 siblings, 1 reply; 17+ messages in thread
From: Richard Purdie @ 2022-04-06 21:05 UTC (permalink / raw)
To: Ferry Toth, openembedded-core
Cc: Xavier Berger, Alexander Kanavin, Alexandre Belloni
On Wed, 2022-04-06 at 21:44 +0200, Ferry Toth wrote:
> Hi
>
> Op 06-04-2022 om 17:23 schreef Richard Purdie:
> > On Wed, 2022-04-06 at 16:43 +0200, Ferry Toth wrote:
> > > Op 06-04-2022 om 13:40 schreef Richard Purdie:
> > > > On Tue, 2022-04-05 at 17:23 +0200, Ferry Toth wrote:
> > > > > Op 04-04-2022 om 22:39 schreef Richard Purdie:
> > > > > On Mon, 2022-04-04 at 19:35 +0200, Ferry Toth wrote:
> > > > > > Op 04-04-2022 om 15:58 schreef Richard Purdie:
> > > > > > > > On Sun, 2022-04-03 at 21:50 +0200, Ferry Toth wrote:
> > > > > > > > Looking at the patches I wondered if this would break testimage and
> > > > > > > > unfortunately it does:
> > > > > > > >
> > > > > > > > https://autobuilder.yoctoproject.org/typhoon/#/builders/50/builds/50
> > > > > > > > 13/s
> > > > > > > > teps/12/logs/stdio
> > > > > > > > https://autobuilder.yoctoproject.org/typhoon/#/builders/76/builds/49
> > > > > > > > 75
> > > > > > > That is weird, do I understand correctly that it fails on:
> > > > > > > apt-get remove --yes run-postinsts-dev
> > > > > > > Reading package lists...
> > > > > > > Building dependency tree...
> > > > > > > E: Unable to locate package run-postinsts-dev
> > > > > > >
> > > > > > > That is actually *) one line I didn't touch. I did note while testing
> > > > > > > that I saw this exact message, however that was not counted as a fail.
> > > > > > >
> > > > > > > What could cause this? Because the complaint is it can't remove the
> > > > > > > package because it was not installed.
> > > > > > >
> > > > > > > It would be trivial to remove the line
> > > > > > >
> > > > > > > *) self.pkg('remove --yes run-postinsts-dev')
> > > > > > >
> > > > > > > but how could it have passed the test before?
> > > > > > I think the issue is you edited testimage which is a different set of
> > > > > > tests
> > > > > > which aren't just called by oe-selftest but by things like
> > > > > That would be my first thought too, but...
> > > > > because the failure seems to be on the line self.pkg('remove --yes run-
> > > > > postinsts-dev'), that would mean the line self.pkg('update') passed.
> > > > > And that should only pass if it finds a signed repository and has the key
> > > > > installed (and believe me, I saw a log of that in the last week).
> > > > > So, there may be a second thing wrong?
> > > > I was easily able to reproduce this locally and it shows the
> > > > setup_source_config_for_package_install() step fails and hence the sources
> > > > aren't setup correctly, hence the update probably works.
> > > not correct, hence works. You lost me here, but I'll try to reproduce.
> > I mean the command doesn't work correctly. In my local logs I see:
> >
> > DEBUG: Command: cd /etc/apt/; cp sources.list sources.list.bak; sed -i 's/\[trusted=yes\] http:\/\/bogus_ip:bogus_port/http:\/\/192.168.7.1:46599/g' sources.list
> > Status: 1 Output: cp: can't stat 'sources.list': No such file or directory
> > sed: sources.list: No such file or directory
> >
> > >
> > > > go in and find/share them. The issue does locally reproduce for me with a
> > > > "bitbake core-image-sato -c testimage" with package_deb set as the backend.
> > > ..in conf. But without PACKAGE_CLASSES, PACKAGE_FEED_GPG_NAME,
> > > PACKAGE_FEED_GPG_PASSPHRASE_FILE?
> > Yes.
> >
> > > > > Yes, regardless the above, we need to either make signing always enabled
> > > > > in
> > > > > all test cases or detect whether signing is used.
> > > > > Do you have a hint if there is a variable to test in class AptRepoTest if
> > > > > PACKAGE_FEED_GPG_NAME has been set?
> > > > > Otherwise I could just duplicate code and create
> > > > > apt.AptRepoTest.test_apt_install_from_repo_signed.
> > > > > What would you prefer?
> > > > >
> > > > We should be able to test self.tc.td.get('PACKAGE_FEED_GPG_NAME') in the
> > > > test
> > > > and handle accordingly?
> > > >
> > > > I did merge the base changes into the release since I thought it was fair to
> > > > get
> > > > the fixes in before it was built. We just need to get the test sorted now, I
> > > > think it is close.
> > > Thanks for merging.
> > > I'll fix the test, that's only fair.
> > > One thing, the test "test_testimage_apt" is new. It needs to be scheduled
> > > somewhere (where "test_testimage_dnf" is called i guess), I didn't add that.
> > > Is that correct?
> > >
> > No, the autobuilder runs all the tests in oe-selftest so it should be covered
> > (and is why we saw the failures on the autobuilder).
>
> I was running 'oe-selftest -K -r
> runtime_test.TestImage.test_testimage_apt' whereas buildbot seems to be
> running 'apt.AptRepoTest.test_apt_install_from_repo' directly.
We have several types of test. There are two types in play here,
"oe-selftest -K -r runtime_test.TestImage.test_testimage_apt"
and
"bitbake core-image-sato -c testimage"
The latter testimage tests are often run every time we create images and running
testimage will trigger 'apt.AptRepoTest.test_apt_install_from_repo' if the image
has apt present and is built using debian package management.
We also run oe-selftest which triggers a testimage of it's own for a specfic
test case.
So we'd expect the normal testimage calls to not have the package signing
enabled and then we'd have the oe-selftest which specifically tests signing.
> However, test_testimage_apt is where keys are setup (keys found in
> meta-selftest). So, where/when is test_testimage_apt called (or
> test_testimage_dnf for that matter)?
Those are called by the oe-selftest call on the autobuilder. We run the oe-
selftest with a mask on the autobuilder so pretty much all of them run.
Cheers,
Richard
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH v2 3/3] apt: add apt selftest to test signed package feeds
2022-04-06 21:05 ` Richard Purdie
@ 2022-04-07 9:59 ` Ferry Toth
0 siblings, 0 replies; 17+ messages in thread
From: Ferry Toth @ 2022-04-07 9:59 UTC (permalink / raw)
To: Richard Purdie, openembedded-core
Cc: Xavier Berger, Alexander Kanavin, Alexandre Belloni
Op 06-04-2022 om 23:05 schreef Richard Purdie:
> On Wed, 2022-04-06 at 21:44 +0200, Ferry Toth wrote:
>> Hi
>>
>> Op 06-04-2022 om 17:23 schreef Richard Purdie:
>>> On Wed, 2022-04-06 at 16:43 +0200, Ferry Toth wrote:
>>>> Op 06-04-2022 om 13:40 schreef Richard Purdie:
>>>>> On Tue, 2022-04-05 at 17:23 +0200, Ferry Toth wrote:
>>>>>> Op 04-04-2022 om 22:39 schreef Richard Purdie:
>>>>>> On Mon, 2022-04-04 at 19:35 +0200, Ferry Toth wrote:
>>>>>>> Op 04-04-2022 om 15:58 schreef Richard Purdie:
>>>>>>>>> On Sun, 2022-04-03 at 21:50 +0200, Ferry Toth wrote:
>>>>>>>>> Looking at the patches I wondered if this would break testimage and
>>>>>>>>> unfortunately it does:
>>>>>>>>>
>>>>>>>>> https://autobuilder.yoctoproject.org/typhoon/#/builders/50/builds/50
>>>>>>>>> 13/s
>>>>>>>>> teps/12/logs/stdio
>>>>>>>>> https://autobuilder.yoctoproject.org/typhoon/#/builders/76/builds/49
>>>>>>>>> 75
>>>>>>>> That is weird, do I understand correctly that it fails on:
>>>>>>>> apt-get remove --yes run-postinsts-dev
>>>>>>>> Reading package lists...
>>>>>>>> Building dependency tree...
>>>>>>>> E: Unable to locate package run-postinsts-dev
>>>>>>>>
>>>>>>>> That is actually *) one line I didn't touch. I did note while testing
>>>>>>>> that I saw this exact message, however that was not counted as a fail.
>>>>>>>>
>>>>>>>> What could cause this? Because the complaint is it can't remove the
>>>>>>>> package because it was not installed.
>>>>>>>>
>>>>>>>> It would be trivial to remove the line
>>>>>>>>
>>>>>>>> *) self.pkg('remove --yes run-postinsts-dev')
>>>>>>>>
>>>>>>>> but how could it have passed the test before?
>>>>>>> I think the issue is you edited testimage which is a different set of
>>>>>>> tests
>>>>>>> which aren't just called by oe-selftest but by things like
>>>>>> That would be my first thought too, but...
>>>>>> because the failure seems to be on the line self.pkg('remove --yes run-
>>>>>> postinsts-dev'), that would mean the line self.pkg('update') passed.
>>>>>> And that should only pass if it finds a signed repository and has the key
>>>>>> installed (and believe me, I saw a log of that in the last week).
>>>>>> So, there may be a second thing wrong?
>>>>> I was easily able to reproduce this locally and it shows the
>>>>> setup_source_config_for_package_install() step fails and hence the sources
>>>>> aren't setup correctly, hence the update probably works.
>>>> not correct, hence works. You lost me here, but I'll try to reproduce.
>>> I mean the command doesn't work correctly. In my local logs I see:
>>>
>>> DEBUG: Command: cd /etc/apt/; cp sources.list sources.list.bak; sed -i 's/\[trusted=yes\] http:\/\/bogus_ip:bogus_port/http:\/\/192.168.7.1:46599/g' sources.list
>>> Status: 1 Output: cp: can't stat 'sources.list': No such file or directory
>>> sed: sources.list: No such file or directory
>>>
>>>>
>>>>> go in and find/share them. The issue does locally reproduce for me with a
>>>>> "bitbake core-image-sato -c testimage" with package_deb set as the backend.
>>>> ..in conf. But without PACKAGE_CLASSES, PACKAGE_FEED_GPG_NAME,
>>>> PACKAGE_FEED_GPG_PASSPHRASE_FILE?
>>> Yes.
>>>
>>>>>> Yes, regardless the above, we need to either make signing always enabled
>>>>>> in
>>>>>> all test cases or detect whether signing is used.
>>>>>> Do you have a hint if there is a variable to test in class AptRepoTest if
>>>>>> PACKAGE_FEED_GPG_NAME has been set?
>>>>>> Otherwise I could just duplicate code and create
>>>>>> apt.AptRepoTest.test_apt_install_from_repo_signed.
>>>>>> What would you prefer?
>>>>>>
>>>>> We should be able to test self.tc.td.get('PACKAGE_FEED_GPG_NAME') in the
>>>>> test
>>>>> and handle accordingly?
I'll try this for autodetecting the test case.
>>>>>
>>>>> I did merge the base changes into the release since I thought it was fair to
>>>>> get
>>>>> the fixes in before it was built. We just need to get the test sorted now, I
>>>>> think it is close.
>>>> Thanks for merging.
>>>> I'll fix the test, that's only fair.
>>>> One thing, the test "test_testimage_apt" is new. It needs to be scheduled
>>>> somewhere (where "test_testimage_dnf" is called i guess), I didn't add that.
>>>> Is that correct?
>>>>
>>> No, the autobuilder runs all the tests in oe-selftest so it should be covered
>>> (and is why we saw the failures on the autobuilder).
>> I was running 'oe-selftest -K -r
>> runtime_test.TestImage.test_testimage_apt' whereas buildbot seems to be
>> running 'apt.AptRepoTest.test_apt_install_from_repo' directly.
> We have several types of test. There are two types in play here,
>
> "oe-selftest -K -r runtime_test.TestImage.test_testimage_apt"
>
> and
>
> "bitbake core-image-sato -c testimage"
Thanks for the explanation. Yesterday evening I built Sato and was able
to reproduce the issue.
(that was a big build with half of build time spent on building Rust).
core-image-sato does not have a /etc/apt/sources.list, the selftest public key is not on the image and gpg is not installed. This explains the errors in the log.
> The latter testimage tests are often run every time we create images and running
> testimage will trigger 'apt.AptRepoTest.test_apt_install_from_repo' if the image
> has apt present and is built using debian package management.
>
> We also run oe-selftest which triggers a testimage of it's own for a specfic
> test case.
Got it.
> So we'd expect the normal testimage calls to not have the package signing
> enabled and then we'd have the oe-selftest which specifically tests signing.
>
>> However, test_testimage_apt is where keys are setup (keys found in
>> meta-selftest). So, where/when is test_testimage_apt called (or
>> test_testimage_dnf for that matter)?
> Those are called by the oe-selftest call on the autobuilder. We run the oe-
> selftest with a mask on the autobuilder so pretty much all of them run.
After looking at more detail at autobuilder schedules I get it.
> Cheers,
>
> Richard
>
>
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [OE-core] [PATCH v2 3/3] apt: add apt selftest to test signed package feeds
2022-04-04 13:58 ` Richard Purdie
2022-04-04 17:35 ` Ferry Toth
@ 2022-04-06 10:10 ` Alexandre Belloni
2022-04-06 15:16 ` Ferry Toth
1 sibling, 1 reply; 17+ messages in thread
From: Alexandre Belloni @ 2022-04-06 10:10 UTC (permalink / raw)
To: Richard Purdie
Cc: Ferry Toth, openembedded-core, Xavier Berger, Alexander Kanavin,
Ferry Toth
Hello,
On 04/04/2022 14:58:07+0100, Richard Purdie wrote:
> On Sun, 2022-04-03 at 21:50 +0200, Ferry Toth wrote:
> > From: Ferry Toth <ftoth@exalondelft.nl>
> >
> > Since Gatesgarth apt (1.8.2) has become more strict and doesn’t allow unsigned repositories by default.
> > Currently when building images this requirement is worked around by using [allow-insecure=yes] and
> > equivalently when performing selftest.
> >
> > Patches "gpg-sign: Add parameters to gpg signature function" and "package_manager: sign DEB package feeds"
> > enable signed DEB package feeds. This patch adds a runtime test for apt derived from the test_testimage_dnf
> > test. It creates a signed deb package feed, runs a qemu image to install the key and performs some package
> > management. To be able to install the key the gnupg package is added to the testimage.
> >
> > Signed-off-by: Ferry Toth <ftoth@exalondelft.nl>
> > ---
> > meta/lib/oeqa/runtime/cases/apt.py | 16 ++++++---
> > meta/lib/oeqa/selftest/cases/runtime_test.py | 38 ++++++++++++++++++++
> > 2 files changed, 49 insertions(+), 5 deletions(-)
> >
> > diff --git a/meta/lib/oeqa/runtime/cases/apt.py b/meta/lib/oeqa/runtime/cases/apt.py
> > index 53745df93f..49f8714730 100644
> > --- a/meta/lib/oeqa/runtime/cases/apt.py
> > +++ b/meta/lib/oeqa/runtime/cases/apt.py
> > @@ -21,7 +21,7 @@ class AptRepoTest(AptTest):
> >
> > @classmethod
> > def setUpClass(cls):
> > - service_repo = os.path.join(cls.tc.td['DEPLOY_DIR_DEB'], 'all')
> > + service_repo = os.path.join(cls.tc.td['DEPLOY_DIR_DEB'], '')
> > cls.repo_server = HTTPService(service_repo,
> > '0.0.0.0', port=cls.tc.target.server_port,
> > logger=cls.tc.logger)
> > @@ -32,13 +32,18 @@ class AptRepoTest(AptTest):
> > cls.repo_server.stop()
> >
> > def setup_source_config_for_package_install(self):
> > - apt_get_source_server = 'http://%s:%s/' % (self.tc.target.server_ip, self.repo_server.port)
> > + apt_get_source_server = 'http:\/\/%s:%s' % (self.tc.target.server_ip, self.repo_server.port)
> > apt_get_sourceslist_dir = '/etc/apt/'
> > - self.target.run('cd %s; echo deb [ allow-insecure=yes ] %s ./ > sources.list' % (apt_get_sourceslist_dir, apt_get_source_server))
> > + self.target.run("cd %s; cp sources.list sources.list.bak; sed -i 's/\[trusted=yes\] http:\/\/bogus_ip:bogus_port/%s/g' sources.list" % (apt_get_sourceslist_dir, apt_get_source_server))
> >
> > def cleanup_source_config_for_package_install(self):
> > apt_get_sourceslist_dir = '/etc/apt/'
> > - self.target.run('cd %s; rm sources.list' % (apt_get_sourceslist_dir))
> > + self.target.run('cd %s; mv sources.list.bak sources.list' % (apt_get_sourceslist_dir))
> > +
> > + def setup_key(self):
> > + # the key is found on the target /etc/pki/packagefeed-gpg/
> > + # named PACKAGEFEED-GPG-KEY-poky-branch
> > + self.target.run('cd %s; apt-key add P*' % ('/etc/pki/packagefeed-gpg'))
> >
> > @skipIfNotFeature('package-management',
> > 'Test requires package-management to be in IMAGE_FEATURES')
> > @@ -47,7 +52,8 @@ class AptRepoTest(AptTest):
> > @OEHasPackage(['apt'])
> > def test_apt_install_from_repo(self):
> > self.setup_source_config_for_package_install()
> > + self.setup_key()
> > self.pkg('update')
> > self.pkg('remove --yes run-postinsts-dev')
> > - self.pkg('install --yes --allow-unauthenticated run-postinsts-dev')
> > + self.pkg('install --yes run-postinsts-dev')
> > self.cleanup_source_config_for_package_install()
> > diff --git a/meta/lib/oeqa/selftest/cases/runtime_test.py b/meta/lib/oeqa/selftest/cases/runtime_test.py
> > index 642f0eb637..7a75b95a99 100644
> > --- a/meta/lib/oeqa/selftest/cases/runtime_test.py
> > +++ b/meta/lib/oeqa/selftest/cases/runtime_test.py
> > @@ -162,6 +162,44 @@ class TestImage(OESelftestTestCase):
> > bitbake('core-image-full-cmdline socat')
> > bitbake('-c testimage core-image-full-cmdline')
> >
> > + def test_testimage_apt(self):
> > + """
> > + Summary: Check package feeds functionality for apt
> > + Expected: 1. Check that remote package feeds can be accessed
> > + Product: oe-core
> > + Author: Ferry Toth <fntoth@gmail.com>
> > + """
> > + if get_bb_var('DISTRO') == 'poky-tiny':
> > + self.skipTest('core-image-full-cmdline not buildable for poky-tiny')
> > +
> > + features = 'INHERIT += "testimage"\n'
> > + features += 'TEST_SUITES = "ping ssh apt.AptRepoTest.test_apt_install_from_repo"\n'
> > + # We don't yet know what the server ip and port will be - they will be patched
> > + # in at the start of the on-image test
> > + features += 'PACKAGE_FEED_URIS = "http://bogus_ip:bogus_port"\n'
> > + features += 'EXTRA_IMAGE_FEATURES += "package-management"\n'
> > + features += 'PACKAGE_CLASSES = "package_deb"\n'
> > + # We need gnupg on the target to install keys
> > + features += 'IMAGE_INSTALL:append:pn-core-image-full-cmdline = " gnupg"\n'
> > +
> > + bitbake('gnupg-native -c addto_recipe_sysroot')
> > +
> > + # Enable package feed signing
> > + self.gpg_home = tempfile.mkdtemp(prefix="oeqa-feed-sign-")
> > + self.track_for_cleanup(self.gpg_home)
> > + signing_key_dir = os.path.join(self.testlayer_path, 'files', 'signing')
> > + runCmd('gpgconf --list-dirs --homedir %s; gpg -v --batch --homedir %s --import %s' % (self.gpg_home, self.gpg_home, os.path.join(signing_key_dir, 'key.secret')), native_sysroot=get_bb_var("RECIPE_SYSROOT_NATIVE", "gnupg-native"), shell=True)
> > + features += 'INHERIT += "sign_package_feed"\n'
> > + features += 'PACKAGE_FEED_GPG_NAME = "testuser"\n'
> > + features += 'PACKAGE_FEED_GPG_PASSPHRASE_FILE = "%s"\n' % os.path.join(signing_key_dir, 'key.passphrase')
> > + features += 'GPG_PATH = "%s"\n' % self.gpg_home
> > + features += 'PSEUDO_IGNORE_PATHS .= ",%s"\n' % self.gpg_home
> > + self.write_config(features)
> > +
> > + # Build core-image-sato and testimage
> > + bitbake('core-image-full-cmdline socat')
> > + bitbake('-c testimage core-image-full-cmdline')
> > +
> > def test_testimage_virgl_gtk_sdl(self):
> > """
> > Summary: Check host-assisted accelerate OpenGL functionality in qemu with gtk and SDL frontends
>
> Thanks for working on this!
>
> Looking at the patches I wondered if this would break testimage and
> unfortunately it does:
>
> https://autobuilder.yoctoproject.org/typhoon/#/builders/50/builds/5013/steps/12/logs/stdio
> https://autobuilder.yoctoproject.org/typhoon/#/builders/76/builds/4975
>
> however hopefully these shouldn't be too hard to fix?
>
> The rest of the build is still running.
I missed it at the time but I believe this is also the cause of:
https://autobuilder.yoctoproject.org/typhoon/#/builders/80/builds/3352/steps/15/logs/stdio
ERROR: package-index-1.0-r0 do_package_index: GPG exited with code 2: gpg: can't connect to the agent: IPC connect call failed
gpg: skipped "testuser": No secret key
gpg: signing failed: No secret key
--
Alexandre Belloni, co-owner and COO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [OE-core] [PATCH v2 3/3] apt: add apt selftest to test signed package feeds
2022-04-06 10:10 ` [OE-core] " Alexandre Belloni
@ 2022-04-06 15:16 ` Ferry Toth
0 siblings, 0 replies; 17+ messages in thread
From: Ferry Toth @ 2022-04-06 15:16 UTC (permalink / raw)
To: Alexandre Belloni, Richard Purdie
Cc: openembedded-core, Xavier Berger, Alexander Kanavin
[-- Attachment #1: Type: text/plain, Size: 7885 bytes --]
Hi
Op 06-04-2022 om 12:10 schreef Alexandre Belloni:
> Hello,
>
> On 04/04/2022 14:58:07+0100, Richard Purdie wrote:
>> On Sun, 2022-04-03 at 21:50 +0200, Ferry Toth wrote:
>>> From: Ferry Toth<ftoth@exalondelft.nl>
>>>
>>> Since Gatesgarth apt (1.8.2) has become more strict and doesn’t allow unsigned repositories by default.
>>> Currently when building images this requirement is worked around by using [allow-insecure=yes] and
>>> equivalently when performing selftest.
>>>
>>> Patches "gpg-sign: Add parameters to gpg signature function" and "package_manager: sign DEB package feeds"
>>> enable signed DEB package feeds. This patch adds a runtime test for apt derived from the test_testimage_dnf
>>> test. It creates a signed deb package feed, runs a qemu image to install the key and performs some package
>>> management. To be able to install the key the gnupg package is added to the testimage.
>>>
>>> Signed-off-by: Ferry Toth<ftoth@exalondelft.nl>
>>> ---
>>> meta/lib/oeqa/runtime/cases/apt.py | 16 ++++++---
>>> meta/lib/oeqa/selftest/cases/runtime_test.py | 38 ++++++++++++++++++++
>>> 2 files changed, 49 insertions(+), 5 deletions(-)
>>>
>>> diff --git a/meta/lib/oeqa/runtime/cases/apt.py b/meta/lib/oeqa/runtime/cases/apt.py
>>> index 53745df93f..49f8714730 100644
>>> --- a/meta/lib/oeqa/runtime/cases/apt.py
>>> +++ b/meta/lib/oeqa/runtime/cases/apt.py
>>> @@ -21,7 +21,7 @@ class AptRepoTest(AptTest):
>>>
>>> @classmethod
>>> def setUpClass(cls):
>>> - service_repo = os.path.join(cls.tc.td['DEPLOY_DIR_DEB'], 'all')
>>> + service_repo = os.path.join(cls.tc.td['DEPLOY_DIR_DEB'], '')
>>> cls.repo_server = HTTPService(service_repo,
>>> '0.0.0.0', port=cls.tc.target.server_port,
>>> logger=cls.tc.logger)
>>> @@ -32,13 +32,18 @@ class AptRepoTest(AptTest):
>>> cls.repo_server.stop()
>>>
>>> def setup_source_config_for_package_install(self):
>>> - apt_get_source_server = 'http://%s:%s/' % (self.tc.target.server_ip, self.repo_server.port)
>>> + apt_get_source_server = 'http:\/\/%s:%s' % (self.tc.target.server_ip, self.repo_server.port)
>>> apt_get_sourceslist_dir = '/etc/apt/'
>>> - self.target.run('cd %s; echo deb [ allow-insecure=yes ] %s ./ > sources.list' % (apt_get_sourceslist_dir, apt_get_source_server))
>>> + self.target.run("cd %s; cp sources.list sources.list.bak; sed -i 's/\[trusted=yes\] http:\/\/bogus_ip:bogus_port/%s/g' sources.list" % (apt_get_sourceslist_dir, apt_get_source_server))
>>>
>>> def cleanup_source_config_for_package_install(self):
>>> apt_get_sourceslist_dir = '/etc/apt/'
>>> - self.target.run('cd %s; rm sources.list' % (apt_get_sourceslist_dir))
>>> + self.target.run('cd %s; mv sources.list.bak sources.list' % (apt_get_sourceslist_dir))
>>> +
>>> + def setup_key(self):
>>> + # the key is found on the target /etc/pki/packagefeed-gpg/
>>> + # named PACKAGEFEED-GPG-KEY-poky-branch
>>> + self.target.run('cd %s; apt-key add P*' % ('/etc/pki/packagefeed-gpg'))
>>>
>>> @skipIfNotFeature('package-management',
>>> 'Test requires package-management to be in IMAGE_FEATURES')
>>> @@ -47,7 +52,8 @@ class AptRepoTest(AptTest):
>>> @OEHasPackage(['apt'])
>>> def test_apt_install_from_repo(self):
>>> self.setup_source_config_for_package_install()
>>> + self.setup_key()
>>> self.pkg('update')
>>> self.pkg('remove --yes run-postinsts-dev')
>>> - self.pkg('install --yes --allow-unauthenticated run-postinsts-dev')
>>> + self.pkg('install --yes run-postinsts-dev')
>>> self.cleanup_source_config_for_package_install()
>>> diff --git a/meta/lib/oeqa/selftest/cases/runtime_test.py b/meta/lib/oeqa/selftest/cases/runtime_test.py
>>> index 642f0eb637..7a75b95a99 100644
>>> --- a/meta/lib/oeqa/selftest/cases/runtime_test.py
>>> +++ b/meta/lib/oeqa/selftest/cases/runtime_test.py
>>> @@ -162,6 +162,44 @@ class TestImage(OESelftestTestCase):
>>> bitbake('core-image-full-cmdline socat')
>>> bitbake('-c testimage core-image-full-cmdline')
>>>
>>> + def test_testimage_apt(self):
>>> + """
>>> + Summary: Check package feeds functionality for apt
>>> + Expected: 1. Check that remote package feeds can be accessed
>>> + Product: oe-core
>>> + Author: Ferry Toth<fntoth@gmail.com>
>>> + """
>>> + if get_bb_var('DISTRO') == 'poky-tiny':
>>> + self.skipTest('core-image-full-cmdline not buildable for poky-tiny')
>>> +
>>> + features = 'INHERIT += "testimage"\n'
>>> + features += 'TEST_SUITES = "ping ssh apt.AptRepoTest.test_apt_install_from_repo"\n'
>>> + # We don't yet know what the server ip and port will be - they will be patched
>>> + # in at the start of the on-image test
>>> + features += 'PACKAGE_FEED_URIS = "http://bogus_ip:bogus_port"\n'
>>> + features += 'EXTRA_IMAGE_FEATURES += "package-management"\n'
>>> + features += 'PACKAGE_CLASSES = "package_deb"\n'
>>> + # We need gnupg on the target to install keys
>>> + features += 'IMAGE_INSTALL:append:pn-core-image-full-cmdline = " gnupg"\n'
>>> +
>>> + bitbake('gnupg-native -c addto_recipe_sysroot')
>>> +
>>> + # Enable package feed signing
>>> + self.gpg_home = tempfile.mkdtemp(prefix="oeqa-feed-sign-")
>>> + self.track_for_cleanup(self.gpg_home)
>>> + signing_key_dir = os.path.join(self.testlayer_path, 'files', 'signing')
>>> + runCmd('gpgconf --list-dirs --homedir %s; gpg -v --batch --homedir %s --import %s' % (self.gpg_home, self.gpg_home, os.path.join(signing_key_dir, 'key.secret')), native_sysroot=get_bb_var("RECIPE_SYSROOT_NATIVE", "gnupg-native"), shell=True)
>>> + features += 'INHERIT += "sign_package_feed"\n'
>>> + features += 'PACKAGE_FEED_GPG_NAME = "testuser"\n'
>>> + features += 'PACKAGE_FEED_GPG_PASSPHRASE_FILE = "%s"\n' % os.path.join(signing_key_dir, 'key.passphrase')
>>> + features += 'GPG_PATH = "%s"\n' % self.gpg_home
>>> + features += 'PSEUDO_IGNORE_PATHS .= ",%s"\n' % self.gpg_home
>>> + self.write_config(features)
>>> +
>>> + # Build core-image-sato and testimage
>>> + bitbake('core-image-full-cmdline socat')
>>> + bitbake('-c testimage core-image-full-cmdline')
>>> +
>>> def test_testimage_virgl_gtk_sdl(self):
>>> """
>>> Summary: Check host-assisted accelerate OpenGL functionality in qemu with gtk and SDL frontends
>> Thanks for working on this!
>>
>> Looking at the patches I wondered if this would break testimage and
>> unfortunately it does:
>>
>> https://autobuilder.yoctoproject.org/typhoon/#/builders/50/builds/5013/steps/12/logs/stdio
>> https://autobuilder.yoctoproject.org/typhoon/#/builders/76/builds/4975
>>
>> however hopefully these shouldn't be too hard to fix?
>>
>> The rest of the build is still running.
> I missed it at the time but I believe this is also the cause of:
>
> https://autobuilder.yoctoproject.org/typhoon/#/builders/80/builds/3352/steps/15/logs/stdio
>
> ERROR: package-index-1.0-r0 do_package_index: GPG exited with code 2: gpg: can't connect to the agent: IPC connect call failed
> gpg: skipped "testuser": No secret key
> gpg: signing failed: No secret key
This seems related but not exact the same.
It seems do_package_index wants to generate a signed deb repo but no key
is provided. But IIUC you have PACKAGE_CLASSES = "package_rpm", so why
is runtime_test.TestImage.test_testimage_apt run?
[-- Attachment #2: Type: text/html, Size: 8783 bytes --]
^ permalink raw reply [flat|nested] 17+ messages in thread