package_manager: support for signed DEB package feeds

package_manager: support for signed DEB package feeds

[Ferry Toth]

[PATCH v2 0/3] package_manager: support for signed DEB package feeds
[PATCH v2 1/3] gpg-sign: Add parameters to gpg signature function
[PATCH v2 2/3] package_manager: sign DEB package feeds
[PATCH v2 3/3] apt: add apt selftest to test signed package feeds

Since Gatesgarth apt (1.8.2) has become more strict and doesn’t allow unsigned repositories by default.
Currently when building images this requirement is worked around by using [allow-insecure=yes] and
equivalently when performing selftest.
    
Patches "gpg-sign: Add parameters to gpg signature function" and "package_manager: sign deb package feeds"
enable signed deb package feeds. This patch adds a runtime test for apt derived from the test_testimage_dnf
test. It creates a signed deb package feed, runs a qemu image to install the key and performs some package
management. To be able to install the key the gnupg package is added to the testimage.
    
These patches makes deb a first class citizen as ipk and rpm.

Patches have been in use in meta-intel-edison since Gatesgarth, 
see https://edison-fw.github.io/meta-intel-edison/5.0-Creating-a-deb-repository.html

Changes in V2:
 - Added runtime test for signed deb package feeds (Richard Purdie)

[PATCH v2 0/3] *** SUBJECT HERE ***

[Ferry Toth]

From: Ferry Toth <ftoth@exalondelft.nl>

*** BLURB HERE ***

Ferry Toth (2):
  package_manager: sign DEB package feeds
  apt: add apt selftest to test signed package feeds

Xavier Berger (1):
  gpg-sign: Add parameters to gpg signature function

 meta/lib/oe/gpg_sign.py                      |  6 +++-
 meta/lib/oe/package_manager/deb/__init__.py  | 19 ++++++++--
 meta/lib/oeqa/runtime/cases/apt.py           | 16 ++++++---
 meta/lib/oeqa/selftest/cases/runtime_test.py | 38 ++++++++++++++++++++
 4 files changed, 70 insertions(+), 9 deletions(-)

-- 
2.32.0

[PATCH v2 1/3] gpg-sign: Add parameters to gpg signature function

[Ferry Toth]

From: Xavier Berger <xavier.berger@bio-logic.net>

output_suffix: If defined, add output_suffix as file name extension.
use_sha256: If True, use sha256 for gpg as digest algorithm

Signed-off-by: Xavier Berger <xavier.berger@bio-logic.net>
Signed-off-by: Ferry Toth <ftoth@exalondelft.nl>
---
 meta/lib/oe/gpg_sign.py | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/meta/lib/oe/gpg_sign.py b/meta/lib/oe/gpg_sign.py
index 1bce6cb792..aa9bb49f2c 100644
--- a/meta/lib/oe/gpg_sign.py
+++ b/meta/lib/oe/gpg_sign.py
@@ -58,7 +58,7 @@ class LocalSigner(object):
         for i in range(0, len(files), sign_chunk):
             subprocess.check_output(shlex.split(cmd + ' '.join(files[i:i+sign_chunk])), stderr=subprocess.STDOUT)
 
-    def detach_sign(self, input_file, keyid, passphrase_file, passphrase=None, armor=True):
+    def detach_sign(self, input_file, keyid, passphrase_file, passphrase=None, armor=True, output_suffix=None, use_sha256=False):
         """Create a detached signature of a file"""
 
         if passphrase_file and passphrase:
@@ -71,6 +71,10 @@ class LocalSigner(object):
             cmd += ['--homedir', self.gpg_path]
         if armor:
             cmd += ['--armor']
+        if output_suffix:
+            cmd += ['-o', input_file + "." + output_suffix]
+        if use_sha256:
+            cmd += ['--digest-algo', "SHA256"]
 
         #gpg > 2.1 supports password pipes only through the loopback interface
         #gpg < 2.1 errors out if given unknown parameters
-- 
2.32.0

[PATCH v2 2/3] package_manager: sign DEB package feeds

[Ferry Toth]

From: Ferry Toth <ftoth@exalondelft.nl>

Implement debian package repository signature.
For each Release file created in repository subdirectory, a signature
Release.gpg is created.

Signature is performed using gpg backend when the following variables
are set in local.conf:
PACKAGE_CLASSES += "sign_package_feed"
PACKAGE_FEED_GPG_NAME = "<Id of GPG key>"
PACKAGE_FEED_GPG_PASSPHRASE_FILE="<path to password file>"

Signed-off-by: Xavier Berger <xavier.berger@bio-logic.net>
Signed-off-by: Ferry Toth <ftoth@exalondelft.nl>
---
 meta/lib/oe/package_manager/deb/__init__.py | 19 ++++++++++++++++---
 1 file changed, 16 insertions(+), 3 deletions(-)

diff --git a/meta/lib/oe/package_manager/deb/__init__.py b/meta/lib/oe/package_manager/deb/__init__.py
index 9f112ae25b..86ddb130ad 100644
--- a/meta/lib/oe/package_manager/deb/__init__.py
+++ b/meta/lib/oe/package_manager/deb/__init__.py
@@ -53,6 +53,7 @@ class DpkgIndexer(Indexer):
 
         index_cmds = []
         deb_dirs_found = False
+        index_sign_files = set()
         for arch in arch_list:
             arch_dir = os.path.join(self.deploy_dir, arch)
             if not os.path.isdir(arch_dir):
@@ -62,7 +63,10 @@ class DpkgIndexer(Indexer):
 
             cmd += "%s -fcn Packages > Packages.gz;" % gzip
 
-            with open(os.path.join(arch_dir, "Release"), "w+") as release:
+            release_file = os.path.join(arch_dir, "Release")
+            index_sign_files.add(release_file)
+
+            with open(release_file, "w+") as release:
                 release.write("Label: %s\n" % arch)
 
             cmd += "PSEUDO_UNLOAD=1 %s release . >> Release" % apt_ftparchive
@@ -76,8 +80,17 @@ class DpkgIndexer(Indexer):
             return
 
         oe.utils.multiprocess_launch(create_index, index_cmds, self.d)
-        if self.d.getVar('PACKAGE_FEED_SIGN') == '1':
-            raise NotImplementedError('Package feed signing not implementd for dpkg')
+        if self.d.getVar('PACKAGE_FEED_SIGN', True) == '1':
+            signer = get_signer(self.d, self.d.getVar('PACKAGE_FEED_GPG_BACKEND', True))
+        else:
+            signer = None
+        if signer:
+            for f in index_sign_files:
+                signer.detach_sign(f,
+                                   self.d.getVar('PACKAGE_FEED_GPG_NAME', True),
+                                   self.d.getVar('PACKAGE_FEED_GPG_PASSPHRASE_FILE', True),
+                                   output_suffix="gpg",
+                                   use_sha256=True)
 
 class PMPkgsList(PkgsList):
 
-- 
2.32.0

[PATCH v2 3/3] apt: add apt selftest to test signed package feeds

[Ferry Toth]

From: Ferry Toth <ftoth@exalondelft.nl>

Since Gatesgarth apt (1.8.2) has become more strict and doesn’t allow unsigned repositories by default.
Currently when building images this requirement is worked around by using [allow-insecure=yes] and
equivalently when performing selftest.

Patches "gpg-sign: Add parameters to gpg signature function" and "package_manager: sign DEB package feeds"
enable signed DEB package feeds. This patch adds a runtime test for apt derived from the test_testimage_dnf
test. It creates a signed deb package feed, runs a qemu image to install the key and performs some package
management. To be able to install the key the gnupg package is added to the testimage.

Signed-off-by: Ferry Toth <ftoth@exalondelft.nl>
---
 meta/lib/oeqa/runtime/cases/apt.py           | 16 ++++++---
 meta/lib/oeqa/selftest/cases/runtime_test.py | 38 ++++++++++++++++++++
 2 files changed, 49 insertions(+), 5 deletions(-)

diff --git a/meta/lib/oeqa/runtime/cases/apt.py b/meta/lib/oeqa/runtime/cases/apt.py
index 53745df93f..49f8714730 100644
--- a/meta/lib/oeqa/runtime/cases/apt.py
+++ b/meta/lib/oeqa/runtime/cases/apt.py
@@ -21,7 +21,7 @@ class AptRepoTest(AptTest):
 
     @classmethod
     def setUpClass(cls):
-        service_repo = os.path.join(cls.tc.td['DEPLOY_DIR_DEB'], 'all')
+        service_repo = os.path.join(cls.tc.td['DEPLOY_DIR_DEB'], '')
         cls.repo_server = HTTPService(service_repo,
                                       '0.0.0.0', port=cls.tc.target.server_port,
                                       logger=cls.tc.logger)
@@ -32,13 +32,18 @@ class AptRepoTest(AptTest):
         cls.repo_server.stop()
 
     def setup_source_config_for_package_install(self):
-        apt_get_source_server = 'http://%s:%s/' % (self.tc.target.server_ip, self.repo_server.port)
+        apt_get_source_server = 'http:\/\/%s:%s' % (self.tc.target.server_ip, self.repo_server.port)
         apt_get_sourceslist_dir = '/etc/apt/'
-        self.target.run('cd %s; echo deb [ allow-insecure=yes ] %s ./ > sources.list' % (apt_get_sourceslist_dir, apt_get_source_server))
+        self.target.run("cd %s; cp sources.list sources.list.bak; sed -i 's/\[trusted=yes\] http:\/\/bogus_ip:bogus_port/%s/g' sources.list" % (apt_get_sourceslist_dir, apt_get_source_server))
 
     def cleanup_source_config_for_package_install(self):
         apt_get_sourceslist_dir = '/etc/apt/'
-        self.target.run('cd %s; rm sources.list' % (apt_get_sourceslist_dir))
+        self.target.run('cd %s; mv sources.list.bak sources.list' % (apt_get_sourceslist_dir))
+
+    def setup_key(self):
+        # the key is found on the target /etc/pki/packagefeed-gpg/
+        # named PACKAGEFEED-GPG-KEY-poky-branch
+        self.target.run('cd %s; apt-key add P*' % ('/etc/pki/packagefeed-gpg'))
 
     @skipIfNotFeature('package-management',
                       'Test requires package-management to be in IMAGE_FEATURES')
@@ -47,7 +52,8 @@ class AptRepoTest(AptTest):
     @OEHasPackage(['apt'])
     def test_apt_install_from_repo(self):
         self.setup_source_config_for_package_install()
+        self.setup_key()
         self.pkg('update')
         self.pkg('remove --yes run-postinsts-dev')
-        self.pkg('install --yes --allow-unauthenticated run-postinsts-dev')
+        self.pkg('install --yes run-postinsts-dev')
         self.cleanup_source_config_for_package_install()
diff --git a/meta/lib/oeqa/selftest/cases/runtime_test.py b/meta/lib/oeqa/selftest/cases/runtime_test.py
index 642f0eb637..7a75b95a99 100644
--- a/meta/lib/oeqa/selftest/cases/runtime_test.py
+++ b/meta/lib/oeqa/selftest/cases/runtime_test.py
@@ -162,6 +162,44 @@ class TestImage(OESelftestTestCase):
         bitbake('core-image-full-cmdline socat')
         bitbake('-c testimage core-image-full-cmdline')
 
+    def test_testimage_apt(self):
+        """
+        Summary: Check package feeds functionality for apt
+        Expected: 1. Check that remote package feeds can be accessed
+        Product: oe-core
+        Author: Ferry Toth <fntoth@gmail.com>
+        """
+        if get_bb_var('DISTRO') == 'poky-tiny':
+            self.skipTest('core-image-full-cmdline not buildable for poky-tiny')
+
+        features = 'INHERIT += "testimage"\n'
+        features += 'TEST_SUITES = "ping ssh apt.AptRepoTest.test_apt_install_from_repo"\n'
+        # We don't yet know what the server ip and port will be - they will be patched
+        # in at the start of the on-image test
+        features += 'PACKAGE_FEED_URIS = "http://bogus_ip:bogus_port"\n'
+        features += 'EXTRA_IMAGE_FEATURES += "package-management"\n'
+        features += 'PACKAGE_CLASSES = "package_deb"\n'
+        # We need  gnupg on the target to install keys
+        features += 'IMAGE_INSTALL:append:pn-core-image-full-cmdline = " gnupg"\n'
+
+        bitbake('gnupg-native -c addto_recipe_sysroot')
+
+        # Enable package feed signing
+        self.gpg_home = tempfile.mkdtemp(prefix="oeqa-feed-sign-")
+        self.track_for_cleanup(self.gpg_home)
+        signing_key_dir = os.path.join(self.testlayer_path, 'files', 'signing')
+        runCmd('gpgconf --list-dirs --homedir %s; gpg -v --batch --homedir %s --import %s' % (self.gpg_home, self.gpg_home, os.path.join(signing_key_dir, 'key.secret')), native_sysroot=get_bb_var("RECIPE_SYSROOT_NATIVE", "gnupg-native"), shell=True)
+        features += 'INHERIT += "sign_package_feed"\n'
+        features += 'PACKAGE_FEED_GPG_NAME = "testuser"\n'
+        features += 'PACKAGE_FEED_GPG_PASSPHRASE_FILE = "%s"\n' % os.path.join(signing_key_dir, 'key.passphrase')
+        features += 'GPG_PATH = "%s"\n' % self.gpg_home
+        features += 'PSEUDO_IGNORE_PATHS .= ",%s"\n' % self.gpg_home
+        self.write_config(features)
+
+        # Build core-image-sato and testimage
+        bitbake('core-image-full-cmdline socat')
+        bitbake('-c testimage core-image-full-cmdline')
+
     def test_testimage_virgl_gtk_sdl(self):
         """
         Summary: Check host-assisted accelerate OpenGL functionality in qemu with gtk and SDL frontends
-- 
2.32.0

Re: [PATCH v2 3/3] apt: add apt selftest to test signed package feeds

[Richard Purdie]

On Sun, 2022-04-03 at 21:50 +0200, Ferry Toth wrote:
> From: Ferry Toth <ftoth@exalondelft.nl>
> 
> Since Gatesgarth apt (1.8.2) has become more strict and doesn’t allow unsigned repositories by default.
> Currently when building images this requirement is worked around by using [allow-insecure=yes] and
> equivalently when performing selftest.
> 
> Patches "gpg-sign: Add parameters to gpg signature function" and "package_manager: sign DEB package feeds"
> enable signed DEB package feeds. This patch adds a runtime test for apt derived from the test_testimage_dnf
> test. It creates a signed deb package feed, runs a qemu image to install the key and performs some package
> management. To be able to install the key the gnupg package is added to the testimage.
> 
> Signed-off-by: Ferry Toth <ftoth@exalondelft.nl>
> ---
>  meta/lib/oeqa/runtime/cases/apt.py           | 16 ++++++---
>  meta/lib/oeqa/selftest/cases/runtime_test.py | 38 ++++++++++++++++++++
>  2 files changed, 49 insertions(+), 5 deletions(-)
> 
> diff --git a/meta/lib/oeqa/runtime/cases/apt.py b/meta/lib/oeqa/runtime/cases/apt.py
> index 53745df93f..49f8714730 100644
> --- a/meta/lib/oeqa/runtime/cases/apt.py
> +++ b/meta/lib/oeqa/runtime/cases/apt.py
> @@ -21,7 +21,7 @@ class AptRepoTest(AptTest):
>  
>      @classmethod
>      def setUpClass(cls):
> -        service_repo = os.path.join(cls.tc.td['DEPLOY_DIR_DEB'], 'all')
> +        service_repo = os.path.join(cls.tc.td['DEPLOY_DIR_DEB'], '')
>          cls.repo_server = HTTPService(service_repo,
>                                        '0.0.0.0', port=cls.tc.target.server_port,
>                                        logger=cls.tc.logger)
> @@ -32,13 +32,18 @@ class AptRepoTest(AptTest):
>          cls.repo_server.stop()
>  
>      def setup_source_config_for_package_install(self):
> -        apt_get_source_server = 'http://%s:%s/' % (self.tc.target.server_ip, self.repo_server.port)
> +        apt_get_source_server = 'http:\/\/%s:%s' % (self.tc.target.server_ip, self.repo_server.port)
>          apt_get_sourceslist_dir = '/etc/apt/'
> -        self.target.run('cd %s; echo deb [ allow-insecure=yes ] %s ./ > sources.list' % (apt_get_sourceslist_dir, apt_get_source_server))
> +        self.target.run("cd %s; cp sources.list sources.list.bak; sed -i 's/\[trusted=yes\] http:\/\/bogus_ip:bogus_port/%s/g' sources.list" % (apt_get_sourceslist_dir, apt_get_source_server))
>  
>      def cleanup_source_config_for_package_install(self):
>          apt_get_sourceslist_dir = '/etc/apt/'
> -        self.target.run('cd %s; rm sources.list' % (apt_get_sourceslist_dir))
> +        self.target.run('cd %s; mv sources.list.bak sources.list' % (apt_get_sourceslist_dir))
> +
> +    def setup_key(self):
> +        # the key is found on the target /etc/pki/packagefeed-gpg/
> +        # named PACKAGEFEED-GPG-KEY-poky-branch
> +        self.target.run('cd %s; apt-key add P*' % ('/etc/pki/packagefeed-gpg'))
>  
>      @skipIfNotFeature('package-management',
>                        'Test requires package-management to be in IMAGE_FEATURES')
> @@ -47,7 +52,8 @@ class AptRepoTest(AptTest):
>      @OEHasPackage(['apt'])
>      def test_apt_install_from_repo(self):
>          self.setup_source_config_for_package_install()
> +        self.setup_key()
>          self.pkg('update')
>          self.pkg('remove --yes run-postinsts-dev')
> -        self.pkg('install --yes --allow-unauthenticated run-postinsts-dev')
> +        self.pkg('install --yes run-postinsts-dev')
>          self.cleanup_source_config_for_package_install()
> diff --git a/meta/lib/oeqa/selftest/cases/runtime_test.py b/meta/lib/oeqa/selftest/cases/runtime_test.py
> index 642f0eb637..7a75b95a99 100644
> --- a/meta/lib/oeqa/selftest/cases/runtime_test.py
> +++ b/meta/lib/oeqa/selftest/cases/runtime_test.py
> @@ -162,6 +162,44 @@ class TestImage(OESelftestTestCase):
>          bitbake('core-image-full-cmdline socat')
>          bitbake('-c testimage core-image-full-cmdline')
>  
> +    def test_testimage_apt(self):
> +        """
> +        Summary: Check package feeds functionality for apt
> +        Expected: 1. Check that remote package feeds can be accessed
> +        Product: oe-core
> +        Author: Ferry Toth <fntoth@gmail.com>
> +        """
> +        if get_bb_var('DISTRO') == 'poky-tiny':
> +            self.skipTest('core-image-full-cmdline not buildable for poky-tiny')
> +
> +        features = 'INHERIT += "testimage"\n'
> +        features += 'TEST_SUITES = "ping ssh apt.AptRepoTest.test_apt_install_from_repo"\n'
> +        # We don't yet know what the server ip and port will be - they will be patched
> +        # in at the start of the on-image test
> +        features += 'PACKAGE_FEED_URIS = "http://bogus_ip:bogus_port"\n'
> +        features += 'EXTRA_IMAGE_FEATURES += "package-management"\n'
> +        features += 'PACKAGE_CLASSES = "package_deb"\n'
> +        # We need  gnupg on the target to install keys
> +        features += 'IMAGE_INSTALL:append:pn-core-image-full-cmdline = " gnupg"\n'
> +
> +        bitbake('gnupg-native -c addto_recipe_sysroot')
> +
> +        # Enable package feed signing
> +        self.gpg_home = tempfile.mkdtemp(prefix="oeqa-feed-sign-")
> +        self.track_for_cleanup(self.gpg_home)
> +        signing_key_dir = os.path.join(self.testlayer_path, 'files', 'signing')
> +        runCmd('gpgconf --list-dirs --homedir %s; gpg -v --batch --homedir %s --import %s' % (self.gpg_home, self.gpg_home, os.path.join(signing_key_dir, 'key.secret')), native_sysroot=get_bb_var("RECIPE_SYSROOT_NATIVE", "gnupg-native"), shell=True)
> +        features += 'INHERIT += "sign_package_feed"\n'
> +        features += 'PACKAGE_FEED_GPG_NAME = "testuser"\n'
> +        features += 'PACKAGE_FEED_GPG_PASSPHRASE_FILE = "%s"\n' % os.path.join(signing_key_dir, 'key.passphrase')
> +        features += 'GPG_PATH = "%s"\n' % self.gpg_home
> +        features += 'PSEUDO_IGNORE_PATHS .= ",%s"\n' % self.gpg_home
> +        self.write_config(features)
> +
> +        # Build core-image-sato and testimage
> +        bitbake('core-image-full-cmdline socat')
> +        bitbake('-c testimage core-image-full-cmdline')
> +
>      def test_testimage_virgl_gtk_sdl(self):
>          """
>          Summary: Check host-assisted accelerate OpenGL functionality in qemu with gtk and SDL frontends

Thanks for working on this!

Looking at the patches I wondered if this would break testimage and
unfortunately it does:

https://autobuilder.yoctoproject.org/typhoon/#/builders/50/builds/5013/steps/12/logs/stdio
https://autobuilder.yoctoproject.org/typhoon/#/builders/76/builds/4975

however hopefully these shouldn't be too hard to fix?

The rest of the build is still running.

Cheers,

Richard

Re: [PATCH v2 3/3] apt: add apt selftest to test signed package feeds

[Ferry Toth]

Hi,

Op 04-04-2022 om 15:58 schreef Richard Purdie:
> On Sun, 2022-04-03 at 21:50 +0200, Ferry Toth wrote:
>> From: Ferry Toth <ftoth@exalondelft.nl>
>>
>> Since Gatesgarth apt (1.8.2) has become more strict and doesn’t allow unsigned repositories by default.
>> Currently when building images this requirement is worked around by using [allow-insecure=yes] and
>> equivalently when performing selftest.
>>
>> Patches "gpg-sign: Add parameters to gpg signature function" and "package_manager: sign DEB package feeds"
>> enable signed DEB package feeds. This patch adds a runtime test for apt derived from the test_testimage_dnf
>> test. It creates a signed deb package feed, runs a qemu image to install the key and performs some package
>> management. To be able to install the key the gnupg package is added to the testimage.
>>
>> Signed-off-by: Ferry Toth <ftoth@exalondelft.nl>
>> ---
>>   meta/lib/oeqa/runtime/cases/apt.py           | 16 ++++++---
>>   meta/lib/oeqa/selftest/cases/runtime_test.py | 38 ++++++++++++++++++++
>>   2 files changed, 49 insertions(+), 5 deletions(-)
>>
>> diff --git a/meta/lib/oeqa/runtime/cases/apt.py b/meta/lib/oeqa/runtime/cases/apt.py
>> index 53745df93f..49f8714730 100644
>> --- a/meta/lib/oeqa/runtime/cases/apt.py
>> +++ b/meta/lib/oeqa/runtime/cases/apt.py
>> @@ -21,7 +21,7 @@ class AptRepoTest(AptTest):
>>   
>>       @classmethod
>>       def setUpClass(cls):
>> -        service_repo = os.path.join(cls.tc.td['DEPLOY_DIR_DEB'], 'all')
>> +        service_repo = os.path.join(cls.tc.td['DEPLOY_DIR_DEB'], '')
>>           cls.repo_server = HTTPService(service_repo,
>>                                         '0.0.0.0', port=cls.tc.target.server_port,
>>                                         logger=cls.tc.logger)
>> @@ -32,13 +32,18 @@ class AptRepoTest(AptTest):
>>           cls.repo_server.stop()
>>   
>>       def setup_source_config_for_package_install(self):
>> -        apt_get_source_server = 'http://%s:%s/' % (self.tc.target.server_ip, self.repo_server.port)
>> +        apt_get_source_server = 'http:\/\/%s:%s' % (self.tc.target.server_ip, self.repo_server.port)
>>           apt_get_sourceslist_dir = '/etc/apt/'
>> -        self.target.run('cd %s; echo deb [ allow-insecure=yes ] %s ./ > sources.list' % (apt_get_sourceslist_dir, apt_get_source_server))
>> +        self.target.run("cd %s; cp sources.list sources.list.bak; sed -i 's/\[trusted=yes\] http:\/\/bogus_ip:bogus_port/%s/g' sources.list" % (apt_get_sourceslist_dir, apt_get_source_server))
>>   
>>       def cleanup_source_config_for_package_install(self):
>>           apt_get_sourceslist_dir = '/etc/apt/'
>> -        self.target.run('cd %s; rm sources.list' % (apt_get_sourceslist_dir))
>> +        self.target.run('cd %s; mv sources.list.bak sources.list' % (apt_get_sourceslist_dir))
>> +
>> +    def setup_key(self):
>> +        # the key is found on the target /etc/pki/packagefeed-gpg/
>> +        # named PACKAGEFEED-GPG-KEY-poky-branch
>> +        self.target.run('cd %s; apt-key add P*' % ('/etc/pki/packagefeed-gpg'))
>>   
>>       @skipIfNotFeature('package-management',
>>                         'Test requires package-management to be in IMAGE_FEATURES')
>> @@ -47,7 +52,8 @@ class AptRepoTest(AptTest):
>>       @OEHasPackage(['apt'])
>>       def test_apt_install_from_repo(self):
>>           self.setup_source_config_for_package_install()
>> +        self.setup_key()
>>           self.pkg('update')
>>           self.pkg('remove --yes run-postinsts-dev')
>> -        self.pkg('install --yes --allow-unauthenticated run-postinsts-dev')
>> +        self.pkg('install --yes run-postinsts-dev')
>>           self.cleanup_source_config_for_package_install()
>> diff --git a/meta/lib/oeqa/selftest/cases/runtime_test.py b/meta/lib/oeqa/selftest/cases/runtime_test.py
>> index 642f0eb637..7a75b95a99 100644
>> --- a/meta/lib/oeqa/selftest/cases/runtime_test.py
>> +++ b/meta/lib/oeqa/selftest/cases/runtime_test.py
>> @@ -162,6 +162,44 @@ class TestImage(OESelftestTestCase):
>>           bitbake('core-image-full-cmdline socat')
>>           bitbake('-c testimage core-image-full-cmdline')
>>   
>> +    def test_testimage_apt(self):
>> +        """
>> +        Summary: Check package feeds functionality for apt
>> +        Expected: 1. Check that remote package feeds can be accessed
>> +        Product: oe-core
>> +        Author: Ferry Toth <fntoth@gmail.com>
>> +        """
>> +        if get_bb_var('DISTRO') == 'poky-tiny':
>> +            self.skipTest('core-image-full-cmdline not buildable for poky-tiny')
>> +
>> +        features = 'INHERIT += "testimage"\n'
>> +        features += 'TEST_SUITES = "ping ssh apt.AptRepoTest.test_apt_install_from_repo"\n'
>> +        # We don't yet know what the server ip and port will be - they will be patched
>> +        # in at the start of the on-image test
>> +        features += 'PACKAGE_FEED_URIS = "http://bogus_ip:bogus_port"\n'
>> +        features += 'EXTRA_IMAGE_FEATURES += "package-management"\n'
>> +        features += 'PACKAGE_CLASSES = "package_deb"\n'
>> +        # We need  gnupg on the target to install keys
>> +        features += 'IMAGE_INSTALL:append:pn-core-image-full-cmdline = " gnupg"\n'
>> +
>> +        bitbake('gnupg-native -c addto_recipe_sysroot')
>> +
>> +        # Enable package feed signing
>> +        self.gpg_home = tempfile.mkdtemp(prefix="oeqa-feed-sign-")
>> +        self.track_for_cleanup(self.gpg_home)
>> +        signing_key_dir = os.path.join(self.testlayer_path, 'files', 'signing')
>> +        runCmd('gpgconf --list-dirs --homedir %s; gpg -v --batch --homedir %s --import %s' % (self.gpg_home, self.gpg_home, os.path.join(signing_key_dir, 'key.secret')), native_sysroot=get_bb_var("RECIPE_SYSROOT_NATIVE", "gnupg-native"), shell=True)
>> +        features += 'INHERIT += "sign_package_feed"\n'
>> +        features += 'PACKAGE_FEED_GPG_NAME = "testuser"\n'
>> +        features += 'PACKAGE_FEED_GPG_PASSPHRASE_FILE = "%s"\n' % os.path.join(signing_key_dir, 'key.passphrase')
>> +        features += 'GPG_PATH = "%s"\n' % self.gpg_home
>> +        features += 'PSEUDO_IGNORE_PATHS .= ",%s"\n' % self.gpg_home
>> +        self.write_config(features)
>> +
>> +        # Build core-image-sato and testimage
>> +        bitbake('core-image-full-cmdline socat')
>> +        bitbake('-c testimage core-image-full-cmdline')
>> +
>>       def test_testimage_virgl_gtk_sdl(self):
>>           """
>>           Summary: Check host-assisted accelerate OpenGL functionality in qemu with gtk and SDL frontends
> 
> Thanks for working on this!
> 
> Looking at the patches I wondered if this would break testimage and
> unfortunately it does:
> 
> https://autobuilder.yoctoproject.org/typhoon/#/builders/50/builds/5013/steps/12/logs/stdio
> https://autobuilder.yoctoproject.org/typhoon/#/builders/76/builds/4975

That is weird, do I understand correctly that it fails on:
  apt-get remove --yes run-postinsts-dev
Reading package lists...
Building dependency tree...
E: Unable to locate package run-postinsts-dev

That is actually *) one line I didn't touch. I did note while testing 
that I saw this exact message, however that was not counted as a fail.

What could cause this? Because the complaint is it can't remove the 
package because it was not installed.

It would be trivial to remove the line

*) self.pkg('remove --yes run-postinsts-dev')

but how could it have passed the test before?

> however hopefully these shouldn't be too hard to fix?
> 
> The rest of the build is still running.
> 
> Cheers,
> 
> Richard
> 

Re: [PATCH v2 3/3] apt: add apt selftest to test signed package feeds

[Richard Purdie]

On Mon, 2022-04-04 at 19:35 +0200, Ferry Toth wrote:
> Hi,
> 
> Op 04-04-2022 om 15:58 schreef Richard Purdie:
> > On Sun, 2022-04-03 at 21:50 +0200, Ferry Toth wrote:
> > > From: Ferry Toth <ftoth@exalondelft.nl>
> > > 
> > > Since Gatesgarth apt (1.8.2) has become more strict and doesn’t allow unsigned repositories by default.
> > > Currently when building images this requirement is worked around by using [allow-insecure=yes] and
> > > equivalently when performing selftest.
> > > 
> > > Patches "gpg-sign: Add parameters to gpg signature function" and "package_manager: sign DEB package feeds"
> > > enable signed DEB package feeds. This patch adds a runtime test for apt derived from the test_testimage_dnf
> > > test. It creates a signed deb package feed, runs a qemu image to install the key and performs some package
> > > management. To be able to install the key the gnupg package is added to the testimage.
> > > 
> > > Signed-off-by: Ferry Toth <ftoth@exalondelft.nl>
> > > ---
> > >   meta/lib/oeqa/runtime/cases/apt.py           | 16 ++++++---
> > >   meta/lib/oeqa/selftest/cases/runtime_test.py | 38 ++++++++++++++++++++
> > >   2 files changed, 49 insertions(+), 5 deletions(-)
> > > 
> > > diff --git a/meta/lib/oeqa/runtime/cases/apt.py b/meta/lib/oeqa/runtime/cases/apt.py
> > > index 53745df93f..49f8714730 100644
> > > --- a/meta/lib/oeqa/runtime/cases/apt.py
> > > +++ b/meta/lib/oeqa/runtime/cases/apt.py
> > > @@ -21,7 +21,7 @@ class AptRepoTest(AptTest):
> > >   
> > >       @classmethod
> > >       def setUpClass(cls):
> > > -        service_repo = os.path.join(cls.tc.td['DEPLOY_DIR_DEB'], 'all')
> > > +        service_repo = os.path.join(cls.tc.td['DEPLOY_DIR_DEB'], '')
> > >           cls.repo_server = HTTPService(service_repo,
> > >                                         '0.0.0.0', port=cls.tc.target.server_port,
> > >                                         logger=cls.tc.logger)
> > > @@ -32,13 +32,18 @@ class AptRepoTest(AptTest):
> > >           cls.repo_server.stop()
> > >   
> > >       def setup_source_config_for_package_install(self):
> > > -        apt_get_source_server = 'http://%s:%s/' % (self.tc.target.server_ip, self.repo_server.port)
> > > +        apt_get_source_server = 'http:\/\/%s:%s' % (self.tc.target.server_ip, self.repo_server.port)
> > >           apt_get_sourceslist_dir = '/etc/apt/'
> > > -        self.target.run('cd %s; echo deb [ allow-insecure=yes ] %s ./ > sources.list' % (apt_get_sourceslist_dir, apt_get_source_server))
> > > +        self.target.run("cd %s; cp sources.list sources.list.bak; sed -i 's/\[trusted=yes\] http:\/\/bogus_ip:bogus_port/%s/g' sources.list" % (apt_get_sourceslist_dir, apt_get_source_server))
> > >   
> > >       def cleanup_source_config_for_package_install(self):
> > >           apt_get_sourceslist_dir = '/etc/apt/'
> > > -        self.target.run('cd %s; rm sources.list' % (apt_get_sourceslist_dir))
> > > +        self.target.run('cd %s; mv sources.list.bak sources.list' % (apt_get_sourceslist_dir))
> > > +
> > > +    def setup_key(self):
> > > +        # the key is found on the target /etc/pki/packagefeed-gpg/
> > > +        # named PACKAGEFEED-GPG-KEY-poky-branch
> > > +        self.target.run('cd %s; apt-key add P*' % ('/etc/pki/packagefeed-gpg'))
> > >   
> > >       @skipIfNotFeature('package-management',
> > >                         'Test requires package-management to be in IMAGE_FEATURES')
> > > @@ -47,7 +52,8 @@ class AptRepoTest(AptTest):
> > >       @OEHasPackage(['apt'])
> > >       def test_apt_install_from_repo(self):
> > >           self.setup_source_config_for_package_install()
> > > +        self.setup_key()
> > >           self.pkg('update')
> > >           self.pkg('remove --yes run-postinsts-dev')
> > > -        self.pkg('install --yes --allow-unauthenticated run-postinsts-dev')
> > > +        self.pkg('install --yes run-postinsts-dev')
> > >           self.cleanup_source_config_for_package_install()
> > > diff --git a/meta/lib/oeqa/selftest/cases/runtime_test.py b/meta/lib/oeqa/selftest/cases/runtime_test.py
> > > index 642f0eb637..7a75b95a99 100644
> > > --- a/meta/lib/oeqa/selftest/cases/runtime_test.py
> > > +++ b/meta/lib/oeqa/selftest/cases/runtime_test.py
> > > @@ -162,6 +162,44 @@ class TestImage(OESelftestTestCase):
> > >           bitbake('core-image-full-cmdline socat')
> > >           bitbake('-c testimage core-image-full-cmdline')
> > >   
> > > +    def test_testimage_apt(self):
> > > +        """
> > > +        Summary: Check package feeds functionality for apt
> > > +        Expected: 1. Check that remote package feeds can be accessed
> > > +        Product: oe-core
> > > +        Author: Ferry Toth <fntoth@gmail.com>
> > > +        """
> > > +        if get_bb_var('DISTRO') == 'poky-tiny':
> > > +            self.skipTest('core-image-full-cmdline not buildable for poky-tiny')
> > > +
> > > +        features = 'INHERIT += "testimage"\n'
> > > +        features += 'TEST_SUITES = "ping ssh apt.AptRepoTest.test_apt_install_from_repo"\n'
> > > +        # We don't yet know what the server ip and port will be - they will be patched
> > > +        # in at the start of the on-image test
> > > +        features += 'PACKAGE_FEED_URIS = "http://bogus_ip:bogus_port"\n'
> > > +        features += 'EXTRA_IMAGE_FEATURES += "package-management"\n'
> > > +        features += 'PACKAGE_CLASSES = "package_deb"\n'
> > > +        # We need  gnupg on the target to install keys
> > > +        features += 'IMAGE_INSTALL:append:pn-core-image-full-cmdline = " gnupg"\n'
> > > +
> > > +        bitbake('gnupg-native -c addto_recipe_sysroot')
> > > +
> > > +        # Enable package feed signing
> > > +        self.gpg_home = tempfile.mkdtemp(prefix="oeqa-feed-sign-")
> > > +        self.track_for_cleanup(self.gpg_home)
> > > +        signing_key_dir = os.path.join(self.testlayer_path, 'files', 'signing')
> > > +        runCmd('gpgconf --list-dirs --homedir %s; gpg -v --batch --homedir %s --import %s' % (self.gpg_home, self.gpg_home, os.path.join(signing_key_dir, 'key.secret')), native_sysroot=get_bb_var("RECIPE_SYSROOT_NATIVE", "gnupg-native"), shell=True)
> > > +        features += 'INHERIT += "sign_package_feed"\n'
> > > +        features += 'PACKAGE_FEED_GPG_NAME = "testuser"\n'
> > > +        features += 'PACKAGE_FEED_GPG_PASSPHRASE_FILE = "%s"\n' % os.path.join(signing_key_dir, 'key.passphrase')
> > > +        features += 'GPG_PATH = "%s"\n' % self.gpg_home
> > > +        features += 'PSEUDO_IGNORE_PATHS .= ",%s"\n' % self.gpg_home
> > > +        self.write_config(features)
> > > +
> > > +        # Build core-image-sato and testimage
> > > +        bitbake('core-image-full-cmdline socat')
> > > +        bitbake('-c testimage core-image-full-cmdline')
> > > +
> > >       def test_testimage_virgl_gtk_sdl(self):
> > >           """
> > >           Summary: Check host-assisted accelerate OpenGL functionality in qemu with gtk and SDL frontends
> > 
> > Thanks for working on this!
> > 
> > Looking at the patches I wondered if this would break testimage and
> > unfortunately it does:
> > 
> > https://autobuilder.yoctoproject.org/typhoon/#/builders/50/builds/5013/steps/12/logs/stdio
> > https://autobuilder.yoctoproject.org/typhoon/#/builders/76/builds/4975
> 
> That is weird, do I understand correctly that it fails on:
>   apt-get remove --yes run-postinsts-dev
> Reading package lists...
> Building dependency tree...
> E: Unable to locate package run-postinsts-dev
> 
> That is actually *) one line I didn't touch. I did note while testing 
> that I saw this exact message, however that was not counted as a fail.
> 
> What could cause this? Because the complaint is it can't remove the 
> package because it was not installed.
> 
> It would be trivial to remove the line
> 
> *) self.pkg('remove --yes run-postinsts-dev')
> 
> but how could it have passed the test before?


I think the issue is you edited testimage which is a different set of tests
which aren't just called by oe-selftest but by things like 

"bitbake core-image-sato -c testimage"

as well. I'd suggest making the changes in testimage conditional on signing
being configured.

Cheers,

Richard

Re: [PATCH v2 3/3] apt: add apt selftest to test signed package feeds

[Ferry Toth]

[-- Attachment #1: Type: text/plain, Size: 9640 bytes --]

Hi,

Op 04-04-2022 om 22:39 schreef Richard Purdie:
> On Mon, 2022-04-04 at 19:35 +0200, Ferry Toth wrote:
>> Hi,
>>
>> Op 04-04-2022 om 15:58 schreef Richard Purdie:
>>> On Sun, 2022-04-03 at 21:50 +0200, Ferry Toth wrote:
>>>> From: Ferry Toth<ftoth@exalondelft.nl>
>>>>
>>>> Since Gatesgarth apt (1.8.2) has become more strict and doesn’t allow unsigned repositories by default.
>>>> Currently when building images this requirement is worked around by using [allow-insecure=yes] and
>>>> equivalently when performing selftest.
>>>>
>>>> Patches "gpg-sign: Add parameters to gpg signature function" and "package_manager: sign DEB package feeds"
>>>> enable signed DEB package feeds. This patch adds a runtime test for apt derived from the test_testimage_dnf
>>>> test. It creates a signed deb package feed, runs a qemu image to install the key and performs some package
>>>> management. To be able to install the key the gnupg package is added to the testimage.
>>>>
>>>> Signed-off-by: Ferry Toth<ftoth@exalondelft.nl>
>>>> ---
>>>>    meta/lib/oeqa/runtime/cases/apt.py           | 16 ++++++---
>>>>    meta/lib/oeqa/selftest/cases/runtime_test.py | 38 ++++++++++++++++++++
>>>>    2 files changed, 49 insertions(+), 5 deletions(-)
>>>>
>>>> diff --git a/meta/lib/oeqa/runtime/cases/apt.py b/meta/lib/oeqa/runtime/cases/apt.py
>>>> index 53745df93f..49f8714730 100644
>>>> --- a/meta/lib/oeqa/runtime/cases/apt.py
>>>> +++ b/meta/lib/oeqa/runtime/cases/apt.py
>>>> @@ -21,7 +21,7 @@ class AptRepoTest(AptTest):
>>>>    
>>>>        @classmethod
>>>>        def setUpClass(cls):
>>>> -        service_repo = os.path.join(cls.tc.td['DEPLOY_DIR_DEB'], 'all')
>>>> +        service_repo = os.path.join(cls.tc.td['DEPLOY_DIR_DEB'], '')
>>>>            cls.repo_server = HTTPService(service_repo,
>>>>                                          '0.0.0.0', port=cls.tc.target.server_port,
>>>>                                          logger=cls.tc.logger)
>>>> @@ -32,13 +32,18 @@ class AptRepoTest(AptTest):
>>>>            cls.repo_server.stop()
>>>>    
>>>>        def setup_source_config_for_package_install(self):
>>>> -        apt_get_source_server = 'http://%s:%s/' % (self.tc.target.server_ip, self.repo_server.port)
>>>> +        apt_get_source_server = 'http:\/\/%s:%s' % (self.tc.target.server_ip, self.repo_server.port)
>>>>            apt_get_sourceslist_dir = '/etc/apt/'
>>>> -        self.target.run('cd %s; echo deb [ allow-insecure=yes ] %s ./ > sources.list' % (apt_get_sourceslist_dir, apt_get_source_server))
>>>> +        self.target.run("cd %s; cp sources.list sources.list.bak; sed -i 's/\[trusted=yes\] http:\/\/bogus_ip:bogus_port/%s/g' sources.list" % (apt_get_sourceslist_dir, apt_get_source_server))
>>>>    
>>>>        def cleanup_source_config_for_package_install(self):
>>>>            apt_get_sourceslist_dir = '/etc/apt/'
>>>> -        self.target.run('cd %s; rm sources.list' % (apt_get_sourceslist_dir))
>>>> +        self.target.run('cd %s; mv sources.list.bak sources.list' % (apt_get_sourceslist_dir))
>>>> +
>>>> +    def setup_key(self):
>>>> +        # the key is found on the target /etc/pki/packagefeed-gpg/
>>>> +        # named PACKAGEFEED-GPG-KEY-poky-branch
>>>> +        self.target.run('cd %s; apt-key add P*' % ('/etc/pki/packagefeed-gpg'))
>>>>    
>>>>        @skipIfNotFeature('package-management',
>>>>                          'Test requires package-management to be in IMAGE_FEATURES')
>>>> @@ -47,7 +52,8 @@ class AptRepoTest(AptTest):
>>>>        @OEHasPackage(['apt'])
>>>>        def test_apt_install_from_repo(self):
>>>>            self.setup_source_config_for_package_install()
>>>> +        self.setup_key()
>>>>            self.pkg('update')
>>>>            self.pkg('remove --yes run-postinsts-dev')
>>>> -        self.pkg('install --yes --allow-unauthenticated run-postinsts-dev')
>>>> +        self.pkg('install --yes run-postinsts-dev')
>>>>            self.cleanup_source_config_for_package_install()
>>>> diff --git a/meta/lib/oeqa/selftest/cases/runtime_test.py b/meta/lib/oeqa/selftest/cases/runtime_test.py
>>>> index 642f0eb637..7a75b95a99 100644
>>>> --- a/meta/lib/oeqa/selftest/cases/runtime_test.py
>>>> +++ b/meta/lib/oeqa/selftest/cases/runtime_test.py
>>>> @@ -162,6 +162,44 @@ class TestImage(OESelftestTestCase):
>>>>            bitbake('core-image-full-cmdline socat')
>>>>            bitbake('-c testimage core-image-full-cmdline')
>>>>    
>>>> +    def test_testimage_apt(self):
>>>> +        """
>>>> +        Summary: Check package feeds functionality for apt
>>>> +        Expected: 1. Check that remote package feeds can be accessed
>>>> +        Product: oe-core
>>>> +        Author: Ferry Toth<fntoth@gmail.com>
>>>> +        """
>>>> +        if get_bb_var('DISTRO') == 'poky-tiny':
>>>> +            self.skipTest('core-image-full-cmdline not buildable for poky-tiny')
>>>> +
>>>> +        features = 'INHERIT += "testimage"\n'
>>>> +        features += 'TEST_SUITES = "ping ssh apt.AptRepoTest.test_apt_install_from_repo"\n'
>>>> +        # We don't yet know what the server ip and port will be - they will be patched
>>>> +        # in at the start of the on-image test
>>>> +        features += 'PACKAGE_FEED_URIS = "http://bogus_ip:bogus_port"\n'
>>>> +        features += 'EXTRA_IMAGE_FEATURES += "package-management"\n'
>>>> +        features += 'PACKAGE_CLASSES = "package_deb"\n'
>>>> +        # We need  gnupg on the target to install keys
>>>> +        features += 'IMAGE_INSTALL:append:pn-core-image-full-cmdline = " gnupg"\n'
>>>> +
>>>> +        bitbake('gnupg-native -c addto_recipe_sysroot')
>>>> +
>>>> +        # Enable package feed signing
>>>> +        self.gpg_home = tempfile.mkdtemp(prefix="oeqa-feed-sign-")
>>>> +        self.track_for_cleanup(self.gpg_home)
>>>> +        signing_key_dir = os.path.join(self.testlayer_path, 'files', 'signing')
>>>> +        runCmd('gpgconf --list-dirs --homedir %s; gpg -v --batch --homedir %s --import %s' % (self.gpg_home, self.gpg_home, os.path.join(signing_key_dir, 'key.secret')), native_sysroot=get_bb_var("RECIPE_SYSROOT_NATIVE", "gnupg-native"), shell=True)
>>>> +        features += 'INHERIT += "sign_package_feed"\n'
>>>> +        features += 'PACKAGE_FEED_GPG_NAME = "testuser"\n'
>>>> +        features += 'PACKAGE_FEED_GPG_PASSPHRASE_FILE = "%s"\n' % os.path.join(signing_key_dir, 'key.passphrase')
>>>> +        features += 'GPG_PATH = "%s"\n' % self.gpg_home
>>>> +        features += 'PSEUDO_IGNORE_PATHS .= ",%s"\n' % self.gpg_home
>>>> +        self.write_config(features)
>>>> +
>>>> +        # Build core-image-sato and testimage
>>>> +        bitbake('core-image-full-cmdline socat')
>>>> +        bitbake('-c testimage core-image-full-cmdline')
>>>> +
>>>>        def test_testimage_virgl_gtk_sdl(self):
>>>>            """
>>>>            Summary: Check host-assisted accelerate OpenGL functionality in qemu with gtk and SDL frontends
>>> Thanks for working on this!
>>>
>>> Looking at the patches I wondered if this would break testimage and
>>> unfortunately it does:
>>>
>>> https://autobuilder.yoctoproject.org/typhoon/#/builders/50/builds/5013/steps/12/logs/stdio
>>> https://autobuilder.yoctoproject.org/typhoon/#/builders/76/builds/4975
>> That is weird, do I understand correctly that it fails on:
>>    apt-get remove --yes run-postinsts-dev
>> Reading package lists...
>> Building dependency tree...
>> E: Unable to locate package run-postinsts-dev
>>
>> That is actually *) one line I didn't touch. I did note while testing
>> that I saw this exact message, however that was not counted as a fail.
>>
>> What could cause this? Because the complaint is it can't remove the
>> package because it was not installed.
>>
>> It would be trivial to remove the line
>>
>> *) self.pkg('remove --yes run-postinsts-dev')
>>
>> but how could it have passed the test before?
>
> I think the issue is you edited testimage which is a different set of tests
> which aren't just called by oe-selftest but by things like

That would be my first thought too, but...

because the failure seems to be on the line self.pkg('remove --yes 
run-postinsts-dev'),  that would mean the line self.pkg('update') passed.

And that should only pass if it finds a signed repository and has the 
key installed (and believe me, I saw a log of that in the last week).

So, there may be a second thing wrong?

Do you know where I can find the log files referred to:

<..>tmp/work/qemux86-poky-linux/core-image-sato/1.0-r0/temp/log.do_testimage.35553

<..>tmp/work/qemux86-poky-linux/core-image-sato-sdk/1.0-r0/temp/log.do_testimage.35362

or could we do a 'quick' check by changing

         self.pkg('update')
         self.pkg('remove --yes run-postinsts-dev')
         self.pkg('install --yes run-postinsts-dev')
to
         self.pkg('update')
         self.pkg('install --yes run-postinsts-dev')
         self.pkg('remove --yes run-postinsts-dev')
?

>
> "bitbake core-image-sato -c testimage"
>
> as well. I'd suggest making the changes in testimage conditional on signing
> being configured.

Yes, regardless the above, we need to either make signing always enabled 
in all test cases or detect whether signing is used.

Do you have a hint if there is a variable to test in class AptRepoTest 
if PACKAGE_FEED_GPG_NAME has been set?

Otherwise I could just duplicate code and create 
apt.AptRepoTest.test_apt_install_from_repo_signed.

What would you prefer?

> Cheers,
>
> Richard
>
>

[-- Attachment #2: Type: text/html, Size: 10907 bytes --]

Re: [PATCH v2 3/3] apt: add apt selftest to test signed package feeds

[Richard Purdie]

On Tue, 2022-04-05 at 17:23 +0200, Ferry Toth wrote:
> Op 04-04-2022 om 22:39 schreef Richard Purdie:
>  On Mon, 2022-04-04 at 19:35 +0200, Ferry Toth wrote:
> >  Op 04-04-2022 om 15:58 schreef Richard Purdie:
> > > 
> > > > On Sun, 2022-04-03 at 21:50 +0200, Ferry Toth wrote:
> > > > Looking at the patches I wondered if this would break testimage and
> > > > unfortunately it does:
> > > > 
> > > > https://autobuilder.yoctoproject.org/typhoon/#/builders/50/builds/5013/s
> > > > teps/12/logs/stdio
> > > > https://autobuilder.yoctoproject.org/typhoon/#/builders/76/builds/4975
> > > That is weird, do I understand correctly that it fails on:
> > >   apt-get remove --yes run-postinsts-dev
> > > Reading package lists...
> > > Building dependency tree...
> > > E: Unable to locate package run-postinsts-dev
> > > 
> > > That is actually *) one line I didn't touch. I did note while testing 
> > > that I saw this exact message, however that was not counted as a fail.
> > > 
> > > What could cause this? Because the complaint is it can't remove the 
> > > package because it was not installed.
> > > 
> > > It would be trivial to remove the line
> > > 
> > > *) self.pkg('remove --yes run-postinsts-dev')
> > > 
> > > but how could it have passed the test before?
> > 
> > I think the issue is you edited testimage which is a different set of tests
> > which aren't just called by oe-selftest but by things like 
> That would be my first thought too, but...
> because the failure seems to be on the line self.pkg('remove --yes run-
> postinsts-dev'),  that would mean the line self.pkg('update') passed.
> And that should only pass if it finds a signed repository and has the key
> installed (and believe me, I saw a log of that in the last week).
> So, there may be a second thing wrong?

I was easily able to reproduce this locally and it shows the
setup_source_config_for_package_install() step fails and hence the sources
aren't setup correctly, hence the update probably works.

> Do you know where I can find the log files referred to:
> <..>tmp/work/qemux86-poky-linux/core-image-sato/1.0-
> r0/temp/log.do_testimage.35553
> <..>tmp/work/qemux86-poky-linux/core-image-sato-sdk/1.0-
> r0/temp/log.do_testimage.35362

We can get them off the autobuilder if needed but someone would have to manually
go in and find/share them. The issue does locally reproduce for me with a
"bitbake core-image-sato -c testimage" with package_deb set as the backend.

> or could we do a 'quick' check by changing
>         self.pkg('update')
>         self.pkg('remove --yes run-postinsts-dev')
>         self.pkg('install --yes run-postinsts-dev')
> to 
>         self.pkg('update')
>         self.pkg('install --yes run-postinsts-dev')
>         self.pkg('remove --yes run-postinsts-dev')
> ?

I'm not convinced that would help us...

>  
> > 
> > "bitbake core-image-sato -c testimage"
> > 
> > as well. I'd suggest making the changes in testimage conditional on signing
> > being configured.
> Yes, regardless the above, we need to either make signing always enabled in
> all test cases or detect whether signing is used.
> Do you have a hint if there is a variable to test in class AptRepoTest if
> PACKAGE_FEED_GPG_NAME has been set?
> Otherwise I could just duplicate code and create
> apt.AptRepoTest.test_apt_install_from_repo_signed.
> What would you prefer?
> 

We should be able to test self.tc.td.get('PACKAGE_FEED_GPG_NAME') in the test
and handle accordingly?

I did merge the base changes into the release since I thought it was fair to get
the fixes in before it was built. We just need to get the test sorted now, I
think it is close.

Cheers,

Richard

Re: [PATCH v2 3/3] apt: add apt selftest to test signed package feeds

[Ferry Toth]

[-- Attachment #1: Type: text/plain, Size: 4362 bytes --]

Hi,

Op 06-04-2022 om 13:40 schreef Richard Purdie:
> On Tue, 2022-04-05 at 17:23 +0200, Ferry Toth wrote:
>> Op 04-04-2022 om 22:39 schreef Richard Purdie:
>>   On Mon, 2022-04-04 at 19:35 +0200, Ferry Toth wrote:
>>>   Op 04-04-2022 om 15:58 schreef Richard Purdie:
>>>>> On Sun, 2022-04-03 at 21:50 +0200, Ferry Toth wrote:
>>>>> Looking at the patches I wondered if this would break testimage and
>>>>> unfortunately it does:
>>>>>
>>>>> https://autobuilder.yoctoproject.org/typhoon/#/builders/50/builds/5013/s
>>>>> teps/12/logs/stdio
>>>>> https://autobuilder.yoctoproject.org/typhoon/#/builders/76/builds/4975
>>>> That is weird, do I understand correctly that it fails on:
>>>>    apt-get remove --yes run-postinsts-dev
>>>> Reading package lists...
>>>> Building dependency tree...
>>>> E: Unable to locate package run-postinsts-dev
>>>>
>>>> That is actually *) one line I didn't touch. I did note while testing
>>>> that I saw this exact message, however that was not counted as a fail.
>>>>
>>>> What could cause this? Because the complaint is it can't remove the
>>>> package because it was not installed.
>>>>
>>>> It would be trivial to remove the line
>>>>
>>>> *) self.pkg('remove --yes run-postinsts-dev')
>>>>
>>>> but how could it have passed the test before?
>>> I think the issue is you edited testimage which is a different set of tests
>>> which aren't just called by oe-selftest but by things like
>> That would be my first thought too, but...
>> because the failure seems to be on the line self.pkg('remove --yes run-
>> postinsts-dev'),  that would mean the line self.pkg('update') passed.
>> And that should only pass if it finds a signed repository and has the key
>> installed (and believe me, I saw a log of that in the last week).
>> So, there may be a second thing wrong?
> I was easily able to reproduce this locally and it shows the
> setup_source_config_for_package_install() step fails and hence the sources
> aren't setup correctly, hence the update probably works.
not correct, hence works. You lost me here, but I'll try to reproduce.
>> Do you know where I can find the log files referred to:
>> <..>tmp/work/qemux86-poky-linux/core-image-sato/1.0-
>> r0/temp/log.do_testimage.35553
>> <..>tmp/work/qemux86-poky-linux/core-image-sato-sdk/1.0-
>> r0/temp/log.do_testimage.35362
> We can get them off the autobuilder if needed but someone would have to manually
No, that would be too much work. I'll try to reproduce myself.
> go in and find/share them. The issue does locally reproduce for me with a
> "bitbake core-image-sato -c testimage" with package_deb set as the backend.

..in conf. But without PACKAGE_CLASSES, PACKAGE_FEED_GPG_NAME, 
PACKAGE_FEED_GPG_PASSPHRASE_FILE?

>> or could we do a 'quick' check by changing
>>          self.pkg('update')
>>          self.pkg('remove --yes run-postinsts-dev')
>>          self.pkg('install --yes run-postinsts-dev')
>> to
>>          self.pkg('update')
>>          self.pkg('install --yes run-postinsts-dev')
>>          self.pkg('remove --yes run-postinsts-dev')
>> ?
> I'm not convinced that would help us...
I'll try locally.
>>   
>>> "bitbake core-image-sato -c testimage"
>>>
>>> as well. I'd suggest making the changes in testimage conditional on signing
>>> being configured.
>> Yes, regardless the above, we need to either make signing always enabled in
>> all test cases or detect whether signing is used.
>> Do you have a hint if there is a variable to test in class AptRepoTest if
>> PACKAGE_FEED_GPG_NAME has been set?
>> Otherwise I could just duplicate code and create
>> apt.AptRepoTest.test_apt_install_from_repo_signed.
>> What would you prefer?
>>
> We should be able to test self.tc.td.get('PACKAGE_FEED_GPG_NAME') in the test
> and handle accordingly?
>
> I did merge the base changes into the release since I thought it was fair to get
> the fixes in before it was built. We just need to get the test sorted now, I
> think it is close.

Thanks for merging.

I'll fix the test, that's only fair.

One thing, the test "test_testimage_apt" is new. It needs to be 
scheduled somewhere (where "test_testimage_dnf" is called i guess), I 
didn't add that. Is that correct?

>
> Cheers,
>
> Richard
>
>
>

[-- Attachment #2: Type: text/html, Size: 7024 bytes --]

Re: [PATCH v2 3/3] apt: add apt selftest to test signed package feeds

[Richard Purdie]

On Wed, 2022-04-06 at 16:43 +0200, Ferry Toth wrote:
> Op 06-04-2022 om 13:40 schreef Richard Purdie:
> > On Tue, 2022-04-05 at 17:23 +0200, Ferry Toth wrote:
> > > Op 04-04-2022 om 22:39 schreef Richard Purdie:
> > >  On Mon, 2022-04-04 at 19:35 +0200, Ferry Toth wrote:
> > > >  Op 04-04-2022 om 15:58 schreef Richard Purdie:
> > > > > > On Sun, 2022-04-03 at 21:50 +0200, Ferry Toth wrote:
> > > > > > Looking at the patches I wondered if this would break testimage and
> > > > > > unfortunately it does:
> > > > > > 
> > > > > > https://autobuilder.yoctoproject.org/typhoon/#/builders/50/builds/50
> > > > > > 13/s
> > > > > > teps/12/logs/stdio
> > > > > > https://autobuilder.yoctoproject.org/typhoon/#/builders/76/builds/49
> > > > > > 75
> > > > > That is weird, do I understand correctly that it fails on:
> > > > >   apt-get remove --yes run-postinsts-dev
> > > > > Reading package lists...
> > > > > Building dependency tree...
> > > > > E: Unable to locate package run-postinsts-dev
> > > > > 
> > > > > That is actually *) one line I didn't touch. I did note while testing 
> > > > > that I saw this exact message, however that was not counted as a fail.
> > > > > 
> > > > > What could cause this? Because the complaint is it can't remove the 
> > > > > package because it was not installed.
> > > > > 
> > > > > It would be trivial to remove the line
> > > > > 
> > > > > *) self.pkg('remove --yes run-postinsts-dev')
> > > > > 
> > > > > but how could it have passed the test before?
> > > > I think the issue is you edited testimage which is a different set of
> > > > tests
> > > > which aren't just called by oe-selftest but by things like 
> > > That would be my first thought too, but...
> > > because the failure seems to be on the line self.pkg('remove --yes run-
> > > postinsts-dev'),  that would mean the line self.pkg('update') passed.
> > > And that should only pass if it finds a signed repository and has the key
> > > installed (and believe me, I saw a log of that in the last week).
> > > So, there may be a second thing wrong?
> > I was easily able to reproduce this locally and it shows the
> > setup_source_config_for_package_install() step fails and hence the sources
> > aren't setup correctly, hence the update probably works.
>  not correct, hence works. You lost me here, but I'll try to reproduce.

I mean the command doesn't work correctly. In my local logs I see:

DEBUG: Command: cd /etc/apt/; cp sources.list sources.list.bak; sed -i 's/\[trusted=yes\] http:\/\/bogus_ip:bogus_port/http:\/\/192.168.7.1:46599/g' sources.list
Status: 1 Output:  cp: can't stat 'sources.list': No such file or directory
sed: sources.list: No such file or directory

> 
>  
> > go in and find/share them. The issue does locally reproduce for me with a
> > "bitbake core-image-sato -c testimage" with package_deb set as the backend.
> ..in conf. But without PACKAGE_CLASSES, PACKAGE_FEED_GPG_NAME,
> PACKAGE_FEED_GPG_PASSPHRASE_FILE?

Yes.

> > 
> > > Yes, regardless the above, we need to either make signing always enabled
> > > in
> > > all test cases or detect whether signing is used.
> > > Do you have a hint if there is a variable to test in class AptRepoTest if
> > > PACKAGE_FEED_GPG_NAME has been set?
> > > Otherwise I could just duplicate code and create
> > > apt.AptRepoTest.test_apt_install_from_repo_signed.
> > > What would you prefer?
> > > 
> > We should be able to test self.tc.td.get('PACKAGE_FEED_GPG_NAME') in the
> > test
> > and handle accordingly?
> > 
> > I did merge the base changes into the release since I thought it was fair to
> > get
> > the fixes in before it was built. We just need to get the test sorted now, I
> > think it is close.
> Thanks for merging.
> I'll fix the test, that's only fair. 
> One thing, the test "test_testimage_apt" is new. It needs to be scheduled
> somewhere (where "test_testimage_dnf" is called i guess), I didn't add that.
> Is that correct?
>  

No, the autobuilder runs all the tests in oe-selftest so it should be covered
(and is why we saw the failures on the autobuilder).

Cheers,

Richard

Re: [PATCH v2 3/3] apt: add apt selftest to test signed package feeds

[Ferry Toth]

Hi

Op 06-04-2022 om 17:23 schreef Richard Purdie:
> On Wed, 2022-04-06 at 16:43 +0200, Ferry Toth wrote:
>> Op 06-04-2022 om 13:40 schreef Richard Purdie:
>>> On Tue, 2022-04-05 at 17:23 +0200, Ferry Toth wrote:
>>>> Op 04-04-2022 om 22:39 schreef Richard Purdie:
>>>>   On Mon, 2022-04-04 at 19:35 +0200, Ferry Toth wrote:
>>>>>   Op 04-04-2022 om 15:58 schreef Richard Purdie:
>>>>>>> On Sun, 2022-04-03 at 21:50 +0200, Ferry Toth wrote:
>>>>>>> Looking at the patches I wondered if this would break testimage and
>>>>>>> unfortunately it does:
>>>>>>>
>>>>>>> https://autobuilder.yoctoproject.org/typhoon/#/builders/50/builds/50
>>>>>>> 13/s
>>>>>>> teps/12/logs/stdio
>>>>>>> https://autobuilder.yoctoproject.org/typhoon/#/builders/76/builds/49
>>>>>>> 75
>>>>>> That is weird, do I understand correctly that it fails on:
>>>>>>    apt-get remove --yes run-postinsts-dev
>>>>>> Reading package lists...
>>>>>> Building dependency tree...
>>>>>> E: Unable to locate package run-postinsts-dev
>>>>>>
>>>>>> That is actually *) one line I didn't touch. I did note while testing
>>>>>> that I saw this exact message, however that was not counted as a fail.
>>>>>>
>>>>>> What could cause this? Because the complaint is it can't remove the
>>>>>> package because it was not installed.
>>>>>>
>>>>>> It would be trivial to remove the line
>>>>>>
>>>>>> *) self.pkg('remove --yes run-postinsts-dev')
>>>>>>
>>>>>> but how could it have passed the test before?
>>>>> I think the issue is you edited testimage which is a different set of
>>>>> tests
>>>>> which aren't just called by oe-selftest but by things like
>>>> That would be my first thought too, but...
>>>> because the failure seems to be on the line self.pkg('remove --yes run-
>>>> postinsts-dev'),  that would mean the line self.pkg('update') passed.
>>>> And that should only pass if it finds a signed repository and has the key
>>>> installed (and believe me, I saw a log of that in the last week).
>>>> So, there may be a second thing wrong?
>>> I was easily able to reproduce this locally and it shows the
>>> setup_source_config_for_package_install() step fails and hence the sources
>>> aren't setup correctly, hence the update probably works.
>>   not correct, hence works. You lost me here, but I'll try to reproduce.
> I mean the command doesn't work correctly. In my local logs I see:
>
> DEBUG: Command: cd /etc/apt/; cp sources.list sources.list.bak; sed -i 's/\[trusted=yes\] http:\/\/bogus_ip:bogus_port/http:\/\/192.168.7.1:46599/g' sources.list
> Status: 1 Output:  cp: can't stat 'sources.list': No such file or directory
> sed: sources.list: No such file or directory
>
>>   
>>> go in and find/share them. The issue does locally reproduce for me with a
>>> "bitbake core-image-sato -c testimage" with package_deb set as the backend.
>> ..in conf. But without PACKAGE_CLASSES, PACKAGE_FEED_GPG_NAME,
>> PACKAGE_FEED_GPG_PASSPHRASE_FILE?
> Yes.
>
>>>> Yes, regardless the above, we need to either make signing always enabled
>>>> in
>>>> all test cases or detect whether signing is used.
>>>> Do you have a hint if there is a variable to test in class AptRepoTest if
>>>> PACKAGE_FEED_GPG_NAME has been set?
>>>> Otherwise I could just duplicate code and create
>>>> apt.AptRepoTest.test_apt_install_from_repo_signed.
>>>> What would you prefer?
>>>>
>>> We should be able to test self.tc.td.get('PACKAGE_FEED_GPG_NAME') in the
>>> test
>>> and handle accordingly?
>>>
>>> I did merge the base changes into the release since I thought it was fair to
>>> get
>>> the fixes in before it was built. We just need to get the test sorted now, I
>>> think it is close.
>> Thanks for merging.
>> I'll fix the test, that's only fair.
>> One thing, the test "test_testimage_apt" is new. It needs to be scheduled
>> somewhere (where "test_testimage_dnf" is called i guess), I didn't add that.
>> Is that correct?
>>   
> No, the autobuilder runs all the tests in oe-selftest so it should be covered
> (and is why we saw the failures on the autobuilder).

I was running 'oe-selftest -K -r 
runtime_test.TestImage.test_testimage_apt' whereas buildbot seems to be 
running 'apt.AptRepoTest.test_apt_install_from_repo' directly.

However, test_testimage_apt is where keys are setup (keys found in 
meta-selftest). So, where/when is test_testimage_apt called (or 
test_testimage_dnf for that matter)?

> Cheers,
>
> Richard
>

Re: [PATCH v2 3/3] apt: add apt selftest to test signed package feeds

[Richard Purdie]

On Wed, 2022-04-06 at 21:44 +0200, Ferry Toth wrote:
> Hi
> 
> Op 06-04-2022 om 17:23 schreef Richard Purdie:
> > On Wed, 2022-04-06 at 16:43 +0200, Ferry Toth wrote:
> > > Op 06-04-2022 om 13:40 schreef Richard Purdie:
> > > > On Tue, 2022-04-05 at 17:23 +0200, Ferry Toth wrote:
> > > > > Op 04-04-2022 om 22:39 schreef Richard Purdie:
> > > > >   On Mon, 2022-04-04 at 19:35 +0200, Ferry Toth wrote:
> > > > > >   Op 04-04-2022 om 15:58 schreef Richard Purdie:
> > > > > > > > On Sun, 2022-04-03 at 21:50 +0200, Ferry Toth wrote:
> > > > > > > > Looking at the patches I wondered if this would break testimage and
> > > > > > > > unfortunately it does:
> > > > > > > > 
> > > > > > > > https://autobuilder.yoctoproject.org/typhoon/#/builders/50/builds/50
> > > > > > > > 13/s
> > > > > > > > teps/12/logs/stdio
> > > > > > > > https://autobuilder.yoctoproject.org/typhoon/#/builders/76/builds/49
> > > > > > > > 75
> > > > > > > That is weird, do I understand correctly that it fails on:
> > > > > > >    apt-get remove --yes run-postinsts-dev
> > > > > > > Reading package lists...
> > > > > > > Building dependency tree...
> > > > > > > E: Unable to locate package run-postinsts-dev
> > > > > > > 
> > > > > > > That is actually *) one line I didn't touch. I did note while testing
> > > > > > > that I saw this exact message, however that was not counted as a fail.
> > > > > > > 
> > > > > > > What could cause this? Because the complaint is it can't remove the
> > > > > > > package because it was not installed.
> > > > > > > 
> > > > > > > It would be trivial to remove the line
> > > > > > > 
> > > > > > > *) self.pkg('remove --yes run-postinsts-dev')
> > > > > > > 
> > > > > > > but how could it have passed the test before?
> > > > > > I think the issue is you edited testimage which is a different set of
> > > > > > tests
> > > > > > which aren't just called by oe-selftest but by things like
> > > > > That would be my first thought too, but...
> > > > > because the failure seems to be on the line self.pkg('remove --yes run-
> > > > > postinsts-dev'),  that would mean the line self.pkg('update') passed.
> > > > > And that should only pass if it finds a signed repository and has the key
> > > > > installed (and believe me, I saw a log of that in the last week).
> > > > > So, there may be a second thing wrong?
> > > > I was easily able to reproduce this locally and it shows the
> > > > setup_source_config_for_package_install() step fails and hence the sources
> > > > aren't setup correctly, hence the update probably works.
> > >   not correct, hence works. You lost me here, but I'll try to reproduce.
> > I mean the command doesn't work correctly. In my local logs I see:
> > 
> > DEBUG: Command: cd /etc/apt/; cp sources.list sources.list.bak; sed -i 's/\[trusted=yes\] http:\/\/bogus_ip:bogus_port/http:\/\/192.168.7.1:46599/g' sources.list
> > Status: 1 Output:  cp: can't stat 'sources.list': No such file or directory
> > sed: sources.list: No such file or directory
> > 
> > >   
> > > > go in and find/share them. The issue does locally reproduce for me with a
> > > > "bitbake core-image-sato -c testimage" with package_deb set as the backend.
> > > ..in conf. But without PACKAGE_CLASSES, PACKAGE_FEED_GPG_NAME,
> > > PACKAGE_FEED_GPG_PASSPHRASE_FILE?
> > Yes.
> > 
> > > > > Yes, regardless the above, we need to either make signing always enabled
> > > > > in
> > > > > all test cases or detect whether signing is used.
> > > > > Do you have a hint if there is a variable to test in class AptRepoTest if
> > > > > PACKAGE_FEED_GPG_NAME has been set?
> > > > > Otherwise I could just duplicate code and create
> > > > > apt.AptRepoTest.test_apt_install_from_repo_signed.
> > > > > What would you prefer?
> > > > > 
> > > > We should be able to test self.tc.td.get('PACKAGE_FEED_GPG_NAME') in the
> > > > test
> > > > and handle accordingly?
> > > > 
> > > > I did merge the base changes into the release since I thought it was fair to
> > > > get
> > > > the fixes in before it was built. We just need to get the test sorted now, I
> > > > think it is close.
> > > Thanks for merging.
> > > I'll fix the test, that's only fair.
> > > One thing, the test "test_testimage_apt" is new. It needs to be scheduled
> > > somewhere (where "test_testimage_dnf" is called i guess), I didn't add that.
> > > Is that correct?
> > >   
> > No, the autobuilder runs all the tests in oe-selftest so it should be covered
> > (and is why we saw the failures on the autobuilder).
> 
> I was running 'oe-selftest -K -r 
> runtime_test.TestImage.test_testimage_apt' whereas buildbot seems to be 
> running 'apt.AptRepoTest.test_apt_install_from_repo' directly.

We have several types of test. There are two types in play here,

"oe-selftest -K -r runtime_test.TestImage.test_testimage_apt"

and

"bitbake core-image-sato -c testimage"

The latter testimage tests are often run every time we create images and running
testimage will trigger 'apt.AptRepoTest.test_apt_install_from_repo' if the image
has apt present and is built using debian package management.

We also run oe-selftest which triggers a testimage of it's own for a specfic
test case.

So we'd expect the normal testimage calls to not have the package signing
enabled and then we'd have the oe-selftest which specifically tests signing.

> However, test_testimage_apt is where keys are setup (keys found in 
> meta-selftest). So, where/when is test_testimage_apt called (or 
> test_testimage_dnf for that matter)?

Those are called by the oe-selftest call on the autobuilder. We run the oe-
selftest with a mask on the autobuilder so pretty much all of them run.

Cheers,

Richard

Re: [PATCH v2 3/3] apt: add apt selftest to test signed package feeds

[Ferry Toth]

Op 06-04-2022 om 23:05 schreef Richard Purdie:
> On Wed, 2022-04-06 at 21:44 +0200, Ferry Toth wrote:
>> Hi
>>
>> Op 06-04-2022 om 17:23 schreef Richard Purdie:
>>> On Wed, 2022-04-06 at 16:43 +0200, Ferry Toth wrote:
>>>> Op 06-04-2022 om 13:40 schreef Richard Purdie:
>>>>> On Tue, 2022-04-05 at 17:23 +0200, Ferry Toth wrote:
>>>>>> Op 04-04-2022 om 22:39 schreef Richard Purdie:
>>>>>>    On Mon, 2022-04-04 at 19:35 +0200, Ferry Toth wrote:
>>>>>>>    Op 04-04-2022 om 15:58 schreef Richard Purdie:
>>>>>>>>> On Sun, 2022-04-03 at 21:50 +0200, Ferry Toth wrote:
>>>>>>>>> Looking at the patches I wondered if this would break testimage and
>>>>>>>>> unfortunately it does:
>>>>>>>>>
>>>>>>>>> https://autobuilder.yoctoproject.org/typhoon/#/builders/50/builds/50
>>>>>>>>> 13/s
>>>>>>>>> teps/12/logs/stdio
>>>>>>>>> https://autobuilder.yoctoproject.org/typhoon/#/builders/76/builds/49
>>>>>>>>> 75
>>>>>>>> That is weird, do I understand correctly that it fails on:
>>>>>>>>     apt-get remove --yes run-postinsts-dev
>>>>>>>> Reading package lists...
>>>>>>>> Building dependency tree...
>>>>>>>> E: Unable to locate package run-postinsts-dev
>>>>>>>>
>>>>>>>> That is actually *) one line I didn't touch. I did note while testing
>>>>>>>> that I saw this exact message, however that was not counted as a fail.
>>>>>>>>
>>>>>>>> What could cause this? Because the complaint is it can't remove the
>>>>>>>> package because it was not installed.
>>>>>>>>
>>>>>>>> It would be trivial to remove the line
>>>>>>>>
>>>>>>>> *) self.pkg('remove --yes run-postinsts-dev')
>>>>>>>>
>>>>>>>> but how could it have passed the test before?
>>>>>>> I think the issue is you edited testimage which is a different set of
>>>>>>> tests
>>>>>>> which aren't just called by oe-selftest but by things like
>>>>>> That would be my first thought too, but...
>>>>>> because the failure seems to be on the line self.pkg('remove --yes run-
>>>>>> postinsts-dev'),  that would mean the line self.pkg('update') passed.
>>>>>> And that should only pass if it finds a signed repository and has the key
>>>>>> installed (and believe me, I saw a log of that in the last week).
>>>>>> So, there may be a second thing wrong?
>>>>> I was easily able to reproduce this locally and it shows the
>>>>> setup_source_config_for_package_install() step fails and hence the sources
>>>>> aren't setup correctly, hence the update probably works.
>>>>    not correct, hence works. You lost me here, but I'll try to reproduce.
>>> I mean the command doesn't work correctly. In my local logs I see:
>>>
>>> DEBUG: Command: cd /etc/apt/; cp sources.list sources.list.bak; sed -i 's/\[trusted=yes\] http:\/\/bogus_ip:bogus_port/http:\/\/192.168.7.1:46599/g' sources.list
>>> Status: 1 Output:  cp: can't stat 'sources.list': No such file or directory
>>> sed: sources.list: No such file or directory
>>>
>>>>    
>>>>> go in and find/share them. The issue does locally reproduce for me with a
>>>>> "bitbake core-image-sato -c testimage" with package_deb set as the backend.
>>>> ..in conf. But without PACKAGE_CLASSES, PACKAGE_FEED_GPG_NAME,
>>>> PACKAGE_FEED_GPG_PASSPHRASE_FILE?
>>> Yes.
>>>
>>>>>> Yes, regardless the above, we need to either make signing always enabled
>>>>>> in
>>>>>> all test cases or detect whether signing is used.
>>>>>> Do you have a hint if there is a variable to test in class AptRepoTest if
>>>>>> PACKAGE_FEED_GPG_NAME has been set?
>>>>>> Otherwise I could just duplicate code and create
>>>>>> apt.AptRepoTest.test_apt_install_from_repo_signed.
>>>>>> What would you prefer?
>>>>>>
>>>>> We should be able to test self.tc.td.get('PACKAGE_FEED_GPG_NAME') in the
>>>>> test
>>>>> and handle accordingly?
I'll try this for autodetecting the test case.
>>>>>
>>>>> I did merge the base changes into the release since I thought it was fair to
>>>>> get
>>>>> the fixes in before it was built. We just need to get the test sorted now, I
>>>>> think it is close.
>>>> Thanks for merging.
>>>> I'll fix the test, that's only fair.
>>>> One thing, the test "test_testimage_apt" is new. It needs to be scheduled
>>>> somewhere (where "test_testimage_dnf" is called i guess), I didn't add that.
>>>> Is that correct?
>>>>    
>>> No, the autobuilder runs all the tests in oe-selftest so it should be covered
>>> (and is why we saw the failures on the autobuilder).
>> I was running 'oe-selftest -K -r
>> runtime_test.TestImage.test_testimage_apt' whereas buildbot seems to be
>> running 'apt.AptRepoTest.test_apt_install_from_repo' directly.
> We have several types of test. There are two types in play here,
>
> "oe-selftest -K -r runtime_test.TestImage.test_testimage_apt"
>
> and
>
> "bitbake core-image-sato -c testimage"

Thanks for the explanation. Yesterday evening I built Sato and was able 
to reproduce  the issue.

(that was a big build with half of build time spent on building Rust).

core-image-sato does not have a /etc/apt/sources.list, the selftest public key is not on the image and gpg is not installed. This explains the errors in the log.

> The latter testimage tests are often run every time we create images and running
> testimage will trigger 'apt.AptRepoTest.test_apt_install_from_repo' if the image
> has apt present and is built using debian package management.
>
> We also run oe-selftest which triggers a testimage of it's own for a specfic
> test case.
Got it.
> So we'd expect the normal testimage calls to not have the package signing
> enabled and then we'd have the oe-selftest which specifically tests signing.
>
>> However, test_testimage_apt is where keys are setup (keys found in
>> meta-selftest). So, where/when is test_testimage_apt called (or
>> test_testimage_dnf for that matter)?
> Those are called by the oe-selftest call on the autobuilder. We run the oe-
> selftest with a mask on the autobuilder so pretty much all of them run.
After looking at more detail at autobuilder schedules I get it.
> Cheers,
>
> Richard
>
>

Re: [OE-core] [PATCH v2 3/3] apt: add apt selftest to test signed package feeds

[Alexandre Belloni]

Hello,

On 04/04/2022 14:58:07+0100, Richard Purdie wrote:
> On Sun, 2022-04-03 at 21:50 +0200, Ferry Toth wrote:
> > From: Ferry Toth <ftoth@exalondelft.nl>
> > 
> > Since Gatesgarth apt (1.8.2) has become more strict and doesn’t allow unsigned repositories by default.
> > Currently when building images this requirement is worked around by using [allow-insecure=yes] and
> > equivalently when performing selftest.
> > 
> > Patches "gpg-sign: Add parameters to gpg signature function" and "package_manager: sign DEB package feeds"
> > enable signed DEB package feeds. This patch adds a runtime test for apt derived from the test_testimage_dnf
> > test. It creates a signed deb package feed, runs a qemu image to install the key and performs some package
> > management. To be able to install the key the gnupg package is added to the testimage.
> > 
> > Signed-off-by: Ferry Toth <ftoth@exalondelft.nl>
> > ---
> >  meta/lib/oeqa/runtime/cases/apt.py           | 16 ++++++---
> >  meta/lib/oeqa/selftest/cases/runtime_test.py | 38 ++++++++++++++++++++
> >  2 files changed, 49 insertions(+), 5 deletions(-)
> > 
> > diff --git a/meta/lib/oeqa/runtime/cases/apt.py b/meta/lib/oeqa/runtime/cases/apt.py
> > index 53745df93f..49f8714730 100644
> > --- a/meta/lib/oeqa/runtime/cases/apt.py
> > +++ b/meta/lib/oeqa/runtime/cases/apt.py
> > @@ -21,7 +21,7 @@ class AptRepoTest(AptTest):
> >  
> >      @classmethod
> >      def setUpClass(cls):
> > -        service_repo = os.path.join(cls.tc.td['DEPLOY_DIR_DEB'], 'all')
> > +        service_repo = os.path.join(cls.tc.td['DEPLOY_DIR_DEB'], '')
> >          cls.repo_server = HTTPService(service_repo,
> >                                        '0.0.0.0', port=cls.tc.target.server_port,
> >                                        logger=cls.tc.logger)
> > @@ -32,13 +32,18 @@ class AptRepoTest(AptTest):
> >          cls.repo_server.stop()
> >  
> >      def setup_source_config_for_package_install(self):
> > -        apt_get_source_server = 'http://%s:%s/' % (self.tc.target.server_ip, self.repo_server.port)
> > +        apt_get_source_server = 'http:\/\/%s:%s' % (self.tc.target.server_ip, self.repo_server.port)
> >          apt_get_sourceslist_dir = '/etc/apt/'
> > -        self.target.run('cd %s; echo deb [ allow-insecure=yes ] %s ./ > sources.list' % (apt_get_sourceslist_dir, apt_get_source_server))
> > +        self.target.run("cd %s; cp sources.list sources.list.bak; sed -i 's/\[trusted=yes\] http:\/\/bogus_ip:bogus_port/%s/g' sources.list" % (apt_get_sourceslist_dir, apt_get_source_server))
> >  
> >      def cleanup_source_config_for_package_install(self):
> >          apt_get_sourceslist_dir = '/etc/apt/'
> > -        self.target.run('cd %s; rm sources.list' % (apt_get_sourceslist_dir))
> > +        self.target.run('cd %s; mv sources.list.bak sources.list' % (apt_get_sourceslist_dir))
> > +
> > +    def setup_key(self):
> > +        # the key is found on the target /etc/pki/packagefeed-gpg/
> > +        # named PACKAGEFEED-GPG-KEY-poky-branch
> > +        self.target.run('cd %s; apt-key add P*' % ('/etc/pki/packagefeed-gpg'))
> >  
> >      @skipIfNotFeature('package-management',
> >                        'Test requires package-management to be in IMAGE_FEATURES')
> > @@ -47,7 +52,8 @@ class AptRepoTest(AptTest):
> >      @OEHasPackage(['apt'])
> >      def test_apt_install_from_repo(self):
> >          self.setup_source_config_for_package_install()
> > +        self.setup_key()
> >          self.pkg('update')
> >          self.pkg('remove --yes run-postinsts-dev')
> > -        self.pkg('install --yes --allow-unauthenticated run-postinsts-dev')
> > +        self.pkg('install --yes run-postinsts-dev')
> >          self.cleanup_source_config_for_package_install()
> > diff --git a/meta/lib/oeqa/selftest/cases/runtime_test.py b/meta/lib/oeqa/selftest/cases/runtime_test.py
> > index 642f0eb637..7a75b95a99 100644
> > --- a/meta/lib/oeqa/selftest/cases/runtime_test.py
> > +++ b/meta/lib/oeqa/selftest/cases/runtime_test.py
> > @@ -162,6 +162,44 @@ class TestImage(OESelftestTestCase):
> >          bitbake('core-image-full-cmdline socat')
> >          bitbake('-c testimage core-image-full-cmdline')
> >  
> > +    def test_testimage_apt(self):
> > +        """
> > +        Summary: Check package feeds functionality for apt
> > +        Expected: 1. Check that remote package feeds can be accessed
> > +        Product: oe-core
> > +        Author: Ferry Toth <fntoth@gmail.com>
> > +        """
> > +        if get_bb_var('DISTRO') == 'poky-tiny':
> > +            self.skipTest('core-image-full-cmdline not buildable for poky-tiny')
> > +
> > +        features = 'INHERIT += "testimage"\n'
> > +        features += 'TEST_SUITES = "ping ssh apt.AptRepoTest.test_apt_install_from_repo"\n'
> > +        # We don't yet know what the server ip and port will be - they will be patched
> > +        # in at the start of the on-image test
> > +        features += 'PACKAGE_FEED_URIS = "http://bogus_ip:bogus_port"\n'
> > +        features += 'EXTRA_IMAGE_FEATURES += "package-management"\n'
> > +        features += 'PACKAGE_CLASSES = "package_deb"\n'
> > +        # We need  gnupg on the target to install keys
> > +        features += 'IMAGE_INSTALL:append:pn-core-image-full-cmdline = " gnupg"\n'
> > +
> > +        bitbake('gnupg-native -c addto_recipe_sysroot')
> > +
> > +        # Enable package feed signing
> > +        self.gpg_home = tempfile.mkdtemp(prefix="oeqa-feed-sign-")
> > +        self.track_for_cleanup(self.gpg_home)
> > +        signing_key_dir = os.path.join(self.testlayer_path, 'files', 'signing')
> > +        runCmd('gpgconf --list-dirs --homedir %s; gpg -v --batch --homedir %s --import %s' % (self.gpg_home, self.gpg_home, os.path.join(signing_key_dir, 'key.secret')), native_sysroot=get_bb_var("RECIPE_SYSROOT_NATIVE", "gnupg-native"), shell=True)
> > +        features += 'INHERIT += "sign_package_feed"\n'
> > +        features += 'PACKAGE_FEED_GPG_NAME = "testuser"\n'
> > +        features += 'PACKAGE_FEED_GPG_PASSPHRASE_FILE = "%s"\n' % os.path.join(signing_key_dir, 'key.passphrase')
> > +        features += 'GPG_PATH = "%s"\n' % self.gpg_home
> > +        features += 'PSEUDO_IGNORE_PATHS .= ",%s"\n' % self.gpg_home
> > +        self.write_config(features)
> > +
> > +        # Build core-image-sato and testimage
> > +        bitbake('core-image-full-cmdline socat')
> > +        bitbake('-c testimage core-image-full-cmdline')
> > +
> >      def test_testimage_virgl_gtk_sdl(self):
> >          """
> >          Summary: Check host-assisted accelerate OpenGL functionality in qemu with gtk and SDL frontends
> 
> Thanks for working on this!
> 
> Looking at the patches I wondered if this would break testimage and
> unfortunately it does:
> 
> https://autobuilder.yoctoproject.org/typhoon/#/builders/50/builds/5013/steps/12/logs/stdio
> https://autobuilder.yoctoproject.org/typhoon/#/builders/76/builds/4975
> 
> however hopefully these shouldn't be too hard to fix?
> 
> The rest of the build is still running.

I missed it at the time but I believe this is also the cause of:

https://autobuilder.yoctoproject.org/typhoon/#/builders/80/builds/3352/steps/15/logs/stdio

ERROR: package-index-1.0-r0 do_package_index: GPG exited with code 2: gpg: can't connect to the agent: IPC connect call failed
gpg: skipped "testuser": No secret key
gpg: signing failed: No secret key



-- 
Alexandre Belloni, co-owner and COO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

Re: [OE-core] [PATCH v2 3/3] apt: add apt selftest to test signed package feeds

[Ferry Toth]

[-- Attachment #1: Type: text/plain, Size: 7885 bytes --]

Hi

Op 06-04-2022 om 12:10 schreef Alexandre Belloni:
> Hello,
>
> On 04/04/2022 14:58:07+0100, Richard Purdie wrote:
>> On Sun, 2022-04-03 at 21:50 +0200, Ferry Toth wrote:
>>> From: Ferry Toth<ftoth@exalondelft.nl>
>>>
>>> Since Gatesgarth apt (1.8.2) has become more strict and doesn’t allow unsigned repositories by default.
>>> Currently when building images this requirement is worked around by using [allow-insecure=yes] and
>>> equivalently when performing selftest.
>>>
>>> Patches "gpg-sign: Add parameters to gpg signature function" and "package_manager: sign DEB package feeds"
>>> enable signed DEB package feeds. This patch adds a runtime test for apt derived from the test_testimage_dnf
>>> test. It creates a signed deb package feed, runs a qemu image to install the key and performs some package
>>> management. To be able to install the key the gnupg package is added to the testimage.
>>>
>>> Signed-off-by: Ferry Toth<ftoth@exalondelft.nl>
>>> ---
>>>   meta/lib/oeqa/runtime/cases/apt.py           | 16 ++++++---
>>>   meta/lib/oeqa/selftest/cases/runtime_test.py | 38 ++++++++++++++++++++
>>>   2 files changed, 49 insertions(+), 5 deletions(-)
>>>
>>> diff --git a/meta/lib/oeqa/runtime/cases/apt.py b/meta/lib/oeqa/runtime/cases/apt.py
>>> index 53745df93f..49f8714730 100644
>>> --- a/meta/lib/oeqa/runtime/cases/apt.py
>>> +++ b/meta/lib/oeqa/runtime/cases/apt.py
>>> @@ -21,7 +21,7 @@ class AptRepoTest(AptTest):
>>>   
>>>       @classmethod
>>>       def setUpClass(cls):
>>> -        service_repo = os.path.join(cls.tc.td['DEPLOY_DIR_DEB'], 'all')
>>> +        service_repo = os.path.join(cls.tc.td['DEPLOY_DIR_DEB'], '')
>>>           cls.repo_server = HTTPService(service_repo,
>>>                                         '0.0.0.0', port=cls.tc.target.server_port,
>>>                                         logger=cls.tc.logger)
>>> @@ -32,13 +32,18 @@ class AptRepoTest(AptTest):
>>>           cls.repo_server.stop()
>>>   
>>>       def setup_source_config_for_package_install(self):
>>> -        apt_get_source_server = 'http://%s:%s/' % (self.tc.target.server_ip, self.repo_server.port)
>>> +        apt_get_source_server = 'http:\/\/%s:%s' % (self.tc.target.server_ip, self.repo_server.port)
>>>           apt_get_sourceslist_dir = '/etc/apt/'
>>> -        self.target.run('cd %s; echo deb [ allow-insecure=yes ] %s ./ > sources.list' % (apt_get_sourceslist_dir, apt_get_source_server))
>>> +        self.target.run("cd %s; cp sources.list sources.list.bak; sed -i 's/\[trusted=yes\] http:\/\/bogus_ip:bogus_port/%s/g' sources.list" % (apt_get_sourceslist_dir, apt_get_source_server))
>>>   
>>>       def cleanup_source_config_for_package_install(self):
>>>           apt_get_sourceslist_dir = '/etc/apt/'
>>> -        self.target.run('cd %s; rm sources.list' % (apt_get_sourceslist_dir))
>>> +        self.target.run('cd %s; mv sources.list.bak sources.list' % (apt_get_sourceslist_dir))
>>> +
>>> +    def setup_key(self):
>>> +        # the key is found on the target /etc/pki/packagefeed-gpg/
>>> +        # named PACKAGEFEED-GPG-KEY-poky-branch
>>> +        self.target.run('cd %s; apt-key add P*' % ('/etc/pki/packagefeed-gpg'))
>>>   
>>>       @skipIfNotFeature('package-management',
>>>                         'Test requires package-management to be in IMAGE_FEATURES')
>>> @@ -47,7 +52,8 @@ class AptRepoTest(AptTest):
>>>       @OEHasPackage(['apt'])
>>>       def test_apt_install_from_repo(self):
>>>           self.setup_source_config_for_package_install()
>>> +        self.setup_key()
>>>           self.pkg('update')
>>>           self.pkg('remove --yes run-postinsts-dev')
>>> -        self.pkg('install --yes --allow-unauthenticated run-postinsts-dev')
>>> +        self.pkg('install --yes run-postinsts-dev')
>>>           self.cleanup_source_config_for_package_install()
>>> diff --git a/meta/lib/oeqa/selftest/cases/runtime_test.py b/meta/lib/oeqa/selftest/cases/runtime_test.py
>>> index 642f0eb637..7a75b95a99 100644
>>> --- a/meta/lib/oeqa/selftest/cases/runtime_test.py
>>> +++ b/meta/lib/oeqa/selftest/cases/runtime_test.py
>>> @@ -162,6 +162,44 @@ class TestImage(OESelftestTestCase):
>>>           bitbake('core-image-full-cmdline socat')
>>>           bitbake('-c testimage core-image-full-cmdline')
>>>   
>>> +    def test_testimage_apt(self):
>>> +        """
>>> +        Summary: Check package feeds functionality for apt
>>> +        Expected: 1. Check that remote package feeds can be accessed
>>> +        Product: oe-core
>>> +        Author: Ferry Toth<fntoth@gmail.com>
>>> +        """
>>> +        if get_bb_var('DISTRO') == 'poky-tiny':
>>> +            self.skipTest('core-image-full-cmdline not buildable for poky-tiny')
>>> +
>>> +        features = 'INHERIT += "testimage"\n'
>>> +        features += 'TEST_SUITES = "ping ssh apt.AptRepoTest.test_apt_install_from_repo"\n'
>>> +        # We don't yet know what the server ip and port will be - they will be patched
>>> +        # in at the start of the on-image test
>>> +        features += 'PACKAGE_FEED_URIS = "http://bogus_ip:bogus_port"\n'
>>> +        features += 'EXTRA_IMAGE_FEATURES += "package-management"\n'
>>> +        features += 'PACKAGE_CLASSES = "package_deb"\n'
>>> +        # We need  gnupg on the target to install keys
>>> +        features += 'IMAGE_INSTALL:append:pn-core-image-full-cmdline = " gnupg"\n'
>>> +
>>> +        bitbake('gnupg-native -c addto_recipe_sysroot')
>>> +
>>> +        # Enable package feed signing
>>> +        self.gpg_home = tempfile.mkdtemp(prefix="oeqa-feed-sign-")
>>> +        self.track_for_cleanup(self.gpg_home)
>>> +        signing_key_dir = os.path.join(self.testlayer_path, 'files', 'signing')
>>> +        runCmd('gpgconf --list-dirs --homedir %s; gpg -v --batch --homedir %s --import %s' % (self.gpg_home, self.gpg_home, os.path.join(signing_key_dir, 'key.secret')), native_sysroot=get_bb_var("RECIPE_SYSROOT_NATIVE", "gnupg-native"), shell=True)
>>> +        features += 'INHERIT += "sign_package_feed"\n'
>>> +        features += 'PACKAGE_FEED_GPG_NAME = "testuser"\n'
>>> +        features += 'PACKAGE_FEED_GPG_PASSPHRASE_FILE = "%s"\n' % os.path.join(signing_key_dir, 'key.passphrase')
>>> +        features += 'GPG_PATH = "%s"\n' % self.gpg_home
>>> +        features += 'PSEUDO_IGNORE_PATHS .= ",%s"\n' % self.gpg_home
>>> +        self.write_config(features)
>>> +
>>> +        # Build core-image-sato and testimage
>>> +        bitbake('core-image-full-cmdline socat')
>>> +        bitbake('-c testimage core-image-full-cmdline')
>>> +
>>>       def test_testimage_virgl_gtk_sdl(self):
>>>           """
>>>           Summary: Check host-assisted accelerate OpenGL functionality in qemu with gtk and SDL frontends
>> Thanks for working on this!
>>
>> Looking at the patches I wondered if this would break testimage and
>> unfortunately it does:
>>
>> https://autobuilder.yoctoproject.org/typhoon/#/builders/50/builds/5013/steps/12/logs/stdio
>> https://autobuilder.yoctoproject.org/typhoon/#/builders/76/builds/4975
>>
>> however hopefully these shouldn't be too hard to fix?
>>
>> The rest of the build is still running.
> I missed it at the time but I believe this is also the cause of:
>
> https://autobuilder.yoctoproject.org/typhoon/#/builders/80/builds/3352/steps/15/logs/stdio
>
> ERROR: package-index-1.0-r0 do_package_index: GPG exited with code 2: gpg: can't connect to the agent: IPC connect call failed
> gpg: skipped "testuser": No secret key
> gpg: signing failed: No secret key

This seems related but not exact the same.

It seems do_package_index wants to generate a signed deb repo but no key 
is provided. But IIUC you have PACKAGE_CLASSES = "package_rpm", so why 
is runtime_test.TestImage.test_testimage_apt run?

[-- Attachment #2: Type: text/html, Size: 8783 bytes --]