[PULL][PATCH 0/2] Implement debian packages GPG signing

[PULL][PATCH 0/2] Implement debian packages GPG signing

[Xavier Berger]

The following patches add signing feature for debian packages.


The following changes since commit 3bc700a12d4f21a5c7a1575da0972f9f78b4a091:

  mesa: update 19.0.0 -> 19.0.1 (2019-03-29 08:28:53 +0000)

are available in the git repository at:

  git://push.yoctoproject.org/poky-contrib xberger/deb_sign

Xavier Berger (2):
  gpg-sign: Add parameters to gpg signature function
  package_manager: sign DEB package feeds

 meta/lib/oe/gpg_sign.py        |  6 +++++-
 meta/lib/oe/package_manager.py | 19 ++++++++++++++++---
 2 files changed, 21 insertions(+), 4 deletions(-)

-- 
2.7.4

[PATCH 1/2] gpg-sign: Add parameters to gpg signature function

[Xavier Berger]

output_suffix: If defined, add output_suffix as file name extension.
use_sha256: If True, use sha256 for gpg as digest algorithm

Signed-off-by: Xavier Berger <xavier.berger@bio-logic.net>
---
 meta/lib/oe/gpg_sign.py | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/meta/lib/oe/gpg_sign.py b/meta/lib/oe/gpg_sign.py
index ccd5aee..212ac3a 100644
--- a/meta/lib/oe/gpg_sign.py
+++ b/meta/lib/oe/gpg_sign.py
@@ -49,7 +49,7 @@ class LocalSigner(object):
         for i in range(0, len(files), sign_chunk):
             subprocess.check_output(shlex.split(cmd + ' '.join(files[i:i+sign_chunk])), stderr=subprocess.STDOUT)
 
-    def detach_sign(self, input_file, keyid, passphrase_file, passphrase=None, armor=True):
+    def detach_sign(self, input_file, keyid, passphrase_file, passphrase=None, armor=True, output_suffix=None, use_sha256=False):
         """Create a detached signature of a file"""
 
         if passphrase_file and passphrase:
@@ -62,6 +62,10 @@ class LocalSigner(object):
             cmd += ['--homedir', self.gpg_path]
         if armor:
             cmd += ['--armor']
+        if output_suffix:
+            cmd += ['-o', input_file + "." + output_suffix]
+        if use_sha256:
+            cmd += ['--digest-algo', "SHA256"]
 
         #gpg > 2.1 supports password pipes only through the loopback interface
         #gpg < 2.1 errors out if given unknown parameters
-- 
2.7.4

[PATCH 2/2] package_manager: sign DEB package feeds

[Xavier Berger]

Implement debian package repository signature.
For each Release file created in repository subdirectory, a signature
Release.gpg is created.

Signature is performed using gpg backend when the following variables
are set in local.conf:
PACKAGE_CLASSES += "sign_package_feed"
PACKAGE_FEED_GPG_NAME = "<Id of GPG key>"
PACKAGE_FEED_GPG_PASSPHRASE_FILE="<path to password file>"

Signed-off-by: Xavier Berger <xavier.berger@bio-logic.net>
---
 meta/lib/oe/package_manager.py | 19 ++++++++++++++++---
 1 file changed, 16 insertions(+), 3 deletions(-)

diff --git a/meta/lib/oe/package_manager.py b/meta/lib/oe/package_manager.py
index 2835c1d..215d143 100644
--- a/meta/lib/oe/package_manager.py
+++ b/meta/lib/oe/package_manager.py
@@ -281,6 +281,7 @@ class DpkgIndexer(Indexer):
 
         index_cmds = []
         deb_dirs_found = False
+        index_sign_files = set()
         for arch in arch_list:
             arch_dir = os.path.join(self.deploy_dir, arch)
             if not os.path.isdir(arch_dir):
@@ -290,7 +291,10 @@ class DpkgIndexer(Indexer):
 
             cmd += "%s -fcn Packages > Packages.gz;" % gzip
 
-            with open(os.path.join(arch_dir, "Release"), "w+") as release:
+            release_file = os.path.join(arch_dir, "Release")
+            index_sign_files.add(release_file)
+
+            with open(release_file, "w+") as release:
                 release.write("Label: %s\n" % arch)
 
             cmd += "PSEUDO_UNLOAD=1 %s release . >> Release" % apt_ftparchive
@@ -304,8 +308,17 @@ class DpkgIndexer(Indexer):
             return
 
         oe.utils.multiprocess_launch(create_index, index_cmds, self.d)
-        if self.d.getVar('PACKAGE_FEED_SIGN') == '1':
-            raise NotImplementedError('Package feed signing not implementd for dpkg')
+        if self.d.getVar('PACKAGE_FEED_SIGN', True) == '1':
+            signer = get_signer(self.d, self.d.getVar('PACKAGE_FEED_GPG_BACKEND', True))
+        else:
+            signer = None
+        if signer:
+            for f in index_sign_files:
+                signer.detach_sign(f,
+                                   self.d.getVar('PACKAGE_FEED_GPG_NAME', True),
+                                   self.d.getVar('PACKAGE_FEED_GPG_PASSPHRASE_FILE', True),
+                                   output_suffix="gpg",
+                                   use_sha256=True)
 
 
 
-- 
2.7.4

Re: [PULL][PATCH 0/2] Implement debian packages GPG signing

[Alexander Kanavin]

Thanks, please send these patches to openembedded-core mailing list.

Some kind of test case similar to the rpm feed sign test in
meta/lib/oeqa/selftest/cases/runtime_test.py would be good to have.

Alex


On Mon, 1 Apr 2019 at 10:45, Xavier Berger <xavier.berger@bio-logic.net> wrote:
>
>
> The following patches add signing feature for debian packages.
>
>
> The following changes since commit 3bc700a12d4f21a5c7a1575da0972f9f78b4a091:
>
>   mesa: update 19.0.0 -> 19.0.1 (2019-03-29 08:28:53 +0000)
>
> are available in the git repository at:
>
>   git://push.yoctoproject.org/poky-contrib xberger/deb_sign
>
> Xavier Berger (2):
>   gpg-sign: Add parameters to gpg signature function
>   package_manager: sign DEB package feeds
>
>  meta/lib/oe/gpg_sign.py        |  6 +++++-
>  meta/lib/oe/package_manager.py | 19 ++++++++++++++++---
>  2 files changed, 21 insertions(+), 4 deletions(-)
>
> --
> 2.7.4
>
> --
> _______________________________________________
> poky mailing list
> poky@yoctoproject.org
> https://lists.yoctoproject.org/listinfo/poky

[PULL][PATCH 0/2] Implement debian packages GPG signing

[Xavier Berger]

The following patches add signing feature for debian packages.


The following changes since commit 3bc700a12d4f21a5c7a1575da0972f9f78b4a091:

  mesa: update 19.0.0 -> 19.0.1 (2019-03-29 08:28:53 +0000)

are available in the git repository at:

  git://push.yoctoproject.org/poky-contrib xberger/deb_sign

Xavier Berger (2):
  gpg-sign: Add parameters to gpg signature function
  package_manager: sign DEB package feeds

 meta/lib/oe/gpg_sign.py        |  6 +++++-
 meta/lib/oe/package_manager.py | 19 ++++++++++++++++---
 2 files changed, 21 insertions(+), 4 deletions(-)

-- 
2.7.4

[PATCH 1/2] gpg-sign: Add parameters to gpg signature function

[Xavier Berger]

output_suffix: If defined, add output_suffix as file name extension.
use_sha256: If True, use sha256 for gpg as digest algorithm

Signed-off-by: Xavier Berger <xavier.berger@bio-logic.net>
---
 meta/lib/oe/gpg_sign.py | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/meta/lib/oe/gpg_sign.py b/meta/lib/oe/gpg_sign.py
index ccd5aee..212ac3a 100644
--- a/meta/lib/oe/gpg_sign.py
+++ b/meta/lib/oe/gpg_sign.py
@@ -49,7 +49,7 @@ class LocalSigner(object):
         for i in range(0, len(files), sign_chunk):
             subprocess.check_output(shlex.split(cmd + ' '.join(files[i:i+sign_chunk])), stderr=subprocess.STDOUT)
 
-    def detach_sign(self, input_file, keyid, passphrase_file, passphrase=None, armor=True):
+    def detach_sign(self, input_file, keyid, passphrase_file, passphrase=None, armor=True, output_suffix=None, use_sha256=False):
         """Create a detached signature of a file"""
 
         if passphrase_file and passphrase:
@@ -62,6 +62,10 @@ class LocalSigner(object):
             cmd += ['--homedir', self.gpg_path]
         if armor:
             cmd += ['--armor']
+        if output_suffix:
+            cmd += ['-o', input_file + "." + output_suffix]
+        if use_sha256:
+            cmd += ['--digest-algo', "SHA256"]
 
         #gpg > 2.1 supports password pipes only through the loopback interface
         #gpg < 2.1 errors out if given unknown parameters
-- 
2.7.4

[PATCH 2/2] package_manager: sign DEB package feeds

[Xavier Berger]

Implement debian package repository signature.
For each Release file created in repository subdirectory, a signature
Release.gpg is created.

Signature is performed using gpg backend when the following variables
are set in local.conf:
PACKAGE_CLASSES += "sign_package_feed"
PACKAGE_FEED_GPG_NAME = "<Id of GPG key>"
PACKAGE_FEED_GPG_PASSPHRASE_FILE="<path to password file>"

Signed-off-by: Xavier Berger <xavier.berger@bio-logic.net>
---
 meta/lib/oe/package_manager.py | 19 ++++++++++++++++---
 1 file changed, 16 insertions(+), 3 deletions(-)

diff --git a/meta/lib/oe/package_manager.py b/meta/lib/oe/package_manager.py
index 2835c1d..215d143 100644
--- a/meta/lib/oe/package_manager.py
+++ b/meta/lib/oe/package_manager.py
@@ -281,6 +281,7 @@ class DpkgIndexer(Indexer):
 
         index_cmds = []
         deb_dirs_found = False
+        index_sign_files = set()
         for arch in arch_list:
             arch_dir = os.path.join(self.deploy_dir, arch)
             if not os.path.isdir(arch_dir):
@@ -290,7 +291,10 @@ class DpkgIndexer(Indexer):
 
             cmd += "%s -fcn Packages > Packages.gz;" % gzip
 
-            with open(os.path.join(arch_dir, "Release"), "w+") as release:
+            release_file = os.path.join(arch_dir, "Release")
+            index_sign_files.add(release_file)
+
+            with open(release_file, "w+") as release:
                 release.write("Label: %s\n" % arch)
 
             cmd += "PSEUDO_UNLOAD=1 %s release . >> Release" % apt_ftparchive
@@ -304,8 +308,17 @@ class DpkgIndexer(Indexer):
             return
 
         oe.utils.multiprocess_launch(create_index, index_cmds, self.d)
-        if self.d.getVar('PACKAGE_FEED_SIGN') == '1':
-            raise NotImplementedError('Package feed signing not implementd for dpkg')
+        if self.d.getVar('PACKAGE_FEED_SIGN', True) == '1':
+            signer = get_signer(self.d, self.d.getVar('PACKAGE_FEED_GPG_BACKEND', True))
+        else:
+            signer = None
+        if signer:
+            for f in index_sign_files:
+                signer.detach_sign(f,
+                                   self.d.getVar('PACKAGE_FEED_GPG_NAME', True),
+                                   self.d.getVar('PACKAGE_FEED_GPG_PASSPHRASE_FILE', True),
+                                   output_suffix="gpg",
+                                   use_sha256=True)
 
 
 
-- 
2.7.4

Re: [PULL][PATCH 0/2] Implement debian packages GPG signing

[Alexander Kanavin]

On Mon, 1 Apr 2019 at 15:57, Xavier Berger <xavier.berger@bio-logic.net> wrote:
> Xavier Berger (2):
>   gpg-sign: Add parameters to gpg signature function
>   package_manager: sign DEB package feeds
>
>  meta/lib/oe/gpg_sign.py        |  6 +++++-
>  meta/lib/oe/package_manager.py | 19 ++++++++++++++++---

How do we make sure this does not regress? Can you add a test case
similar to one for rpm feed signing?

Alex

Re: [PULL][PATCH 0/2] Implement debian packages GPG signing

[Xavier Berger]

Hi Alexender, 

I'm working on test case but I've a lot of things to understand (and setup to perform).
I'm basically taking dnf as example and creating a test for apt. Please let me know if you think my strategy is not correct.

I'll let you know here when it will work.

Xavier

-----Original Message-----
From: Alexander Kanavin <alex.kanavin@gmail.com> 
Sent: lundi 1 avril 2019 16:50
To: Xavier Berger <xavier.berger@bio-logic.net>
Cc: OE-core <openembedded-core@lists.openembedded.org>
Subject: Re: [OE-core] [PULL][PATCH 0/2] Implement debian packages GPG signing

On Mon, 1 Apr 2019 at 15:57, Xavier Berger <xavier.berger@bio-logic.net> wrote:
> Xavier Berger (2):
>   gpg-sign: Add parameters to gpg signature function
>   package_manager: sign DEB package feeds
>
>  meta/lib/oe/gpg_sign.py        |  6 +++++-
>  meta/lib/oe/package_manager.py | 19 ++++++++++++++++---

How do we make sure this does not regress? Can you add a test case similar to one for rpm feed signing?

Alex

Re: [PULL][PATCH 0/2] Implement debian packages GPG signing

[Alexander Kanavin]

Yes, that is fine. Let me know if something is not clear in that test case.

Btw, the apt version we have is badly out of date (and has a significant security issue I think) so if you can update that, that’d be awesome.

Alex

> On 2 Apr 2019, at 9.36, Xavier Berger <xavier.berger@bio-logic.net> wrote:
> 
> Hi Alexender, 
> 
> I'm working on test case but I've a lot of things to understand (and setup to perform).
> I'm basically taking dnf as example and creating a test for apt. Please let me know if you think my strategy is not correct.
> 
> I'll let you know here when it will work.
> 
> Xavier
> 
> -----Original Message-----
> From: Alexander Kanavin <alex.kanavin@gmail.com> 
> Sent: lundi 1 avril 2019 16:50
> To: Xavier Berger <xavier.berger@bio-logic.net>
> Cc: OE-core <openembedded-core@lists.openembedded.org>
> Subject: Re: [OE-core] [PULL][PATCH 0/2] Implement debian packages GPG signing
> 
>> On Mon, 1 Apr 2019 at 15:57, Xavier Berger <xavier.berger@bio-logic.net> wrote:
>> Xavier Berger (2):
>>  gpg-sign: Add parameters to gpg signature function
>>  package_manager: sign DEB package feeds
>> 
>> meta/lib/oe/gpg_sign.py        |  6 +++++-
>> meta/lib/oe/package_manager.py | 19 ++++++++++++++++---
> 
> How do we make sure this does not regress? Can you add a test case similar to one for rpm feed signing?
> 
> Alex

Re: [PULL][PATCH 0/2] Implement debian packages GPG signing

[Ferry Toth]

[-- Attachment #1: Type: text/plain, Size: 2238 bytes --]

On Mon, Apr 1, 2019 at 02:19 AM, Alexander Kanavin wrote:

> 
> Thanks, please send these patches to openembedded-core mailing list.

Yes, Xavier please don't forget to send these patches. They have become essential even now that apt will not allow unsigned repositories.

I have refreshed your patches so they apply to gatesgarth. You can find them here if you are interested: https://github.com/htot/meta-intel-edison/tree/gatesgarth/utils
0001-gpg-sign-Add-parameters-to-gpg-signature-function.patch
0002-package_manager-sign-DEB-package-feeds.patch

I have one question though. When building the sdk I get:

Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
signing-keys-dev : Depends: signing-keys (= 1.0-r0) but it is not installable
E: Unable to correct problems, you have held broken packages.

I don't know how to resolve that. Any ideas?

> 
> Some kind of test case similar to the rpm feed sign test in
> meta/lib/oeqa/selftest/cases/runtime_test.py would be good to have.
> 
> Alex
> 
> 
> On Mon, 1 Apr 2019 at 10:45, Xavier Berger <xavier.berger@bio-logic.net>
> wrote:
> 
>> 
>> The following patches add signing feature for debian packages.
>> 
>> 
>> The following changes since commit
>> 3bc700a12d4f21a5c7a1575da0972f9f78b4a091:
>> 
>> mesa: update 19.0.0 -> 19.0.1 (2019-03-29 08:28:53 +0000)
>> 
>> are available in the git repository at:
>> 
>> git://push.yoctoproject.org/poky-contrib xberger/deb_sign
>> 
>> Xavier Berger (2):
>> gpg-sign: Add parameters to gpg signature function
>> package_manager: sign DEB package feeds
>> 
>> meta/lib/oe/gpg_sign.py | 6 +++++-
>> meta/lib/oe/package_manager.py | 19 ++++++++++++++++---
>> 2 files changed, 21 insertions(+), 4 deletions(-)
>> 
>> --
>> 2.7.4
>> 
>> --
>> _______________________________________________
>> poky mailing list
>> poky@yoctoproject.org
>> https://lists.yoctoproject.org/listinfo/poky
> 
>

[-- Attachment #2: Type: text/html, Size: 2632 bytes --]

Re: [PULL][PATCH 0/2] Implement debian packages GPG signing

[Ferry Toth]

[-- Attachment #1: Type: text/plain, Size: 578 bytes --]

It seems nobody is interested in using apt for package management? Strange.

Let me explain: since gatesgart apt requires a repository to be signed. The above linked patches do exactly that. The can still be applied to hardknott and honister. To me seems to be a good addition that brings deb to the same level as rpm and ipk.

On the signing-keys issue, it may be unrelated. Let me self answer this:

signing-keys-dev depends on signing-keys which doesn't exist. It is not built because it is empty.

This is resolved by adding to the recipe:
ALLOW_EMPTY_${PN} = "1"

[-- Attachment #2: Type: text/html, Size: 636 bytes --]

Re: [poky] [PULL][PATCH 0/2] Implement debian packages GPG signing

[Alexander Kanavin]

You need to rebase any patches you have to master and send them to the
oe-core list, otherwise nothing is going to happen.

Alex

On Thu, 17 Mar 2022 at 18:59, Ferry Toth <fntoth@gmail.com> wrote:
>
> It seems nobody is interested in using apt for package management? Strange.
>
> Let me explain: since gatesgart apt requires a repository to be signed. The above linked patches do exactly that. The can still be applied to hardknott and honister. To me seems to be a good addition that brings deb to the same level as rpm and ipk.
>
> On the signing-keys issue, it may be unrelated. Let me self answer this:
>
> signing-keys-dev depends on signing-keys which doesn't exist. It is not built because it is empty.
>
> This is resolved by adding to the recipe:
>
> ALLOW_EMPTY_${PN} = "1"
>
>
> 
>

Re: [poky] [PULL][PATCH 0/2] Implement debian packages GPG signing

[Ferry Toth]

[-- Attachment #1: Type: text/plain, Size: 1076 bytes --]

Hi Alex,

Clear. But they are not originally mine, I only fixed them up after 
rebase to gatesgarth.

I was hoping Xavier would respond. I put him in To: now.

Thanks,

Ferry

Op 17-03-2022 om 21:22 schreef Alexander Kanavin:
> You need to rebase any patches you have to master and send them to the
> oe-core list, otherwise nothing is going to happen.
>
> Alex
>
> On Thu, 17 Mar 2022 at 18:59, Ferry Toth<fntoth@gmail.com>  wrote:
>> It seems nobody is interested in using apt for package management? Strange.
>>
>> Let me explain: since gatesgart apt requires a repository to be signed. The above linked patches do exactly that. The can still be applied to hardknott and honister. To me seems to be a good addition that brings deb to the same level as rpm and ipk.
>>
>> On the signing-keys issue, it may be unrelated. Let me self answer this:
>>
>> signing-keys-dev depends on signing-keys which doesn't exist. It is not built because it is empty.
>>
>> This is resolved by adding to the recipe:
>>
>> ALLOW_EMPTY_${PN} = "1"
>>
>>
>> 
>>

[-- Attachment #2: Type: text/html, Size: 1795 bytes --]

Re: [poky] [PULL][PATCH 0/2] Implement debian packages GPG signing

[Ferry Toth]

If you don't mind I can send them to oe-core?

Op 18-03-2022 om 16:49 schreef Xavier Berger:
> Hello,
> 
> To avoid the signing issue we had with deb, we decided to switch our 
> project to rpm.
> 
> Creating the required tests to validate my merge request was too complex 
> in regards to my knowledge so I gave up and I will no more working on 
> such branch.
> 
> Best regards,
> 
> Xavier
> 
> *De :*Ferry Toth <fntoth@gmail.com>
> *Envoyé :* vendredi 18 mars 2022 13:09
> *À :* Alexander Kanavin <alex.kanavin@gmail.com>; Xavier Berger 
> <xavier.berger@biologic.net>
> *Cc :* poky@lists.yoctoproject.org
> *Objet :* Re: [poky] [PULL][PATCH 0/2] Implement debian packages GPG signing
> 
> Hi Alex,
> 
> Clear. But they are not originally mine, I only fixed them up after 
> rebase to gatesgarth.
> 
> I was hoping Xavier would respond. I put him in To: now.
> 
> Thanks,
> 
> Ferry
> 
> Op 17-03-2022 om 21:22 schreef Alexander Kanavin:
> 
>     You need to rebase any patches you have to master and send them to the
> 
>     oe-core list, otherwise nothing is going to happen.
> 
>     Alex
> 
>     On Thu, 17 Mar 2022 at 18:59, Ferry Toth<fntoth@gmail.com>  <mailto:fntoth@gmail.com>  wrote:
> 
>         It seems nobody is interested in using apt for package management? Strange.
> 
>         Let me explain: since gatesgart apt requires a repository to be signed. The above linked patches do exactly that. The can still be applied to hardknott and honister. To me seems to be a good addition that brings deb to the same level as rpm and ipk.
> 
>         On the signing-keys issue, it may be unrelated. Let me self answer this:
> 
>         signing-keys-dev depends on signing-keys which doesn't exist. It is not built because it is empty.
> 
>         This is resolved by adding to the recipe:
> 
>         ALLOW_EMPTY_${PN} = "1"
> 
>         
>