* [PATCH v5 0/3] IPK signing for the gpg_sign module
@ 2016-02-17 15:41 Ioan-Adrian Ratiu
2016-02-17 15:41 ` [PATCH v5 1/3] gpg_sign: add local ipk package signing functionality Ioan-Adrian Ratiu
` (2 more replies)
0 siblings, 3 replies; 11+ messages in thread
From: Ioan-Adrian Ratiu @ 2016-02-17 15:41 UTC (permalink / raw)
To: openembedded-core
This patch series extends the gpg_sign module to support ipk signing.
v5 is a rebase on top of Markus' refactoring of the gpg_sign module.
Most notably signature types have been reworked to function parameters
with default values in accordance with the refactoring.
Ioan-Adrian Ratiu (3):
gpg_sign: add local ipk package signing functionality
gpg_sign: detached_sign: add signature type support
package_manager: sign IPK package feeds
meta/classes/package_ipk.bbclass | 6 ++++
meta/classes/sign_ipk.bbclass | 55 ++++++++++++++++++++++++++++++++++
meta/classes/sign_package_feed.bbclass | 10 ++++++-
meta/lib/oe/gpg_sign.py | 45 ++++++++++++++++++++++++++--
meta/lib/oe/package_manager.py | 17 +++++++++--
5 files changed, 128 insertions(+), 5 deletions(-)
create mode 100644 meta/classes/sign_ipk.bbclass
--
2.7.1
^ permalink raw reply [flat|nested] 11+ messages in thread
* [PATCH v5 1/3] gpg_sign: add local ipk package signing functionality
2016-02-17 15:41 [PATCH v5 0/3] IPK signing for the gpg_sign module Ioan-Adrian Ratiu
@ 2016-02-17 15:41 ` Ioan-Adrian Ratiu
2016-02-18 9:04 ` Markus Lehtonen
2016-02-17 15:41 ` [PATCH v5 2/3] gpg_sign: detached_sign: add signature type support Ioan-Adrian Ratiu
2016-02-17 15:41 ` [PATCH v5 3/3] package_manager: sign IPK package feeds Ioan-Adrian Ratiu
2 siblings, 1 reply; 11+ messages in thread
From: Ioan-Adrian Ratiu @ 2016-02-17 15:41 UTC (permalink / raw)
To: openembedded-core
Implement local ipk signing logic inside the gpg backend and add a new
bbclass which configures signing similar to how rpm does it.
The ipk signing process is a bit different from rpm:
- Signatures are stored outside ipk files; opkg connects to a feed
server and downloads them to verify a package.
- Signatures are of two types (both supported by opkg): binary or
ascii armoured. By default we sign using ascii armoured.
- Public keys are stored on targets to verify ipks using the
opkg-keyrings recipe.
Signed-off-by: Ioan-Adrian Ratiu <adrian.ratiu@ni.com>
---
meta/classes/package_ipk.bbclass | 6 +++++
meta/classes/sign_ipk.bbclass | 55 ++++++++++++++++++++++++++++++++++++++++
meta/lib/oe/gpg_sign.py | 39 ++++++++++++++++++++++++++++
3 files changed, 100 insertions(+)
create mode 100644 meta/classes/sign_ipk.bbclass
diff --git a/meta/classes/package_ipk.bbclass b/meta/classes/package_ipk.bbclass
index 51bee28..4f5bbd0 100644
--- a/meta/classes/package_ipk.bbclass
+++ b/meta/classes/package_ipk.bbclass
@@ -246,6 +246,12 @@ python do_package_ipk () {
bb.utils.unlockfile(lf)
raise bb.build.FuncFailed("opkg-build execution failed")
+ if d.getVar('IPK_SIGN_PACKAGES', True) == '1':
+ ipkver = "%s-%s" % (d.getVar('PKGV'), d.getVar('PKGR'))
+ ipk_to_sign = "%s/%s_%s_%s.ipk" % (pkgoutdir, pkgname, ipkver, d.getVar('PACKAGE_ARCH', True))
+ d.setVar('IPK_TO_SIGN', ipk_to_sign)
+ bb.build.exec_func("sign_ipk", d)
+
cleanupcontrol(root)
bb.utils.unlockfile(lf)
diff --git a/meta/classes/sign_ipk.bbclass b/meta/classes/sign_ipk.bbclass
new file mode 100644
index 0000000..cb22bb4
--- /dev/null
+++ b/meta/classes/sign_ipk.bbclass
@@ -0,0 +1,55 @@
+# Class for generating signed IPK packages.
+#
+# Configuration variables used by this class:
+# IPK_GPG_PASSPHRASE_FILE
+# Path to a file containing the passphrase of the signing key.
+# IPK_GPG_NAME
+# Name of the key to sign with.
+# IPK_GPG_BACKEND
+# Optional variable for specifying the backend to use for signing.
+# Currently the only available option is 'local', i.e. local signing
+# on the build host.
+# IPK_GPG_SIGNATURE_TYPE
+# Optional variable for specifying the type of gpg signatures, can be:
+# 1. Ascii armored (ASC), default if not set
+# 2. Binary (BIN)
+# GPG_BIN
+# Optional variable for specifying the gpg binary/wrapper to use for
+# signing.
+# GPG_PATH
+# Optional variable for specifying the gnupg "home" directory:
+#
+
+inherit sanity
+
+IPK_SIGN_PACKAGES = '1'
+IPK_GPG_BACKEND ?= 'local'
+IPK_GPG_SIGNATURE_TYPE ?= 'ASC'
+
+python () {
+ # Check configuration
+ for var in ('IPK_GPG_NAME', 'IPK_GPG_PASSPHRASE_FILE'):
+ if not d.getVar(var, True):
+ raise_sanity_error("You need to define %s in the config" % var, d)
+
+ sigtype = d.getVar("IPK_GPG_SIGNATURE_TYPE", True)
+ if sigtype.upper() != "ASC" and sigtype.upper() != "BIN":
+ raise_sanity_error("Bad value for IPK_GPG_SIGNATURE_TYPE (%s), use either ASC or BIN" % sigtype)
+}
+
+python sign_ipk () {
+ from oe.gpg_sign import get_signer
+
+ ipk_file = d.getVar('IPK_TO_SIGN')
+ bb.debug(1, 'Signing ipk: %s' % ipk_file)
+
+ signer = get_signer(d, d.getVar('IPK_GPG_BACKEND', True))
+
+ sig_type = d.getVar('IPK_GPG_SIGNATURE_TYPE', True)
+ is_ascii_sig = (sig_type.upper() != "BIN")
+
+ signer.sign_ipk(ipk_file,
+ d.getVar('IPK_GPG_NAME', True),
+ d.getVar('IPK_GPG_PASSPHRASE_FILE', True),
+ is_ascii_sig)
+}
diff --git a/meta/lib/oe/gpg_sign.py b/meta/lib/oe/gpg_sign.py
index ada1b2f..138499b 100644
--- a/meta/lib/oe/gpg_sign.py
+++ b/meta/lib/oe/gpg_sign.py
@@ -1,5 +1,6 @@
"""Helper module for GPG signing"""
import os
+import sys
import bb
import oe.utils
@@ -50,6 +51,44 @@ class LocalSigner(object):
bb.error('rpmsign failed: %s' % proc.before.strip())
raise bb.build.FuncFailed("Failed to sign RPM packages")
+ def sign_ipk(self, ipkfile, keyid, passphrase_file, armor=True):
+ """Sign IPK files"""
+ import subprocess
+ from subprocess import Popen
+
+ cmd = [self.gpg_bin, "-q", "--batch", "--yes", "-b", "-u", keyid]
+ if self.gpg_path:
+ cmd += ["--homedir", self.gpg_path]
+ if armor:
+ cmd += ["--armor"]
+
+ try:
+ keypipe = os.pipe()
+
+ # Need to add '\n' in case the passfile does not have it
+ with open(passphrase_file) as fobj:
+ os.write(keypipe[1], fobj.readline() + '\n')
+
+ cmd += ["--passphrase-fd", str(keypipe[0])]
+ cmd += [ipkfile]
+
+ gpg_proc = Popen(cmd, stdin=subprocess.PIPE)
+ gpg_proc.wait()
+
+ os.close(keypipe[1]);
+ os.close(keypipe[0]);
+
+ except IOError as e:
+ bb.error("IO error ({0}): {1}".format(e.errno, e.strerror))
+ raise bb.build.FuncFailed("Failed to sign IPK packages")
+ except OSError as e:
+ bb.error("OS error ({0}): {1}".format(e.errno, e.strerror))
+ raise bb.build.FuncFailed("Failed to sign IPK packages")
+ except:
+ bb.error("Unexpected error: {1}".format(sys.exc_info()[0]))
+ raise bb.build.FuncFailed("Failed to sign IPK packages")
+
+
def detach_sign(self, input_file, keyid, passphrase_file, passphrase=None, armor=True):
"""Create a detached signature of a file"""
import subprocess
--
2.7.1
^ permalink raw reply related [flat|nested] 11+ messages in thread
* [PATCH v5 2/3] gpg_sign: detached_sign: add signature type support
2016-02-17 15:41 [PATCH v5 0/3] IPK signing for the gpg_sign module Ioan-Adrian Ratiu
2016-02-17 15:41 ` [PATCH v5 1/3] gpg_sign: add local ipk package signing functionality Ioan-Adrian Ratiu
@ 2016-02-17 15:41 ` Ioan-Adrian Ratiu
2016-02-18 9:06 ` Markus Lehtonen
2016-02-17 15:41 ` [PATCH v5 3/3] package_manager: sign IPK package feeds Ioan-Adrian Ratiu
2 siblings, 1 reply; 11+ messages in thread
From: Ioan-Adrian Ratiu @ 2016-02-17 15:41 UTC (permalink / raw)
To: openembedded-core
Add support for multiple types of signatures (binary or ascii)
in export_pubkey(). There is no change in behaviour for the function,
the previous implicit default is the new parameter "armor" default.
Signed-off-by: Ioan-Adrian Ratiu <adrian.ratiu@ni.com>
---
meta/lib/oe/gpg_sign.py | 6 ++++--
1 file changed, 4 insertions(+), 2 deletions(-)
diff --git a/meta/lib/oe/gpg_sign.py b/meta/lib/oe/gpg_sign.py
index 138499b..14888c0 100644
--- a/meta/lib/oe/gpg_sign.py
+++ b/meta/lib/oe/gpg_sign.py
@@ -13,12 +13,14 @@ class LocalSigner(object):
self.gpg_path = d.getVar('GPG_PATH', True)
self.rpm_bin = bb.utils.which(os.getenv('PATH'), "rpm")
- def export_pubkey(self, output_file, keyid):
+ def export_pubkey(self, output_file, keyid, armor=True):
"""Export GPG public key to a file"""
- cmd = '%s --batch --yes --export --armor -o %s ' % \
+ cmd = '%s --batch --yes --export -o %s ' % \
(self.gpg_bin, output_file)
if self.gpg_path:
cmd += "--homedir %s " % self.gpg_path
+ if armor:
+ cmd += "--armor "
cmd += keyid
status, output = oe.utils.getstatusoutput(cmd)
if status:
--
2.7.1
^ permalink raw reply related [flat|nested] 11+ messages in thread
* [PATCH v5 3/3] package_manager: sign IPK package feeds
2016-02-17 15:41 [PATCH v5 0/3] IPK signing for the gpg_sign module Ioan-Adrian Ratiu
2016-02-17 15:41 ` [PATCH v5 1/3] gpg_sign: add local ipk package signing functionality Ioan-Adrian Ratiu
2016-02-17 15:41 ` [PATCH v5 2/3] gpg_sign: detached_sign: add signature type support Ioan-Adrian Ratiu
@ 2016-02-17 15:41 ` Ioan-Adrian Ratiu
2016-02-18 9:09 ` Markus Lehtonen
2 siblings, 1 reply; 11+ messages in thread
From: Ioan-Adrian Ratiu @ 2016-02-17 15:41 UTC (permalink / raw)
To: openembedded-core
Create gpg signed ipk package feeds using the gpg backend if configured.
Signed-off-by: Ioan-Adrian Ratiu <adrian.ratiu@ni.com>
---
meta/classes/sign_package_feed.bbclass | 10 +++++++++-
meta/lib/oe/package_manager.py | 17 +++++++++++++++--
2 files changed, 24 insertions(+), 3 deletions(-)
diff --git a/meta/classes/sign_package_feed.bbclass b/meta/classes/sign_package_feed.bbclass
index 63ca02f..2b0548a 100644
--- a/meta/classes/sign_package_feed.bbclass
+++ b/meta/classes/sign_package_feed.bbclass
@@ -10,6 +10,10 @@
# Optional variable for specifying the backend to use for signing.
# Currently the only available option is 'local', i.e. local signing
# on the build host.
+# PACKAGE_FEED_GPG_SIGNATURE_TYPE
+# Optional variable for specifying the type of gpg signature, can be:
+# 1. Ascii armored (ASC), default if not set
+# 2. Binary (BIN)
# GPG_BIN
# Optional variable for specifying the gpg binary/wrapper to use for
# signing.
@@ -20,7 +24,7 @@ inherit sanity
PACKAGE_FEED_SIGN = '1'
PACKAGE_FEED_GPG_BACKEND ?= 'local'
-
+PACKAGE_FEED_GPG_SIGNATURE_TYPE ?= 'ASC'
python () {
# Check sanity of configuration
@@ -28,6 +32,10 @@ python () {
if not d.getVar(var, True):
raise_sanity_error("You need to define %s in the config" % var, d)
+ sigtype = d.getVar("PACKAGE_FEED_GPG_SIGNATURE_TYPE", True)
+ if sigtype.upper() != "ASC" and sigtype.upper() != "BIN":
+ raise_sanity_error("Bad value for PACKAGE_FEED_GPG_SIGNATURE_TYPE (%s), use either ASC or BIN" % sigtype)
+
# Set expected location of the public key
d.setVar('PACKAGE_FEED_GPG_PUBKEY',
os.path.join(d.getVar('STAGING_ETCDIR_NATIVE', False),
diff --git a/meta/lib/oe/package_manager.py b/meta/lib/oe/package_manager.py
index b30a4da..606ba24 100644
--- a/meta/lib/oe/package_manager.py
+++ b/meta/lib/oe/package_manager.py
@@ -163,11 +163,16 @@ class OpkgIndexer(Indexer):
"MULTILIB_ARCHS"]
opkg_index_cmd = bb.utils.which(os.getenv('PATH'), "opkg-make-index")
+ if self.d.getVar('PACKAGE_FEED_SIGN', True) == '1':
+ signer = get_signer(self.d, self.d.getVar('PACKAGE_FEED_GPG_BACKEND', True))
+ else:
+ signer = None
if not os.path.exists(os.path.join(self.deploy_dir, "Packages")):
open(os.path.join(self.deploy_dir, "Packages"), "w").close()
index_cmds = []
+ index_sign_files = []
for arch_var in arch_vars:
archs = self.d.getVar(arch_var, True)
if archs is None:
@@ -186,6 +191,8 @@ class OpkgIndexer(Indexer):
index_cmds.append('%s -r %s -p %s -m %s' %
(opkg_index_cmd, pkgs_file, pkgs_file, pkgs_dir))
+ index_sign_files.append(pkgs_file)
+
if len(index_cmds) == 0:
bb.note("There are no packages in %s!" % self.deploy_dir)
return
@@ -193,9 +200,15 @@ class OpkgIndexer(Indexer):
result = oe.utils.multiprocess_exec(index_cmds, create_index)
if result:
bb.fatal('%s' % ('\n'.join(result)))
- if self.d.getVar('PACKAGE_FEED_SIGN', True) == '1':
- raise NotImplementedError('Package feed signing not implementd for ipk')
+ if signer:
+ feed_sig_type = self.d.getVar('PACKAGE_FEED_GPG_SIGNATURE_TYPE', True)
+ is_ascii_sig = (feed_sig_type.upper() != "BIN")
+ for f in index_sign_files:
+ signer.detach_sign(f,
+ self.d.getVar('PACKAGE_FEED_GPG_NAME', True),
+ self.d.getVar('PACKAGE_FEED_GPG_PASSPHRASE_FILE', True),
+ armor=is_ascii_sig)
class DpkgIndexer(Indexer):
--
2.7.1
^ permalink raw reply related [flat|nested] 11+ messages in thread
* Re: [PATCH v5 1/3] gpg_sign: add local ipk package signing functionality
2016-02-17 15:41 ` [PATCH v5 1/3] gpg_sign: add local ipk package signing functionality Ioan-Adrian Ratiu
@ 2016-02-18 9:04 ` Markus Lehtonen
2016-02-18 9:28 ` Ioan-Adrian Ratiu
0 siblings, 1 reply; 11+ messages in thread
From: Markus Lehtonen @ 2016-02-18 9:04 UTC (permalink / raw)
To: Ioan-Adrian Ratiu, openembedded-core
Hi,
On 17/02/16 17:41, "Ioan-Adrian Ratiu" <openembedded-core-bounces@lists.openembedded.org on behalf of adrian.ratiu@ni.com> wrote:
>Implement local ipk signing logic inside the gpg backend and add a new
>bbclass which configures signing similar to how rpm does it.
>
>The ipk signing process is a bit different from rpm:
> - Signatures are stored outside ipk files; opkg connects to a feed
>server and downloads them to verify a package.
> - Signatures are of two types (both supported by opkg): binary or
>ascii armoured. By default we sign using ascii armoured.
> - Public keys are stored on targets to verify ipks using the
>opkg-keyrings recipe.
>
>Signed-off-by: Ioan-Adrian Ratiu <adrian.ratiu@ni.com>
>---
> meta/classes/package_ipk.bbclass | 6 +++++
> meta/classes/sign_ipk.bbclass | 55 ++++++++++++++++++++++++++++++++++++++++
> meta/lib/oe/gpg_sign.py | 39 ++++++++++++++++++++++++++++
> 3 files changed, 100 insertions(+)
> create mode 100644 meta/classes/sign_ipk.bbclass
>
>diff --git a/meta/classes/package_ipk.bbclass b/meta/classes/package_ipk.bbclass
>index 51bee28..4f5bbd0 100644
>--- a/meta/classes/package_ipk.bbclass
>+++ b/meta/classes/package_ipk.bbclass
>@@ -246,6 +246,12 @@ python do_package_ipk () {
> bb.utils.unlockfile(lf)
> raise bb.build.FuncFailed("opkg-build execution failed")
>
>+ if d.getVar('IPK_SIGN_PACKAGES', True) == '1':
>+ ipkver = "%s-%s" % (d.getVar('PKGV'), d.getVar('PKGR'))
>+ ipk_to_sign = "%s/%s_%s_%s.ipk" % (pkgoutdir, pkgname, ipkver, d.getVar('PACKAGE_ARCH', True))
>+ d.setVar('IPK_TO_SIGN', ipk_to_sign)
>+ bb.build.exec_func("sign_ipk", d)
>+
> cleanupcontrol(root)
> bb.utils.unlockfile(lf)
>
>diff --git a/meta/classes/sign_ipk.bbclass b/meta/classes/sign_ipk.bbclass
>new file mode 100644
>index 0000000..cb22bb4
>--- /dev/null
>+++ b/meta/classes/sign_ipk.bbclass
>@@ -0,0 +1,55 @@
>+# Class for generating signed IPK packages.
>+#
>+# Configuration variables used by this class:
>+# IPK_GPG_PASSPHRASE_FILE
>+# Path to a file containing the passphrase of the signing key.
>+# IPK_GPG_NAME
>+# Name of the key to sign with.
>+# IPK_GPG_BACKEND
>+# Optional variable for specifying the backend to use for signing.
>+# Currently the only available option is 'local', i.e. local signing
>+# on the build host.
>+# IPK_GPG_SIGNATURE_TYPE
>+# Optional variable for specifying the type of gpg signatures, can be:
>+# 1. Ascii armored (ASC), default if not set
>+# 2. Binary (BIN)
>+# GPG_BIN
>+# Optional variable for specifying the gpg binary/wrapper to use for
>+# signing.
>+# GPG_PATH
>+# Optional variable for specifying the gnupg "home" directory:
>+#
>+
>+inherit sanity
>+
>+IPK_SIGN_PACKAGES = '1'
>+IPK_GPG_BACKEND ?= 'local'
>+IPK_GPG_SIGNATURE_TYPE ?= 'ASC'
>+
>+python () {
>+ # Check configuration
>+ for var in ('IPK_GPG_NAME', 'IPK_GPG_PASSPHRASE_FILE'):
>+ if not d.getVar(var, True):
>+ raise_sanity_error("You need to define %s in the config" % var, d)
>+
>+ sigtype = d.getVar("IPK_GPG_SIGNATURE_TYPE", True)
>+ if sigtype.upper() != "ASC" and sigtype.upper() != "BIN":
>+ raise_sanity_error("Bad value for IPK_GPG_SIGNATURE_TYPE (%s), use either ASC or BIN" % sigtype)
>+}
>+
>+python sign_ipk () {
>+ from oe.gpg_sign import get_signer
>+
>+ ipk_file = d.getVar('IPK_TO_SIGN')
>+ bb.debug(1, 'Signing ipk: %s' % ipk_file)
>+
>+ signer = get_signer(d, d.getVar('IPK_GPG_BACKEND', True))
>+
>+ sig_type = d.getVar('IPK_GPG_SIGNATURE_TYPE', True)
>+ is_ascii_sig = (sig_type.upper() != "BIN")
>+
>+ signer.sign_ipk(ipk_file,
>+ d.getVar('IPK_GPG_NAME', True),
>+ d.getVar('IPK_GPG_PASSPHRASE_FILE', True),
>+ is_ascii_sig)
>+}
To me, it would be seem more straightforward to not circulate ipk_to_sign through 'd'. Just define a regular python function like
def sign_ipk(d, ipk_to_sign):
...
And then in package_ipk.bbclass just do "sign_ipk(d, ipk_to_sign)" instead of bb.build.exec_func("sign_ipk", d)"
>diff --git a/meta/lib/oe/gpg_sign.py b/meta/lib/oe/gpg_sign.py
>index ada1b2f..138499b 100644
>--- a/meta/lib/oe/gpg_sign.py
>+++ b/meta/lib/oe/gpg_sign.py
>@@ -1,5 +1,6 @@
> """Helper module for GPG signing"""
> import os
>+import sys
>
> import bb
> import oe.utils
>@@ -50,6 +51,44 @@ class LocalSigner(object):
> bb.error('rpmsign failed: %s' % proc.before.strip())
> raise bb.build.FuncFailed("Failed to sign RPM packages")
>
>+ def sign_ipk(self, ipkfile, keyid, passphrase_file, armor=True):
>+ """Sign IPK files"""
>+ import subprocess
>+ from subprocess import Popen
>+
>+ cmd = [self.gpg_bin, "-q", "--batch", "--yes", "-b", "-u", keyid]
>+ if self.gpg_path:
>+ cmd += ["--homedir", self.gpg_path]
>+ if armor:
>+ cmd += ["--armor"]
>+
>+ try:
>+ keypipe = os.pipe()
>+
>+ # Need to add '\n' in case the passfile does not have it
>+ with open(passphrase_file) as fobj:
>+ os.write(keypipe[1], fobj.readline() + '\n')
>+
>+ cmd += ["--passphrase-fd", str(keypipe[0])]
>+ cmd += [ipkfile]
>+
>+ gpg_proc = Popen(cmd, stdin=subprocess.PIPE)
>+ gpg_proc.wait()
>+
>+ os.close(keypipe[1]);
>+ os.close(keypipe[0]);
>+
>+ except IOError as e:
>+ bb.error("IO error ({0}): {1}".format(e.errno, e.strerror))
>+ raise bb.build.FuncFailed("Failed to sign IPK packages")
>+ except OSError as e:
>+ bb.error("OS error ({0}): {1}".format(e.errno, e.strerror))
>+ raise bb.build.FuncFailed("Failed to sign IPK packages")
>+ except:
>+ bb.error("Unexpected error: {1}".format(sys.exc_info()[0]))
>+ raise bb.build.FuncFailed("Failed to sign IPK packages")
>+
>+
> def detach_sign(self, input_file, keyid, passphrase_file, passphrase=None, armor=True):
> """Create a detached signature of a file"""
> import subprocess
Couldn't you just use detach_sign() instead of introducing sign_ipk(). To me the functionality seems identical.
Thanks,
Markus
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH v5 2/3] gpg_sign: detached_sign: add signature type support
2016-02-17 15:41 ` [PATCH v5 2/3] gpg_sign: detached_sign: add signature type support Ioan-Adrian Ratiu
@ 2016-02-18 9:06 ` Markus Lehtonen
0 siblings, 0 replies; 11+ messages in thread
From: Markus Lehtonen @ 2016-02-18 9:06 UTC (permalink / raw)
To: Ioan-Adrian Ratiu, openembedded-core
On 17/02/16 17:41, "Ioan-Adrian Ratiu" <openembedded-core-bounces@lists.openembedded.org on behalf of adrian.ratiu@ni.com> wrote:
>Add support for multiple types of signatures (binary or ascii)
>in export_pubkey(). There is no change in behaviour for the function,
>the previous implicit default is the new parameter "armor" default.
>
>Signed-off-by: Ioan-Adrian Ratiu <adrian.ratiu@ni.com>
>---
> meta/lib/oe/gpg_sign.py | 6 ++++--
> 1 file changed, 4 insertions(+), 2 deletions(-)
>
>diff --git a/meta/lib/oe/gpg_sign.py b/meta/lib/oe/gpg_sign.py
>index 138499b..14888c0 100644
>--- a/meta/lib/oe/gpg_sign.py
>+++ b/meta/lib/oe/gpg_sign.py
>@@ -13,12 +13,14 @@ class LocalSigner(object):
> self.gpg_path = d.getVar('GPG_PATH', True)
> self.rpm_bin = bb.utils.which(os.getenv('PATH'), "rpm")
>
>- def export_pubkey(self, output_file, keyid):
>+ def export_pubkey(self, output_file, keyid, armor=True):
> """Export GPG public key to a file"""
>- cmd = '%s --batch --yes --export --armor -o %s ' % \
>+ cmd = '%s --batch --yes --export -o %s ' % \
> (self.gpg_bin, output_file)
> if self.gpg_path:
> cmd += "--homedir %s " % self.gpg_path
>+ if armor:
>+ cmd += "--armor "
> cmd += keyid
> status, output = oe.utils.getstatusoutput(cmd)
> if status:
Just change "detached_sign" in the commit subject to "export_pubkey"
Thanks,
Markus
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH v5 3/3] package_manager: sign IPK package feeds
2016-02-17 15:41 ` [PATCH v5 3/3] package_manager: sign IPK package feeds Ioan-Adrian Ratiu
@ 2016-02-18 9:09 ` Markus Lehtonen
0 siblings, 0 replies; 11+ messages in thread
From: Markus Lehtonen @ 2016-02-18 9:09 UTC (permalink / raw)
To: Ioan-Adrian Ratiu, openembedded-core
On 17/02/16 17:41, "Ioan-Adrian Ratiu" <openembedded-core-bounces@lists.openembedded.org on behalf of adrian.ratiu@ni.com> wrote:
>Create gpg signed ipk package feeds using the gpg backend if configured.
>
>Signed-off-by: Ioan-Adrian Ratiu <adrian.ratiu@ni.com>
>---
> meta/classes/sign_package_feed.bbclass | 10 +++++++++-
> meta/lib/oe/package_manager.py | 17 +++++++++++++++--
> 2 files changed, 24 insertions(+), 3 deletions(-)
>
>diff --git a/meta/classes/sign_package_feed.bbclass b/meta/classes/sign_package_feed.bbclass
>index 63ca02f..2b0548a 100644
>--- a/meta/classes/sign_package_feed.bbclass
>+++ b/meta/classes/sign_package_feed.bbclass
>@@ -10,6 +10,10 @@
> # Optional variable for specifying the backend to use for signing.
> # Currently the only available option is 'local', i.e. local signing
> # on the build host.
>+# PACKAGE_FEED_GPG_SIGNATURE_TYPE
>+# Optional variable for specifying the type of gpg signature, can be:
>+# 1. Ascii armored (ASC), default if not set
>+# 2. Binary (BIN)
I'd add a note that PACKAGE_FEED_GPG_SIGNATURE_TYPE is only supported for ipk feeds. This setting is ignored for RPM feeds and afaiu only .asc signatures are supported in rpm-md repositories.
Thanks,
Markus
> # GPG_BIN
> # Optional variable for specifying the gpg binary/wrapper to use for
> # signing.
>@@ -20,7 +24,7 @@ inherit sanity
>
> PACKAGE_FEED_SIGN = '1'
> PACKAGE_FEED_GPG_BACKEND ?= 'local'
>-
>+PACKAGE_FEED_GPG_SIGNATURE_TYPE ?= 'ASC'
>
> python () {
> # Check sanity of configuration
>@@ -28,6 +32,10 @@ python () {
> if not d.getVar(var, True):
> raise_sanity_error("You need to define %s in the config" % var, d)
>
>+ sigtype = d.getVar("PACKAGE_FEED_GPG_SIGNATURE_TYPE", True)
>+ if sigtype.upper() != "ASC" and sigtype.upper() != "BIN":
>+ raise_sanity_error("Bad value for PACKAGE_FEED_GPG_SIGNATURE_TYPE (%s), use either ASC or BIN" % sigtype)
>+
> # Set expected location of the public key
> d.setVar('PACKAGE_FEED_GPG_PUBKEY',
> os.path.join(d.getVar('STAGING_ETCDIR_NATIVE', False),
>diff --git a/meta/lib/oe/package_manager.py b/meta/lib/oe/package_manager.py
>index b30a4da..606ba24 100644
>--- a/meta/lib/oe/package_manager.py
>+++ b/meta/lib/oe/package_manager.py
>@@ -163,11 +163,16 @@ class OpkgIndexer(Indexer):
> "MULTILIB_ARCHS"]
>
> opkg_index_cmd = bb.utils.which(os.getenv('PATH'), "opkg-make-index")
>+ if self.d.getVar('PACKAGE_FEED_SIGN', True) == '1':
>+ signer = get_signer(self.d, self.d.getVar('PACKAGE_FEED_GPG_BACKEND', True))
>+ else:
>+ signer = None
>
> if not os.path.exists(os.path.join(self.deploy_dir, "Packages")):
> open(os.path.join(self.deploy_dir, "Packages"), "w").close()
>
> index_cmds = []
>+ index_sign_files = []
> for arch_var in arch_vars:
> archs = self.d.getVar(arch_var, True)
> if archs is None:
>@@ -186,6 +191,8 @@ class OpkgIndexer(Indexer):
> index_cmds.append('%s -r %s -p %s -m %s' %
> (opkg_index_cmd, pkgs_file, pkgs_file, pkgs_dir))
>
>+ index_sign_files.append(pkgs_file)
>+
> if len(index_cmds) == 0:
> bb.note("There are no packages in %s!" % self.deploy_dir)
> return
>@@ -193,9 +200,15 @@ class OpkgIndexer(Indexer):
> result = oe.utils.multiprocess_exec(index_cmds, create_index)
> if result:
> bb.fatal('%s' % ('\n'.join(result)))
>- if self.d.getVar('PACKAGE_FEED_SIGN', True) == '1':
>- raise NotImplementedError('Package feed signing not implementd for ipk')
>
>+ if signer:
>+ feed_sig_type = self.d.getVar('PACKAGE_FEED_GPG_SIGNATURE_TYPE', True)
>+ is_ascii_sig = (feed_sig_type.upper() != "BIN")
>+ for f in index_sign_files:
>+ signer.detach_sign(f,
>+ self.d.getVar('PACKAGE_FEED_GPG_NAME', True),
>+ self.d.getVar('PACKAGE_FEED_GPG_PASSPHRASE_FILE', True),
>+ armor=is_ascii_sig)
>
>
> class DpkgIndexer(Indexer):
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH v5 1/3] gpg_sign: add local ipk package signing functionality
2016-02-18 9:04 ` Markus Lehtonen
@ 2016-02-18 9:28 ` Ioan-Adrian Ratiu
2016-02-18 15:26 ` Markus Lehtonen
2016-02-19 11:18 ` Ioan-Adrian Ratiu
0 siblings, 2 replies; 11+ messages in thread
From: Ioan-Adrian Ratiu @ 2016-02-18 9:28 UTC (permalink / raw)
To: Markus Lehtonen; +Cc: openembedded-core
Hello
On Thu, 18 Feb 2016 11:04:22 +0200
Markus Lehtonen <markus.lehtonen@linux.intel.com> wrote:
> Hi,
>
>
>
> On 17/02/16 17:41, "Ioan-Adrian Ratiu" <openembedded-core-bounces@lists.openembedded.org on behalf of adrian.ratiu@ni.com> wrote:
>
> >Implement local ipk signing logic inside the gpg backend and add a new
> >bbclass which configures signing similar to how rpm does it.
> >
> >The ipk signing process is a bit different from rpm:
> > - Signatures are stored outside ipk files; opkg connects to a feed
> >server and downloads them to verify a package.
> > - Signatures are of two types (both supported by opkg): binary or
> >ascii armoured. By default we sign using ascii armoured.
> > - Public keys are stored on targets to verify ipks using the
> >opkg-keyrings recipe.
> >
> >Signed-off-by: Ioan-Adrian Ratiu <adrian.ratiu@ni.com>
> >---
> > meta/classes/package_ipk.bbclass | 6 +++++
> > meta/classes/sign_ipk.bbclass | 55 ++++++++++++++++++++++++++++++++++++++++
> > meta/lib/oe/gpg_sign.py | 39 ++++++++++++++++++++++++++++
> > 3 files changed, 100 insertions(+)
> > create mode 100644 meta/classes/sign_ipk.bbclass
> >
> >diff --git a/meta/classes/package_ipk.bbclass b/meta/classes/package_ipk.bbclass
> >index 51bee28..4f5bbd0 100644
> >--- a/meta/classes/package_ipk.bbclass
> >+++ b/meta/classes/package_ipk.bbclass
> >@@ -246,6 +246,12 @@ python do_package_ipk () {
> > bb.utils.unlockfile(lf)
> > raise bb.build.FuncFailed("opkg-build execution failed")
> >
> >+ if d.getVar('IPK_SIGN_PACKAGES', True) == '1':
> >+ ipkver = "%s-%s" % (d.getVar('PKGV'), d.getVar('PKGR'))
> >+ ipk_to_sign = "%s/%s_%s_%s.ipk" % (pkgoutdir, pkgname, ipkver, d.getVar('PACKAGE_ARCH', True))
> >+ d.setVar('IPK_TO_SIGN', ipk_to_sign)
> >+ bb.build.exec_func("sign_ipk", d)
> >+
> > cleanupcontrol(root)
> > bb.utils.unlockfile(lf)
> >
> >diff --git a/meta/classes/sign_ipk.bbclass b/meta/classes/sign_ipk.bbclass
> >new file mode 100644
> >index 0000000..cb22bb4
> >--- /dev/null
> >+++ b/meta/classes/sign_ipk.bbclass
> >@@ -0,0 +1,55 @@
> >+# Class for generating signed IPK packages.
> >+#
> >+# Configuration variables used by this class:
> >+# IPK_GPG_PASSPHRASE_FILE
> >+# Path to a file containing the passphrase of the signing key.
> >+# IPK_GPG_NAME
> >+# Name of the key to sign with.
> >+# IPK_GPG_BACKEND
> >+# Optional variable for specifying the backend to use for signing.
> >+# Currently the only available option is 'local', i.e. local signing
> >+# on the build host.
> >+# IPK_GPG_SIGNATURE_TYPE
> >+# Optional variable for specifying the type of gpg signatures, can be:
> >+# 1. Ascii armored (ASC), default if not set
> >+# 2. Binary (BIN)
> >+# GPG_BIN
> >+# Optional variable for specifying the gpg binary/wrapper to use for
> >+# signing.
> >+# GPG_PATH
> >+# Optional variable for specifying the gnupg "home" directory:
> >+#
> >+
> >+inherit sanity
> >+
> >+IPK_SIGN_PACKAGES = '1'
> >+IPK_GPG_BACKEND ?= 'local'
> >+IPK_GPG_SIGNATURE_TYPE ?= 'ASC'
> >+
> >+python () {
> >+ # Check configuration
> >+ for var in ('IPK_GPG_NAME', 'IPK_GPG_PASSPHRASE_FILE'):
> >+ if not d.getVar(var, True):
> >+ raise_sanity_error("You need to define %s in the config" % var, d)
> >+
> >+ sigtype = d.getVar("IPK_GPG_SIGNATURE_TYPE", True)
> >+ if sigtype.upper() != "ASC" and sigtype.upper() != "BIN":
> >+ raise_sanity_error("Bad value for IPK_GPG_SIGNATURE_TYPE (%s), use either ASC or BIN" % sigtype)
> >+}
> >+
> >+python sign_ipk () {
> >+ from oe.gpg_sign import get_signer
> >+
> >+ ipk_file = d.getVar('IPK_TO_SIGN')
> >+ bb.debug(1, 'Signing ipk: %s' % ipk_file)
> >+
> >+ signer = get_signer(d, d.getVar('IPK_GPG_BACKEND', True))
> >+
> >+ sig_type = d.getVar('IPK_GPG_SIGNATURE_TYPE', True)
> >+ is_ascii_sig = (sig_type.upper() != "BIN")
> >+
> >+ signer.sign_ipk(ipk_file,
> >+ d.getVar('IPK_GPG_NAME', True),
> >+ d.getVar('IPK_GPG_PASSPHRASE_FILE', True),
> >+ is_ascii_sig)
> >+}
>
> To me, it would be seem more straightforward to not circulate ipk_to_sign through 'd'. Just define a regular python function like
> def sign_ipk(d, ipk_to_sign):
> ...
>
> And then in package_ipk.bbclass just do "sign_ipk(d, ipk_to_sign)" instead of bb.build.exec_func("sign_ipk", d)"
>
>
>
>
> >diff --git a/meta/lib/oe/gpg_sign.py b/meta/lib/oe/gpg_sign.py
> >index ada1b2f..138499b 100644
> >--- a/meta/lib/oe/gpg_sign.py
> >+++ b/meta/lib/oe/gpg_sign.py
> >@@ -1,5 +1,6 @@
> > """Helper module for GPG signing"""
> > import os
> >+import sys
> >
> > import bb
> > import oe.utils
> >@@ -50,6 +51,44 @@ class LocalSigner(object):
> > bb.error('rpmsign failed: %s' % proc.before.strip())
> > raise bb.build.FuncFailed("Failed to sign RPM packages")
> >
> >+ def sign_ipk(self, ipkfile, keyid, passphrase_file, armor=True):
> >+ """Sign IPK files"""
> >+ import subprocess
> >+ from subprocess import Popen
> >+
> >+ cmd = [self.gpg_bin, "-q", "--batch", "--yes", "-b", "-u", keyid]
> >+ if self.gpg_path:
> >+ cmd += ["--homedir", self.gpg_path]
> >+ if armor:
> >+ cmd += ["--armor"]
> >+
> >+ try:
> >+ keypipe = os.pipe()
> >+
> >+ # Need to add '\n' in case the passfile does not have it
> >+ with open(passphrase_file) as fobj:
> >+ os.write(keypipe[1], fobj.readline() + '\n')
> >+
> >+ cmd += ["--passphrase-fd", str(keypipe[0])]
> >+ cmd += [ipkfile]
> >+
> >+ gpg_proc = Popen(cmd, stdin=subprocess.PIPE)
> >+ gpg_proc.wait()
> >+
> >+ os.close(keypipe[1]);
> >+ os.close(keypipe[0]);
> >+
> >+ except IOError as e:
> >+ bb.error("IO error ({0}): {1}".format(e.errno, e.strerror))
> >+ raise bb.build.FuncFailed("Failed to sign IPK packages")
> >+ except OSError as e:
> >+ bb.error("OS error ({0}): {1}".format(e.errno, e.strerror))
> >+ raise bb.build.FuncFailed("Failed to sign IPK packages")
> >+ except:
> >+ bb.error("Unexpected error: {1}".format(sys.exc_info()[0]))
> >+ raise bb.build.FuncFailed("Failed to sign IPK packages")
> >+
> >+
> > def detach_sign(self, input_file, keyid, passphrase_file, passphrase=None, armor=True):
> > """Create a detached signature of a file"""
> > import subprocess
>
> Couldn't you just use detach_sign() instead of introducing sign_ipk(). To me the functionality seems identical.
The functionality is almost identical, yes, and consolidating it into one function is a very good idea. I'll do it but I have one question.
The only diference between them is the usage in detach-sign of gpg's "--with-passphrase" arg, and that arg seems to cause some errors on my system:
"gpg: signing failed: Inappropriate ioctl for device"
I have not managed to reliably reproduce and find the cause of this issue. However, if we always open the file in python and read directly in a pipe which
we always pass to gpg using "--passphrase-fd", the error goes away.
Is using something like the following in detach_sign() ok with you?
with open(passphrase_file) as fobj:
os.write(keypipe[1], fobj.readline() + '\n')
cmd += ["--passphrase-fd", str(keypipe[0])]
>
>
> Thanks,
> Markus
>
>
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH v5 1/3] gpg_sign: add local ipk package signing functionality
2016-02-18 9:28 ` Ioan-Adrian Ratiu
@ 2016-02-18 15:26 ` Markus Lehtonen
2016-02-19 11:18 ` Ioan-Adrian Ratiu
1 sibling, 0 replies; 11+ messages in thread
From: Markus Lehtonen @ 2016-02-18 15:26 UTC (permalink / raw)
To: Ioan-Adrian Ratiu; +Cc: openembedded-core
On 18/02/16 11:28, "Ioan-Adrian Ratiu" <adrian.ratiu@ni.com> wrote:
>Hello
>
>On Thu, 18 Feb 2016 11:04:22 +0200
>Markus Lehtonen <markus.lehtonen@linux.intel.com> wrote:
>
>> Hi,
>>
>>
>>
>> On 17/02/16 17:41, "Ioan-Adrian Ratiu" <openembedded-core-bounces@lists.openembedded.org on behalf of adrian.ratiu@ni.com> wrote:
>>
>> >Implement local ipk signing logic inside the gpg backend and add a new
>> >bbclass which configures signing similar to how rpm does it.
>> >
>> >The ipk signing process is a bit different from rpm:
>> > - Signatures are stored outside ipk files; opkg connects to a feed
>> >server and downloads them to verify a package.
>> > - Signatures are of two types (both supported by opkg): binary or
>> >ascii armoured. By default we sign using ascii armoured.
>> > - Public keys are stored on targets to verify ipks using the
>> >opkg-keyrings recipe.
>> >
>> >Signed-off-by: Ioan-Adrian Ratiu <adrian.ratiu@ni.com>
>> >---
>> > meta/classes/package_ipk.bbclass | 6 +++++
>> > meta/classes/sign_ipk.bbclass | 55 ++++++++++++++++++++++++++++++++++++++++
>> > meta/lib/oe/gpg_sign.py | 39 ++++++++++++++++++++++++++++
>> > 3 files changed, 100 insertions(+)
>> > create mode 100644 meta/classes/sign_ipk.bbclass
>> >
>> >diff --git a/meta/classes/package_ipk.bbclass b/meta/classes/package_ipk.bbclass
>> >index 51bee28..4f5bbd0 100644
>> >--- a/meta/classes/package_ipk.bbclass
>> >+++ b/meta/classes/package_ipk.bbclass
>> >@@ -246,6 +246,12 @@ python do_package_ipk () {
>> > bb.utils.unlockfile(lf)
>> > raise bb.build.FuncFailed("opkg-build execution failed")
>> >
>> >+ if d.getVar('IPK_SIGN_PACKAGES', True) == '1':
>> >+ ipkver = "%s-%s" % (d.getVar('PKGV'), d.getVar('PKGR'))
>> >+ ipk_to_sign = "%s/%s_%s_%s.ipk" % (pkgoutdir, pkgname, ipkver, d.getVar('PACKAGE_ARCH', True))
>> >+ d.setVar('IPK_TO_SIGN', ipk_to_sign)
>> >+ bb.build.exec_func("sign_ipk", d)
>> >+
>> > cleanupcontrol(root)
>> > bb.utils.unlockfile(lf)
>> >
>> >diff --git a/meta/classes/sign_ipk.bbclass b/meta/classes/sign_ipk.bbclass
>> >new file mode 100644
>> >index 0000000..cb22bb4
>> >--- /dev/null
>> >+++ b/meta/classes/sign_ipk.bbclass
>> >@@ -0,0 +1,55 @@
>> >+# Class for generating signed IPK packages.
>> >+#
>> >+# Configuration variables used by this class:
>> >+# IPK_GPG_PASSPHRASE_FILE
>> >+# Path to a file containing the passphrase of the signing key.
>> >+# IPK_GPG_NAME
>> >+# Name of the key to sign with.
>> >+# IPK_GPG_BACKEND
>> >+# Optional variable for specifying the backend to use for signing.
>> >+# Currently the only available option is 'local', i.e. local signing
>> >+# on the build host.
>> >+# IPK_GPG_SIGNATURE_TYPE
>> >+# Optional variable for specifying the type of gpg signatures, can be:
>> >+# 1. Ascii armored (ASC), default if not set
>> >+# 2. Binary (BIN)
>> >+# GPG_BIN
>> >+# Optional variable for specifying the gpg binary/wrapper to use for
>> >+# signing.
>> >+# GPG_PATH
>> >+# Optional variable for specifying the gnupg "home" directory:
>> >+#
>> >+
>> >+inherit sanity
>> >+
>> >+IPK_SIGN_PACKAGES = '1'
>> >+IPK_GPG_BACKEND ?= 'local'
>> >+IPK_GPG_SIGNATURE_TYPE ?= 'ASC'
>> >+
>> >+python () {
>> >+ # Check configuration
>> >+ for var in ('IPK_GPG_NAME', 'IPK_GPG_PASSPHRASE_FILE'):
>> >+ if not d.getVar(var, True):
>> >+ raise_sanity_error("You need to define %s in the config" % var, d)
>> >+
>> >+ sigtype = d.getVar("IPK_GPG_SIGNATURE_TYPE", True)
>> >+ if sigtype.upper() != "ASC" and sigtype.upper() != "BIN":
>> >+ raise_sanity_error("Bad value for IPK_GPG_SIGNATURE_TYPE (%s), use either ASC or BIN" % sigtype)
>> >+}
>> >+
>> >+python sign_ipk () {
>> >+ from oe.gpg_sign import get_signer
>> >+
>> >+ ipk_file = d.getVar('IPK_TO_SIGN')
>> >+ bb.debug(1, 'Signing ipk: %s' % ipk_file)
>> >+
>> >+ signer = get_signer(d, d.getVar('IPK_GPG_BACKEND', True))
>> >+
>> >+ sig_type = d.getVar('IPK_GPG_SIGNATURE_TYPE', True)
>> >+ is_ascii_sig = (sig_type.upper() != "BIN")
>> >+
>> >+ signer.sign_ipk(ipk_file,
>> >+ d.getVar('IPK_GPG_NAME', True),
>> >+ d.getVar('IPK_GPG_PASSPHRASE_FILE', True),
>> >+ is_ascii_sig)
>> >+}
>>
>> To me, it would be seem more straightforward to not circulate ipk_to_sign through 'd'. Just define a regular python function like
>> def sign_ipk(d, ipk_to_sign):
>> ...
>>
>> And then in package_ipk.bbclass just do "sign_ipk(d, ipk_to_sign)" instead of bb.build.exec_func("sign_ipk", d)"
>>
>>
>>
>>
>> >diff --git a/meta/lib/oe/gpg_sign.py b/meta/lib/oe/gpg_sign.py
>> >index ada1b2f..138499b 100644
>> >--- a/meta/lib/oe/gpg_sign.py
>> >+++ b/meta/lib/oe/gpg_sign.py
>> >@@ -1,5 +1,6 @@
>> > """Helper module for GPG signing"""
>> > import os
>> >+import sys
>> >
>> > import bb
>> > import oe.utils
>> >@@ -50,6 +51,44 @@ class LocalSigner(object):
>> > bb.error('rpmsign failed: %s' % proc.before.strip())
>> > raise bb.build.FuncFailed("Failed to sign RPM packages")
>> >
>> >+ def sign_ipk(self, ipkfile, keyid, passphrase_file, armor=True):
>> >+ """Sign IPK files"""
>> >+ import subprocess
>> >+ from subprocess import Popen
>> >+
>> >+ cmd = [self.gpg_bin, "-q", "--batch", "--yes", "-b", "-u", keyid]
>> >+ if self.gpg_path:
>> >+ cmd += ["--homedir", self.gpg_path]
>> >+ if armor:
>> >+ cmd += ["--armor"]
>> >+
>> >+ try:
>> >+ keypipe = os.pipe()
>> >+
>> >+ # Need to add '\n' in case the passfile does not have it
>> >+ with open(passphrase_file) as fobj:
>> >+ os.write(keypipe[1], fobj.readline() + '\n')
>> >+
>> >+ cmd += ["--passphrase-fd", str(keypipe[0])]
>> >+ cmd += [ipkfile]
>> >+
>> >+ gpg_proc = Popen(cmd, stdin=subprocess.PIPE)
>> >+ gpg_proc.wait()
>> >+
>> >+ os.close(keypipe[1]);
>> >+ os.close(keypipe[0]);
>> >+
>> >+ except IOError as e:
>> >+ bb.error("IO error ({0}): {1}".format(e.errno, e.strerror))
>> >+ raise bb.build.FuncFailed("Failed to sign IPK packages")
>> >+ except OSError as e:
>> >+ bb.error("OS error ({0}): {1}".format(e.errno, e.strerror))
>> >+ raise bb.build.FuncFailed("Failed to sign IPK packages")
>> >+ except:
>> >+ bb.error("Unexpected error: {1}".format(sys.exc_info()[0]))
>> >+ raise bb.build.FuncFailed("Failed to sign IPK packages")
>> >+
>> >+
>> > def detach_sign(self, input_file, keyid, passphrase_file, passphrase=None, armor=True):
>> > """Create a detached signature of a file"""
>> > import subprocess
>>
>> Couldn't you just use detach_sign() instead of introducing sign_ipk(). To me the functionality seems identical.
>
>The functionality is almost identical, yes, and consolidating it into one function is a very good idea. I'll do it but I have one question.
>
>The only diference between them is the usage in detach-sign of gpg's "--with-passphrase" arg, and that arg seems to cause some errors on my system:
>"gpg: signing failed: Inappropriate ioctl for device"
You mean the "--passphrase-file" option? I don't see any "--with-passphrase" option anywhere.
The problem sounds really strange. What host OS and gpg version do you have? I'd like to understand what is happening there.
>I have not managed to reliably reproduce and find the cause of this issue. However, if we always open the file in python and read directly in a pipe which
>we always pass to gpg using "--passphrase-fd", the error goes away.
>
>Is using something like the following in detach_sign() ok with you?
>
>with open(passphrase_file) as fobj:
> os.write(keypipe[1], fobj.readline() + '\n')
>
>cmd += ["--passphrase-fd", str(keypipe[0])]
If we need to do this, why do you want to use pipes? Why not just something like
with open(passphrase_file) as fobj:
job.communicate(fobj.readline() + '\n')
Thanks,
Markus
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH v5 1/3] gpg_sign: add local ipk package signing functionality
2016-02-18 9:28 ` Ioan-Adrian Ratiu
2016-02-18 15:26 ` Markus Lehtonen
@ 2016-02-19 11:18 ` Ioan-Adrian Ratiu
2016-02-19 13:42 ` Ioan-Adrian Ratiu
1 sibling, 1 reply; 11+ messages in thread
From: Ioan-Adrian Ratiu @ 2016-02-19 11:18 UTC (permalink / raw)
To: Markus Lehtonen; +Cc: openembedded-core
On Thu, 18 Feb 2016 11:28:58 +0200
Ioan-Adrian Ratiu <adrian.ratiu@ni.com> wrote:
> Hello
>
> On Thu, 18 Feb 2016 11:04:22 +0200
> Markus Lehtonen <markus.lehtonen@linux.intel.com> wrote:
>
> > Hi,
> >
> >
> >
> > On 17/02/16 17:41, "Ioan-Adrian Ratiu" <openembedded-core-bounces@lists.openembedded.org on behalf of adrian.ratiu@ni.com> wrote:
> >
> > >Implement local ipk signing logic inside the gpg backend and add a new
> > >bbclass which configures signing similar to how rpm does it.
> > >
> > >The ipk signing process is a bit different from rpm:
> > > - Signatures are stored outside ipk files; opkg connects to a feed
> > >server and downloads them to verify a package.
> > > - Signatures are of two types (both supported by opkg): binary or
> > >ascii armoured. By default we sign using ascii armoured.
> > > - Public keys are stored on targets to verify ipks using the
> > >opkg-keyrings recipe.
> > >
> > >Signed-off-by: Ioan-Adrian Ratiu <adrian.ratiu@ni.com>
> > >---
> > > meta/classes/package_ipk.bbclass | 6 +++++
> > > meta/classes/sign_ipk.bbclass | 55 ++++++++++++++++++++++++++++++++++++++++
> > > meta/lib/oe/gpg_sign.py | 39 ++++++++++++++++++++++++++++
> > > 3 files changed, 100 insertions(+)
> > > create mode 100644 meta/classes/sign_ipk.bbclass
> > >
> > >diff --git a/meta/classes/package_ipk.bbclass b/meta/classes/package_ipk.bbclass
> > >index 51bee28..4f5bbd0 100644
> > >--- a/meta/classes/package_ipk.bbclass
> > >+++ b/meta/classes/package_ipk.bbclass
> > >@@ -246,6 +246,12 @@ python do_package_ipk () {
> > > bb.utils.unlockfile(lf)
> > > raise bb.build.FuncFailed("opkg-build execution failed")
> > >
> > >+ if d.getVar('IPK_SIGN_PACKAGES', True) == '1':
> > >+ ipkver = "%s-%s" % (d.getVar('PKGV'), d.getVar('PKGR'))
> > >+ ipk_to_sign = "%s/%s_%s_%s.ipk" % (pkgoutdir, pkgname, ipkver, d.getVar('PACKAGE_ARCH', True))
> > >+ d.setVar('IPK_TO_SIGN', ipk_to_sign)
> > >+ bb.build.exec_func("sign_ipk", d)
> > >+
> > > cleanupcontrol(root)
> > > bb.utils.unlockfile(lf)
> > >
> > >diff --git a/meta/classes/sign_ipk.bbclass b/meta/classes/sign_ipk.bbclass
> > >new file mode 100644
> > >index 0000000..cb22bb4
> > >--- /dev/null
> > >+++ b/meta/classes/sign_ipk.bbclass
> > >@@ -0,0 +1,55 @@
> > >+# Class for generating signed IPK packages.
> > >+#
> > >+# Configuration variables used by this class:
> > >+# IPK_GPG_PASSPHRASE_FILE
> > >+# Path to a file containing the passphrase of the signing key.
> > >+# IPK_GPG_NAME
> > >+# Name of the key to sign with.
> > >+# IPK_GPG_BACKEND
> > >+# Optional variable for specifying the backend to use for signing.
> > >+# Currently the only available option is 'local', i.e. local signing
> > >+# on the build host.
> > >+# IPK_GPG_SIGNATURE_TYPE
> > >+# Optional variable for specifying the type of gpg signatures, can be:
> > >+# 1. Ascii armored (ASC), default if not set
> > >+# 2. Binary (BIN)
> > >+# GPG_BIN
> > >+# Optional variable for specifying the gpg binary/wrapper to use for
> > >+# signing.
> > >+# GPG_PATH
> > >+# Optional variable for specifying the gnupg "home" directory:
> > >+#
> > >+
> > >+inherit sanity
> > >+
> > >+IPK_SIGN_PACKAGES = '1'
> > >+IPK_GPG_BACKEND ?= 'local'
> > >+IPK_GPG_SIGNATURE_TYPE ?= 'ASC'
> > >+
> > >+python () {
> > >+ # Check configuration
> > >+ for var in ('IPK_GPG_NAME', 'IPK_GPG_PASSPHRASE_FILE'):
> > >+ if not d.getVar(var, True):
> > >+ raise_sanity_error("You need to define %s in the config" % var, d)
> > >+
> > >+ sigtype = d.getVar("IPK_GPG_SIGNATURE_TYPE", True)
> > >+ if sigtype.upper() != "ASC" and sigtype.upper() != "BIN":
> > >+ raise_sanity_error("Bad value for IPK_GPG_SIGNATURE_TYPE (%s), use either ASC or BIN" % sigtype)
> > >+}
> > >+
> > >+python sign_ipk () {
> > >+ from oe.gpg_sign import get_signer
> > >+
> > >+ ipk_file = d.getVar('IPK_TO_SIGN')
> > >+ bb.debug(1, 'Signing ipk: %s' % ipk_file)
> > >+
> > >+ signer = get_signer(d, d.getVar('IPK_GPG_BACKEND', True))
> > >+
> > >+ sig_type = d.getVar('IPK_GPG_SIGNATURE_TYPE', True)
> > >+ is_ascii_sig = (sig_type.upper() != "BIN")
> > >+
> > >+ signer.sign_ipk(ipk_file,
> > >+ d.getVar('IPK_GPG_NAME', True),
> > >+ d.getVar('IPK_GPG_PASSPHRASE_FILE', True),
> > >+ is_ascii_sig)
> > >+}
> >
> > To me, it would be seem more straightforward to not circulate ipk_to_sign through 'd'. Just define a regular python function like
> > def sign_ipk(d, ipk_to_sign):
> > ...
> >
> > And then in package_ipk.bbclass just do "sign_ipk(d, ipk_to_sign)" instead of bb.build.exec_func("sign_ipk", d)"
> >
> >
> >
> >
> > >diff --git a/meta/lib/oe/gpg_sign.py b/meta/lib/oe/gpg_sign.py
> > >index ada1b2f..138499b 100644
> > >--- a/meta/lib/oe/gpg_sign.py
> > >+++ b/meta/lib/oe/gpg_sign.py
> > >@@ -1,5 +1,6 @@
> > > """Helper module for GPG signing"""
> > > import os
> > >+import sys
> > >
> > > import bb
> > > import oe.utils
> > >@@ -50,6 +51,44 @@ class LocalSigner(object):
> > > bb.error('rpmsign failed: %s' % proc.before.strip())
> > > raise bb.build.FuncFailed("Failed to sign RPM packages")
> > >
> > >+ def sign_ipk(self, ipkfile, keyid, passphrase_file, armor=True):
> > >+ """Sign IPK files"""
> > >+ import subprocess
> > >+ from subprocess import Popen
> > >+
> > >+ cmd = [self.gpg_bin, "-q", "--batch", "--yes", "-b", "-u", keyid]
> > >+ if self.gpg_path:
> > >+ cmd += ["--homedir", self.gpg_path]
> > >+ if armor:
> > >+ cmd += ["--armor"]
> > >+
> > >+ try:
> > >+ keypipe = os.pipe()
> > >+
> > >+ # Need to add '\n' in case the passfile does not have it
> > >+ with open(passphrase_file) as fobj:
> > >+ os.write(keypipe[1], fobj.readline() + '\n')
> > >+
> > >+ cmd += ["--passphrase-fd", str(keypipe[0])]
> > >+ cmd += [ipkfile]
> > >+
> > >+ gpg_proc = Popen(cmd, stdin=subprocess.PIPE)
> > >+ gpg_proc.wait()
> > >+
> > >+ os.close(keypipe[1]);
> > >+ os.close(keypipe[0]);
> > >+
> > >+ except IOError as e:
> > >+ bb.error("IO error ({0}): {1}".format(e.errno, e.strerror))
> > >+ raise bb.build.FuncFailed("Failed to sign IPK packages")
> > >+ except OSError as e:
> > >+ bb.error("OS error ({0}): {1}".format(e.errno, e.strerror))
> > >+ raise bb.build.FuncFailed("Failed to sign IPK packages")
> > >+ except:
> > >+ bb.error("Unexpected error: {1}".format(sys.exc_info()[0]))
> > >+ raise bb.build.FuncFailed("Failed to sign IPK packages")
> > >+
> > >+
> > > def detach_sign(self, input_file, keyid, passphrase_file, passphrase=None, armor=True):
> > > """Create a detached signature of a file"""
> > > import subprocess
> >
> > Couldn't you just use detach_sign() instead of introducing sign_ipk(). To me the functionality seems identical.
>
> The functionality is almost identical, yes, and consolidating it into one function is a very good idea. I'll do it but I have one question.
>
> The only diference between them is the usage in detach-sign of gpg's "--with-passphrase" arg, and that arg seems to cause some errors on my system:
> "gpg: signing failed: Inappropriate ioctl for device"
>
> I have not managed to reliably reproduce and find the cause of this issue. However, if we always open the file in python and read directly in a pipe which
> we always pass to gpg using "--passphrase-fd", the error goes away.
>
> Is using something like the following in detach_sign() ok with you?
>
> with open(passphrase_file) as fobj:
> os.write(keypipe[1], fobj.readline() + '\n')
>
> cmd += ["--passphrase-fd", str(keypipe[0])]
Good news: I managed to reproduce and find the cause of the problem: pinentry mode.
Gpg has a parameter "--pinentry-mode" which by default is set to ask, but when doing
batch singing and sending the passphrases through pipes it needs to be set to cancel.
So now both methods work! :) I'll go with your method of using --passphrase-file and
--passphrase-fd 0 because it is more clearer.
>
> >
> >
> > Thanks,
> > Markus
> >
> >
>
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH v5 1/3] gpg_sign: add local ipk package signing functionality
2016-02-19 11:18 ` Ioan-Adrian Ratiu
@ 2016-02-19 13:42 ` Ioan-Adrian Ratiu
0 siblings, 0 replies; 11+ messages in thread
From: Ioan-Adrian Ratiu @ 2016-02-19 13:42 UTC (permalink / raw)
To: Markus Lehtonen; +Cc: openembedded-core
On Fri, 19 Feb 2016 13:18:12 +0200
Ioan-Adrian Ratiu <adrian.ratiu@ni.com> wrote:
> On Thu, 18 Feb 2016 11:28:58 +0200
> Ioan-Adrian Ratiu <adrian.ratiu@ni.com> wrote:
>
> > Hello
> >
> > On Thu, 18 Feb 2016 11:04:22 +0200
> > Markus Lehtonen <markus.lehtonen@linux.intel.com> wrote:
> >
> > > Hi,
> > >
> > >
> > >
> > > On 17/02/16 17:41, "Ioan-Adrian Ratiu" <openembedded-core-bounces@lists.openembedded.org on behalf of adrian.ratiu@ni.com> wrote:
> > >
> > > >Implement local ipk signing logic inside the gpg backend and add a new
> > > >bbclass which configures signing similar to how rpm does it.
> > > >
> > > >The ipk signing process is a bit different from rpm:
> > > > - Signatures are stored outside ipk files; opkg connects to a feed
> > > >server and downloads them to verify a package.
> > > > - Signatures are of two types (both supported by opkg): binary or
> > > >ascii armoured. By default we sign using ascii armoured.
> > > > - Public keys are stored on targets to verify ipks using the
> > > >opkg-keyrings recipe.
> > > >
> > > >Signed-off-by: Ioan-Adrian Ratiu <adrian.ratiu@ni.com>
> > > >---
> > > > meta/classes/package_ipk.bbclass | 6 +++++
> > > > meta/classes/sign_ipk.bbclass | 55 ++++++++++++++++++++++++++++++++++++++++
> > > > meta/lib/oe/gpg_sign.py | 39 ++++++++++++++++++++++++++++
> > > > 3 files changed, 100 insertions(+)
> > > > create mode 100644 meta/classes/sign_ipk.bbclass
> > > >
> > > >diff --git a/meta/classes/package_ipk.bbclass b/meta/classes/package_ipk.bbclass
> > > >index 51bee28..4f5bbd0 100644
> > > >--- a/meta/classes/package_ipk.bbclass
> > > >+++ b/meta/classes/package_ipk.bbclass
> > > >@@ -246,6 +246,12 @@ python do_package_ipk () {
> > > > bb.utils.unlockfile(lf)
> > > > raise bb.build.FuncFailed("opkg-build execution failed")
> > > >
> > > >+ if d.getVar('IPK_SIGN_PACKAGES', True) == '1':
> > > >+ ipkver = "%s-%s" % (d.getVar('PKGV'), d.getVar('PKGR'))
> > > >+ ipk_to_sign = "%s/%s_%s_%s.ipk" % (pkgoutdir, pkgname, ipkver, d.getVar('PACKAGE_ARCH', True))
> > > >+ d.setVar('IPK_TO_SIGN', ipk_to_sign)
> > > >+ bb.build.exec_func("sign_ipk", d)
> > > >+
> > > > cleanupcontrol(root)
> > > > bb.utils.unlockfile(lf)
> > > >
> > > >diff --git a/meta/classes/sign_ipk.bbclass b/meta/classes/sign_ipk.bbclass
> > > >new file mode 100644
> > > >index 0000000..cb22bb4
> > > >--- /dev/null
> > > >+++ b/meta/classes/sign_ipk.bbclass
> > > >@@ -0,0 +1,55 @@
> > > >+# Class for generating signed IPK packages.
> > > >+#
> > > >+# Configuration variables used by this class:
> > > >+# IPK_GPG_PASSPHRASE_FILE
> > > >+# Path to a file containing the passphrase of the signing key.
> > > >+# IPK_GPG_NAME
> > > >+# Name of the key to sign with.
> > > >+# IPK_GPG_BACKEND
> > > >+# Optional variable for specifying the backend to use for signing.
> > > >+# Currently the only available option is 'local', i.e. local signing
> > > >+# on the build host.
> > > >+# IPK_GPG_SIGNATURE_TYPE
> > > >+# Optional variable for specifying the type of gpg signatures, can be:
> > > >+# 1. Ascii armored (ASC), default if not set
> > > >+# 2. Binary (BIN)
> > > >+# GPG_BIN
> > > >+# Optional variable for specifying the gpg binary/wrapper to use for
> > > >+# signing.
> > > >+# GPG_PATH
> > > >+# Optional variable for specifying the gnupg "home" directory:
> > > >+#
> > > >+
> > > >+inherit sanity
> > > >+
> > > >+IPK_SIGN_PACKAGES = '1'
> > > >+IPK_GPG_BACKEND ?= 'local'
> > > >+IPK_GPG_SIGNATURE_TYPE ?= 'ASC'
> > > >+
> > > >+python () {
> > > >+ # Check configuration
> > > >+ for var in ('IPK_GPG_NAME', 'IPK_GPG_PASSPHRASE_FILE'):
> > > >+ if not d.getVar(var, True):
> > > >+ raise_sanity_error("You need to define %s in the config" % var, d)
> > > >+
> > > >+ sigtype = d.getVar("IPK_GPG_SIGNATURE_TYPE", True)
> > > >+ if sigtype.upper() != "ASC" and sigtype.upper() != "BIN":
> > > >+ raise_sanity_error("Bad value for IPK_GPG_SIGNATURE_TYPE (%s), use either ASC or BIN" % sigtype)
> > > >+}
> > > >+
> > > >+python sign_ipk () {
> > > >+ from oe.gpg_sign import get_signer
> > > >+
> > > >+ ipk_file = d.getVar('IPK_TO_SIGN')
> > > >+ bb.debug(1, 'Signing ipk: %s' % ipk_file)
> > > >+
> > > >+ signer = get_signer(d, d.getVar('IPK_GPG_BACKEND', True))
> > > >+
> > > >+ sig_type = d.getVar('IPK_GPG_SIGNATURE_TYPE', True)
> > > >+ is_ascii_sig = (sig_type.upper() != "BIN")
> > > >+
> > > >+ signer.sign_ipk(ipk_file,
> > > >+ d.getVar('IPK_GPG_NAME', True),
> > > >+ d.getVar('IPK_GPG_PASSPHRASE_FILE', True),
> > > >+ is_ascii_sig)
> > > >+}
> > >
> > > To me, it would be seem more straightforward to not circulate ipk_to_sign through 'd'. Just define a regular python function like
> > > def sign_ipk(d, ipk_to_sign):
> > > ...
> > >
> > > And then in package_ipk.bbclass just do "sign_ipk(d, ipk_to_sign)" instead of bb.build.exec_func("sign_ipk", d)"
> > >
> > >
> > >
> > >
> > > >diff --git a/meta/lib/oe/gpg_sign.py b/meta/lib/oe/gpg_sign.py
> > > >index ada1b2f..138499b 100644
> > > >--- a/meta/lib/oe/gpg_sign.py
> > > >+++ b/meta/lib/oe/gpg_sign.py
> > > >@@ -1,5 +1,6 @@
> > > > """Helper module for GPG signing"""
> > > > import os
> > > >+import sys
> > > >
> > > > import bb
> > > > import oe.utils
> > > >@@ -50,6 +51,44 @@ class LocalSigner(object):
> > > > bb.error('rpmsign failed: %s' % proc.before.strip())
> > > > raise bb.build.FuncFailed("Failed to sign RPM packages")
> > > >
> > > >+ def sign_ipk(self, ipkfile, keyid, passphrase_file, armor=True):
> > > >+ """Sign IPK files"""
> > > >+ import subprocess
> > > >+ from subprocess import Popen
> > > >+
> > > >+ cmd = [self.gpg_bin, "-q", "--batch", "--yes", "-b", "-u", keyid]
> > > >+ if self.gpg_path:
> > > >+ cmd += ["--homedir", self.gpg_path]
> > > >+ if armor:
> > > >+ cmd += ["--armor"]
> > > >+
> > > >+ try:
> > > >+ keypipe = os.pipe()
> > > >+
> > > >+ # Need to add '\n' in case the passfile does not have it
> > > >+ with open(passphrase_file) as fobj:
> > > >+ os.write(keypipe[1], fobj.readline() + '\n')
> > > >+
> > > >+ cmd += ["--passphrase-fd", str(keypipe[0])]
> > > >+ cmd += [ipkfile]
> > > >+
> > > >+ gpg_proc = Popen(cmd, stdin=subprocess.PIPE)
> > > >+ gpg_proc.wait()
> > > >+
> > > >+ os.close(keypipe[1]);
> > > >+ os.close(keypipe[0]);
> > > >+
> > > >+ except IOError as e:
> > > >+ bb.error("IO error ({0}): {1}".format(e.errno, e.strerror))
> > > >+ raise bb.build.FuncFailed("Failed to sign IPK packages")
> > > >+ except OSError as e:
> > > >+ bb.error("OS error ({0}): {1}".format(e.errno, e.strerror))
> > > >+ raise bb.build.FuncFailed("Failed to sign IPK packages")
> > > >+ except:
> > > >+ bb.error("Unexpected error: {1}".format(sys.exc_info()[0]))
> > > >+ raise bb.build.FuncFailed("Failed to sign IPK packages")
> > > >+
> > > >+
> > > > def detach_sign(self, input_file, keyid, passphrase_file, passphrase=None, armor=True):
> > > > """Create a detached signature of a file"""
> > > > import subprocess
> > >
> > > Couldn't you just use detach_sign() instead of introducing sign_ipk(). To me the functionality seems identical.
> >
> > The functionality is almost identical, yes, and consolidating it into one function is a very good idea. I'll do it but I have one question.
> >
> > The only diference between them is the usage in detach-sign of gpg's "--with-passphrase" arg, and that arg seems to cause some errors on my system:
> > "gpg: signing failed: Inappropriate ioctl for device"
> >
> > I have not managed to reliably reproduce and find the cause of this issue. However, if we always open the file in python and read directly in a pipe which
> > we always pass to gpg using "--passphrase-fd", the error goes away.
> >
> > Is using something like the following in detach_sign() ok with you?
> >
> > with open(passphrase_file) as fobj:
> > os.write(keypipe[1], fobj.readline() + '\n')
> >
> > cmd += ["--passphrase-fd", str(keypipe[0])]
>
> Good news: I managed to reproduce and find the cause of the problem: pinentry mode.
> Gpg has a parameter "--pinentry-mode" which by default is set to ask, but when doing
> batch singing and sending the passphrases through pipes it needs to be set to cancel.
>
> So now both methods work! :) I'll go with your method of using --passphrase-file and
> --passphrase-fd 0 because it is more clearer.
*facepalm*
I mistyped the value of the pinentry-mode parameter it's loopback not cancel.
More info at https://wiki.archlinux.org/index.php/GnuPG#Unattended_passphrase
>
> >
> > >
> > >
> > > Thanks,
> > > Markus
> > >
> > >
> >
>
^ permalink raw reply [flat|nested] 11+ messages in thread
end of thread, other threads:[~2016-02-19 13:42 UTC | newest]
Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-02-17 15:41 [PATCH v5 0/3] IPK signing for the gpg_sign module Ioan-Adrian Ratiu
2016-02-17 15:41 ` [PATCH v5 1/3] gpg_sign: add local ipk package signing functionality Ioan-Adrian Ratiu
2016-02-18 9:04 ` Markus Lehtonen
2016-02-18 9:28 ` Ioan-Adrian Ratiu
2016-02-18 15:26 ` Markus Lehtonen
2016-02-19 11:18 ` Ioan-Adrian Ratiu
2016-02-19 13:42 ` Ioan-Adrian Ratiu
2016-02-17 15:41 ` [PATCH v5 2/3] gpg_sign: detached_sign: add signature type support Ioan-Adrian Ratiu
2016-02-18 9:06 ` Markus Lehtonen
2016-02-17 15:41 ` [PATCH v5 3/3] package_manager: sign IPK package feeds Ioan-Adrian Ratiu
2016-02-18 9:09 ` Markus Lehtonen
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.