[Buildroot] [PATCH 0/3] allow to recompile wireless-regdb

[Buildroot] [PATCH 0/3] allow to recompile wireless-regdb

[Kurt Van Dijck]

This series adds an option to recompile wireless-regdb.
This became necessary in our product for outdoor use.
In some countries, different regulations apply for indoor
and outdoor use. The precompiled version of wireless-regdb
is appropriate for indoor use only.

For this to work, we compile using python3, and added
host-python3-m2crypto prerequisite in buildroot.

Finally, we patched wireless-regdb for hiding the username
of the build-server.

Kind regards,
Kurt

[Buildroot] [PATCH 1/3] python3-m2crypto: import package

[Kurt Van Dijck]

Signed-off-by: Kurt Van Dijck <dev.kurt@vandijck-laurijssen.be>
---
 package/python3-m2crypto/python3-m2crypto.mk | 21 ++++++++++++++++++++
 1 file changed, 21 insertions(+)
 create mode 100644 package/python3-m2crypto/python3-m2crypto.mk

diff --git a/package/python3-m2crypto/python3-m2crypto.mk b/package/python3-m2crypto/python3-m2crypto.mk
new file mode 100644
index 0000000000..4394226067
--- /dev/null
+++ b/package/python3-m2crypto/python3-m2crypto.mk
@@ -0,0 +1,21 @@
+################################################################################
+#
+# python3-m2crypto
+#
+################################################################################
+
+# Please keep in sync with
+PYTHON3_M2CRYPTO_VERSION = 0.35.2
+PYTHON3_M2CRYPTO_SOURCE = M2Crypto-$(PYTHON3_M2CRYPTO_VERSION).tar.gz
+PYTHON3_M2CRYPTO_SITE = https://files.pythonhosted.org/packages/74/18/3beedd4ac48b52d1a4d12f2a8c5cf0ae342ce974859fba838cbbc1580249
+PYTHON3_M2CRYPTO_SETUP_TYPE = setuptools
+PYTHON3_M2CRYPTO_LICENSE = MIT
+PYTHON3_M2CRYPTO_LICENSE_FILES = LICENSE
+HOST_PYTHON3_M2CRYPTO_DL_SUBDIR = python-m2crypto
+HOST_PYTHON3_M2CRYPTO_NEEDS_HOST_PYTHON = python3
+
+HOST_PYTHON3_M2CRYPTO_DEPENDENCIES = host-swig
+
+HOST_PYTHON3_M2CRYPTO_BUILD_OPTS = --openssl $(HOST_DIR)
+
+$(eval $(host-python-package))
-- 
2.25.0

[Buildroot] [PATCH 2/3] wireless-regdb: add option to rebuild

[Kurt Van Dijck]

Signed-off-by: Kurt Van Dijck <dev.kurt@vandijck-laurijssen.be>
---
 package/wireless-regdb/0003-python3.patch | 24 +++++++++++++++++++++++
 package/wireless-regdb/Config.in          | 10 ++++++++++
 package/wireless-regdb/wireless-regdb.mk  | 13 ++++++++++--
 3 files changed, 45 insertions(+), 2 deletions(-)
 create mode 100644 package/wireless-regdb/0003-python3.patch

diff --git a/package/wireless-regdb/0003-python3.patch b/package/wireless-regdb/0003-python3.patch
new file mode 100644
index 0000000000..87226b763f
--- /dev/null
+++ b/package/wireless-regdb/0003-python3.patch
@@ -0,0 +1,24 @@
+--- a/db2bin.py	2020-06-04 12:10:59.825073819 +0200
++++ b/db2bin.py	2020-06-04 12:11:52.355229867 +0200
+@@ -1,4 +1,4 @@
+-#!/usr/bin/env python
++#!/usr/bin/env python3
+ 
+ from io import BytesIO, open
+ import struct
+--- a/db2fw.py	2020-06-04 12:10:59.825073819 +0200
++++ b/db2fw.py	2020-06-04 12:11:52.365229898 +0200
+@@ -1,4 +1,4 @@
+-#!/usr/bin/env python
++#!/usr/bin/env python3
+ 
+ from io import BytesIO, open
+ import struct
+--- a/dbparse.py	2020-06-04 12:10:59.825073819 +0200
++++ b/dbparse.py	2020-06-04 12:11:52.365229898 +0200
+@@ -1,4 +1,4 @@
+-#!/usr/bin/env python
++#!/usr/bin/env python3
+ 
+ from builtins import bytes
+ from functools import total_ordering
diff --git a/package/wireless-regdb/Config.in b/package/wireless-regdb/Config.in
index 17350a41d8..cb8ec20977 100644
--- a/package/wireless-regdb/Config.in
+++ b/package/wireless-regdb/Config.in
@@ -12,3 +12,13 @@ config BR2_PACKAGE_WIRELESS_REGDB
 	  application is needed.
 
 	  https://wireless.wiki.kernel.org/en/developers/regulatory/wireless-regdb
+
+if BR2_PACKAGE_WIRELESS_REGDB
+
+config BR2_WIRELESS_REGDB_REBUILD
+	bool "rebuild wireless-regdb"
+	help
+	  Say no to use the pre-built wireless regdb.
+
+	  Say yes to rebuild, and make sure to deploy your wireless-regdb public key.
+endif
diff --git a/package/wireless-regdb/wireless-regdb.mk b/package/wireless-regdb/wireless-regdb.mk
index 31b62e36e1..d3b30c7fce 100644
--- a/package/wireless-regdb/wireless-regdb.mk
+++ b/package/wireless-regdb/wireless-regdb.mk
@@ -10,12 +10,21 @@ WIRELESS_REGDB_SITE = $(BR2_KERNEL_MIRROR)/software/network/wireless-regdb
 WIRELESS_REGDB_LICENSE = ISC
 WIRELESS_REGDB_LICENSE_FILES = LICENSE
 
+ifeq ($(BR2_WIRELESS_REGDB_REBUILD),y)
+WIRELESS_REGDB_DEPENDENCIES += host-python3-m2crypto
+define WIRELESS_REGDB_BUILD_CMDS
+	$(TARGET_MAKE_ENV) $(MAKE) -C $(@D) clean
+	$(TARGET_MAKE_ENV) $(MAKE) -C $(@D)
+endef
+endif
+
 ifeq ($(BR2_PACKAGE_CRDA),y)
 define  WIRELESS_REGDB_INSTALL_CRDA_TARGET_CMDS
 	$(INSTALL) -m 644 -D -T $(@D)/regulatory.bin \
 		$(TARGET_DIR)/usr/lib/crda/regulatory.bin
-	$(INSTALL) -m 644 -D -T $(@D)/sforshee.key.pub.pem \
-		$(TARGET_DIR)/etc/wireless-regdb/pubkeys/sforshee.key.pub.pem
+	$(INSTALL) -d $(TARGET_DIR)/etc/wireless-regdb/pubkeys
+	$(INSTALL) -m 644 $(wildcard $(@D)/*.key.pub.pem) \
+		$(TARGET_DIR)/etc/wireless-regdb/pubkeys/
 endef
 endif
 
-- 
2.25.0

[Buildroot] [PATCH 3/3] wireless-regdb: omit author info from private key in home directory

[Kurt Van Dijck]

Signed-off-by: Kurt Van Dijck <dev.kurt@vandijck-laurijssen.be>
---
 .../002-no-author-from-user-privkey.patch            | 12 ++++++++++++
 1 file changed, 12 insertions(+)
 create mode 100644 package/wireless-regdb/002-no-author-from-user-privkey.patch

diff --git a/package/wireless-regdb/002-no-author-from-user-privkey.patch b/package/wireless-regdb/002-no-author-from-user-privkey.patch
new file mode 100644
index 0000000000..8cab1e169c
--- /dev/null
+++ b/package/wireless-regdb/002-no-author-from-user-privkey.patch
@@ -0,0 +1,12 @@
+diff -Naur wireless-regdb.orig/Makefile wireless-regdb/Makefile
+--- wireless-regdb.orig/Makefile	2017-03-07 14:13:09.000000000 +0100
++++ wireless-regdb/Makefile	2019-01-07 14:15:14.296020922 +0100
+@@ -29,7 +29,7 @@
+ 			echo custom-user; \
+ 		fi)
+ 
+-REGDB_PRIVKEY ?= ~/.wireless-regdb-$(REGDB_AUTHOR).key.priv.pem
++REGDB_PRIVKEY ?= ~/.wireless-regdb.key.priv.pem
+ REGDB_PUBKEY ?= $(REGDB_AUTHOR).key.pub.pem
+ 
+ REGDB_UPSTREAM_PUBKEY ?= sforshee.key.pub.pem
-- 
2.25.0

[Buildroot] [PATCH 0/3] allow to recompile wireless-regdb

[Matthew Weber]

Kurt,


On Fri, Jun 12, 2020 at 12:47 AM Kurt Van Dijck
<dev.kurt@vandijck-laurijssen.be> wrote:
>
> This series adds an option to recompile wireless-regdb.
> This became necessary in our product for outdoor use.
> In some countries, different regulations apply for indoor
> and outdoor use. The precompiled version of wireless-regdb
> is appropriate for indoor use only.

Thanks for sending this series!

We ran into a similar situation but never tried to upstream an
approach with the wireless-regdb project to have an outdoor profile.
Have you guys looked to see if that would be possible?  With how the
product wireless certs go it could be beneficial to have the outdoor
profile as a standard configuration.

Best Regards,
Matt

[Buildroot] [PATCH 0/3] allow to recompile wireless-regdb

[Kurt Van Dijck]

On ma, 15 jun 2020 06:31:54 -0500, Matthew Weber wrote:
> Kurt,
> 
> 
> On Fri, Jun 12, 2020 at 12:47 AM Kurt Van Dijck
> <dev.kurt@vandijck-laurijssen.be> wrote:
> >
> > This series adds an option to recompile wireless-regdb.
> > This became necessary in our product for outdoor use.
> > In some countries, different regulations apply for indoor
> > and outdoor use. The precompiled version of wireless-regdb
> > is appropriate for indoor use only.
> 
> Thanks for sending this series!
> 
> We ran into a similar situation but never tried to upstream an
> approach with the wireless-regdb project to have an outdoor profile.
> Have you guys looked to see if that would be possible?

We did not attempt to change wireless-regdb itself.
If I remember well, it could in theory support indoor and outdoor regulations,
but you need to tell the driver to do so, and that seemed not really
implemented for our driver (ralink).
Our specific usecase where the differences apply, are in Japan, where
* the regulations db was not yet up to date with the outdoor restrictions
* we wanted to avoid the risks of incomplete implementations

Since wireless-regdb supports recompiling your own db, we figured that
that stripping the indoor-only bands from the db alltogether was the
simplest and safest way to success.

Only recently, our new build-server stumbled over python2/python3 and
our modification became too big.

> With how the
> product wireless certs go it could be beneficial to have the outdoor
> profile as a standard configuration.

We remove the indoor-only bands from the db. That seems quite good.
I wasn't involved in the actual certification, but I can't imagine
how our device would have used the indoor-only bands.

Kurt

[Buildroot] [PATCH 1/3] python3-m2crypto: import package

[Thomas Petazzoni]

Hello Kurt,

Thanks for this contribution. The commit title needs to be:

	package/python3-m2crypto: new package

It should be mentioned that we used to have a python-m2crypto package,
which was removed in commit 484c04878e5e76cdc0cc33c7646f98ab41698a7e
because it was only used by crda, and crda no longer needed it.

On Fri, 12 Jun 2020 07:31:08 +0200
Kurt Van Dijck <dev.kurt@vandijck-laurijssen.be> wrote:

> Signed-off-by: Kurt Van Dijck <dev.kurt@vandijck-laurijssen.be>
> ---
>  package/python3-m2crypto/python3-m2crypto.mk | 21 ++++++++++++++++++++
>  1 file changed, 21 insertions(+)

You're missing a .hash file, as well as an entry in the DEVELOPERS file.

>  create mode 100644 package/python3-m2crypto/python3-m2crypto.mk
> 
> diff --git a/package/python3-m2crypto/python3-m2crypto.mk b/package/python3-m2crypto/python3-m2crypto.mk
> new file mode 100644
> index 0000000000..4394226067
> --- /dev/null
> +++ b/package/python3-m2crypto/python3-m2crypto.mk
> @@ -0,0 +1,21 @@
> +################################################################################
> +#
> +# python3-m2crypto
> +#
> +################################################################################
> +
> +# Please keep in sync with

in sync with ?

> +PYTHON3_M2CRYPTO_VERSION = 0.35.2
> +PYTHON3_M2CRYPTO_SOURCE = M2Crypto-$(PYTHON3_M2CRYPTO_VERSION).tar.gz
> +PYTHON3_M2CRYPTO_SITE = https://files.pythonhosted.org/packages/74/18/3beedd4ac48b52d1a4d12f2a8c5cf0ae342ce974859fba838cbbc1580249
> +PYTHON3_M2CRYPTO_SETUP_TYPE = setuptools
> +PYTHON3_M2CRYPTO_LICENSE = MIT
> +PYTHON3_M2CRYPTO_LICENSE_FILES = LICENSE
> +HOST_PYTHON3_M2CRYPTO_DL_SUBDIR = python-m2crypto
> +HOST_PYTHON3_M2CRYPTO_NEEDS_HOST_PYTHON = python3
> +
> +HOST_PYTHON3_M2CRYPTO_DEPENDENCIES = host-swig
> +
> +HOST_PYTHON3_M2CRYPTO_BUILD_OPTS = --openssl $(HOST_DIR)

It needs OpenSSL ? Then I guess host-openssl should be in the dependencies, no ?

Thanks,

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

[Buildroot] [PATCH 2/3] wireless-regdb: add option to rebuild

[Thomas Petazzoni]

Hello Kurt,

On Fri, 12 Jun 2020 07:31:09 +0200
Kurt Van Dijck <dev.kurt@vandijck-laurijssen.be> wrote:

> Signed-off-by: Kurt Van Dijck <dev.kurt@vandijck-laurijssen.be>

Could you expand the commit log to explain why rebuilding the wireless
regdb might be necessary ?

> diff --git a/package/wireless-regdb/0003-python3.patch b/package/wireless-regdb/0003-python3.patch
> new file mode 100644
> index 0000000000..87226b763f

We need all patches to have a description + Signed-off-by line, and be
generated using "git format-patch -N" if the upstream project uses Git.

> diff --git a/package/wireless-regdb/Config.in b/package/wireless-regdb/Config.in
> index 17350a41d8..cb8ec20977 100644
> --- a/package/wireless-regdb/Config.in
> +++ b/package/wireless-regdb/Config.in
> @@ -12,3 +12,13 @@ config BR2_PACKAGE_WIRELESS_REGDB
>  	  application is needed.
>  
>  	  https://wireless.wiki.kernel.org/en/developers/regulatory/wireless-regdb
> +
> +if BR2_PACKAGE_WIRELESS_REGDB
> +
> +config BR2_WIRELESS_REGDB_REBUILD
> +	bool "rebuild wireless-regdb"
> +	help
> +	  Say no to use the pre-built wireless regdb.
> +
> +	  Say yes to rebuild, and make sure to deploy your wireless-regdb public key.

Lines are too long.

Could you give a bit of background about this wireless-regdb public
key? I see that the Makefile is using some private/public key pair from
the user home directory, this looks absolutely horrible in the context
of Buildroot. Should we instead have Config.in options to provide the
path to the private/public key pair ?

> +endif
> diff --git a/package/wireless-regdb/wireless-regdb.mk b/package/wireless-regdb/wireless-regdb.mk
> index 31b62e36e1..d3b30c7fce 100644
> --- a/package/wireless-regdb/wireless-regdb.mk
> +++ b/package/wireless-regdb/wireless-regdb.mk
> @@ -10,12 +10,21 @@ WIRELESS_REGDB_SITE = $(BR2_KERNEL_MIRROR)/software/network/wireless-regdb
>  WIRELESS_REGDB_LICENSE = ISC
>  WIRELESS_REGDB_LICENSE_FILES = LICENSE
>  
> +ifeq ($(BR2_WIRELESS_REGDB_REBUILD),y)
> +WIRELESS_REGDB_DEPENDENCIES += host-python3-m2crypto
> +define WIRELESS_REGDB_BUILD_CMDS
> +	$(TARGET_MAKE_ENV) $(MAKE) -C $(@D) clean
> +	$(TARGET_MAKE_ENV) $(MAKE) -C $(@D)

Why do you need the "clean" step here ? To force removing the
pre-generated wireless regdb, and therefore have the next make
invocation re-generate it ?

> +endef
> +endif
> +
>  ifeq ($(BR2_PACKAGE_CRDA),y)
>  define  WIRELESS_REGDB_INSTALL_CRDA_TARGET_CMDS
>  	$(INSTALL) -m 644 -D -T $(@D)/regulatory.bin \
>  		$(TARGET_DIR)/usr/lib/crda/regulatory.bin
> -	$(INSTALL) -m 644 -D -T $(@D)/sforshee.key.pub.pem \
> -		$(TARGET_DIR)/etc/wireless-regdb/pubkeys/sforshee.key.pub.pem
> +	$(INSTALL) -d $(TARGET_DIR)/etc/wireless-regdb/pubkeys
> +	$(INSTALL) -m 644 $(wildcard $(@D)/*.key.pub.pem) \
> +		$(TARGET_DIR)/etc/wireless-regdb/pubkeys/

So here you install all public keys. But how are they going to end up
in the wireless-regdb build directory ?

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

[Buildroot] [PATCH 3/3] wireless-regdb: omit author info from private key in home directory

[Thomas Petazzoni]

Hello,

On Fri, 12 Jun 2020 07:31:10 +0200
Kurt Van Dijck <dev.kurt@vandijck-laurijssen.be> wrote:

> Signed-off-by: Kurt Van Dijck <dev.kurt@vandijck-laurijssen.be>
> ---
>  .../002-no-author-from-user-privkey.patch            | 12 ++++++++++++
>  1 file changed, 12 insertions(+)
>  create mode 100644 package/wireless-regdb/002-no-author-from-user-privkey.patch
> 
> diff --git a/package/wireless-regdb/002-no-author-from-user-privkey.patch b/package/wireless-regdb/002-no-author-from-user-privkey.patch
> new file mode 100644
> index 0000000000..8cab1e169c
> --- /dev/null
> +++ b/package/wireless-regdb/002-no-author-from-user-privkey.patch
> @@ -0,0 +1,12 @@
> +diff -Naur wireless-regdb.orig/Makefile wireless-regdb/Makefile
> +--- wireless-regdb.orig/Makefile	2017-03-07 14:13:09.000000000 +0100
> ++++ wireless-regdb/Makefile	2019-01-07 14:15:14.296020922 +0100
> +@@ -29,7 +29,7 @@
> + 			echo custom-user; \
> + 		fi)
> + 
> +-REGDB_PRIVKEY ?= ~/.wireless-regdb-$(REGDB_AUTHOR).key.priv.pem
> ++REGDB_PRIVKEY ?= ~/.wireless-regdb.key.priv.pem
> + REGDB_PUBKEY ?= $(REGDB_AUTHOR).key.pub.pem
> + 
> + REGDB_UPSTREAM_PUBKEY ?= sforshee.key.pub.pem

All patches need a description and Signed-off-by line. But what about
passing REGDB_AUTHOR=buildroot from wireless-regdb.mk, or something
like that ?

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

[Buildroot] [PATCH 3/3] wireless-regdb: omit author info from private key in home directory

[Kurt Van Dijck]

Hey Thomas,

On do, 18 jun 2020 22:28:36 +0200, Thomas Petazzoni wrote:
> Hello,
> 
> On Fri, 12 Jun 2020 07:31:10 +0200
> Kurt Van Dijck <dev.kurt@vandijck-laurijssen.be> wrote:
> 
...
> > +-REGDB_PRIVKEY ?= ~/.wireless-regdb-$(REGDB_AUTHOR).key.priv.pem
> > ++REGDB_PRIVKEY ?= ~/.wireless-regdb.key.priv.pem
> > + REGDB_PUBKEY ?= $(REGDB_AUTHOR).key.pub.pem
> > + 
> > + REGDB_UPSTREAM_PUBKEY ?= sforshee.key.pub.pem
> 
> All patches need a description and Signed-off-by line. But what about
> passing REGDB_AUTHOR=buildroot from wireless-regdb.mk, or something
> like that ?

I'm in the process of discovering that the crda-less system
requires the keys in the kernel (a good choice) which can best
be accomplished by your suggestion here.

I have a few things for a rework
1. put some more comments
2. avoid a python3 patch and use python-m2crypto.
3. prepare for an explicit priv key for both wireless-regdb and kernel

Thanks for the review.
Kurt