[Buildroot] [PATCH 1/1] refpolicy: bump to 2.20170805

[Buildroot] [PATCH 1/1] refpolicy: bump to 2.20170805

[Adam Duskett]

In addition to a simple bump, the following extra changes have occured:
  - Change the refpolicy site to the official release URL.
  - Remove REFPOLICY_SITE_METHOD.
  - Remove REFPOLICY_GIT_SUBMODULES.
  - Refpolicy is now compatible with python3, as such, remove host-python.
    from the dependencies and add a check for python3 or python in it's place.
  - Add upstreamed 0001-fix-regex-escape-sequence-error.patch to fix building
    against python3.6.
  - Add sha256 license hash to hash file.

Signed-off-by: Adam Duskett <aduskett@gmail.com>
---
 .../0001-fix-regex-escape-sequence-error.patch     | 59 ++++++++++++++++++++++
 package/refpolicy/refpolicy.hash                   |  7 ++-
 package/refpolicy/refpolicy.mk                     | 20 +++++---
 3 files changed, 76 insertions(+), 10 deletions(-)
 create mode 100644 package/refpolicy/0001-fix-regex-escape-sequence-error.patch

diff --git a/package/refpolicy/0001-fix-regex-escape-sequence-error.patch b/package/refpolicy/0001-fix-regex-escape-sequence-error.patch
new file mode 100644
index 0000000000..1aa01892d0
--- /dev/null
+++ b/package/refpolicy/0001-fix-regex-escape-sequence-error.patch
@@ -0,0 +1,59 @@
+From 6c9cc47e6cc9c6e67b1b822f7a1a2e1f6d836118 Mon Sep 17 00:00:00 2001
+From: Adam Duskett <Adamduskett@outlook.com>
+Date: Tue, 10 Oct 2017 18:00:30 -0400
+Subject: [PATCH] fix regex escape sequence error.
+
+python3.6 will error out with the message "invalid escape sequence"
+in genhomedircon.py.  This patch fixes these errors by turning the string
+in the into a raw string.
+
+Upstream status: accepted
+
+Signed-off-by: Adam Duskett <aduskett@gmail.com>
+---
+ support/genhomedircon.py | 16 ++++++++--------
+ 1 file changed, 8 insertions(+), 8 deletions(-)
+
+diff --git a/support/genhomedircon.py b/support/genhomedircon.py
+index 036f5cc9..6662f412 100644
+--- a/support/genhomedircon.py
++++ b/support/genhomedircon.py
+@@ -189,13 +189,13 @@ def oldgenhomedircon(filecontextdir, filecontext):
+                 addme = 1
+                 for regex in prefix_regex:
+                         #match a trailing (/*)? which is actually a bug in rpc_pipefs
+-                        regex = re.sub("\(/\*\)\?$", "", regex)
++                        regex = re.sub(r"\(/\*\)\?$", "", regex)
+                         #match a trailing .+
+-                        regex = re.sub("\.+$", "", regex)
++                        regex = re.sub(r"\.+$", "", regex)
+                         #match a trailing .*
+-                        regex = re.sub("\.\*$", "", regex)
++                        regex = re.sub(r"\.\*$", "", regex)
+                         #strip a (/.*)? which matches anything trailing to a /*$ which matches trailing /'s
+-                        regex = re.sub("\(\/\.\*\)\?", "", regex)
++                        regex = re.sub(r"\(\/\.\*\)\?", "", regex)
+                         regex = regex + "/*$"
+                         if re.search(regex, potential, 0):
+                                 addme = 0
+@@ -391,13 +391,13 @@ class selinuxConfig:
+ 		exists=1
+ 		for regex in prefix_regex:
+ 			#match a trailing (/*)? which is actually a bug in rpc_pipefs
+-			regex = re.sub("\(/\*\)\?$", "", regex)
++			regex = re.sub(r"\(/\*\)\?$", "", regex)
+ 			#match a trailing .+
+-			regex = re.sub("\.+$", "", regex)
++			regex = re.sub(r"\.+$", "", regex)
+ 			#match a trailing .*
+-			regex = re.sub("\.\*$", "", regex)
++			regex = re.sub(r"\.\*$", "", regex)
+ 			#strip a (/.*)? which matches anything trailing to a /*$ which matches trailing /'s
+-			regex = re.sub("\(\/\.\*\)\?", "", regex)
++			regex = re.sub(r"\(\/\.\*\)\?", "", regex)
+ 			regex = regex + "/*$"
+ 			if re.search(regex, home, 0):
+ 				exists = 0
+-- 
+2.13.6
+
diff --git a/package/refpolicy/refpolicy.hash b/package/refpolicy/refpolicy.hash
index 7aeac4113d..905ac88542 100644
--- a/package/refpolicy/refpolicy.hash
+++ b/package/refpolicy/refpolicy.hash
@@ -1,2 +1,5 @@
-#From https://github.com/TresysTechnology/refpolicy/wiki/DownloadRelease
-sha256 08f9e2afc5e4939c23e56deeec7c47da029d7b85d82fb4ded01a36eb5da0651e  refpolicy-RELEASE_2_20170204.tar.gz
+# From https://github.com/TresysTechnology/refpolicy/wiki/DownloadRelease
+sha256 045709f5e44199f402149b31c6aab9666bdb1540a5c5ed0312a46c90dedfa52d  refpolicy-2.20170805.tar.bz2
+
+# Locally computed
+sha256 204d8eff92f95aac4df6c8122bc1505f468f3a901e5a4cc08940e0ede1938994 COPYING
diff --git a/package/refpolicy/refpolicy.mk b/package/refpolicy/refpolicy.mk
index 2da151f321..1f1d8a2985 100644
--- a/package/refpolicy/refpolicy.mk
+++ b/package/refpolicy/refpolicy.mk
@@ -4,12 +4,9 @@
 #
 ################################################################################
 
-REFPOLICY_VERSION = RELEASE_2_20170204
-
-# Do not use GitHub helper as git submodules are needed for refpolicy-contrib
-REFPOLICY_SITE = https://github.com/TresysTechnology/refpolicy.git
-REFPOLICY_SITE_METHOD = git
-REFPOLICY_GIT_SUBMODULES = y # Required for refpolicy-contrib
+REFPOLICY_VERSION = 2.20170805
+REFPOLICY_SOURCE = refpolicy-$(REFPOLICY_VERSION).tar.bz2
+REFPOLICY_SITE = https://raw.githubusercontent.com/wiki/TresysTechnology/refpolicy/files
 REFPOLICY_LICENSE = GPL-2.0
 REFPOLICY_LICENSE_FILES = COPYING
 REFPOLICY_INSTALL_STAGING = YES
@@ -19,13 +16,20 @@ REFPOLICY_DEPENDENCIES = \
 	host-policycoreutils \
 	host-setools \
 	host-gawk \
-	host-python \
 	policycoreutils
 
+ifeq ($(BR2_PACKAGE_PYTHON3),y)
+REFPOLICY_DEPENDENCIES += host-python3
+REFPOLICY_PYTHON="$(HOST_DIR)/bin/python(PYTHON3_VERSION_MAJOR)"
+else
+REFPOLICY_DEPENDENCIES += host-python
+REFPOLICY_PYTHON="$(HOST_DIR)/bin/python(PYTHON_VERSION_MAJOR)"
+endif
+
 # Cannot use multiple threads to build the reference policy
 REFPOLICY_MAKE = \
 	TEST_TOOLCHAIN=$(HOST_DIR) \
-	PYTHON="$(HOST_DIR)/bin/python2" \
+	PYHON=$(REFPOLICY_PYTHON) \
 	$(TARGET_MAKE_ENV) \
 	$(MAKE1)
 
-- 
2.13.6

[Buildroot] [PATCH 1/1] refpolicy: bump to 2.20170805

[Matthew Weber]

Adam,

On Fri, Nov 3, 2017 at 7:58 AM, Adam Duskett <aduskett@gmail.com> wrote:
> In addition to a simple bump, the following extra changes have occured:
>   - Change the refpolicy site to the official release URL.
>   - Remove REFPOLICY_SITE_METHOD.
>   - Remove REFPOLICY_GIT_SUBMODULES.

Interesting, do you know how the upstream plan changed for the contrib
material?  Doesn't look like the refpolicy repo has any of that at
first glance.


-Matt

[Buildroot] [PATCH 1/1] refpolicy: bump to 2.20170805

[Thomas Petazzoni]

Hello,

On Fri,  3 Nov 2017 08:58:56 -0400, Adam Duskett wrote:

> +ifeq ($(BR2_PACKAGE_PYTHON3),y)
> +REFPOLICY_DEPENDENCIES += host-python3
> +REFPOLICY_PYTHON="$(HOST_DIR)/bin/python(PYTHON3_VERSION_MAJOR)"
> +else
> +REFPOLICY_DEPENDENCIES += host-python
> +REFPOLICY_PYTHON="$(HOST_DIR)/bin/python(PYTHON_VERSION_MAJOR)"
> +endif

I'm not sure why this is needed. Indeed, if BR2_PACKAGE_PYTHON3=y,
$(HOST_DIR)/bin/python points to python3, and if
BR2_PACKAGE_PYTHON=y, $(HOST_DIR)/bin/python points to python2.

Therefore, you should be able to just use $(HOST_DIR)/bin/python.

Could you try that instead?

Also, please answer Matt's concerns before sending a v2. Thanks!

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux and Kernel engineering
http://free-electrons.com

[Buildroot] [PATCH 1/1] refpolicy: bump to 2.20170805

[Arnout Vandecappelle]

On 03-11-17 14:08, Matthew Weber wrote:
> Adam,
> 
> On Fri, Nov 3, 2017 at 7:58 AM, Adam Duskett <aduskett@gmail.com> wrote:
>> In addition to a simple bump, the following extra changes have occured:
>>   - Change the refpolicy site to the official release URL.
>>   - Remove REFPOLICY_SITE_METHOD.
>>   - Remove REFPOLICY_GIT_SUBMODULES.
> 
> Interesting, do you know how the upstream plan changed for the contrib
> material?  Doesn't look like the refpolicy repo has any of that at
> first glance.

 As far as I can see, the contrib submodule is included in the tarball, isn't it?

 Regards,
 Arnout

-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF

[Buildroot] [PATCH 1/1] refpolicy: bump to 2.20170805

[Adam Duskett]

Hello;

On Fri, Nov 3, 2017 at 1:56 PM, Thomas Petazzoni
<thomas.petazzoni@free-electrons.com> wrote:
> Hello,
>
> On Fri,  3 Nov 2017 08:58:56 -0400, Adam Duskett wrote:
>
>> +ifeq ($(BR2_PACKAGE_PYTHON3),y)
>> +REFPOLICY_DEPENDENCIES += host-python3
>> +REFPOLICY_PYTHON="$(HOST_DIR)/bin/python(PYTHON3_VERSION_MAJOR)"
>> +else
>> +REFPOLICY_DEPENDENCIES += host-python
>> +REFPOLICY_PYTHON="$(HOST_DIR)/bin/python(PYTHON_VERSION_MAJOR)"
>> +endif
>
> I'm not sure why this is needed. Indeed, if BR2_PACKAGE_PYTHON3=y,
> $(HOST_DIR)/bin/python points to python3, and if
> BR2_PACKAGE_PYTHON=y, $(HOST_DIR)/bin/python points to python2.
>
Huh, didn't know that.  I will change that asap!

> Therefore, you should be able to just use $(HOST_DIR)/bin/python.
>
> Could you try that instead?
>
> Also, please answer Matt's concerns before sending a v2. Thanks!
>
We talked on IRC. :)  The contrib module is included in the tarball as
Arnout has said.
> Thomas
> --
> Thomas Petazzoni, CTO, Free Electrons
> Embedded Linux and Kernel engineering
> http://free-electrons.com

[Buildroot] [PATCH 1/1] refpolicy: bump to 2.20170805

[Matthew Weber]

Adam,

On Sun, Nov 5, 2017 at 11:34 AM, Adam Duskett <aduskett@gmail.com> wrote:
>
> Hello;
>
> On Fri, Nov 3, 2017 at 1:56 PM, Thomas Petazzoni
> <thomas.petazzoni@free-electrons.com> wrote:
> > Hello,
> >
> > On Fri,  3 Nov 2017 08:58:56 -0400, Adam Duskett wrote:
> >
> >> +ifeq ($(BR2_PACKAGE_PYTHON3),y)
> >> +REFPOLICY_DEPENDENCIES += host-python3
> >> +REFPOLICY_PYTHON="$(HOST_DIR)/bin/python(PYTHON3_VERSION_MAJOR)"
> >> +else
> >> +REFPOLICY_DEPENDENCIES += host-python
> >> +REFPOLICY_PYTHON="$(HOST_DIR)/bin/python(PYTHON_VERSION_MAJOR)"
> >> +endif
> >
> > I'm not sure why this is needed. Indeed, if BR2_PACKAGE_PYTHON3=y,
> > $(HOST_DIR)/bin/python points to python3, and if
> > BR2_PACKAGE_PYTHON=y, $(HOST_DIR)/bin/python points to python2.
> >
> Huh, didn't know that.  I will change that asap!
>
> > Therefore, you should be able to just use $(HOST_DIR)/bin/python.
> >
> > Could you try that instead?
> >
> > Also, please answer Matt's concerns before sending a v2. Thanks!
> >
> We talked on IRC. :)  The contrib module is included in the tarball as
> Arnout has said.
>
>

Sorry took a bit to get back to this email.  I've figured out what
happened.  Nothing has changed in how refpolicy and refpolicy-contrib
are released.  This patch series changes the package to use a released
archive vs the git repo.  In the upstream git repo it still uses the
submodule for retrieving the contrib repository.  I don't have a good
reason to go either way, but my development approach for refpolicy
leverage's the fact it's a GIT clone (less lines of change in a patch
I maintain on top of buildroot providing selectable GIT repo for this
package).  I could propose this patch if there is interest and that
would clear up which way to go....  we found it was hard to do
refpolicy development purely via patches.


Matt

[Buildroot] [PATCH 1/1] refpolicy: bump to 2.20170805

[Arnout Vandecappelle]

On 06-11-17 14:51, Matthew Weber wrote:
> Adam,
> 
> On Sun, Nov 5, 2017 at 11:34 AM, Adam Duskett <aduskett@gmail.com> wrote:
>>
>> Hello;
>>
>> On Fri, Nov 3, 2017 at 1:56 PM, Thomas Petazzoni
>> <thomas.petazzoni@free-electrons.com> wrote:
>>> Hello,
>>>
>>> On Fri,  3 Nov 2017 08:58:56 -0400, Adam Duskett wrote:
>>>
>>>> +ifeq ($(BR2_PACKAGE_PYTHON3),y)
>>>> +REFPOLICY_DEPENDENCIES += host-python3
>>>> +REFPOLICY_PYTHON="$(HOST_DIR)/bin/python(PYTHON3_VERSION_MAJOR)"
>>>> +else
>>>> +REFPOLICY_DEPENDENCIES += host-python
>>>> +REFPOLICY_PYTHON="$(HOST_DIR)/bin/python(PYTHON_VERSION_MAJOR)"
>>>> +endif
>>>
>>> I'm not sure why this is needed. Indeed, if BR2_PACKAGE_PYTHON3=y,
>>> $(HOST_DIR)/bin/python points to python3, and if
>>> BR2_PACKAGE_PYTHON=y, $(HOST_DIR)/bin/python points to python2.
>>>
>> Huh, didn't know that.  I will change that asap!
>>
>>> Therefore, you should be able to just use $(HOST_DIR)/bin/python.
>>>
>>> Could you try that instead?
>>>
>>> Also, please answer Matt's concerns before sending a v2. Thanks!
>>>
>> We talked on IRC. :)  The contrib module is included in the tarball as
>> Arnout has said.
>>
>>
> 
> Sorry took a bit to get back to this email.  I've figured out what
> happened.  Nothing has changed in how refpolicy and refpolicy-contrib
> are released.  This patch series changes the package to use a released
> archive vs the git repo.  In the upstream git repo it still uses the
> submodule for retrieving the contrib repository.  I don't have a good
> reason to go either way, but my development approach for refpolicy
> leverage's the fact it's a GIT clone (less lines of change in a patch
> I maintain on top of buildroot providing selectable GIT repo for this
> package).  I could propose this patch if there is interest and that
> would clear up which way to go....  we found it was hard to do
> refpolicy development purely via patches.

 In that case I think the way to go is to make a custom clone of refpolicy
(including submodules) and using OVERRIDE_SRCDIR to build it. I usually have the
following in my external:

OVERRIDE_BASE_DIR := $(dir $(lastword $(MAKEFILE_LIST)))
AUTO_OVERRIDES = $(notdir $(wildcard $(OVERRIDE_BASE_DIR)src/*))
do_override = $(1)_OVERRIDE_SRCDIR = $(OVERRIDE_BASE_DIR)src/$(2)
$(foreach override,$(AUTO_OVERRIDES),\
        $(eval $(call do_override,$(call UPPERCASE,$(override)),$(override))))


and then I put anything that needs a custom version as a git submodule under
src/. I usually also put custom packages there, i.e. I don't use any versioning
for them other than git itself.

 Regards,
 Arnout

-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF

[Buildroot] [PATCH 1/1] refpolicy: bump to 2.20170805

[Matthew Weber]

Arnout,

On Mon, Nov 6, 2017 at 4:11 PM, Arnout Vandecappelle <arnout@mind.be> wrote:
>
>
> On 06-11-17 14:51, Matthew Weber wrote:
>> Adam,

>>> We talked on IRC. :)  The contrib module is included in the tarball as
>>> Arnout has said.
>>>
>>>
>>
>> Sorry took a bit to get back to this email.  I've figured out what
>> happened.  Nothing has changed in how refpolicy and refpolicy-contrib
>> are released.  This patch series changes the package to use a released
>> archive vs the git repo.  In the upstream git repo it still uses the
>> submodule for retrieving the contrib repository.  I don't have a good
>> reason to go either way, but my development approach for refpolicy
>> leverage's the fact it's a GIT clone (less lines of change in a patch
>> I maintain on top of buildroot providing selectable GIT repo for this
>> package).  I could propose this patch if there is interest and that
>> would clear up which way to go....  we found it was hard to do
>> refpolicy development purely via patches.
>
>  In that case I think the way to go is to make a custom clone of refpolicy
> (including submodules) and using OVERRIDE_SRCDIR to build it. I usually have the
> following in my external:
>
> OVERRIDE_BASE_DIR := $(dir $(lastword $(MAKEFILE_LIST)))
> AUTO_OVERRIDES = $(notdir $(wildcard $(OVERRIDE_BASE_DIR)src/*))
> do_override = $(1)_OVERRIDE_SRCDIR = $(OVERRIDE_BASE_DIR)src/$(2)
> $(foreach override,$(AUTO_OVERRIDES),\
>         $(eval $(call do_override,$(call UPPERCASE,$(override)),$(override))))
>
>
> and then I put anything that needs a custom version as a git submodule under
> src/. I usually also put custom packages there, i.e. I don't use any versioning
> for them other than git itself.
>

Yep, I think we're headed the path of a out-of-tree build of refpolicy
using the SDK instead.  So no issue on my end.  Thanks for the update
Adam!

Matt