[Xen-devel] [PATCH for-4.13 0/2] xen: Build fixes related to Python3

[Xen-devel] [PATCH for-4.13 0/2] xen: Build fixes related to Python3

[Andrew Cooper]

Patch 2 is a fix for a problem reported on IRC.  It is a very-nice-to-have
considering our attempt to make Xen 4.13 Py3-clean.

While testing patch 2, it became apparent that XSM/Flask isn't Py3-clean, and
this is a blocker.  It is addressed in patch 1.

Andrew Cooper (2):
  xen/flask: Fix Python 3 problems with gen-policy.py
  xen/build: Automatically locate a suitable python interpreter

 xen/Makefile                 |  2 ++
 xen/xsm/flask/Makefile       |  6 ++----
 xen/xsm/flask/flask-policy.S | 20 ++++++++++++++++++++
 xen/xsm/flask/gen-policy.py  | 23 -----------------------
 4 files changed, 24 insertions(+), 27 deletions(-)
 create mode 100644 xen/xsm/flask/flask-policy.S
 delete mode 100644 xen/xsm/flask/gen-policy.py

-- 
2.11.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH 1/2] xen/flask: Fix Python 3 problems with gen-policy.py

[Andrew Cooper]

The script is Python 2 specific, and fails with string/binary issues with
Python 3:

  Traceback (most recent call last):
    File "gen-policy.py", line 14, in <module>
      for char in sys.stdin.read():
    File "/usr/lib/python3.5/codecs.py", line 321, in decode
      (result, consumed) = self._buffer_decode(data, self.errors, final)
  UnicodeDecodeError: 'utf-8' codec can't decode byte 0x8c in position 0: invalid start byte

Fixing the script to be compatible isn't hard, but using python here is
wasteful.  Drop the script entirely, and write a short flask-policy.S instead.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Daniel De Graaf <dgdegra@tycho.nsa.gov>
CC: Juergen Gross <jgross@suse.com>

For 4.13.  This is a blocker to our intent to by Py3-clean in this release

Discovered entirely accidently when testing the following patch.
---
 xen/xsm/flask/Makefile       |  6 ++----
 xen/xsm/flask/flask-policy.S | 20 ++++++++++++++++++++
 xen/xsm/flask/gen-policy.py  | 23 -----------------------
 3 files changed, 22 insertions(+), 27 deletions(-)
 create mode 100644 xen/xsm/flask/flask-policy.S
 delete mode 100644 xen/xsm/flask/gen-policy.py

diff --git a/xen/xsm/flask/Makefile b/xen/xsm/flask/Makefile
index f5ffab1226..7c3f381287 100644
--- a/xen/xsm/flask/Makefile
+++ b/xen/xsm/flask/Makefile
@@ -27,7 +27,8 @@ $(FLASK_H_FILES): $(FLASK_H_DEPEND)
 $(AV_H_FILES): $(AV_H_DEPEND)
 	$(CONFIG_SHELL) policy/mkaccess_vector.sh $(AWK) $(AV_H_DEPEND)
 
-obj-$(CONFIG_XSM_FLASK_POLICY) += policy.o
+obj-bin-$(CONFIG_XSM_FLASK_POLICY) += flask-policy.o
+flask-policy.o: policy.bin
 
 FLASK_BUILD_DIR := $(CURDIR)
 POLICY_SRC := $(FLASK_BUILD_DIR)/xenpolicy-$(XEN_FULLVERSION)
@@ -36,9 +37,6 @@ policy.bin: FORCE
 	$(MAKE) -f $(XEN_ROOT)/tools/flask/policy/Makefile.common -C $(XEN_ROOT)/tools/flask/policy FLASK_BUILD_DIR=$(FLASK_BUILD_DIR)
 	cmp -s $(POLICY_SRC) $@ || cp $(POLICY_SRC) $@
 
-policy.c: policy.bin gen-policy.py
-	$(PYTHON) gen-policy.py < $< > $@
-
 .PHONY: clean
 clean::
 	rm -f $(ALL_H_FILES) *.o $(DEPS_RM) policy.* $(POLICY_SRC)
diff --git a/xen/xsm/flask/flask-policy.S b/xen/xsm/flask/flask-policy.S
new file mode 100644
index 0000000000..d78ce77fd6
--- /dev/null
+++ b/xen/xsm/flask/flask-policy.S
@@ -0,0 +1,20 @@
+        .section .init.rodata, "a", @progbits
+
+/* const unsigned char xsm_flask_init_policy[] __initconst */
+	.align 4
+        .global xsm_flask_init_policy
+xsm_flask_init_policy:
+        .incbin "policy.bin"
+.Lend:
+
+        .type xsm_flask_init_policy, @object
+        .size xsm_flask_init_policy, . - xsm_flask_init_policy
+
+/* const unsigned int __initconst xsm_flask_init_policy_size */
+	.align 4
+        .global xsm_flask_init_policy_size
+xsm_flask_init_policy_size:
+        .long .Lend - xsm_flask_init_policy
+
+        .type xsm_flask_init_policy_size, @object
+        .size xsm_flask_init_policy_size, . - xsm_flask_init_policy_size
diff --git a/xen/xsm/flask/gen-policy.py b/xen/xsm/flask/gen-policy.py
deleted file mode 100644
index c7501e4614..0000000000
--- a/xen/xsm/flask/gen-policy.py
+++ /dev/null
@@ -1,23 +0,0 @@
-#!/usr/bin/env python
-import sys
-
-policy_size = 0
-
-sys.stdout.write("""
-/* This file is autogenerated by gen_policy.py */
-#include <xen/init.h>
-#include <xsm/xsm.h>
-
-const unsigned char xsm_flask_init_policy[] __initconst = {
-""")
-
-for char in sys.stdin.read():
-    sys.stdout.write(" 0x%02x," % ord(char))
-    policy_size = policy_size + 1
-    if policy_size % 13 == 0:
-        sys.stdout.write("\n")
-
-sys.stdout.write("""
-};
-const unsigned int __initconst xsm_flask_init_policy_size = %d;
-""" % policy_size)
-- 
2.11.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH 2/2] xen/build: Automatically locate a suitable python interpreter

[Andrew Cooper]

Needing to pass PYTHON=python3 into hypervisor builds is irritating and
unnecessary.  Locate a suitable interpreter automatically, defaulting to Py3
if it is available.

Reported-by: Steven Haigh <netwiz@crc.id.au>
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: George Dunlap <George.Dunlap@eu.citrix.com>
CC: Ian Jackson <ian.jackson@citrix.com>
CC: Jan Beulich <JBeulich@suse.com>
CC: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
CC: Stefano Stabellini <sstabellini@kernel.org>
CC: Wei Liu <wl@xen.org>
CC: Julien Grall <julien@xen.org>
CC: Steven Haigh <netwiz@crc.id.au>
CC: Juergen Gross <jgross@suse.com>

For 4.13.  This is a very-nice-to-have WRT our Py3-clean intention.
---
 xen/Makefile | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/xen/Makefile b/xen/Makefile
index 99701e3165..b936d1812b 100644
--- a/xen/Makefile
+++ b/xen/Makefile
@@ -12,6 +12,8 @@ export XEN_BUILD_DATE	?= $(shell LC_ALL=C date)
 export XEN_BUILD_TIME	?= $(shell LC_ALL=C date +%T)
 export XEN_BUILD_HOST	?= $(shell hostname)
 export XEN_CONFIG_EXPERT ?= n
+# Best effort attempt to find a python interpreter
+export PYTHON		?= $(word 1,$(shell which python3 python python2))
 
 export BASEDIR := $(CURDIR)
 export XEN_ROOT := $(BASEDIR)/..
-- 
2.11.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel