[PATCH for-next v2 v2 0/5] tools: Python 3 compatibility

[PATCH for-next v2 v2 0/5] tools: Python 3 compatibility

[Wei Liu]

This series makes tools work with Python 2 and 3.

No more RFC because I have tested it on my testbox.

You need Andrew's "tools/xen-foreign: Update python scripts to be                                                                                   
Py3 compatible" as well.

Wei.

Wei Liu (5):
  build/m4: make python_devel.m4 work with both python 2 and 3
  libxl: make python scripts work with python 2.6 and up
  pygrub: convert python scripts to work with 2.6 and up
  pygrub: make it build with python 3
  Update python requirement

 README                             | 10 +---
 m4/python_devel.m4                 | 23 ++++-----
 tools/configure                    | 72 +++++-----------------------
 tools/configure.ac                 |  2 +-
 tools/libxl/gentest.py             |  2 +-
 tools/libxl/gentypes.py            | 10 ++--
 tools/libxl/idl.py                 | 13 +++---
 tools/pygrub/src/ExtLinuxConf.py   | 15 +++---
 tools/pygrub/src/GrubConf.py       | 36 +++++++-------
 tools/pygrub/src/LiloConf.py       | 15 +++---
 tools/pygrub/src/fsimage/fsimage.c | 96 ++++++++++++++++++++++++++++++++++----
 11 files changed, 157 insertions(+), 137 deletions(-)

-- 
2.11.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH for-next v2 v2 1/5] build/m4: make python_devel.m4 work with both python 2 and 3

[Wei Liu]

Do the following:

1. Change the form of "print".
2. Use AC_CHECK_FUNC to avoid the need to generate library name.
3. Remove unused stuff.

Signed-off-by: Wei Liu <wei.liu2@citrix.com>
---
I doubt the non-python-config branch works, because the paths generated
seem rather off. It definitely doesn't work on my machine, but I
don't know how other systems could possibly be configured before the
existence of python-config.
---
 m4/python_devel.m4 | 23 +++++++-------------
 tools/configure    | 64 +++++++-----------------------------------------------
 2 files changed, 16 insertions(+), 71 deletions(-)

diff --git a/m4/python_devel.m4 b/m4/python_devel.m4
index 05ea4ef7e2..f9cb23aee1 100644
--- a/m4/python_devel.m4
+++ b/m4/python_devel.m4
@@ -1,38 +1,31 @@
 AC_DEFUN([AX_CHECK_PYTHON_DEVEL], [
 ac_previous_cppflags=$CPPFLAGS
 ac_previous_ldflags=$LDFLAGS
-ac_python_version=`$PYTHON -c 'import distutils.sysconfig; \
-    print distutils.sysconfig.get_config_var("VERSION")'`
 AC_PATH_PROG([pyconfig], [$PYTHON-config], [no])
 AS_IF([test x"$pyconfig" = x"no"], [
     dnl For those that don't have python-config
     CPPFLAGS="$CFLAGS `$PYTHON -c 'import distutils.sysconfig; \
         print "-I" + distutils.sysconfig.get_config_var("INCLUDEPY")'`"
     CPPFLAGS="$CPPFLAGS `$PYTHON -c 'import distutils.sysconfig; \
-        print distutils.sysconfig.get_config_var("CFLAGS")'`"
-    PYTHON_LIBS="$LDFLAGS `$PYTHON -c 'import distutils.sysconfig; \
-        print distutils.sysconfig.get_config_var("LIBS")'`"
-    PYTHON_LIBS="$LDFLAGS `$PYTHON -c 'import distutils.sysconfig; \
-        print distutils.sysconfig.get_config_var("SYSLIBS")'`"
+        print(distutils.sysconfig.get_config_var("CFLAGS"))'`"
     LDFLAGS="$LDFLAGS `$PYTHON -c 'import distutils.sysconfig; \
-        print "-L" + distutils.sysconfig.get_python_lib(plat_specific=1,\
-        standard_lib=1) + "/config"'`"
+        print("-L" + distutils.sysconfig.get_python_lib(plat_specific=1,\
+        standard_lib=1) + "/config")'`"
     LDFLAGS="$LDFLAGS `$PYTHON -c 'import distutils.sysconfig; \
-        print distutils.sysconfig.get_config_var("LINKFORSHARED")'`"
+        print(distutils.sysconfig.get_config_var("LINKFORSHARED"))'`"
     LDFLAGS="$LDFLAGS `$PYTHON -c 'import distutils.sysconfig; \
-        print distutils.sysconfig.get_config_var("LDFLAGS")'`"
+        print(distutils.sysconfig.get_config_var("LDFLAGS"))'`"
 ], [
     dnl If python-config is found use it
     CPPFLAGS="$CFLAGS `$PYTHON-config --cflags`"
     LDFLAGS="$LDFLAGS `$PYTHON-config --ldflags`"
-    PYTHON_LIBS="$LIBS `$PYTHON-config --libs`"
 ])
 
 AC_CHECK_HEADER([Python.h], [],
     [AC_MSG_ERROR([Unable to find Python development headers])],)
-AC_CHECK_LIB(python$ac_python_version, PyArg_ParseTuple, [],
-    [AC_MSG_ERROR([Unable to find a suitable python development library])],
-    [$PYTHON_LIBS])
+AC_CHECK_FUNC([PyArg_ParseTuple], [],
+    [AC_MSG_ERROR([Unable to find a suitable python development library])]) 
+
 CPPFLAGS=$ac_previous_cppflags
 LDFLAGS=$ac_previous_ldflags
 ])
diff --git a/tools/configure b/tools/configure
index acc857510e..632ccce445 100755
--- a/tools/configure
+++ b/tools/configure
@@ -7418,8 +7418,6 @@ if test "$cross_compiling" != yes; then :
 
 ac_previous_cppflags=$CPPFLAGS
 ac_previous_ldflags=$LDFLAGS
-ac_python_version=`$PYTHON -c 'import distutils.sysconfig; \
-    print distutils.sysconfig.get_config_var("VERSION")'`
 # Extract the first word of "$PYTHON-config", so it can be a program name with args.
 set dummy $PYTHON-config; ac_word=$2
 { $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5
@@ -7466,24 +7464,19 @@ if test x"$pyconfig" = x"no"; then :
         CPPFLAGS="$CFLAGS `$PYTHON -c 'import distutils.sysconfig; \
         print "-I" + distutils.sysconfig.get_config_var("INCLUDEPY")'`"
     CPPFLAGS="$CPPFLAGS `$PYTHON -c 'import distutils.sysconfig; \
-        print distutils.sysconfig.get_config_var("CFLAGS")'`"
-    PYTHON_LIBS="$LDFLAGS `$PYTHON -c 'import distutils.sysconfig; \
-        print distutils.sysconfig.get_config_var("LIBS")'`"
-    PYTHON_LIBS="$LDFLAGS `$PYTHON -c 'import distutils.sysconfig; \
-        print distutils.sysconfig.get_config_var("SYSLIBS")'`"
+        print(distutils.sysconfig.get_config_var("CFLAGS"))'`"
     LDFLAGS="$LDFLAGS `$PYTHON -c 'import distutils.sysconfig; \
-        print "-L" + distutils.sysconfig.get_python_lib(plat_specific=1,\
-        standard_lib=1) + "/config"'`"
+        print("-L" + distutils.sysconfig.get_python_lib(plat_specific=1,\
+        standard_lib=1) + "/config")'`"
     LDFLAGS="$LDFLAGS `$PYTHON -c 'import distutils.sysconfig; \
-        print distutils.sysconfig.get_config_var("LINKFORSHARED")'`"
+        print(distutils.sysconfig.get_config_var("LINKFORSHARED"))'`"
     LDFLAGS="$LDFLAGS `$PYTHON -c 'import distutils.sysconfig; \
-        print distutils.sysconfig.get_config_var("LDFLAGS")'`"
+        print(distutils.sysconfig.get_config_var("LDFLAGS"))'`"
 
 else
 
         CPPFLAGS="$CFLAGS `$PYTHON-config --cflags`"
     LDFLAGS="$LDFLAGS `$PYTHON-config --ldflags`"
-    PYTHON_LIBS="$LIBS `$PYTHON-config --libs`"
 
 fi
 
@@ -7495,55 +7488,14 @@ else
 fi
 
 
-as_ac_Lib=`$as_echo "ac_cv_lib_python$ac_python_version''_PyArg_ParseTuple" | $as_tr_sh`
-{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for PyArg_ParseTuple in -lpython$ac_python_version" >&5
-$as_echo_n "checking for PyArg_ParseTuple in -lpython$ac_python_version... " >&6; }
-if eval \${$as_ac_Lib+:} false; then :
-  $as_echo_n "(cached) " >&6
-else
-  ac_check_lib_save_LIBS=$LIBS
-LIBS="-lpython$ac_python_version $PYTHON_LIBS $LIBS"
-cat confdefs.h - <<_ACEOF >conftest.$ac_ext
-/* end confdefs.h.  */
-
-/* Override any GCC internal prototype to avoid an error.
-   Use char because int might match the return type of a GCC
-   builtin and then its argument prototype would still apply.  */
-#ifdef __cplusplus
-extern "C"
-#endif
-char PyArg_ParseTuple ();
-int
-main ()
-{
-return PyArg_ParseTuple ();
-  ;
-  return 0;
-}
-_ACEOF
-if ac_fn_c_try_link "$LINENO"; then :
-  eval "$as_ac_Lib=yes"
-else
-  eval "$as_ac_Lib=no"
-fi
-rm -f core conftest.err conftest.$ac_objext \
-    conftest$ac_exeext conftest.$ac_ext
-LIBS=$ac_check_lib_save_LIBS
-fi
-eval ac_res=\$$as_ac_Lib
-	       { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_res" >&5
-$as_echo "$ac_res" >&6; }
-if eval test \"x\$"$as_ac_Lib"\" = x"yes"; then :
-  cat >>confdefs.h <<_ACEOF
-#define `$as_echo "HAVE_LIBpython$ac_python_version" | $as_tr_cpp` 1
-_ACEOF
-
-  LIBS="-lpython$ac_python_version $LIBS"
+ac_fn_c_check_func "$LINENO" "PyArg_ParseTuple" "ac_cv_func_PyArg_ParseTuple"
+if test "x$ac_cv_func_PyArg_ParseTuple" = xyes; then :
 
 else
   as_fn_error $? "Unable to find a suitable python development library" "$LINENO" 5
 fi
 
+
 CPPFLAGS=$ac_previous_cppflags
 LDFLAGS=$ac_previous_ldflags
 
-- 
2.11.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH for-next v2 v2 2/5] libxl: make python scripts work with python 2.6 and up

[Wei Liu]

Go through the transformations suggested by 2to3 and pick the
necessary ones.

Use sys.stderr.write to avoid importing from __future__.

Signed-off-by: Wei Liu <wei.liu2@citrix.com>
---
 tools/libxl/gentest.py  |  2 +-
 tools/libxl/gentypes.py | 10 +++++-----
 tools/libxl/idl.py      | 13 ++++++-------
 3 files changed, 12 insertions(+), 13 deletions(-)

diff --git a/tools/libxl/gentest.py b/tools/libxl/gentest.py
index 989959fc68..81e13b437c 100644
--- a/tools/libxl/gentest.py
+++ b/tools/libxl/gentest.py
@@ -86,7 +86,7 @@ def gen_rand_init(ty, v, indent = "    ", parent = None):
 
 if __name__ == '__main__':
     if len(sys.argv) < 3:
-        print >>sys.stderr, "Usage: gentest.py <idl> <implementation>"
+        sys.stderr.write("Usage: gentest.py <idl> <implementation>\n")
         sys.exit(1)
 
     random.seed(os.getenv('LIBXL_TESTIDL_SEED'))
diff --git a/tools/libxl/gentypes.py b/tools/libxl/gentypes.py
index 88e5c5f30e..656c157c01 100644
--- a/tools/libxl/gentypes.py
+++ b/tools/libxl/gentypes.py
@@ -576,14 +576,14 @@ def libxl_C_enum_from_string(ty, str, e, indent = "    "):
 
 if __name__ == '__main__':
     if len(sys.argv) != 6:
-        print >>sys.stderr, "Usage: gentypes.py <idl> <header> <header-private> <header-json> <implementation>"
+        sys.stderr.write("Usage: gentypes.py <idl> <header> <header-private> <header-json> <implementation>\n")
         sys.exit(1)
 
     (_, idlname, header, header_private, header_json, impl) = sys.argv
 
     (builtins,types) = idl.parse(idlname)
 
-    print "outputting libxl type definitions to %s" % header
+    print("outputting libxl type definitions to %s" % header)
 
     f = open(header, "w")
 
@@ -633,7 +633,7 @@ if __name__ == '__main__':
     f.write("""#endif /* %s */\n""" % (header_define))
     f.close()
 
-    print "outputting libxl JSON definitions to %s" % header_json
+    print("outputting libxl JSON definitions to %s" % header_json)
 
     f = open(header_json, "w")
 
@@ -657,7 +657,7 @@ if __name__ == '__main__':
     f.write("""#endif /* %s */\n""" % header_json_define)
     f.close()
 
-    print "outputting libxl type internal definitions to %s" % header_private
+    print("outputting libxl type internal definitions to %s" % header_private)
 
     f = open(header_private, "w")
 
@@ -683,7 +683,7 @@ if __name__ == '__main__':
     f.write("""#endif /* %s */\n""" % header_json_define)
     f.close()
 
-    print "outputting libxl type implementations to %s" % impl
+    print("outputting libxl type implementations to %s" % impl)
 
     f = open(impl, "w")
     f.write("""
diff --git a/tools/libxl/idl.py b/tools/libxl/idl.py
index 2a7f3c44fe..b5bfc66b50 100644
--- a/tools/libxl/idl.py
+++ b/tools/libxl/idl.py
@@ -11,7 +11,7 @@ DIR_BOTH = 3
 _default_namespace = ""
 def namespace(s):
     if type(s) != str:
-        raise TypeError, "Require a string for the default namespace."
+        raise TypeError("Require a string for the default namespace.")
     global _default_namespace
     _default_namespace = s
 
@@ -346,7 +346,7 @@ class OrderedDict(dict):
         return [(x,self[x]) for x in self.__ordered]
 
 def parse(f):
-    print >>sys.stderr, "Parsing %s" % f
+    sys.stderr.write("Parsing %s\n" % f)
 
     globs = {}
     locs = OrderedDict()
@@ -362,11 +362,10 @@ def parse(f):
             globs[n] = t
 
     try:
-        execfile(f, globs, locs)
-    except SyntaxError,e:
-        raise SyntaxError, \
-              "Errors were found at line %d while processing %s:\n\t%s"\
-              %(e.lineno,f,e.text)
+        exec(compile(open(f).read(), f, 'exec'), globs, locs)
+    except SyntaxError as e:
+        raise SyntaxError("Errors were found at line %d while processing %s:\n\t%s"\
+              %(e.lineno,f,e.text))
 
     types = [t for t in locs.ordered_values() if isinstance(t,Type)]
 
-- 
2.11.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH for-next v2 v2 3/5] pygrub: convert python scripts to work with 2.6 and up

[Wei Liu]

Run 2to3 and pick the sensible suggestions.

Signed-off-by: Wei Liu <wei.liu2@citrix.com>
---
 tools/pygrub/src/ExtLinuxConf.py | 15 ++++++++-------
 tools/pygrub/src/GrubConf.py     | 36 ++++++++++++++++++------------------
 tools/pygrub/src/LiloConf.py     | 15 ++++++++-------
 3 files changed, 34 insertions(+), 32 deletions(-)

diff --git a/tools/pygrub/src/ExtLinuxConf.py b/tools/pygrub/src/ExtLinuxConf.py
index d1789bf020..b84bbf8454 100644
--- a/tools/pygrub/src/ExtLinuxConf.py
+++ b/tools/pygrub/src/ExtLinuxConf.py
@@ -32,7 +32,8 @@ class ExtLinuxImage(object):
         self.lines = []
         self.path = path
         self.root = ""
-        map(self.set_from_line, lines)
+        for line in lines:
+            self.set_from_line(line)
 
     def set_from_line(self, line, replace = None):
         (com, arg) = GrubConf.grub_exact_split(line, 2)
@@ -67,7 +68,7 @@ class ExtLinuxImage(object):
                         setattr(self, "initrd", a.replace("initrd=", ""))
                         arg = arg.replace(a, "")
 
-        if com is not None and self.commands.has_key(com):
+        if com is not None and com in self.commands:
             if self.commands[com] is not None:
                 setattr(self, self.commands[com], re.sub('^"(.+)"$', r"\1", arg.strip()))
             else:
@@ -136,7 +137,7 @@ class ExtLinuxConfigFile(object):
     def parse(self, buf = None):
         if buf is None:
             if self.filename is None:
-                raise ValueError, "No config file defined to parse!"
+                raise ValueError("No config file defined to parse!")
 
             f = open(self.filename, 'r')
             lines = f.readlines()
@@ -167,7 +168,7 @@ class ExtLinuxConfigFile(object):
 
             (com, arg) = GrubConf.grub_exact_split(l, 2)
             com = com.lower()
-            if self.commands.has_key(com):
+            if com in self.commands:
                 if self.commands[com] is not None:
                     setattr(self, self.commands[com], arg.strip())
                 else:
@@ -207,8 +208,8 @@ class ExtLinuxConfigFile(object):
         
 if __name__ == "__main__":
     if len(sys.argv) < 2:
-        raise RuntimeError, "Need a configuration file to read"
+        raise RuntimeError("Need a configuration file to read")
     g = ExtLinuxConfigFile(sys.argv[1])
     for i in g.images:
-        print i
-    print g.default
+        print(i)
+    print(g.default)
diff --git a/tools/pygrub/src/GrubConf.py b/tools/pygrub/src/GrubConf.py
index dc810d55cb..97913f3993 100644
--- a/tools/pygrub/src/GrubConf.py
+++ b/tools/pygrub/src/GrubConf.py
@@ -44,7 +44,7 @@ def get_path(s):
         return (None, s)
     idx = s.find(')')
     if idx == -1:
-        raise ValueError, "Unable to find matching ')'"
+        raise ValueError("Unable to find matching ')'")
     d = s[:idx]
     return (GrubDiskPart(d), s[idx + 1:])
 
@@ -100,7 +100,7 @@ class _GrubImage(object):
                 "  initrd: %s\n" %(self.title, self.root, self.kernel,
                                    self.args, self.initrd))
     def _parse(self, lines):
-        map(self.set_from_line, lines)
+        list(map(self.set_from_line, lines))
 
     def reset(self, lines):
         self._root = self._initrd = self._kernel = self._args = None
@@ -141,7 +141,7 @@ class GrubImage(_GrubImage):
     def set_from_line(self, line, replace = None):
         (com, arg) = grub_exact_split(line, 2)
 
-        if self.commands.has_key(com):
+        if com in self.commands:
             if self.commands[com] is not None:
                 setattr(self, self.commands[com], arg.strip())
             else:
@@ -177,7 +177,7 @@ class _GrubConfigFile(object):
             self.parse()
 
     def parse(self, buf = None):
-        raise RuntimeError, "unimplemented parse function"   
+        raise RuntimeError("unimplemented parse function")
 
     def hasPasswordAccess(self):
         return self.passwordAccess
@@ -201,7 +201,7 @@ class _GrubConfigFile(object):
                 import crypt
                 if crypt.crypt(password, pwd[1]) == pwd[1]:
                     return True
-            except Exception, e:
+            except Exception as e:
                 self.passExc = "Can't verify password: %s" % str(e)
                 return False
 
@@ -213,7 +213,7 @@ class _GrubConfigFile(object):
 
     def set(self, line):
         (com, arg) = grub_exact_split(line, 2)
-        if self.commands.has_key(com):
+        if com in self.commands:
             if self.commands[com] is not None:
                 setattr(self, self.commands[com], arg.strip())
             else:
@@ -233,7 +233,7 @@ class _GrubConfigFile(object):
             self._default = val
 
         if self._default < 0:
-            raise ValueError, "default must be positive number"
+            raise ValueError("default must be positive number")
     default = property(_get_default, _set_default)
 
     def set_splash(self, val):
@@ -265,7 +265,7 @@ class GrubConfigFile(_GrubConfigFile):
     def parse(self, buf = None):
         if buf is None:
             if self.filename is None:
-                raise ValueError, "No config file defined to parse!"
+                raise ValueError("No config file defined to parse!")
 
             f = open(self.filename, 'r')
             lines = f.readlines()
@@ -296,7 +296,7 @@ class GrubConfigFile(_GrubConfigFile):
                 continue
 
             (com, arg) = grub_exact_split(l, 2)
-            if self.commands.has_key(com):
+            if com in self.commands:
                 if self.commands[com] is not None:
                     setattr(self, self.commands[com], arg.strip())
                 else:
@@ -328,7 +328,7 @@ class Grub2Image(_GrubImage):
         if com == "set":
             (com,arg) = grub2_handle_set(arg)
             
-        if self.commands.has_key(com):
+        if com in self.commands:
             if self.commands[com] is not None:
                 setattr(self, self.commands[com], arg.strip())
             else:
@@ -364,7 +364,7 @@ class Grub2ConfigFile(_GrubConfigFile):
     def parse(self, buf = None):
         if buf is None:
             if self.filename is None:
-                raise ValueError, "No config file defined to parse!"
+                raise ValueError("No config file defined to parse!")
 
             f = open(self.filename, 'r')
             lines = f.readlines()
@@ -398,7 +398,7 @@ class Grub2ConfigFile(_GrubConfigFile):
             title_match = re.match('^menuentry ["\'](.*?)["\'] (.*){', l)
             if title_match:
                 if img is not None:
-                    raise RuntimeError, "syntax error: cannot nest menuentry (%d %s)" % (len(img),img)
+                    raise RuntimeError("syntax error: cannot nest menuentry (%d %s)" % (len(img),img))
                 img = []
                 title = title_match.group(1)
                 continue
@@ -413,7 +413,7 @@ class Grub2ConfigFile(_GrubConfigFile):
                         menu_level -= 1
                         continue
                     else:
-                        raise RuntimeError, "syntax error: closing brace without menuentry"
+                        raise RuntimeError("syntax error: closing brace without menuentry")
 
                 self.add_image(Grub2Image(title, img))
                 img = None
@@ -428,7 +428,7 @@ class Grub2ConfigFile(_GrubConfigFile):
             if com == "set":
                 (com,arg) = grub2_handle_set(arg)
                 
-            if self.commands.has_key(com):
+            if com in self.commands:
                 if self.commands[com] is not None:
                     arg_strip = arg.strip()
                     if arg_strip == "${saved_entry}" or arg_strip == "${next_entry}":
@@ -443,7 +443,7 @@ class Grub2ConfigFile(_GrubConfigFile):
                 logging.warning("Unknown directive %s" %(com,))
             
         if img is not None:
-            raise RuntimeError, "syntax error: end of file with open menuentry(%d %s)" % (len(img),img)
+            raise RuntimeError("syntax error: end of file with open menuentry(%d %s)" % (len(img),img))
 
         if self.hasPassword():
             self.setPasswordAccess(False)
@@ -462,12 +462,12 @@ class Grub2ConfigFile(_GrubConfigFile):
         
 if __name__ == "__main__":
     if len(sys.argv) < 3:
-        raise RuntimeError, "Need a grub version (\"grub\" or \"grub2\") and a grub.conf or grub.cfg to read"
+        raise RuntimeError("Need a grub version (\"grub\" or \"grub2\") and a grub.conf or grub.cfg to read")
     if sys.argv[1] == "grub":
         g = GrubConfigFile(sys.argv[2])
     elif sys.argv[1] == "grub2":
         g = Grub2ConfigFile(sys.argv[2])
     else:
-        raise RuntimeError, "Unknown config type %s" % sys.argv[1]
+        raise RuntimeError("Unknown config type %s" % sys.argv[1])
     for i in g.images:
-        print i #, i.title, i.root, i.kernel, i.args, i.initrd
+        print(i) #, i.title, i.root, i.kernel, i.args, i.initrd
diff --git a/tools/pygrub/src/LiloConf.py b/tools/pygrub/src/LiloConf.py
index 2cb649f115..f4d26737f5 100644
--- a/tools/pygrub/src/LiloConf.py
+++ b/tools/pygrub/src/LiloConf.py
@@ -24,12 +24,13 @@ class LiloImage(object):
         self.lines = []
         self.path = path
         self.root = ""
-        map(self.set_from_line, lines)
+        for line in lines:
+            self.set_from_line(line)
 
     def set_from_line(self, line, replace = None):
         (com, arg) = GrubConf.grub_exact_split(line, 2)
 
-        if self.commands.has_key(com):
+        if com in self.commands:
             if self.commands[com] is not None:
                 setattr(self, self.commands[com], re.sub('^"(.+)"$', r"\1", arg.strip()))
             else:
@@ -97,7 +98,7 @@ class LiloConfigFile(object):
     def parse(self, buf = None):
         if buf is None:
             if self.filename is None:
-                raise ValueError, "No config file defined to parse!"
+                raise ValueError("No config file defined to parse!")
 
             f = open(self.filename, 'r')
             lines = f.readlines()
@@ -127,7 +128,7 @@ class LiloConfigFile(object):
                 continue
 
             (com, arg) = GrubConf.grub_exact_split(l, 2)
-            if self.commands.has_key(com):
+            if com in self.commands:
                 if self.commands[com] is not None:
                     setattr(self, self.commands[com], arg.strip())
                 else:
@@ -170,8 +171,8 @@ class LiloConfigFile(object):
 
 if __name__ == "__main__":
     if len(sys.argv) < 2:
-        raise RuntimeError, "Need a lilo.conf to read"
+        raise RuntimeError("Need a lilo.conf to read")
     g = LiloConfigFile(sys.argv[1])
     for i in g.images:
-        print i #, i.title, i.root, i.kernel, i.args, i.initrd
-    print g.default
+        print(i) #, i.title, i.root, i.kernel, i.args, i.initrd
+    print(g.default)
-- 
2.11.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH for-next v2 v2 4/5] pygrub: make it build with python 3

[Wei Liu]

With the help of two porting guides and cpython source code:

1. Use PyBytes to replace PyString counterparts.
2. Use PyVarObject_HEAD_INIT and provide compatibility for 2.5 and
   earlier.
3. Remove usage of Py_FindMethod.
4. Use new module initialisation routine.

For #3, Py_FindMethod was removed, yet an alternative wasn't
documented.  The code is the result of reverse-engineering cpython
commit 6116d4a1d1

https://docs.python.org/3/howto/cporting.html
http://python3porting.com/cextensions.html

Signed-off-by: Wei Liu <wei.liu2@citrix.com>
---
v2: use PyBytes.
---
 tools/pygrub/src/fsimage/fsimage.c | 96 ++++++++++++++++++++++++++++++++++----
 1 file changed, 88 insertions(+), 8 deletions(-)

diff --git a/tools/pygrub/src/fsimage/fsimage.c b/tools/pygrub/src/fsimage/fsimage.c
index 743a3fb7b8..7e124f7bb3 100644
--- a/tools/pygrub/src/fsimage/fsimage.c
+++ b/tools/pygrub/src/fsimage/fsimage.c
@@ -26,11 +26,15 @@
 #include <xenfsimage.h>
 #include <stdlib.h>
 
+#if PY_MAJOR_VERSION < 3
 #if (PYTHON_API_VERSION >= 1011)
 #define PY_PAD 0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L
 #else
 #define PY_PAD 0L,0L,0L,0L
 #endif
+#else
+#define PY_PAD 0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L
+#endif
 
 typedef struct fsimage_fs {
 	PyObject_HEAD
@@ -66,12 +70,24 @@ fsimage_file_read(fsimage_file_t *file, PyObject *args, PyObject *kwargs)
 
 	bufsize = size ? size : 4096;
 
-	if ((buffer = PyString_FromStringAndSize(NULL, bufsize)) == NULL)
+	buffer =
+#if PY_MAJOR_VERSION < 3
+		PyString_FromStringAndSize(NULL, bufsize);
+#else
+		PyBytes_FromStringAndSize(NULL, bufsize);
+#endif
+
+	if (buffer == NULL)
 		return (NULL);
  
 	while (1) {
 		int err;
-		void *buf = PyString_AS_STRING(buffer) + bytesread;
+		void *buf =
+#if PY_MAJOR_VERSION < 3
+			PyString_AS_STRING(buffer) + bytesread;
+#else
+			PyBytes_AS_STRING(buffer) + bytesread;
+#endif
 
 		err = fsi_pread_file(file->file, buf, bufsize,
 		    bytesread + offset);
@@ -91,12 +107,20 @@ fsimage_file_read(fsimage_file_t *file, PyObject *args, PyObject *kwargs)
 			if (bufsize == 0)
 				break;
 		} else {
+#if PY_MAJOR_VERSION < 3
 			if (_PyString_Resize(&buffer, bytesread + bufsize) < 0)
+#else
+			if (_PyBytes_Resize(&buffer, bytesread + bufsize) < 0)
+#endif
 				return (NULL);
 		}
 	}
 
+#if PY_MAJOR_VERSION < 3
 	_PyString_Resize(&buffer, bytesread);
+#else
+	_PyBytes_Resize(&buffer, bytesread);
+#endif
 	return (buffer);
 }
 
@@ -113,11 +137,13 @@ static struct PyMethodDef fsimage_file_methods[] = {
 	{ NULL, NULL, 0, NULL }	
 };
 
+#if PY_MAJOR_VERSION < 3
 static PyObject *
 fsimage_file_getattr(fsimage_file_t *file, char *name)
 {
 	return (Py_FindMethod(fsimage_file_methods, (PyObject *)file, name));
 }
+#endif
 
 static void
 fsimage_file_dealloc(fsimage_file_t *file)
@@ -128,16 +154,25 @@ fsimage_file_dealloc(fsimage_file_t *file)
 	PyObject_DEL(file);
 }
 
+/* Compatibility for 2.5 and earlier */
+#ifndef PyVarObject_HEAD_INIT
+#define PyVarObject_HEAD_INIT(type, size) \
+	PyObject_HEAD_INIT(type) size,
+#endif
+
 static char fsimage_file_type__doc__[] = "Filesystem image file";
 PyTypeObject fsimage_file_type = {
-	PyObject_HEAD_INIT(&PyType_Type)
-	0,					/* ob_size */
+	PyVarObject_HEAD_INIT(&PyType_Type, 0)
 	"xenfsimage.file",			/* tp_name */
 	sizeof(fsimage_file_t),			/* tp_size */
 	0,					/* tp_itemsize */
 	(destructor) fsimage_file_dealloc, 	/* tp_dealloc */
 	0,					/* tp_print */
+#if PY_MAJOR_VERSION < 3
 	(getattrfunc) fsimage_file_getattr, 	/* tp_getattr */
+#else
+	0,					/* tp_getattr */
+#endif
 	0,					/* tp_setattr */
 	0,					/* tp_compare */
 	0,					/* tp_repr */
@@ -151,7 +186,16 @@ PyTypeObject fsimage_file_type = {
 	0,					/* tp_setattro */
 	0,					/* tp_as_buffer */
 	Py_TPFLAGS_DEFAULT,			/* tp_flags */
-	fsimage_file_type__doc__,
+	fsimage_file_type__doc__,		/* tp_doc */
+#if PY_MAJOR_VERSION >= 3
+	0,					/* tp_traverse */
+	0,					/* tp_clear */
+	0,					/* tp_richcompare */
+	0,					/* tp_weaklistoffset */
+	0,					/* tp_iter */
+	0,					/* tp_iternext */
+	fsimage_file_methods,			/* tp_methods */
+#endif
 	PY_PAD
 };
 
@@ -215,11 +259,13 @@ static struct PyMethodDef fsimage_fs_methods[] = {
 	{ NULL, NULL, 0, NULL }	
 };
 
+#if PY_MAJOR_VERSION < 3
 static PyObject *
 fsimage_fs_getattr(fsimage_fs_t *fs, char *name)
 {
 	return (Py_FindMethod(fsimage_fs_methods, (PyObject *)fs, name));
 }
+#endif
 
 static void
 fsimage_fs_dealloc (fsimage_fs_t *fs)
@@ -232,14 +278,17 @@ fsimage_fs_dealloc (fsimage_fs_t *fs)
 PyDoc_STRVAR(fsimage_fs_type__doc__, "Filesystem image");
 
 PyTypeObject fsimage_fs_type = {
-	PyObject_HEAD_INIT(&PyType_Type)
-	0,					/* ob_size */
+	PyVarObject_HEAD_INIT(&PyType_Type, 0)
 	"xenfsimage.fs",			/* tp_name */
 	sizeof(fsimage_fs_t),			/* tp_size */
 	0,					/* tp_itemsize */
 	(destructor) fsimage_fs_dealloc, 	/* tp_dealloc */
 	0,					/* tp_print */
+#if PY_MAJOR_VERSION < 3
 	(getattrfunc) fsimage_fs_getattr, 	/* tp_getattr */
+#else
+	0,					/* tp_getattr */
+#endif
 	0,					/* tp_setattr */
 	0,					/* tp_compare */
 	0,					/* tp_repr */
@@ -253,7 +302,16 @@ PyTypeObject fsimage_fs_type = {
 	0,					/* tp_setattro */
 	0,					/* tp_as_buffer */
 	Py_TPFLAGS_DEFAULT,			/* tp_flags */
-	fsimage_fs_type__doc__,
+	fsimage_fs_type__doc__,			/* tp_doc */
+#if PY_MAJOR_VERSION >= 3
+	0,					/* tp_traverse */
+	0,					/* tp_clear */
+	0,					/* tp_richcompare */
+	0,					/* tp_weaklistoffset */
+	0,					/* tp_iter */
+	0,					/* tp_iternext */
+	fsimage_fs_methods,			/* tp_methods */
+#endif
 	PY_PAD
 };
 
@@ -316,8 +374,30 @@ static struct PyMethodDef fsimage_module_methods[] = {
 	{ NULL, NULL, 0, NULL }
 };
 
+#if PY_MAJOR_VERSION >= 3
+static struct PyModuleDef fsimage_module_def = {
+	PyModuleDef_HEAD_INIT,
+	"xenfsimage",		/* m_name */
+	"",			/* m_doc */
+	-1,			/* m_size */
+	fsimage_module_methods,	/* m_methods */
+	NULL,			/* m_reload */
+	NULL,			/* m_traverse */
+	NULL,			/* m_clear */
+	NULL			/* m_free */
+};
+#endif
+
 PyMODINIT_FUNC
+#if PY_MAJOR_VERSION >= 3
+PyInit_xenfsimage(void)
+#else
 initxenfsimage(void)
+#endif
 {
+#if PY_MAJOR_VERSION < 3
 	Py_InitModule("xenfsimage", fsimage_module_methods);
+#else
+	return PyModule_Create(&fsimage_module_def);
+#endif
 }
-- 
2.11.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH for-next v2 v2 5/5] Update python requirement

[Wei Liu]

CentOS 5, which was the reason of the 2.4 restriction, is EOL. CentOS
6 ships 2.6.

Bump the version to 2.6 in README. Now that all scripts are 3
compatible, remove the restriction on python 2 as well.

Update the check in configure.ac.

Signed-off-by: Wei Liu <wei.liu2@citrix.com>
---
 README             | 10 ++--------
 tools/configure    |  8 ++++----
 tools/configure.ac |  2 +-
 3 files changed, 7 insertions(+), 13 deletions(-)

diff --git a/README b/README
index c19409efa2..3049201d91 100644
--- a/README
+++ b/README
@@ -46,7 +46,7 @@ provided by your OS distributor:
         - GCC 4.8 or later
         - GNU Binutils 2.24 or later
     * Development install of zlib (e.g., zlib-dev)
-    * Development install of Python 2, v2.4 or later (e.g., python-dev)
+    * Development install of Python v2.6 or later (e.g., python-dev)
     * Development install of curses (e.g., libncurses-dev)
     * Development install of openssl (e.g., openssl-dev)
     * Development install of x11 (e.g. xorg-x11-dev)
@@ -177,16 +177,10 @@ Python Runtime Libraries
 
 Various tools, such as pygrub, have the following runtime dependencies:
 
-    * Python 2, v2.4 or later.
+    * Python v2.6 or later.
           URL:    http://www.python.org/
           Debian: python
 
-Note that the build system expects `python` to be python2.  If your system
-has `python` pointing to python3 (as in the case of Arch Linux or Anaconda),
-you'll need to specify a path to a python2 binary when running configure:
-
-    PYTHON=/usr/bin/python2 ./configure
-
 Intel(R) Trusted Execution Technology Support
 =============================================
 
diff --git a/tools/configure b/tools/configure
index 632ccce445..e1fa5d6b0f 100755
--- a/tools/configure
+++ b/tools/configure
@@ -7002,15 +7002,15 @@ if test x"${PYTHONPATH}" = x"no"
 then
     as_fn_error $? "Unable to find $PYTHON, please install $PYTHON" "$LINENO" 5
 fi
-{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for python version >= 2.3 " >&5
-$as_echo_n "checking for python version >= 2.3 ... " >&6; }
-`$PYTHON -c 'import sys; sys.exit(eval("sys.version_info < (2, 3)"))'`
+{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for python version >= 2.6 " >&5
+$as_echo_n "checking for python version >= 2.6 ... " >&6; }
+`$PYTHON -c 'import sys; sys.exit(eval("sys.version_info < (2, 6)"))'`
 if test "$?" != "0"
 then
     python_version=`$PYTHON -V 2>&1`
     { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5
 $as_echo "no" >&6; }
-    as_fn_error $? "$python_version is too old, minimum required version is 2.3" "$LINENO" 5
+    as_fn_error $? "$python_version is too old, minimum required version is 2.6" "$LINENO" 5
 else
     { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5
 $as_echo "yes" >&6; }
diff --git a/tools/configure.ac b/tools/configure.ac
index 1499344ce6..c9fd69ddfa 100644
--- a/tools/configure.ac
+++ b/tools/configure.ac
@@ -358,7 +358,7 @@ AS_IF([echo "$PYTHON" | grep -q "^/"], [
 ],[test -z "$PYTHON"], [PYTHON="python"],
 [AC_MSG_ERROR([PYTHON specified, but is not an absolute path])])
 AX_PATH_PROG_OR_FAIL([PYTHONPATH], [$PYTHON])
-AX_CHECK_PYTHON_VERSION([2], [3])
+AX_CHECK_PYTHON_VERSION([2], [6])
 
 AS_IF([test "$cross_compiling" != yes], [
     AX_CHECK_PYTHON_DEVEL()
-- 
2.11.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH for-next v2 v2 0/5] tools: Python 3 compatibility

[Wei Liu]

On Wed, Mar 06, 2019 at 05:52:05PM +0000, Wei Liu wrote:
> This series makes tools work with Python 2 and 3.
> 
> No more RFC because I have tested it on my testbox.

Spoke too soon. The testbox wasn't using the files I installed. I will
retest tomorrow.

Wei.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH for-next v2 v2 2/5] libxl: make python scripts work with python 2.6 and up

[Andrew Cooper]

On 06/03/2019 17:52, Wei Liu wrote:
> @@ -362,11 +362,10 @@ def parse(f):
>              globs[n] = t
>  
>      try:
> -        execfile(f, globs, locs)
> -    except SyntaxError,e:
> -        raise SyntaxError, \
> -              "Errors were found at line %d while processing %s:\n\t%s"\
> -              %(e.lineno,f,e.text)
> +        exec(compile(open(f).read(), f, 'exec'), globs, locs)
> +    except SyntaxError as e:
> +        raise SyntaxError("Errors were found at line %d while processing %s:\n\t%s"\
> +              %(e.lineno,f,e.text))

As you're editing this, the \ can go, and it would be nice to properly
indent the second line.  (Even better if we can get spaces in sensible
places).

Otherwise, Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>

~Andrew

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH for-next v2 v2 3/5] pygrub: convert python scripts to work with 2.6 and up

[Andrew Cooper]

On 06/03/2019 17:52, Wei Liu wrote:
> diff --git a/tools/pygrub/src/GrubConf.py b/tools/pygrub/src/GrubConf.py
> index dc810d55cb..97913f3993 100644
> --- a/tools/pygrub/src/GrubConf.py
> +++ b/tools/pygrub/src/GrubConf.py
> @@ -100,7 +100,7 @@ class _GrubImage(object):
>                  "  initrd: %s\n" %(self.title, self.root, self.kernel,
>                                     self.args, self.initrd))
>      def _parse(self, lines):
> -        map(self.set_from_line, lines)
> +        list(map(self.set_from_line, lines))

Another site which wants a transform into a for loop.

>  
>      def reset(self, lines):
>          self._root = self._initrd = self._kernel = self._args = None
> @@ -462,12 +462,12 @@ class Grub2ConfigFile(_GrubConfigFile):
>          
>  if __name__ == "__main__":
>      if len(sys.argv) < 3:
> -        raise RuntimeError, "Need a grub version (\"grub\" or \"grub2\") and a grub.conf or grub.cfg to read"
> +        raise RuntimeError("Need a grub version (\"grub\" or \"grub2\") and a grub.conf or grub.cfg to read")

As you're editing this anyway, it would be better to switch to

RuntimeError('Need a grub version ("grub" or "grub2") and a grub.conf or grub.cfg to read')


To drop the unnecessary escaping.

Otherwise, Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH for-next v2 v2 4/5] pygrub: make it build with python 3

[Andrew Cooper]

On 06/03/2019 17:52, Wei Liu wrote:
> @@ -128,16 +154,25 @@ fsimage_file_dealloc(fsimage_file_t *file)
>  	PyObject_DEL(file);
>  }
>  
> +/* Compatibility for 2.5 and earlier */
> +#ifndef PyVarObject_HEAD_INIT
> +#define PyVarObject_HEAD_INIT(type, size) \
> +	PyObject_HEAD_INIT(type) size,
> +#endif
> +
>  static char fsimage_file_type__doc__[] = "Filesystem image file";
>  PyTypeObject fsimage_file_type = {
> -	PyObject_HEAD_INIT(&PyType_Type)
> -	0,					/* ob_size */
> +	PyVarObject_HEAD_INIT(&PyType_Type, 0)
>  	"xenfsimage.file",			/* tp_name */
>  	sizeof(fsimage_file_t),			/* tp_size */
>  	0,					/* tp_itemsize */
>  	(destructor) fsimage_file_dealloc, 	/* tp_dealloc */
>  	0,					/* tp_print */
> +#if PY_MAJOR_VERSION < 3
>  	(getattrfunc) fsimage_file_getattr, 	/* tp_getattr */
> +#else
> +	0,					/* tp_getattr */
> +#endif
>  	0,					/* tp_setattr */
>  	0,					/* tp_compare */
>  	0,					/* tp_repr */
> @@ -151,7 +186,16 @@ PyTypeObject fsimage_file_type = {
>  	0,					/* tp_setattro */
>  	0,					/* tp_as_buffer */
>  	Py_TPFLAGS_DEFAULT,			/* tp_flags */
> -	fsimage_file_type__doc__,
> +	fsimage_file_type__doc__,		/* tp_doc */
> +#if PY_MAJOR_VERSION >= 3
> +	0,					/* tp_traverse */
> +	0,					/* tp_clear */
> +	0,					/* tp_richcompare */
> +	0,					/* tp_weaklistoffset */
> +	0,					/* tp_iter */
> +	0,					/* tp_iternext */
> +	fsimage_file_methods,			/* tp_methods */
> +#endif
>  	PY_PAD

PY_PAD is very WTF.  I've got no idea why it is necessary in the first
place.

Either way, most of the fields are zero, so why not use named
initialisation?

PyTypeObject fsimage_file_type = {
    PyVarObject_HEAD_INIT(&PyType_Type, 0)
    .tp_name = "xenfsimage.file",
    .tp_size = sizeof(fsimage_file_t),
    ...
};


~Andrew

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH for-next v2 v2 2/5] libxl: make python scripts work with python 2.6 and up

[Hans van Kranenburg]

On 3/6/19 6:52 PM, Wei Liu wrote:
> Go through the transformations suggested by 2to3 and pick the
> necessary ones.
> 
> Use sys.stderr.write to avoid importing from __future__.
> 
> Signed-off-by: Wei Liu <wei.liu2@citrix.com>
> ---
>  tools/libxl/gentest.py  |  2 +-
>  tools/libxl/gentypes.py | 10 +++++-----
>  tools/libxl/idl.py      | 13 ++++++-------
>  3 files changed, 12 insertions(+), 13 deletions(-)
> 
> diff --git a/tools/libxl/gentest.py b/tools/libxl/gentest.py
> index 989959fc68..81e13b437c 100644
> --- a/tools/libxl/gentest.py
> +++ b/tools/libxl/gentest.py
> @@ -86,7 +86,7 @@ def gen_rand_init(ty, v, indent = "    ", parent = None):
>  
>  if __name__ == '__main__':
>      if len(sys.argv) < 3:
> -        print >>sys.stderr, "Usage: gentest.py <idl> <implementation>"
> +        sys.stderr.write("Usage: gentest.py <idl> <implementation>\n")
>          sys.exit(1)
>  
>      random.seed(os.getenv('LIBXL_TESTIDL_SEED'))
> diff --git a/tools/libxl/gentypes.py b/tools/libxl/gentypes.py
> index 88e5c5f30e..656c157c01 100644
> --- a/tools/libxl/gentypes.py
> +++ b/tools/libxl/gentypes.py
> @@ -576,14 +576,14 @@ def libxl_C_enum_from_string(ty, str, e, indent = "    "):
>  
>  if __name__ == '__main__':
>      if len(sys.argv) != 6:
> -        print >>sys.stderr, "Usage: gentypes.py <idl> <header> <header-private> <header-json> <implementation>"
> +        sys.stderr.write("Usage: gentypes.py <idl> <header> <header-private> <header-json> <implementation>\n")
>          sys.exit(1)
>  
>      (_, idlname, header, header_private, header_json, impl) = sys.argv
>  
>      (builtins,types) = idl.parse(idlname)
>  
> -    print "outputting libxl type definitions to %s" % header
> +    print("outputting libxl type definitions to %s" % header)

Note that print is *not* a function in python 2, unless you..

  from __future__ import print_function

...which should not be avoided.

It looks like print() just works as a function, but it's not, and I
would recommend against introducing this kind of misleading code.

In print("hallo"), the parentheses are part of an expression, like in
  ((1+ 1) * 2)

Other syntax with parentheses creates tuples.

Python 2.7.13 (>>> replaced with ===)

=== type(())
<type 'tuple'>
=== type((1+1))
<type 'int'>
=== type((1, 2))
<type 'tuple'>

=== print () <- tuple
()
=== print (1+1) <- expression
2
=== print (1, 2) <- tuple
(1, 2)

=== from __future__ import print_function
=== print()

=== print(1+1)
2
=== print(1, 2)
1 2

>      f = open(header, "w")
>  
> @@ -633,7 +633,7 @@ if __name__ == '__main__':
>      f.write("""#endif /* %s */\n""" % (header_define))
>      f.close()
>  
> -    print "outputting libxl JSON definitions to %s" % header_json
> +    print("outputting libxl JSON definitions to %s" % header_json)
>  
>      f = open(header_json, "w")
>  
> @@ -657,7 +657,7 @@ if __name__ == '__main__':
>      f.write("""#endif /* %s */\n""" % header_json_define)
>      f.close()
>  
> -    print "outputting libxl type internal definitions to %s" % header_private
> +    print("outputting libxl type internal definitions to %s" % header_private)
>  
>      f = open(header_private, "w")
>  
> @@ -683,7 +683,7 @@ if __name__ == '__main__':
>      f.write("""#endif /* %s */\n""" % header_json_define)
>      f.close()
>  
> -    print "outputting libxl type implementations to %s" % impl
> +    print("outputting libxl type implementations to %s" % impl)
>  
>      f = open(impl, "w")
>      f.write("""
> diff --git a/tools/libxl/idl.py b/tools/libxl/idl.py
> index 2a7f3c44fe..b5bfc66b50 100644
> --- a/tools/libxl/idl.py
> +++ b/tools/libxl/idl.py
> @@ -11,7 +11,7 @@ DIR_BOTH = 3
>  _default_namespace = ""
>  def namespace(s):
>      if type(s) != str:
> -        raise TypeError, "Require a string for the default namespace."
> +        raise TypeError("Require a string for the default namespace.")
>      global _default_namespace
>      _default_namespace = s
>  
> @@ -346,7 +346,7 @@ class OrderedDict(dict):
>          return [(x,self[x]) for x in self.__ordered]
>  
>  def parse(f):
> -    print >>sys.stderr, "Parsing %s" % f
> +    sys.stderr.write("Parsing %s\n" % f)

And if you have it from future anyway...

print("Parsing", f, file=sys.stderr)

>      globs = {}
>      locs = OrderedDict()
> @@ -362,11 +362,10 @@ def parse(f):
>              globs[n] = t
>  
>      try:
> -        execfile(f, globs, locs)
> -    except SyntaxError,e:
> -        raise SyntaxError, \
> -              "Errors were found at line %d while processing %s:\n\t%s"\
> -              %(e.lineno,f,e.text)
> +        exec(compile(open(f).read(), f, 'exec'), globs, locs)
> +    except SyntaxError as e:
> +        raise SyntaxError("Errors were found at line %d while processing %s:\n\t%s"\
> +              %(e.lineno,f,e.text))
>  
>      types = [t for t in locs.ordered_values() if isinstance(t,Type)]
>  
> 

Hans

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH for-next v2 v2 2/5] libxl: make python scripts work with python 2.6 and up

[Wei Liu]

On Wed, Mar 06, 2019 at 09:58:19PM +0100, Hans van Kranenburg wrote:
> On 3/6/19 6:52 PM, Wei Liu wrote:
> > Go through the transformations suggested by 2to3 and pick the
> > necessary ones.
> > 
> > Use sys.stderr.write to avoid importing from __future__.
> > 
> > Signed-off-by: Wei Liu <wei.liu2@citrix.com>
> > ---
> >  tools/libxl/gentest.py  |  2 +-
> >  tools/libxl/gentypes.py | 10 +++++-----
> >  tools/libxl/idl.py      | 13 ++++++-------
> >  3 files changed, 12 insertions(+), 13 deletions(-)
> > 
> > diff --git a/tools/libxl/gentest.py b/tools/libxl/gentest.py
> > index 989959fc68..81e13b437c 100644
> > --- a/tools/libxl/gentest.py
> > +++ b/tools/libxl/gentest.py
> > @@ -86,7 +86,7 @@ def gen_rand_init(ty, v, indent = "    ", parent = None):
> >  
> >  if __name__ == '__main__':
> >      if len(sys.argv) < 3:
> > -        print >>sys.stderr, "Usage: gentest.py <idl> <implementation>"
> > +        sys.stderr.write("Usage: gentest.py <idl> <implementation>\n")
> >          sys.exit(1)
> >  
> >      random.seed(os.getenv('LIBXL_TESTIDL_SEED'))
> > diff --git a/tools/libxl/gentypes.py b/tools/libxl/gentypes.py
> > index 88e5c5f30e..656c157c01 100644
> > --- a/tools/libxl/gentypes.py
> > +++ b/tools/libxl/gentypes.py
> > @@ -576,14 +576,14 @@ def libxl_C_enum_from_string(ty, str, e, indent = "    "):
> >  
> >  if __name__ == '__main__':
> >      if len(sys.argv) != 6:
> > -        print >>sys.stderr, "Usage: gentypes.py <idl> <header> <header-private> <header-json> <implementation>"
> > +        sys.stderr.write("Usage: gentypes.py <idl> <header> <header-private> <header-json> <implementation>\n")
> >          sys.exit(1)
> >  
> >      (_, idlname, header, header_private, header_json, impl) = sys.argv
> >  
> >      (builtins,types) = idl.parse(idlname)
> >  
> > -    print "outputting libxl type definitions to %s" % header
> > +    print("outputting libxl type definitions to %s" % header)
> 
> Note that print is *not* a function in python 2, unless you..
> 
>   from __future__ import print_function
> 
> ...which should not be avoided.
> 

I know. print in 2 is a statement but in 3 is a function.

Yet this transformation is done by 2to3 _without_ importing
print_function. I thought it was a blessed way of doing stuff. I went
through those transformations. They looked correct (although I agree
they were confusing).

Importing print_function was specifically avoided because we wanted 2.4
compatibility at first. But now I propose to bump the requirement to
2.6, the from __future__ import print_function may become available. I
will need to check if that works with 2.6.

> It looks like print() just works as a function, but it's not, and I
> would recommend against introducing this kind of misleading code.
> 
> In print("hallo"), the parentheses are part of an expression, like in
>   ((1+ 1) * 2)
> 
> Other syntax with parentheses creates tuples.
> 
> Python 2.7.13 (>>> replaced with ===)
> 
> === type(())
> <type 'tuple'>
> === type((1+1))
> <type 'int'>
> === type((1, 2))
> <type 'tuple'>
> 
> === print () <- tuple
> ()
> === print (1+1) <- expression
> 2
> === print (1, 2) <- tuple
> (1, 2)
> 
> === from __future__ import print_function
> === print()
> 
> === print(1+1)
> 2
> === print(1, 2)
> 1 2
> 
> >      f = open(header, "w")
> >  
> > @@ -633,7 +633,7 @@ if __name__ == '__main__':
> >      f.write("""#endif /* %s */\n""" % (header_define))
> >      f.close()
> >  
> > -    print "outputting libxl JSON definitions to %s" % header_json
> > +    print("outputting libxl JSON definitions to %s" % header_json)
> >  
> >      f = open(header_json, "w")
> >  
> > @@ -657,7 +657,7 @@ if __name__ == '__main__':
> >      f.write("""#endif /* %s */\n""" % header_json_define)
> >      f.close()
> >  
> > -    print "outputting libxl type internal definitions to %s" % header_private
> > +    print("outputting libxl type internal definitions to %s" % header_private)
> >  
> >      f = open(header_private, "w")
> >  
> > @@ -683,7 +683,7 @@ if __name__ == '__main__':
> >      f.write("""#endif /* %s */\n""" % header_json_define)
> >      f.close()
> >  
> > -    print "outputting libxl type implementations to %s" % impl
> > +    print("outputting libxl type implementations to %s" % impl)
> >  
> >      f = open(impl, "w")
> >      f.write("""
> > diff --git a/tools/libxl/idl.py b/tools/libxl/idl.py
> > index 2a7f3c44fe..b5bfc66b50 100644
> > --- a/tools/libxl/idl.py
> > +++ b/tools/libxl/idl.py
> > @@ -11,7 +11,7 @@ DIR_BOTH = 3
> >  _default_namespace = ""
> >  def namespace(s):
> >      if type(s) != str:
> > -        raise TypeError, "Require a string for the default namespace."
> > +        raise TypeError("Require a string for the default namespace.")
> >      global _default_namespace
> >      _default_namespace = s
> >  
> > @@ -346,7 +346,7 @@ class OrderedDict(dict):
> >          return [(x,self[x]) for x in self.__ordered]
> >  
> >  def parse(f):
> > -    print >>sys.stderr, "Parsing %s" % f
> > +    sys.stderr.write("Parsing %s\n" % f)
> 
> And if you have it from future anyway...
> 
> print("Parsing", f, file=sys.stderr)
> 

Yeah, the first version had this. But then I deleted the import and rewrote
the patch.

Wei.

> >      globs = {}
> >      locs = OrderedDict()
> > @@ -362,11 +362,10 @@ def parse(f):
> >              globs[n] = t
> >  
> >      try:
> > -        execfile(f, globs, locs)
> > -    except SyntaxError,e:
> > -        raise SyntaxError, \
> > -              "Errors were found at line %d while processing %s:\n\t%s"\
> > -              %(e.lineno,f,e.text)
> > +        exec(compile(open(f).read(), f, 'exec'), globs, locs)
> > +    except SyntaxError as e:
> > +        raise SyntaxError("Errors were found at line %d while processing %s:\n\t%s"\
> > +              %(e.lineno,f,e.text))
> >  
> >      types = [t for t in locs.ordered_values() if isinstance(t,Type)]
> >  
> > 
> 
> Hans

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH for-next v2 v2 1/5] build/m4: make python_devel.m4 work with both python 2 and 3

[Anthony PERARD]

On Wed, Mar 06, 2019 at 05:52:06PM +0000, Wei Liu wrote:
> Do the following:
> 
> 1. Change the form of "print".
> 2. Use AC_CHECK_FUNC to avoid the need to generate library name.
> 3. Remove unused stuff.
> 
> Signed-off-by: Wei Liu <wei.liu2@citrix.com>

Reviewed-by: Anthony PERARD <anthony.perard@citrix.com>

Thanks.

> ---
> I doubt the non-python-config branch works, because the paths generated
> seem rather off. It definitely doesn't work on my machine, but I
> don't know how other systems could possibly be configured before the
> existence of python-config.

It's probably fine. On Arch, the paths for python2 looks fine. With
python3, that would generates "-L/usr/lib/python3.7/config" which
doesn't exist, but the python lib is in /usr/lib, so it's probably going
to work just fine. Anyway, an installation of python3 without its
python-config is probably broken.

-- 
Anthony PERARD

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH for-next v2 v2 5/5] Update python requirement

[Anthony PERARD]

On Wed, Mar 06, 2019 at 05:52:10PM +0000, Wei Liu wrote:
> CentOS 5, which was the reason of the 2.4 restriction, is EOL. CentOS
> 6 ships 2.6.
> 
> Bump the version to 2.6 in README. Now that all scripts are 3
> compatible, remove the restriction on python 2 as well.
> 
> Update the check in configure.ac.
> 
> Signed-off-by: Wei Liu <wei.liu2@citrix.com>

I think this patch should be first in the series, otherwise there is
going to be python code non-compatible with 2.4 before it is deemed too
old ;-).

-- 
Anthony PERARD

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH for-next v2 v2 5/5] Update python requirement

[Wei Liu]

On Thu, Mar 07, 2019 at 11:21:29AM +0000, Anthony PERARD wrote:
> On Wed, Mar 06, 2019 at 05:52:10PM +0000, Wei Liu wrote:
> > CentOS 5, which was the reason of the 2.4 restriction, is EOL. CentOS
> > 6 ships 2.6.
> > 
> > Bump the version to 2.6 in README. Now that all scripts are 3
> > compatible, remove the restriction on python 2 as well.
> > 
> > Update the check in configure.ac.
> > 
> > Signed-off-by: Wei Liu <wei.liu2@citrix.com>
> 
> I think this patch should be first in the series, otherwise there is
> going to be python code non-compatible with 2.4 before it is deemed too
> old ;-).

Works for me.

Wei.

> 
> -- 
> Anthony PERARD

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH for-next v2 v2 2/5] libxl: make python scripts work with python 2.6 and up

[Wei Liu]

On Thu, Mar 07, 2019 at 10:37:45AM +0000, Wei Liu wrote:
> 
> Importing print_function was specifically avoided because we wanted 2.4
> compatibility at first. But now I propose to bump the requirement to
> 2.6, the from __future__ import print_function may become available. I
> will need to check if that works with 2.6.

According to:

https://docs.python.org/2.6/library/__future__.html

from __future__ import print_function should be available in 2.6.

I will use it and fix the sys.stderr.write in the next version.

Wei.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel