[PATCH v3 0/7] tools: Python 3 compatibility

[PATCH v3 0/7] tools: Python 3 compatibility

[Wei Liu]

This series makes tools work with Python 2 and 3

Tested on my testbox with Python 2 and 3. Unfortunately I didn't have a
suitable disk image for pygrub to consume, but at least while running pygrub 2
and 3 it gave the same error message.

Andrew's "tools/xen-foreign: Update python scripts to be
Py3 compatible" is a pre-requisite.

Wei.

Wei Liu (7):
  Update python requirement
  pygrub/fsimage: drop unused struct
  pygrub: change tabs into spaces
  build/m4: make python_devel.m4 work with both python 2 and 3
  libxl: make python scripts work with python 2.6 and up
  pygrub: make python scripts work with 2.6 and up
  pygrub/fsimage: make it work with python 3

 README                             | 10 +---
 m4/python_devel.m4                 | 23 +++------
 tools/configure                    | 72 +++++----------------------
 tools/configure.ac                 |  2 +-
 tools/libxl/gentest.py             |  4 +-
 tools/libxl/gentypes.py            | 12 +++--
 tools/libxl/idl.py                 | 15 +++---
 tools/pygrub/src/ExtLinuxConf.py   | 19 +++++---
 tools/pygrub/src/GrubConf.py       | 39 ++++++++-------
 tools/pygrub/src/LiloConf.py       | 19 +++++---
 tools/pygrub/src/fsimage/fsimage.c | 99 ++++++++++++++++++++++++++++++++------
 tools/pygrub/src/pygrub            | 75 ++++++++++++++---------------
 12 files changed, 205 insertions(+), 184 deletions(-)

-- 
2.11.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v3 1/7] Update python requirement

[Wei Liu]

CentOS 5, which was the reason of the 2.4 restriction, is EOL. CentOS
6 ships 2.6.

Bump the version to 2.6 in README. Now that all scripts are 3
compatible, remove the restriction on python 2 as well.

Update the check in configure.ac.

Signed-off-by: Wei Liu <wei.liu2@citrix.com>
---
 README             | 10 ++--------
 tools/configure    |  8 ++++----
 tools/configure.ac |  2 +-
 3 files changed, 7 insertions(+), 13 deletions(-)

diff --git a/README b/README
index c19409efa2..3049201d91 100644
--- a/README
+++ b/README
@@ -46,7 +46,7 @@ provided by your OS distributor:
         - GCC 4.8 or later
         - GNU Binutils 2.24 or later
     * Development install of zlib (e.g., zlib-dev)
-    * Development install of Python 2, v2.4 or later (e.g., python-dev)
+    * Development install of Python v2.6 or later (e.g., python-dev)
     * Development install of curses (e.g., libncurses-dev)
     * Development install of openssl (e.g., openssl-dev)
     * Development install of x11 (e.g. xorg-x11-dev)
@@ -177,16 +177,10 @@ Python Runtime Libraries
 
 Various tools, such as pygrub, have the following runtime dependencies:
 
-    * Python 2, v2.4 or later.
+    * Python v2.6 or later.
           URL:    http://www.python.org/
           Debian: python
 
-Note that the build system expects `python` to be python2.  If your system
-has `python` pointing to python3 (as in the case of Arch Linux or Anaconda),
-you'll need to specify a path to a python2 binary when running configure:
-
-    PYTHON=/usr/bin/python2 ./configure
-
 Intel(R) Trusted Execution Technology Support
 =============================================
 
diff --git a/tools/configure b/tools/configure
index acc857510e..d0065b811d 100755
--- a/tools/configure
+++ b/tools/configure
@@ -7002,15 +7002,15 @@ if test x"${PYTHONPATH}" = x"no"
 then
     as_fn_error $? "Unable to find $PYTHON, please install $PYTHON" "$LINENO" 5
 fi
-{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for python version >= 2.3 " >&5
-$as_echo_n "checking for python version >= 2.3 ... " >&6; }
-`$PYTHON -c 'import sys; sys.exit(eval("sys.version_info < (2, 3)"))'`
+{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for python version >= 2.6 " >&5
+$as_echo_n "checking for python version >= 2.6 ... " >&6; }
+`$PYTHON -c 'import sys; sys.exit(eval("sys.version_info < (2, 6)"))'`
 if test "$?" != "0"
 then
     python_version=`$PYTHON -V 2>&1`
     { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5
 $as_echo "no" >&6; }
-    as_fn_error $? "$python_version is too old, minimum required version is 2.3" "$LINENO" 5
+    as_fn_error $? "$python_version is too old, minimum required version is 2.6" "$LINENO" 5
 else
     { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5
 $as_echo "yes" >&6; }
diff --git a/tools/configure.ac b/tools/configure.ac
index 1499344ce6..c9fd69ddfa 100644
--- a/tools/configure.ac
+++ b/tools/configure.ac
@@ -358,7 +358,7 @@ AS_IF([echo "$PYTHON" | grep -q "^/"], [
 ],[test -z "$PYTHON"], [PYTHON="python"],
 [AC_MSG_ERROR([PYTHON specified, but is not an absolute path])])
 AX_PATH_PROG_OR_FAIL([PYTHONPATH], [$PYTHON])
-AX_CHECK_PYTHON_VERSION([2], [3])
+AX_CHECK_PYTHON_VERSION([2], [6])
 
 AS_IF([test "$cross_compiling" != yes], [
     AX_CHECK_PYTHON_DEVEL()
-- 
2.11.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v3 2/7] pygrub/fsimage: drop unused struct

[Wei Liu]

Signed-off-by: Wei Liu <wei.liu2@citrix.com>
---
 tools/pygrub/src/fsimage/fsimage.c | 7 -------
 1 file changed, 7 deletions(-)

diff --git a/tools/pygrub/src/fsimage/fsimage.c b/tools/pygrub/src/fsimage/fsimage.c
index 743a3fb7b8..780207791c 100644
--- a/tools/pygrub/src/fsimage/fsimage.c
+++ b/tools/pygrub/src/fsimage/fsimage.c
@@ -43,13 +43,6 @@ typedef struct fsimage_file {
 	fsi_file_t *file;
 } fsimage_file_t;
 
-struct foo {
-	int ref;
-	int size;
-	long hash;
-	int state;
-};
-
 static PyObject *
 fsimage_file_read(fsimage_file_t *file, PyObject *args, PyObject *kwargs)
 {
-- 
2.11.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v3 3/7] pygrub: change tabs into spaces

[Wei Liu]

Not sure why Python 2 never complained, but Python 3 does.

Change tabs to spaces.

Signed-off-by: Wei Liu <wei.liu2@citrix.com>
---
Backport candidate.
---
 tools/pygrub/src/pygrub | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/tools/pygrub/src/pygrub b/tools/pygrub/src/pygrub
index 52a8965ad9..1189b1ca48 100755
--- a/tools/pygrub/src/pygrub
+++ b/tools/pygrub/src/pygrub
@@ -858,7 +858,7 @@ if __name__ == "__main__":
             output_directory = a
 
     if debug:
-	logging.basicConfig(level=logging.DEBUG)
+        logging.basicConfig(level=logging.DEBUG)
 
 
     try:
@@ -917,7 +917,7 @@ if __name__ == "__main__":
             # IOErrors raised by fsimage.open
             # RuntimeErrors raised by run_grub if no menu.lst present
             if debug:
-		traceback.print_exc()
+                traceback.print_exc()
             fs = None
             continue
 
-- 
2.11.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v3 4/7] build/m4: make python_devel.m4 work with both python 2 and 3

[Wei Liu]

Do the following:

1. Change the form of "print".
2. Use AC_CHECK_FUNC to avoid the need to generate library name.
3. Remove unused stuff.

Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Reviewed-by: Anthony PERARD <anthony.perard@citrix.com>
---
 m4/python_devel.m4 | 23 +++++++-------------
 tools/configure    | 64 +++++++-----------------------------------------------
 2 files changed, 16 insertions(+), 71 deletions(-)

diff --git a/m4/python_devel.m4 b/m4/python_devel.m4
index 05ea4ef7e2..f9cb23aee1 100644
--- a/m4/python_devel.m4
+++ b/m4/python_devel.m4
@@ -1,38 +1,31 @@
 AC_DEFUN([AX_CHECK_PYTHON_DEVEL], [
 ac_previous_cppflags=$CPPFLAGS
 ac_previous_ldflags=$LDFLAGS
-ac_python_version=`$PYTHON -c 'import distutils.sysconfig; \
-    print distutils.sysconfig.get_config_var("VERSION")'`
 AC_PATH_PROG([pyconfig], [$PYTHON-config], [no])
 AS_IF([test x"$pyconfig" = x"no"], [
     dnl For those that don't have python-config
     CPPFLAGS="$CFLAGS `$PYTHON -c 'import distutils.sysconfig; \
         print "-I" + distutils.sysconfig.get_config_var("INCLUDEPY")'`"
     CPPFLAGS="$CPPFLAGS `$PYTHON -c 'import distutils.sysconfig; \
-        print distutils.sysconfig.get_config_var("CFLAGS")'`"
-    PYTHON_LIBS="$LDFLAGS `$PYTHON -c 'import distutils.sysconfig; \
-        print distutils.sysconfig.get_config_var("LIBS")'`"
-    PYTHON_LIBS="$LDFLAGS `$PYTHON -c 'import distutils.sysconfig; \
-        print distutils.sysconfig.get_config_var("SYSLIBS")'`"
+        print(distutils.sysconfig.get_config_var("CFLAGS"))'`"
     LDFLAGS="$LDFLAGS `$PYTHON -c 'import distutils.sysconfig; \
-        print "-L" + distutils.sysconfig.get_python_lib(plat_specific=1,\
-        standard_lib=1) + "/config"'`"
+        print("-L" + distutils.sysconfig.get_python_lib(plat_specific=1,\
+        standard_lib=1) + "/config")'`"
     LDFLAGS="$LDFLAGS `$PYTHON -c 'import distutils.sysconfig; \
-        print distutils.sysconfig.get_config_var("LINKFORSHARED")'`"
+        print(distutils.sysconfig.get_config_var("LINKFORSHARED"))'`"
     LDFLAGS="$LDFLAGS `$PYTHON -c 'import distutils.sysconfig; \
-        print distutils.sysconfig.get_config_var("LDFLAGS")'`"
+        print(distutils.sysconfig.get_config_var("LDFLAGS"))'`"
 ], [
     dnl If python-config is found use it
     CPPFLAGS="$CFLAGS `$PYTHON-config --cflags`"
     LDFLAGS="$LDFLAGS `$PYTHON-config --ldflags`"
-    PYTHON_LIBS="$LIBS `$PYTHON-config --libs`"
 ])
 
 AC_CHECK_HEADER([Python.h], [],
     [AC_MSG_ERROR([Unable to find Python development headers])],)
-AC_CHECK_LIB(python$ac_python_version, PyArg_ParseTuple, [],
-    [AC_MSG_ERROR([Unable to find a suitable python development library])],
-    [$PYTHON_LIBS])
+AC_CHECK_FUNC([PyArg_ParseTuple], [],
+    [AC_MSG_ERROR([Unable to find a suitable python development library])]) 
+
 CPPFLAGS=$ac_previous_cppflags
 LDFLAGS=$ac_previous_ldflags
 ])
diff --git a/tools/configure b/tools/configure
index d0065b811d..e1fa5d6b0f 100755
--- a/tools/configure
+++ b/tools/configure
@@ -7418,8 +7418,6 @@ if test "$cross_compiling" != yes; then :
 
 ac_previous_cppflags=$CPPFLAGS
 ac_previous_ldflags=$LDFLAGS
-ac_python_version=`$PYTHON -c 'import distutils.sysconfig; \
-    print distutils.sysconfig.get_config_var("VERSION")'`
 # Extract the first word of "$PYTHON-config", so it can be a program name with args.
 set dummy $PYTHON-config; ac_word=$2
 { $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5
@@ -7466,24 +7464,19 @@ if test x"$pyconfig" = x"no"; then :
         CPPFLAGS="$CFLAGS `$PYTHON -c 'import distutils.sysconfig; \
         print "-I" + distutils.sysconfig.get_config_var("INCLUDEPY")'`"
     CPPFLAGS="$CPPFLAGS `$PYTHON -c 'import distutils.sysconfig; \
-        print distutils.sysconfig.get_config_var("CFLAGS")'`"
-    PYTHON_LIBS="$LDFLAGS `$PYTHON -c 'import distutils.sysconfig; \
-        print distutils.sysconfig.get_config_var("LIBS")'`"
-    PYTHON_LIBS="$LDFLAGS `$PYTHON -c 'import distutils.sysconfig; \
-        print distutils.sysconfig.get_config_var("SYSLIBS")'`"
+        print(distutils.sysconfig.get_config_var("CFLAGS"))'`"
     LDFLAGS="$LDFLAGS `$PYTHON -c 'import distutils.sysconfig; \
-        print "-L" + distutils.sysconfig.get_python_lib(plat_specific=1,\
-        standard_lib=1) + "/config"'`"
+        print("-L" + distutils.sysconfig.get_python_lib(plat_specific=1,\
+        standard_lib=1) + "/config")'`"
     LDFLAGS="$LDFLAGS `$PYTHON -c 'import distutils.sysconfig; \
-        print distutils.sysconfig.get_config_var("LINKFORSHARED")'`"
+        print(distutils.sysconfig.get_config_var("LINKFORSHARED"))'`"
     LDFLAGS="$LDFLAGS `$PYTHON -c 'import distutils.sysconfig; \
-        print distutils.sysconfig.get_config_var("LDFLAGS")'`"
+        print(distutils.sysconfig.get_config_var("LDFLAGS"))'`"
 
 else
 
         CPPFLAGS="$CFLAGS `$PYTHON-config --cflags`"
     LDFLAGS="$LDFLAGS `$PYTHON-config --ldflags`"
-    PYTHON_LIBS="$LIBS `$PYTHON-config --libs`"
 
 fi
 
@@ -7495,55 +7488,14 @@ else
 fi
 
 
-as_ac_Lib=`$as_echo "ac_cv_lib_python$ac_python_version''_PyArg_ParseTuple" | $as_tr_sh`
-{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for PyArg_ParseTuple in -lpython$ac_python_version" >&5
-$as_echo_n "checking for PyArg_ParseTuple in -lpython$ac_python_version... " >&6; }
-if eval \${$as_ac_Lib+:} false; then :
-  $as_echo_n "(cached) " >&6
-else
-  ac_check_lib_save_LIBS=$LIBS
-LIBS="-lpython$ac_python_version $PYTHON_LIBS $LIBS"
-cat confdefs.h - <<_ACEOF >conftest.$ac_ext
-/* end confdefs.h.  */
-
-/* Override any GCC internal prototype to avoid an error.
-   Use char because int might match the return type of a GCC
-   builtin and then its argument prototype would still apply.  */
-#ifdef __cplusplus
-extern "C"
-#endif
-char PyArg_ParseTuple ();
-int
-main ()
-{
-return PyArg_ParseTuple ();
-  ;
-  return 0;
-}
-_ACEOF
-if ac_fn_c_try_link "$LINENO"; then :
-  eval "$as_ac_Lib=yes"
-else
-  eval "$as_ac_Lib=no"
-fi
-rm -f core conftest.err conftest.$ac_objext \
-    conftest$ac_exeext conftest.$ac_ext
-LIBS=$ac_check_lib_save_LIBS
-fi
-eval ac_res=\$$as_ac_Lib
-	       { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_res" >&5
-$as_echo "$ac_res" >&6; }
-if eval test \"x\$"$as_ac_Lib"\" = x"yes"; then :
-  cat >>confdefs.h <<_ACEOF
-#define `$as_echo "HAVE_LIBpython$ac_python_version" | $as_tr_cpp` 1
-_ACEOF
-
-  LIBS="-lpython$ac_python_version $LIBS"
+ac_fn_c_check_func "$LINENO" "PyArg_ParseTuple" "ac_cv_func_PyArg_ParseTuple"
+if test "x$ac_cv_func_PyArg_ParseTuple" = xyes; then :
 
 else
   as_fn_error $? "Unable to find a suitable python development library" "$LINENO" 5
 fi
 
+
 CPPFLAGS=$ac_previous_cppflags
 LDFLAGS=$ac_previous_ldflags
 
-- 
2.11.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v3 5/7] libxl: make python scripts work with python 2.6 and up

[Wei Liu]

Go through transformations suggested by 2to3 and pick the necessary
ones.

Signed-off-by: Wei Liu <wei.liu2@citrix.com>
---
 tools/libxl/gentest.py  |  4 +++-
 tools/libxl/gentypes.py | 12 +++++++-----
 tools/libxl/idl.py      | 15 ++++++++-------
 3 files changed, 18 insertions(+), 13 deletions(-)

diff --git a/tools/libxl/gentest.py b/tools/libxl/gentest.py
index 989959fc68..1cc7eebc82 100644
--- a/tools/libxl/gentest.py
+++ b/tools/libxl/gentest.py
@@ -1,5 +1,7 @@
 #!/usr/bin/python
 
+from __future__ import print_function
+
 import os
 import sys
 import re
@@ -86,7 +88,7 @@ def gen_rand_init(ty, v, indent = "    ", parent = None):
 
 if __name__ == '__main__':
     if len(sys.argv) < 3:
-        print >>sys.stderr, "Usage: gentest.py <idl> <implementation>"
+        print("Usage: gentest.py <idl> <implementation>", file=sys.stderr)
         sys.exit(1)
 
     random.seed(os.getenv('LIBXL_TESTIDL_SEED'))
diff --git a/tools/libxl/gentypes.py b/tools/libxl/gentypes.py
index 88e5c5f30e..6417c9dd8c 100644
--- a/tools/libxl/gentypes.py
+++ b/tools/libxl/gentypes.py
@@ -1,5 +1,7 @@
 #!/usr/bin/python
 
+from __future__ import print_function
+
 import sys
 import re
 
@@ -576,14 +578,14 @@ def libxl_C_enum_from_string(ty, str, e, indent = "    "):
 
 if __name__ == '__main__':
     if len(sys.argv) != 6:
-        print >>sys.stderr, "Usage: gentypes.py <idl> <header> <header-private> <header-json> <implementation>"
+        print("Usage: gentypes.py <idl> <header> <header-private> <header-json> <implementation>", file=sys.stderr)
         sys.exit(1)
 
     (_, idlname, header, header_private, header_json, impl) = sys.argv
 
     (builtins,types) = idl.parse(idlname)
 
-    print "outputting libxl type definitions to %s" % header
+    print("outputting libxl type definitions to %s" % header)
 
     f = open(header, "w")
 
@@ -633,7 +635,7 @@ if __name__ == '__main__':
     f.write("""#endif /* %s */\n""" % (header_define))
     f.close()
 
-    print "outputting libxl JSON definitions to %s" % header_json
+    print("outputting libxl JSON definitions to %s" % header_json)
 
     f = open(header_json, "w")
 
@@ -657,7 +659,7 @@ if __name__ == '__main__':
     f.write("""#endif /* %s */\n""" % header_json_define)
     f.close()
 
-    print "outputting libxl type internal definitions to %s" % header_private
+    print("outputting libxl type internal definitions to %s" % header_private)
 
     f = open(header_private, "w")
 
@@ -683,7 +685,7 @@ if __name__ == '__main__':
     f.write("""#endif /* %s */\n""" % header_json_define)
     f.close()
 
-    print "outputting libxl type implementations to %s" % impl
+    print("outputting libxl type implementations to %s" % impl)
 
     f = open(impl, "w")
     f.write("""
diff --git a/tools/libxl/idl.py b/tools/libxl/idl.py
index 2a7f3c44fe..d7367503b4 100644
--- a/tools/libxl/idl.py
+++ b/tools/libxl/idl.py
@@ -1,3 +1,5 @@
+from __future__ import print_function
+
 import sys
 
 PASS_BY_VALUE = 1
@@ -11,7 +13,7 @@ DIR_BOTH = 3
 _default_namespace = ""
 def namespace(s):
     if type(s) != str:
-        raise TypeError, "Require a string for the default namespace."
+        raise TypeError("Require a string for the default namespace.")
     global _default_namespace
     _default_namespace = s
 
@@ -346,7 +348,7 @@ class OrderedDict(dict):
         return [(x,self[x]) for x in self.__ordered]
 
 def parse(f):
-    print >>sys.stderr, "Parsing %s" % f
+    print("Parsing %s" % f, file=sys.stderr)
 
     globs = {}
     locs = OrderedDict()
@@ -362,11 +364,10 @@ def parse(f):
             globs[n] = t
 
     try:
-        execfile(f, globs, locs)
-    except SyntaxError,e:
-        raise SyntaxError, \
-              "Errors were found at line %d while processing %s:\n\t%s"\
-              %(e.lineno,f,e.text)
+        exec(compile(open(f).read(), f, 'exec'), globs, locs)
+    except SyntaxError as e:
+        raise SyntaxError("Errors were found at line %d while processing %s:\n\t%s"
+                          % (e.lineno, f, e.text))
 
     types = [t for t in locs.ordered_values() if isinstance(t,Type)]
 
-- 
2.11.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v3 6/7] pygrub: make python scripts work with 2.6 and up

[Wei Liu]

Run 2to3 and pick the sensible suggestions.

Import print_function and absolute_import so 2.6 can work.

There has never been a curses.wrapper module according to 2.x and 3.x
doc, only a function, so "import curses.wrapper" is not correct. It
happened to work because 2.x implemented a (undocumented) module.

We only need to import curses to make curses.wrapper available to
pygrub.

Signed-off-by: Wei Liu <wei.liu2@citrix.com>
---
 tools/pygrub/src/ExtLinuxConf.py | 19 ++++++-----
 tools/pygrub/src/GrubConf.py     | 39 ++++++++++++----------
 tools/pygrub/src/LiloConf.py     | 19 ++++++-----
 tools/pygrub/src/pygrub          | 71 ++++++++++++++++++++--------------------
 4 files changed, 78 insertions(+), 70 deletions(-)

diff --git a/tools/pygrub/src/ExtLinuxConf.py b/tools/pygrub/src/ExtLinuxConf.py
index d1789bf020..9fd635b9cf 100644
--- a/tools/pygrub/src/ExtLinuxConf.py
+++ b/tools/pygrub/src/ExtLinuxConf.py
@@ -10,9 +10,11 @@
 # along with this program; If not, see <http://www.gnu.org/licenses/>.
 #
 
+from __future__ import print_function, absolute_import
+
 import sys, re, os
 import logging
-import GrubConf
+from . import GrubConf
 
 class ExtLinuxImage(object):
     def __init__(self, lines, path):
@@ -32,7 +34,8 @@ class ExtLinuxImage(object):
         self.lines = []
         self.path = path
         self.root = ""
-        map(self.set_from_line, lines)
+        for line in lines:
+            self.set_from_line(line)
 
     def set_from_line(self, line, replace = None):
         (com, arg) = GrubConf.grub_exact_split(line, 2)
@@ -67,7 +70,7 @@ class ExtLinuxImage(object):
                         setattr(self, "initrd", a.replace("initrd=", ""))
                         arg = arg.replace(a, "")
 
-        if com is not None and self.commands.has_key(com):
+        if com is not None and com in self.commands:
             if self.commands[com] is not None:
                 setattr(self, self.commands[com], re.sub('^"(.+)"$', r"\1", arg.strip()))
             else:
@@ -136,7 +139,7 @@ class ExtLinuxConfigFile(object):
     def parse(self, buf = None):
         if buf is None:
             if self.filename is None:
-                raise ValueError, "No config file defined to parse!"
+                raise ValueError("No config file defined to parse!")
 
             f = open(self.filename, 'r')
             lines = f.readlines()
@@ -167,7 +170,7 @@ class ExtLinuxConfigFile(object):
 
             (com, arg) = GrubConf.grub_exact_split(l, 2)
             com = com.lower()
-            if self.commands.has_key(com):
+            if com in self.commands:
                 if self.commands[com] is not None:
                     setattr(self, self.commands[com], arg.strip())
                 else:
@@ -207,8 +210,8 @@ class ExtLinuxConfigFile(object):
         
 if __name__ == "__main__":
     if len(sys.argv) < 2:
-        raise RuntimeError, "Need a configuration file to read"
+        raise RuntimeError("Need a configuration file to read")
     g = ExtLinuxConfigFile(sys.argv[1])
     for i in g.images:
-        print i
-    print g.default
+        print(i)
+    print(g.default)
diff --git a/tools/pygrub/src/GrubConf.py b/tools/pygrub/src/GrubConf.py
index dc810d55cb..f8d3799dc0 100644
--- a/tools/pygrub/src/GrubConf.py
+++ b/tools/pygrub/src/GrubConf.py
@@ -12,6 +12,8 @@
 # along with this program; If not, see <http://www.gnu.org/licenses/>.
 #
 
+from __future__ import print_function, absolute_import
+
 import os, sys
 import logging
 import re
@@ -44,7 +46,7 @@ def get_path(s):
         return (None, s)
     idx = s.find(')')
     if idx == -1:
-        raise ValueError, "Unable to find matching ')'"
+        raise ValueError("Unable to find matching ')'")
     d = s[:idx]
     return (GrubDiskPart(d), s[idx + 1:])
 
@@ -100,7 +102,8 @@ class _GrubImage(object):
                 "  initrd: %s\n" %(self.title, self.root, self.kernel,
                                    self.args, self.initrd))
     def _parse(self, lines):
-        map(self.set_from_line, lines)
+        for line in lines:
+            self.set_from_line(line)
 
     def reset(self, lines):
         self._root = self._initrd = self._kernel = self._args = None
@@ -141,7 +144,7 @@ class GrubImage(_GrubImage):
     def set_from_line(self, line, replace = None):
         (com, arg) = grub_exact_split(line, 2)
 
-        if self.commands.has_key(com):
+        if com in self.commands:
             if self.commands[com] is not None:
                 setattr(self, self.commands[com], arg.strip())
             else:
@@ -177,7 +180,7 @@ class _GrubConfigFile(object):
             self.parse()
 
     def parse(self, buf = None):
-        raise RuntimeError, "unimplemented parse function"   
+        raise RuntimeError("unimplemented parse function")
 
     def hasPasswordAccess(self):
         return self.passwordAccess
@@ -201,7 +204,7 @@ class _GrubConfigFile(object):
                 import crypt
                 if crypt.crypt(password, pwd[1]) == pwd[1]:
                     return True
-            except Exception, e:
+            except Exception as e:
                 self.passExc = "Can't verify password: %s" % str(e)
                 return False
 
@@ -213,7 +216,7 @@ class _GrubConfigFile(object):
 
     def set(self, line):
         (com, arg) = grub_exact_split(line, 2)
-        if self.commands.has_key(com):
+        if com in self.commands:
             if self.commands[com] is not None:
                 setattr(self, self.commands[com], arg.strip())
             else:
@@ -233,7 +236,7 @@ class _GrubConfigFile(object):
             self._default = val
 
         if self._default < 0:
-            raise ValueError, "default must be positive number"
+            raise ValueError("default must be positive number")
     default = property(_get_default, _set_default)
 
     def set_splash(self, val):
@@ -265,7 +268,7 @@ class GrubConfigFile(_GrubConfigFile):
     def parse(self, buf = None):
         if buf is None:
             if self.filename is None:
-                raise ValueError, "No config file defined to parse!"
+                raise ValueError("No config file defined to parse!")
 
             f = open(self.filename, 'r')
             lines = f.readlines()
@@ -296,7 +299,7 @@ class GrubConfigFile(_GrubConfigFile):
                 continue
 
             (com, arg) = grub_exact_split(l, 2)
-            if self.commands.has_key(com):
+            if com in self.commands:
                 if self.commands[com] is not None:
                     setattr(self, self.commands[com], arg.strip())
                 else:
@@ -328,7 +331,7 @@ class Grub2Image(_GrubImage):
         if com == "set":
             (com,arg) = grub2_handle_set(arg)
             
-        if self.commands.has_key(com):
+        if com in self.commands:
             if self.commands[com] is not None:
                 setattr(self, self.commands[com], arg.strip())
             else:
@@ -364,7 +367,7 @@ class Grub2ConfigFile(_GrubConfigFile):
     def parse(self, buf = None):
         if buf is None:
             if self.filename is None:
-                raise ValueError, "No config file defined to parse!"
+                raise ValueError("No config file defined to parse!")
 
             f = open(self.filename, 'r')
             lines = f.readlines()
@@ -398,7 +401,7 @@ class Grub2ConfigFile(_GrubConfigFile):
             title_match = re.match('^menuentry ["\'](.*?)["\'] (.*){', l)
             if title_match:
                 if img is not None:
-                    raise RuntimeError, "syntax error: cannot nest menuentry (%d %s)" % (len(img),img)
+                    raise RuntimeError("syntax error: cannot nest menuentry (%d %s)" % (len(img),img))
                 img = []
                 title = title_match.group(1)
                 continue
@@ -413,7 +416,7 @@ class Grub2ConfigFile(_GrubConfigFile):
                         menu_level -= 1
                         continue
                     else:
-                        raise RuntimeError, "syntax error: closing brace without menuentry"
+                        raise RuntimeError("syntax error: closing brace without menuentry")
 
                 self.add_image(Grub2Image(title, img))
                 img = None
@@ -428,7 +431,7 @@ class Grub2ConfigFile(_GrubConfigFile):
             if com == "set":
                 (com,arg) = grub2_handle_set(arg)
                 
-            if self.commands.has_key(com):
+            if com in self.commands:
                 if self.commands[com] is not None:
                     arg_strip = arg.strip()
                     if arg_strip == "${saved_entry}" or arg_strip == "${next_entry}":
@@ -443,7 +446,7 @@ class Grub2ConfigFile(_GrubConfigFile):
                 logging.warning("Unknown directive %s" %(com,))
             
         if img is not None:
-            raise RuntimeError, "syntax error: end of file with open menuentry(%d %s)" % (len(img),img)
+            raise RuntimeError("syntax error: end of file with open menuentry(%d %s)" % (len(img),img))
 
         if self.hasPassword():
             self.setPasswordAccess(False)
@@ -462,12 +465,12 @@ class Grub2ConfigFile(_GrubConfigFile):
         
 if __name__ == "__main__":
     if len(sys.argv) < 3:
-        raise RuntimeError, "Need a grub version (\"grub\" or \"grub2\") and a grub.conf or grub.cfg to read"
+        raise RuntimeError('Need a grub version ("grub" or "grub2") and a grub.conf or grub.cfg to read')
     if sys.argv[1] == "grub":
         g = GrubConfigFile(sys.argv[2])
     elif sys.argv[1] == "grub2":
         g = Grub2ConfigFile(sys.argv[2])
     else:
-        raise RuntimeError, "Unknown config type %s" % sys.argv[1]
+        raise RuntimeError("Unknown config type %s" % sys.argv[1])
     for i in g.images:
-        print i #, i.title, i.root, i.kernel, i.args, i.initrd
+        print(i) #, i.title, i.root, i.kernel, i.args, i.initrd
diff --git a/tools/pygrub/src/LiloConf.py b/tools/pygrub/src/LiloConf.py
index 2cb649f115..e3bfcb5244 100644
--- a/tools/pygrub/src/LiloConf.py
+++ b/tools/pygrub/src/LiloConf.py
@@ -2,9 +2,11 @@
 #LiloConf.py
 #
 
+from __future__ import print_function, absolute_import
+
 import sys, re, os
 import logging
-import GrubConf
+from . import GrubConf
 
 class LiloImage(object):
     def __init__(self, lines, path):
@@ -24,12 +26,13 @@ class LiloImage(object):
         self.lines = []
         self.path = path
         self.root = ""
-        map(self.set_from_line, lines)
+        for line in lines:
+            self.set_from_line(line)
 
     def set_from_line(self, line, replace = None):
         (com, arg) = GrubConf.grub_exact_split(line, 2)
 
-        if self.commands.has_key(com):
+        if com in self.commands:
             if self.commands[com] is not None:
                 setattr(self, self.commands[com], re.sub('^"(.+)"$', r"\1", arg.strip()))
             else:
@@ -97,7 +100,7 @@ class LiloConfigFile(object):
     def parse(self, buf = None):
         if buf is None:
             if self.filename is None:
-                raise ValueError, "No config file defined to parse!"
+                raise ValueError("No config file defined to parse!")
 
             f = open(self.filename, 'r')
             lines = f.readlines()
@@ -127,7 +130,7 @@ class LiloConfigFile(object):
                 continue
 
             (com, arg) = GrubConf.grub_exact_split(l, 2)
-            if self.commands.has_key(com):
+            if com in self.commands:
                 if self.commands[com] is not None:
                     setattr(self, self.commands[com], arg.strip())
                 else:
@@ -170,8 +173,8 @@ class LiloConfigFile(object):
 
 if __name__ == "__main__":
     if len(sys.argv) < 2:
-        raise RuntimeError, "Need a lilo.conf to read"
+        raise RuntimeError("Need a lilo.conf to read")
     g = LiloConfigFile(sys.argv[1])
     for i in g.images:
-        print i #, i.title, i.root, i.kernel, i.args, i.initrd
-    print g.default
+        print(i) #, i.title, i.root, i.kernel, i.args, i.initrd
+    print(g.default)
diff --git a/tools/pygrub/src/pygrub b/tools/pygrub/src/pygrub
index 1189b1ca48..dbdce315c6 100755
--- a/tools/pygrub/src/pygrub
+++ b/tools/pygrub/src/pygrub
@@ -12,13 +12,15 @@
 # along with this program; If not, see <http://www.gnu.org/licenses/>.
 #
 
+from __future__ import print_function
+
 import os, sys, string, struct, tempfile, re, traceback, stat, errno
 import copy
 import logging
 import platform
 import xen.lowlevel.xc
 
-import curses, _curses, curses.wrapper, curses.textpad, curses.ascii
+import curses, _curses, curses.textpad, curses.ascii
 import getopt
 
 import xenfsimage
@@ -77,7 +79,7 @@ def get_solaris_slice(file, offset):
     buf = os.read(fd, 512)
     os.close(fd)
     if struct.unpack("<H", buf[508:510])[0] != DKL_MAGIC:
-        raise RuntimeError, "Invalid disklabel magic"
+        raise RuntimeError("Invalid disklabel magic")
 
     nslices = struct.unpack("<H", buf[30:32])[0]
 
@@ -88,7 +90,7 @@ def get_solaris_slice(file, offset):
         if slicetag == V_ROOT:
             return slicesect * SECTOR_SIZE
 
-    raise RuntimeError, "No root slice found"      
+    raise RuntimeError("No root slice found")
 
 def get_fs_offset_gpt(file):
     fd = os.open(file, os.O_RDONLY)
@@ -423,20 +425,17 @@ class Grub:
         we're being given a raw config file rather than a disk image."""
         
         if not os.access(fn, os.R_OK):
-            raise RuntimeError, "Unable to access %s" %(fn,)
+            raise RuntimeError("Unable to access %s" %(fn,))
 
-        cfg_list = map(lambda x: (x,grub.GrubConf.Grub2ConfigFile),
-                       ["/boot/grub/grub.cfg", "/grub/grub.cfg",
-                        "/boot/grub2/grub.cfg", "/grub2/grub.cfg"]) + \
-                   map(lambda x: (x,grub.ExtLinuxConf.ExtLinuxConfigFile),
-                       ["/boot/isolinux/isolinux.cfg",
+        cfg_list = [(x,grub.GrubConf.Grub2ConfigFile) for x in ["/boot/grub/grub.cfg", "/grub/grub.cfg",
+                        "/boot/grub2/grub.cfg", "/grub2/grub.cfg"]] + \
+                   [(x,grub.ExtLinuxConf.ExtLinuxConfigFile) for x in ["/boot/isolinux/isolinux.cfg",
                         "/boot/extlinux/extlinux.conf",
                         "/boot/extlinux.conf",
                         "/extlinux/extlinux.conf",
-                        "/extlinux.conf"]) + \
-                   map(lambda x: (x,grub.GrubConf.GrubConfigFile),
-                       ["/boot/grub/menu.lst", "/boot/grub/grub.conf",
-                        "/grub/menu.lst", "/grub/grub.conf"])
+                        "/extlinux.conf"]] + \
+                   [(x,grub.GrubConf.GrubConfigFile) for x in ["/boot/grub/menu.lst", "/boot/grub/grub.conf",
+                        "/grub/menu.lst", "/grub/grub.conf"]]
 
         if not fs:
             # set the config file and parse it
@@ -448,12 +447,12 @@ class Grub:
 
         for f,parser in cfg_list:
             if fs.file_exists(f):
-                print >>sys.stderr, "Using %s to parse %s" % (parser,f)
+                print("Using %s to parse %s" % (parser,f), file=sys.stderr)
                 self.cf = parser()
                 self.cf.filename = f
                 break
         if self.__dict__.get('cf', None) is None:
-            raise RuntimeError, "couldn't find bootloader config file in the image provided."
+            raise RuntimeError("couldn't find bootloader config file in the image provided.")
         f = fs.open_file(self.cf.filename)
         # limit read size to avoid pathological cases
         buf = f.read(FS_READ_MAX)
@@ -628,11 +627,11 @@ def run_grub(file, entry, fs, cfg_args):
     if list_entries:
         for i in range(len(g.cf.images)):
             img = g.cf.images[i]
-            print "title: %s" % img.title
-            print "  root: %s" % img.root
-            print "  kernel: %s" % img.kernel[1]
-            print "  args: %s" % img.args
-            print "  initrd: %s" % img.initrd[1]
+            print("title: %s" % img.title)
+            print("  root: %s" % img.root)
+            print("  kernel: %s" % img.kernel[1])
+            print("  args: %s" % img.args)
+            print("  initrd: %s" % img.initrd[1])
 
     if interactive and not list_entries:
         curses.wrapper(run_main)
@@ -646,7 +645,7 @@ def run_grub(file, entry, fs, cfg_args):
            sel = idx
 
     if sel == -1:
-        print "No kernel image selected!"
+        print("No kernel image selected!")
         sys.exit(1)
 
     try:
@@ -731,7 +730,7 @@ def format_sxp(kernel, ramdisk, args):
 def format_simple(kernel, ramdisk, args, sep):
     for check in (kernel, ramdisk, args):
         if check is not None and sep in check:
-            raise RuntimeError, "simple format cannot represent delimiter-containing value"
+            raise RuntimeError("simple format cannot represent delimiter-containing value")
     s = ("kernel %s" % kernel) + sep
     if ramdisk:
         s += ("ramdisk %s" % ramdisk) + sep
@@ -744,7 +743,7 @@ if __name__ == "__main__":
     sel = None
     
     def usage():
-        print >> sys.stderr, "Usage: %s [-q|--quiet] [-i|--interactive] [-l|--list-entries] [-n|--not-really] [--output=] [--kernel=] [--ramdisk=] [--args=] [--entry=] [--output-directory=] [--output-format=sxp|simple|simple0] [--offset=] <image>" %(sys.argv[0],)
+        print("Usage: %s [-q|--quiet] [-i|--interactive] [-l|--list-entries] [-n|--not-really] [--output=] [--kernel=] [--ramdisk=] [--args=] [--entry=] [--output-directory=] [--output-format=sxp|simple|simple0] [--offset=] <image>" %(sys.argv[0],), file=sys.stderr)
 
     def copy_from_image(fs, file_to_read, file_type, output_directory,
                         not_really):
@@ -755,8 +754,8 @@ if __name__ == "__main__":
                 sys.exit("The requested %s file does not exist" % file_type)
         try:
             datafile = fs.open_file(file_to_read)
-        except Exception, e:
-            print >>sys.stderr, e
+        except Exception as e:
+            print(e, file=sys.stderr)
             sys.exit("Error opening %s in guest" % file_to_read)
         (tfd, ret) = tempfile.mkstemp(prefix="boot_"+file_type+".",
                                       dir=output_directory)
@@ -769,8 +768,8 @@ if __name__ == "__main__":
                 return ret
             try:
                 os.write(tfd, data)
-            except Exception, e:
-                print >>sys.stderr, e
+            except Exception as e:
+                print(e, file=sys.stderr)
                 os.close(tfd)
                 os.unlink(ret)
                 del datafile
@@ -834,7 +833,7 @@ if __name__ == "__main__":
             try:
                 part_offs = [ int(a) ]
             except ValueError:
-                print "offset value must be an integer"
+                print("offset value must be an integer")
                 usage()
                 sys.exit(1)
         elif o in ("--entry",):
@@ -847,13 +846,13 @@ if __name__ == "__main__":
             debug = True
         elif o in ("--output-format",):
             if a not in ["sxp", "simple", "simple0"]:
-                print "unknown output format %s" % a
+                print("unknown output format %s" % a)
                 usage()
                 sys.exit(1)
             output_format = a
         elif o in ("--output-directory",):
             if not os.path.isdir(a):
-                print "%s is not an existing directory" % a
+                print("%s is not an existing directory" % a)
                 sys.exit(1)
             output_directory = a
 
@@ -862,8 +861,8 @@ if __name__ == "__main__":
 
 
     try:
-        os.makedirs(output_directory, 0700)
-    except OSError,e:
+        os.makedirs(output_directory, 0o700)
+    except OSError as e:
         if (e.errno == errno.EEXIST) and os.path.isdir(output_directory):
             pass
         else:
@@ -877,10 +876,10 @@ if __name__ == "__main__":
     # debug
     if isconfig:
         chosencfg = run_grub(file, entry, fs, incfg["args"])
-        print "  kernel: %s" % chosencfg["kernel"]
+        print("  kernel: %s" % chosencfg["kernel"])
         if chosencfg["ramdisk"]:
-            print "  initrd: %s" % chosencfg["ramdisk"]
-        print "  args: %s" % chosencfg["args"]
+            print("  initrd: %s" % chosencfg["ramdisk"])
+        print("  args: %s" % chosencfg["args"])
         sys.exit(0)
 
     # if boot filesystem is set then pass to fsimage.open
@@ -926,7 +925,7 @@ if __name__ == "__main__":
 
     # Did looping through partitions find us a kernel?
     if fs is None:
-        raise RuntimeError, "Unable to find partition containing kernel"
+        raise RuntimeError("Unable to find partition containing kernel")
 
     bootcfg["kernel"] = copy_from_image(fs, chosencfg["kernel"], "kernel",
                                         output_directory, not_really)
-- 
2.11.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v3 7/7] pygrub/fsimage: make it work with python 3

[Wei Liu]

With the help of two porting guides and cpython source code:

1. Use PyBytes to replace PyString counterparts.
2. Use PyVarObject_HEAD_INIT.
3. Remove usage of Py_FindMethod.
4. Use new module initialisation routine.

For #3, Py_FindMethod was removed, yet an alternative wasn't
documented.  The code is the result of reverse-engineering cpython
commit 6116d4a1d1

https://docs.python.org/3/howto/cporting.html
http://python3porting.com/cextensions.html

Signed-off-by: Wei Liu <wei.liu2@citrix.com>
---
v3: use PyType_Ready to initialise types.
v2: use PyBytes.
---
 tools/pygrub/src/fsimage/fsimage.c | 92 ++++++++++++++++++++++++++++++++++----
 1 file changed, 84 insertions(+), 8 deletions(-)

diff --git a/tools/pygrub/src/fsimage/fsimage.c b/tools/pygrub/src/fsimage/fsimage.c
index 780207791c..057786d678 100644
--- a/tools/pygrub/src/fsimage/fsimage.c
+++ b/tools/pygrub/src/fsimage/fsimage.c
@@ -26,11 +26,15 @@
 #include <xenfsimage.h>
 #include <stdlib.h>
 
+#if PY_MAJOR_VERSION < 3
 #if (PYTHON_API_VERSION >= 1011)
 #define PY_PAD 0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L
 #else
 #define PY_PAD 0L,0L,0L,0L
 #endif
+#else
+#define PY_PAD 0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L
+#endif
 
 typedef struct fsimage_fs {
 	PyObject_HEAD
@@ -59,12 +63,24 @@ fsimage_file_read(fsimage_file_t *file, PyObject *args, PyObject *kwargs)
 
 	bufsize = size ? size : 4096;
 
-	if ((buffer = PyString_FromStringAndSize(NULL, bufsize)) == NULL)
+	buffer =
+#if PY_MAJOR_VERSION < 3
+		PyString_FromStringAndSize(NULL, bufsize);
+#else
+		PyBytes_FromStringAndSize(NULL, bufsize);
+#endif
+
+	if (buffer == NULL)
 		return (NULL);
  
 	while (1) {
 		int err;
-		void *buf = PyString_AS_STRING(buffer) + bytesread;
+		void *buf =
+#if PY_MAJOR_VERSION < 3
+			PyString_AS_STRING(buffer) + bytesread;
+#else
+			PyBytes_AS_STRING(buffer) + bytesread;
+#endif
 
 		err = fsi_pread_file(file->file, buf, bufsize,
 		    bytesread + offset);
@@ -84,12 +100,20 @@ fsimage_file_read(fsimage_file_t *file, PyObject *args, PyObject *kwargs)
 			if (bufsize == 0)
 				break;
 		} else {
+#if PY_MAJOR_VERSION < 3
 			if (_PyString_Resize(&buffer, bytesread + bufsize) < 0)
+#else
+			if (_PyBytes_Resize(&buffer, bytesread + bufsize) < 0)
+#endif
 				return (NULL);
 		}
 	}
 
+#if PY_MAJOR_VERSION < 3
 	_PyString_Resize(&buffer, bytesread);
+#else
+	_PyBytes_Resize(&buffer, bytesread);
+#endif
 	return (buffer);
 }
 
@@ -106,11 +130,13 @@ static struct PyMethodDef fsimage_file_methods[] = {
 	{ NULL, NULL, 0, NULL }	
 };
 
+#if PY_MAJOR_VERSION < 3
 static PyObject *
 fsimage_file_getattr(fsimage_file_t *file, char *name)
 {
 	return (Py_FindMethod(fsimage_file_methods, (PyObject *)file, name));
 }
+#endif
 
 static void
 fsimage_file_dealloc(fsimage_file_t *file)
@@ -123,14 +149,17 @@ fsimage_file_dealloc(fsimage_file_t *file)
 
 static char fsimage_file_type__doc__[] = "Filesystem image file";
 PyTypeObject fsimage_file_type = {
-	PyObject_HEAD_INIT(&PyType_Type)
-	0,					/* ob_size */
+	PyVarObject_HEAD_INIT(&PyType_Type, 0)
 	"xenfsimage.file",			/* tp_name */
 	sizeof(fsimage_file_t),			/* tp_size */
 	0,					/* tp_itemsize */
 	(destructor) fsimage_file_dealloc, 	/* tp_dealloc */
 	0,					/* tp_print */
+#if PY_MAJOR_VERSION < 3
 	(getattrfunc) fsimage_file_getattr, 	/* tp_getattr */
+#else
+	0,					/* tp_getattr */
+#endif
 	0,					/* tp_setattr */
 	0,					/* tp_compare */
 	0,					/* tp_repr */
@@ -144,7 +173,16 @@ PyTypeObject fsimage_file_type = {
 	0,					/* tp_setattro */
 	0,					/* tp_as_buffer */
 	Py_TPFLAGS_DEFAULT,			/* tp_flags */
-	fsimage_file_type__doc__,
+	fsimage_file_type__doc__,		/* tp_doc */
+#if PY_MAJOR_VERSION >= 3
+	0,					/* tp_traverse */
+	0,					/* tp_clear */
+	0,					/* tp_richcompare */
+	0,					/* tp_weaklistoffset */
+	0,					/* tp_iter */
+	0,					/* tp_iternext */
+	fsimage_file_methods,			/* tp_methods */
+#endif
 	PY_PAD
 };
 
@@ -208,11 +246,13 @@ static struct PyMethodDef fsimage_fs_methods[] = {
 	{ NULL, NULL, 0, NULL }	
 };
 
+#if PY_MAJOR_VERSION < 3
 static PyObject *
 fsimage_fs_getattr(fsimage_fs_t *fs, char *name)
 {
 	return (Py_FindMethod(fsimage_fs_methods, (PyObject *)fs, name));
 }
+#endif
 
 static void
 fsimage_fs_dealloc (fsimage_fs_t *fs)
@@ -225,14 +265,17 @@ fsimage_fs_dealloc (fsimage_fs_t *fs)
 PyDoc_STRVAR(fsimage_fs_type__doc__, "Filesystem image");
 
 PyTypeObject fsimage_fs_type = {
-	PyObject_HEAD_INIT(&PyType_Type)
-	0,					/* ob_size */
+	PyVarObject_HEAD_INIT(&PyType_Type, 0)
 	"xenfsimage.fs",			/* tp_name */
 	sizeof(fsimage_fs_t),			/* tp_size */
 	0,					/* tp_itemsize */
 	(destructor) fsimage_fs_dealloc, 	/* tp_dealloc */
 	0,					/* tp_print */
+#if PY_MAJOR_VERSION < 3
 	(getattrfunc) fsimage_fs_getattr, 	/* tp_getattr */
+#else
+	0,					/* tp_getattr */
+#endif
 	0,					/* tp_setattr */
 	0,					/* tp_compare */
 	0,					/* tp_repr */
@@ -246,7 +289,16 @@ PyTypeObject fsimage_fs_type = {
 	0,					/* tp_setattro */
 	0,					/* tp_as_buffer */
 	Py_TPFLAGS_DEFAULT,			/* tp_flags */
-	fsimage_fs_type__doc__,
+	fsimage_fs_type__doc__,			/* tp_doc */
+#if PY_MAJOR_VERSION >= 3
+	0,					/* tp_traverse */
+	0,					/* tp_clear */
+	0,					/* tp_richcompare */
+	0,					/* tp_weaklistoffset */
+	0,					/* tp_iter */
+	0,					/* tp_iternext */
+	fsimage_fs_methods,			/* tp_methods */
+#endif
 	PY_PAD
 };
 
@@ -309,8 +361,32 @@ static struct PyMethodDef fsimage_module_methods[] = {
 	{ NULL, NULL, 0, NULL }
 };
 
+#if PY_MAJOR_VERSION >= 3
+static struct PyModuleDef fsimage_module_def = {
+	PyModuleDef_HEAD_INIT,
+	"xenfsimage",		/* m_name */
+	"",			/* m_doc */
+	-1,			/* m_size */
+	fsimage_module_methods,	/* m_methods */
+	NULL,			/* m_reload */
+	NULL,			/* m_traverse */
+	NULL,			/* m_clear */
+	NULL			/* m_free */
+};
+#endif
+
 PyMODINIT_FUNC
+#if PY_MAJOR_VERSION >= 3
+PyInit_xenfsimage(void)
+#else
 initxenfsimage(void)
+#endif
 {
+#if PY_MAJOR_VERSION < 3
 	Py_InitModule("xenfsimage", fsimage_module_methods);
+#else
+	if (PyType_Ready(&fsimage_fs_type) < 0 || PyType_Ready(&fsimage_file_type) < 0)
+		return NULL;
+	return PyModule_Create(&fsimage_module_def);
+#endif
 }
-- 
2.11.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v3 1/7] Update python requirement

[Andrew Cooper]

On 11/03/2019 13:04, Wei Liu wrote:
> CentOS 5, which was the reason of the 2.4 restriction, is EOL. CentOS
> 6 ships 2.6.
>
> Bump the version to 2.6 in README. Now that all scripts are 3
> compatible, remove the restriction on python 2 as well.

I think the Python 2 restriction needs to stay until the end of the series.

Its fine to update the minimum to 2.6 before the changes are made, but
it isn't really ok to claim that everything is 3-compatible at this point.

~Andrew

>
> Update the check in configure.ac.
>
> Signed-off-by: Wei Liu <wei.liu2@citrix.com>


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v3 2/7] pygrub/fsimage: drop unused struct

[Andrew Cooper]

On 11/03/2019 13:04, Wei Liu wrote:
> Signed-off-by: Wei Liu <wei.liu2@citrix.com>

Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>

Good riddance.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v3 3/7] pygrub: change tabs into spaces

[Andrew Cooper]

On 11/03/2019 13:04, Wei Liu wrote:
> Not sure why Python 2 never complained, but Python 3 does.
>
> Change tabs to spaces.
>
> Signed-off-by: Wei Liu <wei.liu2@citrix.com>

Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>

Tabs vs spaces was an issue with Py2 which got fixed in 3 when it was
fine to break existing scripts.

The problem was that with certain editor settings, you could the
indentation looking correct while evaluating differently with Pythons
idea of indentation.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v3 5/7] libxl: make python scripts work with python 2.6 and up

[Andrew Cooper]

On 11/03/2019 13:04, Wei Liu wrote:
> Go through transformations suggested by 2to3 and pick the necessary
> ones.
>
> Signed-off-by: Wei Liu <wei.liu2@citrix.com>

Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v3 6/7] pygrub: make python scripts work with 2.6 and up

[Andrew Cooper]

On 11/03/2019 13:04, Wei Liu wrote:
> Run 2to3 and pick the sensible suggestions.
>
> Import print_function and absolute_import so 2.6 can work.
>
> There has never been a curses.wrapper module according to 2.x and 3.x
> doc, only a function, so "import curses.wrapper" is not correct. It
> happened to work because 2.x implemented a (undocumented) module.
>
> We only need to import curses to make curses.wrapper available to
> pygrub.
>
> Signed-off-by: Wei Liu <wei.liu2@citrix.com>

Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v3 1/7] Update python requirement

[Wei Liu]

On Mon, Mar 11, 2019 at 01:08:11PM +0000, Andrew Cooper wrote:
> On 11/03/2019 13:04, Wei Liu wrote:
> > CentOS 5, which was the reason of the 2.4 restriction, is EOL. CentOS
> > 6 ships 2.6.
> >
> > Bump the version to 2.6 in README. Now that all scripts are 3
> > compatible, remove the restriction on python 2 as well.
> 
> I think the Python 2 restriction needs to stay until the end of the series.
> 
> Its fine to update the minimum to 2.6 before the changes are made, but
> it isn't really ok to claim that everything is 3-compatible at this point.

It was at the end of the series in previous version. I forgot to update
the description.

I don't care if it is the first or last patch, really...

Wei.

> 
> ~Andrew
> 
> >
> > Update the check in configure.ac.
> >
> > Signed-off-by: Wei Liu <wei.liu2@citrix.com>
> 

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v3 8/7] pygrub/fsimage: use named initialisation in extension

[Wei Liu]

... and drop PY_PAD.

Signed-off-by: Wei Liu <wei.liu2@citrix.com>
---
This can be squashed into previous patch.
---
 tools/pygrub/src/fsimage/fsimage.c | 47 ++++++--------------------------------
 1 file changed, 7 insertions(+), 40 deletions(-)

diff --git a/tools/pygrub/src/fsimage/fsimage.c b/tools/pygrub/src/fsimage/fsimage.c
index cf8089a872..5483aefbfe 100644
--- a/tools/pygrub/src/fsimage/fsimage.c
+++ b/tools/pygrub/src/fsimage/fsimage.c
@@ -26,16 +26,6 @@
 #include <xenfsimage.h>
 #include <stdlib.h>
 
-#if PY_MAJOR_VERSION < 3
-#if (PYTHON_API_VERSION >= 1011)
-#define PY_PAD 0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L
-#else
-#define PY_PAD 0L,0L,0L,0L
-#endif
-#else
-#define PY_PAD 0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L
-#endif
-
 typedef struct fsimage_fs {
 	PyObject_HEAD
 	fsi_t *fs;
@@ -150,40 +140,17 @@ fsimage_file_dealloc(fsimage_file_t *file)
 static char fsimage_file_type__doc__[] = "Filesystem image file";
 PyTypeObject fsimage_file_type = {
 	PyVarObject_HEAD_INIT(&PyType_Type, 0)
-	"xenfsimage.file",			/* tp_name */
-	sizeof(fsimage_file_t),			/* tp_size */
-	0,					/* tp_itemsize */
-	(destructor) fsimage_file_dealloc, 	/* tp_dealloc */
-	0,					/* tp_print */
+	.tp_name = "xenfsimage.file",
+	.tp_basicsize = sizeof(fsimage_file_t),
+	.tp_dealloc = (destructor) fsimage_file_dealloc,
 #if PY_MAJOR_VERSION < 3
-	(getattrfunc) fsimage_file_getattr, 	/* tp_getattr */
-#else
-	0,					/* tp_getattr */
+	.tp_getattr = (getattrfunc) fsimage_file_getattr,
 #endif
-	0,					/* tp_setattr */
-	0,					/* tp_compare */
-	0,					/* tp_repr */
-	0,					/* tp_as_number */
-	0,	 				/* tp_as_sequence */
-	0,					/* tp_as_mapping */
-	0,	   				/* tp_hash */
-	0,					/* tp_call */
-	0,					/* tp_str */
-	0,					/* tp_getattro */
-	0,					/* tp_setattro */
-	0,					/* tp_as_buffer */
-	Py_TPFLAGS_DEFAULT,			/* tp_flags */
-	fsimage_file_type__doc__,		/* tp_doc */
+	.tp_flags = Py_TPFLAGS_DEFAULT,
+	.tp_doc = fsimage_file_type__doc__,
 #if PY_MAJOR_VERSION >= 3
-	0,					/* tp_traverse */
-	0,					/* tp_clear */
-	0,					/* tp_richcompare */
-	0,					/* tp_weaklistoffset */
-	0,					/* tp_iter */
-	0,					/* tp_iternext */
-	fsimage_file_methods,			/* tp_methods */
+	.tp_methods = fsimage_file_methods,
 #endif
-	PY_PAD
 };
 
 static PyObject *
-- 
2.11.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v3 8/7] pygrub/fsimage: use named initialisation in extension

[Andrew Cooper]

On 11/03/2019 14:41, Wei Liu wrote:
> ... and drop PY_PAD.
>
> Signed-off-by: Wei Liu <wei.liu2@citrix.com>

Much nicer.  With this suitably folded, Reviewed-by: Andrew Cooper
<andrew.cooper3@citrix.com>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel