[PATCH 0/8] Python 3 bindings

[PATCH 0/8] Python 3 bindings

[Marek Marczykowski-Górecki]

This patch series add support for Python3 in tools/python. As most python
modules, it looks at $PYTHON variable to what python use, so to use this
PYTHON=python3 needs to be defined in a build environment. If both python2 and
python3 versions are needed, then python bindings needs to be built twice. In
most cases it will be packaging scripts responsibility.

Patches 1-4 are cleanups not really specific to python3, but making it easier.

Marek Marczykowski-Górecki (8):
  python: check return value of PyErr_NewException
  python: drop tp_getattr implementation
  python: use Py_TYPE instead of looking directly into PyObject_HEAD
  python: initialize specific fields of PyTypeObject
  python: use PyBytes/PyUnicode instead of PyString
  python: use PyLong_* for constructing 'int' type in Python3
  python: adjust module initalization for Python3
  python: handle long type in scripts

 tools/python/xen/lowlevel/xc/xc.c   | 148 +++++++++++++++++++-----------------
 tools/python/xen/lowlevel/xs/xs.c   | 118 ++++++++++++++++------------
 tools/python/xen/migration/libxc.py |  32 ++++----
 3 files changed, 165 insertions(+), 133 deletions(-)

-- 
2.7.4


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[PATCH 1/8] python: check return value of PyErr_NewException

[Marek Marczykowski-Górecki]

Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
---
 tools/python/xen/lowlevel/xc/xc.c | 4 ++++
 tools/python/xen/lowlevel/xs/xs.c | 4 ++++
 2 files changed, 8 insertions(+)

diff --git a/tools/python/xen/lowlevel/xc/xc.c b/tools/python/xen/lowlevel/xc/xc.c
index 39be1d5..7fbead5 100644
--- a/tools/python/xen/lowlevel/xc/xc.c
+++ b/tools/python/xen/lowlevel/xc/xc.c
@@ -2735,6 +2735,10 @@ PyMODINIT_FUNC initxc(void)
       return;
 
     xc_error_obj = PyErr_NewException(PKG ".Error", PyExc_RuntimeError, NULL);
+    if (xc_error_obj == NULL) {
+        Py_DECREF(m);
+        return;
+    }
     zero = PyInt_FromLong(0);
 
     /* KAF: This ensures that we get debug output in a timely manner. */
diff --git a/tools/python/xen/lowlevel/xs/xs.c b/tools/python/xen/lowlevel/xs/xs.c
index a86edbe..5772f4b 100644
--- a/tools/python/xen/lowlevel/xs/xs.c
+++ b/tools/python/xen/lowlevel/xs/xs.c
@@ -987,6 +987,10 @@ PyMODINIT_FUNC initxs(void)
       return;
 
     xs_error = PyErr_NewException(PKG ".Error", PyExc_RuntimeError, NULL);
+    if (xs_error == NULL) {
+        Py_DECREF(m);
+        return;
+    }
 
     Py_INCREF(&xshandle_type);
     PyModule_AddObject(m, CLS, (PyObject *)&xshandle_type);
-- 
2.7.4


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[PATCH 2/8] python: drop tp_getattr implementation

[Marek Marczykowski-Górecki]

tp_getattr method of type object is deprecated already in Python2 and
gone in Python3. Default implementation does the same as this custom one.

Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
---
 tools/python/xen/lowlevel/xc/xc.c | 7 +------
 tools/python/xen/lowlevel/xs/xs.c | 7 +------
 2 files changed, 2 insertions(+), 12 deletions(-)

diff --git a/tools/python/xen/lowlevel/xc/xc.c b/tools/python/xen/lowlevel/xc/xc.c
index 7fbead5..109ef57 100644
--- a/tools/python/xen/lowlevel/xc/xc.c
+++ b/tools/python/xen/lowlevel/xc/xc.c
@@ -2640,11 +2640,6 @@ static PyMethodDef pyxc_methods[] = {
 };
 
 
-static PyObject *PyXc_getattr(PyObject *obj, char *name)
-{
-    return Py_FindMethod(pyxc_methods, obj, name);
-}
-
 static PyObject *PyXc_new(PyTypeObject *type, PyObject *args, PyObject *kwds)
 {
     XcObject *self = (XcObject *)type->tp_alloc(type, 0);
@@ -2686,7 +2681,7 @@ static PyTypeObject PyXcType = {
     0,
     (destructor)PyXc_dealloc,     /* tp_dealloc        */
     NULL,                         /* tp_print          */
-    PyXc_getattr,                 /* tp_getattr        */
+    NULL,                         /* tp_getattr        */
     NULL,                         /* tp_setattr        */
     NULL,                         /* tp_compare        */
     NULL,                         /* tp_repr           */
diff --git a/tools/python/xen/lowlevel/xs/xs.c b/tools/python/xen/lowlevel/xs/xs.c
index 5772f4b..e9eef73 100644
--- a/tools/python/xen/lowlevel/xs/xs.c
+++ b/tools/python/xen/lowlevel/xs/xs.c
@@ -870,11 +870,6 @@ static PyMethodDef xshandle_methods[] = {
     { NULL /* Sentinel. */ },
 };
 
-static PyObject *xshandle_getattr(PyObject *self, char *name)
-{
-    return Py_FindMethod(xshandle_methods, self, name);
-}
-
 static PyObject *
 xshandle_new(PyTypeObject *type, PyObject *args, PyObject *kwds)
 {
@@ -938,7 +933,7 @@ static PyTypeObject xshandle_type = {
     0,
     (destructor)xshandle_dealloc, /* tp_dealloc        */
     NULL,                         /* tp_print          */
-    xshandle_getattr,             /* tp_getattr        */
+    NULL,                         /* tp_getattr        */
     NULL,                         /* tp_setattr        */
     NULL,                         /* tp_compare        */
     NULL,                         /* tp_repr           */
-- 
2.7.4


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[PATCH 3/8] python: use Py_TYPE instead of looking directly into PyObject_HEAD

[Marek Marczykowski-Górecki]

Py_TYPE works on both Python2 and Python3, while internals of
PyObject_HEAD have changed.

Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
---
 tools/python/xen/lowlevel/xc/xc.c | 2 +-
 tools/python/xen/lowlevel/xs/xs.c | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/tools/python/xen/lowlevel/xc/xc.c b/tools/python/xen/lowlevel/xc/xc.c
index 109ef57..75842ef 100644
--- a/tools/python/xen/lowlevel/xc/xc.c
+++ b/tools/python/xen/lowlevel/xc/xc.c
@@ -2670,7 +2670,7 @@ static void PyXc_dealloc(XcObject *self)
         self->xc_handle = NULL;
     }
 
-    self->ob_type->tp_free((PyObject *)self);
+    Py_TYPE(self)->tp_free((PyObject *)self);
 }
 
 static PyTypeObject PyXcType = {
diff --git a/tools/python/xen/lowlevel/xs/xs.c b/tools/python/xen/lowlevel/xs/xs.c
index e9eef73..74a80ca 100644
--- a/tools/python/xen/lowlevel/xs/xs.c
+++ b/tools/python/xen/lowlevel/xs/xs.c
@@ -922,7 +922,7 @@ static void xshandle_dealloc(XsHandle *self)
 
     Py_XDECREF(self->watches);
 
-    self->ob_type->tp_free((PyObject *)self);
+    Py_TYPE(self)->tp_free((PyObject *)self);
 }
 
 static PyTypeObject xshandle_type = {
-- 
2.7.4


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[PATCH 4/8] python: initialize specific fields of PyTypeObject

[Marek Marczykowski-Górecki]

Fields not named here will be zero-initialized anyway, but using this
way will be much easier to support both Python2 and Python3.

Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
---
 tools/python/xen/lowlevel/xc/xc.c | 47 ++++++++-------------------------------
 tools/python/xen/lowlevel/xs/xs.c | 47 ++++++++-------------------------------
 2 files changed, 18 insertions(+), 76 deletions(-)

diff --git a/tools/python/xen/lowlevel/xc/xc.c b/tools/python/xen/lowlevel/xc/xc.c
index 75842ef..bcbb7b0 100644
--- a/tools/python/xen/lowlevel/xc/xc.c
+++ b/tools/python/xen/lowlevel/xc/xc.c
@@ -2675,44 +2675,15 @@ static void PyXc_dealloc(XcObject *self)
 
 static PyTypeObject PyXcType = {
     PyObject_HEAD_INIT(NULL)
-    0,
-    PKG "." CLS,
-    sizeof(XcObject),
-    0,
-    (destructor)PyXc_dealloc,     /* tp_dealloc        */
-    NULL,                         /* tp_print          */
-    NULL,                         /* tp_getattr        */
-    NULL,                         /* tp_setattr        */
-    NULL,                         /* tp_compare        */
-    NULL,                         /* tp_repr           */
-    NULL,                         /* tp_as_number      */
-    NULL,                         /* tp_as_sequence    */
-    NULL,                         /* tp_as_mapping     */
-    NULL,                         /* tp_hash           */
-    NULL,                         /* tp_call           */
-    NULL,                         /* tp_str            */
-    NULL,                         /* tp_getattro       */
-    NULL,                         /* tp_setattro       */
-    NULL,                         /* tp_as_buffer      */
-    Py_TPFLAGS_DEFAULT,           /* tp_flags          */
-    "Xen client connections",     /* tp_doc            */
-    NULL,                         /* tp_traverse       */
-    NULL,                         /* tp_clear          */
-    NULL,                         /* tp_richcompare    */
-    0,                            /* tp_weaklistoffset */
-    NULL,                         /* tp_iter           */
-    NULL,                         /* tp_iternext       */
-    pyxc_methods,                 /* tp_methods        */
-    NULL,                         /* tp_members        */
-    NULL,                         /* tp_getset         */
-    NULL,                         /* tp_base           */
-    NULL,                         /* tp_dict           */
-    NULL,                         /* tp_descr_get      */
-    NULL,                         /* tp_descr_set      */
-    0,                            /* tp_dictoffset     */
-    (initproc)PyXc_init,          /* tp_init           */
-    NULL,                         /* tp_alloc          */
-    PyXc_new,                     /* tp_new            */
+    .tp_name = PKG "." CLS,
+    .tp_basicsize = sizeof(XcObject),
+    .tp_itemsize = 0,
+    .tp_dealloc = (destructor)PyXc_dealloc,
+    .tp_flags = Py_TPFLAGS_DEFAULT,
+    .tp_doc = "Xen client connections",
+    .tp_methods = pyxc_methods,
+    .tp_init = (initproc)PyXc_init,
+    .tp_new = PyXc_new,
 };
 
 static PyMethodDef xc_methods[] = { { NULL } };
diff --git a/tools/python/xen/lowlevel/xs/xs.c b/tools/python/xen/lowlevel/xs/xs.c
index 74a80ca..66ab08d 100644
--- a/tools/python/xen/lowlevel/xs/xs.c
+++ b/tools/python/xen/lowlevel/xs/xs.c
@@ -927,44 +927,15 @@ static void xshandle_dealloc(XsHandle *self)
 
 static PyTypeObject xshandle_type = {
     PyObject_HEAD_INIT(NULL)
-    0,
-    PKG "." CLS,
-    sizeof(XsHandle),
-    0,
-    (destructor)xshandle_dealloc, /* tp_dealloc        */
-    NULL,                         /* tp_print          */
-    NULL,                         /* tp_getattr        */
-    NULL,                         /* tp_setattr        */
-    NULL,                         /* tp_compare        */
-    NULL,                         /* tp_repr           */
-    NULL,                         /* tp_as_number      */
-    NULL,                         /* tp_as_sequence    */
-    NULL,                         /* tp_as_mapping     */
-    NULL,                         /* tp_hash           */
-    NULL,                         /* tp_call           */
-    NULL,                         /* tp_str            */
-    NULL,                         /* tp_getattro       */
-    NULL,                         /* tp_setattro       */
-    NULL,                         /* tp_as_buffer      */
-    Py_TPFLAGS_DEFAULT,           /* tp_flags          */
-    "Xenstore connections",       /* tp_doc            */
-    NULL,                         /* tp_traverse       */
-    NULL,                         /* tp_clear          */
-    NULL,                         /* tp_richcompare    */
-    0,                            /* tp_weaklistoffset */
-    NULL,                         /* tp_iter           */
-    NULL,                         /* tp_iternext       */
-    xshandle_methods,             /* tp_methods        */
-    NULL,                         /* tp_members        */
-    NULL,                         /* tp_getset         */
-    NULL,                         /* tp_base           */
-    NULL,                         /* tp_dict           */
-    NULL,                         /* tp_descr_get      */
-    NULL,                         /* tp_descr_set      */
-    0,                            /* tp_dictoffset     */
-    (initproc)xshandle_init,      /* tp_init           */
-    NULL,                         /* tp_alloc          */
-    xshandle_new,                 /* tp_new            */
+    .tp_name = PKG "." CLS,
+    .tp_basicsize = sizeof(XsHandle),
+    .tp_itemsize = 0,
+    .tp_dealloc = (destructor)xshandle_dealloc,
+    .tp_flags = Py_TPFLAGS_DEFAULT,
+    .tp_doc = "Xenstore connections",
+    .tp_methods = xshandle_methods,
+    .tp_init = (initproc)xshandle_init,
+    .tp_new = xshandle_new,
 };
 
 static PyMethodDef xs_methods[] = { { NULL } };
-- 
2.7.4


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[PATCH 5/8] python: use PyBytes/PyUnicode instead of PyString

[Marek Marczykowski-Górecki]

In Python2 PyBytes is the same as PyString, but in Python3 PyString is
gone and 'str' is really PyUnicode in C-API.
When handling arbitrary data, use PyBytes - which is the right thing to
do in Python3, and pose no API change in Python2. When handling
xenstore paths and transaction ids, which have well defined format, use
PyUnicode - to ease API usage - no need to prefix all xenstore paths
with 'b' when migrating scripts to Python3.

Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
---
 tools/python/xen/lowlevel/xc/xc.c |  6 +++---
 tools/python/xen/lowlevel/xs/xs.c | 20 ++++++++++++++++----
 2 files changed, 19 insertions(+), 7 deletions(-)

diff --git a/tools/python/xen/lowlevel/xc/xc.c b/tools/python/xen/lowlevel/xc/xc.c
index bcbb7b0..ce87afb 100644
--- a/tools/python/xen/lowlevel/xc/xc.c
+++ b/tools/python/xen/lowlevel/xc/xc.c
@@ -678,7 +678,7 @@ static void pyxc_dom_extract_cpuid(PyObject *config,
 
     for ( i = 0; i < 4; i++ )
         if ( (obj = PyDict_GetItemString(config, regs_extract[i])) != NULL )
-            regs[i] = PyString_AS_STRING(obj);
+            regs[i] = PyBytes_AS_STRING(obj);
 }
 
 static PyObject *pyxc_create_cpuid_dict(char **regs)
@@ -693,7 +693,7 @@ static PyObject *pyxc_create_cpuid_dict(char **regs)
        if ( regs[i] == NULL )
            continue;
        PyDict_SetItemString(dict, regs_extract[i],
-                            PyString_FromString(regs[i]));
+                            PyBytes_FromString(regs[i]));
        free(regs[i]);
        regs[i] = NULL;
    }
@@ -940,7 +940,7 @@ static PyObject *pyxc_readconsolering(XcObject *self,
         str = ptr;
     }
 
-    obj = PyString_FromStringAndSize(str, count);
+    obj = PyBytes_FromStringAndSize(str, count);
     free(str);
     return obj;
 }
diff --git a/tools/python/xen/lowlevel/xs/xs.c b/tools/python/xen/lowlevel/xs/xs.c
index 66ab08d..c2b4d87 100644
--- a/tools/python/xen/lowlevel/xs/xs.c
+++ b/tools/python/xen/lowlevel/xs/xs.c
@@ -103,7 +103,7 @@ static PyObject *xspy_read(XsHandle *self, PyObject *args)
     xsval = xs_read(xh, th, path, &xsval_n);
     Py_END_ALLOW_THREADS
     if (xsval) {
-        PyObject *val = PyString_FromStringAndSize(xsval, xsval_n);
+        PyObject *val = PyBytes_FromStringAndSize(xsval, xsval_n);
         free(xsval);
         return val;
     }
@@ -179,7 +179,11 @@ static PyObject *xspy_ls(XsHandle *self, PyObject *args)
         int i;
         PyObject *val = PyList_New(xsval_n);
         for (i = 0; i < xsval_n; i++)
-            PyList_SetItem(val, i, PyString_FromString(xsval[i]));
+#if PY_MAJOR_VERSION >= 3
+            PyList_SetItem(val, i, PyUnicode_FromString(xsval[i]));
+#else
+            PyList_SetItem(val, i, PyBytes_FromString(xsval[i]));
+#endif
         free(xsval);
         return val;
     }
@@ -550,7 +554,11 @@ static PyObject *xspy_transaction_start(XsHandle *self)
     }
 
     snprintf(thstr, sizeof(thstr), "%lX", (unsigned long)th);
-    return PyString_FromString(thstr);
+#if PY_MAJOR_VERSION >= 3
+    return PyUnicode_FromString(thstr);
+#else
+    return PyBytes_FromString(thstr);
+#endif
 }
 
 #define xspy_transaction_end_doc "\n"					\
@@ -773,7 +781,11 @@ static PyObject *xspy_get_domain_path(XsHandle *self, PyObject *args)
     Py_END_ALLOW_THREADS
 
     if (xsval) {
-        PyObject *val = PyString_FromString(xsval);
+#if PY_MAJOR_VERSION >= 3
+        PyObject *val = PyUnicode_FromString(xsval);
+#else
+        PyObject *val = PyBytes_FromString(xsval);
+#endif
         free(xsval);
         return val;
     }
-- 
2.7.4


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[PATCH 6/8] python: use PyLong_* for constructing 'int' type in Python3

[Marek Marczykowski-Górecki]

In Python3 'int' and 'long' types are the same, there are no longer
separate PyInt_* functions.  Provide convenient #defines to limit #if in
code.

Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
---
 tools/python/xen/lowlevel/xc/xc.c | 51 ++++++++++++++++++++++++---------------
 tools/python/xen/lowlevel/xs/xs.c |  8 ++++++
 2 files changed, 39 insertions(+), 20 deletions(-)

diff --git a/tools/python/xen/lowlevel/xc/xc.c b/tools/python/xen/lowlevel/xc/xc.c
index ce87afb..df31a1d 100644
--- a/tools/python/xen/lowlevel/xc/xc.c
+++ b/tools/python/xen/lowlevel/xc/xc.c
@@ -34,6 +34,17 @@
 
 #define FLASK_CTX_LEN 1024
 
+/* Python 2 compatibility */
+#if PY_MAJOR_VERSION >= 3
+#define PyLongOrInt_FromLong PyLong_FromLong
+#define PyLongOrInt_Check PyLong_Check
+#define PyLongOrInt_AsLong PyLong_AsLong
+#else
+#define PyLongOrInt_FromLong PyInt_FromLong
+#define PyLongOrInt_Check PyInt_Check
+#define PyLongOrInt_AsLong PyInt_AsLong
+#endif
+
 static PyObject *xc_error_obj, *zero;
 
 typedef struct {
@@ -127,9 +138,9 @@ static PyObject *pyxc_domain_create(XcObject *self,
         for ( i = 0; i < sizeof(xen_domain_handle_t); i++ )
         {
             PyObject *p = PyList_GetItem(pyhandle, i);
-            if ( !PyInt_Check(p) )
+            if ( !PyLongOrInt_Check(p) )
                 goto out_exception;
-            handle[i] = (uint8_t)PyInt_AsLong(p);
+            handle[i] = (uint8_t)PyLongOrInt_AsLong(p);
         }
     }
 
@@ -142,7 +153,7 @@ static PyObject *pyxc_domain_create(XcObject *self,
             return pyxc_error_to_exception(self->xc_handle);
 
 
-    return PyInt_FromLong(dom);
+    return PyLongOrInt_FromLong(dom);
 
 out_exception:
     errno = EINVAL;
@@ -242,7 +253,7 @@ static PyObject *pyxc_vcpu_setaffinity(XcObject *self,
     {
         for ( i = 0; i < PyList_Size(cpulist); i++ ) 
         {
-            long cpu = PyInt_AsLong(PyList_GetItem(cpulist, i));
+            long cpu = PyLongOrInt_AsLong(PyList_GetItem(cpulist, i));
             if ( cpu < 0 || cpu >= nr_cpus )
             {
                 free(cpumap);
@@ -284,9 +295,9 @@ static PyObject *pyxc_domain_sethandle(XcObject *self, PyObject *args)
     for ( i = 0; i < sizeof(xen_domain_handle_t); i++ )
     {
         PyObject *p = PyList_GetItem(pyhandle, i);
-        if ( !PyInt_Check(p) )
+        if ( !PyLongOrInt_Check(p) )
             goto out_exception;
-        handle[i] = (uint8_t)PyInt_AsLong(p);
+        handle[i] = (uint8_t)PyLongOrInt_AsLong(p);
     }
 
     if (xc_domain_sethandle(self->xc_handle, dom, handle) < 0)
@@ -361,7 +372,7 @@ static PyObject *pyxc_domain_getinfo(XcObject *self,
             return NULL;
         }
         for ( j = 0; j < sizeof(xen_domain_handle_t); j++ )
-            PyList_SetItem(pyhandle, j, PyInt_FromLong(info[i].handle[j]));
+            PyList_SetItem(pyhandle, j, PyLongOrInt_FromLong(info[i].handle[j]));
         PyDict_SetItemString(info_dict, "handle", pyhandle);
         Py_DECREF(pyhandle);
         PyList_SetItem(list, i, info_dict);
@@ -420,7 +431,7 @@ static PyObject *pyxc_vcpu_getinfo(XcObject *self,
     for ( i = 0; i < nr_cpus; i++ )
     {
         if (*(cpumap + i / 8) & 1 ) {
-            PyObject *pyint = PyInt_FromLong(i);
+            PyObject *pyint = PyLongOrInt_FromLong(i);
             PyList_Append(cpulist, pyint);
             Py_DECREF(pyint);
         }
@@ -836,7 +847,7 @@ static PyObject *pyxc_evtchn_alloc_unbound(XcObject *self,
     if ( (port = xc_evtchn_alloc_unbound(self->xc_handle, dom, remote_dom)) < 0 )
         return pyxc_error_to_exception(self->xc_handle);
 
-    return PyInt_FromLong(port);
+    return PyLongOrInt_FromLong(port);
 }
 
 static PyObject *pyxc_evtchn_reset(XcObject *self,
@@ -1063,7 +1074,7 @@ static PyObject *pyxc_topologyinfo(XcObject *self)
         }
         else
         {
-            PyObject *pyint = PyInt_FromLong(cputopo[i].core);
+            PyObject *pyint = PyLongOrInt_FromLong(cputopo[i].core);
             PyList_Append(cpu_to_core_obj, pyint);
             Py_DECREF(pyint);
         }
@@ -1074,7 +1085,7 @@ static PyObject *pyxc_topologyinfo(XcObject *self)
         }
         else
         {
-            PyObject *pyint = PyInt_FromLong(cputopo[i].socket);
+            PyObject *pyint = PyLongOrInt_FromLong(cputopo[i].socket);
             PyList_Append(cpu_to_socket_obj, pyint);
             Py_DECREF(pyint);
         }
@@ -1085,7 +1096,7 @@ static PyObject *pyxc_topologyinfo(XcObject *self)
         }
         else
         {
-            PyObject *pyint = PyInt_FromLong(cputopo[i].node);
+            PyObject *pyint = PyLongOrInt_FromLong(cputopo[i].node);
             PyList_Append(cpu_to_node_obj, pyint);
             Py_DECREF(pyint);
         }
@@ -1139,18 +1150,18 @@ static PyObject *pyxc_numainfo(XcObject *self)
         unsigned invalid_node;
 
         /* Total Memory */
-        pyint = PyInt_FromLong(meminfo[i].memsize >> 20); /* MB */
+        pyint = PyLongOrInt_FromLong(meminfo[i].memsize >> 20); /* MB */
         PyList_Append(node_to_memsize_obj, pyint);
         Py_DECREF(pyint);
 
         /* Free Memory */
-        pyint = PyInt_FromLong(meminfo[i].memfree >> 20); /* MB */
+        pyint = PyLongOrInt_FromLong(meminfo[i].memfree >> 20); /* MB */
         PyList_Append(node_to_memfree_obj, pyint);
         Py_DECREF(pyint);
 
         /* DMA memory. */
         xc_availheap(self->xc_handle, 0, 32, i, &free_heap);
-        pyint = PyInt_FromLong(free_heap >> 20); /* MB */
+        pyint = PyLongOrInt_FromLong(free_heap >> 20); /* MB */
         PyList_Append(node_to_dma32_mem_obj, pyint);
         Py_DECREF(pyint);
 
@@ -1166,7 +1177,7 @@ static PyObject *pyxc_numainfo(XcObject *self)
             }
             else
             {
-                pyint = PyInt_FromLong(dist);
+                pyint = PyLongOrInt_FromLong(dist);
                 PyList_Append(node_to_node_dist_obj, pyint);
                 Py_DECREF(pyint);
             }
@@ -1700,7 +1711,7 @@ static PyObject *cpumap_to_cpulist(XcObject *self, xc_cpumap_t cpumap)
     {
         if ( *cpumap & (1 << (i % 8)) )
         {
-            PyObject* pyint = PyInt_FromLong(i);
+            PyObject* pyint = PyLongOrInt_FromLong(i);
 
             PyList_Append(cpulist, pyint);
             Py_DECREF(pyint);
@@ -1726,7 +1737,7 @@ static PyObject *pyxc_cpupool_create(XcObject *self,
     if ( xc_cpupool_create(self->xc_handle, &cpupool, sched) < 0 )
         return pyxc_error_to_exception(self->xc_handle);
 
-    return PyInt_FromLong(cpupool);
+    return PyLongOrInt_FromLong(cpupool);
 }
 
 static PyObject *pyxc_cpupool_destroy(XcObject *self,
@@ -1882,7 +1893,7 @@ static PyObject *pyflask_context_to_sid(PyObject *self, PyObject *args,
         return PyErr_SetFromErrno(xc_error_obj);
     }
 
-    return PyInt_FromLong(sid);
+    return PyLongOrInt_FromLong(sid);
 }
 
 static PyObject *pyflask_sid_to_context(PyObject *self, PyObject *args,
@@ -2705,7 +2716,7 @@ PyMODINIT_FUNC initxc(void)
         Py_DECREF(m);
         return;
     }
-    zero = PyInt_FromLong(0);
+    zero = PyLongOrInt_FromLong(0);
 
     /* KAF: This ensures that we get debug output in a timely manner. */
     setbuf(stdout, NULL);
diff --git a/tools/python/xen/lowlevel/xs/xs.c b/tools/python/xen/lowlevel/xs/xs.c
index c2b4d87..b37daa9 100644
--- a/tools/python/xen/lowlevel/xs/xs.c
+++ b/tools/python/xen/lowlevel/xs/xs.c
@@ -43,6 +43,14 @@
 #define PKG "xen.lowlevel.xs"
 #define CLS "xs"
 
+#if PY_MAJOR_VERSION < 3
+/* Python 2 compatibility */
+#define PyLong_FromLong PyInt_FromLong
+#undef PyLong_Check
+#define PyLong_Check PyInt_Check
+#define PyLong_AsLong PyInt_AsLong
+#endif
+
 static PyObject *xs_error;
 
 /** Python wrapper round an xs handle.
-- 
2.7.4


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[PATCH 7/8] python: adjust module initalization for Python3

[Marek Marczykowski-Górecki]

In Python3, PyTypeObject looks slightly different, and also module
initialization is different. Instead of Py_InitModule, PyModule_Create
should be called on already defined PyModuleDef structure. And then
initialization function should return that module.

Additionally initialization function should be named PyInit_<name>,
instead of init<name>.

Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
---
 tools/python/xen/lowlevel/xc/xc.c | 35 ++++++++++++++++++++++++++++++++---
 tools/python/xen/lowlevel/xs/xs.c | 34 +++++++++++++++++++++++++++++++---
 2 files changed, 63 insertions(+), 6 deletions(-)

diff --git a/tools/python/xen/lowlevel/xc/xc.c b/tools/python/xen/lowlevel/xc/xc.c
index df31a1d..d09c45f 100644
--- a/tools/python/xen/lowlevel/xc/xc.c
+++ b/tools/python/xen/lowlevel/xc/xc.c
@@ -2685,7 +2685,11 @@ static void PyXc_dealloc(XcObject *self)
 }
 
 static PyTypeObject PyXcType = {
+#if PY_MAJOR_VERSION >= 3
+    .ob_base = { PyObject_HEAD_INIT(NULL) },
+#else
     PyObject_HEAD_INIT(NULL)
+#endif
     .tp_name = PKG "." CLS,
     .tp_basicsize = sizeof(XcObject),
     .tp_itemsize = 0,
@@ -2699,22 +2703,44 @@ static PyTypeObject PyXcType = {
 
 static PyMethodDef xc_methods[] = { { NULL } };
 
+
+#if PY_MAJOR_VERSION >= 3
+static PyModuleDef xc_module = {
+    PyModuleDef_HEAD_INIT,
+    PKG,     /* name */
+    NULL,   /* docstring */
+    -1,     /* size of per-interpreter state, -1 means the module use global
+               variables */
+    xc_methods
+};
+#endif
+
+#if PY_MAJOR_VERSION >= 3
+#define INITERROR return NULL
+PyMODINIT_FUNC PyInit_xc(void)
+#else
+#define INITERROR return
 PyMODINIT_FUNC initxc(void)
+#endif
 {
     PyObject *m;
 
     if (PyType_Ready(&PyXcType) < 0)
-        return;
+        INITERROR;
 
+#if PY_MAJOR_VERSION >= 3
+    m = PyModule_Create(&xc_module);
+#else
     m = Py_InitModule(PKG, xc_methods);
+#endif
 
     if (m == NULL)
-      return;
+        INITERROR;
 
     xc_error_obj = PyErr_NewException(PKG ".Error", PyExc_RuntimeError, NULL);
     if (xc_error_obj == NULL) {
         Py_DECREF(m);
-        return;
+        INITERROR;
     }
     zero = PyLongOrInt_FromLong(0);
 
@@ -2732,6 +2758,9 @@ PyMODINIT_FUNC initxc(void)
     PyModule_AddIntConstant(m, "XEN_SCHEDULER_CREDIT", XEN_SCHEDULER_CREDIT);
     PyModule_AddIntConstant(m, "XEN_SCHEDULER_CREDIT2", XEN_SCHEDULER_CREDIT2);
 
+#if PY_MAJOR_VERSION >= 3
+    return m;
+#endif
 }
 
 
diff --git a/tools/python/xen/lowlevel/xs/xs.c b/tools/python/xen/lowlevel/xs/xs.c
index b37daa9..aba5a20 100644
--- a/tools/python/xen/lowlevel/xs/xs.c
+++ b/tools/python/xen/lowlevel/xs/xs.c
@@ -946,7 +946,11 @@ static void xshandle_dealloc(XsHandle *self)
 }
 
 static PyTypeObject xshandle_type = {
+#if PY_MAJOR_VERSION >= 3
+    .ob_base = { PyObject_HEAD_INIT(NULL) },
+#else
     PyObject_HEAD_INIT(NULL)
+#endif
     .tp_name = PKG "." CLS,
     .tp_basicsize = sizeof(XsHandle),
     .tp_itemsize = 0,
@@ -960,22 +964,43 @@ static PyTypeObject xshandle_type = {
 
 static PyMethodDef xs_methods[] = { { NULL } };
 
+#if PY_MAJOR_VERSION >= 3
+static PyModuleDef xs_module = {
+    PyModuleDef_HEAD_INIT,
+    PKG,     /* name */
+    NULL,   /* docstring */
+    -1,     /* size of per-interpreter state, -1 means the module use global
+               variables */
+    xs_methods
+};
+#endif
+
+#if PY_MAJOR_VERSION >= 3
+#define INITERROR return NULL
+PyMODINIT_FUNC PyInit_xs(void)
+#else
+#define INITERROR return
 PyMODINIT_FUNC initxs(void)
+#endif
 {
     PyObject* m;
 
     if (PyType_Ready(&xshandle_type) < 0)
-        return;
+        INITERROR;
 
+#if PY_MAJOR_VERSION >= 3
+    m = PyModule_Create(&xs_module);
+#else
     m = Py_InitModule(PKG, xs_methods);
+#endif
 
     if (m == NULL)
-      return;
+        INITERROR;
 
     xs_error = PyErr_NewException(PKG ".Error", PyExc_RuntimeError, NULL);
     if (xs_error == NULL) {
         Py_DECREF(m);
-        return;
+        INITERROR;
     }
 
     Py_INCREF(&xshandle_type);
@@ -983,6 +1008,9 @@ PyMODINIT_FUNC initxs(void)
 
     Py_INCREF(xs_error);
     PyModule_AddObject(m, "Error", xs_error);
+#if PY_MAJOR_VERSION >= 3
+    return m;
+#endif
 }
 
 
-- 
2.7.4


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[PATCH 8/8] python: handle long type in scripts

[Marek Marczykowski-Górecki]

In Python3 'long' type have been merged into 'int', '1L' syntax is no
longer valid. Assign 'int' type to a 'long' variable in python3, so
'long(1)' will give correct result in both python2 and python3.

Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
---
 tools/python/xen/migration/libxc.py | 32 ++++++++++++++++++--------------
 1 file changed, 18 insertions(+), 14 deletions(-)

diff --git a/tools/python/xen/migration/libxc.py b/tools/python/xen/migration/libxc.py
index 85a78f4..6fd3f6d 100644
--- a/tools/python/xen/migration/libxc.py
+++ b/tools/python/xen/migration/libxc.py
@@ -14,6 +14,10 @@ from struct import calcsize, unpack
 
 from xen.migration.verify import StreamError, RecordError, VerifyBase
 
+# In Python3 long type have been merged into int, 1L syntax is no longer valid
+if sys.version_info > (3,):
+    long = int
+
 # Image Header
 IHDR_FORMAT = "!QIIHHI"
 
@@ -83,23 +87,23 @@ rec_type_to_str = {
 
 # page_data
 PAGE_DATA_FORMAT             = "II"
-PAGE_DATA_PFN_MASK           = (1L << 52) - 1
-PAGE_DATA_PFN_RESZ_MASK      = ((1L << 60) - 1) & ~((1L << 52) - 1)
+PAGE_DATA_PFN_MASK           = (long(1) << 52) - 1
+PAGE_DATA_PFN_RESZ_MASK      = ((long(1) << 60) - 1) & ~((long(1) << 52) - 1)
 
 # flags from xen/public/domctl.h: XEN_DOMCTL_PFINFO_* shifted by 32 bits
 PAGE_DATA_TYPE_SHIFT         = 60
-PAGE_DATA_TYPE_LTABTYPE_MASK = (0x7L << PAGE_DATA_TYPE_SHIFT)
-PAGE_DATA_TYPE_LTAB_MASK     = (0xfL << PAGE_DATA_TYPE_SHIFT)
-PAGE_DATA_TYPE_LPINTAB       = (0x8L << PAGE_DATA_TYPE_SHIFT) # Pinned pagetable
-
-PAGE_DATA_TYPE_NOTAB         = (0x0L << PAGE_DATA_TYPE_SHIFT) # Regular page
-PAGE_DATA_TYPE_L1TAB         = (0x1L << PAGE_DATA_TYPE_SHIFT) # L1 pagetable
-PAGE_DATA_TYPE_L2TAB         = (0x2L << PAGE_DATA_TYPE_SHIFT) # L2 pagetable
-PAGE_DATA_TYPE_L3TAB         = (0x3L << PAGE_DATA_TYPE_SHIFT) # L3 pagetable
-PAGE_DATA_TYPE_L4TAB         = (0x4L << PAGE_DATA_TYPE_SHIFT) # L4 pagetable
-PAGE_DATA_TYPE_BROKEN        = (0xdL << PAGE_DATA_TYPE_SHIFT) # Broken
-PAGE_DATA_TYPE_XALLOC        = (0xeL << PAGE_DATA_TYPE_SHIFT) # Allocate-only
-PAGE_DATA_TYPE_XTAB          = (0xfL << PAGE_DATA_TYPE_SHIFT) # Invalid
+PAGE_DATA_TYPE_LTABTYPE_MASK = (long(0x7) << PAGE_DATA_TYPE_SHIFT)
+PAGE_DATA_TYPE_LTAB_MASK     = (long(0xf) << PAGE_DATA_TYPE_SHIFT)
+PAGE_DATA_TYPE_LPINTAB       = (long(0x8) << PAGE_DATA_TYPE_SHIFT) # Pinned pagetable
+
+PAGE_DATA_TYPE_NOTAB         = (long(0x0) << PAGE_DATA_TYPE_SHIFT) # Regular page
+PAGE_DATA_TYPE_L1TAB         = (long(0x1) << PAGE_DATA_TYPE_SHIFT) # L1 pagetable
+PAGE_DATA_TYPE_L2TAB         = (long(0x2) << PAGE_DATA_TYPE_SHIFT) # L2 pagetable
+PAGE_DATA_TYPE_L3TAB         = (long(0x3) << PAGE_DATA_TYPE_SHIFT) # L3 pagetable
+PAGE_DATA_TYPE_L4TAB         = (long(0x4) << PAGE_DATA_TYPE_SHIFT) # L4 pagetable
+PAGE_DATA_TYPE_BROKEN        = (long(0xd) << PAGE_DATA_TYPE_SHIFT) # Broken
+PAGE_DATA_TYPE_XALLOC        = (long(0xe) << PAGE_DATA_TYPE_SHIFT) # Allocate-only
+PAGE_DATA_TYPE_XTAB          = (long(0xf) << PAGE_DATA_TYPE_SHIFT) # Invalid
 
 # x86_pv_info
 X86_PV_INFO_FORMAT        = "BBHI"
-- 
2.7.4


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [PATCH 0/8] Python 3 bindings

[Wei Liu]

On Thu, Feb 23, 2017 at 11:48:20AM +0100, Marek Marczykowski-Górecki wrote:
> This patch series add support for Python3 in tools/python. As most python
> modules, it looks at $PYTHON variable to what python use, so to use this
> PYTHON=python3 needs to be defined in a build environment. If both python2 and
> python3 versions are needed, then python bindings needs to be built twice. In
> most cases it will be packaging scripts responsibility.
> 
> Patches 1-4 are cleanups not really specific to python3, but making it easier.
> 
> Marek Marczykowski-Górecki (8):
>   python: check return value of PyErr_NewException
>   python: drop tp_getattr implementation
>   python: use Py_TYPE instead of looking directly into PyObject_HEAD
>   python: initialize specific fields of PyTypeObject
>   python: use PyBytes/PyUnicode instead of PyString
>   python: use PyLong_* for constructing 'int' type in Python3
>   python: adjust module initalization for Python3
>   python: handle long type in scripts
> 
>  tools/python/xen/lowlevel/xc/xc.c   | 148 +++++++++++++++++++-----------------
>  tools/python/xen/lowlevel/xs/xs.c   | 118 ++++++++++++++++------------
>  tools/python/xen/migration/libxc.py |  32 ++++----
>  3 files changed, 165 insertions(+), 133 deletions(-)

This series looks rather nice. And, to the best of my knowledge, they
are correct:

Acked-by: Wei Liu <wei.liu2@citrix.com>

> 
> -- 
> 2.7.4
> 

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel