[PATCH v2 0/4] Introduce XEN_PAGE_* definitions for mapping guests memory

[PATCH v2 0/4] Introduce XEN_PAGE_* definitions for mapping guests memory

[Costin Lupu]

This series tries to fix a side-effect introduced by commits 0dbb4be7 and
d1b32abd which added a dependency to xenctrl for foreignmemory and gnntab
libraries library only because they needed to use the XC_PAGE_* values.

These changes introduce the XEN_PAGE_* definitions that will be used by any
toolstack component that doesn't need a dependency to xenctrl library.

Changes since v1:
- Use same page definitions for both x86_64 and ARM (i.e. a single page.h file)
- Introduce xen_mk_long()

Costin Lupu (4):
  public: Add page related definitions for accessing guests memory
  libs/ctrl: Use Xen values for XC_PAGE_* definitions
  libs/foreignmemory: Use XEN_PAGE_* definitions
  libs/gnttab: Use XEN_PAGE_* definitions

 tools/include/xenctrl.h            |  7 +++---
 tools/libs/foreignmemory/core.c    |  2 +-
 tools/libs/foreignmemory/freebsd.c | 10 ++++-----
 tools/libs/foreignmemory/linux.c   | 18 +++++++--------
 tools/libs/foreignmemory/minios.c  | 10 +--------
 tools/libs/foreignmemory/netbsd.c  | 10 ++++-----
 tools/libs/foreignmemory/private.h |  2 +-
 tools/libs/foreignmemory/solaris.c |  6 ++---
 tools/libs/gnttab/freebsd.c        | 20 ++++++++---------
 tools/libs/gnttab/linux.c          | 20 ++++++++---------
 tools/libs/gnttab/netbsd.c         | 20 ++++++++---------
 xen/include/public/page.h          | 36 ++++++++++++++++++++++++++++++
 xen/include/public/xen.h           |  3 +++
 13 files changed, 98 insertions(+), 66 deletions(-)
 create mode 100644 xen/include/public/page.h

-- 
2.20.1

[PATCH v2 1/4] public: Add page related definitions for accessing guests memory

[Costin Lupu]

These changes introduce the page related definitions needed for mapping and
accessing guests memory. These values are intended to be used by any toolstack
component that needs to map guests memory. Until now, the values were defined
by the xenctrl.h header, therefore whenever a component had to use them it also
had to add a dependency for the xenctrl library.

This patch also introduces xen_mk_long() macrodefinition for defining long
constants both for C and assembler code.

Signed-off-by: Costin Lupu <costin.lupu@cs.pub.ro>
---
 xen/include/public/page.h | 36 ++++++++++++++++++++++++++++++++++++
 xen/include/public/xen.h  |  3 +++
 2 files changed, 39 insertions(+)
 create mode 100644 xen/include/public/page.h

diff --git a/xen/include/public/page.h b/xen/include/public/page.h
new file mode 100644
index 0000000000..6b06259bad
--- /dev/null
+++ b/xen/include/public/page.h
@@ -0,0 +1,36 @@
+/******************************************************************************
+ * page.h
+ *
+ * Page definitions for accessing guests memory
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to
+ * deal in the Software without restriction, including without limitation the
+ * rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
+ * sell copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+ * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
+ * DEALINGS IN THE SOFTWARE.
+ *
+ * Copyright (c) 2021, Costin Lupu
+ */
+
+#ifndef __XEN_PUBLIC_PAGE_H__
+#define __XEN_PUBLIC_PAGE_H__
+
+#include "xen.h"
+
+#define XEN_PAGE_SHIFT           12
+#define XEN_PAGE_SIZE            (xen_mk_long(1) << XEN_PAGE_SHIFT)
+#define XEN_PAGE_MASK            (~(XEN_PAGE_SIZE - 1))
+
+#endif /* __XEN_PUBLIC_PAGE_H__ */
diff --git a/xen/include/public/xen.h b/xen/include/public/xen.h
index e373592c33..12531c02b5 100644
--- a/xen/include/public/xen.h
+++ b/xen/include/public/xen.h
@@ -64,11 +64,13 @@ DEFINE_XEN_GUEST_HANDLE(xen_ulong_t);
 
 /* Turn a plain number into a C unsigned (long (long)) constant. */
 #define __xen_mk_uint(x)  x ## U
+#define __xen_mk_long(x)  x ## L
 #define __xen_mk_ulong(x) x ## UL
 #ifndef __xen_mk_ullong
 # define __xen_mk_ullong(x) x ## ULL
 #endif
 #define xen_mk_uint(x)    __xen_mk_uint(x)
+#define xen_mk_long(x)    __xen_mk_long(x)
 #define xen_mk_ulong(x)   __xen_mk_ulong(x)
 #define xen_mk_ullong(x)  __xen_mk_ullong(x)
 
@@ -76,6 +78,7 @@ DEFINE_XEN_GUEST_HANDLE(xen_ulong_t);
 
 /* In assembly code we cannot use C numeric constant suffixes. */
 #define xen_mk_uint(x)   x
+#define xen_mk_long(x)   x
 #define xen_mk_ulong(x)  x
 #define xen_mk_ullong(x) x
 
-- 
2.20.1

[PATCH v2 2/4] libs/ctrl: Use Xen values for XC_PAGE_* definitions

[Costin Lupu]

We use the values provided by the Xen public interface for defining the
XC_PAGE_* macros.

Signed-off-by: Costin Lupu <costin.lupu@cs.pub.ro>
---
 tools/include/xenctrl.h | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/tools/include/xenctrl.h b/tools/include/xenctrl.h
index b77726eab7..2031308458 100644
--- a/tools/include/xenctrl.h
+++ b/tools/include/xenctrl.h
@@ -54,10 +54,11 @@
 #include <xen/foreign/x86_64.h>
 #include <xen/arch-x86/xen-mca.h>
 #endif
+#include <xen/page.h>
 
-#define XC_PAGE_SHIFT           12
-#define XC_PAGE_SIZE            (1UL << XC_PAGE_SHIFT)
-#define XC_PAGE_MASK            (~(XC_PAGE_SIZE-1))
+#define XC_PAGE_SHIFT           XEN_PAGE_SHIFT
+#define XC_PAGE_SIZE            XEN_PAGE_SIZE
+#define XC_PAGE_MASK            XEN_PAGE_MASK
 
 #define INVALID_MFN  (~0UL)
 
-- 
2.20.1

[PATCH v2 3/4] libs/foreignmemory: Use XEN_PAGE_* definitions

[Costin Lupu]

These changes refine the changes in 0dbb4be7 which added a dependency to
xenctrl library. We use the XEN_PAGE_* definitions instead of the XC_PAGE_*
definitions and therefore we get rid of the unnecessary dependency.

Signed-off-by: Costin Lupu <costin.lupu@cs.pub.ro>
---
 tools/libs/foreignmemory/core.c    |  2 +-
 tools/libs/foreignmemory/freebsd.c | 10 +++++-----
 tools/libs/foreignmemory/linux.c   | 18 +++++++++---------
 tools/libs/foreignmemory/minios.c  | 10 +---------
 tools/libs/foreignmemory/netbsd.c  | 10 +++++-----
 tools/libs/foreignmemory/private.h |  2 +-
 tools/libs/foreignmemory/solaris.c |  6 +++---
 7 files changed, 25 insertions(+), 33 deletions(-)

diff --git a/tools/libs/foreignmemory/core.c b/tools/libs/foreignmemory/core.c
index 7edc6f0dbf..ad1ad9fc67 100644
--- a/tools/libs/foreignmemory/core.c
+++ b/tools/libs/foreignmemory/core.c
@@ -202,7 +202,7 @@ int xenforeignmemory_resource_size(
     if ( rc )
         return rc;
 
-    *size = fres.nr_frames << XC_PAGE_SHIFT;
+    *size = fres.nr_frames << XEN_PAGE_SHIFT;
     return 0;
 }
 
diff --git a/tools/libs/foreignmemory/freebsd.c b/tools/libs/foreignmemory/freebsd.c
index 2cf0fa1c38..9439c4ca6a 100644
--- a/tools/libs/foreignmemory/freebsd.c
+++ b/tools/libs/foreignmemory/freebsd.c
@@ -63,7 +63,7 @@ void *osdep_xenforeignmemory_map(xenforeignmemory_handle *fmem,
     privcmd_mmapbatch_t ioctlx;
     int rc;
 
-    addr = mmap(addr, num << XC_PAGE_SHIFT, prot, flags | MAP_SHARED, fd, 0);
+    addr = mmap(addr, num << XEN_PAGE_SHIFT, prot, flags | MAP_SHARED, fd, 0);
     if ( addr == MAP_FAILED )
         return NULL;
 
@@ -78,7 +78,7 @@ void *osdep_xenforeignmemory_map(xenforeignmemory_handle *fmem,
     {
         int saved_errno = errno;
 
-        (void)munmap(addr, num << XC_PAGE_SHIFT);
+        (void)munmap(addr, num << XEN_PAGE_SHIFT);
         errno = saved_errno;
         return NULL;
     }
@@ -89,7 +89,7 @@ void *osdep_xenforeignmemory_map(xenforeignmemory_handle *fmem,
 int osdep_xenforeignmemory_unmap(xenforeignmemory_handle *fmem,
                                  void *addr, size_t num)
 {
-    return munmap(addr, num << XC_PAGE_SHIFT);
+    return munmap(addr, num << XEN_PAGE_SHIFT);
 }
 
 int osdep_xenforeignmemory_restrict(xenforeignmemory_handle *fmem,
@@ -101,7 +101,7 @@ int osdep_xenforeignmemory_restrict(xenforeignmemory_handle *fmem,
 int osdep_xenforeignmemory_unmap_resource(xenforeignmemory_handle *fmem,
                                         xenforeignmemory_resource_handle *fres)
 {
-    return fres ? munmap(fres->addr, fres->nr_frames << XC_PAGE_SHIFT) : 0;
+    return fres ? munmap(fres->addr, fres->nr_frames << XEN_PAGE_SHIFT) : 0;
 }
 
 int osdep_xenforeignmemory_map_resource(xenforeignmemory_handle *fmem,
@@ -120,7 +120,7 @@ int osdep_xenforeignmemory_map_resource(xenforeignmemory_handle *fmem,
         /* Request for resource size.  Skip mmap(). */
         goto skip_mmap;
 
-    fres->addr = mmap(fres->addr, fres->nr_frames << XC_PAGE_SHIFT,
+    fres->addr = mmap(fres->addr, fres->nr_frames << XEN_PAGE_SHIFT,
                       fres->prot, fres->flags | MAP_SHARED, fmem->fd, 0);
     if ( fres->addr == MAP_FAILED )
         return -1;
diff --git a/tools/libs/foreignmemory/linux.c b/tools/libs/foreignmemory/linux.c
index 9062117407..9dabf28cae 100644
--- a/tools/libs/foreignmemory/linux.c
+++ b/tools/libs/foreignmemory/linux.c
@@ -134,7 +134,7 @@ static int retry_paged(int fd, uint32_t dom, void *addr,
         /* At least one gfn is still in paging state */
         ioctlx.num = 1;
         ioctlx.dom = dom;
-        ioctlx.addr = (unsigned long)addr + (i<<XC_PAGE_SHIFT);
+        ioctlx.addr = (unsigned long)addr + (i<<XEN_PAGE_SHIFT);
         ioctlx.arr = arr + i;
         ioctlx.err = err + i;
 
@@ -168,7 +168,7 @@ void *osdep_xenforeignmemory_map(xenforeignmemory_handle *fmem,
     size_t i;
     int rc;
 
-    addr = mmap(addr, num << XC_PAGE_SHIFT, prot, flags | MAP_SHARED,
+    addr = mmap(addr, num << XEN_PAGE_SHIFT, prot, flags | MAP_SHARED,
                 fd, 0);
     if ( addr == MAP_FAILED )
         return NULL;
@@ -198,7 +198,7 @@ void *osdep_xenforeignmemory_map(xenforeignmemory_handle *fmem,
          */
         privcmd_mmapbatch_t ioctlx;
         xen_pfn_t *pfn;
-        unsigned int pfn_arr_size = ROUNDUP((num * sizeof(*pfn)), XC_PAGE_SHIFT);
+        unsigned int pfn_arr_size = ROUNDUP((num * sizeof(*pfn)), XEN_PAGE_SHIFT);
         int os_page_size = sysconf(_SC_PAGESIZE);
 
         if ( pfn_arr_size <= os_page_size )
@@ -210,7 +210,7 @@ void *osdep_xenforeignmemory_map(xenforeignmemory_handle *fmem,
             if ( pfn == MAP_FAILED )
             {
                 PERROR("mmap of pfn array failed");
-                (void)munmap(addr, num << XC_PAGE_SHIFT);
+                (void)munmap(addr, num << XEN_PAGE_SHIFT);
                 return NULL;
             }
         }
@@ -243,7 +243,7 @@ void *osdep_xenforeignmemory_map(xenforeignmemory_handle *fmem,
                     continue;
                 }
                 rc = map_foreign_batch_single(fd, dom, pfn + i,
-                        (unsigned long)addr + (i<<XC_PAGE_SHIFT));
+                        (unsigned long)addr + (i<<XEN_PAGE_SHIFT));
                 if ( rc < 0 )
                 {
                     rc = -errno;
@@ -271,7 +271,7 @@ void *osdep_xenforeignmemory_map(xenforeignmemory_handle *fmem,
     {
         int saved_errno = errno;
 
-        (void)munmap(addr, num << XC_PAGE_SHIFT);
+        (void)munmap(addr, num << XEN_PAGE_SHIFT);
         errno = saved_errno;
         return NULL;
     }
@@ -282,7 +282,7 @@ void *osdep_xenforeignmemory_map(xenforeignmemory_handle *fmem,
 int osdep_xenforeignmemory_unmap(xenforeignmemory_handle *fmem,
                                  void *addr, size_t num)
 {
-    return munmap(addr, num << XC_PAGE_SHIFT);
+    return munmap(addr, num << XEN_PAGE_SHIFT);
 }
 
 int osdep_xenforeignmemory_restrict(xenforeignmemory_handle *fmem,
@@ -294,7 +294,7 @@ int osdep_xenforeignmemory_restrict(xenforeignmemory_handle *fmem,
 int osdep_xenforeignmemory_unmap_resource(
     xenforeignmemory_handle *fmem, xenforeignmemory_resource_handle *fres)
 {
-    return fres ? munmap(fres->addr, fres->nr_frames << XC_PAGE_SHIFT) : 0;
+    return fres ? munmap(fres->addr, fres->nr_frames << XEN_PAGE_SHIFT) : 0;
 }
 
 int osdep_xenforeignmemory_map_resource(
@@ -313,7 +313,7 @@ int osdep_xenforeignmemory_map_resource(
         /* Request for resource size.  Skip mmap(). */
         goto skip_mmap;
 
-    fres->addr = mmap(fres->addr, fres->nr_frames << XC_PAGE_SHIFT,
+    fres->addr = mmap(fres->addr, fres->nr_frames << XEN_PAGE_SHIFT,
                       fres->prot, fres->flags | MAP_SHARED, fmem->fd, 0);
     if ( fres->addr == MAP_FAILED )
         return -1;
diff --git a/tools/libs/foreignmemory/minios.c b/tools/libs/foreignmemory/minios.c
index f2f4dfb2be..2454eb9af3 100644
--- a/tools/libs/foreignmemory/minios.c
+++ b/tools/libs/foreignmemory/minios.c
@@ -17,14 +17,6 @@
  * Copyright 2007-2008 Samuel Thibault <samuel.thibault@eu.citrix.com>.
  */
 
-/*
- * xenctrl.h currently defines __XEN_TOOLS__ which affects what is
- * exposed by Xen headers. As the define needs to be set consistently,
- * we want to include xenctrl.h before the mini-os headers (they include
- * public headers).
- */
-#include <xenctrl.h>
-
 #include <mini-os/types.h>
 #include <mini-os/os.h>
 #include <mini-os/mm.h>
@@ -63,7 +55,7 @@ void *osdep_xenforeignmemory_map(xenforeignmemory_handle *fmem,
 int osdep_xenforeignmemory_unmap(xenforeignmemory_handle *fmem,
                                  void *addr, size_t num)
 {
-    return munmap(addr, num << XC_PAGE_SHIFT);
+    return munmap(addr, num << XEN_PAGE_SHIFT);
 }
 
 /*
diff --git a/tools/libs/foreignmemory/netbsd.c b/tools/libs/foreignmemory/netbsd.c
index 597db775d7..ba69b9c6bb 100644
--- a/tools/libs/foreignmemory/netbsd.c
+++ b/tools/libs/foreignmemory/netbsd.c
@@ -76,7 +76,7 @@ void *osdep_xenforeignmemory_map(xenforeignmemory_handle *fmem,
 {
     int fd = fmem->fd;
     privcmd_mmapbatch_v2_t ioctlx;
-    addr = mmap(addr, num * XC_PAGE_SIZE, prot,
+    addr = mmap(addr, num * XEN_PAGE_SIZE, prot,
                 flags | MAP_ANON | MAP_SHARED, -1, 0);
     if ( addr == MAP_FAILED ) {
         PERROR("osdep_xenforeignmemory_map: mmap failed");
@@ -93,7 +93,7 @@ void *osdep_xenforeignmemory_map(xenforeignmemory_handle *fmem,
     {
         int saved_errno = errno;
         PERROR("osdep_xenforeignmemory_map: ioctl failed");
-        munmap(addr, num * XC_PAGE_SIZE);
+        munmap(addr, num * XEN_PAGE_SIZE);
         errno = saved_errno;
         return NULL;
     }
@@ -104,7 +104,7 @@ void *osdep_xenforeignmemory_map(xenforeignmemory_handle *fmem,
 int osdep_xenforeignmemory_unmap(xenforeignmemory_handle *fmem,
                                  void *addr, size_t num)
 {
-    return munmap(addr, num * XC_PAGE_SIZE);
+    return munmap(addr, num * XEN_PAGE_SIZE);
 }
 
 int osdep_xenforeignmemory_restrict(xenforeignmemory_handle *fmem,
@@ -117,7 +117,7 @@ int osdep_xenforeignmemory_restrict(xenforeignmemory_handle *fmem,
 int osdep_xenforeignmemory_unmap_resource(
     xenforeignmemory_handle *fmem, xenforeignmemory_resource_handle *fres)
 {
-    return fres ? munmap(fres->addr, fres->nr_frames << XC_PAGE_SHIFT) : 0;
+    return fres ? munmap(fres->addr, fres->nr_frames << XEN_PAGE_SHIFT) : 0;
 }
 
 int osdep_xenforeignmemory_map_resource(
@@ -136,7 +136,7 @@ int osdep_xenforeignmemory_map_resource(
         /* Request for resource size.  Skip mmap(). */
         goto skip_mmap;
 
-    fres->addr = mmap(fres->addr, fres->nr_frames << XC_PAGE_SHIFT,
+    fres->addr = mmap(fres->addr, fres->nr_frames << XEN_PAGE_SHIFT,
                       fres->prot, fres->flags | MAP_ANON | MAP_SHARED, -1, 0);
     if ( fres->addr == MAP_FAILED )
         return -1;
diff --git a/tools/libs/foreignmemory/private.h b/tools/libs/foreignmemory/private.h
index 8540303adc..1200f98067 100644
--- a/tools/libs/foreignmemory/private.h
+++ b/tools/libs/foreignmemory/private.h
@@ -1,7 +1,6 @@
 #ifndef XENFOREIGNMEMORY_PRIVATE_H
 #define XENFOREIGNMEMORY_PRIVATE_H
 
-#include <xenctrl.h>
 #include <xentoollog.h>
 
 #include <xenforeignmemory.h>
@@ -9,6 +8,7 @@
 #include <xentoolcore_internal.h>
 
 #include <xen/xen.h>
+#include <xen/page.h>
 #include <xen/sys/privcmd.h>
 
 struct xenforeignmemory_handle {
diff --git a/tools/libs/foreignmemory/solaris.c b/tools/libs/foreignmemory/solaris.c
index 958fb01f6d..4466780bd8 100644
--- a/tools/libs/foreignmemory/solaris.c
+++ b/tools/libs/foreignmemory/solaris.c
@@ -72,7 +72,7 @@ void *osdep_map_foreign_batch(xenforeignmem_handle *fmem, uint32_t dom,
 {
     int fd = fmem->fd;
     privcmd_mmapbatch_t ioctlx;
-    addr = mmap(addr, num*XC_PAGE_SIZE, prot, flags | MAP_SHARED, fd, 0);
+    addr = mmap(addr, num*XEN_PAGE_SIZE, prot, flags | MAP_SHARED, fd, 0);
     if ( addr == MAP_FAILED )
         return NULL;
 
@@ -84,7 +84,7 @@ void *osdep_map_foreign_batch(xenforeignmem_handle *fmem, uint32_t dom,
     {
         int saved_errno = errno;
 
-        (void)munmap(addr, num*XC_PAGE_SIZE);
+        (void)munmap(addr, num*XEN_PAGE_SIZE);
         errno = saved_errno;
         return NULL;
     }
@@ -94,7 +94,7 @@ void *osdep_map_foreign_batch(xenforeignmem_handle *fmem, uint32_t dom,
 int osdep_xenforeignmemory_unmap(xenforeignmemory_handle *fmem,
                                  void *addr, size_t num)
 {
-    return munmap(addr, num*XC_PAGE_SIZE);
+    return munmap(addr, num*XEN_PAGE_SIZE);
 }
 
 /*
-- 
2.20.1

[PATCH v2 4/4] libs/gnttab: Use XEN_PAGE_* definitions

[Costin Lupu]

These changes refine the changes in d1b32abd which added a dependency to
xenctrl library. We use the XEN_PAGE_* definitions instead of the XC_PAGE_*
definitions and therefore we get rid of the unnecessary dependency.

Signed-off-by: Costin Lupu <costin.lupu@cs.pub.ro>
---
 tools/libs/gnttab/freebsd.c | 20 ++++++++++----------
 tools/libs/gnttab/linux.c   | 20 ++++++++++----------
 tools/libs/gnttab/netbsd.c  | 20 ++++++++++----------
 3 files changed, 30 insertions(+), 30 deletions(-)

diff --git a/tools/libs/gnttab/freebsd.c b/tools/libs/gnttab/freebsd.c
index e42ac3fbf3..7ecb0e3b38 100644
--- a/tools/libs/gnttab/freebsd.c
+++ b/tools/libs/gnttab/freebsd.c
@@ -28,9 +28,9 @@
 #include <sys/ioctl.h>
 #include <sys/mman.h>
 
+#include <xen/page.h>
 #include <xen/sys/gntdev.h>
 
-#include <xenctrl.h>
 #include <xen-tools/libs.h>
 
 #include "private.h"
@@ -74,7 +74,7 @@ void *osdep_gnttab_grant_map(xengnttab_handle *xgt,
     int domids_stride;
     unsigned int refs_size = ROUNDUP(count *
                                      sizeof(struct ioctl_gntdev_grant_ref),
-                                     XC_PAGE_SHIFT);
+                                     XEN_PAGE_SHIFT);
     int os_page_size = getpagesize();
 
     domids_stride = (flags & XENGNTTAB_GRANT_MAP_SINGLE_DOMAIN) ? 0 : 1;
@@ -105,7 +105,7 @@ void *osdep_gnttab_grant_map(xengnttab_handle *xgt,
         goto out;
     }
 
-    addr = mmap(NULL, XC_PAGE_SIZE * count, prot, MAP_SHARED, fd,
+    addr = mmap(NULL, XEN_PAGE_SIZE * count, prot, MAP_SHARED, fd,
                 map.index);
     if ( addr != MAP_FAILED )
     {
@@ -114,7 +114,7 @@ void *osdep_gnttab_grant_map(xengnttab_handle *xgt,
 
         notify.index = map.index;
         notify.action = 0;
-        if ( notify_offset < XC_PAGE_SIZE * count )
+        if ( notify_offset < XEN_PAGE_SIZE * count )
         {
             notify.index += notify_offset;
             notify.action |= UNMAP_NOTIFY_CLEAR_BYTE;
@@ -129,7 +129,7 @@ void *osdep_gnttab_grant_map(xengnttab_handle *xgt,
         if ( rv )
         {
             GTERROR(xgt->logger, "ioctl SET_UNMAP_NOTIFY failed");
-            munmap(addr, count * XC_PAGE_SIZE);
+            munmap(addr, count * XEN_PAGE_SIZE);
             addr = MAP_FAILED;
         }
     }
@@ -187,7 +187,7 @@ int osdep_gnttab_unmap(xengnttab_handle *xgt,
     }
 
     /* Next, unmap the memory. */
-    if ( (rc = munmap(start_address, count * XC_PAGE_SIZE)) )
+    if ( (rc = munmap(start_address, count * XEN_PAGE_SIZE)) )
         return rc;
 
     /* Finally, unmap the driver slots used to store the grant information. */
@@ -254,7 +254,7 @@ void *osdep_gntshr_share_pages(xengntshr_handle *xgs,
         goto out;
     }
 
-    area = mmap(NULL, count * XC_PAGE_SIZE, PROT_READ | PROT_WRITE, MAP_SHARED,
+    area = mmap(NULL, count * XEN_PAGE_SIZE, PROT_READ | PROT_WRITE, MAP_SHARED,
                 fd, gref_info.index);
 
     if ( area == MAP_FAILED )
@@ -266,7 +266,7 @@ void *osdep_gntshr_share_pages(xengntshr_handle *xgs,
 
     notify.index = gref_info.index;
     notify.action = 0;
-    if ( notify_offset < XC_PAGE_SIZE * count )
+    if ( notify_offset < XEN_PAGE_SIZE * count )
     {
         notify.index += notify_offset;
         notify.action |= UNMAP_NOTIFY_CLEAR_BYTE;
@@ -281,7 +281,7 @@ void *osdep_gntshr_share_pages(xengntshr_handle *xgs,
     if ( err )
     {
         GSERROR(xgs->logger, "ioctl SET_UNMAP_NOTIFY failed");
-        munmap(area, count * XC_PAGE_SIZE);
+        munmap(area, count * XEN_PAGE_SIZE);
         area = NULL;
     }
 
@@ -304,7 +304,7 @@ void *osdep_gntshr_share_pages(xengntshr_handle *xgs,
 int osdep_gntshr_unshare(xengntshr_handle *xgs,
                          void *start_address, uint32_t count)
 {
-    return munmap(start_address, count * XC_PAGE_SIZE);
+    return munmap(start_address, count * XEN_PAGE_SIZE);
 }
 
 /*
diff --git a/tools/libs/gnttab/linux.c b/tools/libs/gnttab/linux.c
index 5628fd5719..11f1acb771 100644
--- a/tools/libs/gnttab/linux.c
+++ b/tools/libs/gnttab/linux.c
@@ -29,10 +29,10 @@
 #include <sys/ioctl.h>
 #include <sys/mman.h>
 
+#include <xen/page.h>
 #include <xen/sys/gntdev.h>
 #include <xen/sys/gntalloc.h>
 
-#include <xenctrl.h>
 #include <xen-tools/libs.h>
 
 #include "private.h"
@@ -101,7 +101,7 @@ void *osdep_gnttab_grant_map(xengnttab_handle *xgt,
         map = alloca(map_size);
     else
     {
-        map_size = ROUNDUP(map_size, XC_PAGE_SHIFT);
+        map_size = ROUNDUP(map_size, XEN_PAGE_SHIFT);
         map = mmap(NULL, map_size, PROT_READ | PROT_WRITE,
                    MAP_PRIVATE | MAP_ANON | MAP_POPULATE, -1, 0);
         if ( map == MAP_FAILED )
@@ -125,7 +125,7 @@ void *osdep_gnttab_grant_map(xengnttab_handle *xgt,
     }
 
  retry:
-    addr = mmap(NULL, XC_PAGE_SIZE * count, prot, MAP_SHARED, fd,
+    addr = mmap(NULL, XEN_PAGE_SIZE * count, prot, MAP_SHARED, fd,
                 map->index);
 
     if (addr == MAP_FAILED && errno == EAGAIN)
@@ -150,7 +150,7 @@ void *osdep_gnttab_grant_map(xengnttab_handle *xgt,
         struct ioctl_gntdev_unmap_notify notify;
         notify.index = map->index;
         notify.action = 0;
-        if (notify_offset < XC_PAGE_SIZE * count) {
+        if (notify_offset < XEN_PAGE_SIZE * count) {
             notify.index += notify_offset;
             notify.action |= UNMAP_NOTIFY_CLEAR_BYTE;
         }
@@ -162,7 +162,7 @@ void *osdep_gnttab_grant_map(xengnttab_handle *xgt,
             rv = ioctl(fd, IOCTL_GNTDEV_SET_UNMAP_NOTIFY, &notify);
         if (rv) {
             GTERROR(xgt->logger, "ioctl SET_UNMAP_NOTIFY failed");
-            munmap(addr, count * XC_PAGE_SIZE);
+            munmap(addr, count * XEN_PAGE_SIZE);
             addr = MAP_FAILED;
         }
     }
@@ -218,7 +218,7 @@ int osdep_gnttab_unmap(xengnttab_handle *xgt,
     }
 
     /* Next, unmap the memory. */
-    if ( (rc = munmap(start_address, count * XC_PAGE_SIZE)) )
+    if ( (rc = munmap(start_address, count * XEN_PAGE_SIZE)) )
         return rc;
 
     /* Finally, unmap the driver slots used to store the grant information. */
@@ -464,7 +464,7 @@ void *osdep_gntshr_share_pages(xengntshr_handle *xgs,
         goto out;
     }
 
-    area = mmap(NULL, count * XC_PAGE_SIZE, PROT_READ | PROT_WRITE,
+    area = mmap(NULL, count * XEN_PAGE_SIZE, PROT_READ | PROT_WRITE,
         MAP_SHARED, fd, gref_info->index);
 
     if (area == MAP_FAILED) {
@@ -475,7 +475,7 @@ void *osdep_gntshr_share_pages(xengntshr_handle *xgs,
 
     notify.index = gref_info->index;
     notify.action = 0;
-    if (notify_offset < XC_PAGE_SIZE * count) {
+    if (notify_offset < XEN_PAGE_SIZE * count) {
         notify.index += notify_offset;
         notify.action |= UNMAP_NOTIFY_CLEAR_BYTE;
     }
@@ -487,7 +487,7 @@ void *osdep_gntshr_share_pages(xengntshr_handle *xgs,
         err = ioctl(fd, IOCTL_GNTALLOC_SET_UNMAP_NOTIFY, &notify);
     if (err) {
         GSERROR(xgs->logger, "ioctl SET_UNMAP_NOTIFY failed");
-        munmap(area, count * XC_PAGE_SIZE);
+        munmap(area, count * XEN_PAGE_SIZE);
         area = NULL;
     }
 
@@ -508,7 +508,7 @@ void *osdep_gntshr_share_pages(xengntshr_handle *xgs,
 int osdep_gntshr_unshare(xengntshr_handle *xgs,
                          void *start_address, uint32_t count)
 {
-    return munmap(start_address, count * XC_PAGE_SIZE);
+    return munmap(start_address, count * XEN_PAGE_SIZE);
 }
 
 /*
diff --git a/tools/libs/gnttab/netbsd.c b/tools/libs/gnttab/netbsd.c
index a4ad624b54..beb94be468 100644
--- a/tools/libs/gnttab/netbsd.c
+++ b/tools/libs/gnttab/netbsd.c
@@ -28,8 +28,8 @@
 #include <sys/ioctl.h>
 #include <sys/mman.h>
 
-#include <xenctrl.h>
 #include <xen/xen.h>
+#include <xen/page.h>
 #include <xen/xenio.h>
 
 #include "private.h"
@@ -84,19 +84,19 @@ void *osdep_gnttab_grant_map(xengnttab_handle *xgt,
     }
 
     map.count = count;
-    addr = mmap(NULL, count * XC_PAGE_SIZE,
+    addr = mmap(NULL, count * XEN_PAGE_SIZE,
                 prot, flags | MAP_ANON | MAP_SHARED, -1, 0);
     if ( map.va == MAP_FAILED )
     {
         GTERROR(xgt->logger, "osdep_gnttab_grant_map: mmap failed");
-        munmap((void *)map.va, count * XC_PAGE_SIZE);
+        munmap((void *)map.va, count * XEN_PAGE_SIZE);
         addr = MAP_FAILED;
     }
     map.va = addr;
 
     map.notify.offset = 0;
     map.notify.action = 0;
-    if ( notify_offset < XC_PAGE_SIZE * count )
+    if ( notify_offset < XEN_PAGE_SIZE * count )
     {
         map.notify.offset = notify_offset;
         map.notify.action |= UNMAP_NOTIFY_CLEAR_BYTE;
@@ -112,7 +112,7 @@ void *osdep_gnttab_grant_map(xengnttab_handle *xgt,
     {
         GTERROR(xgt->logger,
             "ioctl IOCTL_GNTDEV_MMAP_GRANT_REF failed: %d", rv);
-        munmap(addr, count * XC_PAGE_SIZE);
+        munmap(addr, count * XEN_PAGE_SIZE);
         addr = MAP_FAILED;
     }
 
@@ -133,7 +133,7 @@ int osdep_gnttab_unmap(xengnttab_handle *xgt,
     }
 
     /* Next, unmap the memory. */
-    rc = munmap(start_address, count * XC_PAGE_SIZE);
+    rc = munmap(start_address, count * XEN_PAGE_SIZE);
 
     return rc;
 }
@@ -184,7 +184,7 @@ void *osdep_gntshr_share_pages(xengntshr_handle *xgs,
     alloc.domid = domid;
     alloc.flags = writable ? GNTDEV_ALLOC_FLAG_WRITABLE : 0;
     alloc.count = count;
-    area = mmap(NULL, count * XC_PAGE_SIZE,
+    area = mmap(NULL, count * XEN_PAGE_SIZE,
                 PROT_READ | PROT_WRITE, MAP_ANON | MAP_SHARED, -1, 0);
 
     if ( area == MAP_FAILED )
@@ -197,7 +197,7 @@ void *osdep_gntshr_share_pages(xengntshr_handle *xgs,
 
     alloc.notify.offset = 0;
     alloc.notify.action = 0;
-    if ( notify_offset < XC_PAGE_SIZE * count )
+    if ( notify_offset < XEN_PAGE_SIZE * count )
     {
         alloc.notify.offset = notify_offset;
         alloc.notify.action |= UNMAP_NOTIFY_CLEAR_BYTE;
@@ -212,7 +212,7 @@ void *osdep_gntshr_share_pages(xengntshr_handle *xgs,
     if ( err )
     {
         GSERROR(xgs->logger, "IOCTL_GNTDEV_ALLOC_GRANT_REF failed");
-        munmap(area, count * XC_PAGE_SIZE);
+        munmap(area, count * XEN_PAGE_SIZE);
         area = MAP_FAILED;
         goto out;
     }
@@ -227,7 +227,7 @@ void *osdep_gntshr_share_pages(xengntshr_handle *xgs,
 int osdep_gntshr_unshare(xengntshr_handle *xgs,
                          void *start_address, uint32_t count)
 {
-    return munmap(start_address, count * XC_PAGE_SIZE);
+    return munmap(start_address, count * XEN_PAGE_SIZE);
 }
 
 /*
-- 
2.20.1

Re: [PATCH v2 1/4] public: Add page related definitions for accessing guests memory

[Jan Beulich]

On 19.08.2021 19:50, Costin Lupu wrote:
> These changes introduce the page related definitions needed for mapping and
> accessing guests memory. These values are intended to be used by any toolstack
> component that needs to map guests memory. Until now, the values were defined
> by the xenctrl.h header, therefore whenever a component had to use them it also
> had to add a dependency for the xenctrl library.
> 
> This patch also introduces xen_mk_long() macrodefinition for defining long
> constants both for C and assembler code.

I'm still missing justification for the addition of a new header, especially
as I don't see that header to gain much more contents down the road.

> --- /dev/null
> +++ b/xen/include/public/page.h
> @@ -0,0 +1,36 @@
> +/******************************************************************************
> + * page.h
> + *
> + * Page definitions for accessing guests memory
> + *
> + * Permission is hereby granted, free of charge, to any person obtaining a copy
> + * of this software and associated documentation files (the "Software"), to
> + * deal in the Software without restriction, including without limitation the
> + * rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
> + * sell copies of the Software, and to permit persons to whom the Software is
> + * furnished to do so, subject to the following conditions:
> + *
> + * The above copyright notice and this permission notice shall be included in
> + * all copies or substantial portions of the Software.
> + *
> + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
> + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
> + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
> + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
> + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
> + * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
> + * DEALINGS IN THE SOFTWARE.
> + *
> + * Copyright (c) 2021, Costin Lupu
> + */
> +
> +#ifndef __XEN_PUBLIC_PAGE_H__
> +#define __XEN_PUBLIC_PAGE_H__
> +
> +#include "xen.h"
> +
> +#define XEN_PAGE_SHIFT           12
> +#define XEN_PAGE_SIZE            (xen_mk_long(1) << XEN_PAGE_SHIFT)
> +#define XEN_PAGE_MASK            (~(XEN_PAGE_SIZE - 1))

You went too far here, I'm afraid: In reply to v1 I did say "The latter
two, being identical ..." - XEN_PAGE_SHIFT ought to continue to be a
per-arch constant, even if right now it is the same for Arm and x86.

Thinking of which - with exposing this as a stable ABI (not just the
abstraction, but the specific values and the fact that they're
invariable become part of the stable ABI this way), what is the plan
for supporting 64k(?) page size on Arm in the future? At that point
you _cannot_ simply remove or replace the #define you add here. As
the immediate need is by the tool stack, enclosing in 

#if defined(__XEN__) || defined(__XEN_TOOLS__)

might be an option, with the downside of having stable libraries
(foreignmemory and gnttab) depend on an unstable hypervisor interface
(again). I can't seem to be able to think of anything better ...

Jan

Re: [PATCH v2 1/4] public: Add page related definitions for accessing guests memory

[Costin Lupu]

Hi Jan,

Please see inline.

On 8/20/21 9:52 AM, Jan Beulich wrote:
> On 19.08.2021 19:50, Costin Lupu wrote:
>> These changes introduce the page related definitions needed for mapping and
>> accessing guests memory. These values are intended to be used by any toolstack
>> component that needs to map guests memory. Until now, the values were defined
>> by the xenctrl.h header, therefore whenever a component had to use them it also
>> had to add a dependency for the xenctrl library.
>>
>> This patch also introduces xen_mk_long() macrodefinition for defining long
>> constants both for C and assembler code.
> 
> I'm still missing justification for the addition of a new header, especially
> as I don't see that header to gain much more contents down the road.
> 

For the first version, since it didn't need to include other headers, I
thought it would make sense to isolate the definitions in their own
headers. Now maybe it makes more sense to put the definitions in
arch-x86/xen.h, arch-arm.h and xen.h (the latter two) respectively. What
do you think? I'm open to suggestions here.

>> --- /dev/null
>> +++ b/xen/include/public/page.h
>> @@ -0,0 +1,36 @@
>> +/******************************************************************************
>> + * page.h
>> + *
>> + * Page definitions for accessing guests memory
>> + *
>> + * Permission is hereby granted, free of charge, to any person obtaining a copy
>> + * of this software and associated documentation files (the "Software"), to
>> + * deal in the Software without restriction, including without limitation the
>> + * rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
>> + * sell copies of the Software, and to permit persons to whom the Software is
>> + * furnished to do so, subject to the following conditions:
>> + *
>> + * The above copyright notice and this permission notice shall be included in
>> + * all copies or substantial portions of the Software.
>> + *
>> + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
>> + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
>> + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
>> + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
>> + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
>> + * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
>> + * DEALINGS IN THE SOFTWARE.
>> + *
>> + * Copyright (c) 2021, Costin Lupu
>> + */
>> +
>> +#ifndef __XEN_PUBLIC_PAGE_H__
>> +#define __XEN_PUBLIC_PAGE_H__
>> +
>> +#include "xen.h"
>> +
>> +#define XEN_PAGE_SHIFT           12
>> +#define XEN_PAGE_SIZE            (xen_mk_long(1) << XEN_PAGE_SHIFT)
>> +#define XEN_PAGE_MASK            (~(XEN_PAGE_SIZE - 1))
> 
> You went too far here, I'm afraid: In reply to v1 I did say "The latter
> two, being identical ..." - XEN_PAGE_SHIFT ought to continue to be a
> per-arch constant, even if right now it is the same for Arm and x86.
> 

Alright, now I got it.

> Thinking of which - with exposing this as a stable ABI (not just the
> abstraction, but the specific values and the fact that they're
> invariable become part of the stable ABI this way), what is the plan
> for supporting 64k(?) page size on Arm in the future? At that point
> you _cannot_ simply remove or replace the #define you add here. As
> the immediate need is by the tool stack, enclosing in 
> 
> #if defined(__XEN__) || defined(__XEN_TOOLS__)
> 
> might be an option, with the downside of having stable libraries
> (foreignmemory and gnttab) depend on an unstable hypervisor interface
> (again). I can't seem to be able to think of anything better ...

Here I can be only guessing because I don't know all the requirements
and I would need more input. One would have to add new values for ARM
that would be enabled by some flag and then both the toolstack and the
hypervisor would have to be rebuilt. With the current approach I don't
think there is room for anything else, but I'm curious to hear about
other ideas.

Cheers,
Costin

Re: [PATCH v2 1/4] public: Add page related definitions for accessing guests memory

[Jan Beulich]

On 20.08.2021 09:44, Costin Lupu wrote:
> On 8/20/21 9:52 AM, Jan Beulich wrote:
>> On 19.08.2021 19:50, Costin Lupu wrote:
>>> These changes introduce the page related definitions needed for mapping and
>>> accessing guests memory. These values are intended to be used by any toolstack
>>> component that needs to map guests memory. Until now, the values were defined
>>> by the xenctrl.h header, therefore whenever a component had to use them it also
>>> had to add a dependency for the xenctrl library.
>>>
>>> This patch also introduces xen_mk_long() macrodefinition for defining long
>>> constants both for C and assembler code.
>>
>> I'm still missing justification for the addition of a new header, especially
>> as I don't see that header to gain much more contents down the road.
>>
> 
> For the first version, since it didn't need to include other headers, I
> thought it would make sense to isolate the definitions in their own
> headers. Now maybe it makes more sense to put the definitions in
> arch-x86/xen.h, arch-arm.h and xen.h (the latter two) respectively. What
> do you think? I'm open to suggestions here.

Well, yes, the headers you name are where I thought these new #define-s
would go.

Jan

Re: [PATCH v2 1/4] public: Add page related definitions for accessing guests memory

[Julien Grall]

Hi,

Replying to a single e-mail for simplicity.

On 20/08/2021 08:44, Costin Lupu wrote:
> On 8/20/21 9:52 AM, Jan Beulich wrote:
>>> --- /dev/null
>>> +++ b/xen/include/public/page.h
>>> @@ -0,0 +1,36 @@
>>> +/******************************************************************************
>>> + * page.h
>>> + *
>>> + * Page definitions for accessing guests memory
>>> + *
>>> + * Permission is hereby granted, free of charge, to any person obtaining a copy
>>> + * of this software and associated documentation files (the "Software"), to
>>> + * deal in the Software without restriction, including without limitation the
>>> + * rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
>>> + * sell copies of the Software, and to permit persons to whom the Software is
>>> + * furnished to do so, subject to the following conditions:
>>> + *
>>> + * The above copyright notice and this permission notice shall be included in
>>> + * all copies or substantial portions of the Software.
>>> + *
>>> + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
>>> + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
>>> + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
>>> + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
>>> + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
>>> + * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
>>> + * DEALINGS IN THE SOFTWARE.
>>> + *
>>> + * Copyright (c) 2021, Costin Lupu
>>> + */
>>> +
>>> +#ifndef __XEN_PUBLIC_PAGE_H__
>>> +#define __XEN_PUBLIC_PAGE_H__
>>> +
>>> +#include "xen.h"
>>> +
>>> +#define XEN_PAGE_SHIFT           12
>>> +#define XEN_PAGE_SIZE            (xen_mk_long(1) << XEN_PAGE_SHIFT)

This will use UL whereas on Arm a page frame should always be 64-bit 
regardless the bitness. Shouldn't this be converted to use xen_ulong_t 
instead?

>>> +#define XEN_PAGE_MASK            (~(XEN_PAGE_SIZE - 1))
>>
>> You went too far here, I'm afraid: In reply to v1 I did say "The latter
>> two, being identical ..." - XEN_PAGE_SHIFT ought to continue to be a
>> per-arch constant, even if right now it is the same for Arm and x86.
>>
> 
> Alright, now I got it.
> 
>> Thinking of which - with exposing this as a stable ABI (not just the
>> abstraction, but the specific values and the fact that they're
>> invariable become part of the stable ABI this way), what is the plan
>> for supporting 64k(?) page size on Arm in the future? At that point
>> you _cannot_ simply remove or replace the #define you add here. As
>> the immediate need is by the tool stack, enclosing in

I would like to get support for 64KB and 16KB pages in Xen (we already 
support in guests). But there is not much we can do with the current ABI 
as guests assume this is 4KB (that would break if Xen was using 64KB).

>>
>> #if defined(__XEN__) || defined(__XEN_TOOLS__)
>>
>> might be an option, with the downside of having stable libraries
>> (foreignmemory and gnttab) depend on an unstable hypervisor interface
>> (again). I can't seem to be able to think of anything better ...
I am not sure why you write (again) here. The two libraries always 
assumed the hypervisor was using 4KB page granularity. But until 
recently it was also assuming that the OS page granularity matched.

Our stable ABI has not been designed with multiple page granularity in 
mind. We could introduce a hypercall to query the page size used by the 
ABI. But then, I don't think we have the full picture of how this is 
going to pan out (I haven't try to use another page size on Xen yet).

I think we have three choices here:
   1) Stick with the existing definition in the tools
   2) Move the definition in the public headers and only expose them to 
the tools.
   3) Query the page size via a new hypervisor

As I wrote above, 3) is going to take some time to get it right. So the 
question here is whether 2) is temporarily better than 1).

I think it is because it at least clarify a long standing question on 
what is the granularity of page in the ABI.

Cheers,

-- 
Julien Grall

Re: [PATCH v2 1/4] public: Add page related definitions for accessing guests memory

[Jan Beulich]

On 20.08.2021 11:08, Julien Grall wrote:
> On 20/08/2021 08:44, Costin Lupu wrote:
>> On 8/20/21 9:52 AM, Jan Beulich wrote:
>>>> --- /dev/null
>>>> +++ b/xen/include/public/page.h
>>>> @@ -0,0 +1,36 @@
>>>> +/******************************************************************************
>>>> + * page.h
>>>> + *
>>>> + * Page definitions for accessing guests memory
>>>> + *
>>>> + * Permission is hereby granted, free of charge, to any person obtaining a copy
>>>> + * of this software and associated documentation files (the "Software"), to
>>>> + * deal in the Software without restriction, including without limitation the
>>>> + * rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
>>>> + * sell copies of the Software, and to permit persons to whom the Software is
>>>> + * furnished to do so, subject to the following conditions:
>>>> + *
>>>> + * The above copyright notice and this permission notice shall be included in
>>>> + * all copies or substantial portions of the Software.
>>>> + *
>>>> + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
>>>> + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
>>>> + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
>>>> + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
>>>> + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
>>>> + * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
>>>> + * DEALINGS IN THE SOFTWARE.
>>>> + *
>>>> + * Copyright (c) 2021, Costin Lupu
>>>> + */
>>>> +
>>>> +#ifndef __XEN_PUBLIC_PAGE_H__
>>>> +#define __XEN_PUBLIC_PAGE_H__
>>>> +
>>>> +#include "xen.h"
>>>> +
>>>> +#define XEN_PAGE_SHIFT           12
>>>> +#define XEN_PAGE_SIZE            (xen_mk_long(1) << XEN_PAGE_SHIFT)
> 
> This will use UL whereas on Arm a page frame should always be 64-bit 
> regardless the bitness. Shouldn't this be converted to use xen_ulong_t 
> instead?

As pointed out on v1, XEN_PAGE_SIZE would better not end up as a
value of signed type, for ...

>>>> +#define XEN_PAGE_MASK            (~(XEN_PAGE_SIZE - 1))

... this to suitably sign-extend to wider types is necessary.

Also unless you expect someone to use typeof(XEN_PAGE_SIZE) I'm
afraid I don't see where the constant being long vs xen_long_t
(if such existed) might matter. Otoh perhaps xen_mk_ulong() would
better have produced a xen_ulong_t typed values in the first
place, but I'm afraid we can't alter the existing macro.

>>> Thinking of which - with exposing this as a stable ABI (not just the
>>> abstraction, but the specific values and the fact that they're
>>> invariable become part of the stable ABI this way), what is the plan
>>> for supporting 64k(?) page size on Arm in the future? At that point
>>> you _cannot_ simply remove or replace the #define you add here. As
>>> the immediate need is by the tool stack, enclosing in
> 
> I would like to get support for 64KB and 16KB pages in Xen (we already 
> support in guests). But there is not much we can do with the current ABI 
> as guests assume this is 4KB (that would break if Xen was using 64KB).
> 
>>>
>>> #if defined(__XEN__) || defined(__XEN_TOOLS__)
>>>
>>> might be an option, with the downside of having stable libraries
>>> (foreignmemory and gnttab) depend on an unstable hypervisor interface
>>> (again). I can't seem to be able to think of anything better ...
> I am not sure why you write (again) here. The two libraries always 
> assumed the hypervisor was using 4KB page granularity. But until 
> recently it was also assuming that the OS page granularity matched.

Assuming 4k page size was a plain assumption imo, not reliance on
a stable hypervisor interface. Just like is assuming OS and
hypervisor page sizes would match.

I wrote "(again)" because at least the foreignmemory code was, iirc,
split off of libxc at some point, i.e. used to be unstable and hence
was "fine" to rely on unstable hypervisor interfaces.

> Our stable ABI has not been designed with multiple page granularity in 
> mind. We could introduce a hypercall to query the page size used by the 
> ABI. But then, I don't think we have the full picture of how this is 
> going to pan out (I haven't try to use another page size on Xen yet).
> 
> I think we have three choices here:
>    1) Stick with the existing definition in the tools
>    2) Move the definition in the public headers and only expose them to 
> the tools.
>    3) Query the page size via a new hypervisor
> 
> As I wrote above, 3) is going to take some time to get it right. So the 
> question here is whether 2) is temporarily better than 1).

Because I understand 3) is some way out, and because I think 2) is
better than 1), I wrote "might be an option" for what you call 2).
But I could see people (Andrew for example) to take a different
position and object to such a temporary measure.

Jan

Re: [PATCH v2 1/4] public: Add page related definitions for accessing guests memory

[Julien Grall]

Hi Jan,

On 20/08/2021 10:26, Jan Beulich wrote:
> On 20.08.2021 11:08, Julien Grall wrote:
>> On 20/08/2021 08:44, Costin Lupu wrote:
>>> On 8/20/21 9:52 AM, Jan Beulich wrote:
>>>>> --- /dev/null
>>>>> +++ b/xen/include/public/page.h
>>>>> @@ -0,0 +1,36 @@
>>>>> +/******************************************************************************
>>>>> + * page.h
>>>>> + *
>>>>> + * Page definitions for accessing guests memory
>>>>> + *
>>>>> + * Permission is hereby granted, free of charge, to any person obtaining a copy
>>>>> + * of this software and associated documentation files (the "Software"), to
>>>>> + * deal in the Software without restriction, including without limitation the
>>>>> + * rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
>>>>> + * sell copies of the Software, and to permit persons to whom the Software is
>>>>> + * furnished to do so, subject to the following conditions:
>>>>> + *
>>>>> + * The above copyright notice and this permission notice shall be included in
>>>>> + * all copies or substantial portions of the Software.
>>>>> + *
>>>>> + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
>>>>> + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
>>>>> + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
>>>>> + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
>>>>> + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
>>>>> + * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
>>>>> + * DEALINGS IN THE SOFTWARE.
>>>>> + *
>>>>> + * Copyright (c) 2021, Costin Lupu
>>>>> + */
>>>>> +
>>>>> +#ifndef __XEN_PUBLIC_PAGE_H__
>>>>> +#define __XEN_PUBLIC_PAGE_H__
>>>>> +
>>>>> +#include "xen.h"
>>>>> +
>>>>> +#define XEN_PAGE_SHIFT           12
>>>>> +#define XEN_PAGE_SIZE            (xen_mk_long(1) << XEN_PAGE_SHIFT)
>>
>> This will use UL whereas on Arm a page frame should always be 64-bit
>> regardless the bitness. Shouldn't this be converted to use xen_ulong_t
>> instead?
> 
> As pointed out on v1, XEN_PAGE_SIZE would better not end up as a
> value of signed type, for ...

Did you mean "not end up as a value of **unsigned** type"...

> 
>>>>> +#define XEN_PAGE_MASK            (~(XEN_PAGE_SIZE - 1))
> 
> ... this to suitably sign-extend to wider types is necessary.

... because, if I am not mistaken, the sign-extension wouldn't happen 
with unsigned type. But then on v1 you wrote:

"Imo the smallest type this should evaluate to is xen_ulong_t"

Which I interpreted as this value should be 64-bit on Arm32. If this not 
what you meant then I am lost.

> 
> Also unless you expect someone to use typeof(XEN_PAGE_SIZE) I'm
> afraid I don't see where the constant being long vs xen_long_t
> (if such existed) might matter.
> Otoh perhaps xen_mk_ulong() would
> better have produced a xen_ulong_t typed values in the first
> place, but I'm afraid we can't alter the existing macro.

We can create a new one.

>> Our stable ABI has not been designed with multiple page granularity in
>> mind. We could introduce a hypercall to query the page size used by the
>> ABI. But then, I don't think we have the full picture of how this is
>> going to pan out (I haven't try to use another page size on Xen yet).
>>
>> I think we have three choices here:
>>     1) Stick with the existing definition in the tools
>>     2) Move the definition in the public headers and only expose them to
>> the tools.
>>     3) Query the page size via a new hypervisor
>>
>> As I wrote above, 3) is going to take some time to get it right. So the
>> question here is whether 2) is temporarily better than 1).
> 
> Because I understand 3) is some way out, and because I think 2) is
> better than 1), I wrote "might be an option" for what you call 2).
> But I could see people (Andrew for example) to take a different
> position and object to such a temporary measure.

I think we need to make a decision so Costin doesn't keep sending 
version on something that can't be merged. What does the others thinks?

Cheers,

-- 
Julien Grall

Re: [PATCH v2 1/4] public: Add page related definitions for accessing guests memory

[Jan Beulich]

On 23.08.2021 19:16, Julien Grall wrote:
> Hi Jan,
> 
> On 20/08/2021 10:26, Jan Beulich wrote:
>> On 20.08.2021 11:08, Julien Grall wrote:
>>> On 20/08/2021 08:44, Costin Lupu wrote:
>>>> On 8/20/21 9:52 AM, Jan Beulich wrote:
>>>>>> --- /dev/null
>>>>>> +++ b/xen/include/public/page.h
>>>>>> @@ -0,0 +1,36 @@
>>>>>> +/******************************************************************************
>>>>>> + * page.h
>>>>>> + *
>>>>>> + * Page definitions for accessing guests memory
>>>>>> + *
>>>>>> + * Permission is hereby granted, free of charge, to any person obtaining a copy
>>>>>> + * of this software and associated documentation files (the "Software"), to
>>>>>> + * deal in the Software without restriction, including without limitation the
>>>>>> + * rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
>>>>>> + * sell copies of the Software, and to permit persons to whom the Software is
>>>>>> + * furnished to do so, subject to the following conditions:
>>>>>> + *
>>>>>> + * The above copyright notice and this permission notice shall be included in
>>>>>> + * all copies or substantial portions of the Software.
>>>>>> + *
>>>>>> + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
>>>>>> + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
>>>>>> + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
>>>>>> + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
>>>>>> + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
>>>>>> + * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
>>>>>> + * DEALINGS IN THE SOFTWARE.
>>>>>> + *
>>>>>> + * Copyright (c) 2021, Costin Lupu
>>>>>> + */
>>>>>> +
>>>>>> +#ifndef __XEN_PUBLIC_PAGE_H__
>>>>>> +#define __XEN_PUBLIC_PAGE_H__
>>>>>> +
>>>>>> +#include "xen.h"
>>>>>> +
>>>>>> +#define XEN_PAGE_SHIFT           12
>>>>>> +#define XEN_PAGE_SIZE            (xen_mk_long(1) << XEN_PAGE_SHIFT)
>>>
>>> This will use UL whereas on Arm a page frame should always be 64-bit
>>> regardless the bitness. Shouldn't this be converted to use xen_ulong_t
>>> instead?
>>
>> As pointed out on v1, XEN_PAGE_SIZE would better not end up as a
>> value of signed type, for ...
> 
> Did you mean "not end up as a value of **unsigned** type"...

Oh, of course I did. I'm sorry for the confusion caused.

>>>>>> +#define XEN_PAGE_MASK            (~(XEN_PAGE_SIZE - 1))
>>
>> ... this to suitably sign-extend to wider types is necessary.
> 
> ... because, if I am not mistaken, the sign-extension wouldn't happen 
> with unsigned type. But then on v1 you wrote:
> 
> "Imo the smallest type this should evaluate to is xen_ulong_t"
> 
> Which I interpreted as this value should be 64-bit on Arm32. If this not 
> what you meant then I am lost.

And there I would better have said "If indeed unsigned for whatever
reason, the smallest type this should evaluate to is xen_ulong_t."

>> Also unless you expect someone to use typeof(XEN_PAGE_SIZE) I'm
>> afraid I don't see where the constant being long vs xen_long_t
>> (if such existed) might matter.
>> Otoh perhaps xen_mk_ulong() would
>> better have produced a xen_ulong_t typed values in the first
>> place, but I'm afraid we can't alter the existing macro.
> 
> We can create a new one.

But we shouldn't carelessly add stuff, as we can't later remove it.

Jan

Re: [PATCH v2 1/4] public: Add page related definitions for accessing guests memory

[Costin Lupu]

Hi guys,

On 8/23/21 8:16 PM, Julien Grall wrote:
> Hi Jan,
> 
> On 20/08/2021 10:26, Jan Beulich wrote:
>> On 20.08.2021 11:08, Julien Grall wrote:
>>> On 20/08/2021 08:44, Costin Lupu wrote:
>>>> On 8/20/21 9:52 AM, Jan Beulich wrote:
>>>>>> --- /dev/null
>>>>>> +++ b/xen/include/public/page.h
>>>>>> @@ -0,0 +1,36 @@
>>>>>> +/******************************************************************************
>>>>>>
>>>>>> + * page.h
>>>>>> + *
>>>>>> + * Page definitions for accessing guests memory
>>>>>> + *
>>>>>> + * Permission is hereby granted, free of charge, to any person
>>>>>> obtaining a copy
>>>>>> + * of this software and associated documentation files (the
>>>>>> "Software"), to
>>>>>> + * deal in the Software without restriction, including without
>>>>>> limitation the
>>>>>> + * rights to use, copy, modify, merge, publish, distribute,
>>>>>> sublicense, and/or
>>>>>> + * sell copies of the Software, and to permit persons to whom the
>>>>>> Software is
>>>>>> + * furnished to do so, subject to the following conditions:
>>>>>> + *
>>>>>> + * The above copyright notice and this permission notice shall be
>>>>>> included in
>>>>>> + * all copies or substantial portions of the Software.
>>>>>> + *
>>>>>> + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY
>>>>>> KIND, EXPRESS OR
>>>>>> + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
>>>>>> MERCHANTABILITY,
>>>>>> + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO
>>>>>> EVENT SHALL THE
>>>>>> + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES
>>>>>> OR OTHER
>>>>>> + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR
>>>>>> OTHERWISE, ARISING
>>>>>> + * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
>>>>>> OTHER
>>>>>> + * DEALINGS IN THE SOFTWARE.
>>>>>> + *
>>>>>> + * Copyright (c) 2021, Costin Lupu
>>>>>> + */
>>>>>> +
>>>>>> +#ifndef __XEN_PUBLIC_PAGE_H__
>>>>>> +#define __XEN_PUBLIC_PAGE_H__
>>>>>> +
>>>>>> +#include "xen.h"
>>>>>> +
>>>>>> +#define XEN_PAGE_SHIFT           12
>>>>>> +#define XEN_PAGE_SIZE            (xen_mk_long(1) << XEN_PAGE_SHIFT)
>>>
>>> This will use UL whereas on Arm a page frame should always be 64-bit
>>> regardless the bitness. Shouldn't this be converted to use xen_ulong_t
>>> instead?
>>
>> As pointed out on v1, XEN_PAGE_SIZE would better not end up as a
>> value of signed type, for ...
> 
> Did you mean "not end up as a value of **unsigned** type"...
> 
>>
>>>>>> +#define XEN_PAGE_MASK            (~(XEN_PAGE_SIZE - 1))
>>
>> ... this to suitably sign-extend to wider types is necessary.
> 
> ... because, if I am not mistaken, the sign-extension wouldn't happen
> with unsigned type. But then on v1 you wrote:
> 
> "Imo the smallest type this should evaluate to is xen_ulong_t"
> 
> Which I interpreted as this value should be 64-bit on Arm32. If this not
> what you meant then I am lost.
> 
>>
>> Also unless you expect someone to use typeof(XEN_PAGE_SIZE) I'm
>> afraid I don't see where the constant being long vs xen_long_t
>> (if such existed) might matter.
>> Otoh perhaps xen_mk_ulong() would
>> better have produced a xen_ulong_t typed values in the first
>> place, but I'm afraid we can't alter the existing macro.
> 
> We can create a new one.
> 
>>> Our stable ABI has not been designed with multiple page granularity in
>>> mind. We could introduce a hypercall to query the page size used by the
>>> ABI. But then, I don't think we have the full picture of how this is
>>> going to pan out (I haven't try to use another page size on Xen yet).
>>>
>>> I think we have three choices here:
>>>     1) Stick with the existing definition in the tools
>>>     2) Move the definition in the public headers and only expose them to
>>> the tools.
>>>     3) Query the page size via a new hypervisor
>>>
>>> As I wrote above, 3) is going to take some time to get it right. So the
>>> question here is whether 2) is temporarily better than 1).
>>
>> Because I understand 3) is some way out, and because I think 2) is
>> better than 1), I wrote "might be an option" for what you call 2).
>> But I could see people (Andrew for example) to take a different
>> position and object to such a temporary measure.
> 
> I think we need to make a decision so Costin doesn't keep sending
> version on something that can't be merged. What does the others thinks?

From what I understood, in his last reply to 'stubdom: foreignmemory:
Fix build after 0dbb4be739c5' thread, Andrew was OK with solution 2).

Cheers,
Costin

Re: [PATCH v2 1/4] public: Add page related definitions for accessing guests memory

[Jan Beulich]

On 24.08.2021 15:03, Costin Lupu wrote:
> On 8/23/21 8:16 PM, Julien Grall wrote:
>> On 20/08/2021 10:26, Jan Beulich wrote:
>>> On 20.08.2021 11:08, Julien Grall wrote:
>>>> On 20/08/2021 08:44, Costin Lupu wrote:
>>>>> On 8/20/21 9:52 AM, Jan Beulich wrote:
>>>>>>> --- /dev/null
>>>>>>> +++ b/xen/include/public/page.h
>>>>>>> @@ -0,0 +1,36 @@
>>>>>>> +/******************************************************************************
>>>>>>>
>>>>>>> + * page.h
>>>>>>> + *
>>>>>>> + * Page definitions for accessing guests memory
>>>>>>> + *
>>>>>>> + * Permission is hereby granted, free of charge, to any person
>>>>>>> obtaining a copy
>>>>>>> + * of this software and associated documentation files (the
>>>>>>> "Software"), to
>>>>>>> + * deal in the Software without restriction, including without
>>>>>>> limitation the
>>>>>>> + * rights to use, copy, modify, merge, publish, distribute,
>>>>>>> sublicense, and/or
>>>>>>> + * sell copies of the Software, and to permit persons to whom the
>>>>>>> Software is
>>>>>>> + * furnished to do so, subject to the following conditions:
>>>>>>> + *
>>>>>>> + * The above copyright notice and this permission notice shall be
>>>>>>> included in
>>>>>>> + * all copies or substantial portions of the Software.
>>>>>>> + *
>>>>>>> + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY
>>>>>>> KIND, EXPRESS OR
>>>>>>> + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
>>>>>>> MERCHANTABILITY,
>>>>>>> + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO
>>>>>>> EVENT SHALL THE
>>>>>>> + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES
>>>>>>> OR OTHER
>>>>>>> + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR
>>>>>>> OTHERWISE, ARISING
>>>>>>> + * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
>>>>>>> OTHER
>>>>>>> + * DEALINGS IN THE SOFTWARE.
>>>>>>> + *
>>>>>>> + * Copyright (c) 2021, Costin Lupu
>>>>>>> + */
>>>>>>> +
>>>>>>> +#ifndef __XEN_PUBLIC_PAGE_H__
>>>>>>> +#define __XEN_PUBLIC_PAGE_H__
>>>>>>> +
>>>>>>> +#include "xen.h"
>>>>>>> +
>>>>>>> +#define XEN_PAGE_SHIFT           12
>>>>>>> +#define XEN_PAGE_SIZE            (xen_mk_long(1) << XEN_PAGE_SHIFT)
>>>>
>>>> This will use UL whereas on Arm a page frame should always be 64-bit
>>>> regardless the bitness. Shouldn't this be converted to use xen_ulong_t
>>>> instead?
>>>
>>> As pointed out on v1, XEN_PAGE_SIZE would better not end up as a
>>> value of signed type, for ...
>>
>> Did you mean "not end up as a value of **unsigned** type"...
>>
>>>
>>>>>>> +#define XEN_PAGE_MASK            (~(XEN_PAGE_SIZE - 1))
>>>
>>> ... this to suitably sign-extend to wider types is necessary.
>>
>> ... because, if I am not mistaken, the sign-extension wouldn't happen
>> with unsigned type. But then on v1 you wrote:
>>
>> "Imo the smallest type this should evaluate to is xen_ulong_t"
>>
>> Which I interpreted as this value should be 64-bit on Arm32. If this not
>> what you meant then I am lost.
>>
>>>
>>> Also unless you expect someone to use typeof(XEN_PAGE_SIZE) I'm
>>> afraid I don't see where the constant being long vs xen_long_t
>>> (if such existed) might matter.
>>> Otoh perhaps xen_mk_ulong() would
>>> better have produced a xen_ulong_t typed values in the first
>>> place, but I'm afraid we can't alter the existing macro.
>>
>> We can create a new one.
>>
>>>> Our stable ABI has not been designed with multiple page granularity in
>>>> mind. We could introduce a hypercall to query the page size used by the
>>>> ABI. But then, I don't think we have the full picture of how this is
>>>> going to pan out (I haven't try to use another page size on Xen yet).
>>>>
>>>> I think we have three choices here:
>>>>     1) Stick with the existing definition in the tools
>>>>     2) Move the definition in the public headers and only expose them to
>>>> the tools.
>>>>     3) Query the page size via a new hypervisor
>>>>
>>>> As I wrote above, 3) is going to take some time to get it right. So the
>>>> question here is whether 2) is temporarily better than 1).
>>>
>>> Because I understand 3) is some way out, and because I think 2) is
>>> better than 1), I wrote "might be an option" for what you call 2).
>>> But I could see people (Andrew for example) to take a different
>>> position and object to such a temporary measure.
>>
>> I think we need to make a decision so Costin doesn't keep sending
>> version on something that can't be merged. What does the others thinks?
> 
> From what I understood, in his last reply to 'stubdom: foreignmemory:
> Fix build after 0dbb4be739c5' thread, Andrew was OK with solution 2).

I agree it can be read this way. 

Jan