[PATCH RFC 00/18] System adjustment to customer needs.

[PATCH RFC 00/18] System adjustment to customer needs.

[Andrii Anisov]

This series RFC patches from the currently ongoing production project.
This patch series presents changes needed to fit the system into
customer requirements as well as workaround limitations of the
Jacinto6 SoC.
This patch series is not a full production branch upstream. Lacks of
the board specific patches as well as new PV drivers interfaces.
Patches for new PV drivers interfaces will be submitted soon.

Andrii Anisov (1):
  xen/tools: Fix virtual disks helper scripts.

Andrii Tseglytskyi (2):
  xen/arm: allow to allocate 1/128/256/512 Mb memory chunks
  xen/arm: allow reassigning of hw interrupts to guest domain

Iurii Konovalenko (3):
  arm: Fix 1-to-1 Dom0 memory allocation of any size
  arm: Add ability to relocate Xen in over 4GB space
  arm: Add ability to allocate Xen heap in lowmem

Iurii Mykhalskyi (3):
  xen: flask: Add possiblity to forward irqs into domU domains
  xen: Add dom0_mem_high option & over 4GB memory allocation for Dom0
  tools: Introduce ARM32_SEPAR_MEM_SPLIT option

Oleksandr Dmytryshyn (3):
  tools: Allow to cross-compile xentop
  xen: arm: add batch support to the XENMEM_p2m_lookup operation
  xen: introduce and use 'dom0_rambase_pfn' setting for kernel Dom0

Oleksandr Tyshchenko (1):
  libxl: add ability to set rambase_pfn via cfg file

Sergiy Kibrik (1):
  kbdif: add raw events passing

Viktor Kleinik (4):
  libxl: parse config data during domain reboot
  tools/misc: Modify Xen watchdog daemon
  tools/misc: Set timeout value from watchdog daemon
  libxl: Fix unneeded domain reboot during destroy routine

 tools/flask/policy/policy/modules/xen/xen.te |   1 +
 tools/hotplug/Linux/block                    |   2 +-
 tools/hotplug/Linux/locking.sh               |   9 +-
 tools/hotplug/Linux/xen-hotplug-common.sh.in |   2 +-
 tools/libxc/Makefile                         |   2 +
 tools/libxc/include/xc_dom.h                 |  24 ++-
 tools/libxc/xc_dom_arm.c                     | 103 +++++++++++-
 tools/libxc/xc_dom_compat_linux.c            |   5 +
 tools/libxc/xc_dom_core.c                    |  51 +++++-
 tools/libxl/Makefile                         |   2 +
 tools/libxl/libxl.c                          |   4 +
 tools/libxl/libxl_arm.c                      |  21 ++-
 tools/libxl/libxl_create.c                   |   5 +
 tools/libxl/libxl_dom.c                      |  29 +++-
 tools/libxl/libxl_internal.h                 |   1 +
 tools/libxl/libxl_types.idl                  |   2 +
 tools/libxl/xl_cmdimpl.c                     |  33 ++++
 tools/misc/xenwatchdogd.c                    |  54 ++-----
 tools/xenstat/Makefile                       |   4 +-
 tools/xenstore/init-xenstore-domain.c        |   5 +
 xen/Rules.mk                                 |   3 +
 xen/arch/arm/arm32/head.S                    |  21 ++-
 xen/arch/arm/domain_build.c                  | 224 ++++++++++++++++++++++++++-
 xen/arch/arm/irq.c                           |  19 ++-
 xen/arch/arm/kernel.h                        |   9 ++
 xen/arch/arm/mm.c                            |  33 ++++
 xen/arch/arm/platforms/omap5.c               |  17 +-
 xen/arch/arm/platforms/rcar2.c               |   9 +-
 xen/arch/arm/setup.c                         |  35 ++++-
 xen/arch/arm/smpboot.c                       |  33 +++-
 xen/common/memory.c                          |  16 +-
 xen/common/page_alloc.c                      |  91 +++++++++--
 xen/include/public/io/kbdif.h                |  10 ++
 xen/include/public/memory.h                  |  18 ++-
 xen/include/xen/mm.h                         |   8 +
 35 files changed, 801 insertions(+), 104 deletions(-)

-- 
2.8.2


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[PATCH RFC 01/18] xen/tools: Fix virtual disks helper scripts.

[Andrii Anisov]

This patch makes virtual disks helper scripts be functional
in busybox environment. Actually we call sh insteand of bash and
rewrite loop with counter to be properly parsed by ash.

Signed-off-by: Andrii Anisov <andrii.anisov@globallogic.com>
Signed-off-by: Andrii Tseglytskyi <andrii.tseglytskyi@globallogic.com>
---
 tools/hotplug/Linux/block                    | 2 +-
 tools/hotplug/Linux/locking.sh               | 9 +++++++--
 tools/hotplug/Linux/xen-hotplug-common.sh.in | 2 +-
 3 files changed, 9 insertions(+), 4 deletions(-)

diff --git a/tools/hotplug/Linux/block b/tools/hotplug/Linux/block
index 8d2ee9d..6a725db 100644
--- a/tools/hotplug/Linux/block
+++ b/tools/hotplug/Linux/block
@@ -1,4 +1,4 @@
-#!/bin/bash
+#!/bin/sh
 
 dir=$(dirname "$0")
 . "$dir/block-common.sh"
diff --git a/tools/hotplug/Linux/locking.sh b/tools/hotplug/Linux/locking.sh
index c6a7e96..b8e9515 100644
--- a/tools/hotplug/Linux/locking.sh
+++ b/tools/hotplug/Linux/locking.sh
@@ -23,9 +23,14 @@ LOCK_BASEDIR=/var/run/xen-hotplug
 
 _setlockfd()
 {
+    local lock_
     local i
-    for ((i = 0; i < ${#_lockdict}; i++))
-    do [ -z "${_lockdict[$i]}" -o "${_lockdict[$i]}" = "$1" ] && break
+    let i=0
+
+    for lock_ in _lockdict ;
+    do
+    [ -z "$lock_" -o "$lock_" = "$1" ] && break
+    (( i++ ))
     done
     _lockdict[$i]="$1"
     let _lockfd=200+i
diff --git a/tools/hotplug/Linux/xen-hotplug-common.sh.in b/tools/hotplug/Linux/xen-hotplug-common.sh.in
index d5d0b69..42e46e3 100644
--- a/tools/hotplug/Linux/xen-hotplug-common.sh.in
+++ b/tools/hotplug/Linux/xen-hotplug-common.sh.in
@@ -51,7 +51,7 @@ sigerr() {
   fatal "$0 failed; error detected."
 }
 
-trap sigerr ERR
+#trap sigerr ERR
 
 
 ##
-- 
2.8.2


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH RFC 01/18] xen/tools: Fix virtual disks helper scripts.

[Wei Liu]

On Wed, May 18, 2016 at 07:32:24PM +0300, Andrii Anisov wrote:
> This patch makes virtual disks helper scripts be functional
> in busybox environment. Actually we call sh insteand of bash and
> rewrite loop with counter to be properly parsed by ash.
> 
> Signed-off-by: Andrii Anisov <andrii.anisov@globallogic.com>
> Signed-off-by: Andrii Tseglytskyi <andrii.tseglytskyi@globallogic.com>
> ---
>  tools/hotplug/Linux/block                    | 2 +-
>  tools/hotplug/Linux/locking.sh               | 9 +++++++--
>  tools/hotplug/Linux/xen-hotplug-common.sh.in | 2 +-
>  3 files changed, 9 insertions(+), 4 deletions(-)
> 
> diff --git a/tools/hotplug/Linux/block b/tools/hotplug/Linux/block
> index 8d2ee9d..6a725db 100644
> --- a/tools/hotplug/Linux/block
> +++ b/tools/hotplug/Linux/block
> @@ -1,4 +1,4 @@
> -#!/bin/bash
> +#!/bin/sh
>  
>  dir=$(dirname "$0")
>  . "$dir/block-common.sh"
> diff --git a/tools/hotplug/Linux/locking.sh b/tools/hotplug/Linux/locking.sh
> index c6a7e96..b8e9515 100644
> --- a/tools/hotplug/Linux/locking.sh
> +++ b/tools/hotplug/Linux/locking.sh
> @@ -23,9 +23,14 @@ LOCK_BASEDIR=/var/run/xen-hotplug
>  
>  _setlockfd()
>  {
> +    local lock_
>      local i
> -    for ((i = 0; i < ${#_lockdict}; i++))
> -    do [ -z "${_lockdict[$i]}" -o "${_lockdict[$i]}" = "$1" ] && break
> +    let i=0
> +
> +    for lock_ in _lockdict ;
> +    do
> +    [ -z "$lock_" -o "$lock_" = "$1" ] && break
> +    (( i++ ))
>      done
>      _lockdict[$i]="$1"
>      let _lockfd=200+i
> diff --git a/tools/hotplug/Linux/xen-hotplug-common.sh.in b/tools/hotplug/Linux/xen-hotplug-common.sh.in
> index d5d0b69..42e46e3 100644
> --- a/tools/hotplug/Linux/xen-hotplug-common.sh.in
> +++ b/tools/hotplug/Linux/xen-hotplug-common.sh.in
> @@ -51,7 +51,7 @@ sigerr() {
>    fatal "$0 failed; error detected."
>  }
>  
> -trap sigerr ERR
> +#trap sigerr ERR

I know why you want to comment this out but you basically break the
error handling protocol. See the fatal function at the beginning of this
file.

So we can't take this patch. And you should probably fix your own
environment, too.

Wei.

>  
>  
>  ##
> -- 
> 2.8.2
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH RFC 01/18] xen/tools: Fix virtual disks helper scripts.

[Andrii Anisov]

Wei,

Actually it should be the oldest patch in our current series, it was
initial researches, I tried to fit the dom0 system rootfs into
smallest possible initramfs so lean busybox was there.

>> +#trap sigerr ERR
> > -trap sigerr ERR
>
> I know why you want to comment this out but you basically break the
> error handling protocol. See the fatal function at the beginning of this
> file.
And yes we should check this particular line change carefully 'cause
it was not intelligent adjusting to busybox, just getting rid of nasty
issue.

> And you should probably fix your own environment, too.
I'm not sure I got the point. If we are speaking about our system we
are tied to this
http://processors.wiki.ti.com/index.php/Category:GLSDK in dom0. I
doubt it will be accepted by customer switching to any rich shell.

Andrii Anisov | Associate Manager, Engineering
GlobalLogic
Kyiv, 03038, Protasov Business Park, M.Grinchenka, 2/1
P +38.044.492.9695x3664  M +380505738852  S andriyanisov
www.globallogic.com

http://www.globallogic.com/email_disclaimer.txt

On Thu, May 19, 2016 at 2:34 PM, Wei Liu <wei.liu2@citrix.com> wrote:
>
> On Wed, May 18, 2016 at 07:32:24PM +0300, Andrii Anisov wrote:
> > This patch makes virtual disks helper scripts be functional
> > in busybox environment. Actually we call sh insteand of bash and
> > rewrite loop with counter to be properly parsed by ash.
> >
> > Signed-off-by: Andrii Anisov <andrii.anisov@globallogic.com>
> > Signed-off-by: Andrii Tseglytskyi <andrii.tseglytskyi@globallogic.com>
> > ---
> >  tools/hotplug/Linux/block                    | 2 +-
> >  tools/hotplug/Linux/locking.sh               | 9 +++++++--
> >  tools/hotplug/Linux/xen-hotplug-common.sh.in | 2 +-
> >  3 files changed, 9 insertions(+), 4 deletions(-)
> >
> > diff --git a/tools/hotplug/Linux/block b/tools/hotplug/Linux/block
> > index 8d2ee9d..6a725db 100644
> > --- a/tools/hotplug/Linux/block
> > +++ b/tools/hotplug/Linux/block
> > @@ -1,4 +1,4 @@
> > -#!/bin/bash
> > +#!/bin/sh
> >
> >  dir=$(dirname "$0")
> >  . "$dir/block-common.sh"
> > diff --git a/tools/hotplug/Linux/locking.sh b/tools/hotplug/Linux/locking.sh
> > index c6a7e96..b8e9515 100644
> > --- a/tools/hotplug/Linux/locking.sh
> > +++ b/tools/hotplug/Linux/locking.sh
> > @@ -23,9 +23,14 @@ LOCK_BASEDIR=/var/run/xen-hotplug
> >
> >  _setlockfd()
> >  {
> > +    local lock_
> >      local i
> > -    for ((i = 0; i < ${#_lockdict}; i++))
> > -    do [ -z "${_lockdict[$i]}" -o "${_lockdict[$i]}" = "$1" ] && break
> > +    let i=0
> > +
> > +    for lock_ in _lockdict ;
> > +    do
> > +    [ -z "$lock_" -o "$lock_" = "$1" ] && break
> > +    (( i++ ))
> >      done
> >      _lockdict[$i]="$1"
> >      let _lockfd=200+i
> > diff --git a/tools/hotplug/Linux/xen-hotplug-common.sh.in b/tools/hotplug/Linux/xen-hotplug-common.sh.in
> > index d5d0b69..42e46e3 100644
> > --- a/tools/hotplug/Linux/xen-hotplug-common.sh.in
> > +++ b/tools/hotplug/Linux/xen-hotplug-common.sh.in
> > @@ -51,7 +51,7 @@ sigerr() {
> >    fatal "$0 failed; error detected."
> >  }
> >
>> +#trap sigerr ERR
> > -trap sigerr ERR
>
> I know why you want to comment this out but you basically break the
> error handling protocol. See the fatal function at the beginning of this
> file.
>
> So we can't take this patch. And you should probably fix your own
> environment, too.
>
> Wei.
>
> >
> >
> >  ##
> > --
> > 2.8.2
> >
> >
> > _______________________________________________
> > Xen-devel mailing list
> > Xen-devel@lists.xen.org
> > http://lists.xen.org/xen-devel

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH RFC 01/18] xen/tools: Fix virtual disks helper scripts.

[Wei Liu]

On Thu, May 19, 2016 at 11:54:55PM +0300, Andrii Anisov wrote:
> Wei,
> 
> Actually it should be the oldest patch in our current series, it was
> initial researches, I tried to fit the dom0 system rootfs into
> smallest possible initramfs so lean busybox was there.
> 
> >> +#trap sigerr ERR
> > > -trap sigerr ERR
> >
> > I know why you want to comment this out but you basically break the
> > error handling protocol. See the fatal function at the beginning of this
> > file.
> And yes we should check this particular line change carefully 'cause
> it was not intelligent adjusting to busybox, just getting rid of nasty
> issue.
> 
> > And you should probably fix your own environment, too.
> I'm not sure I got the point. If we are speaking about our system we
> are tied to this
> http://processors.wiki.ti.com/index.php/Category:GLSDK in dom0. I
> doubt it will be accepted by customer switching to any rich shell.
> 

What I mean is this patch changes (breaks?) error reporting protocol so
your deployment might not function as you expect. And that's something
you need to worry about for your customer.

My bottom line is the error reporting mechanism needs to be preserved,
otherwise we risk breaking other users. Libxl currently relies on those
nodes to report error.

Wei.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH RFC 01/18] xen/tools: Fix virtual disks helper scripts.

[Andrii Anisov]

> My bottom line is the error reporting mechanism needs to be preserved,
> otherwise we risk breaking other users. Libxl currently relies on those
> nodes to report error.

Got it.

Andrii Anisov | Associate Manager, Engineering
GlobalLogic
Kyiv, 03038, Protasov Business Park, M.Grinchenka, 2/1
P +38.044.492.9695x3664  M +380505738852  S andriyanisov
www.globallogic.com

http://www.globallogic.com/email_disclaimer.txt

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[PATCH RFC 02/18] kbdif: add raw events passing

[Andrii Anisov]

From: Sergiy Kibrik <sergiy.kibrik@globallogic.com>

xenkbd_raw carries raw linux event structure -- type, code & value,
which allows support of more devices (e.g. touchscreens).

Signed-off-by: Sergiy Kibrik <sergiy.kibrik@globallogic.com>
---
 xen/include/public/io/kbdif.h | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/xen/include/public/io/kbdif.h b/xen/include/public/io/kbdif.h
index 2d2aebd..ad07ed5 100644
--- a/xen/include/public/io/kbdif.h
+++ b/xen/include/public/io/kbdif.h
@@ -45,6 +45,8 @@
  */
 #define XENKBD_TYPE_POS     4
 
+#define XENKBD_TYPE_RAW     5
+
 struct xenkbd_motion
 {
     uint8_t type;        /* XENKBD_TYPE_MOTION */
@@ -68,6 +70,13 @@ struct xenkbd_position
     int32_t rel_z;       /* relative Z motion (wheel) */
 };
 
+struct xenkbd_raw {
+    uint8_t type;        /* XENKBD_TYPE_RAW */
+    uint16_t input_type;
+    uint16_t code;
+    int32_t value;
+};
+
 #define XENKBD_IN_EVENT_SIZE 40
 
 union xenkbd_in_event
@@ -76,6 +85,7 @@ union xenkbd_in_event
     struct xenkbd_motion motion;
     struct xenkbd_key key;
     struct xenkbd_position pos;
+    struct xenkbd_raw raw;
     char pad[XENKBD_IN_EVENT_SIZE];
 };
 
-- 
2.8.2


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH RFC 02/18] kbdif: add raw events passing

[Jan Beulich]

>>> On 18.05.16 at 18:32, <andrii.anisov@globallogic.com> wrote:
> xenkbd_raw carries raw linux event structure -- type, code & value,
> which allows support of more devices (e.g. touchscreens).

The "raw Linux event structure" may change, so you can't defer to
it. All related definitions would need to be added here, namely ...

> @@ -68,6 +70,13 @@ struct xenkbd_position
>      int32_t rel_z;       /* relative Z motion (wheel) */
>  };
>  
> +struct xenkbd_raw {
> +    uint8_t type;        /* XENKBD_TYPE_RAW */
> +    uint16_t input_type;
> +    uint16_t code;
> +    int32_t value;
> +};

... values to be contained in the three fields which don't have a
comment.

Furthermore it looks like it's the backend that would produce the
new type, and hence there would need to be a means for the
frontend to enable their producing (or else an unsuspecting
frontend might get confused). The way such negotiation works
(likely through xenbus) would need to be documented here, too.

Jan


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[PATCH RFC 03/18] xen/arm: allow to allocate 1/128/256/512 Mb memory chunks

[Andrii Anisov]

From: Andrii Tseglytskyi <andrii.tseglytskyi@globallogic.com>

This is done to allow possibility of having 1 to 1 memory mapping chunks
with size 1/128/256/512 Mb what can sagnificantly decrease time of memory
allocation and fragmentation for 1-to-1 mapped domains.

Signed-off-by: Andrii Tseglytskyi <andrii.tseglytskyi@globallogic.com>
Signed-off-by: Iurii Konovalenko <iurii.konovalenko@globallogic.com>
---
 tools/libxc/xc_dom_arm.c | 24 ++++++++++++++++++++++++
 1 file changed, 24 insertions(+)

diff --git a/tools/libxc/xc_dom_arm.c b/tools/libxc/xc_dom_arm.c
index aeaba54..5ee8eef 100644
--- a/tools/libxc/xc_dom_arm.c
+++ b/tools/libxc/xc_dom_arm.c
@@ -33,7 +33,11 @@
 #define LPAE_SHIFT 9
 
 #define PFN_4K_SHIFT  (0)
+#define PFN_1M_SHIFT  (PFN_4K_SHIFT + 8)
 #define PFN_2M_SHIFT  (PFN_4K_SHIFT+LPAE_SHIFT)
+#define PFN_128M_SHIFT  (PFN_2M_SHIFT + 6)
+#define PFN_256M_SHIFT  (PFN_128M_SHIFT + 1)
+#define PFN_512M_SHIFT  (PFN_256M_SHIFT + 1)
 #define PFN_1G_SHIFT  (PFN_2M_SHIFT+LPAE_SHIFT)
 #define PFN_512G_SHIFT (PFN_1G_SHIFT+LPAE_SHIFT)
 
@@ -359,11 +363,31 @@ static int populate_guest_memory(struct xc_dom_image *dom,
         if ( rc < 0 ) break;
         if ( rc > 0 ) continue;
 
+        rc = populate_one_size(dom, PFN_512M_SHIFT,
+                               base_pfn + pfn, &allocsz, extents);
+        if ( rc < 0 ) break;
+        if ( rc > 0 ) continue;
+
+        rc = populate_one_size(dom, PFN_256M_SHIFT,
+                               base_pfn + pfn, &allocsz, extents);
+        if ( rc < 0 ) break;
+        if ( rc > 0 ) continue;
+
+        rc = populate_one_size(dom, PFN_128M_SHIFT,
+                               base_pfn + pfn, &allocsz, extents);
+        if ( rc < 0 ) break;
+        if ( rc > 0 ) continue;
+
         rc = populate_one_size(dom, PFN_2M_SHIFT,
                                base_pfn + pfn, &allocsz, extents);
         if ( rc < 0 ) break;
         if ( rc > 0 ) continue;
 
+        rc = populate_one_size(dom, PFN_1M_SHIFT,
+                               base_pfn + pfn, &allocsz, extents);
+        if ( rc < 0 ) break;
+        if ( rc > 0 ) continue;
+
         rc = populate_one_size(dom, PFN_4K_SHIFT,
                                base_pfn + pfn, &allocsz, extents);
         if ( rc < 0 ) break;
-- 
2.8.2


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH RFC 03/18] xen/arm: allow to allocate 1/128/256/512 Mb memory chunks

[Julien Grall]

Hello,

On 18/05/16 17:32, Andrii Anisov wrote:
> From: Andrii Tseglytskyi <andrii.tseglytskyi@globallogic.com>
>
> This is done to allow possibility of having 1 to 1 memory mapping chunks
> with size 1/128/256/512 Mb what can sagnificantly decrease time of memory

significantly

> allocation and fragmentation for 1-to-1 mapped domains.

I cannot find any patch in this series to add support for direct memory 
mapped guest.

Can you give details on why this patch would be an improvement for any 
guest and how you choose the new allocations size?

Regards,

>
> Signed-off-by: Andrii Tseglytskyi <andrii.tseglytskyi@globallogic.com>
> Signed-off-by: Iurii Konovalenko <iurii.konovalenko@globallogic.com>
> ---
>   tools/libxc/xc_dom_arm.c | 24 ++++++++++++++++++++++++
>   1 file changed, 24 insertions(+)
>
> diff --git a/tools/libxc/xc_dom_arm.c b/tools/libxc/xc_dom_arm.c
> index aeaba54..5ee8eef 100644
> --- a/tools/libxc/xc_dom_arm.c
> +++ b/tools/libxc/xc_dom_arm.c
> @@ -33,7 +33,11 @@
>   #define LPAE_SHIFT 9
>
>   #define PFN_4K_SHIFT  (0)
> +#define PFN_1M_SHIFT  (PFN_4K_SHIFT + 8)
>   #define PFN_2M_SHIFT  (PFN_4K_SHIFT+LPAE_SHIFT)
> +#define PFN_128M_SHIFT  (PFN_2M_SHIFT + 6)
> +#define PFN_256M_SHIFT  (PFN_128M_SHIFT + 1)
> +#define PFN_512M_SHIFT  (PFN_256M_SHIFT + 1)
>   #define PFN_1G_SHIFT  (PFN_2M_SHIFT+LPAE_SHIFT)
>   #define PFN_512G_SHIFT (PFN_1G_SHIFT+LPAE_SHIFT)
>
> @@ -359,11 +363,31 @@ static int populate_guest_memory(struct xc_dom_image *dom,
>           if ( rc < 0 ) break;
>           if ( rc > 0 ) continue;
>
> +        rc = populate_one_size(dom, PFN_512M_SHIFT,
> +                               base_pfn + pfn, &allocsz, extents);
> +        if ( rc < 0 ) break;
> +        if ( rc > 0 ) continue;
> +
> +        rc = populate_one_size(dom, PFN_256M_SHIFT,
> +                               base_pfn + pfn, &allocsz, extents);
> +        if ( rc < 0 ) break;
> +        if ( rc > 0 ) continue;
> +
> +        rc = populate_one_size(dom, PFN_128M_SHIFT,
> +                               base_pfn + pfn, &allocsz, extents);
> +        if ( rc < 0 ) break;
> +        if ( rc > 0 ) continue;
> +
>           rc = populate_one_size(dom, PFN_2M_SHIFT,
>                                  base_pfn + pfn, &allocsz, extents);
>           if ( rc < 0 ) break;
>           if ( rc > 0 ) continue;
>
> +        rc = populate_one_size(dom, PFN_1M_SHIFT,
> +                               base_pfn + pfn, &allocsz, extents);
> +        if ( rc < 0 ) break;
> +        if ( rc > 0 ) continue;
> +
>           rc = populate_one_size(dom, PFN_4K_SHIFT,
>                                  base_pfn + pfn, &allocsz, extents);
>           if ( rc < 0 ) break;
>

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[PATCH RFC 04/18] libxl: add ability to set rambase_pfn via cfg file

[Andrii Anisov]

From: Oleksandr Tyshchenko <oleksandr.tyshchenko@globallogic.com>

In case of missing required property in cfg file
the default value (0x40000) should be used.

Signed-off-by: Oleksandr Tyshchenko <oleksandr.tyshchenko@globallogic.com>
Signed-off-by: Iurii Konovalenko <iurii.konovalenko@globallogic.com>
Signed-off-by: Iurii Mykhalskyi <iurii.mykhalskyi@globallogic.com>
---
 tools/libxc/xc_dom_arm.c     | 13 +++++++------
 tools/libxl/libxl_arm.c      | 10 ++++++++--
 tools/libxl/libxl_create.c   |  5 +++++
 tools/libxl/libxl_dom.c      |  6 +++---
 tools/libxl/libxl_internal.h |  1 +
 tools/libxl/libxl_types.idl  |  1 +
 tools/libxl/xl_cmdimpl.c     | 13 +++++++++++++
 7 files changed, 38 insertions(+), 11 deletions(-)

diff --git a/tools/libxc/xc_dom_arm.c b/tools/libxc/xc_dom_arm.c
index 5ee8eef..96df669 100644
--- a/tools/libxc/xc_dom_arm.c
+++ b/tools/libxc/xc_dom_arm.c
@@ -415,9 +415,12 @@ int arch_setup_meminit(struct xc_dom_image *dom)
     uint64_t modbase;
 
     uint64_t ramsize = (uint64_t)dom->total_pages << XC_PAGE_SHIFT;
+    uint64_t guest_rambase = (uint64_t)dom->rambase_pfn << XC_PAGE_SHIFT;
+    uint64_t guest_ramsize = (GUEST_RAM0_BASE + GUEST_RAM0_SIZE) -
+                              guest_rambase;
 
-    const uint64_t bankbase[] = GUEST_RAM_BANK_BASES;
-    const uint64_t bankmax[] = GUEST_RAM_BANK_SIZES;
+    const uint64_t bankbase[] = {guest_rambase, GUEST_RAM1_BASE};
+    const uint64_t bankmax[] = {guest_ramsize, GUEST_RAM1_SIZE};
 
     /* Convenient */
     const uint64_t kernbase = dom->kernel_seg.vstart;
@@ -433,8 +436,6 @@ int arch_setup_meminit(struct xc_dom_image *dom)
     xen_pfn_t p2m_size;
     uint64_t bank0end;
 
-    assert(dom->rambase_pfn << XC_PAGE_SHIFT == bankbase[0]);
-
     if ( modsize + kernsize > bankmax[0] )
     {
         DOMPRINTF("%s: Not enough memory for the kernel+dtb+initrd",
@@ -448,11 +449,11 @@ int arch_setup_meminit(struct xc_dom_image *dom)
         return -1;
     }
 
-    if ( ramsize > GUEST_RAM_MAX )
+    if ( ramsize > (bankmax[0] + bankmax[1]) )
     {
         DOMPRINTF("%s: ram size is too large for guest address space: "
                   "%"PRIx64" > %llx",
-                  __FUNCTION__, ramsize, GUEST_RAM_MAX);
+                  __FUNCTION__, ramsize, bankmax[0] + bankmax[1]);
         return -1;
     }
 
diff --git a/tools/libxl/libxl_arm.c b/tools/libxl/libxl_arm.c
index 0af8010..7f9547b 100644
--- a/tools/libxl/libxl_arm.c
+++ b/tools/libxl/libxl_arm.c
@@ -372,7 +372,10 @@ static int make_memory_nodes(libxl__gc *gc, void *fdt,
 {
     int res, i;
     const char *name;
-    const uint64_t bankbase[] = GUEST_RAM_BANK_BASES;
+
+    uint64_t guest_rambase = (uint64_t)dom->rambase_pfn << XC_PAGE_SHIFT;
+
+    const uint64_t bankbase[] = {guest_rambase, GUEST_RAM1_BASE};
 
     for (i = 0; i < GUEST_RAM_BANKS; i++) {
         name = GCSPRINTF("memory@%"PRIx64, bankbase[i]);
@@ -914,7 +917,10 @@ int libxl__arch_domain_finalise_hw_description(libxl__gc *gc,
 {
     void *fdt = dom->devicetree_blob;
     int i;
-    const uint64_t bankbase[] = GUEST_RAM_BANK_BASES;
+
+    uint64_t guest_rambase = (uint64_t)dom->rambase_pfn << XC_PAGE_SHIFT;
+
+    const uint64_t bankbase[] = {guest_rambase, GUEST_RAM1_BASE};
 
     const struct xc_dom_seg *ramdisk = dom->ramdisk_blob ?
         &dom->ramdisk_seg : NULL;
diff --git a/tools/libxl/libxl_create.c b/tools/libxl/libxl_create.c
index 350e274..1c0579c 100644
--- a/tools/libxl/libxl_create.c
+++ b/tools/libxl/libxl_create.c
@@ -182,6 +182,11 @@ int libxl__domain_build_info_setdefault(libxl__gc *gc,
     if (b_info->target_memkb == LIBXL_MEMKB_DEFAULT)
         b_info->target_memkb = b_info->max_memkb;
 
+#ifdef GUEST_RAM_BASE
+    if (b_info->rambase == LIBXL_INVALID_RAM_BASE)
+        b_info->rambase = GUEST_RAM0_BASE;
+#endif
+
     libxl_defbool_setdefault(&b_info->claim_mode, false);
 
     libxl_defbool_setdefault(&b_info->localtime, false);
diff --git a/tools/libxl/libxl_dom.c b/tools/libxl/libxl_dom.c
index 2da3ac4..be6598f 100644
--- a/tools/libxl/libxl_dom.c
+++ b/tools/libxl/libxl_dom.c
@@ -703,12 +703,12 @@ int libxl__build_pv(libxl__gc *gc, uint32_t domid,
         LOGE(ERROR, "xc_dom_boot_xen_init failed");
         goto out;
     }
-#ifdef GUEST_RAM_BASE
-    if ( (ret = xc_dom_rambase_init(dom, GUEST_RAM_BASE)) != 0 ) {
+
+    if ( (ret = xc_dom_rambase_init(dom, info->rambase)) != 0 ) {
         LOGE(ERROR, "xc_dom_rambase failed");
         goto out;
     }
-#endif
+
     if ( (ret = xc_dom_parse_image(dom)) != 0 ) {
         LOGE(ERROR, "xc_dom_parse_image failed");
         goto out;
diff --git a/tools/libxl/libxl_internal.h b/tools/libxl/libxl_internal.h
index 1699f32..5b0b50a 100644
--- a/tools/libxl/libxl_internal.h
+++ b/tools/libxl/libxl_internal.h
@@ -100,6 +100,7 @@
 #define LIBXL_HVM_EXTRA_MEMORY 2048
 #define LIBXL_MIN_DOM0_MEM (128*1024)
 #define LIBXL_INVALID_GFN (~(uint64_t)0)
+#define LIBXL_INVALID_RAM_BASE (~(uint64_t)0)
 /* use 0 as the domid of the toolstack domain for now */
 #define LIBXL_TOOLSTACK_DOMID 0
 #define QEMU_SIGNATURE "DeviceModelRecord0002"
diff --git a/tools/libxl/libxl_types.idl b/tools/libxl/libxl_types.idl
index 502a148..b6cc8d2 100644
--- a/tools/libxl/libxl_types.idl
+++ b/tools/libxl/libxl_types.idl
@@ -416,6 +416,7 @@ libxl_domain_build_info = Struct("domain_build_info",[
     ("target_memkb",    MemKB),
     ("video_memkb",     MemKB),
     ("shadow_memkb",    MemKB),
+    ("rambase",         uint64, {'init_val': 'LIBXL_INVALID_RAM_BASE'}),
     ("rtc_timeoffset",  uint32),
     ("exec_ssidref",    uint32),
     ("exec_ssid_label", string),
diff --git a/tools/libxl/xl_cmdimpl.c b/tools/libxl/xl_cmdimpl.c
index 9a2870e..28d57c3 100644
--- a/tools/libxl/xl_cmdimpl.c
+++ b/tools/libxl/xl_cmdimpl.c
@@ -31,6 +31,7 @@
 #include <sys/select.h>
 #include <sys/utsname.h> /* for utsname in xl info */
 #include <xentoollog.h>
+#include <xenctrl.h>
 #include <ctype.h>
 #include <inttypes.h>
 #include <limits.h>
@@ -1352,6 +1353,18 @@ static void parse_config_data(const char *config_source,
     if (!xlu_cfg_get_long (config, "maxmem", &l, 0))
         b_info->max_memkb = l * 1024;
 
+#ifdef GUEST_RAM_BASE
+    if (!xlu_cfg_get_long (config, "rambase_pfn", &l, 0)) {
+        /* TODO add more detailed check for valid value */
+        uint64_t rambase = (uint64_t)l << XC_PAGE_SHIFT;
+        if (rambase > (GUEST_RAM0_BASE + GUEST_RAM0_SIZE)) {
+            fprintf(stderr, "ERROR: invalid value 0x%lx for \"rambase_pfn\"\n", l);
+            exit (1);
+        }
+        b_info->rambase = rambase;
+    }
+#endif
+
     if (!xlu_cfg_get_long (config, "vcpus", &l, 0)) {
         vcpus = l;
         if (libxl_cpu_bitmap_alloc(ctx, &b_info->avail_vcpus, l)) {
-- 
2.8.2


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH RFC 04/18] libxl: add ability to set rambase_pfn via cfg file

[Wei Liu]

On Wed, May 18, 2016 at 07:32:27PM +0300, Andrii Anisov wrote:
> From: Oleksandr Tyshchenko <oleksandr.tyshchenko@globallogic.com>
> 
> In case of missing required property in cfg file
> the default value (0x40000) should be used.

Assuming this is absolutely needed (whether it is the case I don't
know, I will leave that to ARM maintainers)...

> 
> Signed-off-by: Oleksandr Tyshchenko <oleksandr.tyshchenko@globallogic.com>
> Signed-off-by: Iurii Konovalenko <iurii.konovalenko@globallogic.com>
> Signed-off-by: Iurii Mykhalskyi <iurii.mykhalskyi@globallogic.com>
[...]
> diff --git a/tools/libxl/libxl_internal.h b/tools/libxl/libxl_internal.h
> index 1699f32..5b0b50a 100644
> --- a/tools/libxl/libxl_internal.h
> +++ b/tools/libxl/libxl_internal.h
> @@ -100,6 +100,7 @@
>  #define LIBXL_HVM_EXTRA_MEMORY 2048
>  #define LIBXL_MIN_DOM0_MEM (128*1024)
>  #define LIBXL_INVALID_GFN (~(uint64_t)0)
> +#define LIBXL_INVALID_RAM_BASE (~(uint64_t)0)
>  /* use 0 as the domid of the toolstack domain for now */
>  #define LIBXL_TOOLSTACK_DOMID 0
>  #define QEMU_SIGNATURE "DeviceModelRecord0002"
> diff --git a/tools/libxl/libxl_types.idl b/tools/libxl/libxl_types.idl
> index 502a148..b6cc8d2 100644
> --- a/tools/libxl/libxl_types.idl
> +++ b/tools/libxl/libxl_types.idl
> @@ -416,6 +416,7 @@ libxl_domain_build_info = Struct("domain_build_info",[
>      ("target_memkb",    MemKB),
>      ("video_memkb",     MemKB),
>      ("shadow_memkb",    MemKB),
> +    ("rambase",         uint64, {'init_val': 'LIBXL_INVALID_RAM_BASE'}),

Please make this ARM specific.

You would also need to provide a LIBXL_HAVE macro in libxl.h. See
various examples there.


>      ("rtc_timeoffset",  uint32),
>      ("exec_ssidref",    uint32),
>      ("exec_ssid_label", string),
> diff --git a/tools/libxl/xl_cmdimpl.c b/tools/libxl/xl_cmdimpl.c
> index 9a2870e..28d57c3 100644
> --- a/tools/libxl/xl_cmdimpl.c
> +++ b/tools/libxl/xl_cmdimpl.c
> @@ -31,6 +31,7 @@
>  #include <sys/select.h>
>  #include <sys/utsname.h> /* for utsname in xl info */
>  #include <xentoollog.h>
> +#include <xenctrl.h>

I don't think you need this. In fact, please don't do this. :-)

>  #include <ctype.h>
>  #include <inttypes.h>
>  #include <limits.h>
> @@ -1352,6 +1353,18 @@ static void parse_config_data(const char *config_source,
>      if (!xlu_cfg_get_long (config, "maxmem", &l, 0))
>          b_info->max_memkb = l * 1024;
>  
> +#ifdef GUEST_RAM_BASE
> +    if (!xlu_cfg_get_long (config, "rambase_pfn", &l, 0)) {
> +        /* TODO add more detailed check for valid value */
> +        uint64_t rambase = (uint64_t)l << XC_PAGE_SHIFT;
> +        if (rambase > (GUEST_RAM0_BASE + GUEST_RAM0_SIZE)) {
> +            fprintf(stderr, "ERROR: invalid value 0x%lx for \"rambase_pfn\"\n", l);
> +            exit (1);
> +        }
> +        b_info->rambase = rambase;
> +    }
> +#endif
> +
>      if (!xlu_cfg_get_long (config, "vcpus", &l, 0)) {
>          vcpus = l;
>          if (libxl_cpu_bitmap_alloc(ctx, &b_info->avail_vcpus, l)) {
> -- 
> 2.8.2
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH RFC 04/18] libxl: add ability to set rambase_pfn via cfg file

[Julien Grall]

Hello,

On 18/05/16 17:32, Andrii Anisov wrote:
> From: Oleksandr Tyshchenko <oleksandr.tyshchenko@globallogic.com>
> In case of missing required property in cfg file
> the default value (0x40000) should be used.

The memory layout for the guest is static and may change between 
version. By allowing the user to specify the base of the RAM, you expose 
the Xen internal. The user cannot use this option without knowing the 
version of Xen 4.4 and looking at xen/include/public/arch-arm.h.

Also, you need to explain why a user would want to change the 
rambase_pfn. I suspect it is for supporting direct mapped domain. If so, 
I think the way forward is to expose the same memory layout as the 
hardware rather than trying to make the layout half-static/dynamic.

Exposing the same layout as the hardware would allow you to map the 
device MMIOs 1:1 and avoid unnecessary IPA -> PA translation via an 
hypercall (see patch #10).

Lastly, the commit message should explain the restriction. For instance, 
I would have expect to be able to specify any value. However based on 
the check in xl, this is not possible.

>
> Signed-off-by: Oleksandr Tyshchenko <oleksandr.tyshchenko@globallogic.com>
> Signed-off-by: Iurii Konovalenko <iurii.konovalenko@globallogic.com>
> Signed-off-by: Iurii Mykhalskyi <iurii.mykhalskyi@globallogic.com>
> ---
>   tools/libxc/xc_dom_arm.c     | 13 +++++++------
>   tools/libxl/libxl_arm.c      | 10 ++++++++--
>   tools/libxl/libxl_create.c   |  5 +++++
>   tools/libxl/libxl_dom.c      |  6 +++---
>   tools/libxl/libxl_internal.h |  1 +
>   tools/libxl/libxl_types.idl  |  1 +
>   tools/libxl/xl_cmdimpl.c     | 13 +++++++++++++
>   7 files changed, 38 insertions(+), 11 deletions(-)libxc
>
> diff --git a/tools/libxc/xc_dom_arm.c b/tools/libxc/xc_dom_arm.c
> index 5ee8eef..96df669 100644
> --- a/tools/libxc/xc_dom_arm.c
> +++ b/tools/libxc/xc_dom_arm.c
> @@ -415,9 +415,12 @@ int arch_setup_meminit(struct xc_dom_image *dom)
>       uint64_t modbase;
>
>       uint64_t ramsize = (uint64_t)dom->total_pages << XC_PAGE_SHIFT;
> +    uint64_t guest_rambase = (uint64_t)dom->rambase_pfn << XC_PAGE_SHIFT;
> +    uint64_t guest_ramsize = (GUEST_RAM0_BASE + GUEST_RAM0_SIZE) -
> +                              guest_rambase;
>
> -    const uint64_t bankbase[] = GUEST_RAM_BANK_BASES;
> -    const uint64_t bankmax[] = GUEST_RAM_BANK_SIZES;
> +    const uint64_t bankbase[] = {guest_rambase, GUEST_RAM1_BASE};
> +    const uint64_t bankmax[] = {guest_ramsize, GUEST_RAM1_SIZE};
>
>       /* Convenient */
>       const uint64_t kernbase = dom->kernel_seg.vstart;
> @@ -433,8 +436,6 @@ int arch_setup_meminit(struct xc_dom_image *dom)
>       xen_pfn_t p2m_size;
>       uint64_t bank0end;
>
> -    assert(dom->rambase_pfn << XC_PAGE_SHIFT == bankbase[0]);
> -
>       if ( modsize + kernsize > bankmax[0] )
>       {
>           DOMPRINTF("%s: Not enough memory for the kernel+dtb+initrd",
> @@ -448,11 +449,11 @@ int arch_setup_meminit(struct xc_dom_image *dom)
>           return -1;
>       }
>
> -    if ( ramsize > GUEST_RAM_MAX )
> +    if ( ramsize > (bankmax[0] + bankmax[1]) )
>       {
>           DOMPRINTF("%s: ram size is too large for guest address space: "
>                     "%"PRIx64" > %llx",
> -                  __FUNCTION__, ramsize, GUEST_RAM_MAX);
> +                  __FUNCTION__, ramsize, bankmax[0] + bankmax[1]);
>           return -1;
>       }
>
> diff --git a/tools/libxl/libxl_arm.c b/tools/libxl/libxl_arm.c
> index 0af8010..7f9547b 100644
> --- a/tools/libxl/libxl_arm.c
> +++ b/tools/libxl/libxl_arm.c
> @@ -372,7 +372,10 @@ static int make_memory_nodes(libxl__gc *gc, void *fdt,
>   {
>       int res, i;
>       const char *name;
> -    const uint64_t bankbase[] = GUEST_RAM_BANK_BASES;
> +
> +    uint64_t guest_rambase = (uint64_t)dom->rambase_pfn << XC_PAGE_SHIFT;
> +
> +    const uint64_t bankbase[] = {guest_rambase, GUEST_RAM1_BASE};
>
>       for (i = 0; i < GUEST_RAM_BANKS; i++) {
>           name = GCSPRINTF("memory@%"PRIx64, bankbase[i]);
> @@ -914,7 +917,10 @@ int libxl__arch_domain_finalise_hw_description(libxl__gc *gc,
>   {
>       void *fdt = dom->devicetree_blob;
>       int i;
> -    const uint64_t bankbase[] = GUEST_RAM_BANK_BASES;
> +
> +    uint64_t guest_rambase = (uint64_t)dom->rambase_pfn << XC_PAGE_SHIFT;
> +
> +    const uint64_t bankbase[] = {guest_rambase, GUEST_RAM1_BASE};
>
>       const struct xc_dom_seg *ramdisk = dom->ramdisk_blob ?
>           &dom->ramdisk_seg : NULL;
> diff --git a/tools/libxl/libxl_create.c b/tools/libxl/libxl_create.c
> index 350e274..1c0579c 100644
> --- a/tools/libxl/libxl_create.c
> +++ b/tools/libxl/libxl_create.c
> @@ -182,6 +182,11 @@ int libxl__domain_build_info_setdefault(libxl__gc *gc,
>       if (b_info->target_memkb == LIBXL_MEMKB_DEFAULT)
>           b_info->target_memkb = b_info->max_memkb;
>
> +#ifdef GUEST_RAM_BASE
> +    if (b_info->rambase == LIBXL_INVALID_RAM_BASE)
> +        b_info->rambase = GUEST_RAM0_BASE;

xl is not the only front-end for libxl. So you need to validate the 
value of rambase here.

> +#endif
> +
>       libxl_defbool_setdefault(&b_info->claim_mode, false);
>
>       libxl_defbool_setdefault(&b_info->localtime, false);
> diff --git a/tools/libxl/libxl_dom.c b/tools/libxl/libxl_dom.c
> index 2da3ac4..be6598f 100644
> --- a/tools/libxl/libxl_dom.c
> +++ b/tools/libxl/libxl_dom.c
> @@ -703,12 +703,12 @@ int libxl__build_pv(libxl__gc *gc, uint32_t domid,
>           LOGE(ERROR, "xc_dom_boot_xen_init failed");
>           goto out;
>       }
> -#ifdef GUEST_RAM_BASE
> -    if ( (ret = xc_dom_rambase_init(dom, GUEST_RAM_BASE)) != 0 ) {
> +

Spurious change.

> +    if ( (ret = xc_dom_rambase_init(dom, info->rambase)) != 0 ) {
>           LOGE(ERROR, "xc_dom_rambase failed");
>           goto out;
>       }
> -#endif
> +
>       if ( (ret = xc_dom_parse_image(dom)) != 0 ) {
>           LOGE(ERROR, "xc_dom_parse_image failed");
>           goto out;
> diff --git a/tools/libxl/libxl_internal.h b/tools/libxl/libxl_internal.h
> index 1699f32..5b0b50a 100644
> --- a/tools/libxl/libxl_internal.h
> +++ b/tools/libxl/libxl_internal.h
> @@ -100,6 +100,7 @@
>   #define LIBXL_HVM_EXTRA_MEMORY 2048
>   #define LIBXL_MIN_DOM0_MEM (128*1024)
>   #define LIBXL_INVALID_GFN (~(uint64_t)0)
> +#define LIBXL_INVALID_RAM_BASE (~(uint64_t)0)
>   /* use 0 as the domid of the toolstack domain for now */
>   #define LIBXL_TOOLSTACK_DOMID 0
>   #define QEMU_SIGNATURE "DeviceModelRecord0002"
> diff --git a/tools/libxl/libxl_types.idl b/tools/libxl/libxl_types.idl
> index 502a148..b6cc8d2 100644
> --- a/tools/libxl/libxl_types.idl
> +++ b/tools/libxl/libxl_types.idl
> @@ -416,6 +416,7 @@ libxl_domain_build_info = Struct("domain_build_info",[
>       ("target_memkb",    MemKB),
>       ("video_memkb",     MemKB),
>       ("shadow_memkb",    MemKB),
> +    ("rambase",         uint64, {'init_val': 'LIBXL_INVALID_RAM_BASE'}),

Any change in libxl_types.idl should also come with a new define in 
libxl.h to allow toolstack built on top of libxl works with various 
version of Xen.

>       ("rtc_timeoffset",  uint32),
>       ("exec_ssidref",    uint32),
>       ("exec_ssid_label", string),
> diff --git a/tools/libxl/xl_cmdimpl.c b/tools/libxl/xl_cmdimpl.c
> index 9a2870e..28d57c3 100644
> --- a/tools/libxl/xl_cmdimpl.c
> +++ b/tools/libxl/xl_cmdimpl.c
> @@ -31,6 +31,7 @@
>   #include <sys/select.h>
>   #include <sys/utsname.h> /* for utsname in xl info */
>   #include <xentoollog.h>
> +#include <xenctrl.h>
>   #include <ctype.h>
>   #include <inttypes.h>
>   #include <limits.h>
> @@ -1352,6 +1353,18 @@ static void parse_config_data(const char *config_source,
>       if (!xlu_cfg_get_long (config, "maxmem", &l, 0))
>           b_info->max_memkb = l * 1024;
>
> +#ifdef GUEST_RAM_BASE
> +    if (!xlu_cfg_get_long (config, "rambase_pfn", &l, 0)) {

New options need to be documented in docs/man/xl.cfg.pod.5.

> +        /* TODO add more detailed check for valid value */
> +        uint64_t rambase = (uint64_t)l << XC_PAGE_SHIFT;
> +        if (rambase > (GUEST_RAM0_BASE + GUEST_RAM0_SIZE)) {
> +            fprintf(stderr, "ERROR: invalid value 0x%lx for \"rambase_pfn\"\n", l);
> +            exit (1);
> +        }
> +        b_info->rambase = rambase;
> +    }
> +#endif
> +
>       if (!xlu_cfg_get_long (config, "vcpus", &l, 0)) {
>           vcpus = l;
>           if (libxl_cpu_bitmap_alloc(ctx, &b_info->avail_vcpus, l)) {
>

Regards,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[PATCH RFC 05/18] xen/arm: allow reassigning of hw interrupts to guest domain

[Andrii Anisov]

From: Andrii Tseglytskyi <andrii.tseglytskyi@globallogic.com>

Patch allows reassigning of hardware interrupts from dom0 to
other guest domain.

Signed-off-by: Andrii Tseglytskyi <andrii.tseglytskyi@globallogic.com>
Signed-off-by: Iurii Konovalenko <iurii.konovalenko@globallogic.com>
---
 xen/arch/arm/irq.c | 19 +++++++++++++++----
 1 file changed, 15 insertions(+), 4 deletions(-)

diff --git a/xen/arch/arm/irq.c b/xen/arch/arm/irq.c
index 1f38605..c470613 100644
--- a/xen/arch/arm/irq.c
+++ b/xen/arch/arm/irq.c
@@ -481,12 +481,23 @@ int route_irq_to_guest(struct domain *d, unsigned int virq,
         }
 
         if ( test_bit(_IRQ_GUEST, &desc->status) )
-            printk(XENLOG_G_ERR "IRQ %u is already used by domain %u\n",
-                   irq, ad->domain_id);
+        {
+            printk(XENLOG_G_DEBUG "IRQ %u is reassigned from domain %u to domain %u\n",
+                    irq, ad->domain_id, d->domain_id);
+
+            retval = gic_remove_irq_from_guest(ad, irq, desc);
+            if ( retval )
+                printk(XENLOG_G_ERR "failed to remove IRQ %u from domain %u (%d)\n",
+                        irq, ad->domain_id, retval);
+            xfree(desc->action);
+            desc->action = NULL;
+        }
         else
+        {
             printk(XENLOG_G_ERR "IRQ %u is already used by Xen\n", irq);
-        retval = -EBUSY;
-        goto out;
+			retval = -EBUSY;
+			goto out;
+        }
     }
 
     retval = __setup_irq(desc, 0, action);
-- 
2.8.2


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH RFC 05/18] xen/arm: allow reassigning of hw interrupts to guest domain

[Julien Grall]

Hello,

On 18/05/16 17:32, Andrii Anisov wrote:
> From: Andrii Tseglytskyi <andrii.tseglytskyi@globallogic.com>
>
> Patch allows reassigning of hardware interrupts from dom0 to
> other guest domain.

Can you explain why route_irq_to_guest should  be able to cope with 
reassigning an IRQ rather than having dom0 calling 
XEN_DOMCTL_unbind_pt_irq beforehand?

Also, this patch does more than allowing an IRQ to be reassigned from 
DOM0 to DOMU, it also allows DOMU to DOMU.

Regards,

> Signed-off-by: Andrii Tseglytskyi <andrii.tseglytskyi@globallogic.com>
> Signed-off-by: Iurii Konovalenko <iurii.konovalenko@globallogic.com>
> ---
>   xen/arch/arm/irq.c | 19 +++++++++++++++----
>   1 file changed, 15 insertions(+), 4 deletions(-)
>
> diff --git a/xen/arch/arm/irq.c b/xen/arch/arm/irq.c
> index 1f38605..c470613 100644
> --- a/xen/arch/arm/irq.c
> +++ b/xen/arch/arm/irq.c
> @@ -481,12 +481,23 @@ int route_irq_to_guest(struct domain *d, unsigned int virq,
>           }
>
>           if ( test_bit(_IRQ_GUEST, &desc->status) )
> -            printk(XENLOG_G_ERR "IRQ %u is already used by domain %u\n",
> -                   irq, ad->domain_id);
> +        {
> +            printk(XENLOG_G_DEBUG "IRQ %u is reassigned from domain %u to domain %u\n",
> +                    irq, ad->domain_id, d->domain_id);
> +
> +            retval = gic_remove_irq_from_guest(ad, irq, desc);
> +            if ( retval )
> +                printk(XENLOG_G_ERR "failed to remove IRQ %u from domain %u (%d)\n",
> +                        irq, ad->domain_id, retval);
> +            xfree(desc->action);
> +            desc->action = NULL;
> +        }
>           else
> +        {
>               printk(XENLOG_G_ERR "IRQ %u is already used by Xen\n", irq);
> -        retval = -EBUSY;
> -        goto out;
> +			retval = -EBUSY;
> +			goto out;
> +        }
>       }
>
>       retval = __setup_irq(desc, 0, action);
>

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[PATCH RFC 06/18] libxl: parse config data during domain reboot

[Andrii Anisov]

From: Viktor Kleinik <viktor.kleinik@globallogic.com>

We need to parse config data during domain reboot
to get correct I/O memory regions for mapping.

Signed-off-by: Viktor Kleinik <viktor.kleinik@globallogic.com>
---
 tools/libxl/xl_cmdimpl.c | 15 +++++++++++++++
 1 file changed, 15 insertions(+)

diff --git a/tools/libxl/xl_cmdimpl.c b/tools/libxl/xl_cmdimpl.c
index 28d57c3..98a46bc 100644
--- a/tools/libxl/xl_cmdimpl.c
+++ b/tools/libxl/xl_cmdimpl.c
@@ -2988,6 +2988,21 @@ start:
                     d_config.c_info.name = strdup(common_domname);
                 }
 
+		if (config_file) {
+		    libxl_domain_config_init(&d_config);
+
+		    ret = libxl_read_file_contents(ctx, config_file,
+						   &config_data, &config_len);
+		    if (ret) {
+			LOG("Failed to read config file: %s: %s\n",
+			    config_file, strerror(errno));
+			goto out;
+		    }
+
+		    parse_config_data(config_file, config_data, config_len,
+				      &d_config);
+		}
+
                 /*
                  * XXX FIXME: If this sleep is not there then domain
                  * re-creation fails sometimes.
-- 
2.8.2


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH RFC 06/18] libxl: parse config data during domain reboot

[Wei Liu]

On Wed, May 18, 2016 at 07:32:29PM +0300, Andrii Anisov wrote:
> From: Viktor Kleinik <viktor.kleinik@globallogic.com>
> 
> We need to parse config data during domain reboot
> to get correct I/O memory regions for mapping.
> 
> Signed-off-by: Viktor Kleinik <viktor.kleinik@globallogic.com>
> ---
>  tools/libxl/xl_cmdimpl.c | 15 +++++++++++++++
>  1 file changed, 15 insertions(+)
> 
> diff --git a/tools/libxl/xl_cmdimpl.c b/tools/libxl/xl_cmdimpl.c
> index 28d57c3..98a46bc 100644
> --- a/tools/libxl/xl_cmdimpl.c
> +++ b/tools/libxl/xl_cmdimpl.c
> @@ -2988,6 +2988,21 @@ start:
>                      d_config.c_info.name = strdup(common_domname);
>                  }
>  
> +		if (config_file) {
> +		    libxl_domain_config_init(&d_config);
> +
> +		    ret = libxl_read_file_contents(ctx, config_file,
> +						   &config_data, &config_len);
> +		    if (ret) {
> +			LOG("Failed to read config file: %s: %s\n",
> +			    config_file, strerror(errno));
> +			goto out;
> +		    }
> +
> +		    parse_config_data(config_file, config_data, config_len,
> +				      &d_config);
> +		}
> +

I think libxl has already preserve the configuration for you?

This will break device hotplug I think.

Wei.

>                  /*
>                   * XXX FIXME: If this sleep is not there then domain
>                   * re-creation fails sometimes.
> -- 
> 2.8.2
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[PATCH RFC 07/18] tools/misc: Modify Xen watchdog daemon

[Andrii Anisov]

From: Viktor Kleinik <viktor.kleinik@globallogic.com>

This change allows watchdog daemon to work thru watchdog device
on the file system.

Signed-off-by: Viktor Kleinik <viktor.kleinik@globallogic.com>
---
 tools/misc/xenwatchdogd.c | 52 +++++++++++------------------------------------
 1 file changed, 12 insertions(+), 40 deletions(-)

diff --git a/tools/misc/xenwatchdogd.c b/tools/misc/xenwatchdogd.c
index 254117b..4b27628 100644
--- a/tools/misc/xenwatchdogd.c
+++ b/tools/misc/xenwatchdogd.c
@@ -1,17 +1,17 @@
-
 #include <err.h>
 #include <limits.h>
-#include "xenctrl.h"
 #include <sys/types.h>
 #include <sys/stat.h>
 #include <fcntl.h>
 #include <stdlib.h>
 #include <unistd.h>
-#include <signal.h>
 #include <stdio.h>
+#include <sys/ioctl.h>
+#include <linux/watchdog.h>
+
+#define DEV_NAME "/dev/watchdog"
 
-xc_interface *h;
-int id = 0;
+int fd = -1;
 
 void daemonize(void)
 {
@@ -36,20 +36,6 @@ void daemonize(void)
         err(1, "reopen stderr");
 }
 
-void catch_exit(int sig)
-{
-    if (id)
-        xc_watchdog(h, id, 300);
-    exit(0);
-}
-
-void catch_usr1(int sig)
-{
-    if (id)
-        xc_watchdog(h, id, 0);
-    exit(0);
-}
-
 int main(int argc, char **argv)
 {
     int t, s;
@@ -60,9 +46,9 @@ int main(int argc, char **argv)
 
     daemonize();
 
-    h = xc_interface_open(NULL, NULL, 0);
-    if (h == NULL)
-	err(1, "xc_interface_open");
+    fd = open(DEV_NAME, O_RDWR);
+    if (fd < 0)
+        err(1, "xenwatchdogd: Failed to open %s\n", DEV_NAME);
 
     t = strtoul(argv[1], NULL, 0);
     if (t == ULONG_MAX)
@@ -75,25 +61,11 @@ int main(int argc, char **argv)
 	    err(1, "strtoul");
     }
 
-    if (signal(SIGHUP, &catch_exit) == SIG_ERR)
-	err(1, "signal");
-    if (signal(SIGINT, &catch_exit) == SIG_ERR)
-	err(1, "signal");
-    if (signal(SIGQUIT, &catch_exit) == SIG_ERR)
-	err(1, "signal");
-    if (signal(SIGTERM, &catch_exit) == SIG_ERR)
-	err(1, "signal");
-    if (signal(SIGUSR1, &catch_usr1) == SIG_ERR)
-	err(1, "signal");
-
-    id = xc_watchdog(h, 0, t);
-    if (id <= 0)
-        err(1, "xc_watchdog setup");
-
     for (;;) {
+	ret = ioctl(fd, WDIOC_KEEPALIVE);
+	if (ret)
+	    err(1, "xenwatchdogd: Failed to kick watchdog\n");
+
         sleep(s);
-        ret = xc_watchdog(h, id, t);
-        if (ret != 0)
-            err(1, "xc_watchdog");
     }
 }
-- 
2.8.2


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH RFC 07/18] tools/misc: Modify Xen watchdog daemon

[Wei Liu]

On Wed, May 18, 2016 at 07:32:30PM +0300, Andrii Anisov wrote:
> From: Viktor Kleinik <viktor.kleinik@globallogic.com>
> 
> This change allows watchdog daemon to work thru watchdog device
> on the file system.
> 
> Signed-off-by: Viktor Kleinik <viktor.kleinik@globallogic.com>

The commit message is not clear enough as to why you want to delete a
bunch of code. I'm afraid I'm not able to review a patch like this.

Doesn't this patch break existing behaviour?

Wei.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[PATCH RFC 08/18] tools/misc: Set timeout value from watchdog daemon

[Andrii Anisov]

From: Viktor Kleinik <viktor.kleinik@globallogic.com>

Signed-off-by: Viktor Kleinik <viktor.kleinik@globallogic.com>
---
 tools/misc/xenwatchdogd.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/tools/misc/xenwatchdogd.c b/tools/misc/xenwatchdogd.c
index 4b27628..fcf7d16 100644
--- a/tools/misc/xenwatchdogd.c
+++ b/tools/misc/xenwatchdogd.c
@@ -61,6 +61,10 @@ int main(int argc, char **argv)
 	    err(1, "strtoul");
     }
 
+    ret = ioctl(fd, WDIOC_SETTIMEOUT, &t);
+    if (ret < 0)
+	err(1, "xenwatchdogd: Failed to set timeout\n");
+
     for (;;) {
 	ret = ioctl(fd, WDIOC_KEEPALIVE);
 	if (ret)
-- 
2.8.2


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH RFC 08/18] tools/misc: Set timeout value from watchdog daemon

[Wei Liu]

On Wed, May 18, 2016 at 07:32:31PM +0300, Andrii Anisov wrote:
> From: Viktor Kleinik <viktor.kleinik@globallogic.com>
> 
> Signed-off-by: Viktor Kleinik <viktor.kleinik@globallogic.com>

I have a feeling this can probably be squashed into previous patch. But
please don't squash this yet, let's start with better commit message.

Wei.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[PATCH RFC 09/18] tools: Allow to cross-compile xentop

[Andrii Anisov]

From: Oleksandr Dmytryshyn <oleksandr.dmytryshyn@globallogic.com>

Signed-off-by: Oleksandr Dmytryshyn <oleksandr.dmytryshyn@globallogic.com>
Signed-off-by: Iurii Konovalenko <iurii.konovalenko@globallogic.com>
---
 tools/xenstat/Makefile | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)

diff --git a/tools/xenstat/Makefile b/tools/xenstat/Makefile
index 901be4a..440b1b7 100644
--- a/tools/xenstat/Makefile
+++ b/tools/xenstat/Makefile
@@ -4,12 +4,10 @@ include $(XEN_ROOT)/tools/Rules.mk
 SUBDIRS :=
 SUBDIRS += libxenstat
 
-# This doesn't cross-compile (cross-compile environments rarely have curses)
-ifeq ($(XEN_COMPILE_ARCH),$(XEN_TARGET_ARCH))
+# This compiles if compiler environment has curses
 ifeq ($(wildcard /usr/include/curses.h),/usr/include/curses.h)
 SUBDIRS += xentop
 endif
-endif
 
 .PHONY: all install clean distclean
 
-- 
2.8.2


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH RFC 09/18] tools: Allow to cross-compile xentop

[Wei Liu]

On Wed, May 18, 2016 at 07:32:32PM +0300, Andrii Anisov wrote:
> From: Oleksandr Dmytryshyn <oleksandr.dmytryshyn@globallogic.com>
> 
> Signed-off-by: Oleksandr Dmytryshyn <oleksandr.dmytryshyn@globallogic.com>
> Signed-off-by: Iurii Konovalenko <iurii.konovalenko@globallogic.com>
> ---
>  tools/xenstat/Makefile | 4 +---
>  1 file changed, 1 insertion(+), 3 deletions(-)
> 
> diff --git a/tools/xenstat/Makefile b/tools/xenstat/Makefile
> index 901be4a..440b1b7 100644
> --- a/tools/xenstat/Makefile
> +++ b/tools/xenstat/Makefile
> @@ -4,12 +4,10 @@ include $(XEN_ROOT)/tools/Rules.mk
>  SUBDIRS :=
>  SUBDIRS += libxenstat
>  
> -# This doesn't cross-compile (cross-compile environments rarely have curses)
> -ifeq ($(XEN_COMPILE_ARCH),$(XEN_TARGET_ARCH))
> +# This compiles if compiler environment has curses

You should have a option that is set by configure to guard this.

Wei.

>  ifeq ($(wildcard /usr/include/curses.h),/usr/include/curses.h)
>  SUBDIRS += xentop
>  endif
> -endif
>  
>  .PHONY: all install clean distclean
>  
> -- 
> 2.8.2
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[PATCH RFC 10/18] xen: arm: add batch support to the XENMEM_p2m_lookup operation

[Andrii Anisov]

From: Oleksandr Dmytryshyn <oleksandr.dmytryshyn@globallogic.com>

Signed-off-by: Oleksandr Dmytryshyn <oleksandr.dmytryshyn@globallogic.com>
Signed-off-by: Iurii Konovalenko <iurii.konovalenko@globallogic.com>
---
 xen/arch/arm/mm.c           | 33 +++++++++++++++++++++++++++++++++
 xen/include/public/memory.h | 12 +++++++++++-
 2 files changed, 44 insertions(+), 1 deletion(-)

diff --git a/xen/arch/arm/mm.c b/xen/arch/arm/mm.c
index b5d8c85..04fb813 100644
--- a/xen/arch/arm/mm.c
+++ b/xen/arch/arm/mm.c
@@ -1150,10 +1150,43 @@ int xenmem_add_to_physmap_one(
     return rc;
 }
 
+#define MAX_P2M_ENTRIES_CNT             10000
+
+static long arch_paddr_to_maddr_batch(XEN_GUEST_HANDLE_PARAM(void) arg)
+{
+    struct xen_p2m_lookup p2mr;
+    xen_pfn_t paddr, maddr;
+    unsigned int i;
+
+    if ( copy_from_guest(&p2mr, arg, 1) )
+        return -EFAULT;
+
+    if (p2mr.count < 1 || p2mr.count > MAX_P2M_ENTRIES_CNT)
+        return -EINVAL;
+
+    if ( guest_handle_is_null(p2mr.paddrs) ||
+         guest_handle_is_null(p2mr.maddrs))
+        return -EINVAL;
+
+    for ( i = 0; i < p2mr.count; i++ )
+    {
+        if ( unlikely(__copy_from_guest_offset(&paddr, p2mr.paddrs, i, 1)) )
+            return -EFAULT;
+
+        maddr = p2m_lookup(current->domain, paddr, NULL);
+
+        if ( unlikely(__copy_to_guest_offset(p2mr.maddrs, i, &maddr, 1)) )
+            return -EFAULT;
+    }
+    return 0;
+}
+
 long arch_memory_op(int op, XEN_GUEST_HANDLE_PARAM(void) arg)
 {
     switch ( op )
     {
+    case XENMEM_p2m_lookup:
+        return arch_paddr_to_maddr_batch(arg);
     /* XXX: memsharing not working yet */
     case XENMEM_get_sharing_shared_pages:
     case XENMEM_get_sharing_freed_pages:
diff --git a/xen/include/public/memory.h b/xen/include/public/memory.h
index 320de91..dfc5171 100644
--- a/xen/include/public/memory.h
+++ b/xen/include/public/memory.h
@@ -608,7 +608,17 @@ struct xen_vnuma_topology_info {
 typedef struct xen_vnuma_topology_info xen_vnuma_topology_info_t;
 DEFINE_XEN_GUEST_HANDLE(xen_vnuma_topology_info_t);
 
-/* Next available subop number is 28 */
+struct xen_p2m_lookup {
+    uint32_t count;
+    XEN_GUEST_HANDLE(xen_pfn_t) paddrs; /* IN:  physical addresses */
+    XEN_GUEST_HANDLE(xen_pfn_t) maddrs; /* OUT: machine addresses */
+};
+typedef struct xen_p2m_lookup xen_p2m_lookup_t;
+DEFINE_XEN_GUEST_HANDLE(xen_p2m_lookup_t);
+
+#define XENMEM_p2m_lookup 28
+
+/* Next available subop number is 29 */
 
 #endif /* __XEN_PUBLIC_MEMORY_H__ */
 
-- 
2.8.2


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH RFC 10/18] xen: arm: add batch support to the XENMEM_p2m_lookup operation

[Jan Beulich]

>>> On 18.05.16 at 18:32, <andrii.anisov@globallogic.com> wrote:

First of all, the description is misleading: You don't add anything
_to_ XENMEM_p2m_lookup, you simply add this new sub-op.

And then, we've had requests to add something like this more
than once, and so far they've always got rejected. See the
removed XENMEM_translate_gpfn_list. Hence an empty
description here is definitely insufficient, as you'll need to
explain why _now_ all of the sudden this is needed. Perhaps
it would have been a good idea if you had asked up front
whether something like this can be re-added, or what
alternatives there are without doing so.

I'll therefore not comment on the actual patch, which also has
issues (but which aren't worth addressing if the whole thing is
going to get dropped).

Jan


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[PATCH RFC 11/18] arm: Fix 1-to-1 Dom0 memory allocation of any size

[Andrii Anisov]

From: Iurii Konovalenko <iurii.konovalenko@globallogic.com>

For Dom0 with non-power-two memory size less then 128M allocation
of first memory bank fails. This patch fix it.

Signed-off-by: Iurii Konovalenko <iurii.konovalenko@globallogic.com>
---
 xen/arch/arm/domain_build.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index a059de6..2937ff7 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -244,7 +244,7 @@ fail:
 static void allocate_memory_11(struct domain *d, struct kernel_info *kinfo)
 {
     const unsigned int min_low_order =
-        get_order_from_bytes(min_t(paddr_t, dom0_mem, MB(128)));
+        get_11_allocation_size(min_t(paddr_t, dom0_mem, MB(128)));
     const unsigned int min_order = get_order_from_bytes(MB(4));
     struct page_info *pg;
     unsigned int order = get_11_allocation_size(kinfo->unassigned_mem);
-- 
2.8.2


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[PATCH RFC 12/18] libxl: Fix unneeded domain reboot during destroy routine

[Andrii Anisov]

From: Viktor Kleinik <viktor.kleinik@globallogic.com>

During domain destroy we should change its state from "alive"
to "dying" to prevent reboot because of watchdog timeout.

Signed-off-by: Viktor Kleinik <viktor.kleinik@globallogic.com>
---
 tools/libxl/libxl.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/tools/libxl/libxl.c b/tools/libxl/libxl.c
index 1366177..ac50bda 100644
--- a/tools/libxl/libxl.c
+++ b/tools/libxl/libxl.c
@@ -1634,6 +1634,10 @@ void libxl__destroy_domid(libxl__egc *egc, libxl__destroy_domid_state *dis)
     dis->drs.callback = devices_destroy_cb;
     dis->drs.force = 1;
     libxl__devices_destroy(egc, &dis->drs);
+    rc = xc_domain_destroy(ctx->xch, domid);
+    if (rc < 0) {
+        LIBXL__LOG_ERRNOVAL(ctx, LIBXL__LOG_ERROR, rc, "xc_domain_destroy failed for %d", domid);
+    }
     return;
 
 out:
-- 
2.8.2


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH RFC 12/18] libxl: Fix unneeded domain reboot during destroy routine

[Wei Liu]

On Wed, May 18, 2016 at 07:32:35PM +0300, Andrii Anisov wrote:
> From: Viktor Kleinik <viktor.kleinik@globallogic.com>
> 
> During domain destroy we should change its state from "alive"
> to "dying" to prevent reboot because of watchdog timeout.
> 
> Signed-off-by: Viktor Kleinik <viktor.kleinik@globallogic.com>
> ---
>  tools/libxl/libxl.c | 4 ++++
>  1 file changed, 4 insertions(+)
> 
> diff --git a/tools/libxl/libxl.c b/tools/libxl/libxl.c
> index 1366177..ac50bda 100644
> --- a/tools/libxl/libxl.c
> +++ b/tools/libxl/libxl.c
> @@ -1634,6 +1634,10 @@ void libxl__destroy_domid(libxl__egc *egc, libxl__destroy_domid_state *dis)
>      dis->drs.callback = devices_destroy_cb;
>      dis->drs.force = 1;
>      libxl__devices_destroy(egc, &dis->drs);
> +    rc = xc_domain_destroy(ctx->xch, domid);
> +    if (rc < 0) {
> +        LIBXL__LOG_ERRNOVAL(ctx, LIBXL__LOG_ERROR, rc, "xc_domain_destroy failed for %d", domid);
> +    }

What is the code path here? Have you checked the other place which calls
xc_domain_destroy? Now there are two calls to xc_domain_destroy, which
doesn't seem right to me.

Wei.

>      return;
>  
>  out:
> -- 
> 2.8.2
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[PATCH RFC 13/18] xen: introduce and use 'dom0_rambase_pfn' setting for kernel Dom0

[Andrii Anisov]

From: Oleksandr Dmytryshyn <oleksandr.dmytryshyn@globallogic.com>

This setting is used to adjust starting memory address allocated
for kernel Dom0. To use 'rambase_pfn' setting just add for example
'dom0_rambase_pfn=0x80000' to the hypervisor command line. Note that
'dom0_rambase_pfn' should be aligned with the smallest memory chunk
which use xen memory allocator.

Signed-off-by: Oleksandr Dmytryshyn <oleksandr.dmytryshyn@globallogic.com>
---
 xen/arch/arm/domain_build.c | 24 +++++++++++++---
 xen/common/page_alloc.c     | 68 +++++++++++++++++++++++++++++++++++----------
 xen/include/xen/mm.h        |  2 ++
 3 files changed, 75 insertions(+), 19 deletions(-)

diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index 2937ff7..b48718d 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -27,6 +27,9 @@
 static unsigned int __initdata opt_dom0_max_vcpus;
 integer_param("dom0_max_vcpus", opt_dom0_max_vcpus);
 
+static u64 __initdata opt_dom0_rambase_pfn = 0;
+integer_param("dom0_rambase_pfn", opt_dom0_rambase_pfn);
+
 int dom0_11_mapping = 1;
 
 #define DOM0_MEM_DEFAULT 0x8000000 /* 128 MiB */
@@ -248,6 +251,8 @@ static void allocate_memory_11(struct domain *d, struct kernel_info *kinfo)
     const unsigned int min_order = get_order_from_bytes(MB(4));
     struct page_info *pg;
     unsigned int order = get_11_allocation_size(kinfo->unassigned_mem);
+    u64 rambase_pfn = opt_dom0_rambase_pfn;
+    paddr_t mem_size = kinfo->unassigned_mem;
     int i;
 
     bool_t lowmem = is_32bit_domain(d);
@@ -267,7 +272,7 @@ static void allocate_memory_11(struct domain *d, struct kernel_info *kinfo)
     {
         for ( bits = order ; bits <= (lowmem ? 32 : PADDR_BITS); bits++ )
         {
-            pg = alloc_domheap_pages(d, order, MEMF_bits(bits));
+            pg = alloc_domheap_pages_pfn(d, order, MEMF_bits(bits), rambase_pfn);
             if ( pg != NULL )
                 goto got_bank0;
         }
@@ -284,16 +289,21 @@ static void allocate_memory_11(struct domain *d, struct kernel_info *kinfo)
     /* Now allocate more memory and fill in additional banks */
 
     order = get_11_allocation_size(kinfo->unassigned_mem);
+    if ( opt_dom0_rambase_pfn )
+        rambase_pfn += (mem_size - kinfo->unassigned_mem) >> PAGE_SHIFT;
+
     while ( kinfo->unassigned_mem && kinfo->mem.nr_banks < NR_MEM_BANKS )
     {
-        pg = alloc_domheap_pages(d, order, lowmem ? MEMF_bits(32) : 0);
+        pg = alloc_domheap_pages_pfn(d, order, lowmem ? MEMF_bits(32) : 0,
+                                     rambase_pfn);
         if ( !pg )
         {
             order --;
 
             if ( lowmem && order < min_low_order)
             {
-                D11PRINT("Failed at min_low_order, allow high allocations\n");
+                if ( !opt_dom0_rambase_pfn )
+                    D11PRINT("Failed at min_low_order, allow high allocations\n");
                 order = get_11_allocation_size(kinfo->unassigned_mem);
                 lowmem = false;
                 continue;
@@ -313,7 +323,8 @@ static void allocate_memory_11(struct domain *d, struct kernel_info *kinfo)
 
             if ( lowmem )
             {
-                D11PRINT("Allocation below bank 0, allow high allocations\n");
+                if ( !opt_dom0_rambase_pfn )
+                    D11PRINT("Allocation below bank 0, allow high allocations\n");
                 order = get_11_allocation_size(kinfo->unassigned_mem);
                 lowmem = false;
                 continue;
@@ -330,6 +341,11 @@ static void allocate_memory_11(struct domain *d, struct kernel_info *kinfo)
          * allocation possible.
          */
         order = get_11_allocation_size(kinfo->unassigned_mem);
+        if ( opt_dom0_rambase_pfn )
+        {
+            rambase_pfn += (mem_size - kinfo->unassigned_mem) >> PAGE_SHIFT;
+            mem_size = kinfo->unassigned_mem;
+        }
     }
 
     if ( kinfo->unassigned_mem )
diff --git a/xen/common/page_alloc.c b/xen/common/page_alloc.c
index 74fc1de..d0c0fbb 100644
--- a/xen/common/page_alloc.c
+++ b/xen/common/page_alloc.c
@@ -583,16 +583,17 @@ static void check_low_mem_virq(void)
     }
 }
 
-/* Allocate 2^@order contiguous pages. */
-static struct page_info *alloc_heap_pages(
+/* Allocate 2^@order contiguous pages at given pfn. */
+static struct page_info *alloc_heap_pages_pfn(
     unsigned int zone_lo, unsigned int zone_hi,
     unsigned int order, unsigned int memflags,
-    struct domain *d)
+    struct domain *d, xen_pfn_t pfn)
 {
     unsigned int i, j, zone = 0, nodemask_retry = 0;
     nodeid_t first_node, node = MEMF_get_node(memflags), req_node = node;
     unsigned long request = 1UL << order;
-    struct page_info *pg;
+    struct page_info *pg, *tmp_pg;
+    struct page_list_head *pg_list;
     nodemask_t nodemask = (d != NULL ) ? d->node_affinity : node_online_map;
     bool_t need_tlbflush = 0;
     uint32_t tlbflush_timestamp = 0;
@@ -657,9 +658,25 @@ static struct page_info *alloc_heap_pages(
                 continue;
 
             /* Find smallest order which can satisfy the request. */
-            for ( j = order; j <= MAX_ORDER; j++ )
-                if ( (pg = page_list_remove_head(&heap(node, zone, j))) )
-                    goto found;
+            for ( j = order; j <= MAX_ORDER; j++ ) {
+                pg_list = &heap(node, zone, j);
+                if ( pfn )
+                {
+                    page_list_for_each_safe( pg, tmp_pg, pg_list )
+                    {
+                        if ( pfn == page_to_mfn(pg) )
+                        {
+                            page_list_del(pg, pg_list);
+                            goto found;
+                        }
+                    }
+                }
+                else
+                {
+                    if ( (pg = page_list_remove_head(pg_list)) )
+                        goto found;
+                }
+            }
         } while ( zone-- > zone_lo ); /* careful: unsigned zone may wrap */
 
         if ( (memflags & MEMF_exact_node) && req_node != NUMA_NO_NODE )
@@ -706,9 +723,15 @@ static struct page_info *alloc_heap_pages(
     /* We may have to halve the chunk a number of times. */
     while ( j != order )
     {
-        PFN_ORDER(pg) = --j;
-        page_list_add_tail(pg, &heap(node, zone, j));
-        pg += 1 << j;
+        tmp_pg = pg;
+        if ( pfn )
+            tmp_pg += 1 << (j - 1);
+
+        PFN_ORDER(tmp_pg) = --j;
+        page_list_add_tail(tmp_pg, &heap(node, zone, j));
+
+        if ( !pfn )
+            pg += 1 << j;
     }
 
     ASSERT(avail[node][zone] >= request);
@@ -762,6 +785,15 @@ static struct page_info *alloc_heap_pages(
     return pg;
 }
 
+/* Allocate 2^@order contiguous pages. */
+static struct page_info *alloc_heap_pages(
+    unsigned int zone_lo, unsigned int zone_hi,
+    unsigned int order, unsigned int memflags,
+    struct domain *d)
+{
+    return alloc_heap_pages_pfn(zone_lo, zone_hi, order, memflags, d, 0);
+}
+
 /* Remove any offlined page in the buddy pointed to by head. */
 static int reserve_offlined_page(struct page_info *head)
 {
@@ -1687,8 +1719,8 @@ int assign_pages(
 }
 
 
-struct page_info *alloc_domheap_pages(
-    struct domain *d, unsigned int order, unsigned int memflags)
+struct page_info *alloc_domheap_pages_pfn(
+    struct domain *d, unsigned int order, unsigned int memflags, xen_pfn_t pfn)
 {
     struct page_info *pg = NULL;
     unsigned int bits = memflags >> _MEMF_bits, zone_hi = NR_ZONES - 1;
@@ -1705,12 +1737,12 @@ struct page_info *alloc_domheap_pages(
         memflags |= MEMF_no_refcount;
 
     if ( dma_bitsize && ((dma_zone = bits_to_zone(dma_bitsize)) < zone_hi) )
-        pg = alloc_heap_pages(dma_zone + 1, zone_hi, order, memflags, d);
+        pg = alloc_heap_pages_pfn(dma_zone + 1, zone_hi, order, memflags, d, pfn);
 
     if ( (pg == NULL) &&
          ((memflags & MEMF_no_dma) ||
-          ((pg = alloc_heap_pages(MEMZONE_XEN + 1, zone_hi, order,
-                                  memflags, d)) == NULL)) )
+          ((pg = alloc_heap_pages_pfn(MEMZONE_XEN + 1, zone_hi, order,
+                                  memflags, d, pfn)) == NULL)) )
          return NULL;
 
     if ( d && !(memflags & MEMF_no_owner) &&
@@ -1723,6 +1755,12 @@ struct page_info *alloc_domheap_pages(
     return pg;
 }
 
+struct page_info *alloc_domheap_pages(
+    struct domain *d, unsigned int order, unsigned int memflags)
+{
+    return alloc_domheap_pages_pfn(d, order, memflags, 0);
+}
+
 void free_domheap_pages(struct page_info *pg, unsigned int order)
 {
     struct domain *d = page_get_owner(pg);
diff --git a/xen/include/xen/mm.h b/xen/include/xen/mm.h
index a795dd6..83e4913 100644
--- a/xen/include/xen/mm.h
+++ b/xen/include/xen/mm.h
@@ -117,6 +117,8 @@ void get_outstanding_claims(uint64_t *free_pages, uint64_t *outstanding_pages);
 
 /* Domain suballocator. These functions are *not* interrupt-safe.*/
 void init_domheap_pages(paddr_t ps, paddr_t pe);
+struct page_info *alloc_domheap_pages_pfn(
+    struct domain *d, unsigned int order, unsigned int memflags, xen_pfn_t pfn);
 struct page_info *alloc_domheap_pages(
     struct domain *d, unsigned int order, unsigned int memflags);
 void free_domheap_pages(struct page_info *pg, unsigned int order);
-- 
2.8.2


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH RFC 13/18] xen: introduce and use 'dom0_rambase_pfn' setting for kernel Dom0

[Jan Beulich]

>>> On 18.05.16 at 18:32, <andrii.anisov@globallogic.com> wrote:
> --- a/xen/arch/arm/domain_build.c
> +++ b/xen/arch/arm/domain_build.c
> @@ -27,6 +27,9 @@
>  static unsigned int __initdata opt_dom0_max_vcpus;
>  integer_param("dom0_max_vcpus", opt_dom0_max_vcpus);
>  
> +static u64 __initdata opt_dom0_rambase_pfn = 0;
> +integer_param("dom0_rambase_pfn", opt_dom0_rambase_pfn);

Any addition of a command line option needs to be accompanied by
an entry in the command line doc.

> @@ -248,6 +251,8 @@ static void allocate_memory_11(struct domain *d, struct 
> kernel_info *kinfo)
>      const unsigned int min_order = get_order_from_bytes(MB(4));
>      struct page_info *pg;
>      unsigned int order = get_11_allocation_size(kinfo->unassigned_mem);
> +    u64 rambase_pfn = opt_dom0_rambase_pfn;

Use of __initdata in a non-__init function.

> --- a/xen/common/page_alloc.c
> +++ b/xen/common/page_alloc.c
> @@ -583,16 +583,17 @@ static void check_low_mem_virq(void)
>      }
>  }
>  
> -/* Allocate 2^@order contiguous pages. */
> -static struct page_info *alloc_heap_pages(
> +/* Allocate 2^@order contiguous pages at given pfn. */
> +static struct page_info *alloc_heap_pages_pfn(
>      unsigned int zone_lo, unsigned int zone_hi,
>      unsigned int order, unsigned int memflags,
> -    struct domain *d)
> +    struct domain *d, xen_pfn_t pfn)

Altering generic allocation interfaces like this, for a boot time only
purpose, doesn't seem warranted. Please reconsider the entire
approach.

Jan


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH RFC 13/18] xen: introduce and use 'dom0_rambase_pfn' setting for kernel Dom0

[Julien Grall]

Hi Jan,

On 19/05/16 10:41, Jan Beulich wrote:
>>>> On 18.05.16 at 18:32, <andrii.anisov@globallogic.com> wrote:
>> --- a/xen/arch/arm/domain_build.c
>> +++ b/xen/arch/arm/domain_build.c
>> @@ -27,6 +27,9 @@
>>   static unsigned int __initdata opt_dom0_max_vcpus;
>>   integer_param("dom0_max_vcpus", opt_dom0_max_vcpus);
>>
>> +static u64 __initdata opt_dom0_rambase_pfn = 0;
>> +integer_param("dom0_rambase_pfn", opt_dom0_rambase_pfn);
>
> Any addition of a command line option needs to be accompanied by
> an entry in the command line doc.
>
>> @@ -248,6 +251,8 @@ static void allocate_memory_11(struct domain *d, struct
>> kernel_info *kinfo)
>>       const unsigned int min_order = get_order_from_bytes(MB(4));
>>       struct page_info *pg;
>>       unsigned int order = get_11_allocation_size(kinfo->unassigned_mem);
>> +    u64 rambase_pfn = opt_dom0_rambase_pfn;
>
> Use of __initdata in a non-__init function.

All the functions within domain_build.c should have the attributes 
__init. However, it has been forgotten for half of them.

I am planning to send a patch to enforce it using the Makefile rules.

Regards,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH RFC 13/18] xen: introduce and use 'dom0_rambase_pfn' setting for kernel Dom0

[Jan Beulich]

>>> On 19.05.16 at 14:26, <julien.grall@arm.com> wrote:
> On 19/05/16 10:41, Jan Beulich wrote:
>>>>> On 18.05.16 at 18:32, <andrii.anisov@globallogic.com> wrote:
>>> --- a/xen/arch/arm/domain_build.c
>>> +++ b/xen/arch/arm/domain_build.c
>>> @@ -27,6 +27,9 @@
>>>   static unsigned int __initdata opt_dom0_max_vcpus;
>>>   integer_param("dom0_max_vcpus", opt_dom0_max_vcpus);
>>>
>>> +static u64 __initdata opt_dom0_rambase_pfn = 0;
>>> +integer_param("dom0_rambase_pfn", opt_dom0_rambase_pfn);
>>
>> Any addition of a command line option needs to be accompanied by
>> an entry in the command line doc.
>>
>>> @@ -248,6 +251,8 @@ static void allocate_memory_11(struct domain *d, struct
>>> kernel_info *kinfo)
>>>       const unsigned int min_order = get_order_from_bytes(MB(4));
>>>       struct page_info *pg;
>>>       unsigned int order = get_11_allocation_size(kinfo->unassigned_mem);
>>> +    u64 rambase_pfn = opt_dom0_rambase_pfn;
>>
>> Use of __initdata in a non-__init function.
> 
> All the functions within domain_build.c should have the attributes 
> __init. However, it has been forgotten for half of them.
> 
> I am planning to send a patch to enforce it using the Makefile rules.

I have such a work item (low) on my todo list too, actually.

Jan


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH RFC 13/18] xen: introduce and use 'dom0_rambase_pfn' setting for kernel Dom0

[Julien Grall]

Hello,

On 18/05/16 17:32, Andrii Anisov wrote:
> From: Oleksandr Dmytryshyn <oleksandr.dmytryshyn@globallogic.com>
>
> This setting is used to adjust starting memory address allocated
> for kernel Dom0. To use 'rambase_pfn' setting just add for example
> 'dom0_rambase_pfn=0x80000' to the hypervisor command line. Note that
> 'dom0_rambase_pfn' should be aligned with the smallest memory chunk
> which use xen memory allocator.

Why would a user want to allocate DOM0 RAM bank to a specific address?

If I understand correctly your patch, DOM0 will only able to allocate 
one bank of the given size at the specific address. You also add this 
possibility for guest domain (see patch #4) and try to control where the 
guest memory will be allocated. This will increase a lot the chance of 
the memory allocation to fail.

For instance, the RAM region requested for DOM0 may have been used to 
allocate memory for Xen internal. So you need a way to reserve memory in 
order to avoid Xen using it.

I expect most of the users who want to use direct memory mapped guest to 
know the number of guests which will use this feature.

A such feature is only useful when pass-through a device to the guest on 
platfom without SMMU, so it is by default insecure.

So I would suggest to create a new device-tree binding (or re-use an 
actual one) to reserve memory region to be used for direct memory mapped 
domain.

Those regions could have an identifier to be used later during the 
allocation. This would avoid memory fragmentation, allow multiple RAM 
bank for DOM0,...

Any opinions?

>
> Signed-off-by: Oleksandr Dmytryshyn <oleksandr.dmytryshyn@globallogic.com>
> ---
>   xen/arch/arm/domain_build.c | 24 +++++++++++++---
>   xen/common/page_alloc.c     | 68 +++++++++++++++++++++++++++++++++++----------
>   xen/include/xen/mm.h        |  2 ++
>   3 files changed, 75 insertions(+), 19 deletions(-)
>
> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> index 2937ff7..b48718d 100644
> --- a/xen/arch/arm/domain_build.c
> +++ b/xen/arch/arm/domain_build.c
> @@ -27,6 +27,9 @@
>   static unsigned int __initdata opt_dom0_max_vcpus;
>   integer_param("dom0_max_vcpus", opt_dom0_max_vcpus);
>
> +static u64 __initdata opt_dom0_rambase_pfn = 0;
> +integer_param("dom0_rambase_pfn", opt_dom0_rambase_pfn);
> +
>   int dom0_11_mapping = 1;
>
>   #define DOM0_MEM_DEFAULT 0x8000000 /* 128 MiB */
> @@ -248,6 +251,8 @@ static void allocate_memory_11(struct domain *d, struct kernel_info *kinfo)
>       const unsigned int min_order = get_order_from_bytes(MB(4));
>       struct page_info *pg;
>       unsigned int order = get_11_allocation_size(kinfo->unassigned_mem);
> +    u64 rambase_pfn = opt_dom0_rambase_pfn;
> +    paddr_t mem_size = kinfo->unassigned_mem;
>       int i;
>
>       bool_t lowmem = is_32bit_domain(d);
> @@ -267,7 +272,7 @@ static void allocate_memory_11(struct domain *d, struct kernel_info *kinfo)
>       {
>           for ( bits = order ; bits <= (lowmem ? 32 : PADDR_BITS); bits++ )
>           {
> -            pg = alloc_domheap_pages(d, order, MEMF_bits(bits));
> +            pg = alloc_domheap_pages_pfn(d, order, MEMF_bits(bits), rambase_pfn);
>               if ( pg != NULL )
>                   goto got_bank0;
>           }
> @@ -284,16 +289,21 @@ static void allocate_memory_11(struct domain *d, struct kernel_info *kinfo)
>       /* Now allocate more memory and fill in additional banks */
>
>       order = get_11_allocation_size(kinfo->unassigned_mem);
> +    if ( opt_dom0_rambase_pfn )
> +        rambase_pfn += (mem_size - kinfo->unassigned_mem) >> PAGE_SHIFT;
> +
>       while ( kinfo->unassigned_mem && kinfo->mem.nr_banks < NR_MEM_BANKS )
>       {
> -        pg = alloc_domheap_pages(d, order, lowmem ? MEMF_bits(32) : 0);
> +        pg = alloc_domheap_pages_pfn(d, order, lowmem ? MEMF_bits(32) : 0,
> +                                     rambase_pfn);

 From my understanding, when rambase_pfn is not 0, the memory must be 
allocated contiguously at this specific address. So if the first call of 
alloc_domheap_pages (see a bit above) as failed, then this one will 
always fail because it means that someone has allocated some page in 
this region.

>           if ( !pg )
>           {
>               order --;
>
>               if ( lowmem && order < min_low_order)
>               {
> -                D11PRINT("Failed at min_low_order, allow high allocations\n");
> +                if ( !opt_dom0_rambase_pfn )
> +                    D11PRINT("Failed at min_low_order, allow high allocations\n");
>                   order = get_11_allocation_size(kinfo->unassigned_mem);
>                   lowmem = false;
>                   continue;
> @@ -313,7 +323,8 @@ static void allocate_memory_11(struct domain *d, struct kernel_info *kinfo)
>
>               if ( lowmem )
>               {
> -                D11PRINT("Allocation below bank 0, allow high allocations\n");
> +                if ( !opt_dom0_rambase_pfn )
> +                    D11PRINT("Allocation below bank 0, allow high allocations\n");
>                   order = get_11_allocation_size(kinfo->unassigned_mem);
>                   lowmem = false;
>                   continue;
> @@ -330,6 +341,11 @@ static void allocate_memory_11(struct domain *d, struct kernel_info *kinfo)
>            * allocation possible.
>            */
>           order = get_11_allocation_size(kinfo->unassigned_mem);
> +        if ( opt_dom0_rambase_pfn )
> +        {
> +            rambase_pfn += (mem_size - kinfo->unassigned_mem) >> PAGE_SHIFT;
> +            mem_size = kinfo->unassigned_mem;
> +        }
>       }
>
>       if ( kinfo->unassigned_mem )
> diff --git a/xen/common/page_alloc.c b/xen/common/page_alloc.c
> index 74fc1de..d0c0fbb 100644
> --- a/xen/common/page_alloc.c
> +++ b/xen/common/page_alloc.c
> @@ -583,16 +583,17 @@ static void check_low_mem_virq(void)
>       }
>   }
>
> -/* Allocate 2^@order contiguous pages. */
> -static struct page_info *alloc_heap_pages(
> +/* Allocate 2^@order contiguous pages at given pfn. */
> +static struct page_info *alloc_heap_pages_pfn(
>       unsigned int zone_lo, unsigned int zone_hi,
>       unsigned int order, unsigned int memflags,
> -    struct domain *d)
> +    struct domain *d, xen_pfn_t pfn)
>   {
>       unsigned int i, j, zone = 0, nodemask_retry = 0;
>       nodeid_t first_node, node = MEMF_get_node(memflags), req_node = node;
>       unsigned long request = 1UL << order;
> -    struct page_info *pg;
> +    struct page_info *pg, *tmp_pg;
> +    struct page_list_head *pg_list;
>       nodemask_t nodemask = (d != NULL ) ? d->node_affinity : node_online_map;
>       bool_t need_tlbflush = 0;
>       uint32_t tlbflush_timestamp = 0;
> @@ -657,9 +658,25 @@ static struct page_info *alloc_heap_pages(
>                   continue;
>
>               /* Find smallest order which can satisfy the request. */
> -            for ( j = order; j <= MAX_ORDER; j++ )
> -                if ( (pg = page_list_remove_head(&heap(node, zone, j))) )
> -                    goto found;
> +            for ( j = order; j <= MAX_ORDER; j++ ) {
> +                pg_list = &heap(node, zone, j);
> +                if ( pfn )
> +                {
> +                    page_list_for_each_safe( pg, tmp_pg, pg_list )
> +                    {
> +                        if ( pfn == page_to_mfn(pg) )
> +                        {
> +                            page_list_del(pg, pg_list);
> +                            goto found;
> +                        }
> +                    }
> +                }
> +                else
> +                {
> +                    if ( (pg = page_list_remove_head(pg_list)) )
> +                        goto found;
> +                }
> +            }
>           } while ( zone-- > zone_lo ); /* careful: unsigned zone may wrap */
>
>           if ( (memflags & MEMF_exact_node) && req_node != NUMA_NO_NODE )
> @@ -706,9 +723,15 @@ static struct page_info *alloc_heap_pages(
>       /* We may have to halve the chunk a number of times. */
>       while ( j != order )
>       {
> -        PFN_ORDER(pg) = --j;
> -        page_list_add_tail(pg, &heap(node, zone, j));
> -        pg += 1 << j;
> +        tmp_pg = pg;
> +        if ( pfn )
> +            tmp_pg += 1 << (j - 1);
> +
> +        PFN_ORDER(tmp_pg) = --j;
> +        page_list_add_tail(tmp_pg, &heap(node, zone, j));
> +
> +        if ( !pfn )
> +            pg += 1 << j;
>       }
>
>       ASSERT(avail[node][zone] >= request);
> @@ -762,6 +785,15 @@ static struct page_info *alloc_heap_pages(
>       return pg;
>   }
>
> +/* Allocate 2^@order contiguous pages. */
> +static struct page_info *alloc_heap_pages(
> +    unsigned int zone_lo, unsigned int zone_hi,
> +    unsigned int order, unsigned int memflags,
> +    struct domain *d)
> +{
> +    return alloc_heap_pages_pfn(zone_lo, zone_hi, order, memflags, d, 0);
> +}
> +
>   /* Remove any offlined page in the buddy pointed to by head. */
>   static int reserve_offlined_page(struct page_info *head)
>   {
> @@ -1687,8 +1719,8 @@ int assign_pages(
>   }
>
>
> -struct page_info *alloc_domheap_pages(
> -    struct domain *d, unsigned int order, unsigned int memflags)
> +struct page_info *alloc_domheap_pages_pfn(
> +    struct domain *d, unsigned int order, unsigned int memflags, xen_pfn_t pfn)
>   {
>       struct page_info *pg = NULL;
>       unsigned int bits = memflags >> _MEMF_bits, zone_hi = NR_ZONES - 1;
> @@ -1705,12 +1737,12 @@ struct page_info *alloc_domheap_pages(
>           memflags |= MEMF_no_refcount;
>
>       if ( dma_bitsize && ((dma_zone = bits_to_zone(dma_bitsize)) < zone_hi) )
> -        pg = alloc_heap_pages(dma_zone + 1, zone_hi, order, memflags, d);
> +        pg = alloc_heap_pages_pfn(dma_zone + 1, zone_hi, order, memflags, d, pfn);
>
>       if ( (pg == NULL) &&
>            ((memflags & MEMF_no_dma) ||
> -          ((pg = alloc_heap_pages(MEMZONE_XEN + 1, zone_hi, order,
> -                                  memflags, d)) == NULL)) )
> +          ((pg = alloc_heap_pages_pfn(MEMZONE_XEN + 1, zone_hi, order,
> +                                  memflags, d, pfn)) == NULL)) )
>            return NULL;
>
>       if ( d && !(memflags & MEMF_no_owner) &&
> @@ -1723,6 +1755,12 @@ struct page_info *alloc_domheap_pages(
>       return pg;
>   }
>
> +struct page_info *alloc_domheap_pages(
> +    struct domain *d, unsigned int order, unsigned int memflags)
> +{
> +    return alloc_domheap_pages_pfn(d, order, memflags, 0);
> +}
> +
>   void free_domheap_pages(struct page_info *pg, unsigned int order)
>   {
>       struct domain *d = page_get_owner(pg);
> diff --git a/xen/include/xen/mm.h b/xen/include/xen/mm.h
> index a795dd6..83e4913 100644
> --- a/xen/include/xen/mm.h
> +++ b/xen/include/xen/mm.h
> @@ -117,6 +117,8 @@ void get_outstanding_claims(uint64_t *free_pages, uint64_t *outstanding_pages);
>
>   /* Domain suballocator. These functions are *not* interrupt-safe.*/
>   void init_domheap_pages(paddr_t ps, paddr_t pe);
> +struct page_info *alloc_domheap_pages_pfn(
> +    struct domain *d, unsigned int order, unsigned int memflags, xen_pfn_t pfn);
>   struct page_info *alloc_domheap_pages(
>       struct domain *d, unsigned int order, unsigned int memflags);
>   void free_domheap_pages(struct page_info *pg, unsigned int order);
>

Regards,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH RFC 13/18] xen: introduce and use 'dom0_rambase_pfn' setting for kernel Dom0

[Oleksandr Dmytryshyn]

> Why would a user want to allocate DOM0 RAM bank to a specific address?
>
> If I understand correctly your patch, DOM0 will only able to allocate one bank of the given size at the specific address. You also add this possibility for guest domain (see patch #4) and try to control where the guest memory will be allocated. This will increase a lot the chance of the memory allocation to fail.
>
> For instance, the RAM region requested for DOM0 may have been used to allocate memory for Xen internal. So you need a way to reserve memory in order to avoid Xen using it.
>
> I expect most of the users who want to use direct memory mapped guest to know the number of guests which will use this feature.
>
> A such feature is only useful when pass-through a device to the guest on platfom without SMMU, so it is by default insecure.
>
> So I would suggest to create a new device-tree binding (or re-use an actual one) to reserve memory region to be used for direct memory mapped domain.
>
> Those regions could have an identifier to be used later during the allocation. This would avoid memory fragmentation, allow multiple RAM bank for DOM0,...
>
> Any opinions?

Case 1: Dom0 is driver domain:
There is a Ducati firmware which runs on dedicated M4 core and decodes
video. This firmware uses hardcoded physical addresses for graphics
buffers. Those addresses should be inside address-space of the driver
domain (Dom0). Ducati firmware is proprietary and we have no ability
to rework it. So Dom0 kernel should be placed to the configured
address (to the DOM0 RAM bank with specific address).

Case 2: Dom0 is Thin and DomD is driver domain.
All is the same: Ducati firmware requires special (hardcoded) addresses.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH RFC 13/18] xen: introduce and use 'dom0_rambase_pfn' setting for kernel Dom0

[Julien Grall]

Hello Oleksandr,

On 19/05/16 14:58, Oleksandr Dmytryshyn wrote:
>> Why would a user want to allocate DOM0 RAM bank to a specific address?
>>
>> If I understand correctly your patch, DOM0 will only able to allocate one bank of the given size at the specific address. You also add this possibility for guest domain (see patch #4) and try to control where the guest memory will be allocated. This will increase a lot the chance of the memory allocation to fail.
>>
>> For instance, the RAM region requested for DOM0 may have been used to allocate memory for Xen internal. So you need a way to reserve memory in order to avoid Xen using it.
>>
>> I expect most of the users who want to use direct memory mapped guest to know the number of guests which will use this feature.
>>
>> A such feature is only useful when pass-through a device to the guest on platfom without SMMU, so it is by default insecure.
>>
>> So I would suggest to create a new device-tree binding (or re-use an actual one) to reserve memory region to be used for direct memory mapped domain.
>>
>> Those regions could have an identifier to be used later during the allocation. This would avoid memory fragmentation, allow multiple RAM bank for DOM0,...
>>
>> Any opinions?
>
> Case 1: Dom0 is driver domain:
> There is a Ducati firmware which runs on dedicated M4 core and decodes
> video. This firmware uses hardcoded physical addresses for graphics
> buffers. Those addresses should be inside address-space of the driver
> domain (Dom0). Ducati firmware is proprietary and we have no ability
> to rework it. So Dom0 kernel should be placed to the configured
> address (to the DOM0 RAM bank with specific address).
>
> Case 2: Dom0 is Thin and DomD is driver domain.
> All is the same: Ducati firmware requires special (hardcoded) addresses.

So if I understand correctly, patches #4, #13, #16 are only here to 
workaround a firmware which does not do the right thing?

IHMO, modifying the memory allocator in Xen to make a firmware happy is 
just overkill. We need to explore all the possible solutions before 
going forward.

 From your description, it looks like to me that the device-tree does 
not correctly describe the platform. The graphic buffers should be 
reserved using /memreserve or via a specific binding.

This would be used later by Xen to map the buffer into dom0 or allow 
dom0 to map it to a guest.

Regards,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH RFC 13/18] xen: introduce and use 'dom0_rambase_pfn' setting for kernel Dom0

[Oleksandr Dmytryshyn]

Oleksandr Dmytryshyn | Product Engineering and Development
GlobalLogic
M +38.067.382.2525
www.globallogic.com

http://www.globallogic.com/email_disclaimer.txt


On Thu, May 19, 2016 at 5:34 PM, Julien Grall <julien.grall@arm.com> wrote:
> Hello Oleksandr,
>
>
> On 19/05/16 14:58, Oleksandr Dmytryshyn wrote:
>>>
>>> Why would a user want to allocate DOM0 RAM bank to a specific address?
>>>
>>> If I understand correctly your patch, DOM0 will only able to allocate one
>>> bank of the given size at the specific address. You also add this
>>> possibility for guest domain (see patch #4) and try to control where the
>>> guest memory will be allocated. This will increase a lot the chance of the
>>> memory allocation to fail.
>>>
>>> For instance, the RAM region requested for DOM0 may have been used to
>>> allocate memory for Xen internal. So you need a way to reserve memory in
>>> order to avoid Xen using it.
>>>
>>> I expect most of the users who want to use direct memory mapped guest to
>>> know the number of guests which will use this feature.
>>>
>>> A such feature is only useful when pass-through a device to the guest on
>>> platfom without SMMU, so it is by default insecure.
>>>
>>> So I would suggest to create a new device-tree binding (or re-use an
>>> actual one) to reserve memory region to be used for direct memory mapped
>>> domain.
>>>
>>> Those regions could have an identifier to be used later during the
>>> allocation. This would avoid memory fragmentation, allow multiple RAM bank
>>> for DOM0,...
>>>
>>> Any opinions?
>>
>>
>> Case 1: Dom0 is driver domain:
>> There is a Ducati firmware which runs on dedicated M4 core and decodes
>> video. This firmware uses hardcoded physical addresses for graphics
>> buffers. Those addresses should be inside address-space of the driver
>> domain (Dom0). Ducati firmware is proprietary and we have no ability
>> to rework it. So Dom0 kernel should be placed to the configured
>> address (to the DOM0 RAM bank with specific address).
>>
>> Case 2: Dom0 is Thin and DomD is driver domain.
>> All is the same: Ducati firmware requires special (hardcoded) addresses.
>
>
> So if I understand correctly, patches #4, #13, #16 are only here to
> workaround a firmware which does not do the right thing?
>
> IHMO, modifying the memory allocator in Xen to make a firmware happy is just
> overkill. We need to explore all the possible solutions before going
> forward.
Yes, You are right. This patch written to make a firmware happy.

> From your description, it looks like to me that the device-tree does not
> correctly describe the platform. The graphic buffers should be reserved
> using /memreserve or via a specific binding.
>
> This would be used later by Xen to map the buffer into dom0 or allow dom0 to
> map it to a guest.
>
> Regards,
>
> --
> Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH RFC 13/18] xen: introduce and use 'dom0_rambase_pfn' setting for kernel Dom0

[Jan Beulich]

>>> On 19.05.16 at 15:58, <oleksandr.dmytryshyn@globallogic.com> wrote:
> Case 1: Dom0 is driver domain:
> There is a Ducati firmware which runs on dedicated M4 core and decodes
> video. This firmware uses hardcoded physical addresses for graphics
> buffers. Those addresses should be inside address-space of the driver
> domain (Dom0). Ducati firmware is proprietary and we have no ability
> to rework it. So Dom0 kernel should be placed to the configured
> address (to the DOM0 RAM bank with specific address).
> 
> Case 2: Dom0 is Thin and DomD is driver domain.
> All is the same: Ducati firmware requires special (hardcoded) addresses.

For both of these cases I would then wonder whether such
environments are actually suitable for doing virtualization on.

Jan


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH RFC 13/18] xen: introduce and use 'dom0_rambase_pfn' setting for kernel Dom0

[Oleksandr Dmytryshyn]

On Thu, May 19, 2016 at 5:36 PM, Jan Beulich <JBeulich@suse.com> wrote:
>>>> On 19.05.16 at 15:58, <oleksandr.dmytryshyn@globallogic.com> wrote:
>> Case 1: Dom0 is driver domain:
>> There is a Ducati firmware which runs on dedicated M4 core and decodes
>> video. This firmware uses hardcoded physical addresses for graphics
>> buffers. Those addresses should be inside address-space of the driver
>> domain (Dom0). Ducati firmware is proprietary and we have no ability
>> to rework it. So Dom0 kernel should be placed to the configured
>> address (to the DOM0 RAM bank with specific address).
>>
>> Case 2: Dom0 is Thin and DomD is driver domain.
>> All is the same: Ducati firmware requires special (hardcoded) addresses.
>
> For both of these cases I would then wonder whether such
> environments are actually suitable for doing virtualization on.
Currently we use Jacinto 6 evaluation board with DRA74X processor.
We have both configurations (Thin Dom0 and Thich Dom0).

> Jan
>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH RFC 13/18] xen: introduce and use 'dom0_rambase_pfn' setting for kernel Dom0

[Jan Beulich]

>>> On 20.05.16 at 10:45, <oleksandr.dmytryshyn@globallogic.com> wrote:
> On Thu, May 19, 2016 at 5:36 PM, Jan Beulich <JBeulich@suse.com> wrote:
>>>>> On 19.05.16 at 15:58, <oleksandr.dmytryshyn@globallogic.com> wrote:
>>> Case 1: Dom0 is driver domain:
>>> There is a Ducati firmware which runs on dedicated M4 core and decodes
>>> video. This firmware uses hardcoded physical addresses for graphics
>>> buffers. Those addresses should be inside address-space of the driver
>>> domain (Dom0). Ducati firmware is proprietary and we have no ability
>>> to rework it. So Dom0 kernel should be placed to the configured
>>> address (to the DOM0 RAM bank with specific address).
>>>
>>> Case 2: Dom0 is Thin and DomD is driver domain.
>>> All is the same: Ducati firmware requires special (hardcoded) addresses.
>>
>> For both of these cases I would then wonder whether such
>> environments are actually suitable for doing virtualization on.
> Currently we use Jacinto 6 evaluation board with DRA74X processor.
> We have both configurations (Thin Dom0 and Thich Dom0).

Which says nothing about their suitability for virtualization.

Jan


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH RFC 13/18] xen: introduce and use 'dom0_rambase_pfn' setting for kernel Dom0

[Oleksandr Dmytryshyn]

On Fri, May 20, 2016 at 12:59 PM, Jan Beulich <JBeulich@suse.com> wrote:
>>>> On 20.05.16 at 10:45, <oleksandr.dmytryshyn@globallogic.com> wrote:
>> On Thu, May 19, 2016 at 5:36 PM, Jan Beulich <JBeulich@suse.com> wrote:
>>>>>> On 19.05.16 at 15:58, <oleksandr.dmytryshyn@globallogic.com> wrote:
>>>> Case 1: Dom0 is driver domain:
>>>> There is a Ducati firmware which runs on dedicated M4 core and decodes
>>>> video. This firmware uses hardcoded physical addresses for graphics
>>>> buffers. Those addresses should be inside address-space of the driver
>>>> domain (Dom0). Ducati firmware is proprietary and we have no ability
>>>> to rework it. So Dom0 kernel should be placed to the configured
>>>> address (to the DOM0 RAM bank with specific address).
>>>>
>>>> Case 2: Dom0 is Thin and DomD is driver domain.
>>>> All is the same: Ducati firmware requires special (hardcoded) addresses.
>>>
>>> For both of these cases I would then wonder whether such
>>> environments are actually suitable for doing virtualization on.
>> Currently we use Jacinto 6 evaluation board with DRA74X processor.
>> We have both configurations (Thin Dom0 and Thich Dom0).
>
> Which says nothing about their suitability for virtualization.
Our solution is based on Jacinto 6 evaluation board with DRA74X
processor. We need video-playback. Ducati firmware decodes video and
it works only with hardcoded addresses so we need this patch.

> Jan
>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH RFC 13/18] xen: introduce and use 'dom0_rambase_pfn' setting for kernel Dom0

[Julien Grall]

Hello Oleksandr,

On 20/05/16 15:19, Oleksandr Dmytryshyn wrote:
> On Fri, May 20, 2016 at 12:59 PM, Jan Beulich <JBeulich@suse.com> wrote:
>>>>> On 20.05.16 at 10:45, <oleksandr.dmytryshyn@globallogic.com> wrote:
>>> On Thu, May 19, 2016 at 5:36 PM, Jan Beulich <JBeulich@suse.com> wrote:
>>>>>>> On 19.05.16 at 15:58, <oleksandr.dmytryshyn@globallogic.com> wrote:
>>>>> Case 1: Dom0 is driver domain:
>>>>> There is a Ducati firmware which runs on dedicated M4 core and decodes
>>>>> video. This firmware uses hardcoded physical addresses for graphics
>>>>> buffers. Those addresses should be inside address-space of the driver
>>>>> domain (Dom0). Ducati firmware is proprietary and we have no ability
>>>>> to rework it. So Dom0 kernel should be placed to the configured
>>>>> address (to the DOM0 RAM bank with specific address).
>>>>>
>>>>> Case 2: Dom0 is Thin and DomD is driver domain.
>>>>> All is the same: Ducati firmware requires special (hardcoded) addresses.
>>>>
>>>> For both of these cases I would then wonder whether such
>>>> environments are actually suitable for doing virtualization on.
>>> Currently we use Jacinto 6 evaluation board with DRA74X processor.
>>> We have both configurations (Thin Dom0 and Thich Dom0).
>>
>> Which says nothing about their suitability for virtualization.
> Our solution is based on Jacinto 6 evaluation board with DRA74X
> processor. We need video-playback. Ducati firmware decodes video and
> it works only with hardcoded addresses so we need this patch.

This patch is a way to solve the problem and may not be the only one.
I would like to explore all the possibilities before taking an approach 
that requires to modify the memory allocator in Xen.

In my previous mails, I suggested a different solution (see [1] and 
[2]). If you think it is not suitable, please share more details or 
explain why you think your patch is the only way to solve it.

Regards,

[1] 
http://lists.xenproject.org/archives/html/xen-devel/2016-05/msg01879.html
[2] 
http://lists.xenproject.org/archives/html/xen-devel/2016-05/msg01894.html

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH RFC 13/18] xen: introduce and use 'dom0_rambase_pfn' setting for kernel Dom0

[Edgar E. Iglesias]

On Fri, May 20, 2016 at 04:04:43PM +0100, Julien Grall wrote:
> Hello Oleksandr,
> 
> On 20/05/16 15:19, Oleksandr Dmytryshyn wrote:
> >On Fri, May 20, 2016 at 12:59 PM, Jan Beulich <JBeulich@suse.com> wrote:
> >>>>>On 20.05.16 at 10:45, <oleksandr.dmytryshyn@globallogic.com> wrote:
> >>>On Thu, May 19, 2016 at 5:36 PM, Jan Beulich <JBeulich@suse.com> wrote:
> >>>>>>>On 19.05.16 at 15:58, <oleksandr.dmytryshyn@globallogic.com> wrote:
> >>>>>Case 1: Dom0 is driver domain:
> >>>>>There is a Ducati firmware which runs on dedicated M4 core and decodes
> >>>>>video. This firmware uses hardcoded physical addresses for graphics
> >>>>>buffers. Those addresses should be inside address-space of the driver
> >>>>>domain (Dom0). Ducati firmware is proprietary and we have no ability
> >>>>>to rework it. So Dom0 kernel should be placed to the configured
> >>>>>address (to the DOM0 RAM bank with specific address).
> >>>>>
> >>>>>Case 2: Dom0 is Thin and DomD is driver domain.
> >>>>>All is the same: Ducati firmware requires special (hardcoded) addresses.
> >>>>
> >>>>For both of these cases I would then wonder whether such
> >>>>environments are actually suitable for doing virtualization on.
> >>>Currently we use Jacinto 6 evaluation board with DRA74X processor.
> >>>We have both configurations (Thin Dom0 and Thich Dom0).
> >>
> >>Which says nothing about their suitability for virtualization.
> >Our solution is based on Jacinto 6 evaluation board with DRA74X
> >processor. We need video-playback. Ducati firmware decodes video and
> >it works only with hardcoded addresses so we need this patch.
> 
> This patch is a way to solve the problem and may not be the only one.
> I would like to explore all the possibilities before taking an approach that
> requires to modify the memory allocator in Xen.
> 
> In my previous mails, I suggested a different solution (see [1] and [2]). If
> you think it is not suitable, please share more details or explain why you
> think your patch is the only way to solve it.

Hi,

We have similar needs (not exactly the same) in some of our setups.
We need to map certain OCMs (On Chip Memories) to dom0. Among other things,
these are used to communicate with remote accelerators/CPUs that have
"hardcoded" addresses to these RAMs.

Our approach is more along the lines of Juliens second suggestion. We're
trying to use the mmio-sram DTS bindings to bring in these memories into
dom0.

IIUC the Ducati FW issue correctly, you need to allocate a chunk of DDR.

Another possible solution:
I think you could reserve the memory area by simply not mentioning it
in the main memory node (these nodes support multiple ranges so you can
introduce gaps). Then you could for example create an mmio-sram node to
get the memory explicitely mapped 1:1 into dom0.

Just a moment ago, I posted an RFC for the mmio-sram support to the list.

Cheers,
Edgar


> 
> Regards,
> 
> [1]
> http://lists.xenproject.org/archives/html/xen-devel/2016-05/msg01879.html
> [2]
> http://lists.xenproject.org/archives/html/xen-devel/2016-05/msg01894.html
> 
> -- 
> Julien Grall
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH RFC 13/18] xen: introduce and use 'dom0_rambase_pfn' setting for kernel Dom0

[Oleksandr Dmytryshyn]

Hi, Edgar.

On Fri, May 20, 2016 at 7:05 PM, Edgar E. Iglesias
<edgar.iglesias@gmail.com> wrote:
> Hi,
>
> We have similar needs (not exactly the same) in some of our setups.
> We need to map certain OCMs (On Chip Memories) to dom0. Among other things,
> these are used to communicate with remote accelerators/CPUs that have
> "hardcoded" addresses to these RAMs.
>
> Our approach is more along the lines of Juliens second suggestion. We're
> trying to use the mmio-sram DTS bindings to bring in these memories into
> dom0.
>
> IIUC the Ducati FW issue correctly, you need to allocate a chunk of DDR.
>
> Another possible solution:
> I think you could reserve the memory area by simply not mentioning it
> in the main memory node (these nodes support multiple ranges so you can
> introduce gaps). Then you could for example create an mmio-sram node to
> get the memory explicitely mapped 1:1 into dom0.
>
> Just a moment ago, I posted an RFC for the mmio-sram support to the list.
Thank You for advice.
I'll try to use Your solution.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH RFC 13/18] xen: introduce and use 'dom0_rambase_pfn' setting for kernel Dom0

[Stefano Stabellini]

On Fri, 20 May 2016, Edgar E. Iglesias wrote:
> On Fri, May 20, 2016 at 04:04:43PM +0100, Julien Grall wrote:
> > Hello Oleksandr,
> > 
> > On 20/05/16 15:19, Oleksandr Dmytryshyn wrote:
> > >On Fri, May 20, 2016 at 12:59 PM, Jan Beulich <JBeulich@suse.com> wrote:
> > >>>>>On 20.05.16 at 10:45, <oleksandr.dmytryshyn@globallogic.com> wrote:
> > >>>On Thu, May 19, 2016 at 5:36 PM, Jan Beulich <JBeulich@suse.com> wrote:
> > >>>>>>>On 19.05.16 at 15:58, <oleksandr.dmytryshyn@globallogic.com> wrote:
> > >>>>>Case 1: Dom0 is driver domain:
> > >>>>>There is a Ducati firmware which runs on dedicated M4 core and decodes
> > >>>>>video. This firmware uses hardcoded physical addresses for graphics
> > >>>>>buffers. Those addresses should be inside address-space of the driver
> > >>>>>domain (Dom0). Ducati firmware is proprietary and we have no ability
> > >>>>>to rework it. So Dom0 kernel should be placed to the configured
> > >>>>>address (to the DOM0 RAM bank with specific address).
> > >>>>>
> > >>>>>Case 2: Dom0 is Thin and DomD is driver domain.
> > >>>>>All is the same: Ducati firmware requires special (hardcoded) addresses.
> > >>>>
> > >>>>For both of these cases I would then wonder whether such
> > >>>>environments are actually suitable for doing virtualization on.
> > >>>Currently we use Jacinto 6 evaluation board with DRA74X processor.
> > >>>We have both configurations (Thin Dom0 and Thich Dom0).
> > >>
> > >>Which says nothing about their suitability for virtualization.
> > >Our solution is based on Jacinto 6 evaluation board with DRA74X
> > >processor. We need video-playback. Ducati firmware decodes video and
> > >it works only with hardcoded addresses so we need this patch.
> > 
> > This patch is a way to solve the problem and may not be the only one.
> > I would like to explore all the possibilities before taking an approach that
> > requires to modify the memory allocator in Xen.
> > 
> > In my previous mails, I suggested a different solution (see [1] and [2]). If
> > you think it is not suitable, please share more details or explain why you
> > think your patch is the only way to solve it.
> 
> Hi,
> 
> We have similar needs (not exactly the same) in some of our setups.
> We need to map certain OCMs (On Chip Memories) to dom0. Among other things,
> these are used to communicate with remote accelerators/CPUs that have
> "hardcoded" addresses to these RAMs.
> 
> Our approach is more along the lines of Juliens second suggestion. We're
> trying to use the mmio-sram DTS bindings to bring in these memories into
> dom0.
> 
> IIUC the Ducati FW issue correctly, you need to allocate a chunk of DDR.
> 
> Another possible solution:
> I think you could reserve the memory area by simply not mentioning it
> in the main memory node (these nodes support multiple ranges so you can
> introduce gaps). Then you could for example create an mmio-sram node to
> get the memory explicitely mapped 1:1 into dom0.
> 
> Just a moment ago, I posted an RFC for the mmio-sram support to the list.

This sounds much nicer that the solution proposed here.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH RFC 13/18] xen: introduce and use 'dom0_rambase_pfn' setting for kernel Dom0

[Oleksandr Dmytryshyn]

On Fri, May 20, 2016 at 7:05 PM, Edgar E. Iglesias
<edgar.iglesias@gmail.com> wrote:
> Hi,
>
> We have similar needs (not exactly the same) in some of our setups.
> We need to map certain OCMs (On Chip Memories) to dom0. Among other things,
> these are used to communicate with remote accelerators/CPUs that have
> "hardcoded" addresses to these RAMs.
>
> Our approach is more along the lines of Juliens second suggestion. We're
> trying to use the mmio-sram DTS bindings to bring in these memories into
> dom0.
>
> IIUC the Ducati FW issue correctly, you need to allocate a chunk of DDR.
>
> Another possible solution:
> I think you could reserve the memory area by simply not mentioning it
> in the main memory node (these nodes support multiple ranges so you can
> introduce gaps). Then you could for example create an mmio-sram node to
> get the memory explicitely mapped 1:1 into dom0.
>
> Just a moment ago, I posted an RFC for the mmio-sram support to the list.
Hi, Edgar.

How do You access to the mapped OCMs in dom0?
Are genalloc-related functions (gen_pool_get/_alloc/_virt_to_phys)
only way to work with mmio-sram memory?

> Cheers,
> Edgar
>
>
>>
>> Regards,
>>
>> [1]
>> http://lists.xenproject.org/archives/html/xen-devel/2016-05/msg01879.html
>> [2]
>> http://lists.xenproject.org/archives/html/xen-devel/2016-05/msg01894.html
>>
>> --
>> Julien Grall
>>
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@lists.xen.org
>> http://lists.xen.org/xen-devel

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH RFC 13/18] xen: introduce and use 'dom0_rambase_pfn' setting for kernel Dom0

[Edgar E. Iglesias]

On Tue, May 31, 2016 at 05:04:42PM +0300, Oleksandr Dmytryshyn wrote:
> On Fri, May 20, 2016 at 7:05 PM, Edgar E. Iglesias
> <edgar.iglesias@gmail.com> wrote:
> > Hi,
> >
> > We have similar needs (not exactly the same) in some of our setups.
> > We need to map certain OCMs (On Chip Memories) to dom0. Among other things,
> > these are used to communicate with remote accelerators/CPUs that have
> > "hardcoded" addresses to these RAMs.
> >
> > Our approach is more along the lines of Juliens second suggestion. We're
> > trying to use the mmio-sram DTS bindings to bring in these memories into
> > dom0.
> >
> > IIUC the Ducati FW issue correctly, you need to allocate a chunk of DDR.
> >
> > Another possible solution:
> > I think you could reserve the memory area by simply not mentioning it
> > in the main memory node (these nodes support multiple ranges so you can
> > introduce gaps). Then you could for example create an mmio-sram node to
> > get the memory explicitely mapped 1:1 into dom0.
> >
> > Just a moment ago, I posted an RFC for the mmio-sram support to the list.
> Hi, Edgar.
> 
> How do You access to the mapped OCMs in dom0?
> Are genalloc-related functions (gen_pool_get/_alloc/_virt_to_phys)
> only way to work with mmio-sram memory?


Hi Oleksandr,

I'm not familiar enough with the Linux APIs to give a good answer on which
APIs are considered OK and which not.

There are examples in the tree on other ways of using the srams though.
Look for example at this (search for smp-sram):

arch/arm/mach-rockchip/platsmp.c
Documentation/devicetree/bindings/sram/rockchip-smp-sram.txt

The allwinner,sun4i-a10-emac is another example.

Best regards,
Edgar


> 
> > Cheers,
> > Edgar
> >
> >
> >>
> >> Regards,
> >>
> >> [1]
> >> http://lists.xenproject.org/archives/html/xen-devel/2016-05/msg01879.html
> >> [2]
> >> http://lists.xenproject.org/archives/html/xen-devel/2016-05/msg01894.html
> >>
> >> --
> >> Julien Grall
> >>
> >> _______________________________________________
> >> Xen-devel mailing list
> >> Xen-devel@lists.xen.org
> >> http://lists.xen.org/xen-devel

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[PATCH RFC 14/18] xen: flask: Add possiblity to forward irqs into domU domains

[Andrii Anisov]

From: Iurii Mykhalskyi <iurii.mykhalskyi@globallogic.com>

Looks like that from 4.6 version, Xen didn't allow irq forwarding
into DomU domains - in our setup we forward list of them, so we need
to changed the policy.

Signed-off-by: Iurii Mykhalskyi <iurii.mykhalskyi@globallogic.com>
---
 tools/flask/policy/policy/modules/xen/xen.te | 1 +
 1 file changed, 1 insertion(+)

diff --git a/tools/flask/policy/policy/modules/xen/xen.te b/tools/flask/policy/policy/modules/xen/xen.te
index 5e94ee3..b3eaa15 100644
--- a/tools/flask/policy/policy/modules/xen/xen.te
+++ b/tools/flask/policy/policy/modules/xen/xen.te
@@ -116,6 +116,7 @@ admin_device(dom0_t, device_t)
 admin_device(dom0_t, irq_t)
 admin_device(dom0_t, ioport_t)
 admin_device(dom0_t, iomem_t)
+admin_device(domU_t, irq_t);
 
 domain_comms(dom0_t, dom0_t)
 
-- 
2.8.2


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[PATCH RFC 15/18] arm: Add ability to relocate Xen in over 4GB space

[Andrii Anisov]

From: Iurii Konovalenko <iurii.konovalenko@globallogic.com>

Move Xen to the end of physical memory

Signed-off-by: Iurii Konovalenko <iurii.konovalenko@globallogic.com>
---
 xen/Rules.mk                   |  1 +
 xen/arch/arm/arm32/head.S      | 21 ++++++++++++++++++++-
 xen/arch/arm/domain_build.c    |  2 +-
 xen/arch/arm/platforms/omap5.c | 17 ++++++++++++++---
 xen/arch/arm/platforms/rcar2.c |  9 ++++++++-
 xen/arch/arm/setup.c           | 21 ++++++++++++++++++++-
 xen/arch/arm/smpboot.c         | 33 +++++++++++++++++++++++++++++----
 7 files changed, 93 insertions(+), 11 deletions(-)

diff --git a/xen/Rules.mk b/xen/Rules.mk
index feb08d6..fbd34a5 100644
--- a/xen/Rules.mk
+++ b/xen/Rules.mk
@@ -64,6 +64,7 @@ CFLAGS-$(HAS_PCI)       += -DHAS_PCI
 CFLAGS-$(HAS_IOPORTS)   += -DHAS_IOPORTS
 CFLAGS-$(HAS_PDX)       += -DHAS_PDX
 CFLAGS-$(frame_pointer) += -fno-omit-frame-pointer -DCONFIG_FRAME_POINTER
+CFLAGS-$(ARM32_RELOCATE_OVER_4GB) += -DARM32_RELOCATE_OVER_4GB
 
 ifneq ($(max_phys_cpus),)
 CFLAGS-y                += -DMAX_PHYS_CPUS=$(max_phys_cpus)
diff --git a/xen/arch/arm/arm32/head.S b/xen/arch/arm/arm32/head.S
index e1f29bd..a644d6d 100644
--- a/xen/arch/arm/arm32/head.S
+++ b/xen/arch/arm/arm32/head.S
@@ -262,8 +262,21 @@ cpu_init_done:
         add   r4, r4, r10            /* r4 := paddr (boot_pagetable) */
         mov   r5, #0                 /* r4:r5 is paddr (boot_pagetable) */
         mcrr  CP64(r4, r5, HTTBR)
+#ifdef ARM32_RELOCATE_OVER_4GB
+        teq   r7, #0
+        beq   1f                     /* construct pagetable if CPU0 */
 
-        /* Setup boot_pgtable: */
+        /*Skip constructing TLBs for secondary CPUs, use constracted by CPU0*/
+        PRINT("- Skip construction pagetable, using CPU0 table @")
+        mov   r0, r5
+        bl    putn
+        mov   r0, r4
+        bl    putn
+        PRINT("  -\r\n")
+        bne   skip_constructing
+#endif
+
+1:      /* Setup boot_pgtable: */
         ldr   r1, =boot_second
         add   r1, r1, r10            /* r1 := paddr (boot_second) */
 
@@ -346,6 +359,7 @@ virtphys_clash:
         PRINT("- Unable to build boot page tables - virt and phys addresses clash. -\r\n")
         b     fail
 
+skip_constructing:
 1:
         PRINT("- Turning on paging -\r\n")
 
@@ -427,6 +441,11 @@ paging:
          * setup in init_secondary_pagetables. */
 
         ldr   r4, =init_ttbr         /* VA of HTTBR value stashed by CPU 0 */
+#ifdef ARM32_RELOCATE_OVER_4GB
+        ldr   r1, =_start
+        sub r4, r1
+        add r4, #BOOT_RELOC_VIRT_START
+#endif
         ldrd  r4, r5, [r4]           /* Actual value */
         dsb
         mcrr  CP64(r4, r5, HTTBR)
diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index b48718d..f06792e 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -1487,7 +1487,7 @@ static void __init find_gnttab_region(struct domain *d,
     if ( (kinfo->gnttab_size >> PAGE_SHIFT) < max_grant_frames )
         panic("Cannot find a space for the grant table region\n");
 
-#ifdef CONFIG_ARM_32
+#if defined(CONFIG_ARM_32) && !defined(ARM32_RELOCATE_OVER_4GB)
     /*
      * The gnttab region must be under 4GB in order to work with DOM0
      * using short page table.
diff --git a/xen/arch/arm/platforms/omap5.c b/xen/arch/arm/platforms/omap5.c
index a49ba62..fe77397 100644
--- a/xen/arch/arm/platforms/omap5.c
+++ b/xen/arch/arm/platforms/omap5.c
@@ -25,6 +25,10 @@
 #include <xen/vmap.h>
 #include <asm/io.h>
 
+#ifdef ARM32_RELOCATE_OVER_4GB
+extern paddr_t xen_relocation_offset;
+#endif
+
 static uint16_t num_den[8][2] = {
     {         0,          0 },  /* not used */
     {  26 *  64,  26 *  125 },  /* 12.0 Mhz */
@@ -132,9 +136,16 @@ static int __init omap5_smp_init(void)
     }
 
     printk("Set AuxCoreBoot1 to %"PRIpaddr" (%p)\n",
-           __pa(init_secondary), init_secondary);
-    writel(__pa(init_secondary), wugen_base + OMAP_AUX_CORE_BOOT_1_OFFSET);
-
+           __pa(init_secondary)
+#ifdef ARM32_RELOCATE_OVER_4GB
+            - xen_relocation_offset
+#endif
+			, init_secondary);
+    writel(__pa(init_secondary)
+#ifdef ARM32_RELOCATE_OVER_4GB
+            - xen_relocation_offset
+#endif
+			, wugen_base + OMAP_AUX_CORE_BOOT_1_OFFSET);
     printk("Set AuxCoreBoot0 to 0x20\n");
     writel(0x20, wugen_base + OMAP_AUX_CORE_BOOT_0_OFFSET);
 
diff --git a/xen/arch/arm/platforms/rcar2.c b/xen/arch/arm/platforms/rcar2.c
index bb25751..26973f6 100644
--- a/xen/arch/arm/platforms/rcar2.c
+++ b/xen/arch/arm/platforms/rcar2.c
@@ -25,6 +25,9 @@
 #define RCAR2_RAM_SIZE                         0x1000
 #define RCAR2_SMP_START_OFFSET                 0xFFC
 
+#ifdef ARM32_RELOCATE_OVER_4GB
+extern paddr_t xen_relocation_offset;
+#endif
 static int __init rcar2_smp_init(void)
 {
     void __iomem *pram;
@@ -38,7 +41,11 @@ static int __init rcar2_smp_init(void)
     }
 
     /* setup reset vectors */
-    writel(__pa(init_secondary), pram + RCAR2_SMP_START_OFFSET);
+    writel(__pa(init_secondary)
+#ifdef ARM32_RELOCATE_OVER_4GB
+            - xen_relocation_offset
+#endif
+            , pram + RCAR2_SMP_START_OFFSET);
     iounmap(pram);
 
     sev();
diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c
index 48f734f..7e507bc 100644
--- a/xen/arch/arm/setup.c
+++ b/xen/arch/arm/setup.c
@@ -394,9 +394,13 @@ static paddr_t __init get_xen_paddr(void)
         const struct membank *bank = &mi->bank[i];
         paddr_t s, e;
 
+        //TODO: investigate reason why we need contiguous memory
+        //and how it can affect relocation of Xen in over 4GB space
+#ifndef ARM32_RELOCATE_OVER_4GB
         /* We can only deal with contiguous memory at the moment */
         if ( last_end != bank->start )
             break;
+#endif
 
         last_end = bank->start + bank->size;
 
@@ -407,7 +411,7 @@ static paddr_t __init get_xen_paddr(void)
             if ( !e )
                 continue;
 
-#ifdef CONFIG_ARM_32
+#if defined (CONFIG_ARM_32) && !defined(ARM32_RELOCATE_OVER_4GB)
             /* Xen must be under 4GB */
             if ( e > 0x100000000ULL )
                 e = 0x100000000ULL;
@@ -698,6 +702,9 @@ void __init setup_cache(void)
     cacheline_bytes = 1U << (4 + (ccsid & 0x7));
 }
 
+#ifdef ARM32_RELOCATE_OVER_4GB
+paddr_t xen_relocation_offset;
+#endif
 /* C entry point for boot CPU */
 void __init start_xen(unsigned long boot_phys_offset,
                       unsigned long fdt_paddr,
@@ -739,8 +746,20 @@ void __init start_xen(unsigned long boot_phys_offset,
                              (paddr_t)(uintptr_t)(_end - _start + 1), NULL);
     BUG_ON(!xen_bootmodule);
 
+#ifdef ARM32_RELOCATE_OVER_4GB
+    //save physical address of init_secondary
+    xen_relocation_offset = __pa(init_secondary);
+#endif
     xen_paddr = get_xen_paddr();
     setup_pagetables(boot_phys_offset, xen_paddr);
+#ifdef ARM32_RELOCATE_OVER_4GB
+    //Now Xen is relocated
+    //Calculate offset of Xen relocation
+    //It is difference between new physical address of init_secondary an old one
+    //This offset is needed in several places when we have to write to old Xen location
+    //(secondary CPUs run on old-located Xen and rely on some variables from CPU0)
+    xen_relocation_offset = __pa(init_secondary) - xen_relocation_offset;
+#endif
 
     /* Update Xen's address now that we have relocated. */
     printk("Update BOOTMOD_XEN from %"PRIpaddr"-%"PRIpaddr" => %"PRIpaddr"-%"PRIpaddr"\n",
diff --git a/xen/arch/arm/smpboot.c b/xen/arch/arm/smpboot.c
index a96cda2..731144c 100644
--- a/xen/arch/arm/smpboot.c
+++ b/xen/arch/arm/smpboot.c
@@ -31,6 +31,9 @@
 #include <xen/console.h>
 #include <asm/gic.h>
 #include <asm/psci.h>
+#ifdef ARM32_RELOCATE_OVER_4GB
+#include <xen/vmap.h>
+#endif
 
 cpumask_t cpu_online_map;
 cpumask_t cpu_present_map;
@@ -353,17 +356,33 @@ int __init cpu_up_send_sgi(int cpu)
     return 0;
 }
 
+#ifdef ARM32_RELOCATE_OVER_4GB
+extern paddr_t xen_relocation_offset;
+#endif
 /* Bring up a remote CPU */
 int __cpu_up(unsigned int cpu)
 {
     int rc;
     s_time_t deadline;
+#ifdef ARM32_RELOCATE_OVER_4GB
+    paddr_t p_info = __pa(&smp_up_cpu) - xen_relocation_offset;
+    unsigned long* info = ioremap_nocache(p_info, sizeof(unsigned long));
+#else
+    unsigned long* info = &smp_up_cpu;
+#endif
 
     printk("Bringing up CPU%d\n", cpu);
 
     rc = init_secondary_pagetables(cpu);
     if ( rc < 0 )
+#ifdef ARM32_RELOCATE_OVER_4GB
+    {
+        iounmap(info);
+        return rc;
+    }
+#else
         return rc;
+#endif
 
     console_start_sync(); /* Secondary may use early_printk */
 
@@ -374,8 +393,8 @@ int __cpu_up(unsigned int cpu)
     init_data.cpuid = cpu;
 
     /* Open the gate for this CPU */
-    smp_up_cpu = cpu_logical_map(cpu);
-    clean_dcache(smp_up_cpu);
+    *info = cpu_logical_map(cpu);
+    clean_dcache(*info);
 
     rc = arch_cpu_up(cpu);
 
@@ -383,6 +402,9 @@ int __cpu_up(unsigned int cpu)
 
     if ( rc < 0 )
     {
+#ifdef ARM32_RELOCATE_OVER_4GB
+        iounmap(info);
+#endif
         printk("Failed to bring up CPU%d\n", cpu);
         return rc;
     }
@@ -406,8 +428,11 @@ int __cpu_up(unsigned int cpu)
      */
     init_data.stack = NULL;
     init_data.cpuid = ~0;
-    smp_up_cpu = MPIDR_INVALID;
-    clean_dcache(smp_up_cpu);
+    *info = MPIDR_INVALID;
+    clean_dcache(*info);
+#ifdef ARM32_RELOCATE_OVER_4GB
+    iounmap(info);
+#endif
 
     if ( !cpu_online(cpu) )
     {
-- 
2.8.2


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH RFC 15/18] arm: Add ability to relocate Xen in over 4GB space

[Jan Beulich]

>>> On 18.05.16 at 18:32, <andrii.anisov@globallogic.com> wrote:
> --- a/xen/Rules.mk
> +++ b/xen/Rules.mk
> @@ -64,6 +64,7 @@ CFLAGS-$(HAS_PCI)       += -DHAS_PCI
>  CFLAGS-$(HAS_IOPORTS)   += -DHAS_IOPORTS
>  CFLAGS-$(HAS_PDX)       += -DHAS_PDX
>  CFLAGS-$(frame_pointer) += -fno-omit-frame-pointer -DCONFIG_FRAME_POINTER
> +CFLAGS-$(ARM32_RELOCATE_OVER_4GB) += -DARM32_RELOCATE_OVER_4GB

Things like this should be done via Kconfig nowadays.

Jan


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH RFC 15/18] arm: Add ability to relocate Xen in over 4GB space

[Julien Grall]

Hello,

On 18/05/16 17:32, Andrii Anisov wrote:
> From: Iurii Konovalenko <iurii.konovalenko@globallogic.com>
>
> Move Xen to the end of physical memory

Can you explain why? As Ian campbell said on the previous version sent 
last year [1], Xen itself (i.e .text, .bss and .data, not the xenheap 
itself) is at most 2MB. Are your constraints such that at a 4GB-2MB 
(i.e. 4096M) of lowmem would not be acceptable?

Also, this patch looks very similar to the one you have sent last year 
[2]. Most of the comments in this thread are still valid, so I will not 
repeat them here.

Regards,

>
> Signed-off-by: Iurii Konovalenko <iurii.konovalenko@globallogic.com>
> ---
>   xen/Rules.mk                   |  1 +
>   xen/arch/arm/arm32/head.S      | 21 ++++++++++++++++++++-
>   xen/arch/arm/domain_build.c    |  2 +-
>   xen/arch/arm/platforms/omap5.c | 17 ++++++++++++++---
>   xen/arch/arm/platforms/rcar2.c |  9 ++++++++-
>   xen/arch/arm/setup.c           | 21 ++++++++++
>++++++++++-
>   xen/arch/arm/smpboot.c         | 33 +++++++++++++++++++++++++++++----
>   7 files changed, 93 insertions(+), 11 deletions(-)
>
> diff --git a/xen/Rules.mk b/xen/Rules.mk
> index feb08d6..fbd34a5 100644
> --- a/xen/Rules.mk
> +++ b/xen/Rules.mk
> @@ -64,6 +64,7 @@ CFLAGS-$(HAS_PCI)       += -DHAS_PCI
>   CFLAGS-$(HAS_IOPORTS)   += -DHAS_IOPORTS
>   CFLAGS-$(HAS_PDX)       += -DHAS_PDX
>   CFLAGS-$(frame_pointer) += -fno-omit-frame-pointer -DCONFIG_FRAME_POINTER
> +CFLAGS-$(ARM32_RELOCATE_OVER_4GB) += -DARM32_RELOCATE_OVER_4GB
>
>   ifneq ($(max_phys_cpus),)
>   CFLAGS-y                += -DMAX_PHYS_CPUS=$(max_phys_cpus)
> diff --git a/xen/arch/arm/arm32/head.S b/xen/arch/arm/arm32/head.S
> index e1f29bd..a644d6d 100644
> --- a/xen/arch/arm/arm32/head.S
> +++ b/xen/arch/arm/arm32/head.S
> @@ -262,8 +262,21 @@ cpu_init_done:
>           add   r4, r4, r10            /* r4 := paddr (boot_pagetable) */
>           mov   r5, #0                 /* r4:r5 is paddr (boot_pagetable) */
>           mcrr  CP64(r4, r5, HTTBR)
> +#ifdef ARM32_RELOCATE_OVER_4GB
> +        teq   r7, #0
> +        beq   1f                     /* construct pagetable if CPU0 */
>
> -        /* Setup boot_pgtable: */
> +        /*Skip constructing TLBs for secondary CPUs, use constracted by CPU0*/
> +        PRINT("- Skip construction pagetable, using CPU0 table @")
> +        mov   r0, r5
> +        bl    putn
> +        mov   r0, r4
> +        bl    putn
> +        PRINT("  -\r\n")
> +        bne   skip_constructing
> +#endif
> +
> +1:      /* Setup boot_pgtable: */
>           ldr   r1, =boot_second
>           add   r1, r1, r10            /* r1 := paddr (boot_second) */
>
> @@ -346,6 +359,7 @@ virtphys_clash:
>           PRINT("- Unable to build boot page tables - virt and phys addresses clash. -\r\n")
>           b     fail
>
> +skip_constructing:
>   1:
>           PRINT("- Turning on paging -\r\n")
>
> @@ -427,6 +441,11 @@ paging:
>            * setup in init_secondary_pagetables. */
>
>           ldr   r4, =init_ttbr         /* VA of HTTBR value stashed by CPU 0 */
> +#ifdef ARM32_RELOCATE_OVER_4GB
> +        ldr   r1, =_start
> +        sub r4, r1
> +        add r4, #BOOT_RELOC_VIRT_START
> +#endif
>           ldrd  r4, r5, [r4]           /* Actual value */
>           dsb
>           mcrr  CP64(r4, r5, HTTBR)
> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> index b48718d..f06792e 100644
> --- a/xen/arch/arm/domain_build.c
> +++ b/xen/arch/arm/domain_build.c
> @@ -1487,7 +1487,7 @@ static void __init find_gnttab_region(struct domain *d,
>       if ( (kinfo->gnttab_size >> PAGE_SHIFT) < max_grant_frames )
>           panic("Cannot find a space for the grant table region\n");
>
> -#ifdef CONFIG_ARM_32
> +#if defined(CONFIG_ARM_32) && !defined(ARM32_RELOCATE_OVER_4GB)
>       /*
>        * The gnttab region must be under 4GB in order to work with DOM0
>        * using short page table.
> diff --git a/xen/arch/arm/platforms/omap5.c b/xen/arch/arm/platforms/omap5.c
> index a49ba62..fe77397 100644
> --- a/xen/arch/arm/platforms/omap5.c
> +++ b/xen/arch/arm/platforms/omap5.c
> @@ -25,6 +25,10 @@
>   #include <xen/vmap.h>
>   #include <asm/io.h>
>
> +#ifdef ARM32_RELOCATE_OVER_4GB
> +extern paddr_t xen_relocation_offset;
> +#endif
> +
>   static uint16_t num_den[8][2] = {
>       {         0,          0 },  /* not used */
>       {  26 *  64,  26 *  125 },  /* 12.0 Mhz */
> @@ -132,9 +136,16 @@ static int __init omap5_smp_init(void)
>       }
>
>       printk("Set AuxCoreBoot1 to %"PRIpaddr" (%p)\n",
> -           __pa(init_secondary), init_secondary);
> -    writel(__pa(init_secondary), wugen_base + OMAP_AUX_CORE_BOOT_1_OFFSET);
> -
> +           __pa(init_secondary)
> +#ifdef ARM32_RELOCATE_OVER_4GB
> +            - xen_relocation_offset
> +#endif
> +			, init_secondary);
> +    writel(__pa(init_secondary)
> +#ifdef ARM32_RELOCATE_OVER_4GB
> +            - xen_relocation_offset
> +#endif
> +			, wugen_base + OMAP_AUX_CORE_BOOT_1_OFFSET);
>       printk("Set AuxCoreBoot0 to 0x20\n");
>       writel(0x20, wugen_base + OMAP_AUX_CORE_BOOT_0_OFFSET);
>
> diff --git a/xen/arch/arm/platforms/rcar2.c b/xen/arch/arm/platforms/rcar2.c
> index bb25751..26973f6 100644
> --- a/xen/arch/arm/platforms/rcar2.c
> +++ b/xen/arch/arm/platforms/rcar2.c
> @@ -25,6 +25,9 @@
>   #define RCAR2_RAM_SIZE                         0x1000
>   #define RCAR2_SMP_START_OFFSET                 0xFFC
>
> +#ifdef ARM32_RELOCATE_OVER_4GB
> +extern paddr_t xen_relocation_offset;
> +#endif
>   static int __init rcar2_smp_init(void)
>   {
>       void __iomem *pram;
> @@ -38,7 +41,11 @@ static int __init rcar2_smp_init(void)
>       }
>
>       /* setup reset vectors */
> -    writel(__pa(init_secondary), pram + RCAR2_SMP_START_OFFSET);
> +    writel(__pa(init_secondary)
> +#ifdef ARM32_RELOCATE_OVER_4GB
> +            - xen_relocation_offset
> +#endif
> +            , pram + RCAR2_SMP_START_OFFSET);
>       iounmap(pram);
>
>       sev();
> diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c
> index 48f734f..7e507bc 100644
> --- a/xen/arch/arm/setup.c
> +++ b/xen/arch/arm/setup.c
> @@ -394,9 +394,13 @@ static paddr_t __init get_xen_paddr(void)
>           const struct membank *bank = &mi->bank[i];
>           paddr_t s, e;
>
> +        //TODO: investigate reason why we need contiguous memory
> +        //and how it can affect relocation of Xen in over 4GB space
> +#ifndef ARM32_RELOCATE_OVER_4GB
>           /* We can only deal with contiguous memory at the moment */
>           if ( last_end != bank->start )
>               break;
> +#endif
>
>           last_end = bank->start + bank->size;
>
> @@ -407,7 +411,7 @@ static paddr_t __init get_xen_paddr(void)
>               if ( !e )
>                   continue;
>
> -#ifdef CONFIG_ARM_32
> +#if defined (CONFIG_ARM_32) && !defined(ARM32_RELOCATE_OVER_4GB)
>               /* Xen must be under 4GB */
>               if ( e > 0x100000000ULL )
>                   e = 0x100000000ULL;
> @@ -698,6 +702,9 @@ void __init setup_cache(void)
>       cacheline_bytes = 1U << (4 + (ccsid & 0x7));
>   }
>
> +#ifdef ARM32_RELOCATE_OVER_4GB
> +paddr_t xen_relocation_offset;
> +#endif
>   /* C entry point for boot CPU */
>   void __init start_xen(unsigned long boot_phys_offset,
>                         unsigned long fdt_paddr,
> @@ -739,8 +746,20 @@ void __init start_xen(unsigned long boot_phys_offset,
>                                (paddr_t)(uintptr_t)(_end - _start + 1), NULL);
>       BUG_ON(!xen_bootmodule);
>
> +#ifdef ARM32_RELOCATE_OVER_4GB
> +    //save physical address of init_secondary
> +    xen_relocation_offset = __pa(init_secondary);
> +#endif
>       xen_paddr = get_xen_paddr();
>       setup_pagetables(boot_phys_offset, xen_paddr);
> +#ifdef ARM32_RELOCATE_OVER_4GB
> +    //Now Xen is relocated
> +    //Calculate offset of Xen relocation
> +    //It is difference between new physical address of init_secondary an old one
> +    //This offset is needed in several places when we have to write to old Xen location
> +    //(secondary CPUs run on old-located Xen and rely on some variables from CPU0)
> +    xen_relocation_offset = __pa(init_secondary) - xen_relocation_offset;
> +#endif
>
>       /* Update Xen's address now that we have relocated. */
>       printk("Update BOOTMOD_XEN from %"PRIpaddr"-%"PRIpaddr" => %"PRIpaddr"-%"PRIpaddr"\n",
> diff --git a/xen/arch/arm/smpboot.c b/xen/arch/arm/smpboot.c
> index a96cda2..731144c 100644
> --- a/xen/arch/arm/smpboot.c
> +++ b/xen/arch/arm/smpboot.c
> @@ -31,6 +31,9 @@
>   #include <xen/console.h>
>   #include <asm/gic.h>
>   #include <asm/psci.h>
> +#ifdef ARM32_RELOCATE_OVER_4GB
> +#include <xen/vmap.h>
> +#endif
>
>   cpumask_t cpu_online_map;
>   cpumask_t cpu_present_map;
> @@ -353,17 +356,33 @@ int __init cpu_up_send_sgi(int cpu)
>       return 0;
>   }
>
> +#ifdef ARM32_RELOCATE_OVER_4GB
> +extern paddr_t xen_relocation_offset;
> +#endif
>   /* Bring up a remote CPU */
>   int __cpu_up(unsigned int cpu)
>   {
>       int rc;
>       s_time_t deadline;
> +#ifdef ARM32_RELOCATE_OVER_4GB
> +    paddr_t p_info = __pa(&smp_up_cpu) - xen_relocation_offset;
> +    unsigned long* info = ioremap_nocache(p_info, sizeof(unsigned long));
> +#else
> +    unsigned long* info = &smp_up_cpu;
> +#endif
>
>       printk("Bringing up CPU%d\n", cpu);
>
>       rc = init_secondary_pagetables(cpu);
>       if ( rc < 0 )
> +#ifdef ARM32_RELOCATE_OVER_4GB
> +    {
> +        iounmap(info);
> +        return rc;
> +    }
> +#else
>           return rc;
> +#endif
>
>       console_start_sync(); /* Secondary may use early_printk */
>
> @@ -374,8 +393,8 @@ int __cpu_up(unsigned int cpu)
>       init_data.cpuid = cpu;
>
>       /* Open the gate for this CPU */
> -    smp_up_cpu = cpu_logical_map(cpu);
> -    clean_dcache(smp_up_cpu);
> +    *info = cpu_logical_map(cpu);
> +    clean_dcache(*info);
>
>       rc = arch_cpu_up(cpu);
>
> @@ -383,6 +402,9 @@ int __cpu_up(unsigned int cpu)
>
>       if ( rc < 0 )
>       {
> +#ifdef ARM32_RELOCATE_OVER_4GB
> +        iounmap(info);
> +#endif
>           printk("Failed to bring up CPU%d\n", cpu);
>           return rc;
>       }
> @@ -406,8 +428,11 @@ int __cpu_up(unsigned int cpu)
>        */
>       init_data.stack = NULL;
>       init_data.cpuid = ~0;
> -    smp_up_cpu = MPIDR_INVALID;
> -    clean_dcache(smp_up_cpu);
> +    *info = MPIDR_INVALID;
> +    clean_dcache(*info);
> +#ifdef ARM32_RELOCATE_OVER_4GB
> +    iounmap(info);
> +#endif
>
>       if ( !cpu_online(cpu) )
>       {
>

[1] 
http://lists.xenproject.org/archives/html/xen-devel/2015-05/msg01031.html
[2] 
http://lists.xenproject.org/archives/html/xen-devel/2015-04/msg00662.html

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH RFC 15/18] arm: Add ability to relocate Xen in over 4GB space

[Julien Grall]

Hello,

On 18/05/16 17:32, Andrii Anisov wrote:
> diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c
> index 48f734f..7e507bc 100644
> --- a/xen/arch/arm/setup.c
> +++ b/xen/arch/arm/setup.c
> @@ -394,9 +394,13 @@ static paddr_t __init get_xen_paddr(void)
>           const struct membank *bank = &mi->bank[i];
>           paddr_t s, e;
>
> +        //TODO: investigate reason why we need contiguous memory
> +        //and how it can affect relocation of Xen in over 4GB space

Xen ARM was not able to support sparse memory before Xen 4.5. I.e only 
contiguous banks were supported. Xen had to be relocated on those banks 
to avoid issues when the init memory was released.

I believe this is not necessary anymore. Can you send this fix in a 
separate patch?

> +#ifndef ARM32_RELOCATE_OVER_4GB
>           /* We can only deal with contiguous memory at the moment */
>           if ( last_end != bank->start )
>               break;
> +#endif
>
>           last_end = bank->start + bank->size;
>

Regards,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[PATCH RFC 16/18] xen: Add dom0_mem_high option & over 4GB memory allocation for Dom0

[Andrii Anisov]

From: Iurii Mykhalskyi <iurii.mykhalskyi@globallogic.com>

Add support of custom allocation of over 4GB memory for Dom0. Requested
memory size might be specified by passing dom0_mem_high option in Xen
cmdline

Signed-off-by: Iurii Mykhalskyi <iurii.mykhalskyi@globallogic.com>
---
 xen/Rules.mk                |   1 +
 xen/arch/arm/domain_build.c | 189 +++++++++++++++++++++++++++++++++++++++++++-
 xen/arch/arm/kernel.h       |   9 +++
 3 files changed, 198 insertions(+), 1 deletion(-)

diff --git a/xen/Rules.mk b/xen/Rules.mk
index fbd34a5..51f7124 100644
--- a/xen/Rules.mk
+++ b/xen/Rules.mk
@@ -65,6 +65,7 @@ CFLAGS-$(HAS_IOPORTS)   += -DHAS_IOPORTS
 CFLAGS-$(HAS_PDX)       += -DHAS_PDX
 CFLAGS-$(frame_pointer) += -fno-omit-frame-pointer -DCONFIG_FRAME_POINTER
 CFLAGS-$(ARM32_RELOCATE_OVER_4GB) += -DARM32_RELOCATE_OVER_4GB
+CFLAGS-$(ARM32_SEPAR_MEM_SPLIT) += -DARM32_SEPAR_MEM_SPLIT
 
 ifneq ($(max_phys_cpus),)
 CFLAGS-y                += -DMAX_PHYS_CPUS=$(max_phys_cpus)
diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index f06792e..a63958b 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -35,6 +35,11 @@ int dom0_11_mapping = 1;
 #define DOM0_MEM_DEFAULT 0x8000000 /* 128 MiB */
 static u64 __initdata dom0_mem = DOM0_MEM_DEFAULT;
 
+#ifdef ARM32_SEPAR_MEM_SPLIT
+#define DOM0_MEM_HIGH_DEFAULT 0x0 /* 0 MiB */
+static u64 __initdata dom0_mem_high = DOM0_MEM_HIGH_DEFAULT;
+#endif
+
 static void __init parse_dom0_mem(const char *s)
 {
     dom0_mem = parse_size_and_unit(s, &s);
@@ -43,6 +48,14 @@ static void __init parse_dom0_mem(const char *s)
 }
 custom_param("dom0_mem", parse_dom0_mem);
 
+#ifdef ARM32_SEPAR_MEM_SPLIT
+static void __init parse_dom0_mem_high(const char *s)
+{
+    dom0_mem_high = parse_size_and_unit(s, &s);
+}
+custom_param("dom0_mem_high", parse_dom0_mem_high);
+#endif
+
 //#define DEBUG_DT
 
 #ifdef DEBUG_DT
@@ -130,7 +143,11 @@ static bool_t insert_11_bank(struct domain *d,
     if ( res )
         panic("Failed map pages to DOM0: %d", res);
 
+#ifndef ARM32_SEPAR_MEM_SPLIT
     kinfo->unassigned_mem -= size;
+#else
+    kinfo->unassigned_mem.low -= size;
+#endif
 
     if ( kinfo->mem.nr_banks == 0 )
     {
@@ -192,6 +209,82 @@ fail:
     return false;
 }
 
+#ifdef ARM32_SEPAR_MEM_SPLIT
+static void allocate_dom0_high_memory(struct domain *d, struct kernel_info *kinfo)
+{
+    int i, res, st_banks = kinfo->mem.nr_banks;
+    struct page_info *pg = NULL;
+    int bits;
+    unsigned int order = get_11_allocation_size(dom0_mem_high);
+    const unsigned int min_order = get_order_from_bytes(MB(4));
+    paddr_t spfn;
+    paddr_t start, size;
+    struct membank *bank = NULL;
+
+    if (dom0_mem_high == 0)
+        return;
+
+    printk("Allocating %ldMB of high memory region for dom0\n",
+            (unsigned long)(dom0_mem_high >> 20));
+
+    while ( kinfo->unassigned_mem.high && kinfo->mem.nr_banks < NR_MEM_BANKS )
+    {
+        for (bits = PADDR_BITS ; bits >= min_order; bits-- )
+        {
+            pg = alloc_domheap_pages(d, order, MEMF_bits(bits));
+            if ( pg != NULL )
+                break;
+        }
+
+        if ( !pg )
+        {
+            order --;
+            if ( order >= min_order )
+                continue;
+
+            /* No more we can do */
+            break;
+        }
+
+        spfn = page_to_mfn(pg);
+        start = pfn_to_paddr(spfn);
+        size = pfn_to_paddr((1 << order));
+
+        res = guest_physmap_add_page(d, spfn, spfn, order);
+        if ( res )
+            panic("Failed map pages to DOM0: %d", res);
+
+        kinfo->unassigned_mem.high -= size;
+
+        bank = &kinfo->mem.bank[kinfo->mem.nr_banks];
+
+        bank->start = start;
+        bank->size = size;
+        kinfo->mem.nr_banks++;
+
+        /*
+         * Success, next time around try again to get the largest order
+         * allocation possible.
+         */
+
+        order = get_11_allocation_size(kinfo->unassigned_mem.high);
+     }
+
+    if(kinfo->unassigned_mem.high)
+        panic("Unable to allocate high memory bank");
+
+    for( i = st_banks; i < kinfo->mem.nr_banks; i++ )
+    {
+        printk("BANK[%d] %#"PRIpaddr"-%#"PRIpaddr" (%ldMB)\n",
+               i,
+               kinfo->mem.bank[i].start,
+               kinfo->mem.bank[i].start + kinfo->mem.bank[i].size,
+               /* Don't want format this as PRIpaddr (16 digit hex) */
+               (unsigned long)(kinfo->mem.bank[i].size >> 20));
+    }
+}
+#endif
+
 /*
  * This is all pretty horrible.
  *
@@ -250,9 +343,13 @@ static void allocate_memory_11(struct domain *d, struct kernel_info *kinfo)
         get_11_allocation_size(min_t(paddr_t, dom0_mem, MB(128)));
     const unsigned int min_order = get_order_from_bytes(MB(4));
     struct page_info *pg;
-    unsigned int order = get_11_allocation_size(kinfo->unassigned_mem);
     u64 rambase_pfn = opt_dom0_rambase_pfn;
+#ifndef ARM32_SEPAR_MEM_SPLIT
+    unsigned int order = get_11_allocation_size(kinfo->unassigned_mem);
     paddr_t mem_size = kinfo->unassigned_mem;
+#else
+    unsigned int order = get_11_allocation_size(kinfo->unassigned_mem.low);
+#endif
     int i;
 
     bool_t lowmem = is_32bit_domain(d);
@@ -260,7 +357,11 @@ static void allocate_memory_11(struct domain *d, struct kernel_info *kinfo)
 
     printk("Allocating 1:1 mappings totalling %ldMB for dom0:\n",
            /* Don't want format this as PRIpaddr (16 digit hex) */
+#ifndef ARM32_SEPAR_MEM_SPLIT
            (unsigned long)(kinfo->unassigned_mem >> 20));
+#else
+           (unsigned long)(kinfo->unassigned_mem.low >> 20));
+#endif
 
     kinfo->mem.nr_banks = 0;
 
@@ -288,6 +389,7 @@ static void allocate_memory_11(struct domain *d, struct kernel_info *kinfo)
 
     /* Now allocate more memory and fill in additional banks */
 
+#ifndef ARM32_SEPAR_MEM_SPLIT
     order = get_11_allocation_size(kinfo->unassigned_mem);
     if ( opt_dom0_rambase_pfn )
         rambase_pfn += (mem_size - kinfo->unassigned_mem) >> PAGE_SHIFT;
@@ -353,7 +455,62 @@ static void allocate_memory_11(struct domain *d, struct kernel_info *kinfo)
                /* Don't want format this as PRIpaddr (16 digit hex) */
                " %ldMB unallocated\n",
                (unsigned long)kinfo->unassigned_mem >> 20);
+#else
+    order = get_11_allocation_size(kinfo->unassigned_mem.low);
+    while ( kinfo->unassigned_mem.low && kinfo->mem.nr_banks < NR_MEM_BANKS )
+    {
+        pg = alloc_domheap_pages(d, order, lowmem ? MEMF_bits(32) : 0);
+        if ( !pg )
+        {
+            order --;
+
+            if ( lowmem && order < min_low_order)
+            {
+                D11PRINT("Failed at min_low_order, allow high allocations\n");
+                order = get_11_allocation_size(kinfo->unassigned_mem.low);
+                lowmem = false;
+                continue;
+            }
+            if ( order >= min_order )
+                continue;
+
+            /* No more we can do */
+            break;
+        }
+
+        if ( !insert_11_bank(d, kinfo, pg, order) )
+        {
+            if ( kinfo->mem.nr_banks == NR_MEM_BANKS )
+                /* Nothing more we can do. */
+                break;
+
+            if ( lowmem )
+            {
+                D11PRINT("Allocation below bank 0, allow high allocations\n");
+                order = get_11_allocation_size(kinfo->unassigned_mem.low);
+                lowmem = false;
+                continue;
+            }
+            else
+            {
+                D11PRINT("Allocation below bank 0\n");
+                break;
+            }
+        }
+
+        /*
+         * Success, next time around try again to get the largest order
+         * allocation possible.
+         */
+        order = get_11_allocation_size(kinfo->unassigned_mem.low);
+    }
 
+    if ( kinfo->unassigned_mem.low )
+        printk("WARNING: Failed to allocate requested dom0 memory."
+               /* Don't want format this as PRIpaddr (16 digit hex) */
+               " %ldMB unallocated\n",
+               (unsigned long)kinfo->unassigned_mem.low >> 20);
+#endif
     for( i = 0; i < kinfo->mem.nr_banks; i++ )
     {
         printk("BANK[%d] %#"PRIpaddr"-%#"PRIpaddr" (%ldMB)\n",
@@ -374,7 +531,19 @@ static void allocate_memory(struct domain *d, struct kernel_info *kinfo)
     unsigned int bank = 0;
 
     if ( dom0_11_mapping )
+#ifndef ARM32_SEPAR_MEM_SPLIT
         return allocate_memory_11(d, kinfo);
+#else
+    {
+        allocate_memory_11(d, kinfo);
+        return allocate_dom0_high_memory(d, kinfo);
+    }
+#endif
+
+#ifdef ARM32_SEPAR_MEM_SPLIT
+    if (dom0_mem_high != 0)
+        printk("***WARNING*** We don't implement dom0_mem_high option for non 1:1 mapped domains!\n");
+#endif
 
     while ( (memory = dt_find_node_by_type(memory, "memory")) )
     {
@@ -389,7 +558,11 @@ static void allocate_memory(struct domain *d, struct kernel_info *kinfo)
             panic("Memory node has no reg property");
 
         for ( l = 0;
+#ifndef ARM32_SEPAR_MEM_SPLIT
               kinfo->unassigned_mem > 0 && l + reg_size <= reg_len
+#else
+              kinfo->unassigned_mem.low > 0 && l + reg_size <= reg_len
+#endif
                   && kinfo->mem.nr_banks < NR_MEM_BANKS;
               l += reg_size )
         {
@@ -399,8 +572,13 @@ static void allocate_memory(struct domain *d, struct kernel_info *kinfo)
                 panic("Unable to retrieve the bank %u for %s",
                       bank, dt_node_full_name(memory));
 
+#ifndef ARM32_SEPAR_MEM_SPLIT
             if ( size > kinfo->unassigned_mem )
                 size = kinfo->unassigned_mem;
+#else
+            if ( size > kinfo->unassigned_mem.low )
+                size = kinfo->unassigned_mem.low;
+#endif
 
             printk("Populate P2M %#"PRIx64"->%#"PRIx64"\n",
                    start, start + size);
@@ -410,7 +588,11 @@ static void allocate_memory(struct domain *d, struct kernel_info *kinfo)
             kinfo->mem.bank[kinfo->mem.nr_banks].size = size;
             kinfo->mem.nr_banks++;
 
+#ifndef ARM32_SEPAR_MEM_SPLIT
             kinfo->unassigned_mem -= size;
+#else
+            kinfo->unassigned_mem.low -= size;
+#endif
         }
     }
 }
@@ -1521,7 +1703,12 @@ int construct_dom0(struct domain *d)
 
     d->max_pages = ~0U;
 
+#ifndef ARM32_SEPAR_MEM_SPLIT
     kinfo.unassigned_mem = dom0_mem;
+#else
+    kinfo.unassigned_mem.low = dom0_mem;
+    kinfo.unassigned_mem.high = dom0_mem_high;
+#endif
 
     rc = kernel_probe(&kinfo);
     if ( rc < 0 )
diff --git a/xen/arch/arm/kernel.h b/xen/arch/arm/kernel.h
index c1b07d4..83fa0ed 100644
--- a/xen/arch/arm/kernel.h
+++ b/xen/arch/arm/kernel.h
@@ -16,7 +16,16 @@ struct kernel_info {
 #endif
 
     void *fdt; /* flat device tree */
+
+#ifndef ARM32_SEPAR_MEM_SPLIT
     paddr_t unassigned_mem; /* RAM not (yet) assigned to a bank */
+#else
+    struct {
+        paddr_t low; /* Low RAM not (yet) assigned to a bank */
+        paddr_t high; /* High RAM not (yet) assigned to a bank */
+    } unassigned_mem;
+#endif
+
     struct meminfo mem;
 
     /* kernel entry point */
-- 
2.8.2


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[PATCH RFC 17/18] tools: Introduce ARM32_SEPAR_MEM_SPLIT option

[Andrii Anisov]

From: Iurii Mykhalskyi <iurii.mykhalskyi@globallogic.com>

This option enables separate memory allocation in
low & over 4GB address space.
With this option enabled in domain config files
"memory" parameter are used to specify domain low memory
"memory_high" - to specify over 4GB allocated memory

If you didn't specify high memory with this option enabled,
domain memory will be limited to 4GB (zone 20)

Signed-off-by: Iurii Mykhalskyi <iurii.mykhalskyi@globallogic.com>
Signed-off-by: Iurii Konovalenko <iurii.konovalenko@globallogic.com>
---
 tools/libxc/Makefile                  |  2 ++
 tools/libxc/include/xc_dom.h          | 24 +++++++++++--
 tools/libxc/xc_dom_arm.c              | 66 +++++++++++++++++++++++++++++++++++
 tools/libxc/xc_dom_compat_linux.c     |  5 +++
 tools/libxc/xc_dom_core.c             | 51 ++++++++++++++++++++++-----
 tools/libxl/Makefile                  |  2 ++
 tools/libxl/libxl_arm.c               | 11 ++++++
 tools/libxl/libxl_dom.c               | 23 ++++++++++++
 tools/libxl/libxl_types.idl           |  1 +
 tools/libxl/xl_cmdimpl.c              |  5 +++
 tools/xenstore/init-xenstore-domain.c |  5 +++
 xen/arch/arm/domain_build.c           | 11 +++++-
 xen/common/memory.c                   | 16 ++++++++-
 xen/common/page_alloc.c               | 23 ++++++++++++
 xen/include/public/memory.h           |  6 ++++
 xen/include/xen/mm.h                  |  6 ++++
 16 files changed, 245 insertions(+), 12 deletions(-)

diff --git a/tools/libxc/Makefile b/tools/libxc/Makefile
index a0f899b..84d21b6 100644
--- a/tools/libxc/Makefile
+++ b/tools/libxc/Makefile
@@ -112,6 +112,8 @@ CFLAGS   += -I. -I./include $(CFLAGS_xeninclude)
 # Needed for posix_fadvise64() in xc_linux.c
 CFLAGS-$(CONFIG_Linux) += -D_GNU_SOURCE
 
+CFLAGS-$(ARM32_SEPAR_MEM_SPLIT) += -DARM32_SEPAR_MEM_SPLIT
+
 CFLAGS	+= $(PTHREAD_CFLAGS)
 
 CTRL_LIB_OBJS := $(patsubst %.c,%.o,$(CTRL_SRCS-y))
diff --git a/tools/libxc/include/xc_dom.h b/tools/libxc/include/xc_dom.h
index 600aef6..10bb6ab 100644
--- a/tools/libxc/include/xc_dom.h
+++ b/tools/libxc/include/xc_dom.h
@@ -129,12 +129,22 @@ struct xc_dom_image {
      * in rambase_pfn.
      */
     xen_pfn_t rambase_pfn;
+#ifndef ARM32_SEPAR_MEM_SPLIT
     xen_pfn_t total_pages;
+#else
+    xen_pfn_t low_mem_pages;
+    xen_pfn_t high_mem_pages;
+#endif
     xen_pfn_t p2m_size;         /* number of pfns covered by p2m */
     struct xc_dom_phys *phys_pages;
     int realmodearea_log;
 #if defined (__arm__) || defined(__aarch64__)
+#ifndef ARM32_SEPAR_MEM_SPLIT
     xen_pfn_t rambank_size[GUEST_RAM_BANKS];
+#else
+    xen_pfn_t rambank_size_low[GUEST_RAM_BANKS];
+    xen_pfn_t rambank_size_high[GUEST_RAM_BANKS];
+#endif
 #endif
 
     /* malloc memory pool */
@@ -181,6 +191,12 @@ struct xc_dom_image {
     int (*allocate) (struct xc_dom_image * dom, xen_vaddr_t up_to);
 };
 
+#ifndef ARM32_SEPAR_MEM_SPLIT
+#define XC_DOM_TOTAL_PAGES(dom) ((dom)->total_pages)
+#else
+#define XC_DOM_TOTAL_PAGES(dom) ((dom)->low_mem_pages + (dom)->high_mem_pages)
+#endif
+
 /* --- pluggable kernel loader ------------------------------------- */
 
 struct xc_dom_loader {
@@ -228,7 +244,11 @@ struct xc_dom_image *xc_dom_allocate(xc_interface *xch,
 void xc_dom_release_phys(struct xc_dom_image *dom);
 void xc_dom_release(struct xc_dom_image *dom);
 int xc_dom_rambase_init(struct xc_dom_image *dom, uint64_t rambase);
+#ifndef ARM32_SEPAR_MEM_SPLIT
 int xc_dom_mem_init(struct xc_dom_image *dom, unsigned int mem_mb);
+#else
+int xc_dom_mem_init(struct xc_dom_image *dom, unsigned int mem_mb_low, unsigned int mem_mb_high);
+#endif
 
 /* Set this larger if you have enormous ramdisks/kernels. Note that
  * you should trust all kernels not to be maliciously large (e.g. to
@@ -379,7 +399,7 @@ static inline xen_pfn_t xc_dom_p2m_host(struct xc_dom_image *dom, xen_pfn_t pfn)
 {
     if (dom->shadow_enabled)
         return pfn;
-    if (pfn < dom->rambase_pfn || pfn >= dom->rambase_pfn + dom->total_pages)
+    if (pfn < dom->rambase_pfn || pfn >= dom->rambase_pfn + XC_DOM_TOTAL_PAGES(dom))
         return INVALID_MFN;
     return dom->p2m_host[pfn - dom->rambase_pfn];
 }
@@ -389,7 +409,7 @@ static inline xen_pfn_t xc_dom_p2m_guest(struct xc_dom_image *dom,
 {
     if (xc_dom_feature_translated(dom))
         return pfn;
-    if (pfn < dom->rambase_pfn || pfn >= dom->rambase_pfn + dom->total_pages)
+    if (pfn < dom->rambase_pfn || pfn >= dom->rambase_pfn + XC_DOM_TOTAL_PAGES(dom))
         return INVALID_MFN;
     return dom->p2m_host[pfn - dom->rambase_pfn];
 }
diff --git a/tools/libxc/xc_dom_arm.c b/tools/libxc/xc_dom_arm.c
index 96df669..d995241 100644
--- a/tools/libxc/xc_dom_arm.c
+++ b/tools/libxc/xc_dom_arm.c
@@ -306,8 +306,19 @@ static int populate_one_size(struct xc_dom_image *dom, int pfn_shift,
     for ( i = 0 ; i < count ; i ++ )
         extents[i] = base_pfn + (i<<pfn_shift);
 
+#ifdef ARM32_SEPAR_MEM_SPLIT
+    if (base_pfn < 0x100000) { /* base_pfn less then 4GB */
+        nr = xc_domain_populate_physmap(dom->xch, dom->guest_domid, count,
+                                    pfn_shift, XENMEMF_only_low_mem, extents);
+    } else {
+        nr = xc_domain_populate_physmap(dom->xch, dom->guest_domid, count,
+                                    pfn_shift, XENMEMF_only_high_mem, extents);
+    }
+#else
     nr = xc_domain_populate_physmap(dom->xch, dom->guest_domid, count,
                                     pfn_shift, 0, extents);
+#endif
+
     if ( nr <= 0 ) return nr;
     DOMPRINTF("%s: populated %#x/%#x entries with shift %d",
               __FUNCTION__, nr, count, pfn_shift);
@@ -414,7 +425,11 @@ int arch_setup_meminit(struct xc_dom_image *dom)
     xen_pfn_t pfn;
     uint64_t modbase;
 
+#ifndef ARM32_SEPAR_MEM_SPLIT
     uint64_t ramsize = (uint64_t)dom->total_pages << XC_PAGE_SHIFT;
+#else
+    uint64_t ramsize = (uint64_t)dom->low_mem_pages << XC_PAGE_SHIFT;
+#endif
     uint64_t guest_rambase = (uint64_t)dom->rambase_pfn << XC_PAGE_SHIFT;
     uint64_t guest_ramsize = (GUEST_RAM0_BASE + GUEST_RAM0_SIZE) -
                               guest_rambase;
@@ -471,12 +486,41 @@ int arch_setup_meminit(struct xc_dom_image *dom)
 
         p2m_size = ( bankbase[i] + banksize - bankbase[0] ) >> XC_PAGE_SHIFT;
 
+#ifndef ARM32_SEPAR_MEM_SPLIT
         dom->rambank_size[i] = banksize >> XC_PAGE_SHIFT;
+#else
+        dom->rambank_size_low[i] = banksize >> XC_PAGE_SHIFT;
+#endif
     }
 
+#ifndef ARM32_SEPAR_MEM_SPLIT
     assert(dom->rambank_size[0] != 0);
+#else
+    assert(dom->rambank_size_low[0] != 0);
+#endif
     assert(ramsize == 0); /* Too much RAM is rejected above */
 
+
+#ifdef ARM32_SEPAR_MEM_SPLIT
+    ramsize = (uint64_t)dom->high_mem_pages << XC_PAGE_SHIFT;
+
+    if (ramsize) {
+        for ( i = 0; ramsize && i < GUEST_RAM_BANKS; i++ )
+        {
+            uint64_t banksize = ramsize > bankmax[i] ? bankmax[i] : ramsize;
+
+            ramsize -= banksize;
+            p2m_size += ( bankbase[1] + banksize - bankbase[1] ) >> XC_PAGE_SHIFT;
+
+            dom->rambank_size_high[i] = banksize >> XC_PAGE_SHIFT;
+            DOMPRINTF("%s: rambank_size_high: %"PRIx64, __FUNCTION__, dom->rambank_size_high[i]);
+        }
+
+        assert(ramsize == 0);
+        assert(dom->rambank_size_high[0] != 0);
+    }
+#endif
+
     dom->p2m_size = p2m_size;
     dom->p2m_host = xc_dom_malloc(dom, sizeof(xen_pfn_t) * p2m_size);
     if ( dom->p2m_host == NULL )
@@ -485,6 +529,7 @@ int arch_setup_meminit(struct xc_dom_image *dom)
         dom->p2m_host[pfn] = INVALID_P2M_ENTRY;
 
     /* setup initial p2m and allocate guest memory */
+#ifndef ARM32_SEPAR_MEM_SPLIT
     for ( i = 0; i < GUEST_RAM_BANKS && dom->rambank_size[i]; i++ )
     {
         if ((rc = populate_guest_memory(dom,
@@ -492,6 +537,23 @@ int arch_setup_meminit(struct xc_dom_image *dom)
                                         dom->rambank_size[i])))
             return rc;
     }
+#else
+    for ( i = 0; i < GUEST_RAM_BANKS && dom->rambank_size_low[i]; i++ )
+    {
+        if ((rc = populate_guest_memory(dom,
+                                        bankbase[i] >> XC_PAGE_SHIFT,
+                                        dom->rambank_size_low[i])))
+            return rc;
+    }
+
+    for ( i = 0; i < GUEST_RAM_BANKS && dom->rambank_size_high[i]; i++ )
+    {
+        if ((rc = populate_guest_memory(dom,
+                                        bankbase[1] >> XC_PAGE_SHIFT,
+                                        dom->rambank_size_high[i])))
+            return rc;
+    }
+#endif
 
     /*
      * We try to place dtb+initrd at 128MB or if we have less RAM
@@ -501,7 +563,11 @@ int arch_setup_meminit(struct xc_dom_image *dom)
      * If changing this then consider
      * xen/arch/arm/kernel.c:place_modules as well.
      */
+#ifndef ARM32_SEPAR_MEM_SPLIT
     bank0end = bankbase[0] + ((uint64_t)dom->rambank_size[0] << XC_PAGE_SHIFT);
+#else
+    bank0end = bankbase[0] + ((uint64_t)dom->rambank_size_low[0] << XC_PAGE_SHIFT);
+#endif
 
     if ( bank0end >= ram128mb + modsize && kernend < ram128mb )
         modbase = ram128mb;
diff --git a/tools/libxc/xc_dom_compat_linux.c b/tools/libxc/xc_dom_compat_linux.c
index a3abb99..9f35126 100644
--- a/tools/libxc/xc_dom_compat_linux.c
+++ b/tools/libxc/xc_dom_compat_linux.c
@@ -53,8 +53,13 @@ static int xc_linux_build_internal(struct xc_dom_image *dom,
         goto out;
     if ( (rc = xc_dom_parse_image(dom)) != 0 )
         goto out;
+#ifndef ARM32_SEPAR_MEM_SPLIT
     if ( (rc = xc_dom_mem_init(dom, mem_mb)) != 0 )
         goto out;
+#else
+    if ( (rc = xc_dom_mem_init(dom, mem_mb, 0)) != 0 )
+        goto out;
+#endif
     if ( (rc = xc_dom_boot_mem_init(dom)) != 0 )
         goto out;
     if ( (rc = xc_dom_build_image(dom)) != 0 )
diff --git a/tools/libxc/xc_dom_core.c b/tools/libxc/xc_dom_core.c
index 8466677..5911b1d 100644
--- a/tools/libxc/xc_dom_core.c
+++ b/tools/libxc/xc_dom_core.c
@@ -437,12 +437,12 @@ void *xc_dom_pfn_to_ptr_retcount(struct xc_dom_image *dom, xen_pfn_t pfn,
     *count_out = 0;
 
     offset = pfn - dom->rambase_pfn;
-    if ( offset > dom->total_pages || /* multiple checks to avoid overflows */
-         count > dom->total_pages ||
-         offset > dom->total_pages - count )
+    if ( offset > XC_DOM_TOTAL_PAGES(dom) || /* multiple checks to avoid overflows */
+         count > XC_DOM_TOTAL_PAGES(dom) ||
+         offset > XC_DOM_TOTAL_PAGES(dom) - count )
     {
         DOMPRINTF("%s: pfn %"PRI_xen_pfn" out of range (0x%" PRIpfn " > 0x%" PRIpfn ")",
-                  __FUNCTION__, pfn, offset, dom->total_pages);
+                  __FUNCTION__, pfn, offset, XC_DOM_TOTAL_PAGES(dom));
         return NULL;
     }
 
@@ -566,14 +566,14 @@ int xc_dom_alloc_segment(struct xc_dom_image *dom,
     pfn = (seg->vstart - dom->parms.virt_base) / page_size;
     seg->pfn = pfn + dom->rambase_pfn;
 
-    if ( pages > dom->total_pages || /* multiple test avoids overflow probs */
-         pfn > dom->total_pages ||
-         pages > dom->total_pages - pfn)
+    if ( pages > XC_DOM_TOTAL_PAGES(dom) || /* multiple test avoids overflow probs */
+         pfn > XC_DOM_TOTAL_PAGES(dom) ||
+         pages > XC_DOM_TOTAL_PAGES(dom) - pfn)
     {
         xc_dom_panic(dom->xch, XC_OUT_OF_MEMORY,
                      "%s: segment %s too large (0x%"PRIpfn" > "
                      "0x%"PRIpfn" - 0x%"PRIpfn" pages)",
-                     __FUNCTION__, name, pages, dom->total_pages, pfn);
+                     __FUNCTION__, name, pages, XC_DOM_TOTAL_PAGES(dom), pfn);
         return -1;
     }
 
@@ -891,6 +891,7 @@ int xc_dom_rambase_init(struct xc_dom_image *dom, uint64_t rambase)
     return 0;
 }
 
+#ifndef ARM32_SEPAR_MEM_SPLIT
 int xc_dom_mem_init(struct xc_dom_image *dom, unsigned int mem_mb)
 {
     unsigned int page_shift;
@@ -916,6 +917,40 @@ int xc_dom_mem_init(struct xc_dom_image *dom, unsigned int mem_mb)
 
     return 0;
 }
+#else
+int xc_dom_mem_init(struct xc_dom_image *dom, unsigned int mem_mb_low, unsigned int mem_mb_high)
+{
+    unsigned int page_shift;
+    xen_pfn_t nr_pages_low;
+    xen_pfn_t nr_pages_high;
+
+    dom->arch_hooks = xc_dom_find_arch_hooks(dom->xch, dom->guest_type);
+    if ( dom->arch_hooks == NULL )
+    {
+        xc_dom_panic(dom->xch, XC_INTERNAL_ERROR, "%s: arch hooks not set",
+                     __FUNCTION__);
+        return -1;
+    }
+
+    page_shift = XC_DOM_PAGE_SHIFT(dom);
+    nr_pages_low = mem_mb_low << (20 - page_shift);
+    nr_pages_high = mem_mb_high << (20 - page_shift);
+
+    DOMPRINTF("%s: mem %d MB, low pages 0x%" PRIpfn " pages, %dk each",
+               __FUNCTION__, mem_mb_low, nr_pages_low, 1 << (page_shift-10));
+
+    DOMPRINTF("%s: mem %d MB, high pages 0x%" PRIpfn " pages, %dk each",
+               __FUNCTION__, mem_mb_high, nr_pages_high, 1 << (page_shift-10));
+
+    dom->low_mem_pages = nr_pages_low;
+    dom->high_mem_pages = nr_pages_high;
+
+    DOMPRINTF("%s: Total pages 0x%" PRIpfn,
+              __FUNCTION__, XC_DOM_TOTAL_PAGES(dom));
+
+    return 0;
+}
+#endif
 
 int xc_dom_update_guest_p2m(struct xc_dom_image *dom)
 {
diff --git a/tools/libxl/Makefile b/tools/libxl/Makefile
index c5ecec1..7cf4985 100644
--- a/tools/libxl/Makefile
+++ b/tools/libxl/Makefile
@@ -36,6 +36,8 @@ CFLAGS_LIBXL += -Wshadow
 
 LIBXL_LIBS-$(CONFIG_ARM) += -lfdt
 
+CFLAGS-$(ARM32_SEPAR_MEM_SPLIT) += -DARM32_SEPAR_MEM_SPLIT
+
 CFLAGS += $(PTHREAD_CFLAGS)
 LDFLAGS += $(PTHREAD_LDFLAGS)
 LIBXL_LIBS += $(PTHREAD_LIBS)
diff --git a/tools/libxl/libxl_arm.c b/tools/libxl/libxl_arm.c
index 7f9547b..340ecf9 100644
--- a/tools/libxl/libxl_arm.c
+++ b/tools/libxl/libxl_arm.c
@@ -916,7 +916,11 @@ int libxl__arch_domain_finalise_hw_description(libxl__gc *gc,
                                                struct xc_dom_image *dom)
 {
     void *fdt = dom->devicetree_blob;
+#ifndef ARM32_SEPAR_MEM_SPLIT
     int i;
+#else
+    uint64_t size;
+#endif
 
     uint64_t guest_rambase = (uint64_t)dom->rambase_pfn << XC_PAGE_SHIFT;
 
@@ -953,11 +957,18 @@ int libxl__arch_domain_finalise_hw_description(libxl__gc *gc,
 
     }
 
+#ifndef ARM32_SEPAR_MEM_SPLIT
     for (i = 0; i < GUEST_RAM_BANKS; i++) {
         const uint64_t size = (uint64_t)dom->rambank_size[i] << XC_PAGE_SHIFT;
 
         finalise_one_memory_node(gc, fdt, bankbase[i], size);
     }
+#else
+    size = (uint64_t)dom->rambank_size_low[0] << XC_PAGE_SHIFT;
+    finalise_one_memory_node(gc, fdt, bankbase[0], size);
+    size = (uint64_t)dom->rambank_size_high[0] << XC_PAGE_SHIFT;
+    finalise_one_memory_node(gc, fdt, bankbase[1], size);
+#endif
 
     debug_dump_fdt(gc, fdt);
 
diff --git a/tools/libxl/libxl_dom.c b/tools/libxl/libxl_dom.c
index be6598f..09bb0fa 100644
--- a/tools/libxl/libxl_dom.c
+++ b/tools/libxl/libxl_dom.c
@@ -402,11 +402,19 @@ int libxl__build_pre(libxl__gc *gc, uint32_t domid,
         }
     }
 
+#ifndef ARM32_SEPAR_MEM_SPLIT
     if (xc_domain_setmaxmem(ctx->xch, domid, info->target_memkb +
         LIBXL_MAXMEM_CONSTANT) < 0) {
         LIBXL__LOG_ERRNO(ctx, LIBXL__LOG_ERROR, "Couldn't set max memory");
         return ERROR_FAIL;
     }
+#else
+    if (xc_domain_setmaxmem(ctx->xch, domid, info->target_memkb + info->target_memkb_high +
+        LIBXL_MAXMEM_CONSTANT) < 0) {
+        LIBXL__LOG_ERRNO(ctx, LIBXL__LOG_ERROR, "Couldn't set max memory");
+        return ERROR_FAIL;
+    }
+#endif
 
     xs_domid = xs_read(ctx->xsh, XBT_NULL, "/tool/xenstored/domid", NULL);
     state->store_domid = xs_domid ? atoi(xs_domid) : 0;
@@ -525,11 +533,19 @@ int libxl__build_post(libxl__gc *gc, uint32_t domid,
         ? LIBXL_MAXMEM_CONSTANT : 0;
 
     ents = libxl__calloc(gc, 12 + (info->max_vcpus * 2) + 2, sizeof(char *));
+#ifndef ARM32_SEPAR_MEM_SPLIT
     ents[0] = "memory/static-max";
     ents[1] = GCSPRINTF("%"PRId64, info->max_memkb);
     ents[2] = "memory/target";
     ents[3] = GCSPRINTF("%"PRId64, info->target_memkb - info->video_memkb
                         - mem_target_fudge);
+#else
+    ents[0] = "memory/static-max";
+    ents[1] = GCSPRINTF("%"PRId64, info->max_memkb + info->target_memkb_high);
+    ents[2] = "memory/target";
+    ents[3] = GCSPRINTF("%"PRId64, info->target_memkb + info->target_memkb_high - info->video_memkb
+                        - mem_target_fudge);
+#endif
     ents[4] = "memory/videoram";
     ents[5] = GCSPRINTF("%"PRId64, info->video_memkb);
     ents[6] = "domid";
@@ -717,10 +733,17 @@ int libxl__build_pv(libxl__gc *gc, uint32_t domid,
         LOGE(ERROR, "libxl__arch_domain_init_hw_description failed");
         goto out;
     }
+#ifndef ARM32_SEPAR_MEM_SPLIT
     if ( (ret = xc_dom_mem_init(dom, info->target_memkb / 1024)) != 0 ) {
         LOGE(ERROR, "xc_dom_mem_init failed");
         goto out;
     }
+#else
+    if ( (ret = xc_dom_mem_init(dom, info->target_memkb / 1024, info->target_memkb_high / 1024)) != 0 ) {
+        LOGE(ERROR, "xc_dom_mem_init failed");
+        goto out;
+    }
+#endif
     if ( (ret = xc_dom_boot_mem_init(dom)) != 0 ) {
         LOGE(ERROR, "xc_dom_boot_mem_init failed");
         goto out;
diff --git a/tools/libxl/libxl_types.idl b/tools/libxl/libxl_types.idl
index b6cc8d2..cd12117 100644
--- a/tools/libxl/libxl_types.idl
+++ b/tools/libxl/libxl_types.idl
@@ -414,6 +414,7 @@ libxl_domain_build_info = Struct("domain_build_info",[
     ("tsc_mode",        libxl_tsc_mode),
     ("max_memkb",       MemKB),
     ("target_memkb",    MemKB),
+    ("target_memkb_high",  MemKB, {'init_val': '0'}),
     ("video_memkb",     MemKB),
     ("shadow_memkb",    MemKB),
     ("rambase",         uint64, {'init_val': 'LIBXL_INVALID_RAM_BASE'}),
diff --git a/tools/libxl/xl_cmdimpl.c b/tools/libxl/xl_cmdimpl.c
index 98a46bc..63a6b17 100644
--- a/tools/libxl/xl_cmdimpl.c
+++ b/tools/libxl/xl_cmdimpl.c
@@ -1350,6 +1350,11 @@ static void parse_config_data(const char *config_source,
     if (!xlu_cfg_get_long (config, "memory", &l, 0))
         b_info->target_memkb = l * 1024;
 
+#ifdef ARM32_SEPAR_MEM_SPLIT
+    if (!xlu_cfg_get_long (config, "memory_high", &l, 0))
+        b_info->target_memkb_high = l * 1024;
+#endif
+
     if (!xlu_cfg_get_long (config, "maxmem", &l, 0))
         b_info->max_memkb = l * 1024;
 
diff --git a/tools/xenstore/init-xenstore-domain.c b/tools/xenstore/init-xenstore-domain.c
index 0d12169..c759800 100644
--- a/tools/xenstore/init-xenstore-domain.c
+++ b/tools/xenstore/init-xenstore-domain.c
@@ -54,8 +54,13 @@ static int build(xc_interface *xch, int argc, char** argv)
 	if (rv) goto err;
 	rv = xc_dom_parse_image(dom);
 	if (rv) goto err;
+#ifndef ARM32_SEPAR_MEM_SPLIT
 	rv = xc_dom_mem_init(dom, maxmem);
 	if (rv) goto err;
+#else
+	rv = xc_dom_mem_init(dom, maxmem, 0);
+	if (rv) goto err;
+#endif
 	rv = xc_dom_boot_mem_init(dom);
 	if (rv) goto err;
 	rv = xc_dom_build_image(dom);
diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index a63958b..0678ae7 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -349,6 +349,7 @@ static void allocate_memory_11(struct domain *d, struct kernel_info *kinfo)
     paddr_t mem_size = kinfo->unassigned_mem;
 #else
     unsigned int order = get_11_allocation_size(kinfo->unassigned_mem.low);
+    paddr_t mem_size = kinfo->unassigned_mem.low;
 #endif
     int i;
 
@@ -457,9 +458,12 @@ static void allocate_memory_11(struct domain *d, struct kernel_info *kinfo)
                (unsigned long)kinfo->unassigned_mem >> 20);
 #else
     order = get_11_allocation_size(kinfo->unassigned_mem.low);
+    if ( opt_dom0_rambase_pfn )
+        rambase_pfn += (mem_size - kinfo->unassigned_mem.low) >> PAGE_SHIFT;
+
     while ( kinfo->unassigned_mem.low && kinfo->mem.nr_banks < NR_MEM_BANKS )
     {
-        pg = alloc_domheap_pages(d, order, lowmem ? MEMF_bits(32) : 0);
+        pg = alloc_domheap_pages_pfn(d, order, lowmem ? MEMF_bits(32) : 0, rambase_pfn);
         if ( !pg )
         {
             order --;
@@ -503,6 +507,11 @@ static void allocate_memory_11(struct domain *d, struct kernel_info *kinfo)
          * allocation possible.
          */
         order = get_11_allocation_size(kinfo->unassigned_mem.low);
+        if ( opt_dom0_rambase_pfn )
+        {
+            rambase_pfn += (mem_size - kinfo->unassigned_mem.low) >> PAGE_SHIFT;
+            mem_size = kinfo->unassigned_mem.low;
+        }
     }
 
     if ( kinfo->unassigned_mem.low )
diff --git a/xen/common/memory.c b/xen/common/memory.c
index 8baed72..54d993a 100644
--- a/xen/common/memory.c
+++ b/xen/common/memory.c
@@ -169,7 +169,11 @@ static void populate_physmap(struct memop_args *a)
         }
         else
         {
-            if ( is_domain_direct_mapped(d) )
+            if ( is_domain_direct_mapped(d)
+#ifdef ARM32_SEPAR_MEM_SPLIT
+                && !(a->memflags & MEMF_only_high_mem)
+#endif
+				)
             {
                 mfn = gpfn;
 
@@ -871,6 +875,16 @@ long do_memory_op(unsigned long cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
              && (reservation.mem_flags & XENMEMF_populate_on_demand) )
             args.memflags |= MEMF_populate_on_demand;
 
+#ifdef ARM32_SEPAR_MEM_SPLIT
+        if ( op == XENMEM_populate_physmap
+                && ( reservation.mem_flags & XENMEMF_only_low_mem) )
+            args.memflags |= MEMF_only_low_mem;
+
+        if ( op == XENMEM_populate_physmap
+                && ( reservation.mem_flags & XENMEMF_only_high_mem) )
+            args.memflags |= MEMF_only_high_mem;
+#endif
+
         if ( xsm_memory_adjust_reservation(XSM_TARGET, current->domain, d) )
         {
             rcu_unlock_domain(d);
diff --git a/xen/common/page_alloc.c b/xen/common/page_alloc.c
index d0c0fbb..437efbd 100644
--- a/xen/common/page_alloc.c
+++ b/xen/common/page_alloc.c
@@ -1718,6 +1718,9 @@ int assign_pages(
     return -1;
 }
 
+#ifdef ARM32_SEPAR_MEM_SPLIT
+#define ZONE_4G (20) /* flsl(4G) - PAGE_SHIFT */
+#endif
 
 struct page_info *alloc_domheap_pages_pfn(
     struct domain *d, unsigned int order, unsigned int memflags, xen_pfn_t pfn)
@@ -1739,11 +1742,31 @@ struct page_info *alloc_domheap_pages_pfn(
     if ( dma_bitsize && ((dma_zone = bits_to_zone(dma_bitsize)) < zone_hi) )
         pg = alloc_heap_pages_pfn(dma_zone + 1, zone_hi, order, memflags, d, pfn);
 
+#ifdef ARM32_SEPAR_MEM_SPLIT
+    if ( pg == NULL) {
+        unsigned int low = MEMZONE_XEN + 1;
+        unsigned int high = zone_hi;
+
+        if ( memflags & MEMF_no_dma )
+            return NULL;
+
+       if ( memflags & MEMF_only_low_mem )
+           high = ZONE_4G;
+
+       if ( memflags & MEMF_only_high_mem )
+           low = ZONE_4G;
+
+        if ( (pg = alloc_heap_pages_pfn(low, high, order,
+                                   memflags, d, pfn)) == NULL )
+            return NULL;
+    }
+#else
     if ( (pg == NULL) &&
          ((memflags & MEMF_no_dma) ||
           ((pg = alloc_heap_pages_pfn(MEMZONE_XEN + 1, zone_hi, order,
                                   memflags, d, pfn)) == NULL)) )
          return NULL;
+#endif
 
     if ( d && !(memflags & MEMF_no_owner) &&
          assign_pages(d, pg, order, memflags) )
diff --git a/xen/include/public/memory.h b/xen/include/public/memory.h
index dfc5171..ad7c55d 100644
--- a/xen/include/public/memory.h
+++ b/xen/include/public/memory.h
@@ -51,6 +51,12 @@
 /* NUMA node to allocate from. */
 #define XENMEMF_node(x)     (((x) + 1) << 8)
 #define XENMEMF_get_node(x) ((((x) >> 8) - 1) & 0xffu)
+#ifdef ARM32_SEPAR_MEM_SPLIT
+/* Flag to populate physmap only from low 4GB memory space */
+#define XENMEMF_only_low_mem (1<<6)
+/* Flag to populate physmap only from over 4GB memory space */
+#define XENMEMF_only_high_mem (1<<7)
+#endif
 /* Flag to populate physmap with populate-on-demand entries */
 #define XENMEMF_populate_on_demand (1<<16)
 /* Flag to request allocation only from the node specified */
diff --git a/xen/include/xen/mm.h b/xen/include/xen/mm.h
index 83e4913..a74c7d4 100644
--- a/xen/include/xen/mm.h
+++ b/xen/include/xen/mm.h
@@ -178,6 +178,12 @@ struct npfec {
 #define _MEMF_exact_node  4
 #define  MEMF_exact_node  (1U<<_MEMF_exact_node)
 #define _MEMF_no_owner    5
+#ifdef ARM32_SEPAR_MEM_SPLIT
+#define _MEMF_only_low_mem    6
+#define  MEMF_only_low_mem    (1U<<_MEMF_only_low_mem)
+#define _MEMF_only_high_mem   7
+#define  MEMF_only_high_mem   (1U<<_MEMF_only_high_mem)
+#endif
 #define  MEMF_no_owner    (1U<<_MEMF_no_owner)
 #define _MEMF_node        8
 #define  MEMF_node_mask   ((1U << (8 * sizeof(nodeid_t))) - 1)
-- 
2.8.2


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH RFC 17/18] tools: Introduce ARM32_SEPAR_MEM_SPLIT option

[Wei Liu]

On Wed, May 18, 2016 at 07:32:40PM +0300, Andrii Anisov wrote:
> From: Iurii Mykhalskyi <iurii.mykhalskyi@globallogic.com>
> 
> This option enables separate memory allocation in
> low & over 4GB address space.
> With this option enabled in domain config files
> "memory" parameter are used to specify domain low memory
> "memory_high" - to specify over 4GB allocated memory
> 
> If you didn't specify high memory with this option enabled,
> domain memory will be limited to 4GB (zone 20)
> 

I don't think such option is future proof enough -- what if you suddenly
want several memory regions?

I haven't actually looked at the code, merely asking some (perhaps dumb)
questions first.

Wei.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[PATCH RFC 18/18] arm: Add ability to allocate Xen heap in lowmem

[Andrii Anisov]

From: Iurii Konovalenko <iurii.konovalenko@globallogic.com>

Add abiliyty to allocate Xen heap in 32-bit address range for ARM32.
Due to architecture features some ARM32 platforms require Xen heap
to be allocated in lowmem.

Signed-off-by: Iurii Konovalenko <iurii.konovalenko@globallogic.com>
---
 xen/Rules.mk         |  1 +
 xen/arch/arm/setup.c | 14 ++++++++++++++
 2 files changed, 15 insertions(+)

diff --git a/xen/Rules.mk b/xen/Rules.mk
index 51f7124..30f5227 100644
--- a/xen/Rules.mk
+++ b/xen/Rules.mk
@@ -66,6 +66,7 @@ CFLAGS-$(HAS_PDX)       += -DHAS_PDX
 CFLAGS-$(frame_pointer) += -fno-omit-frame-pointer -DCONFIG_FRAME_POINTER
 CFLAGS-$(ARM32_RELOCATE_OVER_4GB) += -DARM32_RELOCATE_OVER_4GB
 CFLAGS-$(ARM32_SEPAR_MEM_SPLIT) += -DARM32_SEPAR_MEM_SPLIT
+CFLAGS-$(ARM32_XENHEAP_IN_LOWMEM) += -DARM32_XENHEAP_IN_LOWMEM
 
 ifneq ($(max_phys_cpus),)
 CFLAGS-y                += -DMAX_PHYS_CPUS=$(max_phys_cpus)
diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c
index 7e507bc..5510a34 100644
--- a/xen/arch/arm/setup.c
+++ b/xen/arch/arm/setup.c
@@ -475,7 +475,11 @@ static void init_pdx(void)
 #ifdef CONFIG_ARM_32
 static void __init setup_mm(unsigned long dtb_paddr, size_t dtb_size)
 {
+#ifdef ARM32_XENHEAP_IN_LOWMEM
+    paddr_t ram_start, ram_end, ram_size, dma32_end;
+#else
     paddr_t ram_start, ram_end, ram_size;
+#endif
     paddr_t s, e;
     unsigned long ram_pages;
     unsigned long heap_pages, xenheap_pages, domheap_pages;
@@ -492,6 +496,9 @@ static void __init setup_mm(unsigned long dtb_paddr, size_t dtb_size)
     ram_start = bootinfo.mem.bank[0].start;
     ram_size  = bootinfo.mem.bank[0].size;
     ram_end   = ram_start + ram_size;
+#ifdef ARM32_XENHEAP_IN_LOWMEM
+    dma32_end = ram_end > 0x100000000ULL ? 0 : ram_end;
+#endif
 
     for ( i = 1; i < bootinfo.mem.nr_banks; i++ )
     {
@@ -502,6 +509,9 @@ static void __init setup_mm(unsigned long dtb_paddr, size_t dtb_size)
         ram_size  = ram_size + bank_size;
         ram_start = min(ram_start,bank_start);
         ram_end   = max(ram_end,bank_end);
+#ifdef ARM32_XENHEAP_IN_LOWMEM
+        dma32_end = bank_end > 0x100000000ULL ? dma32_end : bank_end;
+#endif
     }
 
     total_pages = ram_pages = ram_size >> PAGE_SHIFT;
@@ -530,7 +540,11 @@ static void __init setup_mm(unsigned long dtb_paddr, size_t dtb_size)
 
     do
     {
+#ifdef ARM32_XENHEAP_IN_LOWMEM
+        e = consider_modules(ram_start, dma32_end,
+#else
         e = consider_modules(ram_start, ram_end,
+#endif
                              pfn_to_paddr(xenheap_pages),
                              32<<20, 0);
         if ( e )
-- 
2.8.2


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH RFC 00/18] System adjustment to customer needs.

[Julien Grall]

Hello Andrii,

Thank you for your series. For the next version, can you CC the relevant 
maintainers using scripts/get_maintainer.pl for each patch?

Regards,

On 18/05/2016 17:32, Andrii Anisov wrote:
> This series RFC patches from the currently ongoing production project.
> This patch series presents changes needed to fit the system into
> customer requirements as well as workaround limitations of the
> Jacinto6 SoC.
> This patch series is not a full production branch upstream. Lacks of
> the board specific patches as well as new PV drivers interfaces.
> Patches for new PV drivers interfaces will be submitted soon.
>
> Andrii Anisov (1):
>   xen/tools: Fix virtual disks helper scripts.
>
> Andrii Tseglytskyi (2):
>   xen/arm: allow to allocate 1/128/256/512 Mb memory chunks
>   xen/arm: allow reassigning of hw interrupts to guest domain
>
> Iurii Konovalenko (3):
>   arm: Fix 1-to-1 Dom0 memory allocation of any size
>   arm: Add ability to relocate Xen in over 4GB space
>   arm: Add ability to allocate Xen heap in lowmem
>
> Iurii Mykhalskyi (3):
>   xen: flask: Add possiblity to forward irqs into domU domains
>   xen: Add dom0_mem_high option & over 4GB memory allocation for Dom0
>   tools: Introduce ARM32_SEPAR_MEM_SPLIT option
>
> Oleksandr Dmytryshyn (3):
>   tools: Allow to cross-compile xentop
>   xen: arm: add batch support to the XENMEM_p2m_lookup operation
>   xen: introduce and use 'dom0_rambase_pfn' setting for kernel Dom0
>
> Oleksandr Tyshchenko (1):
>   libxl: add ability to set rambase_pfn via cfg file
>
> Sergiy Kibrik (1):
>   kbdif: add raw events passing
>
> Viktor Kleinik (4):
>   libxl: parse config data during domain reboot
>   tools/misc: Modify Xen watchdog daemon
>   tools/misc: Set timeout value from watchdog daemon
>   libxl: Fix unneeded domain reboot during destroy routine
>
>  tools/flask/policy/policy/modules/xen/xen.te |   1 +
>  tools/hotplug/Linux/block                    |   2 +-
>  tools/hotplug/Linux/locking.sh               |   9 +-
>  tools/hotplug/Linux/xen-hotplug-common.sh.in |   2 +-
>  tools/libxc/Makefile                         |   2 +
>  tools/libxc/include/xc_dom.h                 |  24 ++-
>  tools/libxc/xc_dom_arm.c                     | 103 +++++++++++-
>  tools/libxc/xc_dom_compat_linux.c            |   5 +
>  tools/libxc/xc_dom_core.c                    |  51 +++++-
>  tools/libxl/Makefile                         |   2 +
>  tools/libxl/libxl.c                          |   4 +
>  tools/libxl/libxl_arm.c                      |  21 ++-
>  tools/libxl/libxl_create.c                   |   5 +
>  tools/libxl/libxl_dom.c                      |  29 +++-
>  tools/libxl/libxl_internal.h                 |   1 +
>  tools/libxl/libxl_types.idl                  |   2 +
>  tools/libxl/xl_cmdimpl.c                     |  33 ++++
>  tools/misc/xenwatchdogd.c                    |  54 ++-----
>  tools/xenstat/Makefile                       |   4 +-
>  tools/xenstore/init-xenstore-domain.c        |   5 +
>  xen/Rules.mk                                 |   3 +
>  xen/arch/arm/arm32/head.S                    |  21 ++-
>  xen/arch/arm/domain_build.c                  | 224 ++++++++++++++++++++++++++-
>  xen/arch/arm/irq.c                           |  19 ++-
>  xen/arch/arm/kernel.h                        |   9 ++
>  xen/arch/arm/mm.c                            |  33 ++++
>  xen/arch/arm/platforms/omap5.c               |  17 +-
>  xen/arch/arm/platforms/rcar2.c               |   9 +-
>  xen/arch/arm/setup.c                         |  35 ++++-
>  xen/arch/arm/smpboot.c                       |  33 +++-
>  xen/common/memory.c                          |  16 +-
>  xen/common/page_alloc.c                      |  91 +++++++++--
>  xen/include/public/io/kbdif.h                |  10 ++
>  xen/include/public/memory.h                  |  18 ++-
>  xen/include/xen/mm.h                         |   8 +
>  35 files changed, 801 insertions(+), 104 deletions(-)
>

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH RFC 00/18] System adjustment to customer needs.

[Andrii Anisov]

Julien,

> Thank you for your series. For the next version, can you CC the relevant maintainers using scripts/get_maintainer.pl for each patch?

We will take care to follow this rule further.

Andrii Anisov | Associate Manager, Engineering
GlobalLogic
Kyiv, 03038, Protasov Business Park, M.Grinchenka, 2/1
P +38.044.492.9695x3664  M +380505738852  S andriyanisov
www.globallogic.com

http://www.globallogic.com/email_disclaimer.txt

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Embedded-pv-devel] [PATCH RFC 00/18] System adjustment to customer needs.

[Meng Xu]

Hi Andrii,

On Wed, May 18, 2016 at 12:32 PM, Andrii Anisov
<andrii.anisov@globallogic.com> wrote:
> This series RFC patches from the currently ongoing production project.
> This patch series presents changes needed to fit the system into
> customer requirements as well as workaround limitations of the
> Jacinto6 SoC.

IMHO, it will be better, if possible, to describe the exact customer
requirements this patch series tries to satisfy. I'm curious at what
the requirements are and if the requirements are general enough for
many other customers. :-)

Similarly, what are the limitations for the Jacinto6 SoC that need to
be workaround? If the board is not supported by Xen, can we say Xen
will support the board with the warkaround?


Thanks and Best Regards,

Meng
-----------
Meng Xu
PhD Student in Computer and Information Science
University of Pennsylvania
http://www.cis.upenn.edu/~mengxu/

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Embedded-pv-devel] [PATCH RFC 00/18] System adjustment to customer needs.

[Julien Grall]

Hello,

On 18/05/16 20:17, Meng Xu wrote:
> On Wed, May 18, 2016 at 12:32 PM, Andrii Anisov
> <andrii.anisov@globallogic.com> wrote:
>> This series RFC patches from the currently ongoing production project.
>> This patch series presents changes needed to fit the system into
>> customer requirements as well as workaround limitations of the
>> Jacinto6 SoC.
>
> IMHO, it will be better, if possible, to describe the exact customer
> requirements this patch series tries to satisfy. I'm curious at what
> the requirements are and if the requirements are general enough for
> many other customers. :-)

I agree with Meng here. We need to understand the use case and see if it 
is possible to generalize it for everyone.

I looked quickly at this patch series and noticed that most of patches 
miss details on why it is necessary and what you are trying solve. Can 
you give us more details?

> Similarly, what are the limitations for the Jacinto6 SoC that need to
> be workaround? If the board is not supported by Xen, can we say Xen
> will support the board with the warkaround?

Regards,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Embedded-pv-devel] [PATCH RFC 00/18] System adjustment to customer needs.

[Andrii Anisov]

[-- Attachment #1.1: Type: text/plain, Size: 2711 bytes --]

All,

See the system details I can reveal below:

   - There are two OS in the system Linux(Dom0) and Android(DomU)
   - Android provides almost all infotainment functionality. Linux covers
   functionality with higher reliability requirements and backup in case if
   Android crashes/glitches.
   - Linux (Dom0) handles all HW except GPU.
   - Android (DomD) runs with number of PV drivers, has an exclusive access
   to GPU.
   - The system has 2Gb(-16MB due to errata) RAM under 4GB, and a memory
   bank mapped over 4GB
   - Due to relatively small amount of dma-able memory both domains should
   have assigned RAM both from under 4GB and over 4GB spaces.
   - Several "data flow" mixing scenarios should be provided on Linux side
   I.e. controlling Android audio level, Android audio mute, mixing Android
   audio stream from stream generated by Linux. Same for Android displaying,
   events passing.
   - Domains should share HW codecs.

>> Similarly, what are the limitations for the Jacinto6 SoC that need to
>> be workaround?

The main limitation is that Jacinto6 lacks of SMMU. All transaction
initiators can address 32bit only, some initiators have no MMU and do not
support sg-lists (require buffers continuous in maddr).

Andrii Anisov | Associate Manager, Engineering
GlobalLogic
Kyiv, 03038, Protasov Business Park, M.Grinchenka, 2/1
P +38.044.492.9695x3664  M +380505738852  S andriyanisov
www.globallogic.com

http://www.globallogic.com/email_disclaimer.txt


On Thu, May 19, 2016 at 2:00 PM, Julien Grall <julien.grall@arm.com> wrote:
> Hello,
>
> On 18/05/16 20:17, Meng Xu wrote:
>>
>> On Wed, May 18, 2016 at 12:32 PM, Andrii Anisov
>> <andrii.anisov@globallogic.com> wrote:
>>>
>>> This series RFC patches from the currently ongoing production project.
>>> This patch series presents changes needed to fit the system into
>>> customer requirements as well as workaround limitations of the
>>> Jacinto6 SoC.
>>
>>
>> IMHO, it will be better, if possible, to describe the exact customer
>> requirements this patch series tries to satisfy. I'm curious at what
>> the requirements are and if the requirements are general enough for
>> many other customers. :-)
>
>
> I agree with Meng here. We need to understand the use case and see if it
is
> possible to generalize it for everyone.
>
> I looked quickly at this patch series and noticed that most of patches
miss
> details on why it is necessary and what you are trying solve. Can you give
> us more details?
>
>> Similarly, what are the limitations for the Jacinto6 SoC that need to
>> be workaround? If the board is not supported by Xen, can we say Xen
>> will support the board with the warkaround?
>
>
> Regards,
>
> --
> Julien Grall

[-- Attachment #1.2: Type: text/html, Size: 3594 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Embedded-pv-devel] [PATCH RFC 00/18] System adjustment to customer needs.

[Julien Grall]

Hello Andrii,

Thank you for sharing details on your use case.

On 19/05/16 22:28, Andrii Anisov wrote:
> See the system details I can reveal below:
>
>   * There are two OS in the system Linux(Dom0) and Android(DomU)
>   * Android provides almost all infotainment functionality. Linux covers
>     functionality with higher reliability requirements and backup in
>     case if Android crashes/glitches.

If a malicious user has access to the Android guest (via USB key, 
wifi,...) he would be able to crash the platform using the GPU because 
there is no SMMU protection.

So can you expand what are your expectations for reliability?

>   * Linux (Dom0) handles all HW except GPU.
>   * Android (DomD) runs with number of PV drivers, has an exclusive
>     access to GPU.
>   * The system has 2Gb(-16MB due to errata) RAM under 4GB, and a memory
>     bank mapped over 4GB
>   * Due to relatively small amount of dma-able memory both domains
>     should have assigned RAM both from under 4GB and over 4GB spaces.
>   * Several "data flow" mixing scenarios should be provided on Linux
>     side I.e. controlling Android audio level, Android audio mute,
>     mixing Android audio stream from stream generated by Linux. Same for
>     Android displaying, events passing.
>   * Domains should share HW codecs.
>
>  >> Similarly, what are the limitations for the Jacinto6 SoC that need to
>  >> be workaround?
>
> The main limitation is that Jacinto6 lacks of SMMU. All transaction
> initiators can address 32bit only, some initiators have no MMU and do
> not support sg-lists (require buffers continuous in maddr).

If I understand correctly, all the initiators but the GPU will be used 
by DOM0 which is already direct mapped. The only issue here is 
allocating memory enough memory below 4GB.

By default Xen will try to allocate as much RAM as possible below 4GB 
for DOM0. If you are concerned about the amount allocated, you could 
pre-allocate them before hand (such as via the DT as propose in [1]).

For the GPU, on another reply you said it was protected by an IPMMU. I 
remembered a series from globallogic to virtualize it in Xen [2]. Can 
you details why this option would not fit in your use case?

Regards,

[1] 
http://lists.xenproject.org/archives/html/xen-devel/2016-05/msg01879.html
[2] 
http://lists.xenproject.org/archives/html/xen-devel/2014-06/msg03359.html

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Embedded-pv-devel] [PATCH RFC 00/18] System adjustment to customer needs.

[Andrii Anisov]

> If a malicious user has access to the Android guest (via USB key, wifi,...)
> he would be able to crash the platform using the GPU because there is no
> SMMU protection.
That's why we are shadowing GPU MMU translation tables in xen heap.
And this is leaded us to need of [PATCH RFC 18/18] arm: Add ability to
allocate Xen heap in lowmem.
Patche for GPU MMU shadowing is not published yet, I still have to
check if it lacks of proprietary information.

Andrii Anisov | Associate Manager, Engineering
GlobalLogic
Kyiv, 03038, Protasov Business Park, M.Grinchenka, 2/1
P +38.044.492.9695x3664  M +380505738852  S andriyanisov
www.globallogic.com

http://www.globallogic.com/email_disclaimer.txt

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Embedded-pv-devel] [PATCH RFC 00/18] System adjustment to customer needs.

[Julien Grall]

Hello Andrii,

On 20/05/16 17:24, Andrii Anisov wrote:
>> If a malicious user has access to the Android guest (via USB key, wifi,...)
>> he would be able to crash the platform using the GPU because there is no
>> SMMU protection.
> That's why we are shadowing GPU MMU translation tables in xen heap.
> And this is leaded us to need of [PATCH RFC 18/18] arm: Add ability to
> allocate Xen heap in lowmem.
> Patche for GPU MMU shadowing is not published yet, I still have to
> check if it lacks of proprietary information.

I would rather delay any review on patches required (such as patch #18) 
for GPU MMU shadowing until you post the new series.

Regards,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Embedded-pv-devel] [PATCH RFC 00/18] System adjustment to customer needs.

[Andrii Anisov]

> If I understand correctly, all the initiators but the GPU will be used by
> DOM0 which is already direct mapped. The only issue here is allocating
> memory enough memory below 4GB.
It's not about memory allocation for domain. It is rather about SDRAM
mapping on the bus. J6 has first 2GB for iomems, starting from
0x80000000 2GB of SDRAM mapping plus more SDRAM mapped over 4GB line.
Changing rambase_pfn is painful for J6 release kernel so we just make
it configurable by
    - [PATCH RFC 04/18] libxl: add ability to set rambase_pfn via cfg file.
    - [PATCH RFC 13/18] xen: introduce and use 'dom0_rambase_pfn'
setting for kernel Dom0


> By default Xen will try to allocate as much RAM as possible below 4GB for
> DOM0.

Also we are providing both domains with a piece of non-dma memory for
applications with a patch [PATCH RFC 16/18] xen: Add dom0_mem_high
option & over 4GB memory allocation for Dom0.

> If you are concerned about the amount allocated, you could pre-allocate them before
> hand (such as via the DT as propose in [1]).
I've quickly checked the proposal, looks interesting we would evaluate.

> For the GPU, on another reply you said it was protected by an IPMMU. I
> remembered a series from globallogic to virtualize it in Xen [2]. Can you
> details why this option would not fit in your use case?
> [2]
> http://lists.xenproject.org/archives/html/xen-devel/2014-06/msg03359.html
We have a successor of that patch series. But the GPU MMU is still
32-bit and is not able to address SDRAM mapped over 4GB on the bus.

Andrii Anisov | Associate Manager, Engineering
GlobalLogic
Kyiv, 03038, Protasov Business Park, M.Grinchenka, 2/1
P +38.044.492.9695x3664  M +380505738852  S andriyanisov
www.globallogic.com

http://www.globallogic.com/email_disclaimer.txt

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Embedded-pv-devel] [PATCH RFC 00/18] System adjustment to customer needs.

[Julien Grall]

Hello Andrii,

On 20/05/16 18:09, Andrii Anisov wrote:
>> If I understand correctly, all the initiators but the GPU will be used by
>> DOM0 which is already direct mapped. The only issue here is allocating
>> memory enough memory below 4GB.
> It's not about memory allocation for domain. It is rather about SDRAM
> mapping on the bus. J6 has first 2GB for iomems, starting from
> 0x80000000 2GB of SDRAM mapping plus more SDRAM mapped over 4GB line.
> Changing rambase_pfn is painful for J6 release kernel so we just make
> it configurable by
>      - [PATCH RFC 04/18] libxl: add ability to set rambase_pfn via cfg file.
>      - [PATCH RFC 13/18] xen: introduce and use 'dom0_rambase_pfn'
> setting for kernel Dom0

I am not sure to understand why you need to change the rambase_pfn. Is 
it because the kernel is using hardcoded address rather than using the 
device tree?

Regards,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Embedded-pv-devel] [PATCH RFC 00/18] System adjustment to customer needs.

[Andrii Anisov]

Julien,

>>We need to understand the use case and see if it is possible to generalize
>>it for everyone.

Well, the thing I would generalize is an understanding that real chips
(automotive IVI, mobile) have no or has limited SMMU functionality.

For a limited SMMU functionality I would name Renesas RCAR H2 chip -
it has its own IPMMU instead of SMMU. It is claimed that IPMMU
implemets VMSAv7. But no VMSAv7 VE are supported by that IPMMU in
fact.

The most outstanding example is a chip with Cortex A15 MPCore, but
without any VE support at all.

Andrii Anisov | Associate Manager, Engineering
GlobalLogic
Kyiv, 03038, Protasov Business Park, M.Grinchenka, 2/1
P +38.044.492.9695x3664  M +380505738852  S andriyanisov
www.globallogic.com

http://www.globallogic.com/email_disclaimer.txt

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Embedded-pv-devel] [PATCH RFC 00/18] System adjustment to customer needs.

[Andrii Anisov]

Meng,

>> If the board is not supported by Xen, can we say Xen will support the
>> board with the warkaround?

I would not say boards are supported by XEN (except earlyprintk).
Rather architectures are supported in general, and SoC's are supported
in architecture implementation defined deviations (i.e. SMMU absence).

Andrii Anisov | Associate Manager, Engineering
GlobalLogic
Kyiv, 03038, Protasov Business Park, M.Grinchenka, 2/1
P +38.044.492.9695x3664  M +380505738852  S andriyanisov
www.globallogic.com

http://www.globallogic.com/email_disclaimer.txt

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Embedded-pv-devel] [PATCH RFC 00/18] System adjustment to customer needs.

[Meng Xu]

On Thu, May 19, 2016 at 5:53 PM, Andrii Anisov
<andrii.anisov@globallogic.com> wrote:
> Meng,
>

Hi Andrii,

Thank you very much for your explanation about the use case in previous email!

>>> If the board is not supported by Xen, can we say Xen will support the
>>> board with the warkaround?
>
> I would not say boards are supported by XEN (except earlyprintk).
> Rather architectures are supported in general, and SoC's are supported
> in architecture implementation defined deviations (i.e. SMMU absence).

Yes. I searched around for the "Jacinto 6" Automotive processor.[1]
It uses Cortex A15 processor...
However, I tried the Arndale Octo board two years ago
(http://www.arndaleboard.org/wiki/index.php/Main_Page). From my
previous experience, the board may not be supported by Xen even though
the processor it uses has virtualization extension.. :-(
That's why I asked if the board itself can run Xen. If the board can
run Xen, I would like to buy one and try it out. :-)

[1] http://www.ti.com/lit/ds/symlink/dra746.pdf

Thanks and Best Regards,

Meng
-----------
Meng Xu
PhD Student in Computer and Information Science
University of Pennsylvania
http://www.cis.upenn.edu/~mengxu/

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Embedded-pv-devel] [PATCH RFC 00/18] System adjustment to customer needs.

[Andrii Anisov]

Meng,

> From my previous experience, the board may not be supported by Xen even though
> the processor it uses has virtualization extension.. :-(

I still insist it depends on SoC ;)

If you are saying about this case:

> Yes. I searched around for the "Jacinto 6" Automotive processor.[1]
> It uses Cortex A15 processor...
> However, I tried the Arndale Octo board two years ago
> (http://www.arndaleboard.org/wiki/index.php/Main_Page).

Cortex A15 just implements ARMv7 architecture more efficiently then
A9, while VE are still optional. You should read chip specs more
careful.
You remember I wrote:
>> The most outstanding example is a chip with Cortex A15 MPCore, but
>> without any VE support at all.
It was another Samsung's SoC in my experience.

Andrii Anisov | Associate Manager, Engineering
GlobalLogic
Kyiv, 03038, Protasov Business Park, M.Grinchenka, 2/1
P +38.044.492.9695x3664  M +380505738852  S andriyanisov
www.globallogic.com

http://www.globallogic.com/email_disclaimer.txt

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Embedded-pv-devel] [PATCH RFC 00/18] System adjustment to customer needs.

[Meng Xu]

On Fri, May 20, 2016 at 1:23 PM, Andrii Anisov
<andrii.anisov@globallogic.com> wrote:
> Meng,
>
>> From my previous experience, the board may not be supported by Xen even though
>> the processor it uses has virtualization extension.. :-(
>
> I still insist it depends on SoC ;)

Ah-ha, I totally agree about this...

>
> If you are saying about this case:
>
>> Yes. I searched around for the "Jacinto 6" Automotive processor.[1]
>> It uses Cortex A15 processor...
>> However, I tried the Arndale Octo board two years ago
>> (http://www.arndaleboard.org/wiki/index.php/Main_Page).
>
> Cortex A15 just implements ARMv7 architecture more efficiently then
> A9, while VE are still optional. You should read chip specs more
> careful.
> You remember I wrote:
>>> The most outstanding example is a chip with Cortex A15 MPCore, but
>>> without any VE support at all.
> It was another Samsung's SoC in my experience.

Yes. I just started looking at the ARM now and tried to evaluate the
RTDS scheduler on ARM. That's why I'm very interested in the use case
here and tried to run it. :-)

I will keep an eye on your patches. ;-)

Thanks and Best Regards,

Meng
-----------
Meng Xu
PhD Student in Computer and Information Science
University of Pennsylvania
http://www.cis.upenn.edu/~mengxu/

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Embedded-pv-devel] [PATCH RFC 00/18] System adjustment to customer needs.

[Julien Grall]

Hello Meng,

On 20/05/2016 16:21, Meng Xu wrote:
> On Thu, May 19, 2016 at 5:53 PM, Andrii Anisov
> <andrii.anisov@globallogic.com> wrote:
>>>> If the board is not supported by Xen, can we say Xen will support the
>>>> board with the warkaround?
>>
>> I would not say boards are supported by XEN (except earlyprintk).
>> Rather architectures are supported in general, and SoC's are supported
>> in architecture implementation defined deviations (i.e. SMMU absence).
>
> Yes. I searched around for the "Jacinto 6" Automotive processor.[1]
> It uses Cortex A15 processor...
> However, I tried the Arndale Octo board two years ago
> (http://www.arndaleboard.org/wiki/index.php/Main_Page). From my
> previous experience, the board may not be supported by Xen even though
> the processor it uses has virtualization extension.. :-(

It would like to correct you here. Xen (and KVM) can run on any board 
providing virtual extensions, a GIC and an architectural timer are 
available.

However, some vendor ship boards with a firmware/bootloader that will 
enter the kernel in EL1 rather than EL2 (i.e the hyp mode).

The bootloader is often U-boot and the upstream version may support the 
board. So if the bootloader is entered in a mode that allows to switch 
to HYP mode (either EL3 or secure EL1), then you can recompile U-boot 
and flash your board. Otherwise, the source may be shipped and you can 
hack around to entered Xen to HYP mode.

Regards,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Embedded-pv-devel] [PATCH RFC 00/18] System adjustment to customer needs.

[Meng Xu]

On Sat, May 21, 2016 at 10:32 AM, Julien Grall <julien.grall@arm.com> wrote:
> Hello Meng,

Hi Julien,

>
> On 20/05/2016 16:21, Meng Xu wrote:
>>
>> On Thu, May 19, 2016 at 5:53 PM, Andrii Anisov
>> <andrii.anisov@globallogic.com> wrote:
>>>>>
>>>>> If the board is not supported by Xen, can we say Xen will support the
>>>>> board with the warkaround?
>>>
>>>
>>> I would not say boards are supported by XEN (except earlyprintk).
>>> Rather architectures are supported in general, and SoC's are supported
>>> in architecture implementation defined deviations (i.e. SMMU absence).
>>
>>
>> Yes. I searched around for the "Jacinto 6" Automotive processor.[1]
>> It uses Cortex A15 processor...
>> However, I tried the Arndale Octo board two years ago
>> (http://www.arndaleboard.org/wiki/index.php/Main_Page). From my
>> previous experience, the board may not be supported by Xen even though
>> the processor it uses has virtualization extension.. :-(
>
>
> It would like to correct you here.

Thank you very much for the correction! :-)

> Xen (and KVM) can run on any board
> providing virtual extensions, a GIC and an architectural timer are
> available.
>

I see.

> However, some vendor ship boards with a firmware/bootloader that will enter
> the kernel in EL1 rather than EL2 (i.e the hyp mode).

Right! That's exactly what we faced two years ago. :-(

>
> The bootloader is often U-boot and the upstream version may support the
> board. So if the bootloader is entered in a mode that allows to switch to
> HYP mode (either EL3 or secure EL1), then you can recompile U-boot and flash
> your board. Otherwise, the source may be shipped and you can hack around to
> entered Xen to HYP mode.

Yes. Sometimes, the board needs the signature for the uboot, that's
why we couldn't just compile the source without the manufacturer's
signature two years ago...

Thank you very much for clarifying this. It makes the statement more
precisely. :-)

Best Regards,

Meng
-----------
Meng Xu
PhD Student in Computer and Information Science
University of Pennsylvania
http://www.cis.upenn.edu/~mengxu/

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel