[PATCH v5 00/17] xen: arm: 64-bit guest support and domU FDT autogeneration

[PATCH v5 00/17] xen: arm: 64-bit guest support and domU FDT autogeneration

[Ian Campbell]

The big change this time is that I actually implemented the review
comments on "xen: arm: allocate dom0 memory separately from preparing
the dtb".

This has resulted in a new precursor patch which moves the GIC and timer
nodes under a new toplevel "xen" node. The memory node is now added to
this parent as well. THis way we get to control the #address-cells etc.

[PATCH v5 01/19] HACK

[Ian Campbell]

---
 xen/include/public/arch-arm.h |   11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)

diff --git a/xen/include/public/arch-arm.h b/xen/include/public/arch-arm.h
index 746df8e..7d452d8 100644
--- a/xen/include/public/arch-arm.h
+++ b/xen/include/public/arch-arm.h
@@ -218,7 +218,16 @@ typedef uint64_t xen_callback_t;
 #define PSR_MODE_SYS 0x1f
 
 /* 64 bit modes */
-#ifdef CONFIG_ARM_64
+#ifdef __aarch64__
+#undef PSR_MODE_BIT
+#undef PSR_MODE_EL3h
+#undef PSR_MODE_EL3t
+#undef PSR_MODE_EL2h
+#undef PSR_MODE_EL2t
+#undef PSR_MODE_EL1h
+#undef PSR_MODE_EL1t
+#undef PSR_MODE_EL0t
+
 #define PSR_MODE_BIT  0x10 /* Set iff AArch32 */
 #define PSR_MODE_EL3h 0x0d
 #define PSR_MODE_EL3t 0x0c
-- 
1.7.10.4

Re: [PATCH v5 01/19] HACK

[Julien Grall]

I guess it's not part of the series ? :)

On 11/13/2013 06:11 PM, Ian Campbell wrote:
> ---
>   xen/include/public/arch-arm.h |   11 ++++++++++-
>   1 file changed, 10 insertions(+), 1 deletion(-)
>
> diff --git a/xen/include/public/arch-arm.h b/xen/include/public/arch-arm.h
> index 746df8e..7d452d8 100644
> --- a/xen/include/public/arch-arm.h
> +++ b/xen/include/public/arch-arm.h
> @@ -218,7 +218,16 @@ typedef uint64_t xen_callback_t;
>   #define PSR_MODE_SYS 0x1f
>
>   /* 64 bit modes */
> -#ifdef CONFIG_ARM_64
> +#ifdef __aarch64__
> +#undef PSR_MODE_BIT
> +#undef PSR_MODE_EL3h
> +#undef PSR_MODE_EL3t
> +#undef PSR_MODE_EL2h
> +#undef PSR_MODE_EL2t
> +#undef PSR_MODE_EL1h
> +#undef PSR_MODE_EL1t
> +#undef PSR_MODE_EL0t
> +
>   #define PSR_MODE_BIT  0x10 /* Set iff AArch32 */
>   #define PSR_MODE_EL3h 0x0d
>   #define PSR_MODE_EL3t 0x0c
>

-- 
Julien Grall

Re: [PATCH v5 01/19] HACK

[Ian Campbell]

On Wed, 2013-11-13 at 18:15 +0000, Julien Grall wrote:
> I guess it's not part of the series ? :)

I'm not having much luck with git send-email recently, am I!

This is a workaround for a bug in the aarch64 glibc which leaks these
symbols when including standard headers. I've filed a bug in launchpad
ages ago...

> 
> On 11/13/2013 06:11 PM, Ian Campbell wrote:
> > ---
> >   xen/include/public/arch-arm.h |   11 ++++++++++-
> >   1 file changed, 10 insertions(+), 1 deletion(-)
> >
> > diff --git a/xen/include/public/arch-arm.h b/xen/include/public/arch-arm.h
> > index 746df8e..7d452d8 100644
> > --- a/xen/include/public/arch-arm.h
> > +++ b/xen/include/public/arch-arm.h
> > @@ -218,7 +218,16 @@ typedef uint64_t xen_callback_t;
> >   #define PSR_MODE_SYS 0x1f
> >
> >   /* 64 bit modes */
> > -#ifdef CONFIG_ARM_64
> > +#ifdef __aarch64__
> > +#undef PSR_MODE_BIT
> > +#undef PSR_MODE_EL3h
> > +#undef PSR_MODE_EL3t
> > +#undef PSR_MODE_EL2h
> > +#undef PSR_MODE_EL2t
> > +#undef PSR_MODE_EL1h
> > +#undef PSR_MODE_EL1t
> > +#undef PSR_MODE_EL0t
> > +
> >   #define PSR_MODE_BIT  0x10 /* Set iff AArch32 */
> >   #define PSR_MODE_EL3h 0x0d
> >   #define PSR_MODE_EL3t 0x0c
> >
> 

[PATCH v5 02/19] xen: arm: drop LDFLAGS_DIRECT emulation specification.

[Ian Campbell]

The current -maarch64elf fails when cross-building arm64 on Ubuntu Raring due
to a missing file "ldscripts/aarch64elf.xr". This is undoubtedly an Ubuntu gcc
bug, hwever when investigating I found that this option was not necessary at
all since we provide an explicit linker script when linking the hypervisor
(AFAICT all -m<foo> does is override the default linker script).

LDFLAGS_DIRECT is also used when linking the intermediate built-in.o files but
-m<emulatin> is not needed for this since it isn't linking the final image and
we are calling the linker with the correct, cross if necessary, name.

However it does appear to be potentially useful to supply -EL in both cases to
ensure that we get little endian images. (I just happened to spot that Linux
does this, for both arm and arm64, although I expect we are unlikely to trip
over such toolchains these days).

Tested with cross-builds of arm32 and arm64 as well as a native arm32 build.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
---
 config/arm32.mk |    5 +----
 config/arm64.mk |    2 +-
 2 files changed, 2 insertions(+), 5 deletions(-)

diff --git a/config/arm32.mk b/config/arm32.mk
index f64f0c1..683531c 100644
--- a/config/arm32.mk
+++ b/config/arm32.mk
@@ -10,9 +10,6 @@ CFLAGS += -marm
 HAS_PL011 := y
 
 # Use only if calling $(LD) directly.
-#LDFLAGS_DIRECT_OpenBSD = _obsd
-#LDFLAGS_DIRECT_FreeBSD = _fbsd
-LDFLAGS_DIRECT_Linux = _linux
-LDFLAGS_DIRECT += -marmelf$(LDFLAGS_DIRECT_$(XEN_OS))_eabi
+LDFLAGS_DIRECT += -EL
 
 CONFIG_LOAD_ADDRESS ?= 0x80000000
diff --git a/config/arm64.mk b/config/arm64.mk
index b2457eb..55b16da 100644
--- a/config/arm64.mk
+++ b/config/arm64.mk
@@ -7,6 +7,6 @@ CFLAGS += #-marm -march= -mcpu= etc
 HAS_PL011 := y
 
 # Use only if calling $(LD) directly.
-LDFLAGS_DIRECT += -maarch64elf
+LDFLAGS_DIRECT += -EL
 
 CONFIG_LOAD_ADDRESS ?= 0x80000000
-- 
1.7.10.4

Re: [PATCH v5 02/19] xen: arm: drop LDFLAGS_DIRECT emulation specification.

[Julien Grall]

Already upstream :)

On 11/13/2013 06:11 PM, Ian Campbell wrote:
> The current -maarch64elf fails when cross-building arm64 on Ubuntu Raring due
> to a missing file "ldscripts/aarch64elf.xr". This is undoubtedly an Ubuntu gcc
> bug, hwever when investigating I found that this option was not necessary at
> all since we provide an explicit linker script when linking the hypervisor
> (AFAICT all -m<foo> does is override the default linker script).
>
> LDFLAGS_DIRECT is also used when linking the intermediate built-in.o files but
> -m<emulatin> is not needed for this since it isn't linking the final image and
> we are calling the linker with the correct, cross if necessary, name.
>
> However it does appear to be potentially useful to supply -EL in both cases to
> ensure that we get little endian images. (I just happened to spot that Linux
> does this, for both arm and arm64, although I expect we are unlikely to trip
> over such toolchains these days).
>
> Tested with cross-builds of arm32 and arm64 as well as a native arm32 build.
>
> Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
> ---
>   config/arm32.mk |    5 +----
>   config/arm64.mk |    2 +-
>   2 files changed, 2 insertions(+), 5 deletions(-)
>
> diff --git a/config/arm32.mk b/config/arm32.mk
> index f64f0c1..683531c 100644
> --- a/config/arm32.mk
> +++ b/config/arm32.mk
> @@ -10,9 +10,6 @@ CFLAGS += -marm
>   HAS_PL011 := y
>
>   # Use only if calling $(LD) directly.
> -#LDFLAGS_DIRECT_OpenBSD = _obsd
> -#LDFLAGS_DIRECT_FreeBSD = _fbsd
> -LDFLAGS_DIRECT_Linux = _linux
> -LDFLAGS_DIRECT += -marmelf$(LDFLAGS_DIRECT_$(XEN_OS))_eabi
> +LDFLAGS_DIRECT += -EL
>
>   CONFIG_LOAD_ADDRESS ?= 0x80000000
> diff --git a/config/arm64.mk b/config/arm64.mk
> index b2457eb..55b16da 100644
> --- a/config/arm64.mk
> +++ b/config/arm64.mk
> @@ -7,6 +7,6 @@ CFLAGS += #-marm -march= -mcpu= etc
>   HAS_PL011 := y
>
>   # Use only if calling $(LD) directly.
> -LDFLAGS_DIRECT += -maarch64elf
> +LDFLAGS_DIRECT += -EL
>
>   CONFIG_LOAD_ADDRESS ?= 0x80000000
>

-- 
Julien Grall

Re: [PATCH v5 02/19] xen: arm: drop LDFLAGS_DIRECT emulation specification.

[Ian Campbell]

On Wed, 2013-11-13 at 21:26 +0000, Julien Grall wrote:
> Already upstream :)

Same cross-hack branch as "HACK", somehow merged into this one. Oops!

[PATCH v5 03/19] xen: update config.{sub, guess} for arm64

[Ian Campbell]

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
---
 config.guess |  406 +++++++++++++++++++++++++++++++++-------------------------
 config.sub   |  293 ++++++++++++++++++++++++++----------------
 2 files changed, 416 insertions(+), 283 deletions(-)

diff --git a/config.guess b/config.guess
index c2246a4..b79252d 100755
--- a/config.guess
+++ b/config.guess
@@ -1,14 +1,12 @@
 #! /bin/sh
 # Attempt to guess a canonical system name.
-#   Copyright (C) 1992, 1993, 1994, 1995, 1996, 1997, 1998, 1999,
-#   2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010
-#   Free Software Foundation, Inc.
+#   Copyright 1992-2013 Free Software Foundation, Inc.
 
-timestamp='2009-12-30'
+timestamp='2013-06-10'
 
 # This file is free software; you can redistribute it and/or modify it
 # under the terms of the GNU General Public License as published by
-# the Free Software Foundation; either version 2 of the License, or
+# the Free Software Foundation; either version 3 of the License, or
 # (at your option) any later version.
 #
 # This program is distributed in the hope that it will be useful, but
@@ -17,26 +15,22 @@ timestamp='2009-12-30'
 # General Public License for more details.
 #
 # You should have received a copy of the GNU General Public License
-# along with this program; if not, write to the Free Software
-# Foundation, Inc., 51 Franklin Street - Fifth Floor, Boston, MA
-# 02110-1301, USA.
+# along with this program; if not, see <http://www.gnu.org/licenses/>.
 #
 # As a special exception to the GNU General Public License, if you
 # distribute this file as part of a program that contains a
 # configuration script generated by Autoconf, you may include it under
-# the same distribution terms that you use for the rest of that program.
-
-
-# Originally written by Per Bothner.  Please send patches (context
-# diff format) to <config-patches@gnu.org> and include a ChangeLog
-# entry.
+# the same distribution terms that you use for the rest of that
+# program.  This Exception is an additional permission under section 7
+# of the GNU General Public License, version 3 ("GPLv3").
 #
-# This script attempts to guess a canonical system name similar to
-# config.sub.  If it succeeds, it prints the system name on stdout, and
-# exits with 0.  Otherwise, it exits with 1.
+# Originally written by Per Bothner.
 #
 # You can get the latest version of this script from:
 # http://git.savannah.gnu.org/gitweb/?p=config.git;a=blob_plain;f=config.guess;hb=HEAD
+#
+# Please send patches with a ChangeLog entry to config-patches@gnu.org.
+
 
 me=`echo "$0" | sed -e 's,.*/,,'`
 
@@ -56,9 +50,7 @@ version="\
 GNU config.guess ($timestamp)
 
 Originally written by Per Bothner.
-Copyright (C) 1992, 1993, 1994, 1995, 1996, 1997, 1998, 1999, 2000,
-2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010 Free
-Software Foundation, Inc.
+Copyright 1992-2013 Free Software Foundation, Inc.
 
 This is free software; see the source for copying conditions.  There is NO
 warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE."
@@ -140,12 +132,33 @@ UNAME_RELEASE=`(uname -r) 2>/dev/null` || UNAME_RELEASE=unknown
 UNAME_SYSTEM=`(uname -s) 2>/dev/null`  || UNAME_SYSTEM=unknown
 UNAME_VERSION=`(uname -v) 2>/dev/null` || UNAME_VERSION=unknown
 
+case "${UNAME_SYSTEM}" in
+Linux|GNU|GNU/*)
+	# If the system lacks a compiler, then just pick glibc.
+	# We could probably try harder.
+	LIBC=gnu
+
+	eval $set_cc_for_build
+	cat <<-EOF > $dummy.c
+	#include <features.h>
+	#if defined(__UCLIBC__)
+	LIBC=uclibc
+	#elif defined(__dietlibc__)
+	LIBC=dietlibc
+	#else
+	LIBC=gnu
+	#endif
+	EOF
+	eval `$CC_FOR_BUILD -E $dummy.c 2>/dev/null | grep '^LIBC'`
+	;;
+esac
+
 # Note: order is significant - the case branches are not exclusive.
 
 case "${UNAME_MACHINE}:${UNAME_SYSTEM}:${UNAME_RELEASE}:${UNAME_VERSION}" in
     *:NetBSD:*:*)
 	# NetBSD (nbsd) targets should (where applicable) match one or
-	# more of the tupples: *-*-netbsdelf*, *-*-netbsdaout*,
+	# more of the tuples: *-*-netbsdelf*, *-*-netbsdaout*,
 	# *-*-netbsdecoff* and *-*-netbsd*.  For targets that recently
 	# switched to ELF, *-*-netbsd* would select the old
 	# object file format.  This provides both forward
@@ -181,7 +194,7 @@ case "${UNAME_MACHINE}:${UNAME_SYSTEM}:${UNAME_RELEASE}:${UNAME_VERSION}" in
 		fi
 		;;
 	    *)
-	        os=netbsd
+		os=netbsd
 		;;
 	esac
 	# The OS release
@@ -202,6 +215,10 @@ case "${UNAME_MACHINE}:${UNAME_SYSTEM}:${UNAME_RELEASE}:${UNAME_VERSION}" in
 	# CPU_TYPE-MANUFACTURER-OPERATING_SYSTEM is used.
 	echo "${machine}-${os}${release}"
 	exit ;;
+    *:Bitrig:*:*)
+	UNAME_MACHINE_ARCH=`arch | sed 's/Bitrig.//'`
+	echo ${UNAME_MACHINE_ARCH}-unknown-bitrig${UNAME_RELEASE}
+	exit ;;
     *:OpenBSD:*:*)
 	UNAME_MACHINE_ARCH=`arch | sed 's/OpenBSD.//'`
 	echo ${UNAME_MACHINE_ARCH}-unknown-openbsd${UNAME_RELEASE}
@@ -224,7 +241,7 @@ case "${UNAME_MACHINE}:${UNAME_SYSTEM}:${UNAME_RELEASE}:${UNAME_VERSION}" in
 		UNAME_RELEASE=`/usr/sbin/sizer -v | awk '{print $3}'`
 		;;
 	*5.*)
-	        UNAME_RELEASE=`/usr/sbin/sizer -v | awk '{print $4}'`
+		UNAME_RELEASE=`/usr/sbin/sizer -v | awk '{print $4}'`
 		;;
 	esac
 	# According to Compaq, /usr/sbin/psrinfo has been available on
@@ -270,7 +287,10 @@ case "${UNAME_MACHINE}:${UNAME_SYSTEM}:${UNAME_RELEASE}:${UNAME_VERSION}" in
 	# A Xn.n version is an unreleased experimental baselevel.
 	# 1.2 uses "1.2" for uname -r.
 	echo ${UNAME_MACHINE}-dec-osf`echo ${UNAME_RELEASE} | sed -e 's/^[PVTX]//' | tr 'ABCDEFGHIJKLMNOPQRSTUVWXYZ' 'abcdefghijklmnopqrstuvwxyz'`
-	exit ;;
+	# Reset EXIT trap before exiting to avoid spurious non-zero exit code.
+	exitcode=$?
+	trap '' 0
+	exit $exitcode ;;
     Alpha\ *:Windows_NT*:*)
 	# How do we know it's Interix rather than the generic POSIX subsystem?
 	# Should we change UNAME_MACHINE based on the output of uname instead
@@ -296,12 +316,12 @@ case "${UNAME_MACHINE}:${UNAME_SYSTEM}:${UNAME_RELEASE}:${UNAME_VERSION}" in
 	echo s390-ibm-zvmoe
 	exit ;;
     *:OS400:*:*)
-        echo powerpc-ibm-os400
+	echo powerpc-ibm-os400
 	exit ;;
     arm:RISC*:1.[012]*:*|arm:riscix:1.[012]*:*)
 	echo arm-acorn-riscix${UNAME_RELEASE}
 	exit ;;
-    arm:riscos:*:*|arm:RISCOS:*:*)
+    arm*:riscos:*:*|arm*:RISCOS:*:*)
 	echo arm-unknown-riscos
 	exit ;;
     SR2?01:HI-UX/MPP:*:* | SR8000:HI-UX/MPP:*:*)
@@ -395,23 +415,23 @@ case "${UNAME_MACHINE}:${UNAME_SYSTEM}:${UNAME_RELEASE}:${UNAME_VERSION}" in
     # MiNT.  But MiNT is downward compatible to TOS, so this should
     # be no problem.
     atarist[e]:*MiNT:*:* | atarist[e]:*mint:*:* | atarist[e]:*TOS:*:*)
-        echo m68k-atari-mint${UNAME_RELEASE}
+	echo m68k-atari-mint${UNAME_RELEASE}
 	exit ;;
     atari*:*MiNT:*:* | atari*:*mint:*:* | atarist[e]:*TOS:*:*)
 	echo m68k-atari-mint${UNAME_RELEASE}
-        exit ;;
+	exit ;;
     *falcon*:*MiNT:*:* | *falcon*:*mint:*:* | *falcon*:*TOS:*:*)
-        echo m68k-atari-mint${UNAME_RELEASE}
+	echo m68k-atari-mint${UNAME_RELEASE}
 	exit ;;
     milan*:*MiNT:*:* | milan*:*mint:*:* | *milan*:*TOS:*:*)
-        echo m68k-milan-mint${UNAME_RELEASE}
-        exit ;;
+	echo m68k-milan-mint${UNAME_RELEASE}
+	exit ;;
     hades*:*MiNT:*:* | hades*:*mint:*:* | *hades*:*TOS:*:*)
-        echo m68k-hades-mint${UNAME_RELEASE}
-        exit ;;
+	echo m68k-hades-mint${UNAME_RELEASE}
+	exit ;;
     *:*MiNT:*:* | *:*mint:*:* | *:*TOS:*:*)
-        echo m68k-unknown-mint${UNAME_RELEASE}
-        exit ;;
+	echo m68k-unknown-mint${UNAME_RELEASE}
+	exit ;;
     m68k:machten:*:*)
 	echo m68k-apple-machten${UNAME_RELEASE}
 	exit ;;
@@ -481,8 +501,8 @@ EOF
 	echo m88k-motorola-sysv3
 	exit ;;
     AViiON:dgux:*:*)
-        # DG/UX returns AViiON for all architectures
-        UNAME_PROCESSOR=`/usr/bin/uname -p`
+	# DG/UX returns AViiON for all architectures
+	UNAME_PROCESSOR=`/usr/bin/uname -p`
 	if [ $UNAME_PROCESSOR = mc88100 ] || [ $UNAME_PROCESSOR = mc88110 ]
 	then
 	    if [ ${TARGET_BINARY_INTERFACE}x = m88kdguxelfx ] || \
@@ -495,7 +515,7 @@ EOF
 	else
 	    echo i586-dg-dgux${UNAME_RELEASE}
 	fi
- 	exit ;;
+	exit ;;
     M88*:DolphinOS:*:*)	# DolphinOS (SVR3)
 	echo m88k-dolphin-sysv3
 	exit ;;
@@ -552,7 +572,7 @@ EOF
 		echo rs6000-ibm-aix3.2
 	fi
 	exit ;;
-    *:AIX:*:[456])
+    *:AIX:*:[4567])
 	IBM_CPU_ID=`/usr/sbin/lsdev -C -c processor -S available | sed 1q | awk '{ print $1 }'`
 	if /usr/sbin/lsattr -El ${IBM_CPU_ID} | grep ' POWER' >/dev/null 2>&1; then
 		IBM_ARCH=rs6000
@@ -595,52 +615,52 @@ EOF
 	    9000/[678][0-9][0-9])
 		if [ -x /usr/bin/getconf ]; then
 		    sc_cpu_version=`/usr/bin/getconf SC_CPU_VERSION 2>/dev/null`
-                    sc_kernel_bits=`/usr/bin/getconf SC_KERNEL_BITS 2>/dev/null`
-                    case "${sc_cpu_version}" in
-                      523) HP_ARCH="hppa1.0" ;; # CPU_PA_RISC1_0
-                      528) HP_ARCH="hppa1.1" ;; # CPU_PA_RISC1_1
-                      532)                      # CPU_PA_RISC2_0
-                        case "${sc_kernel_bits}" in
-                          32) HP_ARCH="hppa2.0n" ;;
-                          64) HP_ARCH="hppa2.0w" ;;
+		    sc_kernel_bits=`/usr/bin/getconf SC_KERNEL_BITS 2>/dev/null`
+		    case "${sc_cpu_version}" in
+		      523) HP_ARCH="hppa1.0" ;; # CPU_PA_RISC1_0
+		      528) HP_ARCH="hppa1.1" ;; # CPU_PA_RISC1_1
+		      532)                      # CPU_PA_RISC2_0
+			case "${sc_kernel_bits}" in
+			  32) HP_ARCH="hppa2.0n" ;;
+			  64) HP_ARCH="hppa2.0w" ;;
 			  '') HP_ARCH="hppa2.0" ;;   # HP-UX 10.20
-                        esac ;;
-                    esac
+			esac ;;
+		    esac
 		fi
 		if [ "${HP_ARCH}" = "" ]; then
 		    eval $set_cc_for_build
-		    sed 's/^              //' << EOF >$dummy.c
+		    sed 's/^		//' << EOF >$dummy.c
 
-              #define _HPUX_SOURCE
-              #include <stdlib.h>
-              #include <unistd.h>
+		#define _HPUX_SOURCE
+		#include <stdlib.h>
+		#include <unistd.h>
 
-              int main ()
-              {
-              #if defined(_SC_KERNEL_BITS)
-                  long bits = sysconf(_SC_KERNEL_BITS);
-              #endif
-                  long cpu  = sysconf (_SC_CPU_VERSION);
+		int main ()
+		{
+		#if defined(_SC_KERNEL_BITS)
+		    long bits = sysconf(_SC_KERNEL_BITS);
+		#endif
+		    long cpu  = sysconf (_SC_CPU_VERSION);
 
-                  switch (cpu)
-              	{
-              	case CPU_PA_RISC1_0: puts ("hppa1.0"); break;
-              	case CPU_PA_RISC1_1: puts ("hppa1.1"); break;
-              	case CPU_PA_RISC2_0:
-              #if defined(_SC_KERNEL_BITS)
-              	    switch (bits)
-              		{
-              		case 64: puts ("hppa2.0w"); break;
-              		case 32: puts ("hppa2.0n"); break;
-              		default: puts ("hppa2.0"); break;
-              		} break;
-              #else  /* !defined(_SC_KERNEL_BITS) */
-              	    puts ("hppa2.0"); break;
-              #endif
-              	default: puts ("hppa1.0"); break;
-              	}
-                  exit (0);
-              }
+		    switch (cpu)
+			{
+			case CPU_PA_RISC1_0: puts ("hppa1.0"); break;
+			case CPU_PA_RISC1_1: puts ("hppa1.1"); break;
+			case CPU_PA_RISC2_0:
+		#if defined(_SC_KERNEL_BITS)
+			    switch (bits)
+				{
+				case 64: puts ("hppa2.0w"); break;
+				case 32: puts ("hppa2.0n"); break;
+				default: puts ("hppa2.0"); break;
+				} break;
+		#else  /* !defined(_SC_KERNEL_BITS) */
+			    puts ("hppa2.0"); break;
+		#endif
+			default: puts ("hppa1.0"); break;
+			}
+		    exit (0);
+		}
 EOF
 		    (CCOPTS= $CC_FOR_BUILD -o $dummy $dummy.c 2>/dev/null) && HP_ARCH=`$dummy`
 		    test -z "$HP_ARCH" && HP_ARCH=hppa
@@ -731,22 +751,22 @@ EOF
 	exit ;;
     C1*:ConvexOS:*:* | convex:ConvexOS:C1*:*)
 	echo c1-convex-bsd
-        exit ;;
+	exit ;;
     C2*:ConvexOS:*:* | convex:ConvexOS:C2*:*)
 	if getsysinfo -f scalar_acc
 	then echo c32-convex-bsd
 	else echo c2-convex-bsd
 	fi
-        exit ;;
+	exit ;;
     C34*:ConvexOS:*:* | convex:ConvexOS:C34*:*)
 	echo c34-convex-bsd
-        exit ;;
+	exit ;;
     C38*:ConvexOS:*:* | convex:ConvexOS:C38*:*)
 	echo c38-convex-bsd
-        exit ;;
+	exit ;;
     C4*:ConvexOS:*:* | convex:ConvexOS:C4*:*)
 	echo c4-convex-bsd
-        exit ;;
+	exit ;;
     CRAY*Y-MP:*:*:*)
 	echo ymp-cray-unicos${UNAME_RELEASE} | sed -e 's/\.[^.]*$/.X/'
 	exit ;;
@@ -770,14 +790,14 @@ EOF
 	exit ;;
     F30[01]:UNIX_System_V:*:* | F700:UNIX_System_V:*:*)
 	FUJITSU_PROC=`uname -m | tr 'ABCDEFGHIJKLMNOPQRSTUVWXYZ' 'abcdefghijklmnopqrstuvwxyz'`
-        FUJITSU_SYS=`uname -p | tr 'ABCDEFGHIJKLMNOPQRSTUVWXYZ' 'abcdefghijklmnopqrstuvwxyz' | sed -e 's/\///'`
-        FUJITSU_REL=`echo ${UNAME_RELEASE} | sed -e 's/ /_/'`
-        echo "${FUJITSU_PROC}-fujitsu-${FUJITSU_SYS}${FUJITSU_REL}"
-        exit ;;
+	FUJITSU_SYS=`uname -p | tr 'ABCDEFGHIJKLMNOPQRSTUVWXYZ' 'abcdefghijklmnopqrstuvwxyz' | sed -e 's/\///'`
+	FUJITSU_REL=`echo ${UNAME_RELEASE} | sed -e 's/ /_/'`
+	echo "${FUJITSU_PROC}-fujitsu-${FUJITSU_SYS}${FUJITSU_REL}"
+	exit ;;
     5000:UNIX_System_V:4.*:*)
-        FUJITSU_SYS=`uname -p | tr 'ABCDEFGHIJKLMNOPQRSTUVWXYZ' 'abcdefghijklmnopqrstuvwxyz' | sed -e 's/\///'`
-        FUJITSU_REL=`echo ${UNAME_RELEASE} | tr 'ABCDEFGHIJKLMNOPQRSTUVWXYZ' 'abcdefghijklmnopqrstuvwxyz' | sed -e 's/ /_/'`
-        echo "sparc-fujitsu-${FUJITSU_SYS}${FUJITSU_REL}"
+	FUJITSU_SYS=`uname -p | tr 'ABCDEFGHIJKLMNOPQRSTUVWXYZ' 'abcdefghijklmnopqrstuvwxyz' | sed -e 's/\///'`
+	FUJITSU_REL=`echo ${UNAME_RELEASE} | tr 'ABCDEFGHIJKLMNOPQRSTUVWXYZ' 'abcdefghijklmnopqrstuvwxyz' | sed -e 's/ /_/'`
+	echo "sparc-fujitsu-${FUJITSU_SYS}${FUJITSU_REL}"
 	exit ;;
     i*86:BSD/386:*:* | i*86:BSD/OS:*:* | *:Ascend\ Embedded/OS:*:*)
 	echo ${UNAME_MACHINE}-pc-bsdi${UNAME_RELEASE}
@@ -789,30 +809,35 @@ EOF
 	echo ${UNAME_MACHINE}-unknown-bsdi${UNAME_RELEASE}
 	exit ;;
     *:FreeBSD:*:*)
-	case ${UNAME_MACHINE} in
-	    pc98)
-		echo i386-unknown-freebsd`echo ${UNAME_RELEASE}|sed -e 's/[-(].*//'` ;;
+	UNAME_PROCESSOR=`/usr/bin/uname -p`
+	case ${UNAME_PROCESSOR} in
 	    amd64)
 		echo x86_64-unknown-freebsd`echo ${UNAME_RELEASE}|sed -e 's/[-(].*//'` ;;
 	    *)
-		echo ${UNAME_MACHINE}-unknown-freebsd`echo ${UNAME_RELEASE}|sed -e 's/[-(].*//'` ;;
+		echo ${UNAME_PROCESSOR}-unknown-freebsd`echo ${UNAME_RELEASE}|sed -e 's/[-(].*//'` ;;
 	esac
 	exit ;;
     i*:CYGWIN*:*)
 	echo ${UNAME_MACHINE}-pc-cygwin
 	exit ;;
+    *:MINGW64*:*)
+	echo ${UNAME_MACHINE}-pc-mingw64
+	exit ;;
     *:MINGW*:*)
 	echo ${UNAME_MACHINE}-pc-mingw32
 	exit ;;
+    i*:MSYS*:*)
+	echo ${UNAME_MACHINE}-pc-msys
+	exit ;;
     i*:windows32*:*)
-    	# uname -m includes "-pc" on this system.
-    	echo ${UNAME_MACHINE}-mingw32
+	# uname -m includes "-pc" on this system.
+	echo ${UNAME_MACHINE}-mingw32
 	exit ;;
     i*:PW*:*)
 	echo ${UNAME_MACHINE}-pc-pw32
 	exit ;;
     *:Interix*:*)
-    	case ${UNAME_MACHINE} in
+	case ${UNAME_MACHINE} in
 	    x86)
 		echo i586-pc-interix${UNAME_RELEASE}
 		exit ;;
@@ -849,15 +874,22 @@ EOF
 	exit ;;
     *:GNU:*:*)
 	# the GNU system
-	echo `echo ${UNAME_MACHINE}|sed -e 's,[-/].*$,,'`-unknown-gnu`echo ${UNAME_RELEASE}|sed -e 's,/.*$,,'`
+	echo `echo ${UNAME_MACHINE}|sed -e 's,[-/].*$,,'`-unknown-${LIBC}`echo ${UNAME_RELEASE}|sed -e 's,/.*$,,'`
 	exit ;;
     *:GNU/*:*:*)
 	# other systems with GNU libc and userland
-	echo ${UNAME_MACHINE}-unknown-`echo ${UNAME_SYSTEM} | sed 's,^[^/]*/,,' | tr '[A-Z]' '[a-z]'``echo ${UNAME_RELEASE}|sed -e 's/[-(].*//'`-gnu
+	echo ${UNAME_MACHINE}-unknown-`echo ${UNAME_SYSTEM} | sed 's,^[^/]*/,,' | tr '[A-Z]' '[a-z]'``echo ${UNAME_RELEASE}|sed -e 's/[-(].*//'`-${LIBC}
 	exit ;;
     i*86:Minix:*:*)
 	echo ${UNAME_MACHINE}-pc-minix
 	exit ;;
+    aarch64:Linux:*:*)
+	echo ${UNAME_MACHINE}-unknown-linux-${LIBC}
+	exit ;;
+    aarch64_be:Linux:*:*)
+	UNAME_MACHINE=aarch64_be
+	echo ${UNAME_MACHINE}-unknown-linux-${LIBC}
+	exit ;;
     alpha:Linux:*:*)
 	case `sed -n '/^cpu model/s/^.*: \(.*\)/\1/p' < /proc/cpuinfo` in
 	  EV5)   UNAME_MACHINE=alphaev5 ;;
@@ -867,52 +899,56 @@ EOF
 	  EV6)   UNAME_MACHINE=alphaev6 ;;
 	  EV67)  UNAME_MACHINE=alphaev67 ;;
 	  EV68*) UNAME_MACHINE=alphaev68 ;;
-        esac
+	esac
 	objdump --private-headers /bin/sh | grep -q ld.so.1
-	if test "$?" = 0 ; then LIBC="libc1" ; else LIBC="" ; fi
-	echo ${UNAME_MACHINE}-unknown-linux-gnu${LIBC}
+	if test "$?" = 0 ; then LIBC="gnulibc1" ; fi
+	echo ${UNAME_MACHINE}-unknown-linux-${LIBC}
+	exit ;;
+    arc:Linux:*:* | arceb:Linux:*:*)
+	echo ${UNAME_MACHINE}-unknown-linux-${LIBC}
 	exit ;;
     arm*:Linux:*:*)
 	eval $set_cc_for_build
 	if echo __ARM_EABI__ | $CC_FOR_BUILD -E - 2>/dev/null \
 	    | grep -q __ARM_EABI__
 	then
-	    echo ${UNAME_MACHINE}-unknown-linux-gnu
+	    echo ${UNAME_MACHINE}-unknown-linux-${LIBC}
 	else
-	    echo ${UNAME_MACHINE}-unknown-linux-gnueabi
+	    if echo __ARM_PCS_VFP | $CC_FOR_BUILD -E - 2>/dev/null \
+		| grep -q __ARM_PCS_VFP
+	    then
+		echo ${UNAME_MACHINE}-unknown-linux-${LIBC}eabi
+	    else
+		echo ${UNAME_MACHINE}-unknown-linux-${LIBC}eabihf
+	    fi
 	fi
 	exit ;;
     avr32*:Linux:*:*)
-	echo ${UNAME_MACHINE}-unknown-linux-gnu
+	echo ${UNAME_MACHINE}-unknown-linux-${LIBC}
 	exit ;;
     cris:Linux:*:*)
-	echo cris-axis-linux-gnu
+	echo ${UNAME_MACHINE}-axis-linux-${LIBC}
 	exit ;;
     crisv32:Linux:*:*)
-	echo crisv32-axis-linux-gnu
+	echo ${UNAME_MACHINE}-axis-linux-${LIBC}
 	exit ;;
     frv:Linux:*:*)
-    	echo frv-unknown-linux-gnu
+	echo ${UNAME_MACHINE}-unknown-linux-${LIBC}
+	exit ;;
+    hexagon:Linux:*:*)
+	echo ${UNAME_MACHINE}-unknown-linux-${LIBC}
 	exit ;;
     i*86:Linux:*:*)
-	LIBC=gnu
-	eval $set_cc_for_build
-	sed 's/^	//' << EOF >$dummy.c
-	#ifdef __dietlibc__
-	LIBC=dietlibc
-	#endif
-EOF
-	eval `$CC_FOR_BUILD -E $dummy.c 2>/dev/null | grep '^LIBC'`
-	echo "${UNAME_MACHINE}-pc-linux-${LIBC}"
+	echo ${UNAME_MACHINE}-pc-linux-${LIBC}
 	exit ;;
     ia64:Linux:*:*)
-	echo ${UNAME_MACHINE}-unknown-linux-gnu
+	echo ${UNAME_MACHINE}-unknown-linux-${LIBC}
 	exit ;;
     m32r*:Linux:*:*)
-	echo ${UNAME_MACHINE}-unknown-linux-gnu
+	echo ${UNAME_MACHINE}-unknown-linux-${LIBC}
 	exit ;;
     m68*:Linux:*:*)
-	echo ${UNAME_MACHINE}-unknown-linux-gnu
+	echo ${UNAME_MACHINE}-unknown-linux-${LIBC}
 	exit ;;
     mips:Linux:*:* | mips64:Linux:*:*)
 	eval $set_cc_for_build
@@ -931,51 +967,63 @@ EOF
 	#endif
 EOF
 	eval `$CC_FOR_BUILD -E $dummy.c 2>/dev/null | grep '^CPU'`
-	test x"${CPU}" != x && { echo "${CPU}-unknown-linux-gnu"; exit; }
+	test x"${CPU}" != x && { echo "${CPU}-unknown-linux-${LIBC}"; exit; }
 	;;
+    or1k:Linux:*:*)
+	echo ${UNAME_MACHINE}-unknown-linux-${LIBC}
+	exit ;;
     or32:Linux:*:*)
-	echo or32-unknown-linux-gnu
+	echo ${UNAME_MACHINE}-unknown-linux-${LIBC}
 	exit ;;
     padre:Linux:*:*)
-	echo sparc-unknown-linux-gnu
+	echo sparc-unknown-linux-${LIBC}
 	exit ;;
     parisc64:Linux:*:* | hppa64:Linux:*:*)
-	echo hppa64-unknown-linux-gnu
+	echo hppa64-unknown-linux-${LIBC}
 	exit ;;
     parisc:Linux:*:* | hppa:Linux:*:*)
 	# Look for CPU level
 	case `grep '^cpu[^a-z]*:' /proc/cpuinfo 2>/dev/null | cut -d' ' -f2` in
-	  PA7*) echo hppa1.1-unknown-linux-gnu ;;
-	  PA8*) echo hppa2.0-unknown-linux-gnu ;;
-	  *)    echo hppa-unknown-linux-gnu ;;
+	  PA7*) echo hppa1.1-unknown-linux-${LIBC} ;;
+	  PA8*) echo hppa2.0-unknown-linux-${LIBC} ;;
+	  *)    echo hppa-unknown-linux-${LIBC} ;;
 	esac
 	exit ;;
     ppc64:Linux:*:*)
-	echo powerpc64-unknown-linux-gnu
+	echo powerpc64-unknown-linux-${LIBC}
 	exit ;;
     ppc:Linux:*:*)
-	echo powerpc-unknown-linux-gnu
+	echo powerpc-unknown-linux-${LIBC}
+	exit ;;
+    ppc64le:Linux:*:*)
+	echo powerpc64le-unknown-linux-${LIBC}
+	exit ;;
+    ppcle:Linux:*:*)
+	echo powerpcle-unknown-linux-${LIBC}
 	exit ;;
     s390:Linux:*:* | s390x:Linux:*:*)
-	echo ${UNAME_MACHINE}-ibm-linux
+	echo ${UNAME_MACHINE}-ibm-linux-${LIBC}
 	exit ;;
     sh64*:Linux:*:*)
-    	echo ${UNAME_MACHINE}-unknown-linux-gnu
+	echo ${UNAME_MACHINE}-unknown-linux-${LIBC}
 	exit ;;
     sh*:Linux:*:*)
-	echo ${UNAME_MACHINE}-unknown-linux-gnu
+	echo ${UNAME_MACHINE}-unknown-linux-${LIBC}
 	exit ;;
     sparc:Linux:*:* | sparc64:Linux:*:*)
-	echo ${UNAME_MACHINE}-unknown-linux-gnu
+	echo ${UNAME_MACHINE}-unknown-linux-${LIBC}
+	exit ;;
+    tile*:Linux:*:*)
+	echo ${UNAME_MACHINE}-unknown-linux-${LIBC}
 	exit ;;
     vax:Linux:*:*)
-	echo ${UNAME_MACHINE}-dec-linux-gnu
+	echo ${UNAME_MACHINE}-dec-linux-${LIBC}
 	exit ;;
     x86_64:Linux:*:*)
-	echo x86_64-unknown-linux-gnu
+	echo ${UNAME_MACHINE}-unknown-linux-${LIBC}
 	exit ;;
     xtensa*:Linux:*:*)
-    	echo ${UNAME_MACHINE}-unknown-linux-gnu
+	echo ${UNAME_MACHINE}-unknown-linux-${LIBC}
 	exit ;;
     i*86:DYNIX/ptx:4*:*)
 	# ptx 4.0 does uname -s correctly, with DYNIX/ptx in there.
@@ -984,11 +1032,11 @@ EOF
 	echo i386-sequent-sysv4
 	exit ;;
     i*86:UNIX_SV:4.2MP:2.*)
-        # Unixware is an offshoot of SVR4, but it has its own version
-        # number series starting with 2...
-        # I am not positive that other SVR4 systems won't match this,
+	# Unixware is an offshoot of SVR4, but it has its own version
+	# number series starting with 2...
+	# I am not positive that other SVR4 systems won't match this,
 	# I just have to hope.  -- rms.
-        # Use sysv4.2uw... so that sysv4* matches it.
+	# Use sysv4.2uw... so that sysv4* matches it.
 	echo ${UNAME_MACHINE}-pc-sysv4.2uw${UNAME_VERSION}
 	exit ;;
     i*86:OS/2:*:*)
@@ -1020,7 +1068,7 @@ EOF
 	fi
 	exit ;;
     i*86:*:5:[678]*)
-    	# UnixWare 7.x, OpenUNIX and OpenServer 6.
+	# UnixWare 7.x, OpenUNIX and OpenServer 6.
 	case `/bin/uname -X | grep "^Machine"` in
 	    *486*)	     UNAME_MACHINE=i486 ;;
 	    *Pentium)	     UNAME_MACHINE=i586 ;;
@@ -1048,13 +1096,13 @@ EOF
 	exit ;;
     pc:*:*:*)
 	# Left here for compatibility:
-        # uname -m prints for DJGPP always 'pc', but it prints nothing about
-        # the processor, so we play safe by assuming i586.
+	# uname -m prints for DJGPP always 'pc', but it prints nothing about
+	# the processor, so we play safe by assuming i586.
 	# Note: whatever this is, it MUST be the same as what config.sub
 	# prints for the "djgpp" host, or else GDB configury will decide that
 	# this is a cross-build.
 	echo i586-pc-msdosdjgpp
-        exit ;;
+	exit ;;
     Intel:Mach:3*:*)
 	echo i386-pc-mach3
 	exit ;;
@@ -1089,8 +1137,8 @@ EOF
 	/bin/uname -p 2>/dev/null | /bin/grep entium >/dev/null \
 	  && { echo i586-ncr-sysv4.3${OS_REL}; exit; } ;;
     3[34]??:*:4.0:* | 3[34]??,*:*:4.0:*)
-        /bin/uname -p 2>/dev/null | grep 86 >/dev/null \
-          && { echo i486-ncr-sysv4; exit; } ;;
+	/bin/uname -p 2>/dev/null | grep 86 >/dev/null \
+	  && { echo i486-ncr-sysv4; exit; } ;;
     NCR*:*:4.2:* | MPRAS*:*:4.2:*)
 	OS_REL='.3'
 	test -r /etc/.relid \
@@ -1133,10 +1181,10 @@ EOF
 		echo ns32k-sni-sysv
 	fi
 	exit ;;
-    PENTIUM:*:4.0*:*) # Unisys `ClearPath HMP IX 4000' SVR4/MP effort
-                      # says <Richard.M.Bartel@ccMail.Census.GOV>
-        echo i586-unisys-sysv4
-        exit ;;
+    PENTIUM:*:4.0*:*)	# Unisys `ClearPath HMP IX 4000' SVR4/MP effort
+			# says <Richard.M.Bartel@ccMail.Census.GOV>
+	echo i586-unisys-sysv4
+	exit ;;
     *:UNIX_System_V:4*:FTX*)
 	# From Gerald Hewes <hewes@openmarket.com>.
 	# How about differentiating between stratus architectures? -djm
@@ -1162,11 +1210,11 @@ EOF
 	exit ;;
     R[34]000:*System_V*:*:* | R4000:UNIX_SYSV:*:* | R*000:UNIX_SV:*:*)
 	if [ -d /usr/nec ]; then
-	        echo mips-nec-sysv${UNAME_RELEASE}
+		echo mips-nec-sysv${UNAME_RELEASE}
 	else
-	        echo mips-unknown-sysv${UNAME_RELEASE}
+		echo mips-unknown-sysv${UNAME_RELEASE}
 	fi
-        exit ;;
+	exit ;;
     BeBox:BeOS:*:*)	# BeOS running on hardware made by Be, PPC only.
 	echo powerpc-be-beos
 	exit ;;
@@ -1179,6 +1227,9 @@ EOF
     BePC:Haiku:*:*)	# Haiku running on Intel PC compatible.
 	echo i586-pc-haiku
 	exit ;;
+    x86_64:Haiku:*:*)
+	echo x86_64-unknown-haiku
+	exit ;;
     SX-4:SUPER-UX:*:*)
 	echo sx4-nec-superux${UNAME_RELEASE}
 	exit ;;
@@ -1205,19 +1256,21 @@ EOF
 	exit ;;
     *:Darwin:*:*)
 	UNAME_PROCESSOR=`uname -p` || UNAME_PROCESSOR=unknown
-	case $UNAME_PROCESSOR in
-	    i386)
-		eval $set_cc_for_build
-		if [ "$CC_FOR_BUILD" != 'no_compiler_found' ]; then
-		  if (echo '#ifdef __LP64__'; echo IS_64BIT_ARCH; echo '#endif') | \
-		      (CCOPTS= $CC_FOR_BUILD -E - 2>/dev/null) | \
-		      grep IS_64BIT_ARCH >/dev/null
-		  then
-		      UNAME_PROCESSOR="x86_64"
-		  fi
-		fi ;;
-	    unknown) UNAME_PROCESSOR=powerpc ;;
-	esac
+	eval $set_cc_for_build
+	if test "$UNAME_PROCESSOR" = unknown ; then
+	    UNAME_PROCESSOR=powerpc
+	fi
+	if [ "$CC_FOR_BUILD" != 'no_compiler_found' ]; then
+	    if (echo '#ifdef __LP64__'; echo IS_64BIT_ARCH; echo '#endif') | \
+		(CCOPTS= $CC_FOR_BUILD -E - 2>/dev/null) | \
+		grep IS_64BIT_ARCH >/dev/null
+	    then
+		case $UNAME_PROCESSOR in
+		    i386) UNAME_PROCESSOR=x86_64 ;;
+		    powerpc) UNAME_PROCESSOR=powerpc64 ;;
+		esac
+	    fi
+	fi
 	echo ${UNAME_PROCESSOR}-apple-darwin${UNAME_RELEASE}
 	exit ;;
     *:procnto*:*:* | *:QNX:[0123456789]*:*)
@@ -1231,7 +1284,10 @@ EOF
     *:QNX:*:4*)
 	echo i386-pc-qnx
 	exit ;;
-    NSE-?:NONSTOP_KERNEL:*:*)
+    NEO-?:NONSTOP_KERNEL:*:*)
+	echo neo-tandem-nsk${UNAME_RELEASE}
+	exit ;;
+    NSE-*:NONSTOP_KERNEL:*:*)
 	echo nse-tandem-nsk${UNAME_RELEASE}
 	exit ;;
     NSR-?:NONSTOP_KERNEL:*:*)
@@ -1276,13 +1332,13 @@ EOF
 	echo pdp10-unknown-its
 	exit ;;
     SEI:*:*:SEIUX)
-        echo mips-sei-seiux${UNAME_RELEASE}
+	echo mips-sei-seiux${UNAME_RELEASE}
 	exit ;;
     *:DragonFly:*:*)
 	echo ${UNAME_MACHINE}-unknown-dragonfly`echo ${UNAME_RELEASE}|sed -e 's/[-(].*//'`
 	exit ;;
     *:*VMS:*:*)
-    	UNAME_MACHINE=`(uname -p) 2>/dev/null`
+	UNAME_MACHINE=`(uname -p) 2>/dev/null`
 	case "${UNAME_MACHINE}" in
 	    A*) echo alpha-dec-vms ; exit ;;
 	    I*) echo ia64-dec-vms ; exit ;;
@@ -1300,11 +1356,11 @@ EOF
     i*86:AROS:*:*)
 	echo ${UNAME_MACHINE}-pc-aros
 	exit ;;
+    x86_64:VMkernel:*:*)
+	echo ${UNAME_MACHINE}-unknown-esx
+	exit ;;
 esac
 
-#echo '(No uname command or uname output not recognized.)' 1>&2
-#echo "${UNAME_MACHINE}:${UNAME_SYSTEM}:${UNAME_RELEASE}:${UNAME_VERSION}" 1>&2
-
 eval $set_cc_for_build
 cat >$dummy.c <<EOF
 #ifdef _SEQUENT_
@@ -1322,11 +1378,11 @@ main ()
 #include <sys/param.h>
   printf ("m68k-sony-newsos%s\n",
 #ifdef NEWSOS4
-          "4"
+	"4"
 #else
-	  ""
+	""
 #endif
-         ); exit (0);
+	); exit (0);
 #endif
 #endif
 
diff --git a/config.sub b/config.sub
index c2d1257..9633db7 100755
--- a/config.sub
+++ b/config.sub
@@ -1,38 +1,31 @@
 #! /bin/sh
 # Configuration validation subroutine script.
-#   Copyright (C) 1992, 1993, 1994, 1995, 1996, 1997, 1998, 1999,
-#   2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010
-#   Free Software Foundation, Inc.
+#   Copyright 1992-2013 Free Software Foundation, Inc.
 
-timestamp='2010-01-22'
+timestamp='2013-08-10'
 
-# This file is (in principle) common to ALL GNU software.
-# The presence of a machine in this file suggests that SOME GNU software
-# can handle that machine.  It does not imply ALL GNU software can.
-#
-# This file is free software; you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation; either version 2 of the License, or
+# This file is free software; you can redistribute it and/or modify it
+# under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3 of the License, or
 # (at your option) any later version.
 #
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU General Public License for more details.
+# This program is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+# General Public License for more details.
 #
 # You should have received a copy of the GNU General Public License
-# along with this program; if not, write to the Free Software
-# Foundation, Inc., 51 Franklin Street - Fifth Floor, Boston, MA
-# 02110-1301, USA.
+# along with this program; if not, see <http://www.gnu.org/licenses/>.
 #
 # As a special exception to the GNU General Public License, if you
 # distribute this file as part of a program that contains a
 # configuration script generated by Autoconf, you may include it under
-# the same distribution terms that you use for the rest of that program.
+# the same distribution terms that you use for the rest of that
+# program.  This Exception is an additional permission under section 7
+# of the GNU General Public License, version 3 ("GPLv3").
 
 
-# Please send patches to <config-patches@gnu.org>.  Submit a context
-# diff and a properly formatted GNU ChangeLog entry.
+# Please send patches with a ChangeLog entry to config-patches@gnu.org.
 #
 # Configuration subroutine to validate and canonicalize a configuration type.
 # Supply the specified configuration type as an argument.
@@ -75,9 +68,7 @@ Report bugs and patches to <config-patches@gnu.org>."
 version="\
 GNU config.sub ($timestamp)
 
-Copyright (C) 1992, 1993, 1994, 1995, 1996, 1997, 1998, 1999, 2000,
-2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010 Free
-Software Foundation, Inc.
+Copyright 1992-2013 Free Software Foundation, Inc.
 
 This is free software; see the source for copying conditions.  There is NO
 warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE."
@@ -124,13 +115,18 @@ esac
 # Here we must recognize all the valid KERNEL-OS combinations.
 maybe_os=`echo $1 | sed 's/^\(.*\)-\([^-]*-[^-]*\)$/\2/'`
 case $maybe_os in
-  nto-qnx* | linux-gnu* | linux-dietlibc | linux-newlib* | linux-uclibc* | \
-  uclinux-uclibc* | uclinux-gnu* | kfreebsd*-gnu* | knetbsd*-gnu* | netbsd*-gnu* | \
+  nto-qnx* | linux-gnu* | linux-android* | linux-dietlibc | linux-newlib* | \
+  linux-musl* | linux-uclibc* | uclinux-uclibc* | uclinux-gnu* | kfreebsd*-gnu* | \
+  knetbsd*-gnu* | netbsd*-gnu* | \
   kopensolaris*-gnu* | \
   storm-chaos* | os2-emx* | rtmk-nova*)
     os=-$maybe_os
     basic_machine=`echo $1 | sed 's/^\(.*\)-\([^-]*-[^-]*\)$/\1/'`
     ;;
+  android-linux)
+    os=-linux-android
+    basic_machine=`echo $1 | sed 's/^\(.*\)-\([^-]*-[^-]*\)$/\1/'`-unknown
+    ;;
   *)
     basic_machine=`echo $1 | sed 's/-[^-]*$//'`
     if [ $basic_machine != $1 ]
@@ -153,12 +149,12 @@ case $os in
 	-convergent* | -ncr* | -news | -32* | -3600* | -3100* | -hitachi* |\
 	-c[123]* | -convex* | -sun | -crds | -omron* | -dg | -ultra | -tti* | \
 	-harris | -dolphin | -highlevel | -gould | -cbm | -ns | -masscomp | \
-	-apple | -axis | -knuth | -cray | -microblaze)
+	-apple | -axis | -knuth | -cray | -microblaze*)
 		os=
 		basic_machine=$1
 		;;
-        -bluegene*)
-	        os=-cnk
+	-bluegene*)
+		os=-cnk
 		;;
 	-sim | -cisco | -oki | -wec | -winbond)
 		os=
@@ -174,10 +170,10 @@ case $os in
 		os=-chorusos
 		basic_machine=$1
 		;;
- 	-chorusrdb)
- 		os=-chorusrdb
+	-chorusrdb)
+		os=-chorusrdb
 		basic_machine=$1
- 		;;
+		;;
 	-hiux*)
 		os=-hiuxwe2
 		;;
@@ -222,6 +218,12 @@ case $os in
 	-isc*)
 		basic_machine=`echo $1 | sed -e 's/86-.*/86-pc/'`
 		;;
+	-lynx*178)
+		os=-lynxos178
+		;;
+	-lynx*5)
+		os=-lynxos5
+		;;
 	-lynx*)
 		os=-lynxos
 		;;
@@ -246,20 +248,27 @@ case $basic_machine in
 	# Some are omitted here because they have special meanings below.
 	1750a | 580 \
 	| a29k \
+	| aarch64 | aarch64_be \
 	| alpha | alphaev[4-8] | alphaev56 | alphaev6[78] | alphapca5[67] \
 	| alpha64 | alpha64ev[4-8] | alpha64ev56 | alpha64ev6[78] | alpha64pca5[67] \
 	| am33_2.0 \
-	| arc | arm | arm[bl]e | arme[lb] | armv[2345] | armv[345][lb] | avr | avr32 \
+	| arc | arceb \
+	| arm | arm[bl]e | arme[lb] | armv[2-8] | armv[3-8][lb] | armv7[arm] \
+	| avr | avr32 \
+	| be32 | be64 \
 	| bfin \
-	| c4x | clipper \
+	| c4x | c8051 | clipper \
 	| d10v | d30v | dlx | dsp16xx \
+	| epiphany \
 	| fido | fr30 | frv \
 	| h8300 | h8500 | hppa | hppa1.[01] | hppa2.0 | hppa2.0[nw] | hppa64 \
+	| hexagon \
 	| i370 | i860 | i960 | ia64 \
 	| ip2k | iq2000 \
+	| le32 | le64 \
 	| lm32 \
 	| m32c | m32r | m32rle | m68000 | m68k | m88k \
-	| maxq | mb | microblaze | mcore | mep | metag \
+	| maxq | mb | microblaze | microblazeel | mcore | mep | metag \
 	| mips | mipsbe | mipseb | mipsel | mipsle \
 	| mips16 \
 	| mips64 | mips64el \
@@ -277,34 +286,45 @@ case $basic_machine in
 	| mipsisa64r2 | mipsisa64r2el \
 	| mipsisa64sb1 | mipsisa64sb1el \
 	| mipsisa64sr71k | mipsisa64sr71kel \
+	| mipsr5900 | mipsr5900el \
 	| mipstx39 | mipstx39el \
 	| mn10200 | mn10300 \
 	| moxie \
 	| mt \
 	| msp430 \
-	| nios | nios2 \
+	| nds32 | nds32le | nds32be \
+	| nios | nios2 | nios2eb | nios2el \
 	| ns16k | ns32k \
-	| or32 \
+	| open8 \
+	| or1k | or32 \
 	| pdp10 | pdp11 | pj | pjl \
-	| powerpc | powerpc64 | powerpc64le | powerpcle | ppcbe \
+	| powerpc | powerpc64 | powerpc64le | powerpcle \
 	| pyramid \
-	| rx \
+	| rl78 | rx \
 	| score \
 	| sh | sh[1234] | sh[24]a | sh[24]aeb | sh[23]e | sh[34]eb | sheb | shbe | shle | sh[1234]le | sh3ele \
 	| sh64 | sh64le \
 	| sparc | sparc64 | sparc64b | sparc64v | sparc86x | sparclet | sparclite \
 	| sparcv8 | sparcv9 | sparcv9b | sparcv9v \
-	| spu | strongarm \
-	| tahoe | thumb | tic4x | tic80 | tron \
+	| spu \
+	| tahoe | tic4x | tic54x | tic55x | tic6x | tic80 | tron \
 	| ubicom32 \
-	| v850 | v850e \
+	| v850 | v850e | v850e1 | v850e2 | v850es | v850e2v3 \
 	| we32k \
-	| x86 | xc16x | xscale | xscalee[bl] | xstormy16 | xtensa \
+	| x86 | xc16x | xstormy16 | xtensa \
 	| z8k | z80)
 		basic_machine=$basic_machine-unknown
 		;;
-	m6811 | m68hc11 | m6812 | m68hc12 | picochip)
-		# Motorola 68HC11/12.
+	c54x)
+		basic_machine=tic54x-unknown
+		;;
+	c55x)
+		basic_machine=tic55x-unknown
+		;;
+	c6x)
+		basic_machine=tic6x-unknown
+		;;
+	m6811 | m68hc11 | m6812 | m68hc12 | m68hcs12x | picochip)
 		basic_machine=$basic_machine-unknown
 		os=-none
 		;;
@@ -314,6 +334,21 @@ case $basic_machine in
 		basic_machine=mt-unknown
 		;;
 
+	strongarm | thumb | xscale)
+		basic_machine=arm-unknown
+		;;
+	xgate)
+		basic_machine=$basic_machine-unknown
+		os=-none
+		;;
+	xscaleeb)
+		basic_machine=armeb-unknown
+		;;
+
+	xscaleel)
+		basic_machine=armel-unknown
+		;;
+
 	# We use `pc' rather than `unknown'
 	# because (1) that's what they normally are, and
 	# (2) the word "unknown" tends to confuse beginning users.
@@ -328,25 +363,30 @@ case $basic_machine in
 	# Recognize the basic CPU types with company name.
 	580-* \
 	| a29k-* \
+	| aarch64-* | aarch64_be-* \
 	| alpha-* | alphaev[4-8]-* | alphaev56-* | alphaev6[78]-* \
 	| alpha64-* | alpha64ev[4-8]-* | alpha64ev56-* | alpha64ev6[78]-* \
-	| alphapca5[67]-* | alpha64pca5[67]-* | arc-* \
+	| alphapca5[67]-* | alpha64pca5[67]-* | arc-* | arceb-* \
 	| arm-*  | armbe-* | armle-* | armeb-* | armv*-* \
 	| avr-* | avr32-* \
+	| be32-* | be64-* \
 	| bfin-* | bs2000-* \
-	| c[123]* | c30-* | [cjt]90-* | c4x-* | c54x-* | c55x-* | c6x-* \
-	| clipper-* | craynv-* | cydra-* \
+	| c[123]* | c30-* | [cjt]90-* | c4x-* \
+	| c8051-* | clipper-* | craynv-* | cydra-* \
 	| d10v-* | d30v-* | dlx-* \
 	| elxsi-* \
 	| f30[01]-* | f700-* | fido-* | fr30-* | frv-* | fx80-* \
 	| h8300-* | h8500-* \
 	| hppa-* | hppa1.[01]-* | hppa2.0-* | hppa2.0[nw]-* | hppa64-* \
+	| hexagon-* \
 	| i*86-* | i860-* | i960-* | ia64-* \
 	| ip2k-* | iq2000-* \
+	| le32-* | le64-* \
 	| lm32-* \
 	| m32c-* | m32r-* | m32rle-* \
 	| m68000-* | m680[012346]0-* | m68360-* | m683?2-* | m68k-* \
-	| m88110-* | m88k-* | maxq-* | mcore-* | metag-* | microblaze-* \
+	| m88110-* | m88k-* | maxq-* | mcore-* | metag-* \
+	| microblaze-* | microblazeel-* \
 	| mips-* | mipsbe-* | mipseb-* | mipsel-* | mipsle-* \
 	| mips16-* \
 	| mips64-* | mips64el-* \
@@ -364,30 +404,34 @@ case $basic_machine in
 	| mipsisa64r2-* | mipsisa64r2el-* \
 	| mipsisa64sb1-* | mipsisa64sb1el-* \
 	| mipsisa64sr71k-* | mipsisa64sr71kel-* \
+	| mipsr5900-* | mipsr5900el-* \
 	| mipstx39-* | mipstx39el-* \
 	| mmix-* \
 	| mt-* \
 	| msp430-* \
-	| nios-* | nios2-* \
+	| nds32-* | nds32le-* | nds32be-* \
+	| nios-* | nios2-* | nios2eb-* | nios2el-* \
 	| none-* | np1-* | ns16k-* | ns32k-* \
+	| open8-* \
 	| orion-* \
 	| pdp10-* | pdp11-* | pj-* | pjl-* | pn-* | power-* \
-	| powerpc-* | powerpc64-* | powerpc64le-* | powerpcle-* | ppcbe-* \
+	| powerpc-* | powerpc64-* | powerpc64le-* | powerpcle-* \
 	| pyramid-* \
-	| romp-* | rs6000-* | rx-* \
+	| rl78-* | romp-* | rs6000-* | rx-* \
 	| sh-* | sh[1234]-* | sh[24]a-* | sh[24]aeb-* | sh[23]e-* | sh[34]eb-* | sheb-* | shbe-* \
 	| shle-* | sh[1234]le-* | sh3ele-* | sh64-* | sh64le-* \
 	| sparc-* | sparc64-* | sparc64b-* | sparc64v-* | sparc86x-* | sparclet-* \
 	| sparclite-* \
-	| sparcv8-* | sparcv9-* | sparcv9b-* | sparcv9v-* | strongarm-* | sv1-* | sx?-* \
-	| tahoe-* | thumb-* \
+	| sparcv8-* | sparcv9-* | sparcv9b-* | sparcv9v-* | sv1-* | sx?-* \
+	| tahoe-* \
 	| tic30-* | tic4x-* | tic54x-* | tic55x-* | tic6x-* | tic80-* \
-	| tile-* | tilegx-* \
+	| tile*-* \
 	| tron-* \
 	| ubicom32-* \
-	| v850-* | v850e-* | vax-* \
+	| v850-* | v850e-* | v850e1-* | v850es-* | v850e2-* | v850e2v3-* \
+	| vax-* \
 	| we32k-* \
-	| x86-* | x86_64-* | xc16x-* | xps100-* | xscale-* | xscalee[bl]-* \
+	| x86-* | x86_64-* | xc16x-* | xps100-* \
 	| xstormy16-* | xtensa*-* \
 	| ymp-* \
 	| z8k-* | z80-*)
@@ -412,7 +456,7 @@ case $basic_machine in
 		basic_machine=a29k-amd
 		os=-udi
 		;;
-    	abacus)
+	abacus)
 		basic_machine=abacus-unknown
 		;;
 	adobe68k)
@@ -482,11 +526,20 @@ case $basic_machine in
 		basic_machine=powerpc-ibm
 		os=-cnk
 		;;
+	c54x-*)
+		basic_machine=tic54x-`echo $basic_machine | sed 's/^[^-]*-//'`
+		;;
+	c55x-*)
+		basic_machine=tic55x-`echo $basic_machine | sed 's/^[^-]*-//'`
+		;;
+	c6x-*)
+		basic_machine=tic6x-`echo $basic_machine | sed 's/^[^-]*-//'`
+		;;
 	c90)
 		basic_machine=c90-cray
 		os=-unicos
 		;;
-        cegcc)
+	cegcc)
 		basic_machine=arm-unknown
 		os=-cegcc
 		;;
@@ -518,7 +571,7 @@ case $basic_machine in
 		basic_machine=craynv-cray
 		os=-unicosmp
 		;;
-	cr16)
+	cr16 | cr16-*)
 		basic_machine=cr16-unknown
 		os=-elf
 		;;
@@ -676,7 +729,6 @@ case $basic_machine in
 	i370-ibm* | ibm*)
 		basic_machine=i370-ibm
 		;;
-# I'm not sure what "Sysv32" means.  Should this be sysv3.2?
 	i*86v32)
 		basic_machine=`echo $1 | sed -e 's/86.*/86-pc/'`
 		os=-sysv32
@@ -734,11 +786,15 @@ case $basic_machine in
 		basic_machine=ns32k-utek
 		os=-sysv
 		;;
-        microblaze)
+	microblaze*)
 		basic_machine=microblaze-xilinx
 		;;
+	mingw64)
+		basic_machine=x86_64-pc
+		os=-mingw64
+		;;
 	mingw32)
-		basic_machine=i386-pc
+		basic_machine=i686-pc
 		os=-mingw32
 		;;
 	mingw32ce)
@@ -773,10 +829,18 @@ case $basic_machine in
 	ms1-*)
 		basic_machine=`echo $basic_machine | sed -e 's/ms1-/mt-/'`
 		;;
+	msys)
+		basic_machine=i686-pc
+		os=-msys
+		;;
 	mvs)
 		basic_machine=i370-ibm
 		os=-mvs
 		;;
+	nacl)
+		basic_machine=le32-unknown
+		os=-nacl
+		;;
 	ncr3000)
 		basic_machine=i486-ncr
 		os=-sysv4
@@ -841,6 +905,12 @@ case $basic_machine in
 	np1)
 		basic_machine=np1-gould
 		;;
+	neo-tandem)
+		basic_machine=neo-tandem
+		;;
+	nse-tandem)
+		basic_machine=nse-tandem
+		;;
 	nsr-tandem)
 		basic_machine=nsr-tandem
 		;;
@@ -923,9 +993,10 @@ case $basic_machine in
 		;;
 	power)	basic_machine=power-ibm
 		;;
-	ppc)	basic_machine=powerpc-unknown
+	ppc | ppcbe)	basic_machine=powerpc-unknown
 		;;
-	ppc-*)	basic_machine=powerpc-`echo $basic_machine | sed 's/^[^-]*-//'`
+	ppc-* | ppcbe-*)
+		basic_machine=powerpc-`echo $basic_machine | sed 's/^[^-]*-//'`
 		;;
 	ppcle | powerpclittle | ppc-le | powerpc-little)
 		basic_machine=powerpcle-unknown
@@ -950,7 +1021,11 @@ case $basic_machine in
 		basic_machine=i586-unknown
 		os=-pw32
 		;;
-	rdos)
+	rdos | rdos64)
+		basic_machine=x86_64-pc
+		os=-rdos
+		;;
+	rdos32)
 		basic_machine=i386-pc
 		os=-rdos
 		;;
@@ -1019,6 +1094,9 @@ case $basic_machine in
 		basic_machine=i860-stratus
 		os=-sysv4
 		;;
+	strongarm-* | thumb-*)
+		basic_machine=arm-`echo $basic_machine | sed 's/^[^-]*-//'`
+		;;
 	sun2)
 		basic_machine=m68000-sun
 		;;
@@ -1075,25 +1153,8 @@ case $basic_machine in
 		basic_machine=t90-cray
 		os=-unicos
 		;;
-	tic54x | c54x*)
-		basic_machine=tic54x-unknown
-		os=-coff
-		;;
-	tic55x | c55x*)
-		basic_machine=tic55x-unknown
-		os=-coff
-		;;
-	tic6x | c6x*)
-		basic_machine=tic6x-unknown
-		os=-coff
-		;;
-        # This must be matched before tile*.
-        tilegx*)
-		basic_machine=tilegx-unknown
-		os=-linux-gnu
-		;;
 	tile*)
-		basic_machine=tile-unknown
+		basic_machine=$basic_machine-unknown
 		os=-linux-gnu
 		;;
 	tx39)
@@ -1163,6 +1224,9 @@ case $basic_machine in
 	xps | xps100)
 		basic_machine=xps100-honeywell
 		;;
+	xscale-* | xscalee[bl]-*)
+		basic_machine=`echo $basic_machine | sed 's/^xscale/arm/'`
+		;;
 	ymp)
 		basic_machine=ymp-cray
 		os=-unicos
@@ -1260,11 +1324,11 @@ esac
 if [ x"$os" != x"" ]
 then
 case $os in
-        # First match some system type aliases
-        # that might get confused with valid system types.
+	# First match some system type aliases
+	# that might get confused with valid system types.
 	# -solaris* is a basic system type, with this one exception.
-        -auroraux)
-	        os=-auroraux
+	-auroraux)
+		os=-auroraux
 		;;
 	-solaris1 | -solaris1.*)
 		os=`echo $os | sed -e 's|solaris1|sunos4|'`
@@ -1288,20 +1352,21 @@ case $os in
 	-gnu* | -bsd* | -mach* | -minix* | -genix* | -ultrix* | -irix* \
 	      | -*vms* | -sco* | -esix* | -isc* | -aix* | -cnk* | -sunos | -sunos[34]*\
 	      | -hpux* | -unos* | -osf* | -luna* | -dgux* | -auroraux* | -solaris* \
-	      | -sym* | -kopensolaris* \
+	      | -sym* | -kopensolaris* | -plan9* \
 	      | -amigaos* | -amigados* | -msdos* | -newsos* | -unicos* | -aof* \
 	      | -aos* | -aros* \
 	      | -nindy* | -vxsim* | -vxworks* | -ebmon* | -hms* | -mvs* \
 	      | -clix* | -riscos* | -uniplus* | -iris* | -rtu* | -xenix* \
 	      | -hiux* | -386bsd* | -knetbsd* | -mirbsd* | -netbsd* \
-	      | -openbsd* | -solidbsd* \
+	      | -bitrig* | -openbsd* | -solidbsd* \
 	      | -ekkobsd* | -kfreebsd* | -freebsd* | -riscix* | -lynxos* \
 	      | -bosx* | -nextstep* | -cxux* | -aout* | -elf* | -oabi* \
 	      | -ptx* | -coff* | -ecoff* | -winnt* | -domain* | -vsta* \
 	      | -udi* | -eabi* | -lites* | -ieee* | -go32* | -aux* \
 	      | -chorusos* | -chorusrdb* | -cegcc* \
-	      | -cygwin* | -pe* | -psos* | -moss* | -proelf* | -rtems* \
-	      | -mingw32* | -linux-gnu* | -linux-newlib* | -linux-uclibc* \
+	      | -cygwin* | -msys* | -pe* | -psos* | -moss* | -proelf* | -rtems* \
+	      | -mingw32* | -mingw64* | -linux-gnu* | -linux-android* \
+	      | -linux-newlib* | -linux-musl* | -linux-uclibc* \
 	      | -uxpv* | -beos* | -mpeix* | -udk* \
 	      | -interix* | -uwin* | -mks* | -rhapsody* | -darwin* | -opened* \
 	      | -openstep* | -oskit* | -conix* | -pw32* | -nonstopux* \
@@ -1348,7 +1413,7 @@ case $os in
 	-opened*)
 		os=-openedition
 		;;
-        -os400*)
+	-os400*)
 		os=-os400
 		;;
 	-wince*)
@@ -1397,7 +1462,7 @@ case $os in
 	-sinix*)
 		os=-sysv4
 		;;
-        -tpf*)
+	-tpf*)
 		os=-tpf
 		;;
 	-triton*)
@@ -1433,17 +1498,14 @@ case $os in
 	-aros*)
 		os=-aros
 		;;
-	-kaos*)
-		os=-kaos
-		;;
 	-zvmoe)
 		os=-zvmoe
 		;;
 	-dicos*)
 		os=-dicos
 		;;
-        -nacl*)
-	        ;;
+	-nacl*)
+		;;
 	-none)
 		;;
 	*)
@@ -1466,10 +1528,10 @@ else
 # system, and we'll never get to this point.
 
 case $basic_machine in
-        score-*)
+	score-*)
 		os=-elf
 		;;
-        spu-*)
+	spu-*)
 		os=-elf
 		;;
 	*-acorn)
@@ -1481,8 +1543,23 @@ case $basic_machine in
 	arm*-semi)
 		os=-aout
 		;;
-        c4x-* | tic4x-*)
-        	os=-coff
+	c4x-* | tic4x-*)
+		os=-coff
+		;;
+	c8051-*)
+		os=-elf
+		;;
+	hexagon-*)
+		os=-elf
+		;;
+	tic54x-*)
+		os=-coff
+		;;
+	tic55x-*)
+		os=-coff
+		;;
+	tic6x-*)
+		os=-coff
 		;;
 	# This must come before the *-dec entry.
 	pdp10-*)
@@ -1502,14 +1579,11 @@ case $basic_machine in
 		;;
 	m68000-sun)
 		os=-sunos3
-		# This also exists in the configure program, but was not the
-		# default.
-		# os=-sunos4
 		;;
 	m68*-cisco)
 		os=-aout
 		;;
-        mep-*)
+	mep-*)
 		os=-elf
 		;;
 	mips*-cisco)
@@ -1518,6 +1592,9 @@ case $basic_machine in
 	mips*-*)
 		os=-elf
 		;;
+	or1k-*)
+		os=-elf
+		;;
 	or32-*)
 		os=-coff
 		;;
@@ -1536,7 +1613,7 @@ case $basic_machine in
 	*-ibm)
 		os=-aix
 		;;
-    	*-knuth)
+	*-knuth)
 		os=-mmixware
 		;;
 	*-wec)
-- 
1.7.10.4

[PATCH v5 04/19] xen: arm: Report aarch64 capability.

[Ian Campbell]

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Julien Grall <julien.grall@linaro.org>
---
 xen/arch/arm/setup.c |    4 ++++
 1 file changed, 4 insertions(+)

diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c
index 1081b43..cdcc2e7 100644
--- a/xen/arch/arm/setup.c
+++ b/xen/arch/arm/setup.c
@@ -731,6 +731,10 @@ void arch_get_xen_caps(xen_capabilities_info_t *info)
 
     (*info)[0] = '\0';
 
+#ifdef CONFIG_ARM_64
+    snprintf(s, sizeof(s), "xen-%d.%d-aarch64 ", major, minor);
+    safe_strcat(*info, s);
+#endif
     snprintf(s, sizeof(s), "xen-%d.%d-armv7l ", major, minor);
     safe_strcat(*info, s);
 }
-- 
1.7.10.4

[PATCH v5 05/19] xen: arm: Add comment regard arm64 zImage v0 vs v1

[Ian Campbell]

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Julien Grall <julien.grall@linaro.org>
---
 xen/arch/arm/kernel.c |    1 +
 1 file changed, 1 insertion(+)

diff --git a/xen/arch/arm/kernel.c b/xen/arch/arm/kernel.c
index 315d12c..7036d94 100644
--- a/xen/arch/arm/kernel.c
+++ b/xen/arch/arm/kernel.c
@@ -139,6 +139,7 @@ static int kernel_try_zimage64_prepare(struct kernel_info *info,
         uint64_t text_offset;  /* Image load offset */
         uint64_t res1;
         uint64_t res2;
+        /* zImage V1 only from here */
         uint64_t res3;
         uint64_t res4;
         uint64_t res5;
-- 
1.7.10.4

[PATCH v5 06/19] xen: arm: move dom0 gic and timer device tree nodes under /xen/

[Ian Campbell]

Julien observed that we were relying on the provided host DTB supplying
suitable #address-cells and #size-cells values to allow us to represent these
addresses, which may not reliably be the case. Moving these under our own
known (somewhat analogous to the use of /soc/ or /motherboard/ on some
platforms) allows us to control these sizes.

Since the xen node is created out of thin air it does not have a corresponding
struct dt_device_node and therefore we cannot use dt_n_addr_cells or
dt_n_size_cells, we can use hardcoded constants instead. For the same reason
we define and use set_xen_range instead of dt_set_range.

The hypervisor, cpus and psci node all either defined #foo-cells for their
children or do not contain reg properties and therefore can remain at the top
level.

The logging in make_gic_node was inconsistent. Fix it.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
---
v5: New patch
---
 xen/arch/arm/domain_build.c |   76 ++++++++++++++++++++++++++++++++++---------
 1 file changed, 61 insertions(+), 15 deletions(-)

diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index 186746c..8645aa1 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -48,6 +48,24 @@ custom_param("dom0_mem", parse_dom0_mem);
  */
 #define DOM0_FDT_EXTRA_SIZE (128 + sizeof(struct fdt_reserve_entry))
 
+/*
+ * Number of cells used for addresses and sizes under the /xen/
+ * node.
+ *
+ * We don't have a struct dt_device_node we can reference as a parent
+ * to get these values, so use these constants instead.
+ */
+#define XEN_FDT_NODE_ADDRESS_CELLS 2
+#define XEN_FDT_NODE_SIZE_CELLS 2
+#define XEN_FDT_NODE_REG_SIZE \
+    dt_cells_to_size(XEN_FDT_NODE_ADDRESS_CELLS + XEN_FDT_NODE_SIZE_CELLS)
+
+static void set_xen_range(__be32 **cellp, u64 address, u64 size)
+{
+    dt_set_cell(cellp, XEN_FDT_NODE_ADDRESS_CELLS, address);
+    dt_set_cell(cellp, XEN_FDT_NODE_SIZE_CELLS, size);
+}
+
 struct vcpu *__init alloc_dom0_vcpu0(void)
 {
     if ( opt_dom0_max_vcpus == 0 )
@@ -477,8 +495,7 @@ static int make_cpus_node(const struct domain *d, void *fdt,
     return res;
 }
 
-static int make_gic_node(const struct domain *d, void *fdt,
-                         const struct dt_device_node *parent)
+static int make_gic_node(const struct domain *d, void *fdt)
 {
     const struct dt_device_node *gic = dt_interrupt_controller;
     const void *compatible = NULL;
@@ -512,20 +529,19 @@ static int make_gic_node(const struct domain *d, void *fdt,
     if ( res )
         return res;
 
-    len = dt_cells_to_size(dt_n_addr_cells(parent) + dt_n_size_cells(parent));
-    len *= 2;
-    new_cells = xzalloc_bytes(dt_cells_to_size(len));
+    len = XEN_FDT_NODE_REG_SIZE * 2;
+    new_cells = xzalloc_bytes(len);
     if ( new_cells == NULL )
         return -FDT_ERR_XEN(ENOMEM);
 
     tmp = new_cells;
     DPRINT("  Set Distributor Base 0x%"PRIpaddr"-0x%"PRIpaddr"\n",
            d->arch.vgic.dbase, d->arch.vgic.dbase + PAGE_SIZE - 1);
-    dt_set_range(&tmp, parent, d->arch.vgic.dbase, PAGE_SIZE);
+    set_xen_range(&tmp, d->arch.vgic.dbase, PAGE_SIZE);
 
-    DPRINT("  Set Cpu Base 0x%"PRIpaddr" size = 0x%"PRIpaddr"\n",
+    DPRINT("  Set Cpu Base 0x%"PRIpaddr"-0x%"PRIpaddr"\n",
            d->arch.vgic.cbase, d->arch.vgic.cbase + (PAGE_SIZE * 2) - 1);
-    dt_set_range(&tmp, parent, d->arch.vgic.cbase, PAGE_SIZE * 2);
+    set_xen_range(&tmp, d->arch.vgic.cbase, PAGE_SIZE * 2);
 
     res = fdt_property(fdt, "reg", new_cells, len);
     xfree(new_cells);
@@ -550,8 +566,7 @@ static int make_gic_node(const struct domain *d, void *fdt,
     return res;
 }
 
-static int make_timer_node(const struct domain *d, void *fdt,
-                           const struct dt_device_node *parent)
+static int make_timer_node(const struct domain *d, void *fdt)
 {
     static const struct dt_device_match timer_ids[] __initconst =
     {
@@ -611,6 +626,41 @@ static int make_timer_node(const struct domain *d, void *fdt,
     return res;
 }
 
+static int make_xen_node(const struct domain *d, void *fdt,
+                         const struct dt_device_node *parent)
+{
+    int res;
+
+    res = fdt_begin_node(fdt, "xen");
+    if ( res )
+        return res;
+
+    res = fdt_property_cell(fdt, "#address-cells",
+                            XEN_FDT_NODE_ADDRESS_CELLS);
+    if ( res )
+        return res;
+
+    res = fdt_property_cell(fdt, "#size-cells",
+                            XEN_FDT_NODE_SIZE_CELLS);
+    if ( res )
+        return res;
+
+    res = fdt_property(fdt, "ranges", NULL, 0);
+    if ( res )
+        return res;
+
+    res = make_gic_node(d, fdt);
+    if ( res )
+        return res;
+
+    res = make_timer_node(d, fdt);
+    if ( res )
+        return res;
+
+    res = fdt_end_node(fdt);
+    return res;
+}
+
 /* Map the device in the domain */
 static int map_device(struct domain *d, const struct dt_device_node *dev)
 {
@@ -776,11 +826,7 @@ static int handle_node(struct domain *d, struct kernel_info *kinfo,
         if ( res )
             return res;
 
-        res = make_gic_node(d, kinfo->fdt, np);
-        if ( res )
-            return res;
-
-        res = make_timer_node(d, kinfo->fdt, np);
+        res = make_xen_node(d, kinfo->fdt, np);
         if ( res )
             return res;
     }
-- 
1.7.10.4

Re: [PATCH v5 06/19] xen: arm: move dom0 gic and timer device tree nodes under /xen/

[Stefano Stabellini]

On Wed, 13 Nov 2013, Ian Campbell wrote:
> Julien observed that we were relying on the provided host DTB supplying
> suitable #address-cells and #size-cells values to allow us to represent these
> addresses, which may not reliably be the case. Moving these under our own
> known (somewhat analogous to the use of /soc/ or /motherboard/ on some
> platforms) allows us to control these sizes.
> 
> Since the xen node is created out of thin air it does not have a corresponding
> struct dt_device_node and therefore we cannot use dt_n_addr_cells or
> dt_n_size_cells, we can use hardcoded constants instead. For the same reason
> we define and use set_xen_range instead of dt_set_range.
> 
> The hypervisor, cpus and psci node all either defined #foo-cells for their
> children or do not contain reg properties and therefore can remain at the top
> level.
> 
> The logging in make_gic_node was inconsistent. Fix it.

I think that using the string "xen" as node name is confusing.
I would use something more informative, as "xen-motherboard" or
"xen-virtual-devices", so that people can guess how it differs from
the hypervisor node.



> Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
> ---
> v5: New patch
> ---
>  xen/arch/arm/domain_build.c |   76 ++++++++++++++++++++++++++++++++++---------
>  1 file changed, 61 insertions(+), 15 deletions(-)
> 
> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> index 186746c..8645aa1 100644
> --- a/xen/arch/arm/domain_build.c
> +++ b/xen/arch/arm/domain_build.c
> @@ -48,6 +48,24 @@ custom_param("dom0_mem", parse_dom0_mem);
>   */
>  #define DOM0_FDT_EXTRA_SIZE (128 + sizeof(struct fdt_reserve_entry))
>  
> +/*
> + * Number of cells used for addresses and sizes under the /xen/
> + * node.
> + *
> + * We don't have a struct dt_device_node we can reference as a parent
> + * to get these values, so use these constants instead.
> + */
> +#define XEN_FDT_NODE_ADDRESS_CELLS 2
> +#define XEN_FDT_NODE_SIZE_CELLS 2
> +#define XEN_FDT_NODE_REG_SIZE \
> +    dt_cells_to_size(XEN_FDT_NODE_ADDRESS_CELLS + XEN_FDT_NODE_SIZE_CELLS)
> +
> +static void set_xen_range(__be32 **cellp, u64 address, u64 size)
> +{
> +    dt_set_cell(cellp, XEN_FDT_NODE_ADDRESS_CELLS, address);
> +    dt_set_cell(cellp, XEN_FDT_NODE_SIZE_CELLS, size);
> +}
> +
>  struct vcpu *__init alloc_dom0_vcpu0(void)
>  {
>      if ( opt_dom0_max_vcpus == 0 )
> @@ -477,8 +495,7 @@ static int make_cpus_node(const struct domain *d, void *fdt,
>      return res;
>  }
>  
> -static int make_gic_node(const struct domain *d, void *fdt,
> -                         const struct dt_device_node *parent)
> +static int make_gic_node(const struct domain *d, void *fdt)
>  {
>      const struct dt_device_node *gic = dt_interrupt_controller;
>      const void *compatible = NULL;
> @@ -512,20 +529,19 @@ static int make_gic_node(const struct domain *d, void *fdt,
>      if ( res )
>          return res;
>  
> -    len = dt_cells_to_size(dt_n_addr_cells(parent) + dt_n_size_cells(parent));
> -    len *= 2;
> -    new_cells = xzalloc_bytes(dt_cells_to_size(len));
> +    len = XEN_FDT_NODE_REG_SIZE * 2;
> +    new_cells = xzalloc_bytes(len);
>      if ( new_cells == NULL )
>          return -FDT_ERR_XEN(ENOMEM);
>  
>      tmp = new_cells;
>      DPRINT("  Set Distributor Base 0x%"PRIpaddr"-0x%"PRIpaddr"\n",
>             d->arch.vgic.dbase, d->arch.vgic.dbase + PAGE_SIZE - 1);
> -    dt_set_range(&tmp, parent, d->arch.vgic.dbase, PAGE_SIZE);
> +    set_xen_range(&tmp, d->arch.vgic.dbase, PAGE_SIZE);
>  
> -    DPRINT("  Set Cpu Base 0x%"PRIpaddr" size = 0x%"PRIpaddr"\n",
> +    DPRINT("  Set Cpu Base 0x%"PRIpaddr"-0x%"PRIpaddr"\n",
>             d->arch.vgic.cbase, d->arch.vgic.cbase + (PAGE_SIZE * 2) - 1);
> -    dt_set_range(&tmp, parent, d->arch.vgic.cbase, PAGE_SIZE * 2);
> +    set_xen_range(&tmp, d->arch.vgic.cbase, PAGE_SIZE * 2);
>  
>      res = fdt_property(fdt, "reg", new_cells, len);
>      xfree(new_cells);
> @@ -550,8 +566,7 @@ static int make_gic_node(const struct domain *d, void *fdt,
>      return res;
>  }
>  
> -static int make_timer_node(const struct domain *d, void *fdt,
> -                           const struct dt_device_node *parent)
> +static int make_timer_node(const struct domain *d, void *fdt)
>  {
>      static const struct dt_device_match timer_ids[] __initconst =
>      {
> @@ -611,6 +626,41 @@ static int make_timer_node(const struct domain *d, void *fdt,
>      return res;
>  }
>  
> +static int make_xen_node(const struct domain *d, void *fdt,
> +                         const struct dt_device_node *parent)
> +{
> +    int res;
> +
> +    res = fdt_begin_node(fdt, "xen");
> +    if ( res )
> +        return res;
> +
> +    res = fdt_property_cell(fdt, "#address-cells",
> +                            XEN_FDT_NODE_ADDRESS_CELLS);
> +    if ( res )
> +        return res;
> +
> +    res = fdt_property_cell(fdt, "#size-cells",
> +                            XEN_FDT_NODE_SIZE_CELLS);
> +    if ( res )
> +        return res;
> +
> +    res = fdt_property(fdt, "ranges", NULL, 0);
> +    if ( res )
> +        return res;
> +
> +    res = make_gic_node(d, fdt);
> +    if ( res )
> +        return res;
> +
> +    res = make_timer_node(d, fdt);
> +    if ( res )
> +        return res;
> +
> +    res = fdt_end_node(fdt);
> +    return res;
> +}
> +
>  /* Map the device in the domain */
>  static int map_device(struct domain *d, const struct dt_device_node *dev)
>  {
> @@ -776,11 +826,7 @@ static int handle_node(struct domain *d, struct kernel_info *kinfo,
>          if ( res )
>              return res;
>  
> -        res = make_gic_node(d, kinfo->fdt, np);
> -        if ( res )
> -            return res;
> -
> -        res = make_timer_node(d, kinfo->fdt, np);
> +        res = make_xen_node(d, kinfo->fdt, np);
>          if ( res )
>              return res;
>      }
> -- 
> 1.7.10.4
> 

Re: [PATCH v5 06/19] xen: arm: move dom0 gic and timer device tree nodes under /xen/

[Ian Campbell]

On Wed, 2013-11-13 at 18:28 +0000, Stefano Stabellini wrote:
> On Wed, 13 Nov 2013, Ian Campbell wrote:
> > Julien observed that we were relying on the provided host DTB supplying
> > suitable #address-cells and #size-cells values to allow us to represent these
> > addresses, which may not reliably be the case. Moving these under our own
> > known (somewhat analogous to the use of /soc/ or /motherboard/ on some
> > platforms) allows us to control these sizes.
> > 
> > Since the xen node is created out of thin air it does not have a corresponding
> > struct dt_device_node and therefore we cannot use dt_n_addr_cells or
> > dt_n_size_cells, we can use hardcoded constants instead. For the same reason
> > we define and use set_xen_range instead of dt_set_range.
> > 
> > The hypervisor, cpus and psci node all either defined #foo-cells for their
> > children or do not contain reg properties and therefore can remain at the top
> > level.
> > 
> > The logging in make_gic_node was inconsistent. Fix it.
> 
> I think that using the string "xen" as node name is confusing.
> I would use something more informative, as "xen-motherboard" or
> "xen-virtual-devices", so that people can guess how it differs from
> the hypervisor node.

The content is the GIC, timer and memory, so neither xen-motherboard nor
xen-virtual-devices sound especially relevant.

No one will ever see this in practice, unless they are deliberately
looking to debug it, since it is built on the fly and only for dom0.

TBH, I think /hypervisor/xen would have been fine.

"xen-system"? Still kinda lame.

Ian.

Re: [PATCH v5 06/19] xen: arm: move dom0 gic and timer device tree nodes under /xen/

[Stefano Stabellini]

On Wed, 13 Nov 2013, Ian Campbell wrote:
> On Wed, 2013-11-13 at 18:28 +0000, Stefano Stabellini wrote:
> > On Wed, 13 Nov 2013, Ian Campbell wrote:
> > > Julien observed that we were relying on the provided host DTB supplying
> > > suitable #address-cells and #size-cells values to allow us to represent these
> > > addresses, which may not reliably be the case. Moving these under our own
> > > known (somewhat analogous to the use of /soc/ or /motherboard/ on some
> > > platforms) allows us to control these sizes.
> > > 
> > > Since the xen node is created out of thin air it does not have a corresponding
> > > struct dt_device_node and therefore we cannot use dt_n_addr_cells or
> > > dt_n_size_cells, we can use hardcoded constants instead. For the same reason
> > > we define and use set_xen_range instead of dt_set_range.
> > > 
> > > The hypervisor, cpus and psci node all either defined #foo-cells for their
> > > children or do not contain reg properties and therefore can remain at the top
> > > level.
> > > 
> > > The logging in make_gic_node was inconsistent. Fix it.
> > 
> > I think that using the string "xen" as node name is confusing.
> > I would use something more informative, as "xen-motherboard" or
> > "xen-virtual-devices", so that people can guess how it differs from
> > the hypervisor node.
> 
> The content is the GIC, timer and memory, so neither xen-motherboard nor
> xen-virtual-devices sound especially relevant.
> 
> No one will ever see this in practice, unless they are deliberately
> looking to debug it, since it is built on the fly and only for dom0.
> 
> TBH, I think /hypervisor/xen would have been fine.
> 
> "xen-system"? Still kinda lame.

xen-core-devices ?

Re: [PATCH v5 06/19] xen: arm: move dom0 gic and timer device tree nodes under /xen/

[Ian Campbell]

On Thu, 2013-11-14 at 12:22 +0000, Stefano Stabellini wrote:
> On Wed, 13 Nov 2013, Ian Campbell wrote:
> > On Wed, 2013-11-13 at 18:28 +0000, Stefano Stabellini wrote:
> > > On Wed, 13 Nov 2013, Ian Campbell wrote:
> > > > Julien observed that we were relying on the provided host DTB supplying
> > > > suitable #address-cells and #size-cells values to allow us to represent these
> > > > addresses, which may not reliably be the case. Moving these under our own
> > > > known (somewhat analogous to the use of /soc/ or /motherboard/ on some
> > > > platforms) allows us to control these sizes.
> > > > 
> > > > Since the xen node is created out of thin air it does not have a corresponding
> > > > struct dt_device_node and therefore we cannot use dt_n_addr_cells or
> > > > dt_n_size_cells, we can use hardcoded constants instead. For the same reason
> > > > we define and use set_xen_range instead of dt_set_range.
> > > > 
> > > > The hypervisor, cpus and psci node all either defined #foo-cells for their
> > > > children or do not contain reg properties and therefore can remain at the top
> > > > level.
> > > > 
> > > > The logging in make_gic_node was inconsistent. Fix it.
> > > 
> > > I think that using the string "xen" as node name is confusing.
> > > I would use something more informative, as "xen-motherboard" or
> > > "xen-virtual-devices", so that people can guess how it differs from
> > > the hypervisor node.
> > 
> > The content is the GIC, timer and memory, so neither xen-motherboard nor
> > xen-virtual-devices sound especially relevant.
> > 
> > No one will ever see this in practice, unless they are deliberately
> > looking to debug it, since it is built on the fly and only for dom0.
> > 
> > TBH, I think /hypervisor/xen would have been fine.
> > 
> > "xen-system"? Still kinda lame.
> 
> xen-core-devices ?

OK, lets go with that.

Ian.

Re: [PATCH v5 06/19] xen: arm: move dom0 gic and timer device tree nodes under /xen/

[Julien Grall]

On 11/13/2013 06:11 PM, Ian Campbell wrote:
> Julien observed that we were relying on the provided host DTB supplying
> suitable #address-cells and #size-cells values to allow us to represent these
> addresses, which may not reliably be the case. Moving these under our own
> known (somewhat analogous to the use of /soc/ or /motherboard/ on some
> platforms) allows us to control these sizes.
>
> Since the xen node is created out of thin air it does not have a corresponding
> struct dt_device_node and therefore we cannot use dt_n_addr_cells or
> dt_n_size_cells, we can use hardcoded constants instead. For the same reason
> we define and use set_xen_range instead of dt_set_range.
>
> The hypervisor, cpus and psci node all either defined #foo-cells for their
> children or do not contain reg properties and therefore can remain at the top
> level.
>
> The logging in make_gic_node was inconsistent. Fix it.
>
> Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
> ---
> v5: New patch
> ---
>   xen/arch/arm/domain_build.c |   76 ++++++++++++++++++++++++++++++++++---------
>   1 file changed, 61 insertions(+), 15 deletions(-)
>
> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> index 186746c..8645aa1 100644
> --- a/xen/arch/arm/domain_build.c
> +++ b/xen/arch/arm/domain_build.c
> @@ -48,6 +48,24 @@ custom_param("dom0_mem", parse_dom0_mem);
>    */
>   #define DOM0_FDT_EXTRA_SIZE (128 + sizeof(struct fdt_reserve_entry))

Don't we need to update DOM0_FDT_EXTRA_SIZE?

[..]

-- 
Julien Grall

Re: [PATCH v5 06/19] xen: arm: move dom0 gic and timer device tree nodes under /xen/

[Ian Campbell]

On Wed, 2013-11-13 at 21:31 +0000, Julien Grall wrote:
> > diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> > index 186746c..8645aa1 100644
> > --- a/xen/arch/arm/domain_build.c
> > +++ b/xen/arch/arm/domain_build.c
> > @@ -48,6 +48,24 @@ custom_param("dom0_mem", parse_dom0_mem);
> >    */
> >   #define DOM0_FDT_EXTRA_SIZE (128 + sizeof(struct fdt_reserve_entry))
> 
> Don't we need to update DOM0_FDT_EXTRA_SIZE?

Nothing is complaining about running out of space, so I guess the
existing 128 slack remains sufficient. We don't actually add all that
much -- most of what we add was previously removed (GIC, timer, memory
nodes etc).

Ian.

[PATCH v5 07/19] xen: arm: allocate dom0 memory separately from preparing the dtb

[Ian Campbell]

Mixing these two together is a pain, it forces us to prepare the dtb before
processing the kernel which means we don't know whether the guest is 32- or
64-bit while we construct its DTB.

Instead split out the memory allocation (including 1:1 workaround handling)
and p2m setup into a separate phase and then create a memory node in the DTB
based on the result.

This allows us to move kernel parsing before DTB setup.

As part of this it was also necessary to rework where the decision regarding
the placement of the DTB and initrd in RAM was made. It is now made when
loading the kernel, which allows it to make use of the zImage/ELF specific
information and therefore to make decisions based on complete knowledge and do
it right rather than guessing in prepare_dtb and relying on a later check to
see if things worked.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
---
v3: Also rework module placement, v2 broke boot because dtb_paddr wasn't set
    soon enough. This ends up cleaner anyway.
v2: Fixed typo in the commit log
    Handle multiple memory nodes as well as individual nodes with several
    entries in them.
    Strip the original memory node and recreate rather than trying to modify.
---
 xen/arch/arm/domain_build.c |  203 ++++++++++++++++++++++---------------------
 xen/arch/arm/kernel.c       |   80 +++++++++++------
 xen/arch/arm/kernel.h       |    2 -
 3 files changed, 158 insertions(+), 127 deletions(-)

diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index 8645aa1..edfcf14 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -81,11 +81,8 @@ struct vcpu *__init alloc_dom0_vcpu0(void)
     return alloc_vcpu(dom0, 0, 0);
 }
 
-static int set_memory_reg_11(struct domain *d, struct kernel_info *kinfo,
-                             const struct dt_property *pp,
-                             const struct dt_device_node *np, __be32 *new_cell)
+static void allocate_memory_11(struct domain *d, struct kernel_info *kinfo)
 {
-    int reg_size = dt_cells_to_size(dt_n_addr_cells(np) + dt_n_size_cells(np));
     paddr_t start;
     paddr_t size;
     struct page_info *pg = NULL;
@@ -116,53 +113,61 @@ static int set_memory_reg_11(struct domain *d, struct kernel_info *kinfo,
     if ( res )
         panic("Unable to add pages in DOM0: %d\n", res);
 
-    dt_set_range(&new_cell, np, start, size);
-
     kinfo->mem.bank[0].start = start;
     kinfo->mem.bank[0].size = size;
     kinfo->mem.nr_banks = 1;
 
-    return reg_size;
+    kinfo->unassigned_mem -= size;
 }
 
-static int set_memory_reg(struct domain *d, struct kernel_info *kinfo,
-                          const struct dt_property *pp,
-                          const struct dt_device_node *np, __be32 *new_cell)
+static void allocate_memory(struct domain *d, struct kernel_info *kinfo)
 {
-    int reg_size = dt_cells_to_size(dt_n_addr_cells(np) + dt_n_size_cells(np));
-    int l = 0;
+
+    struct dt_device_node *memory = NULL;
+    const void *reg;
+    u32 reg_len, reg_size;
     unsigned int bank = 0;
-    u64 start;
-    u64 size;
-    int ret;
 
     if ( platform_has_quirk(PLATFORM_QUIRK_DOM0_MAPPING_11) )
-        return set_memory_reg_11(d, kinfo, pp, np, new_cell);
+        return allocate_memory_11(d, kinfo);
 
-    while ( kinfo->unassigned_mem > 0 && l + reg_size <= pp->length
-            && kinfo->mem.nr_banks < NR_MEM_BANKS )
+    while ( (memory = dt_find_node_by_type(memory, "memory")) )
     {
-        ret = dt_device_get_address(np, bank, &start, &size);
-        if ( ret )
-            panic("Unable to retrieve the bank %u for %s\n",
-                  bank, dt_node_full_name(np));
-
-        if ( size > kinfo->unassigned_mem )
-            size = kinfo->unassigned_mem;
-        dt_set_range(&new_cell, np, start, size);
-
-        printk("Populate P2M %#"PRIx64"->%#"PRIx64"\n", start, start + size);
-        if ( p2m_populate_ram(d, start, start + size) < 0 )
-            panic("Failed to populate P2M\n");
-        kinfo->mem.bank[kinfo->mem.nr_banks].start = start;
-        kinfo->mem.bank[kinfo->mem.nr_banks].size = size;
-        kinfo->mem.nr_banks++;
-        kinfo->unassigned_mem -= size;
-
-        l += reg_size;
-    }
+        int l;
+
+        DPRINT("memory node\n");
+
+        reg_size = dt_cells_to_size(dt_n_addr_cells(memory) + dt_n_size_cells(memory));
+
+        reg = dt_get_property(memory, "reg", &reg_len);
+        if ( reg == NULL )
+            panic("Memory node has no reg property!\n");
+
+        for ( l = 0;
+              kinfo->unassigned_mem > 0 && l + reg_size <= reg_len
+                  && kinfo->mem.nr_banks < NR_MEM_BANKS;
+              l += reg_size )
+        {
+            paddr_t start, size;
 
-    return l;
+            if ( dt_device_get_address(memory, bank, &start, &size) )
+                panic("Unable to retrieve the bank %u for %s\n",
+                      bank, dt_node_full_name(memory));
+
+            if ( size > kinfo->unassigned_mem )
+                size = kinfo->unassigned_mem;
+
+            printk("Populate P2M %#"PRIx64"->%#"PRIx64"\n",
+                   start, start + size);
+            if ( p2m_populate_ram(d, start, start + size) < 0 )
+                panic("Failed to populate P2M\n");
+            kinfo->mem.bank[kinfo->mem.nr_banks].start = start;
+            kinfo->mem.bank[kinfo->mem.nr_banks].size = size;
+            kinfo->mem.nr_banks++;
+
+            kinfo->unassigned_mem -= size;
+        }
+    }
 }
 
 static int write_properties(struct domain *d, struct kernel_info *kinfo,
@@ -211,23 +216,6 @@ static int write_properties(struct domain *d, struct kernel_info *kinfo,
                 continue;
             }
         }
-        /*
-         * In a memory node: adjust reg property.
-         * TODO: handle properly memory node (ie: device_type = "memory")
-         */
-        else if ( dt_node_name_is_equal(np, "memory") )
-        {
-            if ( dt_property_name_is_equal(pp, "reg") )
-            {
-                new_data = xzalloc_bytes(pp->length);
-                if ( new_data  == NULL )
-                    return -FDT_ERR_XEN(ENOMEM);
-
-                prop_len = set_memory_reg(d, kinfo, pp, np,
-                                          (__be32 *)new_data);
-                prop_data = new_data;
-            }
-        }
 
         res = fdt_property(kinfo->fdt, pp->name, prop_data, prop_len);
 
@@ -304,6 +292,46 @@ static int fdt_property_interrupts(void *fdt, gic_interrupt_t *intr,
     return res;
 }
 
+static int make_memory_node(const struct domain *d,
+                            void *fdt,
+                            const struct kernel_info *kinfo)
+{
+    int res, i;
+    int nr_cells = XEN_FDT_NODE_REG_SIZE*kinfo->mem.nr_banks;
+    __be32 reg[nr_cells];
+    __be32 *cells;
+
+    DPRINT("Create memory node\n");
+
+    /* ePAPR 3.4 */
+    res = fdt_begin_node(fdt, "memory");
+    if ( res )
+        return res;
+
+    res = fdt_property_string(fdt, "device_type", "memory");
+    if ( res )
+        return res;
+
+    cells = &reg[0];
+    for ( i = 0 ; i < kinfo->mem.nr_banks; i++ )
+    {
+        u64 start = kinfo->mem.bank[i].start;
+        u64 size = kinfo->mem.bank[i].size;
+
+        DPRINT("  Bank %d: %#"PRIx64"->%#"PRIx64"\n",
+                i, start, start + size);
+
+        set_xen_range(&cells, start, size);
+    }
+
+    res = fdt_property(fdt, "reg", reg, nr_cells);
+    if ( res )
+        return res;
+
+    res = fdt_end_node(fdt);
+
+    return res;
+}
 
 static int make_hypervisor_node(void *fdt, const struct dt_device_node *parent)
 {
@@ -627,7 +655,8 @@ static int make_timer_node(const struct domain *d, void *fdt)
 }
 
 static int make_xen_node(const struct domain *d, void *fdt,
-                         const struct dt_device_node *parent)
+                         const struct dt_device_node *parent,
+                         const struct kernel_info *kinfo)
 {
     int res;
 
@@ -649,6 +678,10 @@ static int make_xen_node(const struct domain *d, void *fdt,
     if ( res )
         return res;
 
+    res = make_memory_node(d, fdt, kinfo);
+    if ( res )
+        return res;
+
     res = make_gic_node(d, fdt);
     if ( res )
         return res;
@@ -750,6 +783,7 @@ static int handle_node(struct domain *d, struct kernel_info *kinfo,
         DT_MATCH_COMPATIBLE("xen,multiboot-module"),
         DT_MATCH_COMPATIBLE("arm,psci"),
         DT_MATCH_PATH("/cpus"),
+        DT_MATCH_TYPE("memory"),
         DT_MATCH_GIC,
         DT_MATCH_TIMER,
         { /* sentinel */ },
@@ -826,7 +860,7 @@ static int handle_node(struct domain *d, struct kernel_info *kinfo,
         if ( res )
             return res;
 
-        res = make_xen_node(d, kinfo->fdt, np);
+        res = make_xen_node(d, kinfo->fdt, np, kinfo);
         if ( res )
             return res;
     }
@@ -841,14 +875,9 @@ static int prepare_dtb(struct domain *d, struct kernel_info *kinfo)
     const void *fdt;
     int new_size;
     int ret;
-    paddr_t end;
-    paddr_t initrd_len;
-    paddr_t dtb_len;
 
     ASSERT(dt_host && (dt_host->sibling == NULL));
 
-    kinfo->unassigned_mem = dom0_mem;
-
     fdt = device_tree_flattened;
 
     new_size = fdt_totalsize(fdt) + DOM0_FDT_EXTRA_SIZE;
@@ -870,36 +899,6 @@ static int prepare_dtb(struct domain *d, struct kernel_info *kinfo)
     if ( ret < 0 )
         goto err;
 
-    /* Align DTB and initrd size to 2Mb. Linux only requires 4 byte alignment */
-    initrd_len = ROUNDUP(early_info.modules.module[MOD_INITRD].size, MB(2));
-    dtb_len = ROUNDUP(fdt_totalsize(kinfo->fdt), MB(2));
-    new_size = initrd_len + dtb_len;
-
-    /*
-     * DTB must be loaded such that it does not conflict with the
-     * kernel decompressor. For 32-bit Linux Documentation/arm/Booting
-     * recommends just after the 128MB boundary while for 64-bit Linux
-     * the recommendation in Documentation/arm64/booting.txt is below
-     * 512MB. Place at 128MB, (or, if we have less RAM, as high as
-     * possible) in order to satisfy both.
-     * If the bootloader provides an initrd, it will be loaded just
-     * after the DTB.
-     */
-    end = kinfo->mem.bank[0].start + kinfo->mem.bank[0].size;
-    end = MIN(kinfo->mem.bank[0].start + (128<<20) + new_size, end);
-
-    kinfo->initrd_paddr = end - initrd_len;
-    kinfo->dtb_paddr = kinfo->initrd_paddr - dtb_len;
-
-    if ( kinfo->dtb_paddr < kinfo->mem.bank[0].start ||
-         kinfo->mem.bank[0].start + new_size > end )
-    {
-        printk(XENLOG_ERR "Not enough memory in the first bank for "
-               "the device tree.");
-        ret = -FDT_ERR_XEN(EINVAL);
-        goto err;
-    }
-
     return 0;
 
   err:
@@ -994,11 +993,19 @@ int construct_dom0(struct domain *d)
 
     d->max_pages = ~0U;
 
-    rc = prepare_dtb(d, &kinfo);
+    kinfo.unassigned_mem = dom0_mem;
+
+    allocate_memory(d, &kinfo);
+
+    rc = kernel_prepare(&kinfo);
     if ( rc < 0 )
         return rc;
 
-    rc = kernel_prepare(&kinfo);
+#ifdef CONFIG_ARM_64
+    d->arch.type = kinfo.type;
+#endif
+
+    rc = prepare_dtb(d, &kinfo);
     if ( rc < 0 )
         return rc;
 
@@ -1006,9 +1013,6 @@ int construct_dom0(struct domain *d)
     if ( rc < 0 )
         return rc;
 
-    if ( kinfo.check_overlap )
-        kinfo.check_overlap(&kinfo);
-
     /* The following loads use the domain's p2m */
     p2m_load_VTTBR(d);
 #ifdef CONFIG_ARM_64
@@ -1019,6 +1023,10 @@ int construct_dom0(struct domain *d)
         WRITE_SYSREG(READ_SYSREG(HCR_EL2) | HCR_RW, HCR_EL2);
 #endif
 
+    /*
+     * kernel_load will determine the placement of the initrd & fdt in
+     * RAM, so call it first.
+     */
     kernel_load(&kinfo);
     /* initrd_load will fix up the fdt, so call it before dtb_load */
     initrd_load(&kinfo);
@@ -1033,7 +1041,6 @@ int construct_dom0(struct domain *d)
 
     regs->pc = (register_t)kinfo.entry;
 
-
     if ( is_pv32_domain(d) )
     {
         regs->cpsr = PSR_GUEST32_INIT;
diff --git a/xen/arch/arm/kernel.c b/xen/arch/arm/kernel.c
index 7036d94..9046797 100644
--- a/xen/arch/arm/kernel.c
+++ b/xen/arch/arm/kernel.c
@@ -68,26 +68,56 @@ void copy_from_paddr(void *dst, paddr_t paddr, unsigned long len, int attrindx)
     clear_fixmap(FIXMAP_MISC);
 }
 
-static void kernel_zimage_check_overlap(struct kernel_info *info)
+static void place_modules(struct kernel_info *info,
+                         paddr_t kernel_start,
+                         paddr_t kernel_end)
 {
-    paddr_t zimage_start = info->zimage.load_addr;
-    paddr_t zimage_end = info->zimage.load_addr + info->zimage.len;
-    paddr_t start = info->dtb_paddr;
-    paddr_t end;
+    /* Align DTB and initrd size to 2Mb. Linux only requires 4 byte alignment */
+    const paddr_t initrd_len =
+        ROUNDUP(early_info.modules.module[MOD_INITRD].size, MB(2));
+    const paddr_t dtb_len = ROUNDUP(fdt_totalsize(info->fdt), MB(2));
+    const paddr_t total = initrd_len + dtb_len;
 
-    end = info->initrd_paddr + early_info.modules.module[MOD_INITRD].size;
+    /* Convenient */
+    const paddr_t mem_start = info->mem.bank[0].start;
+    const paddr_t mem_size = info->mem.bank[0].size;
+    const paddr_t mem_end = mem_start + mem_size;
+    const paddr_t kernel_size = kernel_end - kernel_start;
+
+    paddr_t addr;
+
+    if ( total + kernel_size > mem_size )
+        panic("Not enough memory in the first bank for the dtb+initrd.");
 
     /*
-     * In the dom0 memory, the initrd will be just after the DTB. So we
-     * only need to check if the zImage range will overlap the
-     * DTB-initrd range.
+     * DTB must be loaded such that it does not conflict with the
+     * kernel decompressor. For 32-bit Linux Documentation/arm/Booting
+     * recommends just after the 128MB boundary while for 64-bit Linux
+     * the recommendation in Documentation/arm64/booting.txt is below
+     * 512MB.
+     *
+     * If the bootloader provides an initrd, it will be loaded just
+     * after the DTB.
+     *
+     * We try to place dtb+initrd at 128MB, (or, if we have less RAM,
+     * as high as possible). If there is no space then fallback to
+     * just after the kernel, if there is room, otherwise just before.
      */
-    if ( (start > zimage_end) || (end < zimage_start) )
+
+    if ( kernel_end < MIN(mem_start + MB(128), mem_end - total) )
+        addr = MIN(mem_start + MB(128), mem_end - total);
+    else if ( mem_end - ROUNDUP(kernel_end, MB(2)) >= total )
+        addr = ROUNDUP(kernel_end, MB(2));
+    else if ( kernel_start - mem_start >= total )
+        addr = kernel_start - total;
+    else
+    {
+        panic("Unable to find suitable location for dtb+initrd.");
         return;
+    }
 
-    panic(XENLOG_ERR "The kernel(0x%"PRIpaddr"-0x%"PRIpaddr
-          ") is overlapping the DTB-initrd(0x%"PRIpaddr"-0x%"PRIpaddr")\n",
-          zimage_start, zimage_end, start, end);
+    info->dtb_paddr = addr;
+    info->initrd_paddr = info->dtb_paddr + dtb_len;
 }
 
 static void kernel_zimage_load(struct kernel_info *info)
@@ -98,6 +128,8 @@ static void kernel_zimage_load(struct kernel_info *info)
     paddr_t len = info->zimage.len;
     unsigned long offs;
 
+    place_modules(info, load_addr, load_addr + len);
+
     printk("Loading zImage from %"PRIpaddr" to %"PRIpaddr"-%"PRIpaddr"\n",
            paddr, load_addr, load_addr + len);
     for ( offs = 0; offs < len; )
@@ -176,7 +208,6 @@ static int kernel_try_zimage64_prepare(struct kernel_info *info,
 
     info->entry = info->zimage.load_addr;
     info->load = kernel_zimage_load;
-    info->check_overlap = kernel_zimage_check_overlap;
 
     info->type = DOMAIN_PV64;
 
@@ -236,17 +267,9 @@ static int kernel_try_zimage32_prepare(struct kernel_info *info,
         paddr_t load_end;
 
         load_end = info->mem.bank[0].start + info->mem.bank[0].size;
-        load_end = MIN(info->mem.bank[0].start + (128<<20), load_end);
-
-        /*
-         * FDT is loaded above 128M or as high as possible, so the
-         * only way we can clash is if we have <=128MB, in which case
-         * FDT will be right at the end and so dtb_paddr will be below
-         * the proposed kernel load address. Move the kernel down if
-         * necessary.
-         */
-        if ( load_end >= info->dtb_paddr )
-            load_end = info->dtb_paddr;
+        load_end = MIN(info->mem.bank[0].start + MB(128), load_end);
+
+        load_end += MB(2);
 
         info->zimage.load_addr = load_end - end;
         /* Align to 2MB */
@@ -258,7 +281,6 @@ static int kernel_try_zimage32_prepare(struct kernel_info *info,
 
     info->entry = info->zimage.load_addr;
     info->load = kernel_zimage_load;
-    info->check_overlap = kernel_zimage_check_overlap;
 
 #ifdef CONFIG_ARM_64
     info->type = DOMAIN_PV32;
@@ -269,10 +291,15 @@ static int kernel_try_zimage32_prepare(struct kernel_info *info,
 
 static void kernel_elf_load(struct kernel_info *info)
 {
+    place_modules(info,
+                  info->elf.parms.virt_kstart,
+                  info->elf.parms.virt_kend);
+
     printk("Loading ELF image into guest memory\n");
     info->elf.elf.dest_base = (void*)(unsigned long)info->elf.parms.virt_kstart;
     info->elf.elf.dest_size =
          info->elf.parms.virt_kend - info->elf.parms.virt_kstart;
+
     elf_load_binary(&info->elf.elf);
 
     printk("Free temporary kernel buffer\n");
@@ -321,7 +348,6 @@ static int kernel_try_elf_prepare(struct kernel_info *info,
      */
     info->entry = info->elf.parms.virt_entry;
     info->load = kernel_elf_load;
-    info->check_overlap = NULL;
 
     if ( elf_check_broken(&info->elf.elf) )
         printk("Xen: warning: ELF kernel broken: %s\n",
diff --git a/xen/arch/arm/kernel.h b/xen/arch/arm/kernel.h
index debf590..b48c2c9 100644
--- a/xen/arch/arm/kernel.h
+++ b/xen/arch/arm/kernel.h
@@ -40,8 +40,6 @@ struct kernel_info {
     };
 
     void (*load)(struct kernel_info *info);
-    /* Callback to check overlap between the kernel and the device tree */
-    void (*check_overlap)(struct kernel_info *kinfo);
     int load_attr;
 };
 
-- 
1.7.10.4

Re: [PATCH v5 07/19] xen: arm: allocate dom0 memory separately from preparing the dtb

[Stefano Stabellini]

On Wed, 13 Nov 2013, Ian Campbell wrote:
> Mixing these two together is a pain, it forces us to prepare the dtb before
> processing the kernel which means we don't know whether the guest is 32- or
> 64-bit while we construct its DTB.
> 
> Instead split out the memory allocation (including 1:1 workaround handling)
> and p2m setup into a separate phase and then create a memory node in the DTB
> based on the result.
> 
> This allows us to move kernel parsing before DTB setup.
> 
> As part of this it was also necessary to rework where the decision regarding
> the placement of the DTB and initrd in RAM was made. It is now made when
> loading the kernel, which allows it to make use of the zImage/ELF specific
> information and therefore to make decisions based on complete knowledge and do
> it right rather than guessing in prepare_dtb and relying on a later check to
> see if things worked.
> 
> Signed-off-by: Ian Campbell <ian.campbell@citrix.com>

It looks good


> v3: Also rework module placement, v2 broke boot because dtb_paddr wasn't set
>     soon enough. This ends up cleaner anyway.
> v2: Fixed typo in the commit log
>     Handle multiple memory nodes as well as individual nodes with several
>     entries in them.
>     Strip the original memory node and recreate rather than trying to modify.
> ---
>  xen/arch/arm/domain_build.c |  203 ++++++++++++++++++++++---------------------
>  xen/arch/arm/kernel.c       |   80 +++++++++++------
>  xen/arch/arm/kernel.h       |    2 -
>  3 files changed, 158 insertions(+), 127 deletions(-)
> 
> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> index 8645aa1..edfcf14 100644
> --- a/xen/arch/arm/domain_build.c
> +++ b/xen/arch/arm/domain_build.c
> @@ -81,11 +81,8 @@ struct vcpu *__init alloc_dom0_vcpu0(void)
>      return alloc_vcpu(dom0, 0, 0);
>  }
>  
> -static int set_memory_reg_11(struct domain *d, struct kernel_info *kinfo,
> -                             const struct dt_property *pp,
> -                             const struct dt_device_node *np, __be32 *new_cell)
> +static void allocate_memory_11(struct domain *d, struct kernel_info *kinfo)
>  {
> -    int reg_size = dt_cells_to_size(dt_n_addr_cells(np) + dt_n_size_cells(np));
>      paddr_t start;
>      paddr_t size;
>      struct page_info *pg = NULL;
> @@ -116,53 +113,61 @@ static int set_memory_reg_11(struct domain *d, struct kernel_info *kinfo,
>      if ( res )
>          panic("Unable to add pages in DOM0: %d\n", res);
>  
> -    dt_set_range(&new_cell, np, start, size);
> -
>      kinfo->mem.bank[0].start = start;
>      kinfo->mem.bank[0].size = size;
>      kinfo->mem.nr_banks = 1;
>  
> -    return reg_size;
> +    kinfo->unassigned_mem -= size;
>  }
>  
> -static int set_memory_reg(struct domain *d, struct kernel_info *kinfo,
> -                          const struct dt_property *pp,
> -                          const struct dt_device_node *np, __be32 *new_cell)
> +static void allocate_memory(struct domain *d, struct kernel_info *kinfo)
>  {
> -    int reg_size = dt_cells_to_size(dt_n_addr_cells(np) + dt_n_size_cells(np));
> -    int l = 0;
> +
> +    struct dt_device_node *memory = NULL;
> +    const void *reg;
> +    u32 reg_len, reg_size;
>      unsigned int bank = 0;
> -    u64 start;
> -    u64 size;
> -    int ret;
>  
>      if ( platform_has_quirk(PLATFORM_QUIRK_DOM0_MAPPING_11) )
> -        return set_memory_reg_11(d, kinfo, pp, np, new_cell);
> +        return allocate_memory_11(d, kinfo);
>  
> -    while ( kinfo->unassigned_mem > 0 && l + reg_size <= pp->length
> -            && kinfo->mem.nr_banks < NR_MEM_BANKS )
> +    while ( (memory = dt_find_node_by_type(memory, "memory")) )
>      {
> -        ret = dt_device_get_address(np, bank, &start, &size);
> -        if ( ret )
> -            panic("Unable to retrieve the bank %u for %s\n",
> -                  bank, dt_node_full_name(np));
> -
> -        if ( size > kinfo->unassigned_mem )
> -            size = kinfo->unassigned_mem;
> -        dt_set_range(&new_cell, np, start, size);
> -
> -        printk("Populate P2M %#"PRIx64"->%#"PRIx64"\n", start, start + size);
> -        if ( p2m_populate_ram(d, start, start + size) < 0 )
> -            panic("Failed to populate P2M\n");
> -        kinfo->mem.bank[kinfo->mem.nr_banks].start = start;
> -        kinfo->mem.bank[kinfo->mem.nr_banks].size = size;
> -        kinfo->mem.nr_banks++;
> -        kinfo->unassigned_mem -= size;
> -
> -        l += reg_size;
> -    }
> +        int l;
> +
> +        DPRINT("memory node\n");
> +
> +        reg_size = dt_cells_to_size(dt_n_addr_cells(memory) + dt_n_size_cells(memory));
> +
> +        reg = dt_get_property(memory, "reg", &reg_len);
> +        if ( reg == NULL )
> +            panic("Memory node has no reg property!\n");
> +
> +        for ( l = 0;
> +              kinfo->unassigned_mem > 0 && l + reg_size <= reg_len
> +                  && kinfo->mem.nr_banks < NR_MEM_BANKS;
> +              l += reg_size )
> +        {
> +            paddr_t start, size;
>  
> -    return l;
> +            if ( dt_device_get_address(memory, bank, &start, &size) )
> +                panic("Unable to retrieve the bank %u for %s\n",
> +                      bank, dt_node_full_name(memory));
> +
> +            if ( size > kinfo->unassigned_mem )
> +                size = kinfo->unassigned_mem;
> +
> +            printk("Populate P2M %#"PRIx64"->%#"PRIx64"\n",
> +                   start, start + size);
> +            if ( p2m_populate_ram(d, start, start + size) < 0 )
> +                panic("Failed to populate P2M\n");
> +            kinfo->mem.bank[kinfo->mem.nr_banks].start = start;
> +            kinfo->mem.bank[kinfo->mem.nr_banks].size = size;
> +            kinfo->mem.nr_banks++;
> +
> +            kinfo->unassigned_mem -= size;
> +        }
> +    }
>  }
>  
>  static int write_properties(struct domain *d, struct kernel_info *kinfo,
> @@ -211,23 +216,6 @@ static int write_properties(struct domain *d, struct kernel_info *kinfo,
>                  continue;
>              }
>          }
> -        /*
> -         * In a memory node: adjust reg property.
> -         * TODO: handle properly memory node (ie: device_type = "memory")
> -         */
> -        else if ( dt_node_name_is_equal(np, "memory") )
> -        {
> -            if ( dt_property_name_is_equal(pp, "reg") )
> -            {
> -                new_data = xzalloc_bytes(pp->length);
> -                if ( new_data  == NULL )
> -                    return -FDT_ERR_XEN(ENOMEM);
> -
> -                prop_len = set_memory_reg(d, kinfo, pp, np,
> -                                          (__be32 *)new_data);
> -                prop_data = new_data;
> -            }
> -        }
>  
>          res = fdt_property(kinfo->fdt, pp->name, prop_data, prop_len);
>  
> @@ -304,6 +292,46 @@ static int fdt_property_interrupts(void *fdt, gic_interrupt_t *intr,
>      return res;
>  }
>  
> +static int make_memory_node(const struct domain *d,
> +                            void *fdt,
> +                            const struct kernel_info *kinfo)
> +{
> +    int res, i;
> +    int nr_cells = XEN_FDT_NODE_REG_SIZE*kinfo->mem.nr_banks;
> +    __be32 reg[nr_cells];
> +    __be32 *cells;
> +
> +    DPRINT("Create memory node\n");
> +
> +    /* ePAPR 3.4 */
> +    res = fdt_begin_node(fdt, "memory");
> +    if ( res )
> +        return res;
> +
> +    res = fdt_property_string(fdt, "device_type", "memory");
> +    if ( res )
> +        return res;
> +
> +    cells = &reg[0];
> +    for ( i = 0 ; i < kinfo->mem.nr_banks; i++ )
> +    {
> +        u64 start = kinfo->mem.bank[i].start;
> +        u64 size = kinfo->mem.bank[i].size;
> +
> +        DPRINT("  Bank %d: %#"PRIx64"->%#"PRIx64"\n",
> +                i, start, start + size);
> +
> +        set_xen_range(&cells, start, size);
> +    }
> +
> +    res = fdt_property(fdt, "reg", reg, nr_cells);
> +    if ( res )
> +        return res;
> +
> +    res = fdt_end_node(fdt);
> +
> +    return res;
> +}
>  
>  static int make_hypervisor_node(void *fdt, const struct dt_device_node *parent)
>  {
> @@ -627,7 +655,8 @@ static int make_timer_node(const struct domain *d, void *fdt)
>  }
>  
>  static int make_xen_node(const struct domain *d, void *fdt,
> -                         const struct dt_device_node *parent)
> +                         const struct dt_device_node *parent,
> +                         const struct kernel_info *kinfo)
>  {
>      int res;
>  
> @@ -649,6 +678,10 @@ static int make_xen_node(const struct domain *d, void *fdt,
>      if ( res )
>          return res;
>  
> +    res = make_memory_node(d, fdt, kinfo);
> +    if ( res )
> +        return res;
> +
>      res = make_gic_node(d, fdt);
>      if ( res )
>          return res;
> @@ -750,6 +783,7 @@ static int handle_node(struct domain *d, struct kernel_info *kinfo,
>          DT_MATCH_COMPATIBLE("xen,multiboot-module"),
>          DT_MATCH_COMPATIBLE("arm,psci"),
>          DT_MATCH_PATH("/cpus"),
> +        DT_MATCH_TYPE("memory"),
>          DT_MATCH_GIC,
>          DT_MATCH_TIMER,
>          { /* sentinel */ },
> @@ -826,7 +860,7 @@ static int handle_node(struct domain *d, struct kernel_info *kinfo,
>          if ( res )
>              return res;
>  
> -        res = make_xen_node(d, kinfo->fdt, np);
> +        res = make_xen_node(d, kinfo->fdt, np, kinfo);
>          if ( res )
>              return res;
>      }
> @@ -841,14 +875,9 @@ static int prepare_dtb(struct domain *d, struct kernel_info *kinfo)
>      const void *fdt;
>      int new_size;
>      int ret;
> -    paddr_t end;
> -    paddr_t initrd_len;
> -    paddr_t dtb_len;
>  
>      ASSERT(dt_host && (dt_host->sibling == NULL));
>  
> -    kinfo->unassigned_mem = dom0_mem;
> -
>      fdt = device_tree_flattened;
>  
>      new_size = fdt_totalsize(fdt) + DOM0_FDT_EXTRA_SIZE;
> @@ -870,36 +899,6 @@ static int prepare_dtb(struct domain *d, struct kernel_info *kinfo)
>      if ( ret < 0 )
>          goto err;
>  
> -    /* Align DTB and initrd size to 2Mb. Linux only requires 4 byte alignment */
> -    initrd_len = ROUNDUP(early_info.modules.module[MOD_INITRD].size, MB(2));
> -    dtb_len = ROUNDUP(fdt_totalsize(kinfo->fdt), MB(2));
> -    new_size = initrd_len + dtb_len;
> -
> -    /*
> -     * DTB must be loaded such that it does not conflict with the
> -     * kernel decompressor. For 32-bit Linux Documentation/arm/Booting
> -     * recommends just after the 128MB boundary while for 64-bit Linux
> -     * the recommendation in Documentation/arm64/booting.txt is below
> -     * 512MB. Place at 128MB, (or, if we have less RAM, as high as
> -     * possible) in order to satisfy both.
> -     * If the bootloader provides an initrd, it will be loaded just
> -     * after the DTB.
> -     */
> -    end = kinfo->mem.bank[0].start + kinfo->mem.bank[0].size;
> -    end = MIN(kinfo->mem.bank[0].start + (128<<20) + new_size, end);
> -
> -    kinfo->initrd_paddr = end - initrd_len;
> -    kinfo->dtb_paddr = kinfo->initrd_paddr - dtb_len;
> -
> -    if ( kinfo->dtb_paddr < kinfo->mem.bank[0].start ||
> -         kinfo->mem.bank[0].start + new_size > end )
> -    {
> -        printk(XENLOG_ERR "Not enough memory in the first bank for "
> -               "the device tree.");
> -        ret = -FDT_ERR_XEN(EINVAL);
> -        goto err;
> -    }
> -
>      return 0;
>  
>    err:
> @@ -994,11 +993,19 @@ int construct_dom0(struct domain *d)
>  
>      d->max_pages = ~0U;
>  
> -    rc = prepare_dtb(d, &kinfo);
> +    kinfo.unassigned_mem = dom0_mem;
> +
> +    allocate_memory(d, &kinfo);
> +
> +    rc = kernel_prepare(&kinfo);
>      if ( rc < 0 )
>          return rc;
>  
> -    rc = kernel_prepare(&kinfo);
> +#ifdef CONFIG_ARM_64
> +    d->arch.type = kinfo.type;
> +#endif
> +
> +    rc = prepare_dtb(d, &kinfo);
>      if ( rc < 0 )
>          return rc;
>  
> @@ -1006,9 +1013,6 @@ int construct_dom0(struct domain *d)
>      if ( rc < 0 )
>          return rc;
>  
> -    if ( kinfo.check_overlap )
> -        kinfo.check_overlap(&kinfo);
> -
>      /* The following loads use the domain's p2m */
>      p2m_load_VTTBR(d);
>  #ifdef CONFIG_ARM_64
> @@ -1019,6 +1023,10 @@ int construct_dom0(struct domain *d)
>          WRITE_SYSREG(READ_SYSREG(HCR_EL2) | HCR_RW, HCR_EL2);
>  #endif
>  
> +    /*
> +     * kernel_load will determine the placement of the initrd & fdt in
> +     * RAM, so call it first.
> +     */
>      kernel_load(&kinfo);
>      /* initrd_load will fix up the fdt, so call it before dtb_load */
>      initrd_load(&kinfo);
> @@ -1033,7 +1041,6 @@ int construct_dom0(struct domain *d)
>  
>      regs->pc = (register_t)kinfo.entry;
>  
> -
>      if ( is_pv32_domain(d) )
>      {
>          regs->cpsr = PSR_GUEST32_INIT;
> diff --git a/xen/arch/arm/kernel.c b/xen/arch/arm/kernel.c
> index 7036d94..9046797 100644
> --- a/xen/arch/arm/kernel.c
> +++ b/xen/arch/arm/kernel.c
> @@ -68,26 +68,56 @@ void copy_from_paddr(void *dst, paddr_t paddr, unsigned long len, int attrindx)
>      clear_fixmap(FIXMAP_MISC);
>  }
>  
> -static void kernel_zimage_check_overlap(struct kernel_info *info)
> +static void place_modules(struct kernel_info *info,
> +                         paddr_t kernel_start,
> +                         paddr_t kernel_end)
>  {
> -    paddr_t zimage_start = info->zimage.load_addr;
> -    paddr_t zimage_end = info->zimage.load_addr + info->zimage.len;
> -    paddr_t start = info->dtb_paddr;
> -    paddr_t end;
> +    /* Align DTB and initrd size to 2Mb. Linux only requires 4 byte alignment */
> +    const paddr_t initrd_len =
> +        ROUNDUP(early_info.modules.module[MOD_INITRD].size, MB(2));
> +    const paddr_t dtb_len = ROUNDUP(fdt_totalsize(info->fdt), MB(2));
> +    const paddr_t total = initrd_len + dtb_len;
>  
> -    end = info->initrd_paddr + early_info.modules.module[MOD_INITRD].size;
> +    /* Convenient */
> +    const paddr_t mem_start = info->mem.bank[0].start;
> +    const paddr_t mem_size = info->mem.bank[0].size;
> +    const paddr_t mem_end = mem_start + mem_size;
> +    const paddr_t kernel_size = kernel_end - kernel_start;
> +
> +    paddr_t addr;
> +
> +    if ( total + kernel_size > mem_size )
> +        panic("Not enough memory in the first bank for the dtb+initrd.");
>  
>      /*
> -     * In the dom0 memory, the initrd will be just after the DTB. So we
> -     * only need to check if the zImage range will overlap the
> -     * DTB-initrd range.
> +     * DTB must be loaded such that it does not conflict with the
> +     * kernel decompressor. For 32-bit Linux Documentation/arm/Booting
> +     * recommends just after the 128MB boundary while for 64-bit Linux
> +     * the recommendation in Documentation/arm64/booting.txt is below
> +     * 512MB.
> +     *
> +     * If the bootloader provides an initrd, it will be loaded just
> +     * after the DTB.
> +     *
> +     * We try to place dtb+initrd at 128MB, (or, if we have less RAM,
> +     * as high as possible). If there is no space then fallback to
> +     * just after the kernel, if there is room, otherwise just before.
>       */
> -    if ( (start > zimage_end) || (end < zimage_start) )
> +
> +    if ( kernel_end < MIN(mem_start + MB(128), mem_end - total) )
> +        addr = MIN(mem_start + MB(128), mem_end - total);
> +    else if ( mem_end - ROUNDUP(kernel_end, MB(2)) >= total )
> +        addr = ROUNDUP(kernel_end, MB(2));
> +    else if ( kernel_start - mem_start >= total )
> +        addr = kernel_start - total;
> +    else
> +    {
> +        panic("Unable to find suitable location for dtb+initrd.");
>          return;
> +    }
>  
> -    panic(XENLOG_ERR "The kernel(0x%"PRIpaddr"-0x%"PRIpaddr
> -          ") is overlapping the DTB-initrd(0x%"PRIpaddr"-0x%"PRIpaddr")\n",
> -          zimage_start, zimage_end, start, end);
> +    info->dtb_paddr = addr;
> +    info->initrd_paddr = info->dtb_paddr + dtb_len;
>  }
>  
>  static void kernel_zimage_load(struct kernel_info *info)
> @@ -98,6 +128,8 @@ static void kernel_zimage_load(struct kernel_info *info)
>      paddr_t len = info->zimage.len;
>      unsigned long offs;
>  
> +    place_modules(info, load_addr, load_addr + len);
> +
>      printk("Loading zImage from %"PRIpaddr" to %"PRIpaddr"-%"PRIpaddr"\n",
>             paddr, load_addr, load_addr + len);
>      for ( offs = 0; offs < len; )
> @@ -176,7 +208,6 @@ static int kernel_try_zimage64_prepare(struct kernel_info *info,
>  
>      info->entry = info->zimage.load_addr;
>      info->load = kernel_zimage_load;
> -    info->check_overlap = kernel_zimage_check_overlap;
>  
>      info->type = DOMAIN_PV64;
>  
> @@ -236,17 +267,9 @@ static int kernel_try_zimage32_prepare(struct kernel_info *info,
>          paddr_t load_end;
>  
>          load_end = info->mem.bank[0].start + info->mem.bank[0].size;
> -        load_end = MIN(info->mem.bank[0].start + (128<<20), load_end);
> -
> -        /*
> -         * FDT is loaded above 128M or as high as possible, so the
> -         * only way we can clash is if we have <=128MB, in which case
> -         * FDT will be right at the end and so dtb_paddr will be below
> -         * the proposed kernel load address. Move the kernel down if
> -         * necessary.
> -         */
> -        if ( load_end >= info->dtb_paddr )
> -            load_end = info->dtb_paddr;
> +        load_end = MIN(info->mem.bank[0].start + MB(128), load_end);
> +
> +        load_end += MB(2);
>  
>          info->zimage.load_addr = load_end - end;
>          /* Align to 2MB */
> @@ -258,7 +281,6 @@ static int kernel_try_zimage32_prepare(struct kernel_info *info,
>  
>      info->entry = info->zimage.load_addr;
>      info->load = kernel_zimage_load;
> -    info->check_overlap = kernel_zimage_check_overlap;
>  
>  #ifdef CONFIG_ARM_64
>      info->type = DOMAIN_PV32;
> @@ -269,10 +291,15 @@ static int kernel_try_zimage32_prepare(struct kernel_info *info,
>  
>  static void kernel_elf_load(struct kernel_info *info)
>  {
> +    place_modules(info,
> +                  info->elf.parms.virt_kstart,
> +                  info->elf.parms.virt_kend);
> +
>      printk("Loading ELF image into guest memory\n");
>      info->elf.elf.dest_base = (void*)(unsigned long)info->elf.parms.virt_kstart;
>      info->elf.elf.dest_size =
>           info->elf.parms.virt_kend - info->elf.parms.virt_kstart;
> +
>      elf_load_binary(&info->elf.elf);
>  
>      printk("Free temporary kernel buffer\n");
> @@ -321,7 +348,6 @@ static int kernel_try_elf_prepare(struct kernel_info *info,
>       */
>      info->entry = info->elf.parms.virt_entry;
>      info->load = kernel_elf_load;
> -    info->check_overlap = NULL;
>  
>      if ( elf_check_broken(&info->elf.elf) )
>          printk("Xen: warning: ELF kernel broken: %s\n",
> diff --git a/xen/arch/arm/kernel.h b/xen/arch/arm/kernel.h
> index debf590..b48c2c9 100644
> --- a/xen/arch/arm/kernel.h
> +++ b/xen/arch/arm/kernel.h
> @@ -40,8 +40,6 @@ struct kernel_info {
>      };
>  
>      void (*load)(struct kernel_info *info);
> -    /* Callback to check overlap between the kernel and the device tree */
> -    void (*check_overlap)(struct kernel_info *kinfo);
>      int load_attr;
>  };
>  
> -- 
> 1.7.10.4
> 

Re: [PATCH v5 07/19] xen: arm: allocate dom0 memory separately from preparing the dtb

[Ian Campbell]

On Wed, 2013-11-13 at 19:21 +0000, Stefano Stabellini wrote:
> On Wed, 13 Nov 2013, Ian Campbell wrote:
> > Mixing these two together is a pain, it forces us to prepare the dtb before
> > processing the kernel which means we don't know whether the guest is 32- or
> > 64-bit while we construct its DTB.
> > 
> > Instead split out the memory allocation (including 1:1 workaround handling)
> > and p2m setup into a separate phase and then create a memory node in the DTB
> > based on the result.
> > 
> > This allows us to move kernel parsing before DTB setup.
> > 
> > As part of this it was also necessary to rework where the decision regarding
> > the placement of the DTB and initrd in RAM was made. It is now made when
> > loading the kernel, which allows it to make use of the zImage/ELF specific
> > information and therefore to make decisions based on complete knowledge and do
> > it right rather than guessing in prepare_dtb and relying on a later check to
> > see if things worked.
> > 
> > Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
> 
> It looks good

Can I take that as an Acked-by?

Re: [PATCH v5 07/19] xen: arm: allocate dom0 memory separately from preparing the dtb

[Stefano Stabellini]

On Wed, 13 Nov 2013, Ian Campbell wrote:
> On Wed, 2013-11-13 at 19:21 +0000, Stefano Stabellini wrote:
> > On Wed, 13 Nov 2013, Ian Campbell wrote:
> > > Mixing these two together is a pain, it forces us to prepare the dtb before
> > > processing the kernel which means we don't know whether the guest is 32- or
> > > 64-bit while we construct its DTB.
> > > 
> > > Instead split out the memory allocation (including 1:1 workaround handling)
> > > and p2m setup into a separate phase and then create a memory node in the DTB
> > > based on the result.
> > > 
> > > This allows us to move kernel parsing before DTB setup.
> > > 
> > > As part of this it was also necessary to rework where the decision regarding
> > > the placement of the DTB and initrd in RAM was made. It is now made when
> > > loading the kernel, which allows it to make use of the zImage/ELF specific
> > > information and therefore to make decisions based on complete knowledge and do
> > > it right rather than guessing in prepare_dtb and relying on a later check to
> > > see if things worked.
> > > 
> > > Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
> > 
> > It looks good
> 
> Can I take that as an Acked-by?
> 

yes

Re: [PATCH v5 07/19] xen: arm: allocate dom0 memory separately from preparing the dtb

[Julien Grall]

On 11/13/2013 06:11 PM, Ian Campbell wrote:
> Mixing these two together is a pain, it forces us to prepare the dtb before
> processing the kernel which means we don't know whether the guest is 32- or
> 64-bit while we construct its DTB.
>
> Instead split out the memory allocation (including 1:1 workaround handling)
> and p2m setup into a separate phase and then create a memory node in the DTB
> based on the result.
>
> This allows us to move kernel parsing before DTB setup.
>
> As part of this it was also necessary to rework where the decision regarding
> the placement of the DTB and initrd in RAM was made. It is now made when
> loading the kernel, which allows it to make use of the zImage/ELF specific
> information and therefore to make decisions based on complete knowledge and do
> it right rather than guessing in prepare_dtb and relying on a later check to
> see if things worked.
>
> Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
> ---
> v3: Also rework module placement, v2 broke boot because dtb_paddr wasn't set
>      soon enough. This ends up cleaner anyway.
> v2: Fixed typo in the commit log
>      Handle multiple memory nodes as well as individual nodes with several
>      entries in them.
>      Strip the original memory node and recreate rather than trying to modify.
> ---
>   xen/arch/arm/domain_build.c |  203 ++++++++++++++++++++++---------------------
>   xen/arch/arm/kernel.c       |   80 +++++++++++------
>   xen/arch/arm/kernel.h       |    2 -
>   3 files changed, 158 insertions(+), 127 deletions(-)
>
> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> index 8645aa1..edfcf14 100644
> --- a/xen/arch/arm/domain_build.c
> +++ b/xen/arch/arm/domain_build.c
> @@ -81,11 +81,8 @@ struct vcpu *__init alloc_dom0_vcpu0(void)
>       return alloc_vcpu(dom0, 0, 0);
>   }
>
> -static int set_memory_reg_11(struct domain *d, struct kernel_info *kinfo,
> -                             const struct dt_property *pp,
> -                             const struct dt_device_node *np, __be32 *new_cell)
> +static void allocate_memory_11(struct domain *d, struct kernel_info *kinfo)
>   {
> -    int reg_size = dt_cells_to_size(dt_n_addr_cells(np) + dt_n_size_cells(np));
>       paddr_t start;
>       paddr_t size;
>       struct page_info *pg = NULL;
> @@ -116,53 +113,61 @@ static int set_memory_reg_11(struct domain *d, struct kernel_info *kinfo,
>       if ( res )
>           panic("Unable to add pages in DOM0: %d\n", res);
>
> -    dt_set_range(&new_cell, np, start, size);
> -
>       kinfo->mem.bank[0].start = start;
>       kinfo->mem.bank[0].size = size;
>       kinfo->mem.nr_banks = 1;
>
> -    return reg_size;
> +    kinfo->unassigned_mem -= size;
>   }
>
> -static int set_memory_reg(struct domain *d, struct kernel_info *kinfo,
> -                          const struct dt_property *pp,
> -                          const struct dt_device_node *np, __be32 *new_cell)
> +static void allocate_memory(struct domain *d, struct kernel_info *kinfo)
>   {
> -    int reg_size = dt_cells_to_size(dt_n_addr_cells(np) + dt_n_size_cells(np));
> -    int l = 0;
> +
> +    struct dt_device_node *memory = NULL;
> +    const void *reg;
> +    u32 reg_len, reg_size;
>       unsigned int bank = 0;
> -    u64 start;
> -    u64 size;
> -    int ret;
>
>       if ( platform_has_quirk(PLATFORM_QUIRK_DOM0_MAPPING_11) )
> -        return set_memory_reg_11(d, kinfo, pp, np, new_cell);
> +        return allocate_memory_11(d, kinfo);
>
> -    while ( kinfo->unassigned_mem > 0 && l + reg_size <= pp->length
> -            && kinfo->mem.nr_banks < NR_MEM_BANKS )
> +    while ( (memory = dt_find_node_by_type(memory, "memory")) )
>       {
> -        ret = dt_device_get_address(np, bank, &start, &size);
> -        if ( ret )
> -            panic("Unable to retrieve the bank %u for %s\n",
> -                  bank, dt_node_full_name(np));
> -
> -        if ( size > kinfo->unassigned_mem )
> -            size = kinfo->unassigned_mem;
> -        dt_set_range(&new_cell, np, start, size);
> -
> -        printk("Populate P2M %#"PRIx64"->%#"PRIx64"\n", start, start + size);
> -        if ( p2m_populate_ram(d, start, start + size) < 0 )
> -            panic("Failed to populate P2M\n");
> -        kinfo->mem.bank[kinfo->mem.nr_banks].start = start;
> -        kinfo->mem.bank[kinfo->mem.nr_banks].size = size;
> -        kinfo->mem.nr_banks++;
> -        kinfo->unassigned_mem -= size;
> -
> -        l += reg_size;
> -    }
> +        int l;
> +
> +        DPRINT("memory node\n");
> +
> +        reg_size = dt_cells_to_size(dt_n_addr_cells(memory) + dt_n_size_cells(memory));
> +
> +        reg = dt_get_property(memory, "reg", &reg_len);
> +        if ( reg == NULL )
> +            panic("Memory node has no reg property!\n");
> +
> +        for ( l = 0;
> +              kinfo->unassigned_mem > 0 && l + reg_size <= reg_len
> +                  && kinfo->mem.nr_banks < NR_MEM_BANKS;
> +              l += reg_size )
> +        {
> +            paddr_t start, size;
>
> -    return l;
> +            if ( dt_device_get_address(memory, bank, &start, &size) )
> +                panic("Unable to retrieve the bank %u for %s\n",
> +                      bank, dt_node_full_name(memory));
> +
> +            if ( size > kinfo->unassigned_mem )
> +                size = kinfo->unassigned_mem;
> +
> +            printk("Populate P2M %#"PRIx64"->%#"PRIx64"\n",
> +                   start, start + size);
> +            if ( p2m_populate_ram(d, start, start + size) < 0 )
> +                panic("Failed to populate P2M\n");
> +            kinfo->mem.bank[kinfo->mem.nr_banks].start = start;
> +            kinfo->mem.bank[kinfo->mem.nr_banks].size = size;
> +            kinfo->mem.nr_banks++;
> +
> +            kinfo->unassigned_mem -= size;
> +        }
> +    }
>   }
>
>   static int write_properties(struct domain *d, struct kernel_info *kinfo,
> @@ -211,23 +216,6 @@ static int write_properties(struct domain *d, struct kernel_info *kinfo,
>                   continue;
>               }
>           }
> -        /*
> -         * In a memory node: adjust reg property.
> -         * TODO: handle properly memory node (ie: device_type = "memory")
> -         */
> -        else if ( dt_node_name_is_equal(np, "memory") )
> -        {
> -            if ( dt_property_name_is_equal(pp, "reg") )
> -            {
> -                new_data = xzalloc_bytes(pp->length);
> -                if ( new_data  == NULL )
> -                    return -FDT_ERR_XEN(ENOMEM);
> -
> -                prop_len = set_memory_reg(d, kinfo, pp, np,
> -                                          (__be32 *)new_data);
> -                prop_data = new_data;
> -            }
> -        }
>
>           res = fdt_property(kinfo->fdt, pp->name, prop_data, prop_len);
>
> @@ -304,6 +292,46 @@ static int fdt_property_interrupts(void *fdt, gic_interrupt_t *intr,
>       return res;
>   }
>
> +static int make_memory_node(const struct domain *d,
> +                            void *fdt,
> +                            const struct kernel_info *kinfo)
> +{
> +    int res, i;
> +    int nr_cells = XEN_FDT_NODE_REG_SIZE*kinfo->mem.nr_banks;

What about xzalloc? I don't think it's safe to allocate an uncontrol 
size (we don't know the size of nr_banks, even if now it's hardcoded).

-- 
Julien Grall

Re: [PATCH v5 07/19] xen: arm: allocate dom0 memory separately from preparing the dtb

[Ian Campbell]

On Wed, 2013-11-13 at 21:34 +0000, Julien Grall wrote:
> > @@ -304,6 +292,46 @@ static int fdt_property_interrupts(void *fdt, gic_interrupt_t *intr,
> >       return res;
> >   }
> >
> > +static int make_memory_node(const struct domain *d,
> > +                            void *fdt,
> > +                            const struct kernel_info *kinfo)
> > +{
> > +    int res, i;
> > +    int nr_cells = XEN_FDT_NODE_REG_SIZE*kinfo->mem.nr_banks;
> 
> What about xzalloc? I don't think it's safe to allocate an uncontrol 
> size (we don't know the size of nr_banks, even if now it's hardcoded).

The stack is 8 pages (32k) which would be enough for 2048 memory banks
of 16 bytes each.

I think that is likely to be plenty, given that the function is called
once at start of day and not from an especially deep call chain.

The xzalloc in make_gic_node is unneccessary as well IMHO (even more so,
since the size is at most 32 bytes or so) but I left it alone here.

Ian.

Re: [PATCH v5 07/19] xen: arm: allocate dom0 memory separately from preparing the dtb

[Julien Grall]

On 11/13/2013 06:11 PM, Ian Campbell wrote:
> Mixing these two together is a pain, it forces us to prepare the dtb before
> processing the kernel which means we don't know whether the guest is 32- or
> 64-bit while we construct its DTB.
>
> Instead split out the memory allocation (including 1:1 workaround handling)
> and p2m setup into a separate phase and then create a memory node in the DTB
> based on the result.
>
> This allows us to move kernel parsing before DTB setup.
>
> As part of this it was also necessary to rework where the decision regarding
> the placement of the DTB and initrd in RAM was made. It is now made when
> loading the kernel, which allows it to make use of the zImage/ELF specific
> information and therefore to make decisions based on complete knowledge and do
> it right rather than guessing in prepare_dtb and relying on a later check to
> see if things worked.
>
> Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
> ---
> v3: Also rework module placement, v2 broke boot because dtb_paddr wasn't set
>      soon enough. This ends up cleaner anyway.
> v2: Fixed typo in the commit log
>      Handle multiple memory nodes as well as individual nodes with several
>      entries in them.
>      Strip the original memory node and recreate rather than trying to modify.
> ---
>   xen/arch/arm/domain_build.c |  203 ++++++++++++++++++++++---------------------
>   xen/arch/arm/kernel.c       |   80 +++++++++++------
>   xen/arch/arm/kernel.h       |    2 -
>   3 files changed, 158 insertions(+), 127 deletions(-)

[..]

> @@ -236,17 +267,9 @@ static int kernel_try_zimage32_prepare(struct kernel_info *info,
>           paddr_t load_end;
>
>           load_end = info->mem.bank[0].start + info->mem.bank[0].size;
> -        load_end = MIN(info->mem.bank[0].start + (128<<20), load_end);
> -
> -        /*
> -         * FDT is loaded above 128M or as high as possible, so the
> -         * only way we can clash is if we have <=128MB, in which case
> -         * FDT will be right at the end and so dtb_paddr will be below
> -         * the proposed kernel load address. Move the kernel down if
> -         * necessary.
> -         */
> -        if ( load_end >= info->dtb_paddr )
> -            load_end = info->dtb_paddr;
> +        load_end = MIN(info->mem.bank[0].start + MB(128), load_end);
> +
> +        load_end += MB(2);

I didn't notice this line during my previous review, why do you add MB(2)?

-- 
Julien Grall

Re: [PATCH v5 07/19] xen: arm: allocate dom0 memory separately from preparing the dtb

[Ian Campbell]

On Thu, 2013-11-14 at 00:52 +0000, Julien Grall wrote:
> 
> On 11/13/2013 06:11 PM, Ian Campbell wrote:
> > Mixing these two together is a pain, it forces us to prepare the dtb before
> > processing the kernel which means we don't know whether the guest is 32- or
> > 64-bit while we construct its DTB.
> >
> > Instead split out the memory allocation (including 1:1 workaround handling)
> > and p2m setup into a separate phase and then create a memory node in the DTB
> > based on the result.
> >
> > This allows us to move kernel parsing before DTB setup.
> >
> > As part of this it was also necessary to rework where the decision regarding
> > the placement of the DTB and initrd in RAM was made. It is now made when
> > loading the kernel, which allows it to make use of the zImage/ELF specific
> > information and therefore to make decisions based on complete knowledge and do
> > it right rather than guessing in prepare_dtb and relying on a later check to
> > see if things worked.
> >
> > Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
> > ---
> > v3: Also rework module placement, v2 broke boot because dtb_paddr wasn't set
> >      soon enough. This ends up cleaner anyway.
> > v2: Fixed typo in the commit log
> >      Handle multiple memory nodes as well as individual nodes with several
> >      entries in them.
> >      Strip the original memory node and recreate rather than trying to modify.
> > ---
> >   xen/arch/arm/domain_build.c |  203 ++++++++++++++++++++++---------------------
> >   xen/arch/arm/kernel.c       |   80 +++++++++++------
> >   xen/arch/arm/kernel.h       |    2 -
> >   3 files changed, 158 insertions(+), 127 deletions(-)
> 
> [..]
> 
> > @@ -236,17 +267,9 @@ static int kernel_try_zimage32_prepare(struct kernel_info *info,
> >           paddr_t load_end;
> >
> >           load_end = info->mem.bank[0].start + info->mem.bank[0].size;
> > -        load_end = MIN(info->mem.bank[0].start + (128<<20), load_end);
> > -
> > -        /*
> > -         * FDT is loaded above 128M or as high as possible, so the
> > -         * only way we can clash is if we have <=128MB, in which case
> > -         * FDT will be right at the end and so dtb_paddr will be below
> > -         * the proposed kernel load address. Move the kernel down if
> > -         * necessary.
> > -         */
> > -        if ( load_end >= info->dtb_paddr )
> > -            load_end = info->dtb_paddr;
> > +        load_end = MIN(info->mem.bank[0].start + MB(128), load_end);
> > +
> > +        load_end += MB(2);
> 
> I didn't notice this line during my previous review, why do you add MB(2)?

I've no idea... Perhaps I thought I was aligning it, but that's clearly
wrong. Or maybe it is left over debug from trying to force an error
condition etc. I'll drop it...

Ian.

[PATCH v5 08/19] xen: arm: add enable-method to cpu nodes for arm64 guests.

[Ian Campbell]

This is required by the Linux arm64 boot protocol.

We use PSCI.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Julien Grall <julien.grall@linaro.org>
---
 xen/arch/arm/domain_build.c |    7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index edfcf14..9f22283 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -513,6 +513,13 @@ static int make_cpus_node(const struct domain *d, void *fdt,
                 return res;
         }
 
+        if ( is_pv64_domain(d) )
+        {
+            res = fdt_property_string(fdt, "enable-method", "psci");
+            if ( res )
+                return res;
+        }
+
         res = fdt_end_node(fdt);
         if ( res )
             return res;
-- 
1.7.10.4

[PATCH v5 09/19] xen: arm: include header for for arch_do_{sys, dom}ctl prototype

[Ian Campbell]

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
---
v4: split out from "xen: arm: implement XEN_DOMCTL_set_address_size"
---
 xen/arch/arm/domctl.c |    1 +
 xen/arch/arm/sysctl.c |    1 +
 2 files changed, 2 insertions(+)

diff --git a/xen/arch/arm/domctl.c b/xen/arch/arm/domctl.c
index 851ee40..4cf0294 100644
--- a/xen/arch/arm/domctl.c
+++ b/xen/arch/arm/domctl.c
@@ -9,6 +9,7 @@
 #include <xen/lib.h>
 #include <xen/errno.h>
 #include <xen/sched.h>
+#include <xen/hypercall.h>
 #include <public/domctl.h>
 
 long arch_do_domctl(struct xen_domctl *domctl, struct domain *d,
diff --git a/xen/arch/arm/sysctl.c b/xen/arch/arm/sysctl.c
index 6388204..98bab6a 100644
--- a/xen/arch/arm/sysctl.c
+++ b/xen/arch/arm/sysctl.c
@@ -10,6 +10,7 @@
 #include <xen/types.h>
 #include <xen/lib.h>
 #include <xen/errno.h>
+#include <xen/hypercall.h>
 #include <public/sysctl.h>
 
 void arch_do_physinfo(xen_sysctl_physinfo_t *pi) { }
-- 
1.7.10.4

[PATCH v5 10/19] xen: arm: implement XEN_DOMCTL_set_address_size

[Ian Campbell]

This is subarch specific to plumb through to arm32 and arm64 versions.

The toolstack uses this to select 32- vs 64-bit guests (or rather it does on
x86 and soon will for arm too).

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Julien Grall <julien.grall@linaro.org>
---
v4: moved addition of xen/hypercall.h include into a precursor patch.
---
 xen/arch/arm/arm32/Makefile     |    1 +
 xen/arch/arm/arm32/domctl.c     |   35 +++++++++++++++++++++++
 xen/arch/arm/arm64/Makefile     |    1 +
 xen/arch/arm/arm64/domctl.c     |   59 +++++++++++++++++++++++++++++++++++++++
 xen/arch/arm/domctl.c           |    6 +++-
 xen/include/asm-arm/hypercall.h |    3 ++
 6 files changed, 104 insertions(+), 1 deletion(-)
 create mode 100644 xen/arch/arm/arm32/domctl.c
 create mode 100644 xen/arch/arm/arm64/domctl.c

diff --git a/xen/arch/arm/arm32/Makefile b/xen/arch/arm/arm32/Makefile
index aacdcb9..65ecff4 100644
--- a/xen/arch/arm/arm32/Makefile
+++ b/xen/arch/arm/arm32/Makefile
@@ -7,5 +7,6 @@ obj-y += traps.o
 obj-y += domain.o
 obj-y += vfp.o
 obj-y += smpboot.o
+obj-y += domctl.o
 
 obj-$(EARLY_PRINTK) += debug.o
diff --git a/xen/arch/arm/arm32/domctl.c b/xen/arch/arm/arm32/domctl.c
new file mode 100644
index 0000000..c2ca4d3
--- /dev/null
+++ b/xen/arch/arm/arm32/domctl.c
@@ -0,0 +1,35 @@
+/******************************************************************************
+ * Subarch-specific domctl.c
+ *
+ * Copyright (c) 2013, Citrix Systems
+ */
+
+#include <xen/config.h>
+#include <xen/types.h>
+#include <xen/lib.h>
+#include <xen/errno.h>
+#include <xen/sched.h>
+#include <xen/hypercall.h>
+#include <public/domctl.h>
+
+long subarch_do_domctl(struct xen_domctl *domctl, struct domain *d,
+               XEN_GUEST_HANDLE_PARAM(xen_domctl_t) u_domctl)
+{
+    switch ( domctl->cmd )
+    {
+    case XEN_DOMCTL_set_address_size:
+        return domctl->u.address_size.size == 32 ? 0 : -EINVAL;
+    default:
+        return -ENOSYS;
+    }
+}
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * tab-width: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/arch/arm/arm64/Makefile b/xen/arch/arm/arm64/Makefile
index 5d28bad..d2d5875 100644
--- a/xen/arch/arm/arm64/Makefile
+++ b/xen/arch/arm/arm64/Makefile
@@ -6,5 +6,6 @@ obj-y += traps.o
 obj-y += domain.o
 obj-y += vfp.o
 obj-y += smpboot.o
+obj-y += domctl.o
 
 obj-$(EARLY_PRINTK) += debug.o
diff --git a/xen/arch/arm/arm64/domctl.c b/xen/arch/arm/arm64/domctl.c
new file mode 100644
index 0000000..e2b4617
--- /dev/null
+++ b/xen/arch/arm/arm64/domctl.c
@@ -0,0 +1,59 @@
+/******************************************************************************
+ * Subarch-specific domctl.c
+ *
+ * Copyright (c) 2013, Citrix Systems
+ */
+
+#include <xen/config.h>
+#include <xen/types.h>
+#include <xen/lib.h>
+#include <xen/errno.h>
+#include <xen/sched.h>
+#include <xen/hypercall.h>
+#include <public/domctl.h>
+
+static long switch_mode(struct domain *d, enum domain_type type)
+{
+    if ( d == NULL )
+        return -EINVAL;
+    if ( d->tot_pages != 0 )
+        return -EBUSY;
+    if ( d->arch.type == type )
+        return 0;
+
+    d->arch.type = type;
+
+    return 0;
+}
+
+long subarch_do_domctl(struct xen_domctl *domctl, struct domain *d,
+                       XEN_GUEST_HANDLE_PARAM(xen_domctl_t) u_domctl)
+{
+    switch ( domctl->cmd )
+    {
+    case XEN_DOMCTL_set_address_size:
+        switch ( domctl->u.address_size.size )
+        {
+        case 32:
+            return switch_mode(d, DOMAIN_PV32);
+        case 64:
+            return switch_mode(d, DOMAIN_PV64);
+        default:
+            return -EINVAL;
+        }
+        break;
+
+    default:
+        return -ENOSYS;
+    }
+}
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * tab-width: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/arch/arm/domctl.c b/xen/arch/arm/domctl.c
index 4cf0294..546e86b 100644
--- a/xen/arch/arm/domctl.c
+++ b/xen/arch/arm/domctl.c
@@ -15,7 +15,11 @@
 long arch_do_domctl(struct xen_domctl *domctl, struct domain *d,
                     XEN_GUEST_HANDLE_PARAM(xen_domctl_t) u_domctl)
 {
-    return -ENOSYS;
+    switch ( domctl->cmd )
+    {
+    default:
+        return subarch_do_domctl(domctl, d, u_domctl);
+    }
 }
 
 void arch_get_info_guest(struct vcpu *v, vcpu_guest_context_u c)
diff --git a/xen/include/asm-arm/hypercall.h b/xen/include/asm-arm/hypercall.h
index 3327a96..94a92d4 100644
--- a/xen/include/asm-arm/hypercall.h
+++ b/xen/include/asm-arm/hypercall.h
@@ -6,6 +6,9 @@ int do_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg);
 
 long do_arm_vcpu_op(int cmd, int vcpuid, XEN_GUEST_HANDLE_PARAM(void) arg);
 
+long subarch_do_domctl(struct xen_domctl *domctl, struct domain *d,
+                       XEN_GUEST_HANDLE_PARAM(xen_domctl_t) u_domctl);
+
 #endif /* __ASM_ARM_HYPERCALL_H__ */
 /*
  * Local variables:
-- 
1.7.10.4

[PATCH v5 11/19] xen: arm: implement arch_set_info_guest for 64-bit vcpus

[Ian Campbell]

This all seems too easy...

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Julien Grall <julien.grall@linaro.org>
---
 xen/arch/arm/domain.c |   64 ++++++++++++++++++++++++++++++++++++++-----------
 1 file changed, 50 insertions(+), 14 deletions(-)

diff --git a/xen/arch/arm/domain.c b/xen/arch/arm/domain.c
index cb0424d..5ff7adf 100644
--- a/xen/arch/arm/domain.c
+++ b/xen/arch/arm/domain.c
@@ -550,7 +550,7 @@ void arch_domain_destroy(struct domain *d)
     free_xenheap_page(d->shared_info);
 }
 
-static int is_guest_psr(uint32_t psr)
+static int is_guest_pv32_psr(uint32_t psr)
 {
     switch (psr & PSR_MODE_MASK)
     {
@@ -569,6 +569,29 @@ static int is_guest_psr(uint32_t psr)
     }
 }
 
+
+#ifdef CONFIG_ARM_64
+static int is_guest_pv64_psr(uint32_t psr)
+{
+    if ( psr & PSR_MODE_BIT )
+        return 0;
+
+    switch (psr & PSR_MODE_MASK)
+    {
+    case PSR_MODE_EL1h:
+    case PSR_MODE_EL1t:
+    case PSR_MODE_EL0t:
+        return 1;
+    case PSR_MODE_EL3h:
+    case PSR_MODE_EL3t:
+    case PSR_MODE_EL2h:
+    case PSR_MODE_EL2t:
+    default:
+        return 0;
+    }
+}
+#endif
+
 /*
  * Initialise VCPU state. The context can be supplied by either the
  * toolstack (XEN_DOMCTL_setvcpucontext) or the guest
@@ -580,19 +603,32 @@ int arch_set_info_guest(
     struct vcpu_guest_context *ctxt = c.nat;
     struct vcpu_guest_core_regs *regs = &c.nat->user_regs;
 
-    if ( !is_guest_psr(regs->cpsr) )
-        return -EINVAL;
-
-    if ( regs->spsr_svc && !is_guest_psr(regs->spsr_svc) )
-        return -EINVAL;
-    if ( regs->spsr_abt && !is_guest_psr(regs->spsr_abt) )
-        return -EINVAL;
-    if ( regs->spsr_und && !is_guest_psr(regs->spsr_und) )
-        return -EINVAL;
-    if ( regs->spsr_irq && !is_guest_psr(regs->spsr_irq) )
-        return -EINVAL;
-    if ( regs->spsr_fiq && !is_guest_psr(regs->spsr_fiq) )
-        return -EINVAL;
+    if ( is_pv32_domain(v->domain) )
+    {
+        if ( !is_guest_pv32_psr(regs->cpsr) )
+            return -EINVAL;
+
+        if ( regs->spsr_svc && !is_guest_pv32_psr(regs->spsr_svc) )
+            return -EINVAL;
+        if ( regs->spsr_abt && !is_guest_pv32_psr(regs->spsr_abt) )
+            return -EINVAL;
+        if ( regs->spsr_und && !is_guest_pv32_psr(regs->spsr_und) )
+            return -EINVAL;
+        if ( regs->spsr_irq && !is_guest_pv32_psr(regs->spsr_irq) )
+            return -EINVAL;
+        if ( regs->spsr_fiq && !is_guest_pv32_psr(regs->spsr_fiq) )
+            return -EINVAL;
+    }
+#ifdef CONFIG_ARM_64
+    else
+    {
+        if ( !is_guest_pv64_psr(regs->cpsr) )
+            return -EINVAL;
+
+        if ( regs->spsr_el1 && !is_guest_pv64_psr(regs->spsr_el1) )
+            return -EINVAL;
+    }
+#endif
 
     vcpu_regs_user_to_hyp(v, regs);
 
-- 
1.7.10.4

[PATCH v5 12/19] tools: check for libfdt when building for ARM

[Ian Campbell]

libxl is going to want this to aid in the creation of guest device tree blobs.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
Rerun autogen.sh when committing
---
 tools/config.h.in  |    3 +++
 tools/configure    |   52 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 tools/configure.ac |    6 ++++++
 3 files changed, 61 insertions(+)

diff --git a/tools/config.h.in b/tools/config.h.in
index b1c9531..015f2a1 100644
--- a/tools/config.h.in
+++ b/tools/config.h.in
@@ -9,6 +9,9 @@
 /* Define to 1 if you have the `crypto' library (-lcrypto). */
 #undef HAVE_LIBCRYPTO
 
+/* Define to 1 if you have the `fdt' library (-lfdt). */
+#undef HAVE_LIBFDT
+
 /* Define to 1 if you have the `yajl' library (-lyajl). */
 #undef HAVE_LIBYAJL
 
diff --git a/tools/configure b/tools/configure
index ff82b32..afc3000 100755
--- a/tools/configure
+++ b/tools/configure
@@ -7959,6 +7959,58 @@ fi
 
 
 
+# FDT is needed only on ARM
+case "$host_cpu" in
+arm*|aarch64)
+{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for fdt_create in -lfdt" >&5
+$as_echo_n "checking for fdt_create in -lfdt... " >&6; }
+if ${ac_cv_lib_fdt_fdt_create+:} false; then :
+  $as_echo_n "(cached) " >&6
+else
+  ac_check_lib_save_LIBS=$LIBS
+LIBS="-lfdt  $LIBS"
+cat confdefs.h - <<_ACEOF >conftest.$ac_ext
+/* end confdefs.h.  */
+
+/* Override any GCC internal prototype to avoid an error.
+   Use char because int might match the return type of a GCC
+   builtin and then its argument prototype would still apply.  */
+#ifdef __cplusplus
+extern "C"
+#endif
+char fdt_create ();
+int
+main ()
+{
+return fdt_create ();
+  ;
+  return 0;
+}
+_ACEOF
+if ac_fn_c_try_link "$LINENO"; then :
+  ac_cv_lib_fdt_fdt_create=yes
+else
+  ac_cv_lib_fdt_fdt_create=no
+fi
+rm -f core conftest.err conftest.$ac_objext \
+    conftest$ac_exeext conftest.$ac_ext
+LIBS=$ac_check_lib_save_LIBS
+fi
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_lib_fdt_fdt_create" >&5
+$as_echo "$ac_cv_lib_fdt_fdt_create" >&6; }
+if test "x$ac_cv_lib_fdt_fdt_create" = xyes; then :
+  cat >>confdefs.h <<_ACEOF
+#define HAVE_LIBFDT 1
+_ACEOF
+
+  LIBS="-lfdt $LIBS"
+
+else
+  as_fn_error $? "Could not find libfdt" "$LINENO" 5
+fi
+
+esac
+
 # Checks for header files.
 for ac_header in yajl/yajl_version.h sys/eventfd.h
 do :
diff --git a/tools/configure.ac b/tools/configure.ac
index b2941a4..0754f0e 100644
--- a/tools/configure.ac
+++ b/tools/configure.ac
@@ -227,6 +227,12 @@ AC_CHECK_LIB([z], [deflateCopy], [], [AC_MSG_ERROR([Could not find zlib])])
 AC_CHECK_LIB([iconv], [libiconv_open], [libiconv="y"], [libiconv="n"])
 AC_SUBST(libiconv)
 
+# FDT is needed only on ARM
+case "$host_cpu" in
+arm*|aarch64)
+AC_CHECK_LIB([fdt], [fdt_create], [], [AC_MSG_ERROR([Could not find libfdt])])
+esac
+
 # Checks for header files.
 AC_CHECK_HEADERS([yajl/yajl_version.h sys/eventfd.h])
 
-- 
1.7.10.4

[PATCH v5 13/19] xen: arm: define guest virtual platform in API headers

[Ian Campbell]

The tools and the hypervisor need to agree on various aspects of the guest
environment, such as interrupt numbers, memory layout, initial register values
for registers which are implementation defined etc. Therefore move the
associated defines into the public interface headers, or create them as
necessary.

This just exposes the current de-facto standard guest layout, which may be
subject to change in the future. This deliberately does not make the guest
layout dynamic since there is currently no need.

These values should not be exposed to guests, they should find these things
out via device tree or should not be relying on implementation defined
defaults.

Various bits of the hypervisor needed to change to configure dom0 with the real
platform values while using the virtual platform configuration for guests.
Arrange for this where appropriate and plumb through as needed.

We also need to expose some 64-bit values (e.g. PSR_GUEST64_INIT) for the
benefit of 32 bit toolstacks building 64 bit guests.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Acked-by: Julien Grall <julien.grall@linaro.org>
---
v5: IRQs are unsigned int.
v4: dropped a spurious aarch64 ifdef
    improved some comments and expand the commit message to clarify that this
    is just implementing the current static de-facto setup.
    put back PSR_GUEST32_INIT which disapeared somewhere along the line
---
 tools/libxc/xc_dom_arm.c        |    4 +--
 xen/arch/arm/domain.c           |    8 ++++--
 xen/arch/arm/domain_build.c     |   13 +++++----
 xen/arch/arm/gic.c              |   21 +++++++++-----
 xen/arch/arm/psci.c             |    2 +-
 xen/arch/arm/traps.c            |    2 +-
 xen/arch/arm/vtimer.c           |   13 ++++++---
 xen/include/asm-arm/domain.h    |    1 +
 xen/include/asm-arm/event.h     |    3 +-
 xen/include/asm-arm/gic.h       |    3 --
 xen/include/asm-arm/processor.h |    7 -----
 xen/include/asm-arm/psci.h      |    5 ----
 xen/include/public/arch-arm.h   |   58 +++++++++++++++++++++++++++++++--------
 13 files changed, 90 insertions(+), 50 deletions(-)

diff --git a/tools/libxc/xc_dom_arm.c b/tools/libxc/xc_dom_arm.c
index df59ffb..9f3fdd3 100644
--- a/tools/libxc/xc_dom_arm.c
+++ b/tools/libxc/xc_dom_arm.c
@@ -126,13 +126,13 @@ static int vcpu_arm(struct xc_dom_image *dom, void *ptr)
      */
     ctxt->user_regs.r2_usr = 0xffffffff;
 
-    ctxt->sctlr = /* #define SCTLR_BASE */0x00c50078;
+    ctxt->sctlr = SCTLR_GUEST_INIT;
 
     ctxt->ttbr0 = 0;
     ctxt->ttbr1 = 0;
     ctxt->ttbcr = 0; /* Defined Reset Value */
 
-    ctxt->user_regs.cpsr = PSR_ABT_MASK|PSR_FIQ_MASK|PSR_IRQ_MASK|PSR_MODE_SVC;
+    ctxt->user_regs.cpsr = PSR_GUEST32_INIT;
 
     ctxt->flags = VGCF_online;
 
diff --git a/xen/arch/arm/domain.c b/xen/arch/arm/domain.c
index 5ff7adf..2f57d01 100644
--- a/xen/arch/arm/domain.c
+++ b/xen/arch/arm/domain.c
@@ -461,7 +461,8 @@ int vcpu_initialise(struct vcpu *v)
     if ( is_idle_vcpu(v) )
         return rc;
 
-    v->arch.sctlr = SCTLR_BASE;
+    v->arch.sctlr = SCTLR_GUEST_INIT;
+
     /*
      * By default exposes an SMP system with AFF0 set to the VCPU ID
      * TODO: Handle multi-threading processor and cluster
@@ -525,6 +526,9 @@ int arch_domain_create(struct domain *d, unsigned int domcr_flags)
     if ( (rc = vcpu_domain_init(d)) != 0 )
         goto fail;
 
+    /* XXX dom0 needs more intelligent selection of PPI */
+    d->arch.evtchn_irq = GUEST_EVTCHN_PPI;
+
     /*
      * Virtual UART is only used by linux early printk and decompress code.
      * Only use it for dom0 because the linux kernel may not support
@@ -740,7 +744,7 @@ void vcpu_mark_events_pending(struct vcpu *v)
     if ( already_pending )
         return;
 
-    vgic_vcpu_inject_irq(v, VGIC_IRQ_EVTCHN_CALLBACK, 1);
+    vgic_vcpu_inject_irq(v, v->domain->arch.evtchn_irq, 1);
 }
 
 /*
diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index 9f22283..d2f4cce 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -333,7 +333,8 @@ static int make_memory_node(const struct domain *d,
     return res;
 }
 
-static int make_hypervisor_node(void *fdt, const struct dt_device_node *parent)
+static int make_hypervisor_node(struct domain *d,
+                                void *fdt, const struct dt_device_node *parent)
 {
     const char compat[] =
         "xen,xen-"__stringify(XEN_VERSION)"."__stringify(XEN_SUBVERSION)"\0"
@@ -381,8 +382,8 @@ static int make_hypervisor_node(void *fdt, const struct dt_device_node *parent)
      *
      * TODO: Handle correctly the cpumask
      */
-    DPRINT("  Event channel interrupt to %u\n", VGIC_IRQ_EVTCHN_CALLBACK);
-    set_interrupt_ppi(intr, VGIC_IRQ_EVTCHN_CALLBACK, 0xf,
+    DPRINT("  Event channel interrupt to %u\n", d->arch.evtchn_irq);
+    set_interrupt_ppi(intr, d->arch.evtchn_irq, 0xf,
                    DT_IRQ_TYPE_LEVEL_LOW);
 
     res = fdt_property_interrupts(fdt, &intr, 1);
@@ -413,11 +414,11 @@ static int make_psci_node(void *fdt, const struct dt_device_node *parent)
     if ( res )
         return res;
 
-    res = fdt_property_cell(fdt, "cpu_off", __PSCI_cpu_off);
+    res = fdt_property_cell(fdt, "cpu_off", PSCI_cpu_off);
     if ( res )
         return res;
 
-    res = fdt_property_cell(fdt, "cpu_on", __PSCI_cpu_on);
+    res = fdt_property_cell(fdt, "cpu_on", PSCI_cpu_on);
     if ( res )
         return res;
 
@@ -855,7 +856,7 @@ static int handle_node(struct domain *d, struct kernel_info *kinfo,
 
     if ( np == dt_host )
     {
-        res = make_hypervisor_node(kinfo->fdt, np);
+        res = make_hypervisor_node(d, kinfo->fdt, np);
         if ( res )
             return res;
 
diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c
index 74575cd..33c6b8d 100644
--- a/xen/arch/arm/gic.c
+++ b/xen/arch/arm/gic.c
@@ -717,7 +717,7 @@ int gic_events_need_delivery(void)
 void gic_inject(void)
 {
     if ( vcpu_info(current, evtchn_upcall_pending) )
-        vgic_vcpu_inject_irq(current, VGIC_IRQ_EVTCHN_CALLBACK, 1);
+        vgic_vcpu_inject_irq(current, current->domain->arch.evtchn_irq, 1);
 
     gic_restore_pending_irqs(current);
     if (!gic_events_need_delivery())
@@ -823,13 +823,20 @@ void gic_interrupt(struct cpu_user_regs *regs, int is_fiq)
 
 int gicv_setup(struct domain *d)
 {
-    /* TODO: Retrieve distributor and CPU guest base address from the
-     * guest DTS
-     * For the moment we use dom0 DTS
+    /*
+     * Domain 0 gets the hardware address.
+     * Guests get the virtual platform layout.
      */
-    d->arch.vgic.dbase = gic.dbase;
-    d->arch.vgic.cbase = gic.cbase;
-
+    if ( d == dom0 )
+    {
+        d->arch.vgic.dbase = gic.dbase;
+        d->arch.vgic.cbase = gic.cbase;
+    }
+    else
+    {
+        d->arch.vgic.dbase = GUEST_GICD_BASE;
+        d->arch.vgic.cbase = GUEST_GICC_BASE;
+    }
 
     d->arch.vgic.nr_lines = 0;
 
diff --git a/xen/arch/arm/psci.c b/xen/arch/arm/psci.c
index 6c3be47..c82884f 100644
--- a/xen/arch/arm/psci.c
+++ b/xen/arch/arm/psci.c
@@ -43,7 +43,7 @@ int do_psci_cpu_on(uint32_t vcpuid, register_t entry_point)
 
     memset(ctxt, 0, sizeof(*ctxt));
     ctxt->user_regs.pc64 = (u64) entry_point;
-    ctxt->sctlr = SCTLR_BASE;
+    ctxt->sctlr = SCTLR_GUEST_INIT;
     ctxt->ttbr0 = 0;
     ctxt->ttbr1 = 0;
     ctxt->ttbcr = 0; /* Defined Reset Value */
diff --git a/xen/arch/arm/traps.c b/xen/arch/arm/traps.c
index 10ee498..d39e2d4 100644
--- a/xen/arch/arm/traps.c
+++ b/xen/arch/arm/traps.c
@@ -867,7 +867,7 @@ typedef struct {
 } arm_psci_t;
 
 #define PSCI(_name, _nr_args)                                  \
-    [ __PSCI_ ## _name ] =  {                                  \
+    [ PSCI_ ## _name ] =  {                                    \
         .fn = (arm_psci_fn_t) &do_psci_ ## _name,              \
         .nr_args = _nr_args,                                   \
     }
diff --git a/xen/arch/arm/vtimer.c b/xen/arch/arm/vtimer.c
index d58a630..f323453 100644
--- a/xen/arch/arm/vtimer.c
+++ b/xen/arch/arm/vtimer.c
@@ -54,21 +54,26 @@ int vcpu_domain_init(struct domain *d)
 int vcpu_vtimer_init(struct vcpu *v)
 {
     struct vtimer *t = &v->arch.phys_timer;
+    bool_t d0 = (v->domain == dom0);
 
-    /* TODO: Retrieve physical and virtual timer IRQ from the guest
-     * DT. For the moment we use dom0 DT
+    /*
+     * Domain 0 uses the hardware interrupts, guests get the virtual platform.
      */
 
     init_timer(&t->timer, phys_timer_expired, t, v->processor);
     t->ctl = 0;
     t->cval = NOW();
-    t->irq = timer_dt_irq(TIMER_PHYS_NONSECURE_PPI)->irq;
+    t->irq = d0
+        ? timer_dt_irq(TIMER_PHYS_NONSECURE_PPI)->irq
+        : GUEST_TIMER_PHYS_NS_PPI;
     t->v = v;
 
     t = &v->arch.virt_timer;
     init_timer(&t->timer, virt_timer_expired, t, v->processor);
     t->ctl = 0;
-    t->irq = timer_dt_irq(TIMER_VIRT_PPI)->irq;
+    t->irq = d0
+        ? timer_dt_irq(TIMER_VIRT_PPI)->irq
+        : GUEST_TIMER_VIRT_PPI;
     t->v = v;
 
     return 0;
diff --git a/xen/include/asm-arm/domain.h b/xen/include/asm-arm/domain.h
index 67bfbbc..d5cae2e 100644
--- a/xen/include/asm-arm/domain.h
+++ b/xen/include/asm-arm/domain.h
@@ -112,6 +112,7 @@ struct arch_domain
         spinlock_t                  lock;
     } vuart;
 
+    unsigned int evtchn_irq;
 }  __cacheline_aligned;
 
 struct arch_vcpu
diff --git a/xen/include/asm-arm/event.h b/xen/include/asm-arm/event.h
index 04d854f..dd3ad13 100644
--- a/xen/include/asm-arm/event.h
+++ b/xen/include/asm-arm/event.h
@@ -15,7 +15,8 @@ static inline int vcpu_event_delivery_is_enabled(struct vcpu *v)
 
 static inline int local_events_need_delivery_nomask(void)
 {
-    struct pending_irq *p = irq_to_pending(current, VGIC_IRQ_EVTCHN_CALLBACK);
+    struct pending_irq *p = irq_to_pending(current,
+                                           current->domain->arch.evtchn_irq);
 
     /* XXX: if the first interrupt has already been delivered, we should
      * check whether any other interrupts with priority higher than the
diff --git a/xen/include/asm-arm/gic.h b/xen/include/asm-arm/gic.h
index 0a890be..41f0b3b 100644
--- a/xen/include/asm-arm/gic.h
+++ b/xen/include/asm-arm/gic.h
@@ -129,9 +129,6 @@
 #define GICH_LR_CPUID_SHIFT     9
 #define GICH_VTR_NRLRGS         0x3f
 
-/* XXX: write this into the DT */
-#define VGIC_IRQ_EVTCHN_CALLBACK 31
-
 #ifndef __ASSEMBLY__
 #include <xen/device_tree.h>
 
diff --git a/xen/include/asm-arm/processor.h b/xen/include/asm-arm/processor.h
index 5294421..3da3a3d 100644
--- a/xen/include/asm-arm/processor.h
+++ b/xen/include/asm-arm/processor.h
@@ -48,15 +48,8 @@
 #define SCTLR_A         (1<<1)
 #define SCTLR_M         (1<<0)
 
-#define SCTLR_BASE        0x00c50078
 #define HSCTLR_BASE       0x30c51878
 
-#define PSR_GUEST32_INIT  (PSR_ABT_MASK|PSR_FIQ_MASK|PSR_IRQ_MASK|PSR_MODE_SVC)
-
-#ifdef CONFIG_ARM_64
-#define PSR_GUEST64_INIT (PSR_ABT_MASK|PSR_FIQ_MASK|PSR_IRQ_MASK|PSR_MODE_EL1h)
-#endif
-
 /* HCR Hyp Configuration Register */
 #define HCR_RW          (1<<31) /* Register Width, ARM64 only */
 #define HCR_TGE         (1<<27) /* Trap General Exceptions */
diff --git a/xen/include/asm-arm/psci.h b/xen/include/asm-arm/psci.h
index fdba636..67d4c35 100644
--- a/xen/include/asm-arm/psci.h
+++ b/xen/include/asm-arm/psci.h
@@ -6,11 +6,6 @@
 #define PSCI_EINVAL  -2
 #define PSCI_DENIED  -3
 
-#define __PSCI_cpu_suspend 0
-#define __PSCI_cpu_off     1
-#define __PSCI_cpu_on      2
-#define __PSCI_migrate     3
-
 int do_psci_cpu_on(uint32_t vcpuid, register_t entry_point);
 int do_psci_cpu_off(uint32_t power_state);
 int do_psci_cpu_suspend(uint32_t power_state, register_t entry_point);
diff --git a/xen/include/public/arch-arm.h b/xen/include/public/arch-arm.h
index 1e8aeda..cb41ddc 100644
--- a/xen/include/public/arch-arm.h
+++ b/xen/include/public/arch-arm.h
@@ -302,8 +302,19 @@ typedef uint64_t xen_callback_t;
 
 #endif
 
+#if defined(__XEN__) || defined(__XEN_TOOLS__)
+
 /* PSR bits (CPSR, SPSR)*/
 
+#define PSR_THUMB       (1<<5)        /* Thumb Mode enable */
+#define PSR_FIQ_MASK    (1<<6)        /* Fast Interrupt mask */
+#define PSR_IRQ_MASK    (1<<7)        /* Interrupt mask */
+#define PSR_ABT_MASK    (1<<8)        /* Asynchronous Abort mask */
+#define PSR_BIG_ENDIAN  (1<<9)        /* arm32: Big Endian Mode */
+#define PSR_DBG_MASK    (1<<9)        /* arm64: Debug Exception mask */
+#define PSR_IT_MASK     (0x0600fc00)  /* Thumb If-Then Mask */
+#define PSR_JAZELLE     (1<<24)       /* Jazelle Mode */
+
 /* 32 bit modes */
 #define PSR_MODE_USR 0x10
 #define PSR_MODE_FIQ 0x11
@@ -316,7 +327,6 @@ typedef uint64_t xen_callback_t;
 #define PSR_MODE_SYS 0x1f
 
 /* 64 bit modes */
-#ifdef __aarch64__
 #define PSR_MODE_BIT  0x10 /* Set iff AArch32 */
 #define PSR_MODE_EL3h 0x0d
 #define PSR_MODE_EL3t 0x0c
@@ -325,18 +335,44 @@ typedef uint64_t xen_callback_t;
 #define PSR_MODE_EL1h 0x05
 #define PSR_MODE_EL1t 0x04
 #define PSR_MODE_EL0t 0x00
-#endif
 
-#define PSR_THUMB       (1<<5)        /* Thumb Mode enable */
-#define PSR_FIQ_MASK    (1<<6)        /* Fast Interrupt mask */
-#define PSR_IRQ_MASK    (1<<7)        /* Interrupt mask */
-#define PSR_ABT_MASK    (1<<8)        /* Asynchronous Abort mask */
-#define PSR_BIG_ENDIAN  (1<<9)        /* Big Endian Mode */
-#ifdef __aarch64__ /* For Aarch64 bit 9 is repurposed. */
-#define PSR_DBG_MASK    (1<<9)
+#define PSR_GUEST32_INIT  (PSR_ABT_MASK|PSR_FIQ_MASK|PSR_IRQ_MASK|PSR_MODE_SVC)
+#define PSR_GUEST64_INIT (PSR_ABT_MASK|PSR_FIQ_MASK|PSR_IRQ_MASK|PSR_MODE_EL1h)
+
+#define SCTLR_GUEST_INIT    0x00c50078
+
+/*
+ * Virtual machine platform (memory layout, interrupts)
+ *
+ * These are defined for consistency between the tools and the
+ * hypervisor. Guests must not rely on these hardcoded values but
+ * should instead use the FDT.
+ */
+
+/* Physical Address Space */
+#define GUEST_GICD_BASE   0x2c001000ULL
+#define GUEST_GICD_SIZE   0x1000ULL
+#define GUEST_GICC_BASE   0x2c002000ULL
+#define GUEST_GICC_SIZE   0x100ULL
+
+#define GUEST_RAM_BASE    0x80000000ULL
+
+#define GUEST_GNTTAB_BASE 0xb0000000ULL
+#define GUEST_GNTTAB_SIZE 0x00020000ULL
+
+/* Interrupts */
+#define GUEST_TIMER_VIRT_PPI    27
+#define GUEST_TIMER_PHYS_S_PPI  29
+#define GUEST_TIMER_PHYS_NS_PPI 30
+#define GUEST_EVTCHN_PPI        31
+
+/* PSCI functions */
+#define PSCI_cpu_suspend 0
+#define PSCI_cpu_off     1
+#define PSCI_cpu_on      2
+#define PSCI_migrate     3
+
 #endif
-#define PSR_IT_MASK     (0x0600fc00)  /* Thumb If-Then Mask */
-#define PSR_JAZELLE     (1<<24)       /* Jazelle Mode */
 
 #endif /*  __XEN_PUBLIC_ARCH_ARM_H__ */
 
-- 
1.7.10.4

[PATCH v5 14/19] libxc: arm: rename various bits of zimage load with 32 suffix

[Ian Campbell]

Making room for a 64 bit implementation.

Also fix a typo and stop refering to it as a bzImage, which is an x86-ism.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Julien Grall <julien.grall@linaro.org>
---
 tools/libxc/xc_dom_armzimageloader.c |   44 ++++++++++++++++++++--------------
 1 file changed, 26 insertions(+), 18 deletions(-)

diff --git a/tools/libxc/xc_dom_armzimageloader.c b/tools/libxc/xc_dom_armzimageloader.c
index 54728b8..b693390 100644
--- a/tools/libxc/xc_dom_armzimageloader.c
+++ b/tools/libxc/xc_dom_armzimageloader.c
@@ -36,12 +36,6 @@
  */
 #define GUEST_RAM_BASE 0x80000000
 
-#define ZIMAGE_MAGIC_OFFSET 0x24
-#define ZIMAGE_START_OFFSET 0x28
-#define ZIMAGE_END_OFFSET   0x2c
-
-#define ZIMAGE_MAGIC 0x016f2818
-
 struct minimal_dtb_header {
     uint32_t magic;
     uint32_t total_size;
@@ -50,7 +44,17 @@ struct minimal_dtb_header {
 
 #define DTB_MAGIC 0xd00dfeed
 
-static int xc_dom_probe_zimage_kernel(struct xc_dom_image *dom)
+/* ------------------------------------------------------------ */
+/* 32-bit zImage Support                                        */
+/* ------------------------------------------------------------ */
+
+#define ZIMAGE32_MAGIC_OFFSET 0x24
+#define ZIMAGE32_START_OFFSET 0x28
+#define ZIMAGE32_END_OFFSET   0x2c
+
+#define ZIMAGE32_MAGIC 0x016f2818
+
+static int xc_dom_probe_zimage32_kernel(struct xc_dom_image *dom)
 {
     uint32_t *zimage;
     uint32_t end;
@@ -69,13 +73,13 @@ static int xc_dom_probe_zimage_kernel(struct xc_dom_image *dom)
     }
 
     zimage = (uint32_t *)dom->kernel_blob;
-    if ( zimage[ZIMAGE_MAGIC_OFFSET/4] != ZIMAGE_MAGIC )
+    if ( zimage[ZIMAGE32_MAGIC_OFFSET/4] != ZIMAGE32_MAGIC )
     {
-        xc_dom_printf(dom->xch, "%s: kernel is not a bzImage", __FUNCTION__);
+        xc_dom_printf(dom->xch, "%s: kernel is not an arm32 zImage", __FUNCTION__);
         return -EINVAL;
     }
 
-    end = zimage[ZIMAGE_END_OFFSET/4];
+    end = zimage[ZIMAGE32_END_OFFSET/4];
 
     /*
      * Check for an appended DTB.
@@ -94,7 +98,7 @@ static int xc_dom_probe_zimage_kernel(struct xc_dom_image *dom)
     return 0;
 }
 
-static int xc_dom_parse_zimage_kernel(struct xc_dom_image *dom)
+static int xc_dom_parse_zimage32_kernel(struct xc_dom_image *dom)
 {
     uint32_t *zimage;
     uint32_t start, entry_addr;
@@ -111,7 +115,7 @@ static int xc_dom_parse_zimage_kernel(struct xc_dom_image *dom)
     v_start = rambase + 0x8000;
     v_end = v_start + dom->kernel_size;
 
-    start = zimage[ZIMAGE_START_OFFSET/4];
+    start = zimage[ZIMAGE32_START_OFFSET/4];
 
     if (start == 0)
         entry_addr = v_start;
@@ -134,6 +138,10 @@ static int xc_dom_parse_zimage_kernel(struct xc_dom_image *dom)
     return 0;
 }
 
+/* ------------------------------------------------------------ */
+/* Common zImage Support                                        */
+/* ------------------------------------------------------------ */
+
 static int xc_dom_load_zimage_kernel(struct xc_dom_image *dom)
 {
     void *dst;
@@ -148,7 +156,7 @@ static int xc_dom_load_zimage_kernel(struct xc_dom_image *dom)
         return -1;
     }
 
-    DOMPRINTF("%s: kernel sed %#"PRIx64"-%#"PRIx64,
+    DOMPRINTF("%s: kernel seg %#"PRIx64"-%#"PRIx64,
               __func__, dom->kernel_seg.vstart, dom->kernel_seg.vend);
     DOMPRINTF("%s: copy %zd bytes from blob %p to dst %p",
               __func__, dom->kernel_size, dom->kernel_blob, dst);
@@ -158,16 +166,16 @@ static int xc_dom_load_zimage_kernel(struct xc_dom_image *dom)
     return 0;
 }
 
-static struct xc_dom_loader zimage_loader = {
-    .name = "Linux zImage (ARM)",
-    .probe = xc_dom_probe_zimage_kernel,
-    .parser = xc_dom_parse_zimage_kernel,
+static struct xc_dom_loader zimage32_loader = {
+    .name = "Linux zImage (ARM32)",
+    .probe = xc_dom_probe_zimage32_kernel,
+    .parser = xc_dom_parse_zimage32_kernel,
     .loader = xc_dom_load_zimage_kernel,
 };
 
 static void __init register_loader(void)
 {
-    xc_dom_register_loader(&zimage_loader);
+    xc_dom_register_loader(&zimage32_loader);
 }
 
 /*
-- 
1.7.10.4

[PATCH v5 15/19] libxc: allow caller to specify guest rambase rather than hardcoding

[Ian Campbell]

It's still hardcoded but it could now be plausibly be made variable.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
 tools/libxc/xc_dom.h                 |    1 +
 tools/libxc/xc_dom_armzimageloader.c |   12 +-----------
 tools/libxc/xc_dom_core.c            |    8 ++++++++
 tools/libxl/libxl_dom.c              |    6 ++++++
 4 files changed, 16 insertions(+), 11 deletions(-)

diff --git a/tools/libxc/xc_dom.h b/tools/libxc/xc_dom.h
index 86e23ee..ed49aa7 100644
--- a/tools/libxc/xc_dom.h
+++ b/tools/libxc/xc_dom.h
@@ -197,6 +197,7 @@ struct xc_dom_image *xc_dom_allocate(xc_interface *xch,
                                      const char *cmdline, const char *features);
 void xc_dom_release_phys(struct xc_dom_image *dom);
 void xc_dom_release(struct xc_dom_image *dom);
+int xc_dom_rambase_init(struct xc_dom_image *dom, uint64_t rambase);
 int xc_dom_mem_init(struct xc_dom_image *dom, unsigned int mem_mb);
 
 /* Set this larger if you have enormous ramdisks/kernels. Note that
diff --git a/tools/libxc/xc_dom_armzimageloader.c b/tools/libxc/xc_dom_armzimageloader.c
index b693390..4e3f7ae 100644
--- a/tools/libxc/xc_dom_armzimageloader.c
+++ b/tools/libxc/xc_dom_armzimageloader.c
@@ -30,12 +30,6 @@
 
 #include <arpa/inet.h> /* XXX ntohl is not the right function... */
 
-/*
- * Guest virtual RAM starts here. This must be consistent with the DTB
- * appended to the guest kernel.
- */
-#define GUEST_RAM_BASE 0x80000000
-
 struct minimal_dtb_header {
     uint32_t magic;
     uint32_t total_size;
@@ -103,14 +97,12 @@ static int xc_dom_parse_zimage32_kernel(struct xc_dom_image *dom)
     uint32_t *zimage;
     uint32_t start, entry_addr;
     uint64_t v_start, v_end;
-    uint64_t rambase = GUEST_RAM_BASE;
+    uint64_t rambase = dom->rambase_pfn << XC_PAGE_SHIFT;
 
     DOMPRINTF_CALLED(dom->xch);
 
     zimage = (uint32_t *)dom->kernel_blob;
 
-    dom->rambase_pfn = rambase >> XC_PAGE_SHIFT;
-
     /* Do not load kernel at the very first RAM address */
     v_start = rambase + 0x8000;
     v_end = v_start + dom->kernel_size;
@@ -130,8 +122,6 @@ static int xc_dom_parse_zimage32_kernel(struct xc_dom_image *dom)
     dom->parms.virt_base = rambase;
 
     dom->guest_type = "xen-3.0-armv7l";
-    DOMPRINTF("%s: %s: RAM starts at %"PRI_xen_pfn,
-              __FUNCTION__, dom->guest_type, dom->rambase_pfn);
     DOMPRINTF("%s: %s: 0x%" PRIx64 " -> 0x%" PRIx64 "",
               __FUNCTION__, dom->guest_type,
               dom->kernel_seg.vstart, dom->kernel_seg.vend);
diff --git a/tools/libxc/xc_dom_core.c b/tools/libxc/xc_dom_core.c
index 3bf51ef..6aa0176 100644
--- a/tools/libxc/xc_dom_core.c
+++ b/tools/libxc/xc_dom_core.c
@@ -785,6 +785,14 @@ int xc_dom_parse_image(struct xc_dom_image *dom)
     return -1;
 }
 
+int xc_dom_rambase_init(struct xc_dom_image *dom, uint64_t rambase)
+{
+    dom->rambase_pfn = rambase >> XC_PAGE_SHIFT;
+    DOMPRINTF("%s: RAM starts at %"PRI_xen_pfn,
+              __FUNCTION__, dom->rambase_pfn);
+    return 0;
+}
+
 int xc_dom_mem_init(struct xc_dom_image *dom, unsigned int mem_mb)
 {
     unsigned int page_shift;
diff --git a/tools/libxl/libxl_dom.c b/tools/libxl/libxl_dom.c
index 6cb39c1..55902af 100644
--- a/tools/libxl/libxl_dom.c
+++ b/tools/libxl/libxl_dom.c
@@ -390,6 +390,12 @@ int libxl__build_pv(libxl__gc *gc, uint32_t domid,
         LOGE(ERROR, "xc_dom_boot_xen_init failed");
         goto out;
     }
+#ifdef GUEST_RAM_BASE
+    if ( (ret = xc_dom_rambase_init(dom, GUEST_RAM_BASE)) != 0 ) {
+        LOGE(ERROR, "xc_dom_rambase failed");
+        goto out;
+    }
+#endif
     if ( (ret = xc_dom_parse_image(dom)) != 0 ) {
         LOGE(ERROR, "xc_dom_parse_image failed");
         goto out;
-- 
1.7.10.4

[PATCH v5 16/19] libxc: arm: allow passing a device tree blob to the guest

[Ian Campbell]

Placement of the blob in guest RAM is simplistic but will do for now.

This operation is only supported on ARM.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
v4: Return EINVAL on non-ARM platforms (which do not support device-tree)
    Slightly less rubbish DTB placement which isn't perfect but at least now
    doesn't fail with <128MB of RAM.
---
 tools/libxc/xc_dom.h      |    8 +++++++
 tools/libxc/xc_dom_arm.c  |   22 ++++++++++++++++++-
 tools/libxc/xc_dom_core.c |   53 +++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 82 insertions(+), 1 deletion(-)

diff --git a/tools/libxc/xc_dom.h b/tools/libxc/xc_dom.h
index ed49aa7..bf56436 100644
--- a/tools/libxc/xc_dom.h
+++ b/tools/libxc/xc_dom.h
@@ -54,9 +54,12 @@ struct xc_dom_image {
     size_t kernel_size;
     void *ramdisk_blob;
     size_t ramdisk_size;
+    void *devicetree_blob;
+    size_t devicetree_size;
 
     size_t max_kernel_size;
     size_t max_ramdisk_size;
+    size_t max_devicetree_size;
 
     /* arguments and parameters */
     char *cmdline;
@@ -217,6 +220,8 @@ int xc_dom_kernel_max_size(struct xc_dom_image *dom, size_t sz);
 int xc_dom_ramdisk_check_size(struct xc_dom_image *dom, size_t sz);
 int xc_dom_ramdisk_max_size(struct xc_dom_image *dom, size_t sz);
 
+int xc_dom_devicetree_max_size(struct xc_dom_image *dom, size_t sz);
+
 size_t xc_dom_check_gzip(xc_interface *xch,
                      void *blob, size_t ziplen);
 int xc_dom_do_gunzip(xc_interface *xch,
@@ -229,6 +234,9 @@ int xc_dom_kernel_mem(struct xc_dom_image *dom, const void *mem,
                       size_t memsize);
 int xc_dom_ramdisk_mem(struct xc_dom_image *dom, const void *mem,
                        size_t memsize);
+int xc_dom_devicetree_file(struct xc_dom_image *dom, const char *filename);
+int xc_dom_devicetree_mem(struct xc_dom_image *dom, const void *mem,
+                          size_t memsize);
 
 int xc_dom_parse_image(struct xc_dom_image *dom);
 struct xc_dom_arch *xc_dom_find_arch_hooks(xc_interface *xch, char *guest_type);
diff --git a/tools/libxc/xc_dom_arm.c b/tools/libxc/xc_dom_arm.c
index 9f3fdd3..4c59177 100644
--- a/tools/libxc/xc_dom_arm.c
+++ b/tools/libxc/xc_dom_arm.c
@@ -124,7 +124,8 @@ static int vcpu_arm(struct xc_dom_image *dom, void *ptr)
      * using CONFIG_ARM_APPENDED_DTB. Ensure that r2 does not look
      * like a valid pointer to a set of ATAGS or a DTB.
      */
-    ctxt->user_regs.r2_usr = 0xffffffff;
+    ctxt->user_regs.r2_usr = dom->devicetree_blob ?
+        dom->devicetree_seg.vstart : 0xffffffff;
 
     ctxt->sctlr = SCTLR_GUEST_INIT;
 
@@ -191,6 +192,25 @@ int arch_setup_meminit(struct xc_dom_image *dom)
             0, 0, &dom->p2m_host[i]);
     }
 
+    if ( dom->devicetree_blob )
+    {
+        const uint64_t rambase = dom->rambase_pfn << XC_PAGE_SHIFT;
+        const uint64_t ramend = rambase + ( dom->total_pages << XC_PAGE_SHIFT );
+        const uint64_t dtbsize = ( dom->devicetree_size + 3 ) & ~0x3;
+
+        /* Place at 128MB if there is sufficient RAM */
+        if (ramend >= rambase + 128*1024*1024 + dtbsize )
+            dom->devicetree_seg.vstart = rambase + 128*1024*1024;
+        else /* otherwise at top of RAM */
+            dom->devicetree_seg.vstart = ramend - dtbsize;
+
+        dom->devicetree_seg.vend =
+            dom->devicetree_seg.vstart + dom->devicetree_size;
+        DOMPRINTF("%s: devicetree: 0x%" PRIx64 " -> 0x%" PRIx64 "",
+                  __FUNCTION__,
+                  dom->devicetree_seg.vstart, dom->devicetree_seg.vend);
+    }
+
     return 0;
 }
 
diff --git a/tools/libxc/xc_dom_core.c b/tools/libxc/xc_dom_core.c
index 6aa0176..4bd37cc 100644
--- a/tools/libxc/xc_dom_core.c
+++ b/tools/libxc/xc_dom_core.c
@@ -671,6 +671,7 @@ struct xc_dom_image *xc_dom_allocate(xc_interface *xch,
 
     dom->max_kernel_size = XC_DOM_DECOMPRESS_MAX;
     dom->max_ramdisk_size = XC_DOM_DECOMPRESS_MAX;
+    dom->max_devicetree_size = XC_DOM_DECOMPRESS_MAX;
 
     if ( cmdline )
         dom->cmdline = xc_dom_strdup(dom, cmdline);
@@ -706,6 +707,13 @@ int xc_dom_ramdisk_max_size(struct xc_dom_image *dom, size_t sz)
     return 0;
 }
 
+int xc_dom_devicetree_max_size(struct xc_dom_image *dom, size_t sz)
+{
+    DOMPRINTF("%s: devicetree_max_size=%zx", __FUNCTION__, sz);
+    dom->max_devicetree_size = sz;
+    return 0;
+}
+
 int xc_dom_kernel_file(struct xc_dom_image *dom, const char *filename)
 {
     DOMPRINTF("%s: filename=\"%s\"", __FUNCTION__, filename);
@@ -729,6 +737,23 @@ int xc_dom_ramdisk_file(struct xc_dom_image *dom, const char *filename)
     return 0;
 }
 
+int xc_dom_devicetree_file(struct xc_dom_image *dom, const char *filename)
+{
+#if defined (__arm__) || defined(__aarch64__)
+    DOMPRINTF("%s: filename=\"%s\"", __FUNCTION__, filename);
+    dom->devicetree_blob =
+        xc_dom_malloc_filemap(dom, filename, &dom->devicetree_size,
+                              dom->max_devicetree_size);
+
+    if ( dom->devicetree_blob == NULL )
+        return -1;
+    return 0;
+#else
+    errno = -EINVAL;
+    return -1;
+#endif
+}
+
 int xc_dom_kernel_mem(struct xc_dom_image *dom, const void *mem, size_t memsize)
 {
     DOMPRINTF_CALLED(dom->xch);
@@ -747,6 +772,15 @@ int xc_dom_ramdisk_mem(struct xc_dom_image *dom, const void *mem,
     return 0;
 }
 
+int xc_dom_devicetree_mem(struct xc_dom_image *dom, const void *mem,
+                          size_t memsize)
+{
+    DOMPRINTF_CALLED(dom->xch);
+    dom->devicetree_blob = (void *)mem;
+    dom->devicetree_size = memsize;
+    return 0;
+}
+
 int xc_dom_parse_image(struct xc_dom_image *dom)
 {
     int i;
@@ -916,6 +950,25 @@ int xc_dom_build_image(struct xc_dom_image *dom)
             memcpy(ramdiskmap, dom->ramdisk_blob, dom->ramdisk_size);
     }
 
+    /* load devicetree */
+    if ( dom->devicetree_blob )
+    {
+        void *devicetreemap;
+
+        if ( xc_dom_alloc_segment(dom, &dom->devicetree_seg, "devicetree",
+                                  dom->devicetree_seg.vstart,
+                                  dom->devicetree_size) != 0 )
+            goto err;
+        devicetreemap = xc_dom_seg_to_ptr(dom, &dom->devicetree_seg);
+        if ( devicetreemap == NULL )
+        {
+            DOMPRINTF("%s: xc_dom_seg_to_ptr(dom, &dom->devicetree_seg) => NULL",
+                      __FUNCTION__);
+            goto err;
+        }
+        memcpy(devicetreemap, dom->devicetree_blob, dom->devicetree_size);
+    }
+
     /* allocate other pages */
     if ( dom->arch_hooks->alloc_magic_pages(dom) != 0 )
         goto err;
-- 
1.7.10.4

Re: [PATCH v5 16/19] libxc: arm: allow passing a device tree blob to the guest

[Julien Grall]

On 11/13/2013 06:11 PM, Ian Campbell wrote:
> Placement of the blob in guest RAM is simplistic but will do for now.
>
> This operation is only supported on ARM.
>
> Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
> ---
> v4: Return EINVAL on non-ARM platforms (which do not support device-tree)
>      Slightly less rubbish DTB placement which isn't perfect but at least now
>      doesn't fail with <128MB of RAM.
> ---
>   tools/libxc/xc_dom.h      |    8 +++++++
>   tools/libxc/xc_dom_arm.c  |   22 ++++++++++++++++++-
>   tools/libxc/xc_dom_core.c |   53 +++++++++++++++++++++++++++++++++++++++++++++
>   3 files changed, 82 insertions(+), 1 deletion(-)
>

[..]

> diff --git a/tools/libxc/xc_dom_arm.c b/tools/libxc/xc_dom_arm.c
> index 9f3fdd3..4c59177 100644
> --- a/tools/libxc/xc_dom_arm.c
> +++ b/tools/libxc/xc_dom_arm.c
> @@ -124,7 +124,8 @@ static int vcpu_arm(struct xc_dom_image *dom, void *ptr)
>        * using CONFIG_ARM_APPENDED_DTB. Ensure that r2 does not look
>        * like a valid pointer to a set of ATAGS or a DTB.
>        */
> -    ctxt->user_regs.r2_usr = 0xffffffff;
> +    ctxt->user_regs.r2_usr = dom->devicetree_blob ?
> +        dom->devicetree_seg.vstart : 0xffffffff;
>
>       ctxt->sctlr = SCTLR_GUEST_INIT;
>
> @@ -191,6 +192,25 @@ int arch_setup_meminit(struct xc_dom_image *dom)
>               0, 0, &dom->p2m_host[i]);
>       }
>
> +    if ( dom->devicetree_blob )
> +    {
> +        const uint64_t rambase = dom->rambase_pfn << XC_PAGE_SHIFT;
> +        const uint64_t ramend = rambase + ( dom->total_pages << XC_PAGE_SHIFT );
> +        const uint64_t dtbsize = ( dom->devicetree_size + 3 ) & ~0x3;
> +
> +        /* Place at 128MB if there is sufficient RAM */
> +        if (ramend >= rambase + 128*1024*1024 + dtbsize )

Minor coding style error : if ( ... )

Also, I'm wondering if (128 << 20) will be more clear?

[..]

-- 
Julien Grall

Re: [PATCH v5 16/19] libxc: arm: allow passing a device tree blob to the guest

[Ian Campbell]

On Thu, 2013-11-14 at 00:47 +0000, Julien Grall wrote:
> +        /* Place at 128MB if there is sufficient RAM */
> > +        if (ramend >= rambase + 128*1024*1024 + dtbsize )
> 
> Minor coding style error : if ( ... )

Well spotted

> Also, I'm wondering if (128 << 20) will be more clear?

About the same I think, especially given the preceding comment. I've
left it alone, mostly out of inertia^Wlaziness.

Ian.

[PATCH v5 17/19] libxc: support for arm64 Image format

[Ian Campbell]

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
---
v4: actually use v_end and drop unused entry_addr
---
 tools/libxc/xc_dom_armzimageloader.c |   85 ++++++++++++++++++++++++++++++++++
 1 file changed, 85 insertions(+)

diff --git a/tools/libxc/xc_dom_armzimageloader.c b/tools/libxc/xc_dom_armzimageloader.c
index 4e3f7ae..e6516a1 100644
--- a/tools/libxc/xc_dom_armzimageloader.c
+++ b/tools/libxc/xc_dom_armzimageloader.c
@@ -129,6 +129,83 @@ static int xc_dom_parse_zimage32_kernel(struct xc_dom_image *dom)
 }
 
 /* ------------------------------------------------------------ */
+/* 64-bit zImage Support                                        */
+/* ------------------------------------------------------------ */
+
+#define ZIMAGE64_MAGIC_V0 0x14000008
+#define ZIMAGE64_MAGIC_V1 0x644d5241 /* "ARM\x64" */
+
+/* linux/Documentation/arm64/booting.txt */
+struct zimage64_hdr {
+    uint32_t magic0;
+    uint32_t res0;
+    uint64_t text_offset;  /* Image load offset */
+    uint64_t res1;
+    uint64_t res2;
+    /* zImage V1 only from here */
+    uint64_t res3;
+    uint64_t res4;
+    uint64_t res5;
+    uint32_t magic1;
+    uint32_t res6;
+};
+static int xc_dom_probe_zimage64_kernel(struct xc_dom_image *dom)
+{
+    struct zimage64_hdr *zimage;
+
+    if ( dom->kernel_blob == NULL )
+    {
+        xc_dom_panic(dom->xch, XC_INTERNAL_ERROR,
+                     "%s: no kernel image loaded", __FUNCTION__);
+        return -EINVAL;
+    }
+
+    if ( dom->kernel_size < sizeof(*zimage) )
+    {
+        xc_dom_printf(dom->xch, "%s: kernel image too small", __FUNCTION__);
+        return -EINVAL;
+    }
+
+    zimage =  dom->kernel_blob;
+    if ( zimage->magic0 != ZIMAGE64_MAGIC_V0 &&
+         zimage->magic1 != ZIMAGE64_MAGIC_V1 )
+    {
+        xc_dom_printf(dom->xch, "%s: kernel is not an arm64 Image", __FUNCTION__);
+        return -EINVAL;
+    }
+
+    return 0;
+}
+
+static int xc_dom_parse_zimage64_kernel(struct xc_dom_image *dom)
+{
+    struct zimage64_hdr *zimage;
+    uint64_t v_start, v_end;
+    uint64_t rambase = dom->rambase_pfn << XC_PAGE_SHIFT;
+
+    DOMPRINTF_CALLED(dom->xch);
+
+    zimage = dom->kernel_blob;
+
+    v_start = rambase + zimage->text_offset;
+    v_end = v_start + dom->kernel_size;
+
+    dom->kernel_seg.vstart = v_start;
+    dom->kernel_seg.vend   = v_end;
+
+    /* Call the kernel at offset 0 */
+    dom->parms.virt_entry = v_start;
+    dom->parms.virt_base = rambase;
+
+    dom->guest_type = "xen-3.0-aarch64";
+    DOMPRINTF("%s: %s: 0x%" PRIx64 " -> 0x%" PRIx64 "",
+              __FUNCTION__, dom->guest_type,
+              dom->kernel_seg.vstart, dom->kernel_seg.vend);
+
+    return 0;
+}
+
+/* ------------------------------------------------------------ */
 /* Common zImage Support                                        */
 /* ------------------------------------------------------------ */
 
@@ -163,9 +240,17 @@ static struct xc_dom_loader zimage32_loader = {
     .loader = xc_dom_load_zimage_kernel,
 };
 
+static struct xc_dom_loader zimage64_loader = {
+    .name = "Linux zImage (ARM64)",
+    .probe = xc_dom_probe_zimage64_kernel,
+    .parser = xc_dom_parse_zimage64_kernel,
+    .loader = xc_dom_load_zimage_kernel,
+};
+
 static void __init register_loader(void)
 {
     xc_dom_register_loader(&zimage32_loader);
+    xc_dom_register_loader(&zimage64_loader);
 }
 
 /*
-- 
1.7.10.4

Re: [PATCH v5 17/19] libxc: support for arm64 Image format

[Julien Grall]

On 11/13/2013 06:11 PM, Ian Campbell wrote:
> Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
> ---
> v4: actually use v_end and drop unused entry_addr
> ---
>   tools/libxc/xc_dom_armzimageloader.c |   85 ++++++++++++++++++++++++++++++++++
>   1 file changed, 85 insertions(+)
>
> diff --git a/tools/libxc/xc_dom_armzimageloader.c b/tools/libxc/xc_dom_armzimageloader.c
> index 4e3f7ae..e6516a1 100644
> --- a/tools/libxc/xc_dom_armzimageloader.c
> +++ b/tools/libxc/xc_dom_armzimageloader.c
> @@ -129,6 +129,83 @@ static int xc_dom_parse_zimage32_kernel(struct xc_dom_image *dom)
>   }
>
>   /* ------------------------------------------------------------ */
> +/* 64-bit zImage Support                                        */
> +/* ------------------------------------------------------------ */
> +
> +#define ZIMAGE64_MAGIC_V0 0x14000008
> +#define ZIMAGE64_MAGIC_V1 0x644d5241 /* "ARM\x64" */
> +
> +/* linux/Documentation/arm64/booting.txt */
> +struct zimage64_hdr {
> +    uint32_t magic0;
> +    uint32_t res0;
> +    uint64_t text_offset;  /* Image load offset */
> +    uint64_t res1;
> +    uint64_t res2;
> +    /* zImage V1 only from here */
> +    uint64_t res3;
> +    uint64_t res4;
> +    uint64_t res5;
> +    uint32_t magic1;
> +    uint32_t res6;
> +};
> +static int xc_dom_probe_zimage64_kernel(struct xc_dom_image *dom)
> +{
> +    struct zimage64_hdr *zimage;
> +
> +    if ( dom->kernel_blob == NULL )
> +    {
> +        xc_dom_panic(dom->xch, XC_INTERNAL_ERROR,
> +                     "%s: no kernel image loaded", __FUNCTION__);
> +        return -EINVAL;
> +    }
> +
> +    if ( dom->kernel_size < sizeof(*zimage) )
> +    {
> +        xc_dom_printf(dom->xch, "%s: kernel image too small", __FUNCTION__);

What about using DOMPRINTF instead of xc_dom_printf?

In any case:
Acked-by: Julien Grall <julien.grall@linaro.org>

-- 
Julien Grall

Re: [PATCH v5 17/19] libxc: support for arm64 Image format

[Ian Campbell]

On Thu, 2013-11-14 at 01:17 +0000, Julien Grall wrote:
> 
> On 11/13/2013 06:11 PM, Ian Campbell wrote:
> > Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
> > ---
> > v4: actually use v_end and drop unused entry_addr
> > ---
> >   tools/libxc/xc_dom_armzimageloader.c |   85 ++++++++++++++++++++++++++++++++++
> >   1 file changed, 85 insertions(+)
> >
> > diff --git a/tools/libxc/xc_dom_armzimageloader.c b/tools/libxc/xc_dom_armzimageloader.c
> > index 4e3f7ae..e6516a1 100644
> > --- a/tools/libxc/xc_dom_armzimageloader.c
> > +++ b/tools/libxc/xc_dom_armzimageloader.c
> > @@ -129,6 +129,83 @@ static int xc_dom_parse_zimage32_kernel(struct xc_dom_image *dom)
> >   }
> >
> >   /* ------------------------------------------------------------ */
> > +/* 64-bit zImage Support                                        */
> > +/* ------------------------------------------------------------ */
> > +
> > +#define ZIMAGE64_MAGIC_V0 0x14000008
> > +#define ZIMAGE64_MAGIC_V1 0x644d5241 /* "ARM\x64" */
> > +
> > +/* linux/Documentation/arm64/booting.txt */
> > +struct zimage64_hdr {
> > +    uint32_t magic0;
> > +    uint32_t res0;
> > +    uint64_t text_offset;  /* Image load offset */
> > +    uint64_t res1;
> > +    uint64_t res2;
> > +    /* zImage V1 only from here */
> > +    uint64_t res3;
> > +    uint64_t res4;
> > +    uint64_t res5;
> > +    uint32_t magic1;
> > +    uint32_t res6;
> > +};
> > +static int xc_dom_probe_zimage64_kernel(struct xc_dom_image *dom)
> > +{
> > +    struct zimage64_hdr *zimage;
> > +
> > +    if ( dom->kernel_blob == NULL )
> > +    {
> > +        xc_dom_panic(dom->xch, XC_INTERNAL_ERROR,
> > +                     "%s: no kernel image loaded", __FUNCTION__);
> > +        return -EINVAL;
> > +    }
> > +
> > +    if ( dom->kernel_size < sizeof(*zimage) )
> > +    {
> > +        xc_dom_printf(dom->xch, "%s: kernel image too small", __FUNCTION__);
> 
> What about using DOMPRINTF instead of xc_dom_printf?

We are pretty inconsistent in this file, but it looks like I've managed
to use the same thing the 32-bit version does in each case (without even
thinking about it).

I think it is probably ripe for a future cleanup rather than doing a
partial switch here.

> In any case:
> Acked-by: Julien Grall <julien.grall@linaro.org>

Thanks.

Ian.

[PATCH v5 18/19] libxc: arm64 vcpu initialisation

[Ian Campbell]

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Acked-by: Julien Grall <julien.grall@linaro.org>
---
 tools/libxc/xc_dom_arm.c |   90 ++++++++++++++++++++++++++++++++++++++++++++--
 1 file changed, 88 insertions(+), 2 deletions(-)

diff --git a/tools/libxc/xc_dom_arm.c b/tools/libxc/xc_dom_arm.c
index 4c59177..a5ff01d 100644
--- a/tools/libxc/xc_dom_arm.c
+++ b/tools/libxc/xc_dom_arm.c
@@ -105,7 +105,7 @@ static int shared_info_arm(struct xc_dom_image *dom, void *ptr)
 
 /* ------------------------------------------------------------------------ */
 
-static int vcpu_arm(struct xc_dom_image *dom, void *ptr)
+static int vcpu_arm32(struct xc_dom_image *dom, void *ptr)
 {
     vcpu_guest_context_t *ctxt = ptr;
 
@@ -143,6 +143,41 @@ static int vcpu_arm(struct xc_dom_image *dom, void *ptr)
     return 0;
 }
 
+static int vcpu_arm64(struct xc_dom_image *dom, void *ptr)
+{
+    vcpu_guest_context_t *ctxt = ptr;
+
+    DOMPRINTF_CALLED(dom->xch);
+    /* clear everything */
+    memset(ctxt, 0, sizeof(*ctxt));
+
+    ctxt->user_regs.pc64 = dom->parms.virt_entry;
+
+    /* Linux boot protocol. See linux.Documentation/arm64/booting.txt. */
+    ctxt->user_regs.x0 = dom->devicetree_blob ?
+        dom->devicetree_seg.vstart : 0xffffffff;
+    ctxt->user_regs.x1 = 0;
+    ctxt->user_regs.x2 = 0;
+    ctxt->user_regs.x3 = 0;
+
+    DOMPRINTF("DTB %"PRIx64, ctxt->user_regs.x0);
+
+    ctxt->sctlr = SCTLR_GUEST_INIT;
+
+    ctxt->ttbr0 = 0;
+    ctxt->ttbr1 = 0;
+    ctxt->ttbcr = 0; /* Defined Reset Value */
+
+    ctxt->user_regs.cpsr = PSR_GUEST64_INIT;
+
+    ctxt->flags = VGCF_online;
+
+    DOMPRINTF("Initial state CPSR %#"PRIx32" PC %#"PRIx64,
+           ctxt->user_regs.cpsr, ctxt->user_regs.pc64);
+
+    return 0;
+}
+
 /* ------------------------------------------------------------------------ */
 
 static struct xc_dom_arch xc_dom_32 = {
@@ -155,12 +190,59 @@ static struct xc_dom_arch xc_dom_32 = {
     .setup_pgtables = setup_pgtables_arm,
     .start_info = start_info_arm,
     .shared_info = shared_info_arm,
-    .vcpu = vcpu_arm,
+    .vcpu = vcpu_arm32,
+};
+
+static struct xc_dom_arch xc_dom_64 = {
+    .guest_type = "xen-3.0-aarch64",
+    .native_protocol = XEN_IO_PROTO_ABI_ARM,
+    .page_shift = PAGE_SHIFT_ARM,
+    .sizeof_pfn = 8,
+    .alloc_magic_pages = alloc_magic_pages,
+    .count_pgtables = count_pgtables_arm,
+    .setup_pgtables = setup_pgtables_arm,
+    .start_info = start_info_arm,
+    .shared_info = shared_info_arm,
+    .vcpu = vcpu_arm64,
 };
 
 static void __init register_arch_hooks(void)
 {
     xc_dom_register_arch_hooks(&xc_dom_32);
+    xc_dom_register_arch_hooks(&xc_dom_64);
+}
+
+static int set_mode(xc_interface *xch, domid_t domid, char *guest_type)
+{
+    static const struct {
+        char           *guest;
+        uint32_t        size;
+    } types[] = {
+        { "xen-3.0-aarch64", 64 },
+        { "xen-3.0-armv7l",  32 },
+    };
+    DECLARE_DOMCTL;
+    int i,rc;
+
+    domctl.domain = domid;
+    domctl.cmd    = XEN_DOMCTL_set_address_size;
+    for ( i = 0; i < sizeof(types)/sizeof(types[0]); i++ )
+        if ( !strcmp(types[i].guest, guest_type) )
+            domctl.u.address_size.size = types[i].size;
+    if ( domctl.u.address_size.size == 0 )
+    {
+        xc_dom_printf(xch, "%s: warning: unknown guest type %s",
+                      __FUNCTION__, guest_type);
+        return -EINVAL;
+    }
+
+    xc_dom_printf(xch, "%s: guest %s, address size %" PRId32 "", __FUNCTION__,
+                  guest_type, domctl.u.address_size.size);
+    rc = do_domctl(xch, &domctl);
+    if ( rc != 0 )
+        xc_dom_printf(xch, "%s: warning: failed (rc=%d)",
+                      __FUNCTION__, rc);
+    return rc;
 }
 
 int arch_setup_meminit(struct xc_dom_image *dom)
@@ -168,6 +250,10 @@ int arch_setup_meminit(struct xc_dom_image *dom)
     int rc;
     xen_pfn_t pfn, allocsz, i;
 
+    rc = set_mode(dom->xch, dom->guest_domid, dom->guest_type);
+    if ( rc )
+        return rc;
+
     dom->shadow_enabled = 1;
 
     dom->p2m_host = xc_dom_malloc(dom, sizeof(xen_pfn_t) * dom->total_pages);
-- 
1.7.10.4

[PATCH v5 19/19] libxl: build a device tree for ARM guests

[Ian Campbell]

Uses xc_dom_devicetree_mem which was just added. The call to this needs to be
carefully sequenced to be after xc_dom_parse_image (so we can tell which kind
of guest we are building, although we don't use this yet) and before
xc_dom_mem_init which tries to decide where to place the FDT in guest RAM.

Removes libxl_noarch which would only have been used by IA64 after this
change. Remove IA64 as part of this patch.

There is no attempt to expose this as a configuration setting for the user.

Includes a debug hook to dump the dtb to a file for inspection.

TODO:
- v7 CPU compat is hardcoded to cortex-a15 -- may need to define something more
  generic via mach-virt dt bindngs?

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Cc: Ian Jackson <ian.jackson@eu.citrix.com>
---
v5: Correct error handling in debug_dump_fdt
v4: Drop spurious comment in header
    s/__be32/be32/ and s/gic_interrupt_t/gic_interrupt/ to avoid reserved names
    Coding style fixes
    Use GCSPRINTF
    use for(;;) around FDT creation loop, undef FDT when done
    use libxl__realloc for fdt size increase
    Refactor debug dump into its own function, remove NDEBUG ifdef
v2: base addresses, irq, evtchn etc stuff is now from public API headers,
    avoiding the need to introduce domctls etc until we want to make them
    dynamic.
    fix memory node
    Improve libfdt error handling, especially for FDT_ERR_NOSPACE.
    Derive guest CPU and timer compatiblity nodes from the guest type.

wip
---
 tools/libxl/Makefile       |    6 +-
 tools/libxl/libxl_arch.h   |    3 +
 tools/libxl/libxl_arm.c    |  512 ++++++++++++++++++++++++++++++++++++++++++++
 tools/libxl/libxl_dom.c    |    4 +
 tools/libxl/libxl_noarch.c |    8 -
 tools/libxl/libxl_x86.c    |    7 +
 6 files changed, 530 insertions(+), 10 deletions(-)
 create mode 100644 tools/libxl/libxl_arm.c
 delete mode 100644 tools/libxl/libxl_noarch.c

diff --git a/tools/libxl/Makefile b/tools/libxl/Makefile
index cf214bb..d8495bb 100644
--- a/tools/libxl/Makefile
+++ b/tools/libxl/Makefile
@@ -28,9 +28,12 @@ CFLAGS_LIBXL += $(CFLAGS_libxenstore)
 CFLAGS_LIBXL += $(CFLAGS_libblktapctl) 
 CFLAGS_LIBXL += -Wshadow
 
+LIBXL_LIBS-$(CONFIG_ARM) += -lfdt
+
 CFLAGS += $(PTHREAD_CFLAGS)
 LDFLAGS += $(PTHREAD_LDFLAGS)
 LIBXL_LIBS += $(PTHREAD_LIBS)
+LIBXL_LIBS += $(LIBXL_LIBS-y)
 
 LIBXLU_LIBS =
 
@@ -41,8 +44,7 @@ else
 LIBXL_OBJS-y += libxl_noblktap2.o
 endif
 LIBXL_OBJS-$(CONFIG_X86) += libxl_cpuid.o libxl_x86.o
-LIBXL_OBJS-$(CONFIG_IA64) += libxl_nocpuid.o libxl_noarch.o
-LIBXL_OBJS-$(CONFIG_ARM) += libxl_nocpuid.o libxl_noarch.o
+LIBXL_OBJS-$(CONFIG_ARM) += libxl_nocpuid.o libxl_arm.o
 
 ifeq ($(CONFIG_NetBSD),y)
 LIBXL_OBJS-y += libxl_netbsd.o
diff --git a/tools/libxl/libxl_arch.h b/tools/libxl/libxl_arch.h
index abe6685..aee0a91 100644
--- a/tools/libxl/libxl_arch.h
+++ b/tools/libxl/libxl_arch.h
@@ -19,4 +19,7 @@
 int libxl__arch_domain_create(libxl__gc *gc, libxl_domain_config *d_config,
                uint32_t domid);
 
+int libxl__arch_domain_configure(libxl__gc *gc,
+                                 libxl_domain_build_info *info,
+                                 struct xc_dom_image *dom);
 #endif
diff --git a/tools/libxl/libxl_arm.c b/tools/libxl/libxl_arm.c
new file mode 100644
index 0000000..0a1c8c5
--- /dev/null
+++ b/tools/libxl/libxl_arm.c
@@ -0,0 +1,512 @@
+#include "libxl_internal.h"
+#include "libxl_arch.h"
+
+#include <xc_dom.h>
+#include <libfdt.h>
+#include <assert.h>
+
+int libxl__arch_domain_create(libxl__gc *gc, libxl_domain_config *d_config,
+                              uint32_t domid)
+{
+    return 0;
+}
+
+static struct arch_info {
+    const char *guest_type;
+    const char *timer_compat;
+    const char *cpu_compat;
+} arch_info[] = {
+    {"xen-3.0-armv7l",  "arm,armv7-timer", "arm,cortex-a15" },
+    {"xen-3.0-aarch64", "arm,armv8-timer", "arm,armv8" },
+};
+
+enum {
+    PHANDLE_NONE = 0,
+    PHANDLE_GIC,
+};
+
+typedef uint32_t be32;
+typedef be32 gic_interrupt[3];
+
+#define ROOT_ADDRESS_CELLS 2
+#define ROOT_SIZE_CELLS 2
+
+static void set_cell(be32 **cellp, int size, uint64_t val)
+{
+    int cells = size;
+
+    while (size--) {
+        (*cellp)[size] = cpu_to_fdt32(val);
+        val >>= 32;
+    }
+
+    (*cellp) += cells;
+}
+
+static void set_interrupt_ppi(gic_interrupt interrupt, unsigned int irq,
+                              unsigned int cpumask, unsigned int level)
+{
+    be32 *cells = interrupt;
+
+    /* See linux Documentation/devictree/bindings/arm/gic.txt */
+    set_cell(&cells, 1, 1); /* is a PPI */
+    set_cell(&cells, 1, irq - 16); /* PPIs start at 16 */
+    set_cell(&cells, 1, (cpumask << 8) | level);
+}
+
+static void set_range(be32 **cellp,
+                      int address_cells, int size_cells,
+                      uint64_t address, uint64_t size)
+{
+    set_cell(cellp, address_cells, address);
+    set_cell(cellp, size_cells, size);
+}
+
+static int fdt_property_compat(libxl__gc *gc, void *fdt, unsigned nr_compat, ...)
+{
+    const char *compats[nr_compat];
+    int i;
+    size_t sz;
+    va_list ap;
+    char *compat, *p;
+
+    va_start(ap, nr_compat);
+    sz = 0;
+    for (i = 0; i < nr_compat; i++) {
+        const char *c = va_arg(ap, const char *);
+        compats[i] = c;
+        sz += strlen(compats[i]) + 1;
+    }
+    va_end(ap);
+
+    p = compat = libxl__zalloc(gc, sz);
+    for (i = 0; i < nr_compat; i++) {
+        strcpy(p, compats[i]);
+        p += strlen(compats[i]) + 1;
+    }
+
+    return fdt_property(fdt, "compatible", compat, sz);
+}
+
+static int fdt_property_interrupts(libxl__gc *gc, void *fdt,
+                                   gic_interrupt *intr,
+                                   unsigned num_irq)
+{
+    int res;
+
+    res = fdt_property(fdt, "interrupts", intr, sizeof (intr[0]) * num_irq);
+    if (res) return res;
+
+    res = fdt_property_cell(fdt, "interrupt-parent", PHANDLE_GIC);
+    if (res) return res;
+
+    return 0;
+}
+
+static int fdt_property_regs(libxl__gc *gc, void *fdt,
+                             unsigned addr_cells,
+                             unsigned size_cells,
+                             unsigned num_regs, ...)
+{
+    uint32_t regs[num_regs*(addr_cells+size_cells)];
+    be32 *cells = &regs[0];
+    int i;
+    va_list ap;
+    uint64_t base, size;
+
+    va_start(ap, num_regs);
+    for (i = 0 ; i < num_regs; i++) {
+        base = addr_cells ? va_arg(ap, uint64_t) : 0;
+        size = size_cells ? va_arg(ap, uint64_t) : 0;
+        set_range(&cells, addr_cells, size_cells, base, size);
+    }
+    va_end(ap);
+
+    return fdt_property(fdt, "reg", regs, sizeof(regs));
+}
+
+static int make_root_properties(libxl__gc *gc,
+                                const libxl_version_info *vers,
+                                void *fdt)
+{
+    int res;
+
+    res = fdt_property_string(fdt, "model", GCSPRINTF("XENVM-%d.%d",
+                                                      vers->xen_version_major,
+                                                      vers->xen_version_minor));
+    if (res) return res;
+
+    res = fdt_property_compat(gc, fdt, 2,
+                              GCSPRINTF("xen,xenvm-%d.%d",
+                                        vers->xen_version_major,
+                                        vers->xen_version_minor),
+                              "xen,xenvm");
+    if (res) return res;
+
+    res = fdt_property_cell(fdt, "interrupt-parent", PHANDLE_GIC);
+    if (res) return res;
+
+    res = fdt_property_cell(fdt, "#address-cells", ROOT_ADDRESS_CELLS);
+    if (res) return res;
+
+    res = fdt_property_cell(fdt, "#size-cells", ROOT_SIZE_CELLS);
+    if (res) return res;
+
+    return 0;
+}
+
+static int make_chosen_node(libxl__gc *gc, void *fdt,
+                            const libxl_domain_build_info *info)
+{
+    int res;
+
+    /* See linux Documentation/devicetree/... */
+    res = fdt_begin_node(fdt, "chosen");
+    if (res) return res;
+
+    res = fdt_property_string(fdt, "bootargs", info->u.pv.cmdline);
+    if (res) return res;
+
+    res = fdt_end_node(fdt);
+    if (res) return res;
+
+    return 0;
+}
+
+static int make_cpus_node(libxl__gc *gc, void *fdt, int nr_cpus,
+                          const struct arch_info *ainfo)
+{
+    int res, i;
+
+    res = fdt_begin_node(fdt, "cpus");
+    if (res) return res;
+
+    res = fdt_property_cell(fdt, "#address-cells", 1);
+    if (res) return res;
+
+    res = fdt_property_cell(fdt, "#size-cells", 0);
+    if (res) return res;
+
+    for (i = 0; i < nr_cpus; i++) {
+        const char *name = GCSPRINTF("cpu@%d", i);
+
+        res = fdt_begin_node(fdt, name);
+        if (res) return res;
+
+        res = fdt_property_string(fdt, "device_type", "cpu");
+        if (res) return res;
+
+        res = fdt_property_compat(gc, fdt, 1, ainfo->cpu_compat);
+        if (res) return res;
+
+        res = fdt_property_string(fdt, "enable-method", "psci");
+        if (res) return res;
+
+        res = fdt_property_regs(gc, fdt, 1, 0, 1, (uint64_t)i);
+        if (res) return res;
+
+        res = fdt_end_node(fdt);
+        if (res) return res;
+    }
+
+    res = fdt_end_node(fdt);
+    if (res) return res;
+
+    return 0;
+}
+
+static int make_psci_node(libxl__gc *gc, void *fdt)
+{
+    int res;
+
+    res = fdt_begin_node(fdt, "psci");
+    if (res) return res;
+
+    res = fdt_property_compat(gc, fdt, 1, "arm,psci");
+    if (res) return res;
+
+    res = fdt_property_string(fdt, "method", "hvc");
+    if (res) return res;
+
+    res = fdt_property_cell(fdt, "cpu_off", PSCI_cpu_off);
+    if (res) return res;
+
+    res = fdt_property_cell(fdt, "cpu_on", PSCI_cpu_on);
+    if (res) return res;
+
+    res = fdt_end_node(fdt);
+    if (res) return res;
+
+    return 0;
+}
+
+static int make_memory_node(libxl__gc *gc, void *fdt,
+                            unsigned long long base,
+                            unsigned long long size)
+{
+    int res;
+    const char *name = GCSPRINTF("memory@%08llx", base);
+
+    res = fdt_begin_node(fdt, name);
+    if (res) return res;
+
+    res = fdt_property_string(fdt, "device_type", "memory");
+    if (res) return res;
+
+    res = fdt_property_regs(gc, fdt, ROOT_ADDRESS_CELLS, ROOT_SIZE_CELLS,
+                            1, (uint64_t)base, (uint64_t)size);
+    if (res) return res;
+
+    res = fdt_end_node(fdt);
+    if (res) return res;
+
+    return 0;
+}
+
+static int make_intc_node(libxl__gc *gc, void *fdt,
+                          unsigned long long gicd_base,
+                          unsigned long long gicd_size,
+                          unsigned long long gicc_base,
+                          unsigned long long gicc_size)
+{
+    int res;
+    const char *name = GCSPRINTF("interrupt-controller@%08llx", gicd_base);
+
+    res = fdt_begin_node(fdt, name);
+    if (res) return res;
+
+    res = fdt_property_compat(gc, fdt, 2,
+                              "arm,cortex-a15-gic",
+                              "arm,cortex-a9-gic");
+    if (res) return res;
+
+
+    res = fdt_property_cell(fdt, "#interrupt-cells", 3);
+    if (res) return res;
+
+    res = fdt_property_cell(fdt, "#address-cells", 0);
+    if (res) return res;
+
+    res = fdt_property(fdt, "interrupt-controller", NULL, 0);
+    if (res) return res;
+
+    res = fdt_property_regs(gc, fdt, ROOT_ADDRESS_CELLS, ROOT_SIZE_CELLS,
+                            2,
+                            (uint64_t)gicd_base, (uint64_t)gicd_size,
+                            (uint64_t)gicc_base, (uint64_t)gicc_size);
+    if (res) return res;
+
+    res = fdt_property_cell(fdt, "linux,phandle", PHANDLE_GIC);
+    if (res) return res;
+
+    res = fdt_property_cell(fdt, "phandle", PHANDLE_GIC);
+    if (res) return res;
+
+    res = fdt_end_node(fdt);
+    if (res) return res;
+
+    return 0;
+}
+
+static int make_timer_node(libxl__gc *gc, void *fdt, const struct arch_info *ainfo)
+{
+    int res;
+    gic_interrupt ints[3];
+
+    res = fdt_begin_node(fdt, "timer");
+    if (res) return res;
+
+    res = fdt_property_compat(gc, fdt, 1, ainfo->timer_compat);
+    if (res) return res;
+
+    set_interrupt_ppi(ints[0], GUEST_TIMER_PHYS_S_PPI, 0xf, 0x8);
+    set_interrupt_ppi(ints[1], GUEST_TIMER_PHYS_NS_PPI, 0xf, 0x8);
+    set_interrupt_ppi(ints[2], GUEST_TIMER_VIRT_PPI, 0xf, 0x8);
+
+    res = fdt_property_interrupts(gc, fdt, ints, 3);
+    if (res) return res;
+
+    res = fdt_end_node(fdt);
+    if (res) return res;
+
+    return 0;
+}
+
+static int make_hypervisor_node(libxl__gc *gc, void *fdt,
+                                const libxl_version_info *vers)
+{
+    int res;
+    gic_interrupt intr;
+
+    /* See linux Documentation/devicetree/bindings/arm/xen.txt */
+    res = fdt_begin_node(fdt, "hypervisor");
+    if (res) return res;
+
+    res = fdt_property_compat(gc, fdt, 2,
+                              GCSPRINTF("xen,xen-%d.%d",
+                                        vers->xen_version_major,
+                                        vers->xen_version_minor),
+                              "xen,xen");
+    if (res) return res;
+
+    /* reg 0 is grant table space */
+    res = fdt_property_regs(gc, fdt, ROOT_ADDRESS_CELLS, ROOT_SIZE_CELLS,
+                            1,GUEST_GNTTAB_BASE, GUEST_GNTTAB_SIZE);
+    if (res) return res;
+
+    /*
+     * interrupts is evtchn upcall:
+     *  - Active-low level-sensitive
+     *  - All cpus
+     */
+    set_interrupt_ppi(intr, GUEST_EVTCHN_PPI, 0xf, 0x8);
+
+    res = fdt_property_interrupts(gc, fdt, &intr, 1);
+    if (res) return res;
+
+    res = fdt_end_node(fdt);
+    if (res) return res;
+
+    return 0;
+}
+
+static const struct arch_info *get_arch_info(libxl__gc *gc,
+                                             const struct xc_dom_image *dom)
+{
+    int i;
+
+    for (i=0; i < ARRAY_SIZE(arch_info); i++) {
+        const struct arch_info *info = &arch_info[i];
+        if (!strcmp(dom->guest_type, info->guest_type))
+            return info;
+    }
+    LOG(ERROR, "Unable to find arch FDT info for %s\n", dom->guest_type);
+    return NULL;
+}
+
+static void debug_dump_fdt(libxl__gc *gc, void *fdt)
+{
+    int fd = -1, rc, r;
+
+    const char *dtb = getenv("LIBXL_DEBUG_DUMP_DTB");
+
+    if (!dtb) goto out;
+
+    fd = open(dtb, O_CREAT|O_TRUNC|O_WRONLY, 0666);
+    if (fd < 0) {
+        LOGE(DEBUG, "cannot open %s for LIBXL_DEBUG_DUMP_DTB", dtb);
+        goto out;
+    }
+
+    rc = libxl_write_exactly(CTX, fd, fdt, fdt_totalsize(fdt), dtb, "dtb");
+    if (rc < 0) goto out;
+
+out:
+    if (fd >= 0) {
+        r = close(fd);
+        if (r < 0) LOGE(DEBUG, "failed to close DTB debug dump output");
+    }
+}
+
+#define FDT_MAX_SIZE (1<<20)
+
+int libxl__arch_domain_configure(libxl__gc *gc,
+                                 libxl_domain_build_info *info,
+                                 struct xc_dom_image *dom)
+{
+    void *fdt = NULL;
+    int rc, res;
+    size_t fdt_size = 0;
+
+    const libxl_version_info *vers;
+    const struct arch_info *ainfo;
+
+    assert(info->type == LIBXL_DOMAIN_TYPE_PV);
+
+    vers = libxl_get_version_info(CTX);
+    if (vers == NULL) return ERROR_FAIL;
+
+    ainfo = get_arch_info(gc, dom);
+    if (ainfo == NULL) return ERROR_FAIL;
+
+    LOG(DEBUG, "constructing DTB for Xen version %d.%d guest",
+        vers->xen_version_major, vers->xen_version_minor);
+
+/*
+ * Call "call" handling FDR_ERR_*. Will either:
+ * - loop back to retry_resize
+ * - set rc and goto out
+ * - fall through successfully
+ *
+ * On FDT_ERR_NOSPACE we start again from scratch rather than
+ * realloc+libfdt_open_into because "call" may have failed half way
+ * through a series of steps leaving the partial tree in an
+ * inconsistent state, e.g. leaving a node open.
+ */
+#define FDT( call ) do {                                        \
+    int fdt_res = (call);                                       \
+    if (fdt_res == -FDT_ERR_NOSPACE && fdt_size < FDT_MAX_SIZE) \
+        goto next_resize;                                       \
+    else if (fdt_res < 0) {                                     \
+        LOG(ERROR, "FDT: %s failed: %d = %s",                   \
+            #call, fdt_res, fdt_strerror(fdt_res));             \
+        rc = ERROR_FAIL;                                        \
+        goto out;                                               \
+    }                                                           \
+} while(0)
+
+    for (;;) {
+next_resize:
+        if (fdt_size) {
+            fdt_size <<= 1;
+            LOG(DEBUG, "Increasing FDT size to %zd and retrying", fdt_size);
+        } else {
+            fdt_size = 4096;
+        }
+
+        fdt = libxl__realloc(gc, fdt, fdt_size);
+
+        FDT( fdt_create(fdt, fdt_size) );
+
+        FDT( fdt_finish_reservemap(fdt) );
+
+        FDT( fdt_begin_node(fdt, "") );
+
+        FDT( make_root_properties(gc, vers, fdt) );
+        FDT( make_chosen_node(gc, fdt, info) );
+        FDT( make_cpus_node(gc, fdt, info->max_vcpus, ainfo) );
+        FDT( make_psci_node(gc, fdt) );
+
+        FDT( make_memory_node(gc, fdt,
+                              dom->rambase_pfn << XC_PAGE_SHIFT,
+                              info->target_memkb * 1024) );
+        FDT( make_intc_node(gc, fdt,
+                            GUEST_GICD_BASE, GUEST_GICD_SIZE,
+                            GUEST_GICC_BASE, GUEST_GICD_SIZE) );
+
+        FDT( make_timer_node(gc, fdt, ainfo) );
+        FDT( make_hypervisor_node(gc, fdt, vers) );
+
+        FDT( fdt_end_node(fdt) );
+
+        FDT( fdt_finish(fdt) );
+        break;
+    }
+#undef FDT
+
+    LOG(DEBUG, "fdt total size %d", fdt_totalsize(fdt));
+
+    res = xc_dom_devicetree_mem(dom, fdt, fdt_totalsize(fdt));
+    if (res) {
+        LOGE(ERROR, "xc_dom_devicetree_file failed");
+        rc = ERROR_FAIL;
+        goto out;
+    }
+
+    debug_dump_fdt(gc, fdt);
+
+    rc = 0;
+
+out:
+    return rc;
+}
diff --git a/tools/libxl/libxl_dom.c b/tools/libxl/libxl_dom.c
index 55902af..8ec96e3 100644
--- a/tools/libxl/libxl_dom.c
+++ b/tools/libxl/libxl_dom.c
@@ -400,6 +400,10 @@ int libxl__build_pv(libxl__gc *gc, uint32_t domid,
         LOGE(ERROR, "xc_dom_parse_image failed");
         goto out;
     }
+    if ( (ret = libxl__arch_domain_configure(gc, info, dom)) != 0 ) {
+        LOGE(ERROR, "libxl__arch_domain_configure failed");
+        goto out;
+    }
     if ( (ret = xc_dom_mem_init(dom, info->target_memkb / 1024)) != 0 ) {
         LOGE(ERROR, "xc_dom_mem_init failed");
         goto out;
diff --git a/tools/libxl/libxl_noarch.c b/tools/libxl/libxl_noarch.c
deleted file mode 100644
index 7893535..0000000
--- a/tools/libxl/libxl_noarch.c
+++ /dev/null
@@ -1,8 +0,0 @@
-#include "libxl_internal.h"
-#include "libxl_arch.h"
-
-int libxl__arch_domain_create(libxl__gc *gc, libxl_domain_config *d_config,
-                              uint32_t domid)
-{
-    return 0;
-}
diff --git a/tools/libxl/libxl_x86.c b/tools/libxl/libxl_x86.c
index a78c91d..dd13c45 100644
--- a/tools/libxl/libxl_x86.c
+++ b/tools/libxl/libxl_x86.c
@@ -308,3 +308,10 @@ int libxl__arch_domain_create(libxl__gc *gc, libxl_domain_config *d_config,
 
     return ret;
 }
+
+int libxl__arch_domain_configure(libxl__gc *gc,
+                                 libxl_domain_build_info *info,
+                                 struct xc_dom_image *dom)
+{
+    return 0;
+}
-- 
1.7.10.4

Re: [PATCH v5 19/19] libxl: build a device tree for ARM guests

[Stefano Stabellini]

On Wed, 13 Nov 2013, Ian Campbell wrote:
> Uses xc_dom_devicetree_mem which was just added. The call to this needs to be
> carefully sequenced to be after xc_dom_parse_image (so we can tell which kind
> of guest we are building, although we don't use this yet) and before
> xc_dom_mem_init which tries to decide where to place the FDT in guest RAM.
> 
> Removes libxl_noarch which would only have been used by IA64 after this
> change. Remove IA64 as part of this patch.
> 
> There is no attempt to expose this as a configuration setting for the user.
> 
> Includes a debug hook to dump the dtb to a file for inspection.
> 
> TODO:
> - v7 CPU compat is hardcoded to cortex-a15 -- may need to define something more
>   generic via mach-virt dt bindngs?
> 
> Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
> Cc: Ian Jackson <ian.jackson@eu.citrix.com>

Acked-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>


> v5: Correct error handling in debug_dump_fdt
> v4: Drop spurious comment in header
>     s/__be32/be32/ and s/gic_interrupt_t/gic_interrupt/ to avoid reserved names
>     Coding style fixes
>     Use GCSPRINTF
>     use for(;;) around FDT creation loop, undef FDT when done
>     use libxl__realloc for fdt size increase
>     Refactor debug dump into its own function, remove NDEBUG ifdef
> v2: base addresses, irq, evtchn etc stuff is now from public API headers,
>     avoiding the need to introduce domctls etc until we want to make them
>     dynamic.
>     fix memory node
>     Improve libfdt error handling, especially for FDT_ERR_NOSPACE.
>     Derive guest CPU and timer compatiblity nodes from the guest type.
> 
> wip
> ---
>  tools/libxl/Makefile       |    6 +-
>  tools/libxl/libxl_arch.h   |    3 +
>  tools/libxl/libxl_arm.c    |  512 ++++++++++++++++++++++++++++++++++++++++++++
>  tools/libxl/libxl_dom.c    |    4 +
>  tools/libxl/libxl_noarch.c |    8 -
>  tools/libxl/libxl_x86.c    |    7 +
>  6 files changed, 530 insertions(+), 10 deletions(-)
>  create mode 100644 tools/libxl/libxl_arm.c
>  delete mode 100644 tools/libxl/libxl_noarch.c
> 
> diff --git a/tools/libxl/Makefile b/tools/libxl/Makefile
> index cf214bb..d8495bb 100644
> --- a/tools/libxl/Makefile
> +++ b/tools/libxl/Makefile
> @@ -28,9 +28,12 @@ CFLAGS_LIBXL += $(CFLAGS_libxenstore)
>  CFLAGS_LIBXL += $(CFLAGS_libblktapctl) 
>  CFLAGS_LIBXL += -Wshadow
>  
> +LIBXL_LIBS-$(CONFIG_ARM) += -lfdt
> +
>  CFLAGS += $(PTHREAD_CFLAGS)
>  LDFLAGS += $(PTHREAD_LDFLAGS)
>  LIBXL_LIBS += $(PTHREAD_LIBS)
> +LIBXL_LIBS += $(LIBXL_LIBS-y)
>  
>  LIBXLU_LIBS =
>  
> @@ -41,8 +44,7 @@ else
>  LIBXL_OBJS-y += libxl_noblktap2.o
>  endif
>  LIBXL_OBJS-$(CONFIG_X86) += libxl_cpuid.o libxl_x86.o
> -LIBXL_OBJS-$(CONFIG_IA64) += libxl_nocpuid.o libxl_noarch.o
> -LIBXL_OBJS-$(CONFIG_ARM) += libxl_nocpuid.o libxl_noarch.o
> +LIBXL_OBJS-$(CONFIG_ARM) += libxl_nocpuid.o libxl_arm.o
>  
>  ifeq ($(CONFIG_NetBSD),y)
>  LIBXL_OBJS-y += libxl_netbsd.o
> diff --git a/tools/libxl/libxl_arch.h b/tools/libxl/libxl_arch.h
> index abe6685..aee0a91 100644
> --- a/tools/libxl/libxl_arch.h
> +++ b/tools/libxl/libxl_arch.h
> @@ -19,4 +19,7 @@
>  int libxl__arch_domain_create(libxl__gc *gc, libxl_domain_config *d_config,
>                 uint32_t domid);
>  
> +int libxl__arch_domain_configure(libxl__gc *gc,
> +                                 libxl_domain_build_info *info,
> +                                 struct xc_dom_image *dom);
>  #endif
> diff --git a/tools/libxl/libxl_arm.c b/tools/libxl/libxl_arm.c
> new file mode 100644
> index 0000000..0a1c8c5
> --- /dev/null
> +++ b/tools/libxl/libxl_arm.c
> @@ -0,0 +1,512 @@
> +#include "libxl_internal.h"
> +#include "libxl_arch.h"
> +
> +#include <xc_dom.h>
> +#include <libfdt.h>
> +#include <assert.h>
> +
> +int libxl__arch_domain_create(libxl__gc *gc, libxl_domain_config *d_config,
> +                              uint32_t domid)
> +{
> +    return 0;
> +}
> +
> +static struct arch_info {
> +    const char *guest_type;
> +    const char *timer_compat;
> +    const char *cpu_compat;
> +} arch_info[] = {
> +    {"xen-3.0-armv7l",  "arm,armv7-timer", "arm,cortex-a15" },
> +    {"xen-3.0-aarch64", "arm,armv8-timer", "arm,armv8" },
> +};
> +
> +enum {
> +    PHANDLE_NONE = 0,
> +    PHANDLE_GIC,
> +};
> +
> +typedef uint32_t be32;
> +typedef be32 gic_interrupt[3];
> +
> +#define ROOT_ADDRESS_CELLS 2
> +#define ROOT_SIZE_CELLS 2
> +
> +static void set_cell(be32 **cellp, int size, uint64_t val)
> +{
> +    int cells = size;
> +
> +    while (size--) {
> +        (*cellp)[size] = cpu_to_fdt32(val);
> +        val >>= 32;
> +    }
> +
> +    (*cellp) += cells;
> +}
> +
> +static void set_interrupt_ppi(gic_interrupt interrupt, unsigned int irq,
> +                              unsigned int cpumask, unsigned int level)
> +{
> +    be32 *cells = interrupt;
> +
> +    /* See linux Documentation/devictree/bindings/arm/gic.txt */
> +    set_cell(&cells, 1, 1); /* is a PPI */
> +    set_cell(&cells, 1, irq - 16); /* PPIs start at 16 */
> +    set_cell(&cells, 1, (cpumask << 8) | level);
> +}
> +
> +static void set_range(be32 **cellp,
> +                      int address_cells, int size_cells,
> +                      uint64_t address, uint64_t size)
> +{
> +    set_cell(cellp, address_cells, address);
> +    set_cell(cellp, size_cells, size);
> +}
> +
> +static int fdt_property_compat(libxl__gc *gc, void *fdt, unsigned nr_compat, ...)
> +{
> +    const char *compats[nr_compat];
> +    int i;
> +    size_t sz;
> +    va_list ap;
> +    char *compat, *p;
> +
> +    va_start(ap, nr_compat);
> +    sz = 0;
> +    for (i = 0; i < nr_compat; i++) {
> +        const char *c = va_arg(ap, const char *);
> +        compats[i] = c;
> +        sz += strlen(compats[i]) + 1;
> +    }
> +    va_end(ap);
> +
> +    p = compat = libxl__zalloc(gc, sz);
> +    for (i = 0; i < nr_compat; i++) {
> +        strcpy(p, compats[i]);
> +        p += strlen(compats[i]) + 1;
> +    }
> +
> +    return fdt_property(fdt, "compatible", compat, sz);
> +}
> +
> +static int fdt_property_interrupts(libxl__gc *gc, void *fdt,
> +                                   gic_interrupt *intr,
> +                                   unsigned num_irq)
> +{
> +    int res;
> +
> +    res = fdt_property(fdt, "interrupts", intr, sizeof (intr[0]) * num_irq);
> +    if (res) return res;
> +
> +    res = fdt_property_cell(fdt, "interrupt-parent", PHANDLE_GIC);
> +    if (res) return res;
> +
> +    return 0;
> +}
> +
> +static int fdt_property_regs(libxl__gc *gc, void *fdt,
> +                             unsigned addr_cells,
> +                             unsigned size_cells,
> +                             unsigned num_regs, ...)
> +{
> +    uint32_t regs[num_regs*(addr_cells+size_cells)];
> +    be32 *cells = &regs[0];
> +    int i;
> +    va_list ap;
> +    uint64_t base, size;
> +
> +    va_start(ap, num_regs);
> +    for (i = 0 ; i < num_regs; i++) {
> +        base = addr_cells ? va_arg(ap, uint64_t) : 0;
> +        size = size_cells ? va_arg(ap, uint64_t) : 0;
> +        set_range(&cells, addr_cells, size_cells, base, size);
> +    }
> +    va_end(ap);
> +
> +    return fdt_property(fdt, "reg", regs, sizeof(regs));
> +}
> +
> +static int make_root_properties(libxl__gc *gc,
> +                                const libxl_version_info *vers,
> +                                void *fdt)
> +{
> +    int res;
> +
> +    res = fdt_property_string(fdt, "model", GCSPRINTF("XENVM-%d.%d",
> +                                                      vers->xen_version_major,
> +                                                      vers->xen_version_minor));
> +    if (res) return res;
> +
> +    res = fdt_property_compat(gc, fdt, 2,
> +                              GCSPRINTF("xen,xenvm-%d.%d",
> +                                        vers->xen_version_major,
> +                                        vers->xen_version_minor),
> +                              "xen,xenvm");
> +    if (res) return res;
> +
> +    res = fdt_property_cell(fdt, "interrupt-parent", PHANDLE_GIC);
> +    if (res) return res;
> +
> +    res = fdt_property_cell(fdt, "#address-cells", ROOT_ADDRESS_CELLS);
> +    if (res) return res;
> +
> +    res = fdt_property_cell(fdt, "#size-cells", ROOT_SIZE_CELLS);
> +    if (res) return res;
> +
> +    return 0;
> +}
> +
> +static int make_chosen_node(libxl__gc *gc, void *fdt,
> +                            const libxl_domain_build_info *info)
> +{
> +    int res;
> +
> +    /* See linux Documentation/devicetree/... */
> +    res = fdt_begin_node(fdt, "chosen");
> +    if (res) return res;
> +
> +    res = fdt_property_string(fdt, "bootargs", info->u.pv.cmdline);
> +    if (res) return res;
> +
> +    res = fdt_end_node(fdt);
> +    if (res) return res;
> +
> +    return 0;
> +}
> +
> +static int make_cpus_node(libxl__gc *gc, void *fdt, int nr_cpus,
> +                          const struct arch_info *ainfo)
> +{
> +    int res, i;
> +
> +    res = fdt_begin_node(fdt, "cpus");
> +    if (res) return res;
> +
> +    res = fdt_property_cell(fdt, "#address-cells", 1);
> +    if (res) return res;
> +
> +    res = fdt_property_cell(fdt, "#size-cells", 0);
> +    if (res) return res;
> +
> +    for (i = 0; i < nr_cpus; i++) {
> +        const char *name = GCSPRINTF("cpu@%d", i);
> +
> +        res = fdt_begin_node(fdt, name);
> +        if (res) return res;
> +
> +        res = fdt_property_string(fdt, "device_type", "cpu");
> +        if (res) return res;
> +
> +        res = fdt_property_compat(gc, fdt, 1, ainfo->cpu_compat);
> +        if (res) return res;
> +
> +        res = fdt_property_string(fdt, "enable-method", "psci");
> +        if (res) return res;
> +
> +        res = fdt_property_regs(gc, fdt, 1, 0, 1, (uint64_t)i);
> +        if (res) return res;
> +
> +        res = fdt_end_node(fdt);
> +        if (res) return res;
> +    }
> +
> +    res = fdt_end_node(fdt);
> +    if (res) return res;
> +
> +    return 0;
> +}
> +
> +static int make_psci_node(libxl__gc *gc, void *fdt)
> +{
> +    int res;
> +
> +    res = fdt_begin_node(fdt, "psci");
> +    if (res) return res;
> +
> +    res = fdt_property_compat(gc, fdt, 1, "arm,psci");
> +    if (res) return res;
> +
> +    res = fdt_property_string(fdt, "method", "hvc");
> +    if (res) return res;
> +
> +    res = fdt_property_cell(fdt, "cpu_off", PSCI_cpu_off);
> +    if (res) return res;
> +
> +    res = fdt_property_cell(fdt, "cpu_on", PSCI_cpu_on);
> +    if (res) return res;
> +
> +    res = fdt_end_node(fdt);
> +    if (res) return res;
> +
> +    return 0;
> +}
> +
> +static int make_memory_node(libxl__gc *gc, void *fdt,
> +                            unsigned long long base,
> +                            unsigned long long size)
> +{
> +    int res;
> +    const char *name = GCSPRINTF("memory@%08llx", base);
> +
> +    res = fdt_begin_node(fdt, name);
> +    if (res) return res;
> +
> +    res = fdt_property_string(fdt, "device_type", "memory");
> +    if (res) return res;
> +
> +    res = fdt_property_regs(gc, fdt, ROOT_ADDRESS_CELLS, ROOT_SIZE_CELLS,
> +                            1, (uint64_t)base, (uint64_t)size);
> +    if (res) return res;
> +
> +    res = fdt_end_node(fdt);
> +    if (res) return res;
> +
> +    return 0;
> +}
> +
> +static int make_intc_node(libxl__gc *gc, void *fdt,
> +                          unsigned long long gicd_base,
> +                          unsigned long long gicd_size,
> +                          unsigned long long gicc_base,
> +                          unsigned long long gicc_size)
> +{
> +    int res;
> +    const char *name = GCSPRINTF("interrupt-controller@%08llx", gicd_base);
> +
> +    res = fdt_begin_node(fdt, name);
> +    if (res) return res;
> +
> +    res = fdt_property_compat(gc, fdt, 2,
> +                              "arm,cortex-a15-gic",
> +                              "arm,cortex-a9-gic");
> +    if (res) return res;
> +
> +
> +    res = fdt_property_cell(fdt, "#interrupt-cells", 3);
> +    if (res) return res;
> +
> +    res = fdt_property_cell(fdt, "#address-cells", 0);
> +    if (res) return res;
> +
> +    res = fdt_property(fdt, "interrupt-controller", NULL, 0);
> +    if (res) return res;
> +
> +    res = fdt_property_regs(gc, fdt, ROOT_ADDRESS_CELLS, ROOT_SIZE_CELLS,
> +                            2,
> +                            (uint64_t)gicd_base, (uint64_t)gicd_size,
> +                            (uint64_t)gicc_base, (uint64_t)gicc_size);
> +    if (res) return res;
> +
> +    res = fdt_property_cell(fdt, "linux,phandle", PHANDLE_GIC);
> +    if (res) return res;
> +
> +    res = fdt_property_cell(fdt, "phandle", PHANDLE_GIC);
> +    if (res) return res;
> +
> +    res = fdt_end_node(fdt);
> +    if (res) return res;
> +
> +    return 0;
> +}
> +
> +static int make_timer_node(libxl__gc *gc, void *fdt, const struct arch_info *ainfo)
> +{
> +    int res;
> +    gic_interrupt ints[3];
> +
> +    res = fdt_begin_node(fdt, "timer");
> +    if (res) return res;
> +
> +    res = fdt_property_compat(gc, fdt, 1, ainfo->timer_compat);
> +    if (res) return res;
> +
> +    set_interrupt_ppi(ints[0], GUEST_TIMER_PHYS_S_PPI, 0xf, 0x8);
> +    set_interrupt_ppi(ints[1], GUEST_TIMER_PHYS_NS_PPI, 0xf, 0x8);
> +    set_interrupt_ppi(ints[2], GUEST_TIMER_VIRT_PPI, 0xf, 0x8);
> +
> +    res = fdt_property_interrupts(gc, fdt, ints, 3);
> +    if (res) return res;
> +
> +    res = fdt_end_node(fdt);
> +    if (res) return res;
> +
> +    return 0;
> +}
> +
> +static int make_hypervisor_node(libxl__gc *gc, void *fdt,
> +                                const libxl_version_info *vers)
> +{
> +    int res;
> +    gic_interrupt intr;
> +
> +    /* See linux Documentation/devicetree/bindings/arm/xen.txt */
> +    res = fdt_begin_node(fdt, "hypervisor");
> +    if (res) return res;
> +
> +    res = fdt_property_compat(gc, fdt, 2,
> +                              GCSPRINTF("xen,xen-%d.%d",
> +                                        vers->xen_version_major,
> +                                        vers->xen_version_minor),
> +                              "xen,xen");
> +    if (res) return res;
> +
> +    /* reg 0 is grant table space */
> +    res = fdt_property_regs(gc, fdt, ROOT_ADDRESS_CELLS, ROOT_SIZE_CELLS,
> +                            1,GUEST_GNTTAB_BASE, GUEST_GNTTAB_SIZE);
> +    if (res) return res;
> +
> +    /*
> +     * interrupts is evtchn upcall:
> +     *  - Active-low level-sensitive
> +     *  - All cpus
> +     */
> +    set_interrupt_ppi(intr, GUEST_EVTCHN_PPI, 0xf, 0x8);
> +
> +    res = fdt_property_interrupts(gc, fdt, &intr, 1);
> +    if (res) return res;
> +
> +    res = fdt_end_node(fdt);
> +    if (res) return res;
> +
> +    return 0;
> +}
> +
> +static const struct arch_info *get_arch_info(libxl__gc *gc,
> +                                             const struct xc_dom_image *dom)
> +{
> +    int i;
> +
> +    for (i=0; i < ARRAY_SIZE(arch_info); i++) {
> +        const struct arch_info *info = &arch_info[i];
> +        if (!strcmp(dom->guest_type, info->guest_type))
> +            return info;
> +    }
> +    LOG(ERROR, "Unable to find arch FDT info for %s\n", dom->guest_type);
> +    return NULL;
> +}
> +
> +static void debug_dump_fdt(libxl__gc *gc, void *fdt)
> +{
> +    int fd = -1, rc, r;
> +
> +    const char *dtb = getenv("LIBXL_DEBUG_DUMP_DTB");
> +
> +    if (!dtb) goto out;
> +
> +    fd = open(dtb, O_CREAT|O_TRUNC|O_WRONLY, 0666);
> +    if (fd < 0) {
> +        LOGE(DEBUG, "cannot open %s for LIBXL_DEBUG_DUMP_DTB", dtb);
> +        goto out;
> +    }
> +
> +    rc = libxl_write_exactly(CTX, fd, fdt, fdt_totalsize(fdt), dtb, "dtb");
> +    if (rc < 0) goto out;
> +
> +out:
> +    if (fd >= 0) {
> +        r = close(fd);
> +        if (r < 0) LOGE(DEBUG, "failed to close DTB debug dump output");
> +    }
> +}
> +
> +#define FDT_MAX_SIZE (1<<20)
> +
> +int libxl__arch_domain_configure(libxl__gc *gc,
> +                                 libxl_domain_build_info *info,
> +                                 struct xc_dom_image *dom)
> +{
> +    void *fdt = NULL;
> +    int rc, res;
> +    size_t fdt_size = 0;
> +
> +    const libxl_version_info *vers;
> +    const struct arch_info *ainfo;
> +
> +    assert(info->type == LIBXL_DOMAIN_TYPE_PV);
> +
> +    vers = libxl_get_version_info(CTX);
> +    if (vers == NULL) return ERROR_FAIL;
> +
> +    ainfo = get_arch_info(gc, dom);
> +    if (ainfo == NULL) return ERROR_FAIL;
> +
> +    LOG(DEBUG, "constructing DTB for Xen version %d.%d guest",
> +        vers->xen_version_major, vers->xen_version_minor);
> +
> +/*
> + * Call "call" handling FDR_ERR_*. Will either:
> + * - loop back to retry_resize
> + * - set rc and goto out
> + * - fall through successfully
> + *
> + * On FDT_ERR_NOSPACE we start again from scratch rather than
> + * realloc+libfdt_open_into because "call" may have failed half way
> + * through a series of steps leaving the partial tree in an
> + * inconsistent state, e.g. leaving a node open.
> + */
> +#define FDT( call ) do {                                        \
> +    int fdt_res = (call);                                       \
> +    if (fdt_res == -FDT_ERR_NOSPACE && fdt_size < FDT_MAX_SIZE) \
> +        goto next_resize;                                       \
> +    else if (fdt_res < 0) {                                     \
> +        LOG(ERROR, "FDT: %s failed: %d = %s",                   \
> +            #call, fdt_res, fdt_strerror(fdt_res));             \
> +        rc = ERROR_FAIL;                                        \
> +        goto out;                                               \
> +    }                                                           \
> +} while(0)
> +
> +    for (;;) {
> +next_resize:
> +        if (fdt_size) {
> +            fdt_size <<= 1;
> +            LOG(DEBUG, "Increasing FDT size to %zd and retrying", fdt_size);
> +        } else {
> +            fdt_size = 4096;
> +        }
> +
> +        fdt = libxl__realloc(gc, fdt, fdt_size);
> +
> +        FDT( fdt_create(fdt, fdt_size) );
> +
> +        FDT( fdt_finish_reservemap(fdt) );
> +
> +        FDT( fdt_begin_node(fdt, "") );
> +
> +        FDT( make_root_properties(gc, vers, fdt) );
> +        FDT( make_chosen_node(gc, fdt, info) );
> +        FDT( make_cpus_node(gc, fdt, info->max_vcpus, ainfo) );
> +        FDT( make_psci_node(gc, fdt) );
> +
> +        FDT( make_memory_node(gc, fdt,
> +                              dom->rambase_pfn << XC_PAGE_SHIFT,
> +                              info->target_memkb * 1024) );
> +        FDT( make_intc_node(gc, fdt,
> +                            GUEST_GICD_BASE, GUEST_GICD_SIZE,
> +                            GUEST_GICC_BASE, GUEST_GICD_SIZE) );
> +
> +        FDT( make_timer_node(gc, fdt, ainfo) );
> +        FDT( make_hypervisor_node(gc, fdt, vers) );
> +
> +        FDT( fdt_end_node(fdt) );
> +
> +        FDT( fdt_finish(fdt) );
> +        break;
> +    }
> +#undef FDT
> +
> +    LOG(DEBUG, "fdt total size %d", fdt_totalsize(fdt));
> +
> +    res = xc_dom_devicetree_mem(dom, fdt, fdt_totalsize(fdt));
> +    if (res) {
> +        LOGE(ERROR, "xc_dom_devicetree_file failed");
> +        rc = ERROR_FAIL;
> +        goto out;
> +    }
> +
> +    debug_dump_fdt(gc, fdt);
> +
> +    rc = 0;
> +
> +out:
> +    return rc;
> +}
> diff --git a/tools/libxl/libxl_dom.c b/tools/libxl/libxl_dom.c
> index 55902af..8ec96e3 100644
> --- a/tools/libxl/libxl_dom.c
> +++ b/tools/libxl/libxl_dom.c
> @@ -400,6 +400,10 @@ int libxl__build_pv(libxl__gc *gc, uint32_t domid,
>          LOGE(ERROR, "xc_dom_parse_image failed");
>          goto out;
>      }
> +    if ( (ret = libxl__arch_domain_configure(gc, info, dom)) != 0 ) {
> +        LOGE(ERROR, "libxl__arch_domain_configure failed");
> +        goto out;
> +    }
>      if ( (ret = xc_dom_mem_init(dom, info->target_memkb / 1024)) != 0 ) {
>          LOGE(ERROR, "xc_dom_mem_init failed");
>          goto out;
> diff --git a/tools/libxl/libxl_noarch.c b/tools/libxl/libxl_noarch.c
> deleted file mode 100644
> index 7893535..0000000
> --- a/tools/libxl/libxl_noarch.c
> +++ /dev/null
> @@ -1,8 +0,0 @@
> -#include "libxl_internal.h"
> -#include "libxl_arch.h"
> -
> -int libxl__arch_domain_create(libxl__gc *gc, libxl_domain_config *d_config,
> -                              uint32_t domid)
> -{
> -    return 0;
> -}
> diff --git a/tools/libxl/libxl_x86.c b/tools/libxl/libxl_x86.c
> index a78c91d..dd13c45 100644
> --- a/tools/libxl/libxl_x86.c
> +++ b/tools/libxl/libxl_x86.c
> @@ -308,3 +308,10 @@ int libxl__arch_domain_create(libxl__gc *gc, libxl_domain_config *d_config,
>  
>      return ret;
>  }
> +
> +int libxl__arch_domain_configure(libxl__gc *gc,
> +                                 libxl_domain_build_info *info,
> +                                 struct xc_dom_image *dom)
> +{
> +    return 0;
> +}
> -- 
> 1.7.10.4
> 

Re: [PATCH v5 19/19] libxl: build a device tree for ARM guests

[Julien Grall]

On 11/13/2013 06:11 PM, Ian Campbell wrote:
> Uses xc_dom_devicetree_mem which was just added. The call to this needs to be
> carefully sequenced to be after xc_dom_parse_image (so we can tell which kind
> of guest we are building, although we don't use this yet) and before
> xc_dom_mem_init which tries to decide where to place the FDT in guest RAM.
>
> Removes libxl_noarch which would only have been used by IA64 after this
> change. Remove IA64 as part of this patch.
>
> There is no attempt to expose this as a configuration setting for the user.
>
> Includes a debug hook to dump the dtb to a file for inspection.
>
> TODO:
> - v7 CPU compat is hardcoded to cortex-a15 -- may need to define something more
>    generic via mach-virt dt bindngs?
>
> Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
> Cc: Ian Jackson <ian.jackson@eu.citrix.com>
> ---
> v5: Correct error handling in debug_dump_fdt
> v4: Drop spurious comment in header
>      s/__be32/be32/ and s/gic_interrupt_t/gic_interrupt/ to avoid reserved names
>      Coding style fixes
>      Use GCSPRINTF
>      use for(;;) around FDT creation loop, undef FDT when done
>      use libxl__realloc for fdt size increase
>      Refactor debug dump into its own function, remove NDEBUG ifdef
> v2: base addresses, irq, evtchn etc stuff is now from public API headers,
>      avoiding the need to introduce domctls etc until we want to make them
>      dynamic.
>      fix memory node
>      Improve libfdt error handling, especially for FDT_ERR_NOSPACE.
>      Derive guest CPU and timer compatiblity nodes from the guest type.
>
> wip
> ---
>   tools/libxl/Makefile       |    6 +-
>   tools/libxl/libxl_arch.h   |    3 +
>   tools/libxl/libxl_arm.c    |  512 ++++++++++++++++++++++++++++++++++++++++++++
>   tools/libxl/libxl_dom.c    |    4 +
>   tools/libxl/libxl_noarch.c |    8 -
>   tools/libxl/libxl_x86.c    |    7 +
>   6 files changed, 530 insertions(+), 10 deletions(-)
>   create mode 100644 tools/libxl/libxl_arm.c
>   delete mode 100644 tools/libxl/libxl_noarch.c
>

[..]

> +    for (;;) {
> +next_resize:
> +        if (fdt_size) {
> +            fdt_size <<= 1;
> +            LOG(DEBUG, "Increasing FDT size to %zd and retrying", fdt_size);
> +        } else {
> +            fdt_size = 4096;
> +        }
> +
> +        fdt = libxl__realloc(gc, fdt, fdt_size);
> +
> +        FDT( fdt_create(fdt, fdt_size) );
> +
> +        FDT( fdt_finish_reservemap(fdt) );
> +
> +        FDT( fdt_begin_node(fdt, "") );
> +
> +        FDT( make_root_properties(gc, vers, fdt) );
> +        FDT( make_chosen_node(gc, fdt, info) );
> +        FDT( make_cpus_node(gc, fdt, info->max_vcpus, ainfo) );
> +        FDT( make_psci_node(gc, fdt) );
> +
> +        FDT( make_memory_node(gc, fdt,
> +                              dom->rambase_pfn << XC_PAGE_SHIFT,
> +                              info->target_memkb * 1024) );
> +        FDT( make_intc_node(gc, fdt,
> +                            GUEST_GICD_BASE, GUEST_GICD_SIZE,
> +                            GUEST_GICC_BASE, GUEST_GICD_SIZE) );
> +
> +        FDT( make_timer_node(gc, fdt, ainfo) );
> +        FDT( make_hypervisor_node(gc, fdt, vers) );
> +
> +        FDT( fdt_end_node(fdt) );
> +
> +        FDT( fdt_finish(fdt) );
> +        break;
> +    }
> +#undef FDT

Why didn't you try to use fdt_open_into? It should fit our usage: ie, 
grow up the size of the device tree. Actually, DTC uses this solution 
when the fdt needs to be resized.

-- 
Julien Grall

Re: [PATCH v5 19/19] libxl: build a device tree for ARM guests

[Ian Campbell]

On Thu, 2013-11-14 at 01:04 +0000, Julien Grall wrote:
> Why didn't you try to use fdt_open_into? It should fit our usage: ie, 
> grow up the size of the device tree. Actually, DTC uses this solution 
> when the fdt needs to be resized.

It seemed to be far more complex -- you would need to wrap every
potentially failing fdt operation in a while loop which does the realloc
and open into on ENOSPACE. Now that could be done by a macro but it
would need to have much wider scope than here since it would be needed
all the way down the call chain.

This way the functions building each node don't need to worry about all
that and only the toplevel needs to care. I'd be in favour of increasing
the default allocation size before I wanted to switch the mechanism like
this.

Ian.

Re: [PATCH v5 19/19] libxl: build a device tree for ARM guests

[Julien Grall]

On 11/14/2013 08:28 AM, Ian Campbell wrote:
> On Thu, 2013-11-14 at 01:04 +0000, Julien Grall wrote:
>> Why didn't you try to use fdt_open_into? It should fit our usage: ie,
>> grow up the size of the device tree. Actually, DTC uses this solution
>> when the fdt needs to be resized.
>
> It seemed to be far more complex -- you would need to wrap every
> potentially failing fdt operation in a while loop which does the realloc
> and open into on ENOSPACE. Now that could be done by a macro but it
> would need to have much wider scope than here since it would be needed
> all the way down the call chain.
>
> This way the functions building each node don't need to worry about all
> that and only the toplevel needs to care. I'd be in favour of increasing
> the default allocation size before I wanted to switch the mechanism like
> this.

Ok. Then:

Acked-by: Julien Grall <julien.grall@linaro.org>

-- 
Julien Grall

Re: [PATCH v5 19/19] libxl: build a device tree for ARM guests

[Ian Jackson]

Ian Campbell writes ("[PATCH v5 19/19] libxl: build a device tree for ARM guests"):
> Uses xc_dom_devicetree_mem which was just added. The call to this needs to be
> carefully sequenced to be after xc_dom_parse_image (so we can tell which kind
> of guest we are building, although we don't use this yet) and before
> xc_dom_mem_init which tries to decide where to place the FDT in guest RAM.
> 
> Removes libxl_noarch which would only have been used by IA64 after this
> change. Remove IA64 as part of this patch.
> 
> There is no attempt to expose this as a configuration setting for the user.
> 
> Includes a debug hook to dump the dtb to a file for inspection.

Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>

> TODO:
> - v7 CPU compat is hardcoded to cortex-a15 -- may need to define something more
>   generic via mach-virt dt bindngs?

I don't have an opinion on this.  I hope someone else does :-).

Ian.

Re: [PATCH v5 19/19] libxl: build a device tree for ARM guests

[Stefano Stabellini]

On Thu, 14 Nov 2013, Ian Jackson wrote:
> Ian Campbell writes ("[PATCH v5 19/19] libxl: build a device tree for ARM guests"):
> > Uses xc_dom_devicetree_mem which was just added. The call to this needs to be
> > carefully sequenced to be after xc_dom_parse_image (so we can tell which kind
> > of guest we are building, although we don't use this yet) and before
> > xc_dom_mem_init which tries to decide where to place the FDT in guest RAM.
> > 
> > Removes libxl_noarch which would only have been used by IA64 after this
> > change. Remove IA64 as part of this patch.
> > 
> > There is no attempt to expose this as a configuration setting for the user.
> > 
> > Includes a debug hook to dump the dtb to a file for inspection.
> 
> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
> 
> > TODO:
> > - v7 CPU compat is hardcoded to cortex-a15 -- may need to define something more
> >   generic via mach-virt dt bindngs?
> 
> I don't have an opinion on this.  I hope someone else does :-).

Wouldn't it be better to use the same cpu compatibility string of the
platform? After all it's the physical cpu that we are time slicing for
the guest: if any quirks are present, it is likely that they are going
to affect the guest too.

Re: [PATCH v5 19/19] libxl: build a device tree for ARM guests

[Julien Grall]

On 11/14/2013 12:17 PM, Stefano Stabellini wrote:
> On Thu, 14 Nov 2013, Ian Jackson wrote:
>> Ian Campbell writes ("[PATCH v5 19/19] libxl: build a device tree for ARM guests"):
>>> Uses xc_dom_devicetree_mem which was just added. The call to this needs to be
>>> carefully sequenced to be after xc_dom_parse_image (so we can tell which kind
>>> of guest we are building, although we don't use this yet) and before
>>> xc_dom_mem_init which tries to decide where to place the FDT in guest RAM.
>>>
>>> Removes libxl_noarch which would only have been used by IA64 after this
>>> change. Remove IA64 as part of this patch.
>>>
>>> There is no attempt to expose this as a configuration setting for the user.
>>>
>>> Includes a debug hook to dump the dtb to a file for inspection.
>>
>> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
>>
>>> TODO:
>>> - v7 CPU compat is hardcoded to cortex-a15 -- may need to define something more
>>>    generic via mach-virt dt bindngs?
>>
>> I don't have an opinion on this.  I hope someone else does :-).
>
> Wouldn't it be better to use the same cpu compatibility string of the
> platform? After all it's the physical cpu that we are time slicing for
> the guest: if any quirks are present, it is likely that they are going
> to affect the guest too.
>

/proc doesn't always expose the device tree. It depends on 
CONFIG_PROC_DEVICETREE:
	- if it's enabled, you will still need to browse all the directory to 
find where are the cpus node. But you will need to assume all CPUs are 
homogeneous.
         - if it's not enabled, you will need to create a fake one based 
on, for instance, /proc/cpuinfo.

In any case, Linux doesn't seem to care about the cpu compatible string 
for now.

-- 
Julien Grall

Re: [PATCH v5 19/19] libxl: build a device tree for ARM guests

[Ian Campbell]

On Thu, 2013-11-14 at 12:17 +0000, Stefano Stabellini wrote:
> On Thu, 14 Nov 2013, Ian Jackson wrote:
> > Ian Campbell writes ("[PATCH v5 19/19] libxl: build a device tree for ARM guests"):
> > > Uses xc_dom_devicetree_mem which was just added. The call to this needs to be
> > > carefully sequenced to be after xc_dom_parse_image (so we can tell which kind
> > > of guest we are building, although we don't use this yet) and before
> > > xc_dom_mem_init which tries to decide where to place the FDT in guest RAM.
> > > 
> > > Removes libxl_noarch which would only have been used by IA64 after this
> > > change. Remove IA64 as part of this patch.
> > > 
> > > There is no attempt to expose this as a configuration setting for the user.
> > > 
> > > Includes a debug hook to dump the dtb to a file for inspection.
> > 
> > Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
> > 
> > > TODO:
> > > - v7 CPU compat is hardcoded to cortex-a15 -- may need to define something more
> > >   generic via mach-virt dt bindngs?
> > 
> > I don't have an opinion on this.  I hope someone else does :-).
> 
> Wouldn't it be better to use the same cpu compatibility string of the
> platform? After all it's the physical cpu that we are time slicing for
> the guest: if any quirks are present, it is likely that they are going
> to affect the guest too.

That is one option, but would require us to get at that string somehow
from userspace (perhaps a new domctl, or add it to physinfo etc).

I think we should punt on it for now and coordinate with other mach-virt
users (e.g. KVM) so we all follow the same approach, whatever that might
be.

Ian.

Re: [PATCH v5 19/19] libxl: build a device tree for ARM guests

[Stefano Stabellini]

On Thu, 14 Nov 2013, Ian Campbell wrote:
> On Thu, 2013-11-14 at 12:17 +0000, Stefano Stabellini wrote:
> > On Thu, 14 Nov 2013, Ian Jackson wrote:
> > > Ian Campbell writes ("[PATCH v5 19/19] libxl: build a device tree for ARM guests"):
> > > > Uses xc_dom_devicetree_mem which was just added. The call to this needs to be
> > > > carefully sequenced to be after xc_dom_parse_image (so we can tell which kind
> > > > of guest we are building, although we don't use this yet) and before
> > > > xc_dom_mem_init which tries to decide where to place the FDT in guest RAM.
> > > > 
> > > > Removes libxl_noarch which would only have been used by IA64 after this
> > > > change. Remove IA64 as part of this patch.
> > > > 
> > > > There is no attempt to expose this as a configuration setting for the user.
> > > > 
> > > > Includes a debug hook to dump the dtb to a file for inspection.
> > > 
> > > Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
> > > 
> > > > TODO:
> > > > - v7 CPU compat is hardcoded to cortex-a15 -- may need to define something more
> > > >   generic via mach-virt dt bindngs?
> > > 
> > > I don't have an opinion on this.  I hope someone else does :-).
> > 
> > Wouldn't it be better to use the same cpu compatibility string of the
> > platform? After all it's the physical cpu that we are time slicing for
> > the guest: if any quirks are present, it is likely that they are going
> > to affect the guest too.
> 
> That is one option, but would require us to get at that string somehow
> from userspace (perhaps a new domctl, or add it to physinfo etc).
> 
> I think we should punt on it for now and coordinate with other mach-virt
> users (e.g. KVM) so we all follow the same approach, whatever that might
> be.

I am fine with delaying it. "cortex-a15" is OK for now.

That said, given that cpu virtualization is done very differently on Xen
and KVM, I think that it would be reasonable to make our own decision
on this.

Re: [PATCH v5 19/19] libxl: build a device tree for ARM guests

[Ian Campbell]

On Thu, 2013-11-14 at 14:01 +0000, Stefano Stabellini wrote:
> On Thu, 14 Nov 2013, Ian Campbell wrote:
> > On Thu, 2013-11-14 at 12:17 +0000, Stefano Stabellini wrote:
> > > On Thu, 14 Nov 2013, Ian Jackson wrote:
> > > > Ian Campbell writes ("[PATCH v5 19/19] libxl: build a device tree for ARM guests"):
> > > > > Uses xc_dom_devicetree_mem which was just added. The call to this needs to be
> > > > > carefully sequenced to be after xc_dom_parse_image (so we can tell which kind
> > > > > of guest we are building, although we don't use this yet) and before
> > > > > xc_dom_mem_init which tries to decide where to place the FDT in guest RAM.
> > > > > 
> > > > > Removes libxl_noarch which would only have been used by IA64 after this
> > > > > change. Remove IA64 as part of this patch.
> > > > > 
> > > > > There is no attempt to expose this as a configuration setting for the user.
> > > > > 
> > > > > Includes a debug hook to dump the dtb to a file for inspection.
> > > > 
> > > > Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
> > > > 
> > > > > TODO:
> > > > > - v7 CPU compat is hardcoded to cortex-a15 -- may need to define something more
> > > > >   generic via mach-virt dt bindngs?
> > > > 
> > > > I don't have an opinion on this.  I hope someone else does :-).
> > > 
> > > Wouldn't it be better to use the same cpu compatibility string of the
> > > platform? After all it's the physical cpu that we are time slicing for
> > > the guest: if any quirks are present, it is likely that they are going
> > > to affect the guest too.
> > 
> > That is one option, but would require us to get at that string somehow
> > from userspace (perhaps a new domctl, or add it to physinfo etc).
> > 
> > I think we should punt on it for now and coordinate with other mach-virt
> > users (e.g. KVM) so we all follow the same approach, whatever that might
> > be.
> 
> I am fine with delaying it. "cortex-a15" is OK for now.
> 
> That said, given that cpu virtualization is done very differently on Xen
> and KVM, I think that it would be reasonable to make our own decision
> on this.

This field, when it is used at all, is used to figure out quirks and
enable/disable architectural expectations (e.g. the sorts of flushes
which are available, set ACTLR.SMP etc).

I think at this level Xen and KVM are pretty similar.

Ian.