Xen on Seattle

Xen on Seattle

[Suravee Suthikulanit]

Hi all,

I am running into an issue booting Xen on Seattle platform where dom0 
kernel failed to make the following hypercall in 
arch/arm/xen/enlighten.c: xen_percpu_init().

  HYPERVISOR_vcpu_op(VCPUOP_register_vcpu_info, cpu, &info)

After investigation, the hypercall failed in the xen/common/domain.c: 
map_vcpu_info(), where it check

     if ( offset > (PAGE_SIZE - sizeof(vcpu_info_t)) )
         return -EINVAL;

Here:

(XEN) DEBUG0: map_vcpu_info: offset = 0x6790
(XEN) DEBUG0: map_vcpu_info: PAGE_SIZE = 0x1000
(XEN) DEBUG0: map_vcpu_info: sizeof(vcpu_info_t) = 0x30

If I understand correctly, it seems that the hypervisor is making 
assumption that the Dom0 guest is using 4K page size.  The PAGE_SHIFT in 
include/asm-arm/config.h is currently set to 12.  So, I have tried 
changing this to 16 (for 64K page). However, there is a restriction on 
the 16-bit value of the struct active_grant_entry { length:16; }. So, I 
cannot boot dom0 with 64K page size. Is this accurate?

Thanks,

Suravee

Re: Xen on Seattle

[Stefano Stabellini]

On Tue, 2 Sep 2014, Suravee Suthikulanit wrote:
> Hi all,
> 
> I am running into an issue booting Xen on Seattle platform where dom0 kernel
> failed to make the following hypercall in arch/arm/xen/enlighten.c:
> xen_percpu_init().
> 
>  HYPERVISOR_vcpu_op(VCPUOP_register_vcpu_info, cpu, &info)
> 
> After investigation, the hypercall failed in the xen/common/domain.c:
> map_vcpu_info(), where it check
> 
>     if ( offset > (PAGE_SIZE - sizeof(vcpu_info_t)) )
>         return -EINVAL;
> 
> Here:
> 
> (XEN) DEBUG0: map_vcpu_info: offset = 0x6790
> (XEN) DEBUG0: map_vcpu_info: PAGE_SIZE = 0x1000
> (XEN) DEBUG0: map_vcpu_info: sizeof(vcpu_info_t) = 0x30
> 
> If I understand correctly, it seems that the hypervisor is making assumption
> that the Dom0 guest is using 4K page size.  The PAGE_SHIFT in
> include/asm-arm/config.h is currently set to 12.  So, I have tried changing
> this to 16 (for 64K page). However, there is a restriction on the 16-bit value
> of the struct active_grant_entry { length:16; }. So, I cannot boot dom0 with
> 64K page size. Is this accurate?

Hello Suravee,
yes, that is correct: all pages used in hypercalls and shared with the
hypervisor need to be 4K pages.

Re: Xen on Seattle

[Suravee Suthikulanit]

On 9/2/2014 1:06 PM, Stefano Stabellini wrote:
> On Tue, 2 Sep 2014, Suravee Suthikulanit wrote:
>> Hi all,
>>
>> I am running into an issue booting Xen on Seattle platform where dom0 kernel
>> failed to make the following hypercall in arch/arm/xen/enlighten.c:
>> xen_percpu_init().
>>
>>   HYPERVISOR_vcpu_op(VCPUOP_register_vcpu_info, cpu, &info)
>>
>> After investigation, the hypercall failed in the xen/common/domain.c:
>> map_vcpu_info(), where it check
>>
>>      if ( offset > (PAGE_SIZE - sizeof(vcpu_info_t)) )
>>          return -EINVAL;
>>
>> Here:
>>
>> (XEN) DEBUG0: map_vcpu_info: offset = 0x6790
>> (XEN) DEBUG0: map_vcpu_info: PAGE_SIZE = 0x1000
>> (XEN) DEBUG0: map_vcpu_info: sizeof(vcpu_info_t) = 0x30
>>
>> If I understand correctly, it seems that the hypervisor is making assumption
>> that the Dom0 guest is using 4K page size.  The PAGE_SHIFT in
>> include/asm-arm/config.h is currently set to 12.  So, I have tried changing
>> this to 16 (for 64K page). However, there is a restriction on the 16-bit value
>> of the struct active_grant_entry { length:16; }. So, I cannot boot dom0 with
>> 64K page size. Is this accurate?
>
> Hello Suravee,
> yes, that is correct: all pages used in hypercalls and shared with the
> hypervisor need to be 4K pages.
>

Ok, the next question would be, would it make sense/possible to extend 
the support to allow 64K pages?

Suravee

Re: Xen on Seattle

[Stefano Stabellini]

On Tue, 2 Sep 2014, Suravee Suthikulanit wrote:
> On 9/2/2014 1:06 PM, Stefano Stabellini wrote:
> > On Tue, 2 Sep 2014, Suravee Suthikulanit wrote:
> > > Hi all,
> > > 
> > > I am running into an issue booting Xen on Seattle platform where dom0
> > > kernel
> > > failed to make the following hypercall in arch/arm/xen/enlighten.c:
> > > xen_percpu_init().
> > > 
> > >   HYPERVISOR_vcpu_op(VCPUOP_register_vcpu_info, cpu, &info)
> > > 
> > > After investigation, the hypercall failed in the xen/common/domain.c:
> > > map_vcpu_info(), where it check
> > > 
> > >      if ( offset > (PAGE_SIZE - sizeof(vcpu_info_t)) )
> > >          return -EINVAL;
> > > 
> > > Here:
> > > 
> > > (XEN) DEBUG0: map_vcpu_info: offset = 0x6790
> > > (XEN) DEBUG0: map_vcpu_info: PAGE_SIZE = 0x1000
> > > (XEN) DEBUG0: map_vcpu_info: sizeof(vcpu_info_t) = 0x30
> > > 
> > > If I understand correctly, it seems that the hypervisor is making
> > > assumption
> > > that the Dom0 guest is using 4K page size.  The PAGE_SHIFT in
> > > include/asm-arm/config.h is currently set to 12.  So, I have tried
> > > changing
> > > this to 16 (for 64K page). However, there is a restriction on the 16-bit
> > > value
> > > of the struct active_grant_entry { length:16; }. So, I cannot boot dom0
> > > with
> > > 64K page size. Is this accurate?
> > 
> > Hello Suravee,
> > yes, that is correct: all pages used in hypercalls and shared with the
> > hypervisor need to be 4K pages.
> > 
> 
> Ok, the next question would be, would it make sense/possible to extend the
> support to allow 64K pages?

It might make sense, but it is not trivial: all Xen hypercall interfaces
are built for 4K pages and would need a significant revamp.  Certainly
too late for the next release. It might be something to look at for Xen
4.6.

There is nothing preventing a guest kernel from using 64K pages anywhere
else though.

Re: Xen on Seattle

[Suravee Suthikulanit]

On 9/2/2014 1:15 PM, Stefano Stabellini wrote:
> On Tue, 2 Sep 2014, Suravee Suthikulanit wrote:
>> On 9/2/2014 1:06 PM, Stefano Stabellini wrote:
>>> On Tue, 2 Sep 2014, Suravee Suthikulanit wrote:
>>>> Hi all,
>>>>
>>>> I am running into an issue booting Xen on Seattle platform where dom0
>>>> kernel
>>>> failed to make the following hypercall in arch/arm/xen/enlighten.c:
>>>> xen_percpu_init().
>>>>
>>>>    HYPERVISOR_vcpu_op(VCPUOP_register_vcpu_info, cpu, &info)
>>>>
>>>> After investigation, the hypercall failed in the xen/common/domain.c:
>>>> map_vcpu_info(), where it check
>>>>
>>>>       if ( offset > (PAGE_SIZE - sizeof(vcpu_info_t)) )
>>>>           return -EINVAL;
>>>>
>>>> Here:
>>>>
>>>> (XEN) DEBUG0: map_vcpu_info: offset = 0x6790
>>>> (XEN) DEBUG0: map_vcpu_info: PAGE_SIZE = 0x1000
>>>> (XEN) DEBUG0: map_vcpu_info: sizeof(vcpu_info_t) = 0x30
>>>>
>>>> If I understand correctly, it seems that the hypervisor is making
>>>> assumption
>>>> that the Dom0 guest is using 4K page size.  The PAGE_SHIFT in
>>>> include/asm-arm/config.h is currently set to 12.  So, I have tried
>>>> changing
>>>> this to 16 (for 64K page). However, there is a restriction on the 16-bit
>>>> value
>>>> of the struct active_grant_entry { length:16; }. So, I cannot boot dom0
>>>> with
>>>> 64K page size. Is this accurate?
>>>
>>> Hello Suravee,
>>> yes, that is correct: all pages used in hypercalls and shared with the
>>> hypervisor need to be 4K pages.
>>>
>>
>> Ok, the next question would be, would it make sense/possible to extend the
>> support to allow 64K pages?
>
> It might make sense, but it is not trivial: all Xen hypercall interfaces
> are built for 4K pages and would need a significant revamp.  Certainly
> too late for the next release. It might be something to look at for Xen
> 4.6.
>
> There is nothing preventing a guest kernel from using 64K pages anywhere
> else though.
>
The alternative is to try booting with dom0 kernel configured with 4K 
pages. Currently, with 3.16 kernel, this is not working yet with Seattle 
since we would need level 4 page table support to boot with 4K page size.

There is a patch from Catalin which should add support for level 4 page. 
  Let us experiment with that and get back with you guys.

Suravee

Re: Xen on Seattle

[Stefano Stabellini]

On Tue, 2 Sep 2014, Suravee Suthikulanit wrote:
> On 9/2/2014 1:15 PM, Stefano Stabellini wrote:
> > On Tue, 2 Sep 2014, Suravee Suthikulanit wrote:
> > > On 9/2/2014 1:06 PM, Stefano Stabellini wrote:
> > > > On Tue, 2 Sep 2014, Suravee Suthikulanit wrote:
> > > > > Hi all,
> > > > > 
> > > > > I am running into an issue booting Xen on Seattle platform where dom0
> > > > > kernel
> > > > > failed to make the following hypercall in arch/arm/xen/enlighten.c:
> > > > > xen_percpu_init().
> > > > > 
> > > > >    HYPERVISOR_vcpu_op(VCPUOP_register_vcpu_info, cpu, &info)
> > > > > 
> > > > > After investigation, the hypercall failed in the xen/common/domain.c:
> > > > > map_vcpu_info(), where it check
> > > > > 
> > > > >       if ( offset > (PAGE_SIZE - sizeof(vcpu_info_t)) )
> > > > >           return -EINVAL;
> > > > > 
> > > > > Here:
> > > > > 
> > > > > (XEN) DEBUG0: map_vcpu_info: offset = 0x6790
> > > > > (XEN) DEBUG0: map_vcpu_info: PAGE_SIZE = 0x1000
> > > > > (XEN) DEBUG0: map_vcpu_info: sizeof(vcpu_info_t) = 0x30
> > > > > 
> > > > > If I understand correctly, it seems that the hypervisor is making
> > > > > assumption
> > > > > that the Dom0 guest is using 4K page size.  The PAGE_SHIFT in
> > > > > include/asm-arm/config.h is currently set to 12.  So, I have tried
> > > > > changing
> > > > > this to 16 (for 64K page). However, there is a restriction on the
> > > > > 16-bit
> > > > > value
> > > > > of the struct active_grant_entry { length:16; }. So, I cannot boot
> > > > > dom0
> > > > > with
> > > > > 64K page size. Is this accurate?
> > > > 
> > > > Hello Suravee,
> > > > yes, that is correct: all pages used in hypercalls and shared with the
> > > > hypervisor need to be 4K pages.
> > > > 
> > > 
> > > Ok, the next question would be, would it make sense/possible to extend the
> > > support to allow 64K pages?
> > 
> > It might make sense, but it is not trivial: all Xen hypercall interfaces
> > are built for 4K pages and would need a significant revamp.  Certainly
> > too late for the next release. It might be something to look at for Xen
> > 4.6.
> > 
> > There is nothing preventing a guest kernel from using 64K pages anywhere
> > else though.
> > 
> The alternative is to try booting with dom0 kernel configured with 4K pages.

Yeah, we typically use 4K pages and 2M superpages in guest kernels.


> Currently, with 3.16 kernel, this is not working yet with Seattle since we
> would need level 4 page table support to boot with 4K page size.
> 
> There is a patch from Catalin which should add support for level 4 page.  Let
> us experiment with that and get back with you guys.

That would be great, thanks!

Re: Xen on Seattle

[Ian Campbell]

On Tue, 2014-09-02 at 19:15 +0100, Stefano Stabellini wrote:
> There is nothing preventing a guest kernel from using 64K pages anywhere
> else though.

AFAIK you can't mix and match 4K and 64K pages in a single PT since 4K
vs 64K is controllled by the granule size setting which is global and
affects the entirety of the current PT.

So you would have to e.g. switch to 4K pages just to make hypercalls,
which would be quite a faff.

Ian.

Re: Xen on Seattle

[Suravee Suthikulanit]

[-- Attachment #1: Type: text/plain, Size: 2999 bytes --]

On 9/2/2014 1:31 PM, Suravee Suthikulanit wrote:
> On 9/2/2014 1:15 PM, Stefano Stabellini wrote:
>> On Tue, 2 Sep 2014, Suravee Suthikulanit wrote:
>>> On 9/2/2014 1:06 PM, Stefano Stabellini wrote:
>>>> On Tue, 2 Sep 2014, Suravee Suthikulanit wrote:
>>>>> Hi all,
>>>>>
>>>>> I am running into an issue booting Xen on Seattle platform where dom0
>>>>> kernel
>>>>> failed to make the following hypercall in arch/arm/xen/enlighten.c:
>>>>> xen_percpu_init().
>>>>>
>>>>>    HYPERVISOR_vcpu_op(VCPUOP_register_vcpu_info, cpu, &info)
>>>>>
>>>>> After investigation, the hypercall failed in the xen/common/domain.c:
>>>>> map_vcpu_info(), where it check
>>>>>
>>>>>       if ( offset > (PAGE_SIZE - sizeof(vcpu_info_t)) )
>>>>>           return -EINVAL;
>>>>>
>>>>> Here:
>>>>>
>>>>> (XEN) DEBUG0: map_vcpu_info: offset = 0x6790
>>>>> (XEN) DEBUG0: map_vcpu_info: PAGE_SIZE = 0x1000
>>>>> (XEN) DEBUG0: map_vcpu_info: sizeof(vcpu_info_t) = 0x30
>>>>>
>>>>> If I understand correctly, it seems that the hypervisor is making
>>>>> assumption
>>>>> that the Dom0 guest is using 4K page size.  The PAGE_SHIFT in
>>>>> include/asm-arm/config.h is currently set to 12.  So, I have tried
>>>>> changing
>>>>> this to 16 (for 64K page). However, there is a restriction on the
>>>>> 16-bit
>>>>> value
>>>>> of the struct active_grant_entry { length:16; }. So, I cannot boot
>>>>> dom0
>>>>> with
>>>>> 64K page size. Is this accurate?
>>>>
>>>> Hello Suravee,
>>>> yes, that is correct: all pages used in hypercalls and shared with the
>>>> hypervisor need to be 4K pages.
>>>>
>>>
>>> Ok, the next question would be, would it make sense/possible to
>>> extend the
>>> support to allow 64K pages?
>>
>> It might make sense, but it is not trivial: all Xen hypercall interfaces
>> are built for 4K pages and would need a significant revamp.  Certainly
>> too late for the next release. It might be something to look at for Xen
>> 4.6.
>>
>> There is nothing preventing a guest kernel from using 64K pages anywhere
>> else though.
>>
> The alternative is to try booting with dom0 kernel configured with 4K
> pages. Currently, with 3.16 kernel, this is not working yet with Seattle
> since we would need level 4 page table support to boot with 4K page size.
>
> There is a patch from Catalin which should add support for level 4 page.
>   Let us experiment with that and get back with you guys.
>

Quick update:

After configure kernel w/ 4K page and Using 48-bit (w/ 4 level page 
table), Dom0 kernel seems to be coming along further at this point up 
until the following point:

.....
(XEN) DOM0: NET: Registered protocol family 10
(XEN) DOM0: sit: IPv6 over IPv4 tunneling driver
(XEN) DOM0: NET: Registered protocol family 17
(XEN) DOM0: Bridge firewalling registered
(XEN) DOM0: 9pnet: Installing 9P2000 support
(XEN) DOM0: NET: Registered protocol family 37
(XEN) DOM0: bootconsole [uart0] disabled

Please see attachment for full boot log. I am still working on trying to 
get to Dom0 console.

Suravee


[-- Attachment #2: xen-seattle-4k.txt --]
[-- Type: text/plain, Size: 14954 bytes --]

Loading Xen image..
- UART enabled -
- CPU 00000000 booting -
- Current EL 00000008 -
- Xen starting at EL2 -
- Zero BSS -
- Setting up control registers -
- Turning on paging -
- Ready -
(XEN) Checking for initrd in /chosen
(XEN) RAM: 0000008000000000 - 00000083ffffffff
(XEN)
(XEN) MODULE[0]: 000000801fffe000 - 000000801ffffc79 Device Tree
(XEN) MODULE[1]: 0000008008800000 - 0000008009142a50 Kernel
(XEN)
(XEN) Command line: console=dtuart dtuart=serial0 noreboot sync_console
(XEN) Placing Xen at 0x00000083ffe00000-0x0000008400000000
(XEN) Update BOOTMOD_XEN from 0000008008700000-00000080087fad81 => 00000083ffe00000-00000083ffefad81
(XEN) Domain heap initialised
(XEN) Looking for UART console serial0
 Xen 4.5-unstable
(XEN) Xen version 4.5-unstable (ssuthiku@amd.com) (aarch64-linux-gnu-gcc (crosstool-NG linaro-1.13.1-4.8-2014.01 - Linaro GCC 2013.11) 4.8.3 20140106 (prerelease)) debug=y Wed Sep 10 17:12:52 CDT 2014
(XEN) Latest ChangeSet: Mon Aug 25 22:15:10 2014 -0500 git:8a26eb8           A0
(XEN) Console output is synchronous.
(XEN) Processor: 410fd070: "ARM Limited", variant: 0x0, part 0xd07, rev 0x0
(XEN) 64-bit Execution:
(XEN)   Processor Features: 0000000000002222 0000000000000000
(XEN)     Exception Levels: EL3:64+32 EL2:64+32 EL1:64+32 EL0:64+32
(XEN)     Extensions: FloatingPoint AdvancedSIMD
(XEN)   Debug Features: 0000000010305106 0000000000000000
(XEN)   Auxiliary Features: 0000000000000000 0000000000000000
(XEN)   Memory Model Features: 0000000000001124 0000000000000000
(XEN)   ISA Features:  0000000000011120 0000000000000000
(XEN) 32-bit Execution:
(XEN)   Processor Features: 00000131:00011011
(XEN)     Instruction Sets: AArch32 Thumb Thumb-2 Jazelle
(XEN)     Extensions: GenericTimer Security
(XEN)   Debug Features: 03010066
(XEN)   Auxiliary Features: 00000000
(XEN)   Memory Model Features: 10101105 40000000 01260000 02102211
(XEN)  ISA Features: 02101110 13112111 21232042 01112131 00011142 00011121
(XEN) Platform: SEATTLE
(XEN) Generic Timer IRQ: phys=30 hyp=26 virt=27
(XEN) Using generic timer at 187500 KHz
(XEN) GICv2 initialization:
(XEN)         gic_dist_addr=00000000e1110000
(XEN)         gic_cpu_addr=00000000e112f000
(XEN)         gic_hyp_addr=00000000e1140000
(XEN)         gic_vcpu_addr=00000000e1160000
(XEN)         gic_maintenance_irq=24
(XEN) GICv2: 448 lines, 8 cpus, secure (IID 0200143b).
(XEN) Using scheduler: SMP Credit Scheduler (credit)
(XEN) I/O virtualisation disabled
(XEN) Allocated console ring of 64 KiB.
(XEN) Bringing up CPU1
- CPU 00000201 booting -
- Current EL 00000008 -
- Xen starting at EL2 -
- Setting up control registers -
- Turning on paging -
- Ready -
(XEN) CPU 1 booted.
(XEN) Bringing up CPU2
- CPU 00000200 booting -
- Current EL 00000008 -
- Xen starting at EL2 -
- Setting up control registers -
- Turning on paging -
- Ready -
(XEN) CPU 2 booted.
(XEN) Bringing up CPU3
- CPU 00000101 booting -
- Current EL 00000008 -
- Xen starting at EL2 -
- Setting up control registers -
- Turning on paging -
- Ready -
(XEN) CPU 3 booted.
(XEN) Bringing up CPU4
- CPU 00000100 booting -
- Current EL 00000008 -
- Xen starting at EL2 -
- Setting up control registers -
- Turning on paging -
- Ready -
(XEN) CPU 4 booted.
(XEN) Bringing up CPU5
- CPU 00000001 booting -
- Current EL 00000008 -
- Xen starting at EL2 -
- Setting up control registers -
- Turning on paging -
- Ready -
(XEN) Brought up 6 CPUs
(XEN) CPU 5 booted.
(XEN) *** LOADING DOMAIN 0 ***
(XEN) Loading kernel from boot module @ 0000008008800000
(XEN) Allocating 1:1 mappings totalling 128MB for dom0:
(XEN) BANK[0] 0x00008010000000-0x00008018000000 (128MB)
(XEN) Loading zImage from 0000008008800000 to 0000008010080000-00000080109c2a50
(XEN) Loading dom0 DTB to 0x0000008017e00000-0x0000008017e019eb
(XEN) Scrubbing Free RAM on 1 nodes using 6 CPUs
(XEN) ......................done.
(XEN) Initial low memory virq threshold set at 0x4000 pages.
(XEN) Std. Loglevel: All
(XEN) Guest Loglevel: All
(XEN) **********************************************
(XEN) ******* WARNING: CONSOLE OUTPUT IS SYNCHRONOUS
(XEN) ******* This option is intended to aid debugging of Xen by ensuring
(XEN) ******* that all output is synchronously delivered on the serial line.
(XEN) ******* However it can introduce SIGNIFICANT latencies and affect
(XEN) ******* timekeeping. It is NOT recommended for production use!
(XEN) **********************************************
(XEN) 3... 2... 1...
(XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to switch input to Xen)
(XEN) Freed 252kB init memory.
(XEN) DOM0: Linux version 3.17.0-rc4-styx-xen+ (ssuthiku@sos-arm-dev) (gcc version 4.8.2 20140110 (prerelease) [ibm/gcc-4_8-branch merged
(XEN) DOM0: from gcc-4_8-branch, revision 205847] (Ubuntu/Linaro 4.8.2-13ubuntu1) ) #15 SMP Wed Sep 10 16:54:52 CDT 2014
(XEN) DOM0: CPU: AArch64 Processor [410fd070] revision 0
(XEN) DOM0: Detected PIPT I-cache on CPU0
(XEN) DOM0: Early serial console at I/O port 0x0 (options '')
(XEN) DOM0: bootconsole [uart0] enabled
(XEN) DOM0: efi: Getting EFI parameters from FDT:
(XEN) DOM0: efi: UEFI not found.
(XEN) DOM0: psci: probing for conduit method from DT.
(XEN) DOM0: psci: PSCIv0.2 detected in firmware.
(XEN) DOM0: psci: Using standard PSCI v0.2 function IDs
(XEN) DOM0: PERCPU: Embedded 12 pages/cpu @ffff800007fa7000 s18048 r8192 d22912 u49152
(XEN) DOM0: Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 32320
(XEN) DOM0: Kernel command line: console=ttyAMA0,115200 earlycon=pl011,0xe1010000 show_styx_info root=/dev/sda2 rootwait maxcpus=1
(XEN) DOM0: PID hash table entries: 512 (order: 0, 4096 bytes)
(XEN) DOM0: Dentry cache hash table entries: 16384 (order: 5, 131072 bytes)
(XEN) DOM0: Inode-cache hash table entries: 8192 (order: 4, 65536 bytes)
(XEN) DOM0: Memory: 106164K/131072K available (6470K kernel code, 502K rwdata, 2196K rodata, 305K init, 507K bss, 24908K reserved)
(XEN) DOM0: Virtual kernel memory layout:
(XEN) DOM0:     vmalloc : 0xffff000000000000 - 0xffff7c7fbfff0000   (127486 GB)
(XEN) DOM0:     vmemmap : 0xffff7c7fc0000000 - 0xffff7fffc0000000   (  3584 GB maximum)
(XEN) DOM0:               0xffff7c8180380000 - 0xffff7c8180540000   (     1 MB actual)
(XEN) DOM0:     PCI I/O : 0xffff7ffffa000000 - 0xffff7ffffb000000   (    16 MB)
(XEN) DOM0:     fixed   : 0xffff7ffffbdfe000 - 0xffff7ffffbdff000   (     4 KB)
(XEN) DOM0:     modules : 0xffff7ffffc000000 - 0xffff800000000000   (    64 MB)
(XEN) DOM0:     memory  : 0xffff800000000000 - 0xffff800008000000   (   128 MB)
(XEN) DOM0:       .init : 0xffff8000008f8000 - 0xffff800000944680   (   306 KB)
(XEN) DOM0:       .text : 0xffff800000080000 - 0xffff8000008f7184   (  8669 KB)
(XEN) DOM0:       .data : 0xffff800000945000 - 0xffff8000009c2a50   (   503 KB)
(XEN) DOM0: SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=6, Nodes=1
(XEN) DOM0: Hierarchical RCU implementation.
(XEN) DOM0:     RCU restricting CPUs from NR_CPUS=8 to nr_cpu_ids=6.
(XEN) DOM0: RCU: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=6
(XEN) DOM0: NR_IRQS:64 nr_irqs:64 0
(XEN) DOM0: Architected cp15 timer(s) running at 187.50MHz (virt).
(XEN) DOM0: sched_clock: 56 bits at 187MHz, resolution 5ns, wraps every 2932031094784ns
(XEN) DOM0: Calibrating delay loop (skipped), value calculated using timer frequency.. 375.00 BogoMIPS (lpj=1875000)
(XEN) DOM0: pid_max: default: 32768 minimum: 301
(XEN) DOM0: Mount-cache hash table entries: 512 (order: 0, 4096 bytes)
(XEN) DOM0: Mountpoint-cache hash table entries: 512 (order: 0, 4096 bytes)
(XEN) DOM0: Initializing cgroup subsys perf_event
(XEN) DOM0: hw perfevents: enabled with arm/armv8-pmuv3 PMU driver, 1 counters available
(XEN) DOM0: EFI services will not be available.
(XEN) DOM0: Xen 4.5 support found, events_irq=31 gnttab_frame=0x00000000b0000000
(XEN) DOM0: xen:grant_table: Grant tables using version 1 layout
(XEN) DOM0: Grant table initialized
(XEN) DOM0: xen:events: Using FIFO-based ABI
(XEN) DOM0: Xen: initializing cpu0
(XEN) DOM0: Brought up 1 CPUs
(XEN) DOM0: SMP: Total of 1 processors activated.
(XEN) DOM0: devtmpfs: initialized
(XEN) DOM0: xor: measuring software checksum speed
(XEN) DOM0:    8regs     :  4656.400 MB/sec
(XEN) DOM0:    8regs_prefetch:  4202.800 MB/sec
(XEN) DOM0:    32regs    :  4238.400 MB/sec
(XEN) DOM0:    32regs_prefetch:  3563.600 MB/sec
(XEN) DOM0: xor: using function: 8regs (4656.400 MB/sec)
(XEN) DOM0: NET: Registered protocol family 16
(XEN) DOM0: vdso: 2 pages (1 code @ ffff80000094d000, 1 data @ ffff80000094c000)
(XEN) DOM0: hw-breakpoint: found 6 breakpoint and 4 watchpoint registers.
(XEN) DOM0: software IO TLB [mem 0x8016000000-0x8016400000] (4MB) mapped at [ffff800006000000-ffff8000063fffff]
(XEN) DOM0: software IO TLB [mem 0x8015c00000-0x8016000000] (4MB) mapped at [ffff800005c00000-ffff800005ffffff]
(XEN) DOM0: Serial: AMBA PL011 UART driver
(XEN) DOM0: raid6: int64x1    907 MB/s
(XEN) DOM0: raid6: int64x2   1326 MB/s
(XEN) DOM0: raid6: int64x4   1896 MB/s
(XEN) DOM0: raid6: int64x8   1783 MB/s
(XEN) DOM0: raid6: neonx1    1828 MB/s
(XEN) DOM0: raid6: neonx2    2532 MB/s
(XEN) DOM0: raid6: neonx4    2881 MB/s
(XEN) DOM0: raid6: neonx8    2951 MB/s
(XEN) DOM0: raid6: using algorithm neonx8 (2951 MB/s)
(XEN) DOM0: raid6: using intx1 recovery algorithm
(XEN) DOM0: xen:balloon: Initialising balloon driver
(XEN) DOM0: xen_balloon: Initialising balloon driver
(XEN) DOM0: vgaarb: loaded
(XEN) DOM0: SCSI subsystem initialized
(XEN) DOM0: ssp-pl022 e1020000.ssp: ARM PL022 driver, device ID: 0x00241022
(XEN) DOM0: ssp-pl022 e1020000.ssp: mapped registers from 0x00000000e1020000 to ffff000000010000
(XEN) DOM0: of_dma_request_slave_channel: dma-names property of node '/smb/motherboard/ssp@1,1020000' missing or empty
(XEN) DOM0: ssp-pl022 e1020000.ssp: Failed to work in dma mode, work without dma!
(XEN) DOM0: ssp-pl022 e1030000.ssp: ARM PL022 driver, device ID: 0x00241022
(XEN) DOM0: ssp-pl022 e1030000.ssp: mapped registers from 0x00000000e1030000 to ffff000000012000
(XEN) DOM0: of_dma_request_slave_channel: dma-names property of node '/smb/motherboard/ssp@1,1030000' missing or empty
(XEN) DOM0: ssp-pl022 e1030000.ssp: Failed to work in dma mode, work without dma!
(XEN) DOM0: pps_core: LinuxPPS API ver. 1 registered
(XEN) DOM0: pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>
(XEN) DOM0: PTP clock support registered
(XEN) DOM0: Switched to clocksource arch_sys_counter
(XEN) DOM0: NET: Registered protocol family 2
(XEN) DOM0: TCP established hash table entries: 1024 (order: 1, 8192 bytes)
(XEN) DOM0: TCP bind hash table entries: 1024 (order: 2, 16384 bytes)
(XEN) DOM0: TCP: Hash tables configured (established 1024 bind 1024)
(XEN) DOM0: TCP: reno registered
(XEN) DOM0: UDP hash table entries: 256 (order: 1, 8192 bytes)
(XEN) DOM0: UDP-Lite hash table entries: 256 (order: 1, 8192 bytes)
(XEN) DOM0: NET: Registered protocol family 1
(XEN) DOM0: RPC: Registered named UNIX socket transport module.
(XEN) DOM0: RPC: Registered udp transport module.
(XEN) DOM0: RPC: Registered tcp transport module.
(XEN) DOM0: RPC: Registered tcp NFSv4.1 backchannel transport module.
(XEN) DOM0: futex hash table entries: 2048 (order: 5, 131072 bytes)
(XEN) DOM0: HugeTLB registered 2 MB page size, pre-allocated 0 pages
(XEN) DOM0: Installing knfsd (copyright (C) 1996 okir@monad.swb.de).
(XEN) DOM0: JFS: nTxBlock = 829, nTxLock = 6635
(XEN) DOM0: SGI XFS with security attributes, no debug enabled
(XEN) DOM0: 9p: Installing v9fs 9p2000 file system support
(XEN) DOM0: msgmni has been set to 207
(XEN) DOM0: bounce: pool size: 64 pages
(XEN) DOM0: Block layer SCSI generic (bsg) driver version 0.4 loaded (major 251)
(XEN) DOM0: io scheduler noop registered
(XEN) DOM0: io scheduler deadline registered
(XEN) DOM0: io scheduler cfq registered (default)
(XEN) DOM0: pl061_gpio e1040000.gpio: PL061 GPIO chip @0x00000000e1040000 registered
(XEN) DOM0: pl061_gpio e1050000.gpio: PL061 GPIO chip @0x00000000e1050000 registered
(XEN) DOM0: AMD-PCI DEBUG: amd_pcie_probe
(XEN) DOM0: AMD-PCI: amd_pcie_probe: matched STYX
(XEN) DOM0: AMD-PCI DEBUG: amd_pcie_parse_dt
(XEN) DOM0: amd-pcie pcie-controller: missing "reg" property
(XEN) DOM0: amd-pcie: probe of pcie-controller failed with error -22
(XEN) DOM0: dma-pl330: probe of e0500000.dma failed with error -5
(XEN) DOM0: xen:xen_evtchn: Event-channel device installed
(XEN) DOM0: Serial: 8250/16550 driver, 4 ports, IRQ sharing disabled
(XEN) DOM0: loop: module loaded
(XEN) DOM0: ahci e0300000.sata: SSS flag set, parallel bus scan disabled
(XEN) DOM0: ahci e0300000.sata: AHCI 0001.0300 32 slots 8 ports 6 Gbps 0xff impl platform mode
(XEN) DOM0: ahci e0300000.sata: flags: 64bit ncq sntf stag pm led clo only pmp pio slum part ccc apst
(XEN) DOM0: scsi host0: ahci_platform
(XEN) DOM0: scsi host1: ahci_platform
(XEN) DOM0: scsi host2: ahci_platform
(XEN) DOM0: scsi host3: ahci_platform
(XEN) DOM0: scsi host4: ahci_platform
(XEN) DOM0: scsi host5: ahci_platform
(XEN) DOM0: scsi host6: ahci_platform
(XEN) DOM0: scsi host7: ahci_platform
(XEN) DOM0: ata1: SATA max UDMA/133 mmio [mem 0xe0300000-0xe03007ff] port 0x100 irq 387
(XEN) DOM0: ata2: SATA max UDMA/133 mmio [mem 0xe0300000-0xe03007ff] port 0x180 irq 387
(XEN) DOM0: ata3: SATA max UDMA/133 mmio [mem 0xe0300000-0xe03007ff] port 0x200 irq 387
(XEN) DOM0: ata4: SATA max UDMA/133 mmio [mem 0xe0300000-0xe03007ff] port 0x280 irq 387
(XEN) DOM0: ata5: SATA max UDMA/133 mmio [mem 0xe0300000-0xe03007ff] port 0x300 irq 387
(XEN) DOM0: ata6: SATA max UDMA/133 mmio [mem 0xe0300000-0xe03007ff] port 0x380 irq 387
(XEN) DOM0: ata7: SATA max UDMA/133 mmio [mem 0xe0300000-0xe03007ff] port 0x400 irq 387
(XEN) DOM0: ata8: SATA max UDMA/133 mmio [mem 0xe0300000-0xe03007ff] port 0x480 irq 387
(XEN) DOM0: tun: Universal TUN/TAP device driver, 1.6
(XEN) DOM0: tun: (C) 1999-2004 Max Krasnyansky <maxk@qualcomm.com>
(XEN) DOM0: libphy: amd_xgbe_phy: probed
(XEN) DOM0: amd-xgbe e0700000.xgmac eth0: net device enabled
(XEN) DOM0: libphy: amd_xgbe_phy: probed
(XEN) DOM0: amd-xgbe e0900000.xgmac eth1: net device enabled
(XEN) DOM0: xen_netfront: Initialising Xen virtual ethernet driver
(XEN) DOM0: VFIO - User Level meta-driver version: 0.3
(XEN) DOM0: device-mapper: ioctl: 4.27.0-ioctl (2013-10-30) initialised: dm-devel@redhat.com
(XEN) DOM0: device-mapper: multipath: version 1.7.0 loaded
(XEN) DOM0: device-mapper: multipath round-robin: version 1.0.0 loaded
(XEN) DOM0: mmc_spi spi32765.0: OF: voltage-ranges unspecified
(XEN) DOM0: mmc_spi spi32765.0: ASSUMING 3.2-3.4 V slot power
(XEN) DOM0: mmc_spi spi32765.0: SD/MMC host mmc0, no WP, no poweroff
(XEN) DOM0: TCP: cubic registered
(XEN) DOM0: NET: Registered protocol family 10
(XEN) DOM0: sit: IPv6 over IPv4 tunneling driver
(XEN) DOM0: NET: Registered protocol family 17
(XEN) DOM0: Bridge firewalling registered
(XEN) DOM0: 9pnet: Installing 9P2000 support
(XEN) DOM0: NET: Registered protocol family 37
(XEN) DOM0: bootconsole [uart0] disabled


[-- Attachment #3: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Xen on Seattle

[Julien Grall]

Hi Suravee,

On 10/09/14 15:38, Suravee Suthikulanit wrote:
> .....
> (XEN) DOM0: NET: Registered protocol family 10
> (XEN) DOM0: sit: IPv6 over IPv4 tunneling driver
> (XEN) DOM0: NET: Registered protocol family 17
> (XEN) DOM0: Bridge firewalling registered
> (XEN) DOM0: 9pnet: Installing 9P2000 support
> (XEN) DOM0: NET: Registered protocol family 37
> (XEN) DOM0: bootconsole [uart0] disabled
>
> Please see attachment for full boot log. I am still working on trying to
> get to Dom0 console.

Xen is providing a tiny UART emulation which only handle write. It looks 
like you kernel is using it. Did add "console=hvc0" on your kernel 
command line?

This will allow DOM0 to use the paravirtualised console.

Regards,

-- 
Julien Grall

Re: Xen on Seattle

[Suravee Suthikulanit]

On 9/10/2014 6:00 PM, Julien Grall wrote:
> Hi Suravee,
>
> On 10/09/14 15:38, Suravee Suthikulanit wrote:
>> .....
>> (XEN) DOM0: NET: Registered protocol family 10
>> (XEN) DOM0: sit: IPv6 over IPv4 tunneling driver
>> (XEN) DOM0: NET: Registered protocol family 17
>> (XEN) DOM0: Bridge firewalling registered
>> (XEN) DOM0: 9pnet: Installing 9P2000 support
>> (XEN) DOM0: NET: Registered protocol family 37
>> (XEN) DOM0: bootconsole [uart0] disabled
>>
>> Please see attachment for full boot log. I am still working on trying to
>> get to Dom0 console.
>
> Xen is providing a tiny UART emulation which only handle write. It looks
> like you kernel is using it. Did add "console=hvc0" on your kernel
> command line?
>
> This will allow DOM0 to use the paravirtualised console.
>
> Regards,
>

Ah ok, this got the console out now. I still need to clean up a bit. But 
it looks to be in good shape. Thanks for the pointer ;)

Suravee

......
(XEN) DOM0: bootconsole [uart0] disabled
bootconsole [uart0] disabled
Btrfs loaded
drivers/rtc/hctosys.c: unable to open rtc device (rtc0)
ata1: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
ata1.00: ATA-8: Corsair CSSD-F200GB2, 0.2, max UDMA/133
ata1.00: 390721968 sectors, multi 16: LBA48 NCQ (depth 31/32)
ata1.00: configured for UDMA/133
scsi 0:0:0:0: Direct-Access     ATA      Corsair CSSD-F20 0.2  PQ: 0 ANSI: 5
sd 0:0:0:0: [sda] 390721968 512-byte logical blocks: (200 GB/186 GiB)
sd 0:0:0:0: Attached scsi generic sg0 type 0
sd 0:0:0:0: [sda] Write Protect is off
sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't 
support DPO or FUA
  sda: sda1 sda2 sda3 sda4
sd 0:0:0:0: [sda] Attached SCSI disk
ata2: SATA link down (SStatus 0 SControl 300)
ata3: SATA link down (SStatus 0 SControl 300)
ata4: SATA link down (SStatus 0 SControl 300)
ata5: SATA link down (SStatus 0 SControl 300)
ata6: SATA link down (SStatus 0 SControl 300)
ata7: SATA link down (SStatus 0 SControl 300)
ata8: SATA link down (SStatus 0 SControl 300)
kjournald starting.  Commit interval 5 seconds
EXT3-fs (sda2): mounted filesystem with ordered data mode
VFS: Mounted root (ext3 filesystem) readonly on device 8:2.
devtmpfs: mounted
Freeing unused kernel memory: 304K (ffff8000008f8000 - ffff800000944000)
STYX_INFO: ==== Styx Kernel Info ====
STYX_INFO:  Compile Info:
STYX_INFO:    UTS release : 3.17.0-rc4-styx-xen+
STYX_INFO:    UTS machine : aarch64
STYX_INFO:    UTS version : #15 SMP Wed Sep 10 16:54:52 CDT 2014
STYX_INFO:    Compiled by : Linux version 3.17.0-rc4-styx-xen+ 
(ssuthiku@sos-arm-dev) (gcc version 4.8.2 20140110 (prerelease) 
[ibm/gcc-4_8-branch merged from gcc-4_8-branch, revision 205847] 
(Ubuntu/Linaro 4.8.2-13ubuntu1) ) #15 SMP Wed Sep 10 16:54:52 CDT 2014

STYX_INFO:  Build Info:
STYX_INFO:  Boot Info:
STYX_INFO:    Present/Online/Offline CPUs : 1/1/0
STYX_INFO:  Modules Info:
STYX_INFO:    CONFIG_ARCH_STYX: Yes
STYX_INFO:    CONFIG_SERIAL_AMBA_PL011: Yes
STYX_INFO:    CONFIG_ARM_GIC: Yes
STYX_INFO:    CONFIG_SATA_AHCI_PLATFORM: Yes
STYX_INFO:    CONFIG_PCI: Yes
STYX_INFO:    CONFIG_PCI_MSI: Yes
STYX_INFO:    CONFIG_PCI_AMD: Yes
STYX_INFO:    CONFIG_CRYPTO_DEV_CCP: Yes
STYX_INFO:    CONFIG_CRYPTO_DEV_CCP_DD: Module
STYX_INFO:    CONFIG_CRYPTO_DEV_CCP_CRYPTO: Module
STYX_INFO:    CONFIG_AMD_XGBE: Yes
STYX_INFO:    CONFIG_ARM_SMMU: Yes
STYX_INFO: ========================================
random: init urandom read with 28 bits of entropy available
init: ureadahead main process (94) terminated with status 5

Re: Xen on Seattle

[Stefano Stabellini]

On Wed, 10 Sep 2014, Suravee Suthikulanit wrote:
> On 9/10/2014 6:00 PM, Julien Grall wrote:
> > Hi Suravee,
> > 
> > On 10/09/14 15:38, Suravee Suthikulanit wrote:
> > > .....
> > > (XEN) DOM0: NET: Registered protocol family 10
> > > (XEN) DOM0: sit: IPv6 over IPv4 tunneling driver
> > > (XEN) DOM0: NET: Registered protocol family 17
> > > (XEN) DOM0: Bridge firewalling registered
> > > (XEN) DOM0: 9pnet: Installing 9P2000 support
> > > (XEN) DOM0: NET: Registered protocol family 37
> > > (XEN) DOM0: bootconsole [uart0] disabled
> > > 
> > > Please see attachment for full boot log. I am still working on trying to
> > > get to Dom0 console.
> > 
> > Xen is providing a tiny UART emulation which only handle write. It looks
> > like you kernel is using it. Did add "console=hvc0" on your kernel
> > command line?
> > 
> > This will allow DOM0 to use the paravirtualised console.
> > 
> > Regards,
> > 
> 
> Ah ok, this got the console out now. I still need to clean up a bit. But it
> looks to be in good shape. Thanks for the pointer ;)

Cool!
I wouldn't wait too long before sending the patches for Xen: the feature
freeze is in 2 weeks!


> Suravee
> 
> ......
> (XEN) DOM0: bootconsole [uart0] disabled
> bootconsole [uart0] disabled
> Btrfs loaded
> drivers/rtc/hctosys.c: unable to open rtc device (rtc0)
> ata1: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
> ata1.00: ATA-8: Corsair CSSD-F200GB2, 0.2, max UDMA/133
> ata1.00: 390721968 sectors, multi 16: LBA48 NCQ (depth 31/32)
> ata1.00: configured for UDMA/133
> scsi 0:0:0:0: Direct-Access     ATA      Corsair CSSD-F20 0.2  PQ: 0 ANSI: 5
> sd 0:0:0:0: [sda] 390721968 512-byte logical blocks: (200 GB/186 GiB)
> sd 0:0:0:0: Attached scsi generic sg0 type 0
> sd 0:0:0:0: [sda] Write Protect is off
> sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support
> DPO or FUA
>  sda: sda1 sda2 sda3 sda4
> sd 0:0:0:0: [sda] Attached SCSI disk
> ata2: SATA link down (SStatus 0 SControl 300)
> ata3: SATA link down (SStatus 0 SControl 300)
> ata4: SATA link down (SStatus 0 SControl 300)
> ata5: SATA link down (SStatus 0 SControl 300)
> ata6: SATA link down (SStatus 0 SControl 300)
> ata7: SATA link down (SStatus 0 SControl 300)
> ata8: SATA link down (SStatus 0 SControl 300)
> kjournald starting.  Commit interval 5 seconds
> EXT3-fs (sda2): mounted filesystem with ordered data mode
> VFS: Mounted root (ext3 filesystem) readonly on device 8:2.
> devtmpfs: mounted
> Freeing unused kernel memory: 304K (ffff8000008f8000 - ffff800000944000)
> STYX_INFO: ==== Styx Kernel Info ====
> STYX_INFO:  Compile Info:
> STYX_INFO:    UTS release : 3.17.0-rc4-styx-xen+
> STYX_INFO:    UTS machine : aarch64
> STYX_INFO:    UTS version : #15 SMP Wed Sep 10 16:54:52 CDT 2014
> STYX_INFO:    Compiled by : Linux version 3.17.0-rc4-styx-xen+
> (ssuthiku@sos-arm-dev) (gcc version 4.8.2 20140110 (prerelease)
> [ibm/gcc-4_8-branch merged from gcc-4_8-branch, revision 205847]
> (Ubuntu/Linaro 4.8.2-13ubuntu1) ) #15 SMP Wed Sep 10 16:54:52 CDT 2014
> 
> STYX_INFO:  Build Info:
> STYX_INFO:  Boot Info:
> STYX_INFO:    Present/Online/Offline CPUs : 1/1/0
> STYX_INFO:  Modules Info:
> STYX_INFO:    CONFIG_ARCH_STYX: Yes
> STYX_INFO:    CONFIG_SERIAL_AMBA_PL011: Yes
> STYX_INFO:    CONFIG_ARM_GIC: Yes
> STYX_INFO:    CONFIG_SATA_AHCI_PLATFORM: Yes
> STYX_INFO:    CONFIG_PCI: Yes
> STYX_INFO:    CONFIG_PCI_MSI: Yes
> STYX_INFO:    CONFIG_PCI_AMD: Yes
> STYX_INFO:    CONFIG_CRYPTO_DEV_CCP: Yes
> STYX_INFO:    CONFIG_CRYPTO_DEV_CCP_DD: Module
> STYX_INFO:    CONFIG_CRYPTO_DEV_CCP_CRYPTO: Module
> STYX_INFO:    CONFIG_AMD_XGBE: Yes
> STYX_INFO:    CONFIG_ARM_SMMU: Yes
> STYX_INFO: ========================================
> random: init urandom read with 28 bits of entropy available
> init: ureadahead main process (94) terminated with status 5
>