Problems running Xen 4.4 on OMAP5432 evm

Problems running Xen 4.4 on OMAP5432 evm

[Kapania, Ashish]

Hi,

I am trying to run Xen (+ Linux dom0) on OMAP5432 evm and have been having
trouble just booting xen itself. The xen wiki does not have build instructions
for OMAP5432 but the source code seems to have support for it so I am assuming
that board is supported. I am using xen RELEASE-4.4.0 git tag to build a xen
image. Here's my build command-line:

make dist-xen XEN_TARGET_ARCH=arm32 CROSS_COMPILE=arm-linux-gnueabihf- CONFIG_EARLY_PRINTK=omap5432

I am using uboot 2014.01 with tweaks to put A15 in hypervisor mode and am
using Linux 3.8 kernel (with xen related CONFIGs turned on).

When I boot xen, I see the A15 hangs in setup_pagetables() function in arch/arm/mm.c
file. Here's the UART log I get with earlyprintk turned on:

UART log:
===========================================================================
U-Boot# fatload mmc 0 0x825f0000 omap5-uevm.dtb
reading omap5-uevm.dtb
27990 bytes read in 8 ms (3.3 MiB/s)
U-Boot# fatload mmc 0 0x80300000 zImage
reading zImage
5039880 bytes read in 270 ms (17.8 MiB/s)
U-Boot# fatload mmc 0 0xE0000000 uXen
reading uXen
656208 bytes read in 40 ms (15.6 MiB/s)
U-Boot# setenv fdt_high 0x84000000
U-Boot# fdt addr 0x825f0000
U-Boot# fdt resize
U-Boot# fdt chosen
U-Boot# fdt set /chosen \#address-cells <1>
U-Boot# fdt set /chosen \#size-cells <1>
U-Boot# fdt set /chosen xen,dom0-bootargs "elevator=noop console=hvc0,115200n8 root=/dev/mmcblk0p2 rw rootwait earlyprintk=xen fixrtc";
U-Boot# fdt set /chosen xen,xen-bootargs "nosmp=true bootscrub=false dom0_mem=256M console=dtuart dtuart=serial2 earlyprintk=xen";
U-Boot# fdt mknod /chosen modules
U-Boot# fdt set /chosen/modules comptaible "xen,linux-zimage" "xen,multiboot-module"
U-Boot# fdt set /chosen/modules reg <0x80300000 0x4cf000>
U-Boot# fdt print /chosen
chosen {
        xen,xen-bootargs = "nosmp=true bootscrub=false dom0_mem=256M console=dtuart dtuart=serial2 earlyprintk=xen";
        xen,dom0-bootargs = "elevator=noop console=hvc0,115200n8 root=/dev/mmcblk0p2 rw rootwait earlyprintk=xen fixrtc";
        #size-cells = <0x00000001>;
        #address-cells = <0x00000001>;
        modules {
                reg = <0x80300000 0x004cf000>;
                comptaible = "xen,linux-zimage", "xen,multiboot-module";
        };
};
U-Boot# bootm 0xE0000000 - 0x825F0000
## Booting kernel from Legacy Image at e0000000 ...
   Image Name:   Xen
   Image Type:   ARM Linux Kernel Image (uncompressed)
   Data Size:    656144 Bytes = 640.8 KiB
   Load Address: f0000000
   Entry Point:  f0000000
   Verifying Checksum ... OK
## Flattened Device Tree blob at 825f0000
   Booting using the fdt blob at 0x825f0000
   Loading Kernel Image ... OK
   reserving fdt memory region: addr=825f0000 size=7000
   Loading Device Tree to 83ff6000, end 83ffffff ... OK

Starting kernel ...

- UART enabled -
- CPU 00000000 booting -
- Xen starting in Hyp mode -
- Zero BSS -
- Setting up control registers -
- Turning on paging -
- Ready -
Checking for initrd in /chosen
RAM: 0000000080000000 - 00000000feffffff

MODULE[1]: 0000000083ff6000 - 0000000083ffd000 
 RESVD[0]: 00000000825f0000 - 00000000825f7000
 RESVD[1]: 0000000083ff6000 - 0000000083ffd000

Command line: dom0_mem=256M console=dtuart dtuart=serial2 earlyprintk=xen
Placing Xen at 0x00000000fee00000-0x00000000ff000000

===========================================================================

Any ideas on what I might be doing wrong ? Maybe I am missing some patch
that is required to run xen on omap5432 ?

Thanks,
Ashish

Re: Problems running Xen 4.4 on OMAP5432 evm

[Ian Campbell]

On Wed, 2014-04-16 at 00:23 +0000, Kapania, Ashish wrote:
> Hi,
> 
> I am trying to run Xen (+ Linux dom0) on OMAP5432 evm and have been having
> trouble just booting xen itself. The xen wiki does not have build instructions
> for OMAP5432 but the source code seems to have support for it so I am assuming
> that board is supported. I am using xen RELEASE-4.4.0 git tag to build a xen
> image. Here's my build command-line:
> 
> make dist-xen XEN_TARGET_ARCH=arm32 CROSS_COMPILE=arm-linux-gnueabihf- CONFIG_EARLY_PRINTK=omap5432

Worth trying with "debug=y" on this too (will need a complete clean to
take full affect)

> When I boot xen, I see the A15 hangs in setup_pagetables() function in arch/arm/mm.c
> file.

Can you say exactly where?

> MODULE[1]: 0000000083ff6000 - 0000000083ffd000 
>  RESVD[0]: 00000000825f0000 - 00000000825f7000
>  RESVD[1]: 0000000083ff6000 - 0000000083ffd000
> 
> Command line: dom0_mem=256M console=dtuart dtuart=serial2 earlyprintk=xen
> Placing Xen at 0x00000000fee00000-0x00000000ff000000

At this point Xen will be relocating itself. I'm not sure what has gone
wrong though. I presume 0xfee000000 is actually in RAM?

> Any ideas on what I might be doing wrong ? Maybe I am missing some patch
> that is required to run xen on omap5432 ?

I can't think of any relevant patches which aren't in 4.4.0, but it
would certainly be good to try the development branch to be sure (if
that works then we can figure out which patch needs backporting).

Otherwise I'm afraid the issue will need some more debugging.

Ian.

Re: Problems running Xen 4.4 on OMAP5432 evm

[Julien Grall]

On 04/16/2014 11:43 AM, Ian Campbell wrote:
> On Wed, 2014-04-16 at 00:23 +0000, Kapania, Ashish wrote:
>> Hi,
>>
>> I am trying to run Xen (+ Linux dom0) on OMAP5432 evm and have been having
>> trouble just booting xen itself. The xen wiki does not have build instructions
>> for OMAP5432 but the source code seems to have support for it so I am assuming
>> that board is supported. I am using xen RELEASE-4.4.0 git tag to build a xen
>> image. Here's my build command-line:
>>
>> make dist-xen XEN_TARGET_ARCH=arm32 CROSS_COMPILE=arm-linux-gnueabihf- CONFIG_EARLY_PRINTK=omap5432
> 
> Worth trying with "debug=y" on this too (will need a complete clean to
> take full affect)

I think he already has debug=y because early printk is working.

I'm wondering how ... because xen 4.4 as debug=n by default.

>> When I boot xen, I see the A15 hangs in setup_pagetables() function in arch/arm/mm.c
>> file.
> 
> Can you say exactly where?
> 
>> MODULE[1]: 0000000083ff6000 - 0000000083ffd000 
>>  RESVD[0]: 00000000825f0000 - 00000000825f7000
>>  RESVD[1]: 0000000083ff6000 - 0000000083ffd000
>>
>> Command line: dom0_mem=256M console=dtuart dtuart=serial2 earlyprintk=xen

earlyprintk=xen is not used by Xen.

-- 
Julien Grall

Re: Problems running Xen 4.4 on OMAP5432 evm

[Kapania, Ashish]

Hi Ian, Julien,

Thanks for your response.

I set "debug=y" to make earlyprintk work.

I added a few earlyprintk stmts in setup_pagetable() to narrow
down the problem. The last print msg I get is just before
flush_xen_text_tlb(). I don’t get the prints after that call.

After the above exercise, I decided to check if the A15's
mode somehow got changed from Hypervisor Mode to some other mode.
I read the CPSR using inline assembly and passed it to the printk
call just before flush_xen_text_tlb(). I found that the print
message is no longer printed when I added the CPSR argument.
So, if I try to add an argument to printk before tlb flush,
the message is not printed but if I remove the argument it
gets printed. It does not look like the prints are being
buffered so I am a bit confused by this behavior.

0xfee000000 address should be in RAM. The RAM address range
is 0x80000000 to 0xFFFFFFFF.

I will try building using the development branch to see if that
helps.

Best,
Ashish

-----Original Message-----
From: Ian Campbell [mailto:Ian.Campbell@citrix.com] 
Sent: Wednesday, April 16, 2014 3:43 AM
To: Kapania, Ashish
Cc: xen-devel@lists.xen.org
Subject: Re: [Xen-devel] Problems running Xen 4.4 on OMAP5432 evm

On Wed, 2014-04-16 at 00:23 +0000, Kapania, Ashish wrote:
> Hi,
> 
> I am trying to run Xen (+ Linux dom0) on OMAP5432 evm and have been 
> having trouble just booting xen itself. The xen wiki does not have 
> build instructions for OMAP5432 but the source code seems to have 
> support for it so I am assuming that board is supported. I am using 
> xen RELEASE-4.4.0 git tag to build a xen image. Here's my build command-line:
> 
> make dist-xen XEN_TARGET_ARCH=arm32 CROSS_COMPILE=arm-linux-gnueabihf- 
> CONFIG_EARLY_PRINTK=omap5432

Worth trying with "debug=y" on this too (will need a complete clean to take full affect)

> When I boot xen, I see the A15 hangs in setup_pagetables() function in 
> arch/arm/mm.c file.

Can you say exactly where?

> MODULE[1]: 0000000083ff6000 - 0000000083ffd000
>  RESVD[0]: 00000000825f0000 - 00000000825f7000
>  RESVD[1]: 0000000083ff6000 - 0000000083ffd000
> 
> Command line: dom0_mem=256M console=dtuart dtuart=serial2 
> earlyprintk=xen Placing Xen at 0x00000000fee00000-0x00000000ff000000

At this point Xen will be relocating itself. I'm not sure what has gone wrong though. I presume 0xfee000000 is actually in RAM?

> Any ideas on what I might be doing wrong ? Maybe I am missing some 
> patch that is required to run xen on omap5432 ?

I can't think of any relevant patches which aren't in 4.4.0, but it would certainly be good to try the development branch to be sure (if that works then we can figure out which patch needs backporting).

Otherwise I'm afraid the issue will need some more debugging.

Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Problems running Xen 4.4 on OMAP5432 evm

[Kapania, Ashish]

Hi Ian,

I was able to make progress with Xen 4.4 release itself. The xen image was being
placed at VA 0x200000 (which is not a valid address on OMAP5) so I had updated
xen.lds to change this. It seems that was the wrong thing to do and was causing
the problem (Does xen perform a VA to PA mapping for xen address space ?). After
undoing my changes, I was able to get further.

My latest xen log is shared below. Xen gets to the point where it says loading
Kernel but it does not look like dom0 linux is booting up as I don’t see any
logs printed on uart. I have set "console=hvc0" in the dom0
Bootargs. I am running linux 3.8.13 and am able to boot it by itself so the
Linux image is working.

I am hoping to get some suggestions from you on how to debug this further.

Thanks,
Ashish

==============================================================================

FDT /chosen node:
U-Boot# fdt print /chosen
chosen {
        #size-cells = <0x00000001>;
        #address-cells = <0x00000001>;
        xen,xen-bootargs = "dom0_mem=128M sync_console console=dtuart dtuart=serial2 noreboot ";
        xen,dom0-bootargs = "console=hvc0 root=/dev/mmcblk0p2 rw rootwait fixrtc";
        module@0 {
                reg = <0x80300000 0x00500000>;
                compatible = "xen,linux-zimage", "xen,multiboot-module";
        };
};
==============================================================================

XEN Log:
U-Boot# fatload mmc 0 0x825f0000 omap5-uevm.dtb
reading omap5-uevm.dtb
27990 bytes read in 8 ms (3.3 MiB/s)
U-Boot# fatload mmc 0 0x80300000 zImage
reading zImage
5039880 bytes read in 270 ms (17.8 MiB/s)
U-Boot# fatload mmc 0 0xE0000000 uXen
reading uXen
656208 bytes read in 37 ms (16.9 MiB/s)
...
U-Boot# bootm 0xE0000000 - 0x825F0000
## Booting kernel from Legacy Image at e0000000 ...
   Image Name:   Xen
   Image Type:   ARM Linux Kernel Image (uncompressed)
   Data Size:    656144 Bytes = 640.8 KiB
   Load Address: 85000000
   Entry Point:  85000000
   Verifying Checksum ... OK
## Flattened Device Tree blob at 825f0000
   Booting using the fdt blob at 0x825f0000
   Loading Kernel Image ... OK
   reserving fdt memory region: addr=825f0000 size=7000
   Loading Device Tree to 83ff6000, end 83ffffff ... OK

Starting kernel ...

- UART enabled -
- CPU 00000000 booting -
- Xen starting in Hyp mode -
- Zero BSS -
- Setting up control registers -
- Turning on paging -
- Ready -
Checking for initrd in /chosen
RAM: 0000000080000000 - 00000000feffffff

MODULE[1]: 0000000083ff6000 - 0000000083ffd000 
MODULE[2]: 0000000080300000 - 0000000080800000 
 RESVD[0]: 00000000825f0000 - 00000000825f7000
 RESVD[1]: 0000000083ff6000 - 0000000083ffd000

Command line: dom0_mem=128M sync_console console=dtuart dtuart=serial2 noreboot
Placing Xen at 0x00000000fee00000-0x00000000ff000000
Xen heap: 00000000ee000000-00000000fe000000 (65536 pages)
Dom heap: 454656 pages
Looking for UART console serial2
 Xen 4.4.0
(XEN) Xen version 4.4.0 (a0273866@) (arm-linux-gnueabihf-gcc (crosstool-NG linaro-1.13.1-4.7-2013.03-20130313 - Linaro GCC 2013.03) 4.7.3 20130226 (prerelease)) debug=y Mon Apr 21 17:29:21 PDT 2014
(XEN) Latest ChangeSet: Mon Mar 10 10:23:39 2014 +0000 git:816e8d8-dirty
(XEN) Console output is synchronous.
(XEN) Processor: 412fc0f2: "ARM Limited", variant: 0x2, part 0xc0f, rev 0x2
(XEN) 32-bit Execution:
(XEN)   Processor Features: 00001131:00011011
(XEN)     Instruction Sets: AArch32 Thumb Thumb-2 ThumbEE Jazelle
(XEN)     Extensions: GenericTimer Security
(XEN)   Debug Features: 02010555
(XEN)   Auxiliary Features: 00000000
(XEN)   Memory Model Features: 10201105 20000000 01240000 02102211
(XEN)  ISA Features: 02101110 13112111 21232041 11112131 10011142 00000000
(XEN) Platform: TI OMAP5
(XEN) Set AuxCoreBoot1 to 00000000fee0004c (0020004c)
(XEN) Set AuxCoreBoot0 to 0x20
(XEN) DT missing boot CPU MPIDR[23:0]
(XEN) Using only 1 CPU
(XEN) Generic Timer IRQ: phys=30 hyp=26 virt=27
(XEN) Using generic timer at 6144 KHz
(XEN) GIC initialization:
(XEN)         gic_dist_addr=0000000048211000
(XEN)         gic_cpu_addr=0000000048212000
(XEN)         gic_hyp_addr=0000000048214000
(XEN)         gic_vcpu_addr=0000000048216000
(XEN)         gic_maintenance_irq=25
(XEN) GIC: 192 lines, 2 cpus, secure (IID 0000043b).
(XEN) Using scheduler: SMP Credit Scheduler (credit)
(XEN) Allocated console ring of 16 KiB.
(XEN) VFP implementer 0x41 architecture 4 part 0x30 variant 0xf rev 0x0
(XEN) Brought up 1 CPUs
(XEN) *** LOADING DOMAIN 0 ***
(XEN) Populate P2M 0x88000000->0x90000000 (1:1 mapping for dom0)
(XEN) Loading kernel from boot module 2
(XEN) Loading zImage from 0000000080300000 to 000000008fa00000-000000008fece708
(XEN) Loading dom0 DTB to 0x000000008f800000-0x000000008f806d67
(XEN) Scrubbing Free RAM: .................done.
(XEN) Initial low memory virq threshold set at 0x4000 pages.
(XEN) Std. Loglevel: All
(XEN) Guest Loglevel: All
(XEN) **********************************************
(XEN) ******* WARNING: CONSOLE OUTPUT IS SYNCHRONOUS
(XEN) ******* This option is intended to aid debugging of Xen by ensuring
(XEN) ******* that all output is synchronously delivered on the serial line.
(XEN) ******* However it can introduce SIGNIFICANT latencies and affect
(XEN) ******* timekeeping. It is NOT recommended for production use!
(XEN) **********************************************
(XEN) 3... 2... 1... 
(XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to switch input to Xen)
(XEN) Freed 264kB init memory.

-----Original Message-----
From: Kapania, Ashish 
Sent: Wednesday, April 16, 2014 11:22 AM
To: 'Ian Campbell'
Cc: xen-devel@lists.xen.org
Subject: RE: [Xen-devel] Problems running Xen 4.4 on OMAP5432 evm

Hi Ian, Julien,

Thanks for your response.

I set "debug=y" to make earlyprintk work.

I added a few earlyprintk stmts in setup_pagetable() to narrow down the problem. The last print msg I get is just before flush_xen_text_tlb(). I don’t get the prints after that call.

After the above exercise, I decided to check if the A15's mode somehow got changed from Hypervisor Mode to some other mode.
I read the CPSR using inline assembly and passed it to the printk call just before flush_xen_text_tlb(). I found that the print message is no longer printed when I added the CPSR argument.
So, if I try to add an argument to printk before tlb flush, the message is not printed but if I remove the argument it gets printed. It does not look like the prints are being buffered so I am a bit confused by this behavior.

0xfee000000 address should be in RAM. The RAM address range is 0x80000000 to 0xFFFFFFFF.

I will try building using the development branch to see if that helps.

Best,
Ashish

-----Original Message-----
From: Ian Campbell [mailto:Ian.Campbell@citrix.com]
Sent: Wednesday, April 16, 2014 3:43 AM
To: Kapania, Ashish
Cc: xen-devel@lists.xen.org
Subject: Re: [Xen-devel] Problems running Xen 4.4 on OMAP5432 evm

On Wed, 2014-04-16 at 00:23 +0000, Kapania, Ashish wrote:
> Hi,
> 
> I am trying to run Xen (+ Linux dom0) on OMAP5432 evm and have been 
> having trouble just booting xen itself. The xen wiki does not have 
> build instructions for OMAP5432 but the source code seems to have 
> support for it so I am assuming that board is supported. I am using 
> xen RELEASE-4.4.0 git tag to build a xen image. Here's my build command-line:
> 
> make dist-xen XEN_TARGET_ARCH=arm32 CROSS_COMPILE=arm-linux-gnueabihf-
> CONFIG_EARLY_PRINTK=omap5432

Worth trying with "debug=y" on this too (will need a complete clean to take full affect)

> When I boot xen, I see the A15 hangs in setup_pagetables() function in 
> arch/arm/mm.c file.

Can you say exactly where?

> MODULE[1]: 0000000083ff6000 - 0000000083ffd000
>  RESVD[0]: 00000000825f0000 - 00000000825f7000
>  RESVD[1]: 0000000083ff6000 - 0000000083ffd000
> 
> Command line: dom0_mem=256M console=dtuart dtuart=serial2 
> earlyprintk=xen Placing Xen at 0x00000000fee00000-0x00000000ff000000

At this point Xen will be relocating itself. I'm not sure what has gone wrong though. I presume 0xfee000000 is actually in RAM?

> Any ideas on what I might be doing wrong ? Maybe I am missing some 
> patch that is required to run xen on omap5432 ?

I can't think of any relevant patches which aren't in 4.4.0, but it would certainly be good to try the development branch to be sure (if that works then we can figure out which patch needs backporting).

Otherwise I'm afraid the issue will need some more debugging.

Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Problems running Xen 4.4 on OMAP5432 evm

[Ian Campbell]

On Tue, 2014-04-22 at 02:39 +0000, Kapania, Ashish wrote:
> Hi Ian,
> 
> I was able to make progress with Xen 4.4 release itself. The xen image was being
> placed at VA 0x200000 (which is not a valid address on OMAP5) so I had updated
> xen.lds to change this.

If you are going to randomly hack at the code and then report bugs then
you *must* tell us exactly what you have changed. We are not mind
readers.

>  It seems that was the wrong thing to do 

Yes

> and was causing
> the problem (Does xen perform a VA to PA mapping for xen address space ?).

Yes, it uses paging in the normal way.

>  After
> undoing my changes, I was able to get further.
> 
> My latest xen log is shared below. Xen gets to the point where it says loading
> Kernel but it does not look like dom0 linux is booting up as I don’t see any
> logs printed on uart. I have set "console=hvc0" in the dom0
> Bootargs. I am running linux 3.8.13 and am able to boot it by itself so the
> Linux image is working.
> 
> I am hoping to get some suggestions from you on how to debug this further.

http://article.gmane.org/gmane.comp.emulators.xen.devel/196351 might
shed some light.

My usual next step would be to press Ctrl-A three times and then press
'q' to see if I can identify the PC where dom0 has crashed or is stuck.

After that I would try adding some calls to xen_raw_printk in the kernel
to see if I could identify how far the kernel was getting.

Perhaps someone who has worked with the omap platform can give more
specific advise.

Ian.


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Problems running Xen 4.4 on OMAP5432 evm

[Kapania, Ashish]

Hi Ian,

I apologize for causing any confusion. The xen.lds file I modified is generated and
hence not tracked by git. I had forgotten about the change and running "git status"
was not telling me that I modified the file. Another point I wanted to make was
that I always do a "make distclean" before rebuilding xen but the xen.lds file
does not get regenerated. I had to manually delete it to force regeneration.
Maybe that’s a bug ?

I tried the Ctrl-A * 3 + press 'd' sequence and found that dom0 is in ABT mode.
I tried debugging a little with JTAG (My JTAG setup does not work well in HYP
Mode so I cannot debug xen with it) and found that the ABT happens in
Linux/arch/arm/boot/compressed/head.S's __setup_mmu code. The exact instruction
that causes the abort is a store instruction that is attempting to write a page
table entry to 0x80004000.

MMU stage 1 translation is disabled at this point so it looks like the
Stage 2 translation is not mapping 0x80004000 address. Is there some step that
I am missing that is suppose to tell xen to map the memory used by linux to
store page tables ?

Best,
Ashish

-----Original Message-----
From: Ian Campbell [mailto:Ian.Campbell@citrix.com] 
Sent: Tuesday, April 22, 2014 2:58 AM
To: Kapania, Ashish
Cc: xen-devel@lists.xen.org
Subject: Re: [Xen-devel] Problems running Xen 4.4 on OMAP5432 evm

On Tue, 2014-04-22 at 02:39 +0000, Kapania, Ashish wrote:
> Hi Ian,
> 
> I was able to make progress with Xen 4.4 release itself. The xen image 
> was being placed at VA 0x200000 (which is not a valid address on 
> OMAP5) so I had updated xen.lds to change this.

If you are going to randomly hack at the code and then report bugs then you *must* tell us exactly what you have changed. We are not mind readers.

>  It seems that was the wrong thing to do

Yes

> and was causing
> the problem (Does xen perform a VA to PA mapping for xen address space ?).

Yes, it uses paging in the normal way.

>  After
> undoing my changes, I was able to get further.
> 
> My latest xen log is shared below. Xen gets to the point where it says 
> loading Kernel but it does not look like dom0 linux is booting up as I 
> don’t see any logs printed on uart. I have set "console=hvc0" in the 
> dom0 Bootargs. I am running linux 3.8.13 and am able to boot it by 
> itself so the Linux image is working.
> 
> I am hoping to get some suggestions from you on how to debug this further.

http://article.gmane.org/gmane.comp.emulators.xen.devel/196351 might shed some light.

My usual next step would be to press Ctrl-A three times and then press 'q' to see if I can identify the PC where dom0 has crashed or is stuck.

After that I would try adding some calls to xen_raw_printk in the kernel to see if I could identify how far the kernel was getting.

Perhaps someone who has worked with the omap platform can give more specific advise.

Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Problems running Xen 4.4 on OMAP5432 evm

[Julien Grall]

Hi Anish,

On 25/04/14 00:05, Kapania, Ashish wrote:
> I apologize for causing any confusion. The xen.lds file I modified is generated and
> hence not tracked by git. I had forgotten about the change and running "git status"
> was not telling me that I modified the file. Another point I wanted to make was
> that I always do a "make distclean" before rebuilding xen but the xen.lds file
> does not get regenerated. I had to manually delete it to force regeneration.
> Maybe that’s a bug ?
>
> I tried the Ctrl-A * 3 + press 'd' sequence and found that dom0 is in ABT mode.
> I tried debugging a little with JTAG (My JTAG setup does not work well in HYP
> Mode so I cannot debug xen with it) and found that the ABT happens in
> Linux/arch/arm/boot/compressed/head.S's __setup_mmu code. The exact instruction
> that causes the abort is a store instruction that is attempting to write a page
> table entry to 0x80004000.
>
> MMU stage 1 translation is disabled at this point so it looks like the
> Stage 2 translation is not mapping 0x80004000 address. Is there some step that
> I am missing that is suppose to tell xen to map the memory used by linux to
> store page tables ?

Can you post the console output of Xen and DOM0 (if you have some)? I'd 
like to seems how Xen populates the RAM for DOM0.

Regards,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Problems running Xen 4.4 on OMAP5432 evm

[Kapania, Ashish]

Hi Julien,

Here's the console output I get:

## Booting kernel from Legacy Image at e0000000 ...
   Image Name:   Xen
   Image Type:   ARM Linux Kernel Image (uncompressed)
   Data Size:    656156 Bytes = 640.8 KiB
   Load Address: 80200000
   Entry Point:  80200000
   Verifying Checksum ... OK
## Flattened Device Tree blob at 825f0000
   Booting using the fdt blob at 0x825f0000
   Loading Kernel Image ... OK
   reserving fdt memory region: addr=825f0000 size=7000
   Using Device Tree in place at 825f0000, end 825f9fff

Starting kernel ...

- UART enabled -
- CPU 00000000 booting -
- Xen starting in Hyp mode -
- Zero BSS -
- Setting up control registers -
- Turning on paging -
- Ready -
(XEN) Checking for initrd in /chosen
(XEN) RAM: 0000000080000000 - 00000000feffffff
(XEN) 
(XEN) MODULE[1]: 00000000825f0000 - 00000000825f7000 
(XEN) MODULE[2]: 0000000080300000 - 0000000080800000 
(XEN)  RESVD[0]: 00000000825f0000 - 00000000825f7000
(XEN) 
(XEN) Command line: dom0_mem=128M sync_console console=dtuart dtuart=serial2 noreboot
(XEN) Placing Xen at 0x00000000fee00000-0x00000000ff000000
(XEN) Xen heap: 00000000ee000000-00000000fe000000 (65536 pages)
(XEN) Dom heap: 454656 pages
(XEN) Domain heap initialised
(XEN) Looking for UART console serial2
 Xen 4.5-unstable
(XEN) Xen version 4.5-unstable (arm-linux-gnueabihf-gcc (crosstool-NG linaro-1.13.1-4.7-2013.03-20130313 - Linaro GCC 2013.03) 4.7.3 20130226 (prerelease)) debug=y Thu Apr 24 10:04:50 PDT 2014
(XEN) Latest ChangeSet: Mon Apr 14 15:14:47 2014 +0200 git:c82fbfe-dirty
(XEN) Console output is synchronous.
(XEN) Processor: 412fc0f2: "ARM Limited", variant: 0x2, part 0xc0f, rev 0x2
(XEN) 32-bit Execution:
(XEN)   Processor Features: 00001131:00011011
(XEN)     Instruction Sets: AArch32 Thumb Thumb-2 ThumbEE Jazelle
(XEN)     Extensions: GenericTimer Security
(XEN)   Debug Features: 02010555
(XEN)   Auxiliary Features: 00000000
(XEN)   Memory Model Features: 10201105 20000000 01240000 02102211
(XEN)  ISA Features: 02101110 13112111 21232041 11112131 10011142 00000000
(XEN) Platform: TI OMAP5
(XEN) Set AuxCoreBoot1 to 00000000fee0004c (0020004c)
(XEN) Set AuxCoreBoot0 to 0x20
(XEN) cpu node `/cpus/cpu@1`: #size-cells 1
(XEN) DT missing boot CPU MPIDR[23:0]
(XEN) Using only 1 CPU
(XEN) Generic Timer IRQ: phys=30 hyp=26 virt=27
(XEN) Using generic timer at 6144 KHz
(XEN) GIC initialization:
(XEN)         gic_dist_addr=0000000048211000
(XEN)         gic_cpu_addr=0000000048212000
(XEN)         gic_hyp_addr=0000000048214000
(XEN)         gic_vcpu_addr=0000000048216000
(XEN)         gic_maintenance_irq=25
(XEN) GIC: 192 lines, 2 cpus, secure (IID 0000043b).
(XEN) Using scheduler: SMP Credit Scheduler (credit)
(XEN) Allocated console ring of 16 KiB.
(XEN) VFP implementer 0x41 architecture 4 part 0x30 variant 0xf rev 0x0
(XEN) Brought up 1 CPUs
(XEN) *** LOADING DOMAIN 0 ***
(XEN) Populate P2M 0x88000000->0x90000000 (1:1 mapping for dom0)
(XEN) Loading kernel from boot module 2
(XEN) Loading zImage from 0000000080300000 to 000000008fa00000-000000008feb1cc0
(XEN) Loading dom0 DTB to 0x000000008f800000-0x000000008f806d8e
(XEN) Scrubbing Free RAM: .................done.
(XEN) Initial low memory virq threshold set at 0x4000 pages.
(XEN) Std. Loglevel: All
(XEN) Guest Loglevel: All
(XEN) **********************************************
(XEN) ******* WARNING: CONSOLE OUTPUT IS SYNCHRONOUS
(XEN) ******* This option is intended to aid debugging of Xen by ensuring
(XEN) ******* that all output is synchronously delivered on the serial line.
(XEN) ******* However it can introduce SIGNIFICANT latencies and affect
(XEN) ******* timekeeping. It is NOT recommended for production use!
(XEN) **********************************************
(XEN) 3... 2... 1... 
(XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to switch input to Xen)
(XEN) Freed 264kB init memory.

And the dom0 dump:

(XEN) 'q' pressed -> dumping domain info (now=0x12:5CD00FFD)
(XEN) General information for domain 0:
(XEN)     refcnt=3 dying=0 pause_count=0
(XEN)     nr_pages=32768 xenheap_pages=2 shared_pages=0 paged_pages=0 dirty_cpus={} max_pages=4294967295
(XEN)     handle=00000000-0000-0000-0000-000000000000 vm_assist=00000000
(XEN) GICH_LRs (vcpu 0) mask=0
(XEN)    VCPU_LR[0]=0
(XEN)    VCPU_LR[1]=0
(XEN)    VCPU_LR[2]=0
(XEN)    VCPU_LR[3]=0
(XEN) Rangesets belonging to domain 0:
(XEN)     Interrupts { }
(XEN)     I/O Memory { }
(XEN) NODE affinity for domain 0: [0]
(XEN) VCPU information and callbacks for domain 0:
(XEN)     VCPU0: CPU0 [has=F] poll=0 upcall_pend = 00, upcall_mask = 01 dirty_cpus={} cpu_affinity={0-127}
(XEN)     pause_count=0 pause_flags=0
(XEN)     No periodic timer
(XEN) Notifying guest 0:0 (virq 1, port 0)
(XEN) 'd' pressed -> dumping registers
(XEN) 
(XEN) *** Dumping CPU0 guest state (d0v0): ***
(XEN) ----[ Xen-4.5-unstable  arm32  debug=y  Tainted:    C ]----
(XEN) CPU:    0
(XEN) PC:     0000000c
(XEN) CPSR:   900001d7 MODE:32-bit Guest ABT
(XEN)      R0: 80004000 R1: 00000c12 R2: 80008000 R3: 80004000
(XEN)      R4: 80008000 R5: 00000000 R6: 0000000e R7: ffffffff
(XEN)      R8: 8f800000 R9: 80000000 R10:90000000 R11:10201105 R12:8fa000a8
(XEN) USR: SP: 00000000 LR: 00000000
(XEN) SVC: SP: 00000000 LR: 8fa0036c SPSR:000001d3
(XEN) ABT: SP: 00000000 LR: 0000000c SPSR:900001d7
(XEN) UND: SP: 00000000 LR: 00000000 SPSR:00000000
(XEN) IRQ: SP: 00000000 LR: 00000000 SPSR:00000000
(XEN) FIQ: SP: 00000000 LR: 00000000 SPSR:00000000
(XEN) FIQ: R8: 00000000 R9: 00000000 R10:00000000 R11:00000000 R12:00000000
(XEN) 
(XEN)      SCTLR: 00c50078
(XEN)        TCR: 00000000
(XEN)      TTBR0: 0000000000000000
(XEN)      TTBR1: 0000000000000000
(XEN)       IFAR: 0000000c, IFSR: 00000002
(XEN)       DFAR: 80004000, DFSR: 00000002
(XEN) 
(XEN)   VTCR_EL2: 80003558
(XEN)  VTTBR_EL2: 00010000825fe000
(XEN) 
(XEN)  SCTLR_EL2: 30cd187f
(XEN)    HCR_EL2: 0000000000282435
(XEN)  TTBR0_EL2: 00000000feedf000
(XEN) 
(XEN)    ESR_EL2: 80000005
(XEN)  HPFAR_EL2: 0000000000000000
(XEN)      HDFAR: 80004000
(XEN)      HIFAR: 0000000c
(XEN) 
(XEN) Guest stack trace from sp=0:
(XEN)   Failed to convert stack to physical address
(XEN)

The above output is with development branch xen from Apr14. I get the same output with xen v4.4 too.

Best,
Ashish

-----Original Message-----
From: Julien Grall [mailto:julien.grall@linaro.org] 
Sent: Friday, April 25, 2014 2:17 AM
To: Kapania, Ashish; Ian Campbell
Cc: xen-devel@lists.xen.org
Subject: Re: [Xen-devel] Problems running Xen 4.4 on OMAP5432 evm

Hi Anish,

On 25/04/14 00:05, Kapania, Ashish wrote:
> I apologize for causing any confusion. The xen.lds file I modified is 
> generated and hence not tracked by git. I had forgotten about the change and running "git status"
> was not telling me that I modified the file. Another point I wanted to 
> make was that I always do a "make distclean" before rebuilding xen but 
> the xen.lds file does not get regenerated. I had to manually delete it to force regeneration.
> Maybe that’s a bug ?
>
> I tried the Ctrl-A * 3 + press 'd' sequence and found that dom0 is in ABT mode.
> I tried debugging a little with JTAG (My JTAG setup does not work well 
> in HYP Mode so I cannot debug xen with it) and found that the ABT 
> happens in Linux/arch/arm/boot/compressed/head.S's __setup_mmu code. 
> The exact instruction that causes the abort is a store instruction 
> that is attempting to write a page table entry to 0x80004000.
>
> MMU stage 1 translation is disabled at this point so it looks like the 
> Stage 2 translation is not mapping 0x80004000 address. Is there some 
> step that I am missing that is suppose to tell xen to map the memory 
> used by linux to store page tables ?

Can you post the console output of Xen and DOM0 (if you have some)? I'd like to seems how Xen populates the RAM for DOM0.

Regards,

--
Julien Grall
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Problems running Xen 4.4 on OMAP5432 evm

[Chen Baozi]

Hi Ashish,

On Apr 26, 2014, at 0:56, Kapania, Ashish <akapania@ti.com> wrote:

> Hi Julien,
> 
> Here's the console output I get:
> 
> ## Booting kernel from Legacy Image at e0000000 ...
>   Image Name:   Xen
>   Image Type:   ARM Linux Kernel Image (uncompressed)
>   Data Size:    656156 Bytes = 640.8 KiB
>   Load Address: 80200000
>   Entry Point:  80200000
>   Verifying Checksum ... OK
> ## Flattened Device Tree blob at 825f0000
>   Booting using the fdt blob at 0x825f0000
>   Loading Kernel Image ... OK
>   reserving fdt memory region: addr=825f0000 size=7000
>   Using Device Tree in place at 825f0000, end 825f9fff
> 
> Starting kernel ...
> 
> - UART enabled -
> - CPU 00000000 booting -
> - Xen starting in Hyp mode -
> - Zero BSS -
> - Setting up control registers -
> - Turning on paging -
> - Ready -
> (XEN) Checking for initrd in /chosen
> (XEN) RAM: 0000000080000000 - 00000000feffffff
> (XEN) 
> (XEN) MODULE[1]: 00000000825f0000 - 00000000825f7000 
> (XEN) MODULE[2]: 0000000080300000 - 0000000080800000 
> (XEN)  RESVD[0]: 00000000825f0000 - 00000000825f7000
> (XEN) 
> (XEN) Command line: dom0_mem=128M sync_console console=dtuart dtuart=serial2 noreboot
> (XEN) Placing Xen at 0x00000000fee00000-0x00000000ff000000
> (XEN) Xen heap: 00000000ee000000-00000000fe000000 (65536 pages)
> (XEN) Dom heap: 454656 pages
> (XEN) Domain heap initialised
> (XEN) Looking for UART console serial2
> Xen 4.5-unstable
> (XEN) Xen version 4.5-unstable (arm-linux-gnueabihf-gcc (crosstool-NG linaro-1.13.1-4.7-2013.03-20130313 - Linaro GCC 2013.03) 4.7.3 20130226 (prerelease)) debug=y Thu Apr 24 10:04:50 PDT 2014
> (XEN) Latest ChangeSet: Mon Apr 14 15:14:47 2014 +0200 git:c82fbfe-dirty
> (XEN) Console output is synchronous.
> (XEN) Processor: 412fc0f2: "ARM Limited", variant: 0x2, part 0xc0f, rev 0x2
> (XEN) 32-bit Execution:
> (XEN)   Processor Features: 00001131:00011011
> (XEN)     Instruction Sets: AArch32 Thumb Thumb-2 ThumbEE Jazelle
> (XEN)     Extensions: GenericTimer Security
> (XEN)   Debug Features: 02010555
> (XEN)   Auxiliary Features: 00000000
> (XEN)   Memory Model Features: 10201105 20000000 01240000 02102211
> (XEN)  ISA Features: 02101110 13112111 21232041 11112131 10011142 00000000
> (XEN) Platform: TI OMAP5
> (XEN) Set AuxCoreBoot1 to 00000000fee0004c (0020004c)
> (XEN) Set AuxCoreBoot0 to 0x20
> (XEN) cpu node `/cpus/cpu@1`: #size-cells 1
> (XEN) DT missing boot CPU MPIDR[23:0]
> (XEN) Using only 1 CPU
> (XEN) Generic Timer IRQ: phys=30 hyp=26 virt=27
> (XEN) Using generic timer at 6144 KHz
> (XEN) GIC initialization:
> (XEN)         gic_dist_addr=0000000048211000
> (XEN)         gic_cpu_addr=0000000048212000
> (XEN)         gic_hyp_addr=0000000048214000
> (XEN)         gic_vcpu_addr=0000000048216000
> (XEN)         gic_maintenance_irq=25
> (XEN) GIC: 192 lines, 2 cpus, secure (IID 0000043b).
> (XEN) Using scheduler: SMP Credit Scheduler (credit)
> (XEN) Allocated console ring of 16 KiB.
> (XEN) VFP implementer 0x41 architecture 4 part 0x30 variant 0xf rev 0x0
> (XEN) Brought up 1 CPUs
> (XEN) *** LOADING DOMAIN 0 ***
> (XEN) Populate P2M 0x88000000->0x90000000 (1:1 mapping for dom0)
> (XEN) Loading kernel from boot module 2
> (XEN) Loading zImage from 0000000080300000 to 000000008fa00000-000000008feb1cc0
> (XEN) Loading dom0 DTB to 0x000000008f800000-0x000000008f806d8e
> (XEN) Scrubbing Free RAM: .................done.
> (XEN) Initial low memory virq threshold set at 0x4000 pages.
> (XEN) Std. Loglevel: All
> (XEN) Guest Loglevel: All
> (XEN) **********************************************
> (XEN) ******* WARNING: CONSOLE OUTPUT IS SYNCHRONOUS
> (XEN) ******* This option is intended to aid debugging of Xen by ensuring
> (XEN) ******* that all output is synchronously delivered on the serial line.
> (XEN) ******* However it can introduce SIGNIFICANT latencies and affect
> (XEN) ******* timekeeping. It is NOT recommended for production use!
> (XEN) **********************************************
> (XEN) 3... 2... 1... 
> (XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to switch input to Xen)
> (XEN) Freed 264kB init memory.
> 
> And the dom0 dump:
> 
> (XEN) 'q' pressed -> dumping domain info (now=0x12:5CD00FFD)
> (XEN) General information for domain 0:
> (XEN)     refcnt=3 dying=0 pause_count=0
> (XEN)     nr_pages=32768 xenheap_pages=2 shared_pages=0 paged_pages=0 dirty_cpus={} max_pages=4294967295
> (XEN)     handle=00000000-0000-0000-0000-000000000000 vm_assist=00000000
> (XEN) GICH_LRs (vcpu 0) mask=0
> (XEN)    VCPU_LR[0]=0
> (XEN)    VCPU_LR[1]=0
> (XEN)    VCPU_LR[2]=0
> (XEN)    VCPU_LR[3]=0
> (XEN) Rangesets belonging to domain 0:
> (XEN)     Interrupts { }
> (XEN)     I/O Memory { }
> (XEN) NODE affinity for domain 0: [0]
> (XEN) VCPU information and callbacks for domain 0:
> (XEN)     VCPU0: CPU0 [has=F] poll=0 upcall_pend = 00, upcall_mask = 01 dirty_cpus={} cpu_affinity={0-127}
> (XEN)     pause_count=0 pause_flags=0
> (XEN)     No periodic timer
> (XEN) Notifying guest 0:0 (virq 1, port 0)
> (XEN) 'd' pressed -> dumping registers
> (XEN) 
> (XEN) *** Dumping CPU0 guest state (d0v0): ***
> (XEN) ----[ Xen-4.5-unstable  arm32  debug=y  Tainted:    C ]----
> (XEN) CPU:    0
> (XEN) PC:     0000000c
> (XEN) CPSR:   900001d7 MODE:32-bit Guest ABT
> (XEN)      R0: 80004000 R1: 00000c12 R2: 80008000 R3: 80004000
> (XEN)      R4: 80008000 R5: 00000000 R6: 0000000e R7: ffffffff
> (XEN)      R8: 8f800000 R9: 80000000 R10:90000000 R11:10201105 R12:8fa000a8
> (XEN) USR: SP: 00000000 LR: 00000000
> (XEN) SVC: SP: 00000000 LR: 8fa0036c SPSR:000001d3
> (XEN) ABT: SP: 00000000 LR: 0000000c SPSR:900001d7
> (XEN) UND: SP: 00000000 LR: 00000000 SPSR:00000000
> (XEN) IRQ: SP: 00000000 LR: 00000000 SPSR:00000000
> (XEN) FIQ: SP: 00000000 LR: 00000000 SPSR:00000000
> (XEN) FIQ: R8: 00000000 R9: 00000000 R10:00000000 R11:00000000 R12:00000000
> (XEN) 
> (XEN)      SCTLR: 00c50078
> (XEN)        TCR: 00000000
> (XEN)      TTBR0: 0000000000000000
> (XEN)      TTBR1: 0000000000000000
> (XEN)       IFAR: 0000000c, IFSR: 00000002
> (XEN)       DFAR: 80004000, DFSR: 00000002
> (XEN) 
> (XEN)   VTCR_EL2: 80003558
> (XEN)  VTTBR_EL2: 00010000825fe000
> (XEN) 
> (XEN)  SCTLR_EL2: 30cd187f
> (XEN)    HCR_EL2: 0000000000282435
> (XEN)  TTBR0_EL2: 00000000feedf000
> (XEN) 
> (XEN)    ESR_EL2: 80000005
> (XEN)  HPFAR_EL2: 0000000000000000
> (XEN)      HDFAR: 80004000
> (XEN)      HIFAR: 0000000c
> (XEN) 
> (XEN) Guest stack trace from sp=0:
> (XEN)   Failed to convert stack to physical address
> (XEN)
> 
> The above output is with development branch xen from Apr14. I get the same output with xen v4.4 too.

I just tested the latest xen from git with a 3.11-rc3 dom0 kernel on my omap5432
uevm board. And seems all the problem I got is that dom0 was trying to access
some addresses hardcoded in hwmod that the hypervisor didn’t map according to
the info it got from DT.

I’ll have a test with latest upstream kernel these days and see what would happen.

Cheers,

Baozi

> 
> Best,
> Ashish
> 
> -----Original Message-----
> From: Julien Grall [mailto:julien.grall@linaro.org] 
> Sent: Friday, April 25, 2014 2:17 AM
> To: Kapania, Ashish; Ian Campbell
> Cc: xen-devel@lists.xen.org
> Subject: Re: [Xen-devel] Problems running Xen 4.4 on OMAP5432 evm
> 
> Hi Anish,
> 
> On 25/04/14 00:05, Kapania, Ashish wrote:
>> I apologize for causing any confusion. The xen.lds file I modified is 
>> generated and hence not tracked by git. I had forgotten about the change and running "git status"
>> was not telling me that I modified the file. Another point I wanted to 
>> make was that I always do a "make distclean" before rebuilding xen but 
>> the xen.lds file does not get regenerated. I had to manually delete it to force regeneration.
>> Maybe that’s a bug ?
>> 
>> I tried the Ctrl-A * 3 + press 'd' sequence and found that dom0 is in ABT mode.
>> I tried debugging a little with JTAG (My JTAG setup does not work well 
>> in HYP Mode so I cannot debug xen with it) and found that the ABT 
>> happens in Linux/arch/arm/boot/compressed/head.S's __setup_mmu code. 
>> The exact instruction that causes the abort is a store instruction 
>> that is attempting to write a page table entry to 0x80004000.
>> 
>> MMU stage 1 translation is disabled at this point so it looks like the 
>> Stage 2 translation is not mapping 0x80004000 address. Is there some 
>> step that I am missing that is suppose to tell xen to map the memory 
>> used by linux to store page tables ?
> 
> Can you post the console output of Xen and DOM0 (if you have some)? I'd like to seems how Xen populates the RAM for DOM0.
> 
> Regards,
> 
> --
> Julien Grall
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

Re: Problems running Xen 4.4 on OMAP5432 evm

[Kapania, Ashish]

Hi Baozi,

I managed to resolve the issue of dom0 aborting. The linux kernel I am
using (v3.8.13 that comes with TI's GLSDK) had "zreladdr" set to
0x80008000 which was causing the kernel to load the page table at
0x80004000 (address not mapped by xen stage2 mmu table). To fix
the issue, I enabled "CONFIG_AUTO_ZRELADDR" in the config and
loaded the kernel image to 0xC0800000.This moved the page table
from 0x80004000 to 0xCxxxxxxx address range which is mapped by xen.

Dom0 no longer aborts but the kernel is now stuck in the
__delay_loop function. It never returns. Ian pointed me
to an email from you on the mailing list (http://lists.xen.org/archives/html/xen-devel/2014-04/msg02620.html)
where you recommended to disable UART port in the kernel for OMAP5.
I am not doing that and was wondering if that could be causing the
kernel to hang ? Could you provide more details on what exactly
do I need to change in the kernel to disable UART port ?

I also wanted to ask you another question about how to get
linux printk's on xen console. Is it enough to pass
"console=hvc0"  to linux kernel or do I need to do something
more ?

Thanks for your help.

Best,
Ashish

-----Original Message-----
From: Chen Baozi [mailto:baozich@gmail.com] 
Sent: Saturday, April 26, 2014 8:04 PM
To: Kapania, Ashish
Cc: Julien Grall; Ian Campbell; xen-devel@lists.xen.org
Subject: Re: [Xen-devel] Problems running Xen 4.4 on OMAP5432 evm

Hi Ashish,

On Apr 26, 2014, at 0:56, Kapania, Ashish <akapania@ti.com> wrote:

> Hi Julien,
> 
> Here's the console output I get:
> 
> ## Booting kernel from Legacy Image at e0000000 ...
>   Image Name:   Xen
>   Image Type:   ARM Linux Kernel Image (uncompressed)
>   Data Size:    656156 Bytes = 640.8 KiB
>   Load Address: 80200000
>   Entry Point:  80200000
>   Verifying Checksum ... OK
> ## Flattened Device Tree blob at 825f0000
>   Booting using the fdt blob at 0x825f0000
>   Loading Kernel Image ... OK
>   reserving fdt memory region: addr=825f0000 size=7000
>   Using Device Tree in place at 825f0000, end 825f9fff
> 
> Starting kernel ...
> 
> - UART enabled -
> - CPU 00000000 booting -
> - Xen starting in Hyp mode -
> - Zero BSS -
> - Setting up control registers -
> - Turning on paging -
> - Ready -
> (XEN) Checking for initrd in /chosen
> (XEN) RAM: 0000000080000000 - 00000000feffffff
> (XEN)
> (XEN) MODULE[1]: 00000000825f0000 - 00000000825f7000
> (XEN) MODULE[2]: 0000000080300000 - 0000000080800000
> (XEN)  RESVD[0]: 00000000825f0000 - 00000000825f7000
> (XEN)
> (XEN) Command line: dom0_mem=128M sync_console console=dtuart 
> dtuart=serial2 noreboot
> (XEN) Placing Xen at 0x00000000fee00000-0x00000000ff000000
> (XEN) Xen heap: 00000000ee000000-00000000fe000000 (65536 pages)
> (XEN) Dom heap: 454656 pages
> (XEN) Domain heap initialised
> (XEN) Looking for UART console serial2 Xen 4.5-unstable
> (XEN) Xen version 4.5-unstable (arm-linux-gnueabihf-gcc (crosstool-NG 
> linaro-1.13.1-4.7-2013.03-20130313 - Linaro GCC 2013.03) 4.7.3 
> 20130226 (prerelease)) debug=y Thu Apr 24 10:04:50 PDT 2014
> (XEN) Latest ChangeSet: Mon Apr 14 15:14:47 2014 +0200 
> git:c82fbfe-dirty
> (XEN) Console output is synchronous.
> (XEN) Processor: 412fc0f2: "ARM Limited", variant: 0x2, part 0xc0f, 
> rev 0x2
> (XEN) 32-bit Execution:
> (XEN)   Processor Features: 00001131:00011011
> (XEN)     Instruction Sets: AArch32 Thumb Thumb-2 ThumbEE Jazelle
> (XEN)     Extensions: GenericTimer Security
> (XEN)   Debug Features: 02010555
> (XEN)   Auxiliary Features: 00000000
> (XEN)   Memory Model Features: 10201105 20000000 01240000 02102211
> (XEN)  ISA Features: 02101110 13112111 21232041 11112131 10011142 
> 00000000
> (XEN) Platform: TI OMAP5
> (XEN) Set AuxCoreBoot1 to 00000000fee0004c (0020004c)
> (XEN) Set AuxCoreBoot0 to 0x20
> (XEN) cpu node `/cpus/cpu@1`: #size-cells 1
> (XEN) DT missing boot CPU MPIDR[23:0]
> (XEN) Using only 1 CPU
> (XEN) Generic Timer IRQ: phys=30 hyp=26 virt=27
> (XEN) Using generic timer at 6144 KHz
> (XEN) GIC initialization:
> (XEN)         gic_dist_addr=0000000048211000
> (XEN)         gic_cpu_addr=0000000048212000
> (XEN)         gic_hyp_addr=0000000048214000
> (XEN)         gic_vcpu_addr=0000000048216000
> (XEN)         gic_maintenance_irq=25
> (XEN) GIC: 192 lines, 2 cpus, secure (IID 0000043b).
> (XEN) Using scheduler: SMP Credit Scheduler (credit)
> (XEN) Allocated console ring of 16 KiB.
> (XEN) VFP implementer 0x41 architecture 4 part 0x30 variant 0xf rev 
> 0x0
> (XEN) Brought up 1 CPUs
> (XEN) *** LOADING DOMAIN 0 ***
> (XEN) Populate P2M 0x88000000->0x90000000 (1:1 mapping for dom0)
> (XEN) Loading kernel from boot module 2
> (XEN) Loading zImage from 0000000080300000 to 
> 000000008fa00000-000000008feb1cc0
> (XEN) Loading dom0 DTB to 0x000000008f800000-0x000000008f806d8e
> (XEN) Scrubbing Free RAM: .................done.
> (XEN) Initial low memory virq threshold set at 0x4000 pages.
> (XEN) Std. Loglevel: All
> (XEN) Guest Loglevel: All
> (XEN) **********************************************
> (XEN) ******* WARNING: CONSOLE OUTPUT IS SYNCHRONOUS
> (XEN) ******* This option is intended to aid debugging of Xen by 
> ensuring
> (XEN) ******* that all output is synchronously delivered on the serial line.
> (XEN) ******* However it can introduce SIGNIFICANT latencies and 
> affect
> (XEN) ******* timekeeping. It is NOT recommended for production use!
> (XEN) **********************************************
> (XEN) 3... 2... 1... 
> (XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to switch 
> input to Xen)
> (XEN) Freed 264kB init memory.
> 
> And the dom0 dump:
> 
> (XEN) 'q' pressed -> dumping domain info (now=0x12:5CD00FFD)
> (XEN) General information for domain 0:
> (XEN)     refcnt=3 dying=0 pause_count=0
> (XEN)     nr_pages=32768 xenheap_pages=2 shared_pages=0 paged_pages=0 dirty_cpus={} max_pages=4294967295
> (XEN)     handle=00000000-0000-0000-0000-000000000000 vm_assist=00000000
> (XEN) GICH_LRs (vcpu 0) mask=0
> (XEN)    VCPU_LR[0]=0
> (XEN)    VCPU_LR[1]=0
> (XEN)    VCPU_LR[2]=0
> (XEN)    VCPU_LR[3]=0
> (XEN) Rangesets belonging to domain 0:
> (XEN)     Interrupts { }
> (XEN)     I/O Memory { }
> (XEN) NODE affinity for domain 0: [0]
> (XEN) VCPU information and callbacks for domain 0:
> (XEN)     VCPU0: CPU0 [has=F] poll=0 upcall_pend = 00, upcall_mask = 01 dirty_cpus={} cpu_affinity={0-127}
> (XEN)     pause_count=0 pause_flags=0
> (XEN)     No periodic timer
> (XEN) Notifying guest 0:0 (virq 1, port 0)
> (XEN) 'd' pressed -> dumping registers
> (XEN)
> (XEN) *** Dumping CPU0 guest state (d0v0): ***
> (XEN) ----[ Xen-4.5-unstable  arm32  debug=y  Tainted:    C ]----
> (XEN) CPU:    0
> (XEN) PC:     0000000c
> (XEN) CPSR:   900001d7 MODE:32-bit Guest ABT
> (XEN)      R0: 80004000 R1: 00000c12 R2: 80008000 R3: 80004000
> (XEN)      R4: 80008000 R5: 00000000 R6: 0000000e R7: ffffffff
> (XEN)      R8: 8f800000 R9: 80000000 R10:90000000 R11:10201105 R12:8fa000a8
> (XEN) USR: SP: 00000000 LR: 00000000
> (XEN) SVC: SP: 00000000 LR: 8fa0036c SPSR:000001d3
> (XEN) ABT: SP: 00000000 LR: 0000000c SPSR:900001d7
> (XEN) UND: SP: 00000000 LR: 00000000 SPSR:00000000
> (XEN) IRQ: SP: 00000000 LR: 00000000 SPSR:00000000
> (XEN) FIQ: SP: 00000000 LR: 00000000 SPSR:00000000
> (XEN) FIQ: R8: 00000000 R9: 00000000 R10:00000000 R11:00000000 
> R12:00000000
> (XEN) 
> (XEN)      SCTLR: 00c50078
> (XEN)        TCR: 00000000
> (XEN)      TTBR0: 0000000000000000
> (XEN)      TTBR1: 0000000000000000
> (XEN)       IFAR: 0000000c, IFSR: 00000002
> (XEN)       DFAR: 80004000, DFSR: 00000002
> (XEN) 
> (XEN)   VTCR_EL2: 80003558
> (XEN)  VTTBR_EL2: 00010000825fe000
> (XEN)
> (XEN)  SCTLR_EL2: 30cd187f
> (XEN)    HCR_EL2: 0000000000282435
> (XEN)  TTBR0_EL2: 00000000feedf000
> (XEN) 
> (XEN)    ESR_EL2: 80000005
> (XEN)  HPFAR_EL2: 0000000000000000
> (XEN)      HDFAR: 80004000
> (XEN)      HIFAR: 0000000c
> (XEN)
> (XEN) Guest stack trace from sp=0:
> (XEN)   Failed to convert stack to physical address
> (XEN)
> 
> The above output is with development branch xen from Apr14. I get the same output with xen v4.4 too.

I just tested the latest xen from git with a 3.11-rc3 dom0 kernel on my omap5432 uevm board. And seems all the problem I got is that dom0 was trying to access some addresses hardcoded in hwmod that the hypervisor didn't map according to the info it got from DT.

I'll have a test with latest upstream kernel these days and see what would happen.

Cheers,

Baozi

> 
> Best,
> Ashish
> 
> -----Original Message-----
> From: Julien Grall [mailto:julien.grall@linaro.org]
> Sent: Friday, April 25, 2014 2:17 AM
> To: Kapania, Ashish; Ian Campbell
> Cc: xen-devel@lists.xen.org
> Subject: Re: [Xen-devel] Problems running Xen 4.4 on OMAP5432 evm
> 
> Hi Anish,
> 
> On 25/04/14 00:05, Kapania, Ashish wrote:
>> I apologize for causing any confusion. The xen.lds file I modified is 
>> generated and hence not tracked by git. I had forgotten about the change and running "git status"
>> was not telling me that I modified the file. Another point I wanted 
>> to make was that I always do a "make distclean" before rebuilding xen 
>> but the xen.lds file does not get regenerated. I had to manually delete it to force regeneration.
>> Maybe that's a bug ?
>> 
>> I tried the Ctrl-A * 3 + press 'd' sequence and found that dom0 is in ABT mode.
>> I tried debugging a little with JTAG (My JTAG setup does not work 
>> well in HYP Mode so I cannot debug xen with it) and found that the 
>> ABT happens in Linux/arch/arm/boot/compressed/head.S's __setup_mmu code.
>> The exact instruction that causes the abort is a store instruction 
>> that is attempting to write a page table entry to 0x80004000.
>> 
>> MMU stage 1 translation is disabled at this point so it looks like 
>> the Stage 2 translation is not mapping 0x80004000 address. Is there 
>> some step that I am missing that is suppose to tell xen to map the 
>> memory used by linux to store page tables ?
> 
> Can you post the console output of Xen and DOM0 (if you have some)? I'd like to seems how Xen populates the RAM for DOM0.
> 
> Regards,
> 
> --
> Julien Grall
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

Re: Problems running Xen 4.4 on OMAP5432 evm

[Chen Baozi]

On Apr 27, 2014, at 14:30, Kapania, Ashish <akapania@ti.com> wrote:

> Hi Baozi,
> 
> I managed to resolve the issue of dom0 aborting. The linux kernel I am
> using (v3.8.13 that comes with TI's GLSDK) had "zreladdr" set to
> 0x80008000 which was causing the kernel to load the page table at
> 0x80004000 (address not mapped by xen stage2 mmu table). To fix
> the issue, I enabled "CONFIG_AUTO_ZRELADDR" in the config and
> loaded the kernel image to 0xC0800000.This moved the page table
> from 0x80004000 to 0xCxxxxxxx address range which is mapped by xen.

I test with the following u-boot cmd:

# fatload mmc 0:1 0x825f0000 <dtb-name>
# fatload mmc 0:1 0x90000000 <xen uImage name>
# fatload mmc 0:1 0xa0000000 <dom0 zImage name>
# bootm 0x90000000 - 0x825f0000

Note that the address zImage loaded to should be the same as the address
written in chosen node of DT. And xen image should be 2MiB aligned.

> 
> Dom0 no longer aborts but the kernel is now stuck in the
> __delay_loop function. It never returns. Ian pointed me
> to an email from you on the mailing list (http://lists.xen.org/archives/html/xen-devel/2014-04/msg02620.html)
> where you recommended to disable UART port in the kernel for OMAP5.
> I am not doing that and was wondering if that could be causing the
> kernel to hang ? Could you provide more details on what exactly
> do I need to change in the kernel to disable UART port ?

The reason to disable UART3 hwmod is that the hypervisor won’t map
the corresponding io memory address for dom0 because the UART is taken
by itself. If the dom0 kernel doesn’t use hwmod (only use FDT), everything
would work fine. Because Xen removes the corresponding UART node
in the DT passed to dom0. However, since hwmod structure is used in
OMAP kernel, the dom0 would still try to write the UART’s IO address
space hard-coded in the hwmod structure, which could lead kernel panic.
One of the ugly hack is to remove corresponding	structure in
omap54xx_hwmod_ocp_ifs (arch/arm/mach-omap2/omap_hwmod_54xx_data.c).
I should say I’m still not very familiar with omap kernel and have no
idea whether there would be more elegant ways to solve this problem.

> 
> I also wanted to ask you another question about how to get
> linux printk's on xen console. Is it enough to pass
> "console=hvc0"  to linux kernel or do I need to do something
> more ?

To make the upstream kernel output info to xen-hvc, I did the following hacks:

diff --git a/drivers/tty/hvc/hvc_xen.c b/drivers/tty/hvc/hvc_xen.c
index 2dc2831..931c72a 100644
--- a/drivers/tty/hvc/hvc_xen.c
+++ b/drivers/tty/hvc/hvc_xen.c
@@ -630,7 +630,6 @@ void xen_raw_console_write(const char *str)
        ssize_t len = strlen(str);
        int rc = 0;

-       if (xen_domain()) {
                rc = dom0_write_console(0, str, len);
 #ifdef CONFIG_X86
                if (rc == -ENOSYS && xen_hvm_domain())
@@ -642,7 +641,6 @@ outb_print:
                for (i = 0; i < len; i++)
                        outb(str[i], 0xe9);
 #endif
-       }
 }

 void xen_raw_printk(const char *fmt, ...)
diff --git a/kernel/printk/printk.c b/kernel/printk/printk.c
index a45b509..907922b 100644
--- a/kernel/printk/printk.c
+++ b/kernel/printk/printk.c
@@ -45,6 +45,7 @@
 #include <linux/poll.h>
 #include <linux/irq_work.h>
 #include <linux/utsname.h>
+#include <xen/hvc-console.h>

 #include <asm/uaccess.h>

@@ -1571,6 +1572,8 @@ asmlinkage int vprintk_emit(int facility, int level,
                }
        }

+       xen_raw_console_write(text);
+
        if (level == -1)
                level = default_message_loglevel;

Hopefully this is helpful.

Baozi

> 
> Thanks for your help.
> 
> Best,
> Ashish
> 
> -----Original Message-----
> From: Chen Baozi [mailto:baozich@gmail.com] 
> Sent: Saturday, April 26, 2014 8:04 PM
> To: Kapania, Ashish
> Cc: Julien Grall; Ian Campbell; xen-devel@lists.xen.org
> Subject: Re: [Xen-devel] Problems running Xen 4.4 on OMAP5432 evm
> 
> Hi Ashish,
> 
> On Apr 26, 2014, at 0:56, Kapania, Ashish <akapania@ti.com> wrote:
> 
>> Hi Julien,
>> 
>> Here's the console output I get:
>> 
>> ## Booting kernel from Legacy Image at e0000000 ...
>>  Image Name:   Xen
>>  Image Type:   ARM Linux Kernel Image (uncompressed)
>>  Data Size:    656156 Bytes = 640.8 KiB
>>  Load Address: 80200000
>>  Entry Point:  80200000
>>  Verifying Checksum ... OK
>> ## Flattened Device Tree blob at 825f0000
>>  Booting using the fdt blob at 0x825f0000
>>  Loading Kernel Image ... OK
>>  reserving fdt memory region: addr=825f0000 size=7000
>>  Using Device Tree in place at 825f0000, end 825f9fff
>> 
>> Starting kernel ...
>> 
>> - UART enabled -
>> - CPU 00000000 booting -
>> - Xen starting in Hyp mode -
>> - Zero BSS -
>> - Setting up control registers -
>> - Turning on paging -
>> - Ready -
>> (XEN) Checking for initrd in /chosen
>> (XEN) RAM: 0000000080000000 - 00000000feffffff
>> (XEN)
>> (XEN) MODULE[1]: 00000000825f0000 - 00000000825f7000
>> (XEN) MODULE[2]: 0000000080300000 - 0000000080800000
>> (XEN)  RESVD[0]: 00000000825f0000 - 00000000825f7000
>> (XEN)
>> (XEN) Command line: dom0_mem=128M sync_console console=dtuart 
>> dtuart=serial2 noreboot
>> (XEN) Placing Xen at 0x00000000fee00000-0x00000000ff000000
>> (XEN) Xen heap: 00000000ee000000-00000000fe000000 (65536 pages)
>> (XEN) Dom heap: 454656 pages
>> (XEN) Domain heap initialised
>> (XEN) Looking for UART console serial2 Xen 4.5-unstable
>> (XEN) Xen version 4.5-unstable (arm-linux-gnueabihf-gcc (crosstool-NG 
>> linaro-1.13.1-4.7-2013.03-20130313 - Linaro GCC 2013.03) 4.7.3 
>> 20130226 (prerelease)) debug=y Thu Apr 24 10:04:50 PDT 2014
>> (XEN) Latest ChangeSet: Mon Apr 14 15:14:47 2014 +0200 
>> git:c82fbfe-dirty
>> (XEN) Console output is synchronous.
>> (XEN) Processor: 412fc0f2: "ARM Limited", variant: 0x2, part 0xc0f, 
>> rev 0x2
>> (XEN) 32-bit Execution:
>> (XEN)   Processor Features: 00001131:00011011
>> (XEN)     Instruction Sets: AArch32 Thumb Thumb-2 ThumbEE Jazelle
>> (XEN)     Extensions: GenericTimer Security
>> (XEN)   Debug Features: 02010555
>> (XEN)   Auxiliary Features: 00000000
>> (XEN)   Memory Model Features: 10201105 20000000 01240000 02102211
>> (XEN)  ISA Features: 02101110 13112111 21232041 11112131 10011142 
>> 00000000
>> (XEN) Platform: TI OMAP5
>> (XEN) Set AuxCoreBoot1 to 00000000fee0004c (0020004c)
>> (XEN) Set AuxCoreBoot0 to 0x20
>> (XEN) cpu node `/cpus/cpu@1`: #size-cells 1
>> (XEN) DT missing boot CPU MPIDR[23:0]
>> (XEN) Using only 1 CPU
>> (XEN) Generic Timer IRQ: phys=30 hyp=26 virt=27
>> (XEN) Using generic timer at 6144 KHz
>> (XEN) GIC initialization:
>> (XEN)         gic_dist_addr=0000000048211000
>> (XEN)         gic_cpu_addr=0000000048212000
>> (XEN)         gic_hyp_addr=0000000048214000
>> (XEN)         gic_vcpu_addr=0000000048216000
>> (XEN)         gic_maintenance_irq=25
>> (XEN) GIC: 192 lines, 2 cpus, secure (IID 0000043b).
>> (XEN) Using scheduler: SMP Credit Scheduler (credit)
>> (XEN) Allocated console ring of 16 KiB.
>> (XEN) VFP implementer 0x41 architecture 4 part 0x30 variant 0xf rev 
>> 0x0
>> (XEN) Brought up 1 CPUs
>> (XEN) *** LOADING DOMAIN 0 ***
>> (XEN) Populate P2M 0x88000000->0x90000000 (1:1 mapping for dom0)
>> (XEN) Loading kernel from boot module 2
>> (XEN) Loading zImage from 0000000080300000 to 
>> 000000008fa00000-000000008feb1cc0
>> (XEN) Loading dom0 DTB to 0x000000008f800000-0x000000008f806d8e
>> (XEN) Scrubbing Free RAM: .................done.
>> (XEN) Initial low memory virq threshold set at 0x4000 pages.
>> (XEN) Std. Loglevel: All
>> (XEN) Guest Loglevel: All
>> (XEN) **********************************************
>> (XEN) ******* WARNING: CONSOLE OUTPUT IS SYNCHRONOUS
>> (XEN) ******* This option is intended to aid debugging of Xen by 
>> ensuring
>> (XEN) ******* that all output is synchronously delivered on the serial line.
>> (XEN) ******* However it can introduce SIGNIFICANT latencies and 
>> affect
>> (XEN) ******* timekeeping. It is NOT recommended for production use!
>> (XEN) **********************************************
>> (XEN) 3... 2... 1... 
>> (XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to switch 
>> input to Xen)
>> (XEN) Freed 264kB init memory.
>> 
>> And the dom0 dump:
>> 
>> (XEN) 'q' pressed -> dumping domain info (now=0x12:5CD00FFD)
>> (XEN) General information for domain 0:
>> (XEN)     refcnt=3 dying=0 pause_count=0
>> (XEN)     nr_pages=32768 xenheap_pages=2 shared_pages=0 paged_pages=0 dirty_cpus={} max_pages=4294967295
>> (XEN)     handle=00000000-0000-0000-0000-000000000000 vm_assist=00000000
>> (XEN) GICH_LRs (vcpu 0) mask=0
>> (XEN)    VCPU_LR[0]=0
>> (XEN)    VCPU_LR[1]=0
>> (XEN)    VCPU_LR[2]=0
>> (XEN)    VCPU_LR[3]=0
>> (XEN) Rangesets belonging to domain 0:
>> (XEN)     Interrupts { }
>> (XEN)     I/O Memory { }
>> (XEN) NODE affinity for domain 0: [0]
>> (XEN) VCPU information and callbacks for domain 0:
>> (XEN)     VCPU0: CPU0 [has=F] poll=0 upcall_pend = 00, upcall_mask = 01 dirty_cpus={} cpu_affinity={0-127}
>> (XEN)     pause_count=0 pause_flags=0
>> (XEN)     No periodic timer
>> (XEN) Notifying guest 0:0 (virq 1, port 0)
>> (XEN) 'd' pressed -> dumping registers
>> (XEN)
>> (XEN) *** Dumping CPU0 guest state (d0v0): ***
>> (XEN) ----[ Xen-4.5-unstable  arm32  debug=y  Tainted:    C ]----
>> (XEN) CPU:    0
>> (XEN) PC:     0000000c
>> (XEN) CPSR:   900001d7 MODE:32-bit Guest ABT
>> (XEN)      R0: 80004000 R1: 00000c12 R2: 80008000 R3: 80004000
>> (XEN)      R4: 80008000 R5: 00000000 R6: 0000000e R7: ffffffff
>> (XEN)      R8: 8f800000 R9: 80000000 R10:90000000 R11:10201105 R12:8fa000a8
>> (XEN) USR: SP: 00000000 LR: 00000000
>> (XEN) SVC: SP: 00000000 LR: 8fa0036c SPSR:000001d3
>> (XEN) ABT: SP: 00000000 LR: 0000000c SPSR:900001d7
>> (XEN) UND: SP: 00000000 LR: 00000000 SPSR:00000000
>> (XEN) IRQ: SP: 00000000 LR: 00000000 SPSR:00000000
>> (XEN) FIQ: SP: 00000000 LR: 00000000 SPSR:00000000
>> (XEN) FIQ: R8: 00000000 R9: 00000000 R10:00000000 R11:00000000 
>> R12:00000000
>> (XEN) 
>> (XEN)      SCTLR: 00c50078
>> (XEN)        TCR: 00000000
>> (XEN)      TTBR0: 0000000000000000
>> (XEN)      TTBR1: 0000000000000000
>> (XEN)       IFAR: 0000000c, IFSR: 00000002
>> (XEN)       DFAR: 80004000, DFSR: 00000002
>> (XEN) 
>> (XEN)   VTCR_EL2: 80003558
>> (XEN)  VTTBR_EL2: 00010000825fe000
>> (XEN)
>> (XEN)  SCTLR_EL2: 30cd187f
>> (XEN)    HCR_EL2: 0000000000282435
>> (XEN)  TTBR0_EL2: 00000000feedf000
>> (XEN) 
>> (XEN)    ESR_EL2: 80000005
>> (XEN)  HPFAR_EL2: 0000000000000000
>> (XEN)      HDFAR: 80004000
>> (XEN)      HIFAR: 0000000c
>> (XEN)
>> (XEN) Guest stack trace from sp=0:
>> (XEN)   Failed to convert stack to physical address
>> (XEN)
>> 
>> The above output is with development branch xen from Apr14. I get the same output with xen v4.4 too.
> 
> I just tested the latest xen from git with a 3.11-rc3 dom0 kernel on my omap5432 uevm board. And seems all the problem I got is that dom0 was trying to access some addresses hardcoded in hwmod that the hypervisor didn't map according to the info it got from DT.
> 
> I'll have a test with latest upstream kernel these days and see what would happen.
> 
> Cheers,
> 
> Baozi
> 
>> 
>> Best,
>> Ashish
>> 
>> -----Original Message-----
>> From: Julien Grall [mailto:julien.grall@linaro.org]
>> Sent: Friday, April 25, 2014 2:17 AM
>> To: Kapania, Ashish; Ian Campbell
>> Cc: xen-devel@lists.xen.org
>> Subject: Re: [Xen-devel] Problems running Xen 4.4 on OMAP5432 evm
>> 
>> Hi Anish,
>> 
>> On 25/04/14 00:05, Kapania, Ashish wrote:
>>> I apologize for causing any confusion. The xen.lds file I modified is 
>>> generated and hence not tracked by git. I had forgotten about the change and running "git status"
>>> was not telling me that I modified the file. Another point I wanted 
>>> to make was that I always do a "make distclean" before rebuilding xen 
>>> but the xen.lds file does not get regenerated. I had to manually delete it to force regeneration.
>>> Maybe that's a bug ?
>>> 
>>> I tried the Ctrl-A * 3 + press 'd' sequence and found that dom0 is in ABT mode.
>>> I tried debugging a little with JTAG (My JTAG setup does not work 
>>> well in HYP Mode so I cannot debug xen with it) and found that the 
>>> ABT happens in Linux/arch/arm/boot/compressed/head.S's __setup_mmu code.
>>> The exact instruction that causes the abort is a store instruction 
>>> that is attempting to write a page table entry to 0x80004000.
>>> 
>>> MMU stage 1 translation is disabled at this point so it looks like 
>>> the Stage 2 translation is not mapping 0x80004000 address. Is there 
>>> some step that I am missing that is suppose to tell xen to map the 
>>> memory used by linux to store page tables ?
>> 
>> Can you post the console output of Xen and DOM0 (if you have some)? I'd like to seems how Xen populates the RAM for DOM0.
>> 
>> Regards,
>> 
>> --
>> Julien Grall
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@lists.xen.org
>> http://lists.xen.org/xen-devel
> 

Re: Problems running Xen 4.4 on OMAP5432 evm

[Kapania, Ashish]

> -----Original Message-----
> From: Chen Baozi [mailto:baozich@gmail.com]
> Sent: Monday, April 28, 2014 5:16 AM
> To: Kapania, Ashish
> Cc: Julien Grall; Ian Campbell; xen-devel@lists.xen.org
> Subject: Re: [Xen-devel] Problems running Xen 4.4 on OMAP5432 evm
> 
> 
> On Apr 27, 2014, at 14:30, Kapania, Ashish <akapania@ti.com> wrote:
> 
> > Hi Baozi,
> >
> > I managed to resolve the issue of dom0 aborting. The linux kernel I
> am
> > using (v3.8.13 that comes with TI's GLSDK) had "zreladdr" set to
> > 0x80008000 which was causing the kernel to load the page table at
> > 0x80004000 (address not mapped by xen stage2 mmu table). To fix the
> > issue, I enabled "CONFIG_AUTO_ZRELADDR" in the config and loaded the
> > kernel image to 0xC0800000.This moved the page table from 0x80004000
> > to 0xCxxxxxxx address range which is mapped by xen.
> 
> I test with the following u-boot cmd:
> 
> # fatload mmc 0:1 0x825f0000 <dtb-name>
> # fatload mmc 0:1 0x90000000 <xen uImage name> # fatload mmc 0:1
> 0xa0000000 <dom0 zImage name> # bootm 0x90000000 - 0x825f0000
> 
> Note that the address zImage loaded to should be the same as the
> address written in chosen node of DT. And xen image should be 2MiB
> aligned.
> 
> >
> > Dom0 no longer aborts but the kernel is now stuck in the __delay_loop
> > function. It never returns. Ian pointed me to an email from you on
> the
> > mailing list
> > (http://lists.xen.org/archives/html/xen-devel/2014-04/msg02620.html)
> > where you recommended to disable UART port in the kernel for OMAP5.
> > I am not doing that and was wondering if that could be causing the
> > kernel to hang ? Could you provide more details on what exactly do I
> > need to change in the kernel to disable UART port ?
> 
> The reason to disable UART3 hwmod is that the hypervisor won't map the
> corresponding io memory address for dom0 because the UART is taken by
> itself. If the dom0 kernel doesn't use hwmod (only use FDT), everything
> would work fine. Because Xen removes the corresponding UART node in the
> DT passed to dom0. However, since hwmod structure is used in OMAP
> kernel, the dom0 would still try to write the UART's IO address space
> hard-coded in the hwmod structure, which could lead kernel panic.
> One of the ugly hack is to remove corresponding	structure in
> omap54xx_hwmod_ocp_ifs (arch/arm/mach-omap2/omap_hwmod_54xx_data.c).
> I should say I'm still not very familiar with omap kernel and have no
> idea whether there would be more elegant ways to solve this problem.
> 
> >
> > I also wanted to ask you another question about how to get linux
> > printk's on xen console. Is it enough to pass "console=hvc0"  to
> linux
> > kernel or do I need to do something more ?
> 
> To make the upstream kernel output info to xen-hvc, I did the following
> hacks:
> 
> diff --git a/drivers/tty/hvc/hvc_xen.c b/drivers/tty/hvc/hvc_xen.c
> index 2dc2831..931c72a 100644
> --- a/drivers/tty/hvc/hvc_xen.c
> +++ b/drivers/tty/hvc/hvc_xen.c
> @@ -630,7 +630,6 @@ void xen_raw_console_write(const char *str)
>         ssize_t len = strlen(str);
>         int rc = 0;
> 
> -       if (xen_domain()) {
>                 rc = dom0_write_console(0, str, len);  #ifdef
> CONFIG_X86
>                 if (rc == -ENOSYS && xen_hvm_domain()) @@ -642,7 +641,6
> @@ outb_print:
>                 for (i = 0; i < len; i++)
>                         outb(str[i], 0xe9);  #endif
> -       }
>  }
> 
>  void xen_raw_printk(const char *fmt, ...) diff --git
> a/kernel/printk/printk.c b/kernel/printk/printk.c index
> a45b509..907922b 100644
> --- a/kernel/printk/printk.c
> +++ b/kernel/printk/printk.c
> @@ -45,6 +45,7 @@
>  #include <linux/poll.h>
>  #include <linux/irq_work.h>
>  #include <linux/utsname.h>
> +#include <xen/hvc-console.h>
> 
>  #include <asm/uaccess.h>
> 
> @@ -1571,6 +1572,8 @@ asmlinkage int vprintk_emit(int facility, int
> level,
>                 }
>         }
> 
> +       xen_raw_console_write(text);
> +
>         if (level == -1)
>                 level = default_message_loglevel;
> 
> Hopefully this is helpful.
> 
> Baozi
> 
> >
> > Thanks for your help.
> >
> > Best,
> > Ashish
> >
> > -----Original Message-----
> > From: Chen Baozi [mailto:baozich@gmail.com]
> > Sent: Saturday, April 26, 2014 8:04 PM
> > To: Kapania, Ashish
> > Cc: Julien Grall; Ian Campbell; xen-devel@lists.xen.org
> > Subject: Re: [Xen-devel] Problems running Xen 4.4 on OMAP5432 evm
> >
> > Hi Ashish,
> >
> > On Apr 26, 2014, at 0:56, Kapania, Ashish <akapania@ti.com> wrote:
> >
> >> Hi Julien,
> >>
> >> Here's the console output I get:
> >>
> >> ## Booting kernel from Legacy Image at e0000000 ...
> >>  Image Name:   Xen
> >>  Image Type:   ARM Linux Kernel Image (uncompressed)
> >>  Data Size:    656156 Bytes = 640.8 KiB
> >>  Load Address: 80200000
> >>  Entry Point:  80200000
> >>  Verifying Checksum ... OK
> >> ## Flattened Device Tree blob at 825f0000  Booting using the fdt
> blob
> >> at 0x825f0000  Loading Kernel Image ... OK  reserving fdt memory
> >> region: addr=825f0000 size=7000  Using Device Tree in place at
> >> 825f0000, end 825f9fff
> >>
> >> Starting kernel ...
> >>
> >> - UART enabled -
> >> - CPU 00000000 booting -
> >> - Xen starting in Hyp mode -
> >> - Zero BSS -
> >> - Setting up control registers -
> >> - Turning on paging -
> >> - Ready -
> >> (XEN) Checking for initrd in /chosen
> >> (XEN) RAM: 0000000080000000 - 00000000feffffff
> >> (XEN)
> >> (XEN) MODULE[1]: 00000000825f0000 - 00000000825f7000
> >> (XEN) MODULE[2]: 0000000080300000 - 0000000080800000
> >> (XEN)  RESVD[0]: 00000000825f0000 - 00000000825f7000
> >> (XEN)
> >> (XEN) Command line: dom0_mem=128M sync_console console=dtuart
> >> dtuart=serial2 noreboot
> >> (XEN) Placing Xen at 0x00000000fee00000-0x00000000ff000000
> >> (XEN) Xen heap: 00000000ee000000-00000000fe000000 (65536 pages)
> >> (XEN) Dom heap: 454656 pages
> >> (XEN) Domain heap initialised
> >> (XEN) Looking for UART console serial2 Xen 4.5-unstable
> >> (XEN) Xen version 4.5-unstable (arm-linux-gnueabihf-gcc (crosstool-
> NG
> >> linaro-1.13.1-4.7-2013.03-20130313 - Linaro GCC 2013.03) 4.7.3
> >> 20130226 (prerelease)) debug=y Thu Apr 24 10:04:50 PDT 2014
> >> (XEN) Latest ChangeSet: Mon Apr 14 15:14:47 2014 +0200
> >> git:c82fbfe-dirty
> >> (XEN) Console output is synchronous.
> >> (XEN) Processor: 412fc0f2: "ARM Limited", variant: 0x2, part 0xc0f,
> >> rev 0x2
> >> (XEN) 32-bit Execution:
> >> (XEN)   Processor Features: 00001131:00011011
> >> (XEN)     Instruction Sets: AArch32 Thumb Thumb-2 ThumbEE Jazelle
> >> (XEN)     Extensions: GenericTimer Security
> >> (XEN)   Debug Features: 02010555
> >> (XEN)   Auxiliary Features: 00000000
> >> (XEN)   Memory Model Features: 10201105 20000000 01240000 02102211
> >> (XEN)  ISA Features: 02101110 13112111 21232041 11112131 10011142
> >> 00000000
> >> (XEN) Platform: TI OMAP5
> >> (XEN) Set AuxCoreBoot1 to 00000000fee0004c (0020004c)
> >> (XEN) Set AuxCoreBoot0 to 0x20
> >> (XEN) cpu node `/cpus/cpu@1`: #size-cells 1
> >> (XEN) DT missing boot CPU MPIDR[23:0]
> >> (XEN) Using only 1 CPU
> >> (XEN) Generic Timer IRQ: phys=30 hyp=26 virt=27
> >> (XEN) Using generic timer at 6144 KHz
> >> (XEN) GIC initialization:
> >> (XEN)         gic_dist_addr=0000000048211000
> >> (XEN)         gic_cpu_addr=0000000048212000
> >> (XEN)         gic_hyp_addr=0000000048214000
> >> (XEN)         gic_vcpu_addr=0000000048216000
> >> (XEN)         gic_maintenance_irq=25
> >> (XEN) GIC: 192 lines, 2 cpus, secure (IID 0000043b).
> >> (XEN) Using scheduler: SMP Credit Scheduler (credit)
> >> (XEN) Allocated console ring of 16 KiB.
> >> (XEN) VFP implementer 0x41 architecture 4 part 0x30 variant 0xf rev
> >> 0x0
> >> (XEN) Brought up 1 CPUs
> >> (XEN) *** LOADING DOMAIN 0 ***
> >> (XEN) Populate P2M 0x88000000->0x90000000 (1:1 mapping for dom0)
> >> (XEN) Loading kernel from boot module 2
> >> (XEN) Loading zImage from 0000000080300000 to
> >> 000000008fa00000-000000008feb1cc0
> >> (XEN) Loading dom0 DTB to 0x000000008f800000-0x000000008f806d8e
> >> (XEN) Scrubbing Free RAM: .................done.
> >> (XEN) Initial low memory virq threshold set at 0x4000 pages.
> >> (XEN) Std. Loglevel: All
> >> (XEN) Guest Loglevel: All
> >> (XEN) **********************************************
> >> (XEN) ******* WARNING: CONSOLE OUTPUT IS SYNCHRONOUS
> >> (XEN) ******* This option is intended to aid debugging of Xen by
> >> ensuring
> >> (XEN) ******* that all output is synchronously delivered on the
> serial line.
> >> (XEN) ******* However it can introduce SIGNIFICANT latencies and
> >> affect
> >> (XEN) ******* timekeeping. It is NOT recommended for production use!
> >> (XEN) **********************************************
> >> (XEN) 3... 2... 1...
> >> (XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to switch
> >> input to Xen)
> >> (XEN) Freed 264kB init memory.
> >>
> >> And the dom0 dump:
> >>
> >> (XEN) 'q' pressed -> dumping domain info (now=0x12:5CD00FFD)
> >> (XEN) General information for domain 0:
> >> (XEN)     refcnt=3 dying=0 pause_count=0
> >> (XEN)     nr_pages=32768 xenheap_pages=2 shared_pages=0
> paged_pages=0 dirty_cpus={} max_pages=4294967295
> >> (XEN)     handle=00000000-0000-0000-0000-000000000000
> vm_assist=00000000
> >> (XEN) GICH_LRs (vcpu 0) mask=0
> >> (XEN)    VCPU_LR[0]=0
> >> (XEN)    VCPU_LR[1]=0
> >> (XEN)    VCPU_LR[2]=0
> >> (XEN)    VCPU_LR[3]=0
> >> (XEN) Rangesets belonging to domain 0:
> >> (XEN)     Interrupts { }
> >> (XEN)     I/O Memory { }
> >> (XEN) NODE affinity for domain 0: [0]
> >> (XEN) VCPU information and callbacks for domain 0:
> >> (XEN)     VCPU0: CPU0 [has=F] poll=0 upcall_pend = 00, upcall_mask =
> 01 dirty_cpus={} cpu_affinity={0-127}
> >> (XEN)     pause_count=0 pause_flags=0
> >> (XEN)     No periodic timer
> >> (XEN) Notifying guest 0:0 (virq 1, port 0)
> >> (XEN) 'd' pressed -> dumping registers
> >> (XEN)
> >> (XEN) *** Dumping CPU0 guest state (d0v0): ***
> >> (XEN) ----[ Xen-4.5-unstable  arm32  debug=y  Tainted:    C ]----
> >> (XEN) CPU:    0
> >> (XEN) PC:     0000000c
> >> (XEN) CPSR:   900001d7 MODE:32-bit Guest ABT
> >> (XEN)      R0: 80004000 R1: 00000c12 R2: 80008000 R3: 80004000
> >> (XEN)      R4: 80008000 R5: 00000000 R6: 0000000e R7: ffffffff
> >> (XEN)      R8: 8f800000 R9: 80000000 R10:90000000 R11:10201105
> R12:8fa000a8
> >> (XEN) USR: SP: 00000000 LR: 00000000
> >> (XEN) SVC: SP: 00000000 LR: 8fa0036c SPSR:000001d3
> >> (XEN) ABT: SP: 00000000 LR: 0000000c SPSR:900001d7
> >> (XEN) UND: SP: 00000000 LR: 00000000 SPSR:00000000
> >> (XEN) IRQ: SP: 00000000 LR: 00000000 SPSR:00000000
> >> (XEN) FIQ: SP: 00000000 LR: 00000000 SPSR:00000000
> >> (XEN) FIQ: R8: 00000000 R9: 00000000 R10:00000000 R11:00000000
> >> R12:00000000
> >> (XEN)
> >> (XEN)      SCTLR: 00c50078
> >> (XEN)        TCR: 00000000
> >> (XEN)      TTBR0: 0000000000000000
> >> (XEN)      TTBR1: 0000000000000000
> >> (XEN)       IFAR: 0000000c, IFSR: 00000002
> >> (XEN)       DFAR: 80004000, DFSR: 00000002
> >> (XEN)
> >> (XEN)   VTCR_EL2: 80003558
> >> (XEN)  VTTBR_EL2: 00010000825fe000
> >> (XEN)
> >> (XEN)  SCTLR_EL2: 30cd187f
> >> (XEN)    HCR_EL2: 0000000000282435
> >> (XEN)  TTBR0_EL2: 00000000feedf000
> >> (XEN)
> >> (XEN)    ESR_EL2: 80000005
> >> (XEN)  HPFAR_EL2: 0000000000000000
> >> (XEN)      HDFAR: 80004000
> >> (XEN)      HIFAR: 0000000c
> >> (XEN)
> >> (XEN) Guest stack trace from sp=0:
> >> (XEN)   Failed to convert stack to physical address
> >> (XEN)
> >>
> >> The above output is with development branch xen from Apr14. I get
> the same output with xen v4.4 too.
> >
> > I just tested the latest xen from git with a 3.11-rc3 dom0 kernel on
> my omap5432 uevm board. And seems all the problem I got is that dom0
> was trying to access some addresses hardcoded in hwmod that the
> hypervisor didn't map according to the info it got from DT.
> >
> > I'll have a test with latest upstream kernel these days and see what
> would happen.
> >
> > Cheers,
> >
> > Baozi
> >
> >>
> >> Best,
> >> Ashish
> >>
> >> -----Original Message-----
> >> From: Julien Grall [mailto:julien.grall@linaro.org]
> >> Sent: Friday, April 25, 2014 2:17 AM
> >> To: Kapania, Ashish; Ian Campbell
> >> Cc: xen-devel@lists.xen.org
> >> Subject: Re: [Xen-devel] Problems running Xen 4.4 on OMAP5432 evm
> >>
> >> Hi Anish,
> >>
> >> On 25/04/14 00:05, Kapania, Ashish wrote:
> >>> I apologize for causing any confusion. The xen.lds file I modified
> >>> is generated and hence not tracked by git. I had forgotten about
> the change and running "git status"
> >>> was not telling me that I modified the file. Another point I wanted
> >>> to make was that I always do a "make distclean" before rebuilding
> >>> xen but the xen.lds file does not get regenerated. I had to
> manually delete it to force regeneration.
> >>> Maybe that's a bug ?
> >>>
> >>> I tried the Ctrl-A * 3 + press 'd' sequence and found that dom0 is
> in ABT mode.
> >>> I tried debugging a little with JTAG (My JTAG setup does not work
> >>> well in HYP Mode so I cannot debug xen with it) and found that the
> >>> ABT happens in Linux/arch/arm/boot/compressed/head.S's __setup_mmu
> code.
> >>> The exact instruction that causes the abort is a store instruction
> >>> that is attempting to write a page table entry to 0x80004000.
> >>>
> >>> MMU stage 1 translation is disabled at this point so it looks like
> >>> the Stage 2 translation is not mapping 0x80004000 address. Is there
> >>> some step that I am missing that is suppose to tell xen to map the
> >>> memory used by linux to store page tables ?
> >>
> >> Can you post the console output of Xen and DOM0 (if you have some)?
> I'd like to seems how Xen populates the RAM for DOM0.
> >>
> >> Regards,
> >>
> >> --
> >> Julien Grall
> >> _______________________________________________
> >> Xen-devel mailing list
> >> Xen-devel@lists.xen.org
> >> http://lists.xen.org/xen-devel
> >

Thanks Baozi, the printk patch helped. I am now able to see the kernel printk's on the xen console.

The printk's show that the kernel panics while executing _clktrctrl_write(). I believe clktrctrl is attempting to read from PRCM_MPU registers and that address space is not mapped. Did you see a similar problem ?

The Kernel log I get:
(XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to switch input to Xen)
(XEN) Freed 264kB init memory.
Booting Linux on physical CPU 0x0
Linux version 3.11.0-rc3-dirty (a0273866@a0273866-VirtualBox) (gcc version 4.7.3 20130226 (prerelease) (crosstool-NG linaro-1.13.1-4.7-2013.03-20130313 - Linaro GCC 2013.03) ) #4 SMP Mon Apr 28 14:30:48 PD
T 2014
CPU: ARMv7 Processor [412fc0f2] revision 2 (ARMv7), cr=10c53c7d
CPU: PIPT / VIPT nonaliasing data cache, PIPT instruction cache
Machine: Generic OMAP5 (Flattened Device Tree), model: TI OMAP5 uEVM board
cma: CMA: reserved 16 MiB at ae800000
Memory policy: ECC disabled, Data cache writealloc
On node 0 totalpages: 32512
free_area_init_node: node 0, pgdat c0833d00, node_mem_map c0da0000
  Normal zone: 256 pages used for memmap
  Normal zone: 0 pages reserved
  Normal zone: 32512 pages, LIFO batch:7
psci: probing function IDs from device-tree
OMAP5432 ES2.0
Unhandled fault: terminal exception (0x002) at 0xfc009300
Internal error: : 2 [#1] SMP ARM
Modules linked in:
CPU: 0 PID: 0 Comm: swapper Not tainted 3.11.0-rc3-dirty #4
task: c07a99c8 ti: c079e000 task.ti: c079e000
PC is at _clktrctrl_write+0x1c/0x40
LR is at omap4_clkdm_wakeup+0x18/0x20
pc : [<c0035a88>]    lr : [<c0035ed8>]    psr: a00001d3
sp : c079ff30  ip : c0839cd4  fp : c07ab060
r10: c07ab060  r9 : c0801d90  r8 : c07a6ddc
r7 : 00000002  r6 : c07b3020  r5 : c0839d1c  r4 : c07b4a00
r3 : fc009300  r2 : 00001300  r1 : fc008000  r0 : 00000002
Flags: NzCv  IRQs off  FIQs off  Mode SVC_32  ISA ARM  Segment kernel
Control: 10c53c7d  Table: a800406a  DAC: 00000017
Process swapper (pid: 0, stack limit = 0xc079e240)
Stack: (0xc079ff30 to 0xc07a0000)
ff20:                                     c0035ec0 c0038a74 c07a99c8 c07b4a00
ff40: c0839d1c c0038aac 0000000f c07b4a00 c0839d1c c0038c6c fc002000 c07a6490
ff60: c078008c c07463d4 08000000 c074107c c079ff78 c079ff80 00000000 00000000
ff80: 00000000 00000000 00000000 00000000 00000000 00000001 00000000 00000000
ffa0: c0665f54 00000001 00000000 c07822ec c07ab188 a800406a 412fc0f2 00000000
ffc0: 00000000 c073d58c 00000000 00000000 00000000 00000000 00000000 c07822f0
ffe0: 00000000 10c53c7d c07a650c c07822ec c07ab188 a8008074 00000000 00000000
[<c0035a88>] (_clktrctrl_write+0x1c/0x40) from [<c0035ed8>] (omap4_clkdm_wakeup+0x18/0x20)
[<c0035ed8>] (omap4_clkdm_wakeup+0x18/0x20) from [<c0038a74>] (clkdm_wakeup_nolock+0x48/0x68)
[<c0038a74>] (clkdm_wakeup_nolock+0x48/0x68) from [<c0038aac>] (clkdm_wakeup+0x18/0x2c)
[<c0038aac>] (clkdm_wakeup+0x18/0x2c) from [<c0038c6c>] (clkdm_complete_init+0xb4/0xf8)
[<c0038c6c>] (clkdm_complete_init+0xb4/0xf8) from [<c07463d4>] (omap5_init_early+0x58/0x80)
[<c07463d4>] (omap5_init_early+0x58/0x80) from [<c074107c>] (setup_arch+0x8c8/0x9b8)
[<c074107c>] (setup_arch+0x8c8/0x9b8) from [<c073d58c>] (start_kernel+0x7c/0x320)
[<c073d58c>] (start_kernel+0x7c/0x320) from [<a8008074>] (0xa8008074)
Code: e59fc028 e79c1101 e3510000 0a000006 (e0833002) 
---[ end trace 1b75b31a2719ed1c ]---
Kernel panic - not syncing: Attempted to kill the idle task!

> > I just tested the latest xen from git with a 3.11-rc3 dom0 kernel on
> my omap5432 uevm board. And seems all the problem I got is that dom0
> was trying to access some addresses hardcoded in hwmod that the
> hypervisor didn't map according to the info it got from DT.

How did you resolve the mapping issue ? If you have a patch to resolve this could you share it ?

Thanks,
Ashish

Re: Problems running Xen 4.4 on OMAP5432 evm

[Chen Baozi]

On Apr 29, 2014, at 10:11, Kapania, Ashish <akapania@ti.com> wrote:

> Thanks Baozi, the printk patch helped. I am now able to see the kernel printk's on the xen console.
> 
> The printk's show that the kernel panics while executing _clktrctrl_write(). I believe clktrctrl is attempting to read from PRCM_MPU registers and that address space is not mapped. Did you see a similar problem ?
> 
> The Kernel log I get:
> (XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to switch input to Xen)
> (XEN) Freed 264kB init memory.
> Booting Linux on physical CPU 0x0
> Linux version 3.11.0-rc3-dirty (a0273866@a0273866-VirtualBox) (gcc version 4.7.3 20130226 (prerelease) (crosstool-NG linaro-1.13.1-4.7-2013.03-20130313 - Linaro GCC 2013.03) ) #4 SMP Mon Apr 28 14:30:48 PD
> T 2014
> CPU: ARMv7 Processor [412fc0f2] revision 2 (ARMv7), cr=10c53c7d
> CPU: PIPT / VIPT nonaliasing data cache, PIPT instruction cache
> Machine: Generic OMAP5 (Flattened Device Tree), model: TI OMAP5 uEVM board
> cma: CMA: reserved 16 MiB at ae800000
> Memory policy: ECC disabled, Data cache writealloc
> On node 0 totalpages: 32512
> free_area_init_node: node 0, pgdat c0833d00, node_mem_map c0da0000
>  Normal zone: 256 pages used for memmap
>  Normal zone: 0 pages reserved
>  Normal zone: 32512 pages, LIFO batch:7
> psci: probing function IDs from device-tree
> OMAP5432 ES2.0
> Unhandled fault: terminal exception (0x002) at 0xfc009300
> Internal error: : 2 [#1] SMP ARM
> Modules linked in:
> CPU: 0 PID: 0 Comm: swapper Not tainted 3.11.0-rc3-dirty #4
> task: c07a99c8 ti: c079e000 task.ti: c079e000
> PC is at _clktrctrl_write+0x1c/0x40
> LR is at omap4_clkdm_wakeup+0x18/0x20
> pc : [<c0035a88>]    lr : [<c0035ed8>]    psr: a00001d3
> sp : c079ff30  ip : c0839cd4  fp : c07ab060
> r10: c07ab060  r9 : c0801d90  r8 : c07a6ddc
> r7 : 00000002  r6 : c07b3020  r5 : c0839d1c  r4 : c07b4a00
> r3 : fc009300  r2 : 00001300  r1 : fc008000  r0 : 00000002
> Flags: NzCv  IRQs off  FIQs off  Mode SVC_32  ISA ARM  Segment kernel
> Control: 10c53c7d  Table: a800406a  DAC: 00000017
> Process swapper (pid: 0, stack limit = 0xc079e240)
> Stack: (0xc079ff30 to 0xc07a0000)
> ff20:                                     c0035ec0 c0038a74 c07a99c8 c07b4a00
> ff40: c0839d1c c0038aac 0000000f c07b4a00 c0839d1c c0038c6c fc002000 c07a6490
> ff60: c078008c c07463d4 08000000 c074107c c079ff78 c079ff80 00000000 00000000
> ff80: 00000000 00000000 00000000 00000000 00000000 00000001 00000000 00000000
> ffa0: c0665f54 00000001 00000000 c07822ec c07ab188 a800406a 412fc0f2 00000000
> ffc0: 00000000 c073d58c 00000000 00000000 00000000 00000000 00000000 c07822f0
> ffe0: 00000000 10c53c7d c07a650c c07822ec c07ab188 a8008074 00000000 00000000
> [<c0035a88>] (_clktrctrl_write+0x1c/0x40) from [<c0035ed8>] (omap4_clkdm_wakeup+0x18/0x20)
> [<c0035ed8>] (omap4_clkdm_wakeup+0x18/0x20) from [<c0038a74>] (clkdm_wakeup_nolock+0x48/0x68)
> [<c0038a74>] (clkdm_wakeup_nolock+0x48/0x68) from [<c0038aac>] (clkdm_wakeup+0x18/0x2c)
> [<c0038aac>] (clkdm_wakeup+0x18/0x2c) from [<c0038c6c>] (clkdm_complete_init+0xb4/0xf8)
> [<c0038c6c>] (clkdm_complete_init+0xb4/0xf8) from [<c07463d4>] (omap5_init_early+0x58/0x80)
> [<c07463d4>] (omap5_init_early+0x58/0x80) from [<c074107c>] (setup_arch+0x8c8/0x9b8)
> [<c074107c>] (setup_arch+0x8c8/0x9b8) from [<c073d58c>] (start_kernel+0x7c/0x320)
> [<c073d58c>] (start_kernel+0x7c/0x320) from [<a8008074>] (0xa8008074)
> Code: e59fc028 e79c1101 e3510000 0a000006 (e0833002) 
> ---[ end trace 1b75b31a2719ed1c ]---
> Kernel panic - not syncing: Attempted to kill the idle task!

looks like the hwmod & DT issues.

I post my recently boot steps on omap5 with upstream kernel on the wiki.
see: http://wiki.xenproject.org/wiki/Xen_ARM_with_Virtualization_Extensions/OMAP5432_uEVM

Baozi

Re: Problems running Xen 4.4 on OMAP5432 evm

[Chen Baozi]

On Apr 28, 2014, at 20:16, Chen Baozi <baozich@gmail.com> wrote:

> 
> On Apr 27, 2014, at 14:30, Kapania, Ashish <akapania@ti.com> wrote:
> 
>> Hi Baozi,
>> 
>> I managed to resolve the issue of dom0 aborting. The linux kernel I am
>> using (v3.8.13 that comes with TI's GLSDK) had "zreladdr" set to
>> 0x80008000 which was causing the kernel to load the page table at
>> 0x80004000 (address not mapped by xen stage2 mmu table). To fix
>> the issue, I enabled "CONFIG_AUTO_ZRELADDR" in the config and
>> loaded the kernel image to 0xC0800000.This moved the page table
>> from 0x80004000 to 0xCxxxxxxx address range which is mapped by xen.
> 
> I test with the following u-boot cmd:
> 
> # fatload mmc 0:1 0x825f0000 <dtb-name>
> # fatload mmc 0:1 0x90000000 <xen uImage name>
> # fatload mmc 0:1 0xa0000000 <dom0 zImage name>
> # bootm 0x90000000 - 0x825f0000
> 
> Note that the address zImage loaded to should be the same as the address
> written in chosen node of DT. And xen image should be 2MiB aligned.
> 
>> 
>> Dom0 no longer aborts but the kernel is now stuck in the
>> __delay_loop function. It never returns. Ian pointed me
>> to an email from you on the mailing list (http://lists.xen.org/archives/html/xen-devel/2014-04/msg02620.html)
>> where you recommended to disable UART port in the kernel for OMAP5.
>> I am not doing that and was wondering if that could be causing the
>> kernel to hang ? Could you provide more details on what exactly
>> do I need to change in the kernel to disable UART port ?
> 
> The reason to disable UART3 hwmod is that the hypervisor won’t map
> the corresponding io memory address for dom0 because the UART is taken
> by itself. If the dom0 kernel doesn’t use hwmod (only use FDT), everything
> would work fine. Because Xen removes the corresponding UART node
> in the DT passed to dom0. However, since hwmod structure is used in
> OMAP kernel, the dom0 would still try to write the UART’s IO address
> space hard-coded in the hwmod structure, which could lead kernel panic.
> One of the ugly hack is to remove corresponding	structure in
> omap54xx_hwmod_ocp_ifs (arch/arm/mach-omap2/omap_hwmod_54xx_data.c).
> I should say I’m still not very familiar with omap kernel and have no
> idea whether there would be more elegant ways to solve this problem.

Just found that in the upstream kernel, there is no such issue. So there is
no need to disable hwmod manually in the kernel source if using the latest
upstream kernel :)

> 
>> 
>> I also wanted to ask you another question about how to get
>> linux printk's on xen console. Is it enough to pass
>> "console=hvc0"  to linux kernel or do I need to do something
>> more ?
> 
> To make the upstream kernel output info to xen-hvc, I did the following hacks:
> 
> diff --git a/drivers/tty/hvc/hvc_xen.c b/drivers/tty/hvc/hvc_xen.c
> index 2dc2831..931c72a 100644
> --- a/drivers/tty/hvc/hvc_xen.c
> +++ b/drivers/tty/hvc/hvc_xen.c
> @@ -630,7 +630,6 @@ void xen_raw_console_write(const char *str)
>        ssize_t len = strlen(str);
>        int rc = 0;
> 
> -       if (xen_domain()) {
>                rc = dom0_write_console(0, str, len);
> #ifdef CONFIG_X86
>                if (rc == -ENOSYS && xen_hvm_domain())
> @@ -642,7 +641,6 @@ outb_print:
>                for (i = 0; i < len; i++)
>                        outb(str[i], 0xe9);
> #endif
> -       }
> }
> 
> void xen_raw_printk(const char *fmt, ...)
> diff --git a/kernel/printk/printk.c b/kernel/printk/printk.c
> index a45b509..907922b 100644
> --- a/kernel/printk/printk.c
> +++ b/kernel/printk/printk.c
> @@ -45,6 +45,7 @@
> #include <linux/poll.h>
> #include <linux/irq_work.h>
> #include <linux/utsname.h>
> +#include <xen/hvc-console.h>
> 
> #include <asm/uaccess.h>
> 
> @@ -1571,6 +1572,8 @@ asmlinkage int vprintk_emit(int facility, int level,
>                }
>        }
> 
> +       xen_raw_console_write(text);
> +
>        if (level == -1)
>                level = default_message_loglevel;
> 
> Hopefully this is helpful.
> 
> Baozi
> 
>> 
>> Thanks for your help.
>> 
>> Best,
>> Ashish
>> 
>> -----Original Message-----
>> From: Chen Baozi [mailto:baozich@gmail.com] 
>> Sent: Saturday, April 26, 2014 8:04 PM
>> To: Kapania, Ashish
>> Cc: Julien Grall; Ian Campbell; xen-devel@lists.xen.org
>> Subject: Re: [Xen-devel] Problems running Xen 4.4 on OMAP5432 evm
>> 
>> Hi Ashish,
>> 
>> On Apr 26, 2014, at 0:56, Kapania, Ashish <akapania@ti.com> wrote:
>> 
>>> Hi Julien,
>>> 
>>> Here's the console output I get:
>>> 
>>> ## Booting kernel from Legacy Image at e0000000 ...
>>> Image Name:   Xen
>>> Image Type:   ARM Linux Kernel Image (uncompressed)
>>> Data Size:    656156 Bytes = 640.8 KiB
>>> Load Address: 80200000
>>> Entry Point:  80200000
>>> Verifying Checksum ... OK
>>> ## Flattened Device Tree blob at 825f0000
>>> Booting using the fdt blob at 0x825f0000
>>> Loading Kernel Image ... OK
>>> reserving fdt memory region: addr=825f0000 size=7000
>>> Using Device Tree in place at 825f0000, end 825f9fff
>>> 
>>> Starting kernel ...
>>> 
>>> - UART enabled -
>>> - CPU 00000000 booting -
>>> - Xen starting in Hyp mode -
>>> - Zero BSS -
>>> - Setting up control registers -
>>> - Turning on paging -
>>> - Ready -
>>> (XEN) Checking for initrd in /chosen
>>> (XEN) RAM: 0000000080000000 - 00000000feffffff
>>> (XEN)
>>> (XEN) MODULE[1]: 00000000825f0000 - 00000000825f7000
>>> (XEN) MODULE[2]: 0000000080300000 - 0000000080800000
>>> (XEN)  RESVD[0]: 00000000825f0000 - 00000000825f7000
>>> (XEN)
>>> (XEN) Command line: dom0_mem=128M sync_console console=dtuart 
>>> dtuart=serial2 noreboot
>>> (XEN) Placing Xen at 0x00000000fee00000-0x00000000ff000000
>>> (XEN) Xen heap: 00000000ee000000-00000000fe000000 (65536 pages)
>>> (XEN) Dom heap: 454656 pages
>>> (XEN) Domain heap initialised
>>> (XEN) Looking for UART console serial2 Xen 4.5-unstable
>>> (XEN) Xen version 4.5-unstable (arm-linux-gnueabihf-gcc (crosstool-NG 
>>> linaro-1.13.1-4.7-2013.03-20130313 - Linaro GCC 2013.03) 4.7.3 
>>> 20130226 (prerelease)) debug=y Thu Apr 24 10:04:50 PDT 2014
>>> (XEN) Latest ChangeSet: Mon Apr 14 15:14:47 2014 +0200 
>>> git:c82fbfe-dirty
>>> (XEN) Console output is synchronous.
>>> (XEN) Processor: 412fc0f2: "ARM Limited", variant: 0x2, part 0xc0f, 
>>> rev 0x2
>>> (XEN) 32-bit Execution:
>>> (XEN)   Processor Features: 00001131:00011011
>>> (XEN)     Instruction Sets: AArch32 Thumb Thumb-2 ThumbEE Jazelle
>>> (XEN)     Extensions: GenericTimer Security
>>> (XEN)   Debug Features: 02010555
>>> (XEN)   Auxiliary Features: 00000000
>>> (XEN)   Memory Model Features: 10201105 20000000 01240000 02102211
>>> (XEN)  ISA Features: 02101110 13112111 21232041 11112131 10011142 
>>> 00000000
>>> (XEN) Platform: TI OMAP5
>>> (XEN) Set AuxCoreBoot1 to 00000000fee0004c (0020004c)
>>> (XEN) Set AuxCoreBoot0 to 0x20
>>> (XEN) cpu node `/cpus/cpu@1`: #size-cells 1
>>> (XEN) DT missing boot CPU MPIDR[23:0]
>>> (XEN) Using only 1 CPU
>>> (XEN) Generic Timer IRQ: phys=30 hyp=26 virt=27
>>> (XEN) Using generic timer at 6144 KHz
>>> (XEN) GIC initialization:
>>> (XEN)         gic_dist_addr=0000000048211000
>>> (XEN)         gic_cpu_addr=0000000048212000
>>> (XEN)         gic_hyp_addr=0000000048214000
>>> (XEN)         gic_vcpu_addr=0000000048216000
>>> (XEN)         gic_maintenance_irq=25
>>> (XEN) GIC: 192 lines, 2 cpus, secure (IID 0000043b).
>>> (XEN) Using scheduler: SMP Credit Scheduler (credit)
>>> (XEN) Allocated console ring of 16 KiB.
>>> (XEN) VFP implementer 0x41 architecture 4 part 0x30 variant 0xf rev 
>>> 0x0
>>> (XEN) Brought up 1 CPUs
>>> (XEN) *** LOADING DOMAIN 0 ***
>>> (XEN) Populate P2M 0x88000000->0x90000000 (1:1 mapping for dom0)
>>> (XEN) Loading kernel from boot module 2
>>> (XEN) Loading zImage from 0000000080300000 to 
>>> 000000008fa00000-000000008feb1cc0
>>> (XEN) Loading dom0 DTB to 0x000000008f800000-0x000000008f806d8e
>>> (XEN) Scrubbing Free RAM: .................done.
>>> (XEN) Initial low memory virq threshold set at 0x4000 pages.
>>> (XEN) Std. Loglevel: All
>>> (XEN) Guest Loglevel: All
>>> (XEN) **********************************************
>>> (XEN) ******* WARNING: CONSOLE OUTPUT IS SYNCHRONOUS
>>> (XEN) ******* This option is intended to aid debugging of Xen by 
>>> ensuring
>>> (XEN) ******* that all output is synchronously delivered on the serial line.
>>> (XEN) ******* However it can introduce SIGNIFICANT latencies and 
>>> affect
>>> (XEN) ******* timekeeping. It is NOT recommended for production use!
>>> (XEN) **********************************************
>>> (XEN) 3... 2... 1... 
>>> (XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to switch 
>>> input to Xen)
>>> (XEN) Freed 264kB init memory.
>>> 
>>> And the dom0 dump:
>>> 
>>> (XEN) 'q' pressed -> dumping domain info (now=0x12:5CD00FFD)
>>> (XEN) General information for domain 0:
>>> (XEN)     refcnt=3 dying=0 pause_count=0
>>> (XEN)     nr_pages=32768 xenheap_pages=2 shared_pages=0 paged_pages=0 dirty_cpus={} max_pages=4294967295
>>> (XEN)     handle=00000000-0000-0000-0000-000000000000 vm_assist=00000000
>>> (XEN) GICH_LRs (vcpu 0) mask=0
>>> (XEN)    VCPU_LR[0]=0
>>> (XEN)    VCPU_LR[1]=0
>>> (XEN)    VCPU_LR[2]=0
>>> (XEN)    VCPU_LR[3]=0
>>> (XEN) Rangesets belonging to domain 0:
>>> (XEN)     Interrupts { }
>>> (XEN)     I/O Memory { }
>>> (XEN) NODE affinity for domain 0: [0]
>>> (XEN) VCPU information and callbacks for domain 0:
>>> (XEN)     VCPU0: CPU0 [has=F] poll=0 upcall_pend = 00, upcall_mask = 01 dirty_cpus={} cpu_affinity={0-127}
>>> (XEN)     pause_count=0 pause_flags=0
>>> (XEN)     No periodic timer
>>> (XEN) Notifying guest 0:0 (virq 1, port 0)
>>> (XEN) 'd' pressed -> dumping registers
>>> (XEN)
>>> (XEN) *** Dumping CPU0 guest state (d0v0): ***
>>> (XEN) ----[ Xen-4.5-unstable  arm32  debug=y  Tainted:    C ]----
>>> (XEN) CPU:    0
>>> (XEN) PC:     0000000c
>>> (XEN) CPSR:   900001d7 MODE:32-bit Guest ABT
>>> (XEN)      R0: 80004000 R1: 00000c12 R2: 80008000 R3: 80004000
>>> (XEN)      R4: 80008000 R5: 00000000 R6: 0000000e R7: ffffffff
>>> (XEN)      R8: 8f800000 R9: 80000000 R10:90000000 R11:10201105 R12:8fa000a8
>>> (XEN) USR: SP: 00000000 LR: 00000000
>>> (XEN) SVC: SP: 00000000 LR: 8fa0036c SPSR:000001d3
>>> (XEN) ABT: SP: 00000000 LR: 0000000c SPSR:900001d7
>>> (XEN) UND: SP: 00000000 LR: 00000000 SPSR:00000000
>>> (XEN) IRQ: SP: 00000000 LR: 00000000 SPSR:00000000
>>> (XEN) FIQ: SP: 00000000 LR: 00000000 SPSR:00000000
>>> (XEN) FIQ: R8: 00000000 R9: 00000000 R10:00000000 R11:00000000 
>>> R12:00000000
>>> (XEN) 
>>> (XEN)      SCTLR: 00c50078
>>> (XEN)        TCR: 00000000
>>> (XEN)      TTBR0: 0000000000000000
>>> (XEN)      TTBR1: 0000000000000000
>>> (XEN)       IFAR: 0000000c, IFSR: 00000002
>>> (XEN)       DFAR: 80004000, DFSR: 00000002
>>> (XEN) 
>>> (XEN)   VTCR_EL2: 80003558
>>> (XEN)  VTTBR_EL2: 00010000825fe000
>>> (XEN)
>>> (XEN)  SCTLR_EL2: 30cd187f
>>> (XEN)    HCR_EL2: 0000000000282435
>>> (XEN)  TTBR0_EL2: 00000000feedf000
>>> (XEN) 
>>> (XEN)    ESR_EL2: 80000005
>>> (XEN)  HPFAR_EL2: 0000000000000000
>>> (XEN)      HDFAR: 80004000
>>> (XEN)      HIFAR: 0000000c
>>> (XEN)
>>> (XEN) Guest stack trace from sp=0:
>>> (XEN)   Failed to convert stack to physical address
>>> (XEN)
>>> 
>>> The above output is with development branch xen from Apr14. I get the same output with xen v4.4 too.
>> 
>> I just tested the latest xen from git with a 3.11-rc3 dom0 kernel on my omap5432 uevm board. And seems all the problem I got is that dom0 was trying to access some addresses hardcoded in hwmod that the hypervisor didn't map according to the info it got from DT.
>> 
>> I'll have a test with latest upstream kernel these days and see what would happen.
>> 
>> Cheers,
>> 
>> Baozi
>> 
>>> 
>>> Best,
>>> Ashish
>>> 
>>> -----Original Message-----
>>> From: Julien Grall [mailto:julien.grall@linaro.org]
>>> Sent: Friday, April 25, 2014 2:17 AM
>>> To: Kapania, Ashish; Ian Campbell
>>> Cc: xen-devel@lists.xen.org
>>> Subject: Re: [Xen-devel] Problems running Xen 4.4 on OMAP5432 evm
>>> 
>>> Hi Anish,
>>> 
>>> On 25/04/14 00:05, Kapania, Ashish wrote:
>>>> I apologize for causing any confusion. The xen.lds file I modified is 
>>>> generated and hence not tracked by git. I had forgotten about the change and running "git status"
>>>> was not telling me that I modified the file. Another point I wanted 
>>>> to make was that I always do a "make distclean" before rebuilding xen 
>>>> but the xen.lds file does not get regenerated. I had to manually delete it to force regeneration.
>>>> Maybe that's a bug ?
>>>> 
>>>> I tried the Ctrl-A * 3 + press 'd' sequence and found that dom0 is in ABT mode.
>>>> I tried debugging a little with JTAG (My JTAG setup does not work 
>>>> well in HYP Mode so I cannot debug xen with it) and found that the 
>>>> ABT happens in Linux/arch/arm/boot/compressed/head.S's __setup_mmu code.
>>>> The exact instruction that causes the abort is a store instruction 
>>>> that is attempting to write a page table entry to 0x80004000.
>>>> 
>>>> MMU stage 1 translation is disabled at this point so it looks like 
>>>> the Stage 2 translation is not mapping 0x80004000 address. Is there 
>>>> some step that I am missing that is suppose to tell xen to map the 
>>>> memory used by linux to store page tables ?
>>> 
>>> Can you post the console output of Xen and DOM0 (if you have some)? I'd like to seems how Xen populates the RAM for DOM0.
>>> 
>>> Regards,
>>> 
>>> --
>>> Julien Grall
>>> _______________________________________________
>>> Xen-devel mailing list
>>> Xen-devel@lists.xen.org
>>> http://lists.xen.org/xen-devel
>> 
> 

Re: Problems running Xen 4.4 on OMAP5432 evm

[Kapania, Ashish]

> -----Original Message-----
> From: Chen Baozi [mailto:baozich@gmail.com]
> Sent: Tuesday, April 29, 2014 8:19 PM
> To: Kapania, Ashish
> Cc: Julien Grall; Ian Campbell; xen-devel@lists.xen.org
> Subject: Re: [Xen-devel] Problems running Xen 4.4 on OMAP5432 evm
> 
> 
> On Apr 28, 2014, at 20:16, Chen Baozi <baozich@gmail.com> wrote:
> 
> >
> > On Apr 27, 2014, at 14:30, Kapania, Ashish <akapania@ti.com> wrote:
> >
> >> Hi Baozi,
> >>
> >> I managed to resolve the issue of dom0 aborting. The linux kernel I
> >> am using (v3.8.13 that comes with TI's GLSDK) had "zreladdr" set to
> >> 0x80008000 which was causing the kernel to load the page table at
> >> 0x80004000 (address not mapped by xen stage2 mmu table). To fix the
> >> issue, I enabled "CONFIG_AUTO_ZRELADDR" in the config and loaded the
> >> kernel image to 0xC0800000.This moved the page table from 0x80004000
> >> to 0xCxxxxxxx address range which is mapped by xen.
> >
> > I test with the following u-boot cmd:
> >
> > # fatload mmc 0:1 0x825f0000 <dtb-name> # fatload mmc 0:1 0x90000000
> > <xen uImage name> # fatload mmc 0:1 0xa0000000 <dom0 zImage name> #
> > bootm 0x90000000 - 0x825f0000
> >
> > Note that the address zImage loaded to should be the same as the
> > address written in chosen node of DT. And xen image should be 2MiB
> aligned.
> >
> >>
> >> Dom0 no longer aborts but the kernel is now stuck in the
> __delay_loop
> >> function. It never returns. Ian pointed me to an email from you on
> >> the mailing list
> >> (http://lists.xen.org/archives/html/xen-devel/2014-04/msg02620.html)
> >> where you recommended to disable UART port in the kernel for OMAP5.
> >> I am not doing that and was wondering if that could be causing the
> >> kernel to hang ? Could you provide more details on what exactly do I
> >> need to change in the kernel to disable UART port ?
> >
> > The reason to disable UART3 hwmod is that the hypervisor won't map
> the
> > corresponding io memory address for dom0 because the UART is taken by
> > itself. If the dom0 kernel doesn't use hwmod (only use FDT),
> > everything would work fine. Because Xen removes the corresponding
> UART
> > node in the DT passed to dom0. However, since hwmod structure is used
> > in OMAP kernel, the dom0 would still try to write the UART's IO
> > address space hard-coded in the hwmod structure, which could lead
> kernel panic.
> > One of the ugly hack is to remove corresponding	structure in
> > omap54xx_hwmod_ocp_ifs (arch/arm/mach-omap2/omap_hwmod_54xx_data.c).
> > I should say I'm still not very familiar with omap kernel and have no
> > idea whether there would be more elegant ways to solve this problem.
> 
> Just found that in the upstream kernel, there is no such issue. So
> there is no need to disable hwmod manually in the kernel source if
> using the latest upstream kernel :)
> 

Nice, so I only need to make the DTS changes (to mmc, mcbsp, etc.) as per your wiki ? I will try using the latest upstream kernel. Thanks for all the help.

Best,
Ashish

Re: Problems running Xen 4.4 on OMAP5432 evm

[Kapania, Ashish]

> -----Original Message-----
> From: Kapania, Ashish
> Sent: Tuesday, April 29, 2014 9:43 PM
> To: 'Chen Baozi'
> Cc: Julien Grall; Ian Campbell; xen-devel@lists.xen.org
> Subject: RE: [Xen-devel] Problems running Xen 4.4 on OMAP5432 evm
> 
> > -----Original Message-----
> > From: Chen Baozi [mailto:baozich@gmail.com]
> > Sent: Tuesday, April 29, 2014 8:19 PM
> > To: Kapania, Ashish
> > Cc: Julien Grall; Ian Campbell; xen-devel@lists.xen.org
> > Subject: Re: [Xen-devel] Problems running Xen 4.4 on OMAP5432 evm
> >
> >
> > On Apr 28, 2014, at 20:16, Chen Baozi <baozich@gmail.com> wrote:
> >
> > >
> > > On Apr 27, 2014, at 14:30, Kapania, Ashish <akapania@ti.com> wrote:
> > >
> > >> Hi Baozi,
> > >>
> > >> I managed to resolve the issue of dom0 aborting. The linux kernel
> I
> > >> am using (v3.8.13 that comes with TI's GLSDK) had "zreladdr" set
> to
> > >> 0x80008000 which was causing the kernel to load the page table at
> > >> 0x80004000 (address not mapped by xen stage2 mmu table). To fix
> the
> > >> issue, I enabled "CONFIG_AUTO_ZRELADDR" in the config and loaded
> > >> the kernel image to 0xC0800000.This moved the page table from
> > >> 0x80004000 to 0xCxxxxxxx address range which is mapped by xen.
> > >
> > > I test with the following u-boot cmd:
> > >
> > > # fatload mmc 0:1 0x825f0000 <dtb-name> # fatload mmc 0:1
> 0x90000000
> > > <xen uImage name> # fatload mmc 0:1 0xa0000000 <dom0 zImage name> #
> > > bootm 0x90000000 - 0x825f0000
> > >
> > > Note that the address zImage loaded to should be the same as the
> > > address written in chosen node of DT. And xen image should be 2MiB
> > aligned.
> > >
> > >>
> > >> Dom0 no longer aborts but the kernel is now stuck in the
> > __delay_loop
> > >> function. It never returns. Ian pointed me to an email from you on
> > >> the mailing list
> > >> (http://lists.xen.org/archives/html/xen-devel/2014-
> 04/msg02620.html
> > >> ) where you recommended to disable UART port in the kernel for
> > >> OMAP5.
> > >> I am not doing that and was wondering if that could be causing the
> > >> kernel to hang ? Could you provide more details on what exactly do
> > >> I need to change in the kernel to disable UART port ?
> > >
> > > The reason to disable UART3 hwmod is that the hypervisor won't map
> > the
> > > corresponding io memory address for dom0 because the UART is taken
> > > by itself. If the dom0 kernel doesn't use hwmod (only use FDT),
> > > everything would work fine. Because Xen removes the corresponding
> > UART
> > > node in the DT passed to dom0. However, since hwmod structure is
> > > used in OMAP kernel, the dom0 would still try to write the UART's
> IO
> > > address space hard-coded in the hwmod structure, which could lead
> > kernel panic.
> > > One of the ugly hack is to remove corresponding	structure in
> > > omap54xx_hwmod_ocp_ifs (arch/arm/mach-
> omap2/omap_hwmod_54xx_data.c).
> > > I should say I'm still not very familiar with omap kernel and have
> > > no idea whether there would be more elegant ways to solve this
> problem.
> >
> > Just found that in the upstream kernel, there is no such issue. So
> > there is no need to disable hwmod manually in the kernel source if
> > using the latest upstream kernel :)
> >
> 
> Nice, so I only need to make the DTS changes (to mmc, mcbsp, etc.) as
> per your wiki ? I will try using the latest upstream kernel. Thanks for
> all the help.
> 
> Best,
> Ashish

After moving to linux 3.15-rc3, making the DTS changes suggested on the
wiki and disabling a call to omap_smc1() in mach-omap2/timer.c, I was
able to boot dom0 successfully.

Best,
Ashish