2.6.39-rc6 does not boot on Xen 3.3.1

2.6.39-rc6 does not boot on Xen 3.3.1

[Alex Bligh]

2.6.39-rc6 does not boot on Xen 3.3.1. It hangs on the first
line with Xen Platform PCI: unrecognised magic value.

Is this intentional?

-- 
Alex Bligh

Re: 2.6.39-rc6 does not boot on Xen 3.3.1

[Alex Bligh]

--On 7 May 2011 18:31:49 +0100 Alex Bligh <alex@alex.org.uk> wrote:

> 2.6.39-rc6 does not boot on Xen 3.3.1. It hangs on the first
> line with Xen Platform PCI: unrecognised magic value.
>
> Is this intentional?

Sorry, I should have been mode explicit:

A PV on HVM guest with CONFIG_XEN does not boot on Xen 2.6.39-rc6,
failing with the above error.

$ fgrep XEN .config
CONFIG_XEN=y
CONFIG_XEN_DOM0=y
CONFIG_XEN_PRIVILEGED_GUEST=y
CONFIG_XEN_PVHVM=y
CONFIG_XEN_MAX_DOMAIN_MEMORY=128
CONFIG_XEN_SAVE_RESTORE=y
# CONFIG_XEN_DEBUG_FS is not set
CONFIG_PCI_XEN=y
# CONFIG_XEN_PCIDEV_FRONTEND is not set
# CONFIG_XEN_BLKDEV_FRONTEND is not set
# CONFIG_NETXEN_NIC is not set
# CONFIG_XEN_NETDEV_FRONTEND is not set
CONFIG_HVC_XEN=y
# CONFIG_XEN_FBDEV_FRONTEND is not set
CONFIG_XEN_BALLOON=y
CONFIG_XEN_SCRUB_PAGES=y
# CONFIG_XEN_DEV_EVTCHN is not set
CONFIG_XEN_BACKEND=y
# CONFIG_XENFS is not set
CONFIG_XEN_SYS_HYPERVISOR=y
# CONFIG_XEN_GNTDEV is not set
# CONFIG_XEN_PLATFORM_PCI is not set
CONFIG_SWIOTLB_XEN=y

-- 
Alex Bligh

Re: Re: 2.6.39-rc6 does not boot on Xen 3.3.1

[Ian Campbell]

On Sat, 2011-05-07 at 18:50 +0100, Alex Bligh wrote:
> 
> --On 7 May 2011 18:31:49 +0100 Alex Bligh <alex@alex.org.uk> wrote:
> 
> > 2.6.39-rc6 does not boot on Xen 3.3.1. It hangs on the first
> > line with Xen Platform PCI: unrecognised magic value.
> >
> > Is this intentional?

I expect 3.3.1 didn't support the unplug protocol. However the default
failure behaviour should be to continue with emulated devices and not
hang.

If you have the platform device enabled (which you do not, according the
the .config below) then you can also cause the kernel to ignore the
requirement for unplug using the "xen_emul_unplug" kernel command line
option, assuming you are happy that you have ensured that the emulated
devices cannot be used by mistake.

> Sorry, I should have been mode explicit:
> 
> A PV on HVM guest with CONFIG_XEN does not boot on Xen 2.6.39-rc6,
> failing with the above error.

It's not really a PV on HVM guest since CONFIG_XEN_PLATFORM_PCI is
disabled.

Please can you capture a log of the kernel boot (e.g. setup the guest
kernel to use a serial console and use serial = "file:/tmp/blah.log" in
your guest cfg) and also show us your domU command line and guest cfg
file.

Are you running a self-built 3.3.1 or is this some distro package?

Thanks,
Ian.

> 
> $ fgrep XEN .config
> CONFIG_XEN=y
> CONFIG_XEN_DOM0=y
> CONFIG_XEN_PRIVILEGED_GUEST=y
> CONFIG_XEN_PVHVM=y
> CONFIG_XEN_MAX_DOMAIN_MEMORY=128
> CONFIG_XEN_SAVE_RESTORE=y
> # CONFIG_XEN_DEBUG_FS is not set
> CONFIG_PCI_XEN=y
> # CONFIG_XEN_PCIDEV_FRONTEND is not set
> # CONFIG_XEN_BLKDEV_FRONTEND is not set
> # CONFIG_NETXEN_NIC is not set
> # CONFIG_XEN_NETDEV_FRONTEND is not set
> CONFIG_HVC_XEN=y
> # CONFIG_XEN_FBDEV_FRONTEND is not set
> CONFIG_XEN_BALLOON=y
> CONFIG_XEN_SCRUB_PAGES=y
> # CONFIG_XEN_DEV_EVTCHN is not set
> CONFIG_XEN_BACKEND=y
> # CONFIG_XENFS is not set
> CONFIG_XEN_SYS_HYPERVISOR=y
> # CONFIG_XEN_GNTDEV is not set
> # CONFIG_XEN_PLATFORM_PCI is not set
> CONFIG_SWIOTLB_XEN=y
> 

Re: Re: 2.6.39-rc6 does not boot on Xen 3.3.1

[Stefano Stabellini]

On Sat, 7 May 2011, Alex Bligh wrote:
> --On 7 May 2011 18:31:49 +0100 Alex Bligh <alex@alex.org.uk> wrote:
> 
> > 2.6.39-rc6 does not boot on Xen 3.3.1. It hangs on the first
> > line with Xen Platform PCI: unrecognised magic value.
> >
> > Is this intentional?
> 
> Sorry, I should have been mode explicit:
> 
> A PV on HVM guest with CONFIG_XEN does not boot on Xen 2.6.39-rc6,
> failing with the above error.
> 
> $ fgrep XEN .config
> CONFIG_XEN=y
> CONFIG_XEN_DOM0=y
> CONFIG_XEN_PRIVILEGED_GUEST=y
> CONFIG_XEN_PVHVM=y
> CONFIG_XEN_MAX_DOMAIN_MEMORY=128
> CONFIG_XEN_SAVE_RESTORE=y
> # CONFIG_XEN_DEBUG_FS is not set
> CONFIG_PCI_XEN=y
> # CONFIG_XEN_PCIDEV_FRONTEND is not set
> # CONFIG_XEN_BLKDEV_FRONTEND is not set
> # CONFIG_NETXEN_NIC is not set
> # CONFIG_XEN_NETDEV_FRONTEND is not set
> CONFIG_HVC_XEN=y
> # CONFIG_XEN_FBDEV_FRONTEND is not set
> CONFIG_XEN_BALLOON=y
> CONFIG_XEN_SCRUB_PAGES=y
> # CONFIG_XEN_DEV_EVTCHN is not set
> CONFIG_XEN_BACKEND=y
> # CONFIG_XENFS is not set
> CONFIG_XEN_SYS_HYPERVISOR=y
> # CONFIG_XEN_GNTDEV is not set
> # CONFIG_XEN_PLATFORM_PCI is not set
> CONFIG_SWIOTLB_XEN=y


If you want a good PV on HVM configuration you need to enable also:

CONFIG_XEN_BLKDEV_FRONTEND
CONFIG_XEN_NETDEV_FRONTEND
CONFIG_XEN_PLATFORM_PCI

I suggest to compile CONFIG_XEN_PLATFORM_PCI built-in.

Re: Re: 2.6.39-rc6 does not boot on Xen 3.3.1

[Alex Bligh]

--On 9 May 2011 09:44:11 +0100 Ian Campbell <Ian.Campbell@citrix.com> wrote:

> On Sat, 2011-05-07 at 18:50 +0100, Alex Bligh wrote:
>>
>> --On 7 May 2011 18:31:49 +0100 Alex Bligh <alex@alex.org.uk> wrote:
>>
>> > 2.6.39-rc6 does not boot on Xen 3.3.1. It hangs on the first
>> > line with Xen Platform PCI: unrecognised magic value.
>> >
>> > Is this intentional?
>
> I expect 3.3.1 didn't support the unplug protocol. However the default
> failure behaviour should be to continue with emulated devices and not
> hang.

Indeed it did not, but 2.6.38 booted just fine, so something has changed.

> If you have the platform device enabled (which you do not, according the
> the .config below) then you can also cause the kernel to ignore the
> requirement for unplug using the "xen_emul_unplug" kernel command line
> option, assuming you are happy that you have ensured that the emulated
> devices cannot be used by mistake.

Sure. But whatever the change is means standard vendor kernels with
standard boot lines will not run on 3.3.1.

>> Sorry, I should have been mode explicit:
>>
>> A PV on HVM guest with CONFIG_XEN does not boot on Xen 2.6.39-rc6,
>> failing with the above error.
>
> It's not really a PV on HVM guest since CONFIG_XEN_PLATFORM_PCI is
> disabled.

I should have explained - I just got a stock set of Ubuntu kernel options,
compiled 2.6.39-rc6, and it won't boot on 3.3.1, whereas 2.6.38 does.
I know I can get it to boot.

> Please can you capture a log of the kernel boot (e.g. setup the guest
> kernel to use a serial console and use serial = "file:/tmp/blah.log" in
> your guest cfg) and also show us your domU command line and guest cfg
> file.

There was only one line of kernel output which was the one
I pasted! Just "Xen Platform PCI: unrecognised magic value" then
death.

> Are you running a self-built 3.3.1 or is this some distro package?

It's a self-built version of the Centos package, with a minor
(and irrelevant) mod.

-- 
Alex Bligh

Re: Re: 2.6.39-rc6 does not boot on Xen 3.3.1

[Alex Bligh]

--On 9 May 2011 13:17:15 +0100 Stefano Stabellini 
<stefano.stabellini@eu.citrix.com> wrote:

> If you want a good PV on HVM configuration you need to enable also:
>
> CONFIG_XEN_BLKDEV_FRONTEND
> CONFIG_XEN_NETDEV_FRONTEND
> CONFIG_XEN_PLATFORM_PCI
>
> I suggest to compile CONFIG_XEN_PLATFORM_PCI built-in.

I would not disagree - what I did was just take the stock Ubuntu
generic kernel options, and was somewhat surprised to find the resultant
kernel was unbootable, whereas it isn't on 2.6.38.

-- 
Alex Bligh

Re: Re: 2.6.39-rc6 does not boot on Xen 3.3.1

[Ian Campbell]

On Mon, 2011-05-09 at 19:37 +0100, Alex Bligh wrote:
> 
> --On 9 May 2011 09:44:11 +0100 Ian Campbell <Ian.Campbell@citrix.com> wrote:
> 
> > On Sat, 2011-05-07 at 18:50 +0100, Alex Bligh wrote:
> >>
> >> --On 7 May 2011 18:31:49 +0100 Alex Bligh <alex@alex.org.uk> wrote:
> >>
> >> > 2.6.39-rc6 does not boot on Xen 3.3.1. It hangs on the first
> >> > line with Xen Platform PCI: unrecognised magic value.
> >> >
> >> > Is this intentional?
> >
> > I expect 3.3.1 didn't support the unplug protocol. However the default
> > failure behaviour should be to continue with emulated devices and not
> > hang.
> 
> Indeed it did not, but 2.6.38 booted just fine, so something has changed.

Are you able to bisect?

> > If you have the platform device enabled (which you do not, according the
> > the .config below) then you can also cause the kernel to ignore the
> > requirement for unplug using the "xen_emul_unplug" kernel command line
> > option, assuming you are happy that you have ensured that the emulated
> > devices cannot be used by mistake.
> 
> Sure. But whatever the change is means standard vendor kernels with
> standard boot lines will not run on 3.3.1.

Yes, so we should fix it.

> > Please can you capture a log of the kernel boot (e.g. setup the guest
> > kernel to use a serial console and use serial = "file:/tmp/blah.log" in
> > your guest cfg) and also show us your domU command line and guest cfg
> > file.
> 
> There was only one line of kernel output which was the one
> I pasted! Just "Xen Platform PCI: unrecognised magic value" then
> death.

In which case I think you need to remove "quiet" from your command line
and add "debug" instead and ensure your console= line is correct. You
could also try using the earlyprintk option to see if that produces more
output.

Do you get any output on the hypervisor console ("xl dmesg" or "xm
dmesg" depending on toolstack).

> > Are you running a self-built 3.3.1 or is this some distro package?
> 
> It's a self-built version of the Centos package, with a minor
> (and irrelevant) mod.

Ian.

Re: Re: 2.6.39-rc6 does not boot on Xen 3.3.1

[Alex Bligh]

Ian,

>> Indeed it did not, but 2.6.38 booted just fine, so something has changed.
>
> Are you able to bisect?

I was going to do that but there was a huge pull shortly before rc6.
I wanted to check no one was going to say "oh yes, we disabled
3.3.1 support", rather than say:

> Yes, so we should fix it.

^- that :-)

>> There was only one line of kernel output which was the one
>> I pasted! Just "Xen Platform PCI: unrecognised magic value" then
>> death.
>
> In which case I think you need to remove "quiet" from your command line
> and add "debug" instead and ensure your console= line is correct. You
> could also try using the earlyprintk option to see if that produces more
> output.

Doh! I bet it's Ubuntu's default quiet option. I'd assumed I had
full output because of the "Xen Platform PCI: unrecognised magic value".

> Do you get any output on the hypervisor console ("xl dmesg" or "xm
> dmesg" depending on toolstack).

I will look. Would a full .config help too?

-- 
Alex Bligh

Re: Re: 2.6.39-rc6 does not boot on Xen 3.3.1

[Ian Campbell]

On Mon, 2011-05-09 at 20:20 +0100, Alex Bligh wrote:
> Ian,
> 
> >> Indeed it did not, but 2.6.38 booted just fine, so something has changed.
> >
> > Are you able to bisect?
> 
> I was going to do that but there was a huge pull shortly before rc6.
> I wanted to check no one was going to say "oh yes, we disabled
> 3.3.1 support", rather than say:
> 
> > Yes, so we should fix it.
> 
> ^- that :-)

Sure ;-)

It would probably be useful to do a quick test of the commit before the
huge pull you refer to above before embarking on a full bisection.

> >> There was only one line of kernel output which was the one
> >> I pasted! Just "Xen Platform PCI: unrecognised magic value" then
> >> death.
> >
> > In which case I think you need to remove "quiet" from your command line
> > and add "debug" instead and ensure your console= line is correct. You
> > could also try using the earlyprintk option to see if that produces more
> > output.
> 
> Doh! I bet it's Ubuntu's default quiet option. I'd assumed I had
> full output because of the "Xen Platform PCI: unrecognised magic value".
> 
> > Do you get any output on the hypervisor console ("xl dmesg" or "xm
> > dmesg" depending on toolstack).
> 
> I will look. Would a full .config help too?

Can't hurt. Full guest cfg file would be useful too, plus any file from
under /var/log/xen which has your guest's name in its name.

Ian.