Problems when using latest git tree to boot xen on OMAP5

Problems when using latest git tree to boot xen on OMAP5

[Chen Baozi]

Hi Julien,

Since Ian has merged most my OMAP5 patches, I decide to try to update my working tree to it today. However, it seems it doesn't work as expected. There mainly two kind of problems that I met:

1. It seems that xen no longer maps some of io memory regions described in dts. For example, the memory regions in "ocp" node of omap5.dtsi.

2. commit c88d32f4 seems to be conflict with the UART console hack in DTS, where we mark the status of target UART "disabled".

Both of the problems look like being introduced by new device tree related patches. 

Any ideas？

Cheers,

Baozi
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Problems when using latest git tree to boot xen on OMAP5

[Chen Baozi]

On Oct 4, 2013, at 10:11 PM, Chen Baozi <baozich@gmail.com> wrote:

> Hi Julien,
> 
> Since Ian has merged most my OMAP5 patches, I decide to try to update my working tree to it today. However, it seems it doesn't work as expected. There mainly two kind of problems that I met:
> 
> 1. It seems that xen no longer maps some of io memory regions described in dts. For example, the memory regions in "ocp" node of omap5.dtsi.

I looked into the codes today. It is because of checking dt_device_is_available() before map_device(). Some node, for example /ocp/mmc@480d1000, is disabled in omap5-uevm.dts separately from omap5.dtsi where it is original defined. In this case, xen won't map its memory region for dom0. However, it seems dom0 kernel still accesses those regions.

I guess there would be a bug dealing with this situations?

Cheers,

Baozi

> 
> 2. commit c88d32f4 seems to be conflict with the UART console hack in DTS, where we mark the status of target UART "disabled".
> 
> Both of the problems look like being introduced by new device tree related patches. 
> 
> Any ideas？
> 
> Cheers,
> 
> Baozi


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Problems when using latest git tree to boot xen on OMAP5

[Ian Campbell]

On Sun, 2013-10-06 at 22:21 +0800, Chen Baozi wrote:
> On Oct 4, 2013, at 10:11 PM, Chen Baozi <baozich@gmail.com> wrote:
> 
> > Hi Julien,
> > 
> > Since Ian has merged most my OMAP5 patches, I decide to try to update my working tree to it today. However, it seems it doesn't work as expected. There mainly two kind of problems that I met:
> > 
> > 1. It seems that xen no longer maps some of io memory regions described in dts. For example, the memory regions in "ocp" node of omap5.dtsi.
> 
> I looked into the codes today. It is because of checking
> dt_device_is_available() before map_device(). Some node, for
> example /ocp/mmc@480d1000, is disabled in omap5-uevm.dts separately
> from omap5.dtsi where it is original defined. In this case, xen won't
> map its memory region for dom0. However, it seems dom0 kernel still
> accesses those regions.

> I guess there would be a bug dealing with this situations?

It is certainly a bug in the kernel if it is accessing something which
is disabled. It may also independently be a bug in the dts that this
devices is disabled.

However in v3.12-rc4 I don't see mmc@480d1000 being disabled in
omap5-uevm.dts and I can't see anything in the history of that file
either. Where did your copy come from?

Hopefully we won't need a converse to the device blacklist, i.e. a
whitelist of disabled devices to pass through regardless of the DTS
enabled state.

Ian.

Re: Problems when using latest git tree to boot xen on OMAP5

[Chen Baozi]

On 10/07/2013 04:39 PM, Ian Campbell wrote:
> On Sun, 2013-10-06 at 22:21 +0800, Chen Baozi wrote:
>> On Oct 4, 2013, at 10:11 PM, Chen Baozi <baozich@gmail.com> wrote:
>>
>>> Hi Julien,
>>>
>>> Since Ian has merged most my OMAP5 patches, I decide to try to update my working tree to it today. However, it seems it doesn't work as expected. There mainly two kind of problems that I met:
>>>
>>> 1. It seems that xen no longer maps some of io memory regions described in dts. For example, the memory regions in "ocp" node of omap5.dtsi.
>>
>> I looked into the codes today. It is because of checking
>> dt_device_is_available() before map_device(). Some node, for
>> example /ocp/mmc@480d1000, is disabled in omap5-uevm.dts separately
>> from omap5.dtsi where it is original defined. In this case, xen won't
>> map its memory region for dom0. However, it seems dom0 kernel still
>> accesses those regions.
> 
>> I guess there would be a bug dealing with this situations?
> 
> It is certainly a bug in the kernel if it is accessing something which
> is disabled. It may also independently be a bug in the dts that this
> devices is disabled.
> 
> However in v3.12-rc4 I don't see mmc@480d1000 being disabled in
> omap5-uevm.dts and I can't see anything in the history of that file
> either. Where did your copy come from?

I'm currently working on the "omap5-v3.11-rc3" branch from
git://github.com/rogerq/linux.git, which contains a few necessary
platform patches not upstreamed. In omap5-uevm.dts, there are lines like:

253 &mmc4 {
254     status = "disabled";
255 };
256
257 &mmc5 {
258     status = "disabled";
259 };

the mmc4 refers to mmc@480d1000, which defines at omap5.dtsi:

417         mmc4: mmc@480d1000 {

I checked Linus' mainline git tree. It is the same about disabled mmc4
in omap5-uevm.dts. And the change is introduced in commit 5dd18b0 of the
mainline kernel.

Anyway, I'll see what exactly happened in the dom0 kernel dealing with
those "disabled" regions.

Cheers,

Baozi

> 
> Hopefully we won't need a converse to the device blacklist, i.e. a
> whitelist of disabled devices to pass through regardless of the DTS
> enabled state.
> 
> Ian.
> 
> 

Re: Problems when using latest git tree to boot xen on OMAP5

[Ian Campbell]

On Mon, 2013-10-07 at 17:58 +0800, Chen Baozi wrote:
> On 10/07/2013 04:39 PM, Ian Campbell wrote:
> > On Sun, 2013-10-06 at 22:21 +0800, Chen Baozi wrote:
> >> On Oct 4, 2013, at 10:11 PM, Chen Baozi <baozich@gmail.com> wrote:
> >>
> >>> Hi Julien,
> >>>
> >>> Since Ian has merged most my OMAP5 patches, I decide to try to update my working tree to it today. However, it seems it doesn't work as expected. There mainly two kind of problems that I met:
> >>>
> >>> 1. It seems that xen no longer maps some of io memory regions described in dts. For example, the memory regions in "ocp" node of omap5.dtsi.
> >>
> >> I looked into the codes today. It is because of checking
> >> dt_device_is_available() before map_device(). Some node, for
> >> example /ocp/mmc@480d1000, is disabled in omap5-uevm.dts separately
> >> from omap5.dtsi where it is original defined. In this case, xen won't
> >> map its memory region for dom0. However, it seems dom0 kernel still
> >> accesses those regions.
> > 
> >> I guess there would be a bug dealing with this situations?
> > 
> > It is certainly a bug in the kernel if it is accessing something which
> > is disabled. It may also independently be a bug in the dts that this
> > devices is disabled.
> > 
> > However in v3.12-rc4 I don't see mmc@480d1000 being disabled in
> > omap5-uevm.dts and I can't see anything in the history of that file
> > either. Where did your copy come from?
> 
> I'm currently working on the "omap5-v3.11-rc3" branch from
> git://github.com/rogerq/linux.git, which contains a few necessary
> platform patches not upstreamed. In omap5-uevm.dts, there are lines like:
> 
> 253 &mmc4 {
> 254     status = "disabled";
> 255 };
> 256
> 257 &mmc5 {
> 258     status = "disabled";
> 259 };
> 
> the mmc4 refers to mmc@480d1000, which defines at omap5.dtsi:
> 
> 417         mmc4: mmc@480d1000 {
> 
> I checked Linus' mainline git tree. It is the same about disabled mmc4
> in omap5-uevm.dts. And the change is introduced in commit 5dd18b0 of the
> mainline kernel.

I was grepping for mmc@480d1 not mmc4 so I missed this.

> Anyway, I'll see what exactly happened in the dom0 kernel dealing with
> those "disabled" regions.

Thanks.
Ian.

> 
> Cheers,
> 
> Baozi
> 
> > 
> > Hopefully we won't need a converse to the device blacklist, i.e. a
> > whitelist of disabled devices to pass through regardless of the DTS
> > enabled state.
> > 
> > Ian.
> > 
> > 

Re: Problems when using latest git tree to boot xen on OMAP5

[Julien Grall]

On 10/04/2013 03:11 PM, Chen Baozi wrote:
> Hi Julien,

Hi Chen,

> 2. commit c88d32f4 seems to be conflict with the UART console hack in DTS, where we mark the status of target UART "disabled".

You can remove the hack from the DTS. It was used to avoid dom0 to use
the UART. Now, Xen removes the UART from the device tree.

Cheers,

-- 
Julien Grall

Re: Problems when using latest git tree to boot xen on OMAP5

[Julien Grall]

On 10/07/2013 10:58 AM, Chen Baozi wrote:
> On 10/07/2013 04:39 PM, Ian Campbell wrote:
>>
>> It is certainly a bug in the kernel if it is accessing something which
>> is disabled. It may also independently be a bug in the dts that this
>> devices is disabled.
>>
>> However in v3.12-rc4 I don't see mmc@480d1000 being disabled in
>> omap5-uevm.dts and I can't see anything in the history of that file
>> either. Where did your copy come from?
> 
> I'm currently working on the "omap5-v3.11-rc3" branch from
> git://github.com/rogerq/linux.git, which contains a few necessary
> platform patches not upstreamed. In omap5-uevm.dts, there are lines like:
> 
> 253 &mmc4 {
> 254     status = "disabled";
> 255 };
> 256
> 257 &mmc5 {
> 258     status = "disabled";
> 259 };
> 
> the mmc4 refers to mmc@480d1000, which defines at omap5.dtsi:
> 
> 417         mmc4: mmc@480d1000 {
> 
> I checked Linus' mainline git tree. It is the same about disabled mmc4
> in omap5-uevm.dts. And the change is introduced in commit 5dd18b0 of the
> mainline kernel.
> 
> Anyway, I'll see what exactly happened in the dom0 kernel dealing with
> those "disabled" regions.

I looked at the Linux code. It will populate the different devices via
the of_platform_populate (drivers/of/platform.c).

This function checks in of_platform_create_pdata if the device is
available. So the mmc driver (driver/mmc/host/omap_hsmmc.c) should not
be called for mmc4.

Can you find who generates the data abort? You can use addr2line for
this purpose.

Cheers,

-- 
Julien Grall

Re: Problems when using latest git tree to boot xen on OMAP5

[Chen Baozi]

On Oct 7, 2013, at 6:59 PM, Julien Grall <julien.grall@linaro.org> wrote:

> On 10/07/2013 10:58 AM, Chen Baozi wrote:
>> On 10/07/2013 04:39 PM, Ian Campbell wrote:
>>> 
>>> It is certainly a bug in the kernel if it is accessing something which
>>> is disabled. It may also independently be a bug in the dts that this
>>> devices is disabled.
>>> 
>>> However in v3.12-rc4 I don't see mmc@480d1000 being disabled in
>>> omap5-uevm.dts and I can't see anything in the history of that file
>>> either. Where did your copy come from?
>> 
>> I'm currently working on the "omap5-v3.11-rc3" branch from
>> git://github.com/rogerq/linux.git, which contains a few necessary
>> platform patches not upstreamed. In omap5-uevm.dts, there are lines like:
>> 
>> 253 &mmc4 {
>> 254     status = "disabled";
>> 255 };
>> 256
>> 257 &mmc5 {
>> 258     status = "disabled";
>> 259 };
>> 
>> the mmc4 refers to mmc@480d1000, which defines at omap5.dtsi:
>> 
>> 417         mmc4: mmc@480d1000 {
>> 
>> I checked Linus' mainline git tree. It is the same about disabled mmc4
>> in omap5-uevm.dts. And the change is introduced in commit 5dd18b0 of the
>> mainline kernel.
>> 
>> Anyway, I'll see what exactly happened in the dom0 kernel dealing with
>> those "disabled" regions.
> 
> I looked at the Linux code. It will populate the different devices via
> the of_platform_populate (drivers/of/platform.c).
> 
> This function checks in of_platform_create_pdata if the device is
> available. So the mmc driver (driver/mmc/host/omap_hsmmc.c) should not
> be called for mmc4.
> 
> Can you find who generates the data abort? You can use addr2line for
> this purpose.

addr2line would lead to __raw_readl() in arch/arm/include/asm/io.h. So
it doesn't that useful.

I guess omap platform codes access the memory address when doing low-level
setup and there are some hard coded address in its implementation. Note that
there are hard coded "mmc4" structure in 
arch/arm/mach-omap2/omap_hwmod_54xx_data.c

Cheers,

Baozi

Re: Problems when using latest git tree to boot xen on OMAP5

[Julien Grall]

On 10/07/2013 04:28 PM, Chen Baozi wrote:

> 
> addr2line would lead to __raw_readl() in arch/arm/include/asm/io.h. So
> it doesn't that useful.

Ah right ... but __raw_readl is usually inlined so you can use objdump
and find the instruction. After that, if you go a bit above you can find
the caller name.

-- 
Julien Grall

Re: Problems when using latest git tree to boot xen on OMAP5

[Chen Baozi]

On Oct 7, 2013, at 11:44 PM, Julien Grall <julien.grall@linaro.org> wrote:

> On 10/07/2013 04:28 PM, Chen Baozi wrote:
> 
>> 
>> addr2line would lead to __raw_readl() in arch/arm/include/asm/io.h. So
>> it doesn't that useful.
> 
> Ah right ... but __raw_readl is usually inlined so you can use objdump
> and find the instruction. After that, if you go a bit above you can find
> the caller name.

It is called by omap_hwmod_read(). And it is mostly like to be called
ultimately by omap_device_build_from_dt in _omap_device_notifier_call.

I guess it might relate to the OCP platform bus initialization and 
hard coded "omap_hwmod" information. But I can't point out the exact
calling stack of omap_hwmod_read sadly...

Cheers,

Baozi

Re: Problems when using latest git tree to boot xen on OMAP5

[Chen Baozi]

On Mon, Oct 07, 2013 at 11:59:16AM +0100, Julien Grall wrote:
> On 10/07/2013 10:58 AM, Chen Baozi wrote:
> > On 10/07/2013 04:39 PM, Ian Campbell wrote:
> >>
> >> It is certainly a bug in the kernel if it is accessing something which
> >> is disabled. It may also independently be a bug in the dts that this
> >> devices is disabled.
> >>
> >> However in v3.12-rc4 I don't see mmc@480d1000 being disabled in
> >> omap5-uevm.dts and I can't see anything in the history of that file
> >> either. Where did your copy come from?
> > 
> > I'm currently working on the "omap5-v3.11-rc3" branch from
> > git://github.com/rogerq/linux.git, which contains a few necessary
> > platform patches not upstreamed. In omap5-uevm.dts, there are lines like:
> > 
> > 253 &mmc4 {
> > 254     status = "disabled";
> > 255 };
> > 256
> > 257 &mmc5 {
> > 258     status = "disabled";
> > 259 };
> > 
> > the mmc4 refers to mmc@480d1000, which defines at omap5.dtsi:
> > 
> > 417         mmc4: mmc@480d1000 {
> > 
> > I checked Linus' mainline git tree. It is the same about disabled mmc4
> > in omap5-uevm.dts. And the change is introduced in commit 5dd18b0 of the
> > mainline kernel.
> > 
> > Anyway, I'll see what exactly happened in the dom0 kernel dealing with
> > those "disabled" regions.
> 
> I looked at the Linux code. It will populate the different devices via
> the of_platform_populate (drivers/of/platform.c).
> 
> This function checks in of_platform_create_pdata if the device is
> available. So the mmc driver (driver/mmc/host/omap_hsmmc.c) should not
> be called for mmc4.

I've discuessed this issue on linux-omap@vger.kernel.org. The TI guy says
that "DT disabled" means that device won't be created but hwmod bus
initially would try to initialize all supported modules by doing reset
access in their io memory address regions. That's why dom0 have accessed
mmc4 address even though it has been disabled.

Cheers,

Baozi

Re: Problems when using latest git tree to boot xen on OMAP5

[Ian Campbell]

On Wed, 2013-10-09 at 15:46 +0800, Chen Baozi wrote:
> On Mon, Oct 07, 2013 at 11:59:16AM +0100, Julien Grall wrote:
> > On 10/07/2013 10:58 AM, Chen Baozi wrote:
> > > On 10/07/2013 04:39 PM, Ian Campbell wrote:
> > >>
> > >> It is certainly a bug in the kernel if it is accessing something which
> > >> is disabled. It may also independently be a bug in the dts that this
> > >> devices is disabled.
> > >>
> > >> However in v3.12-rc4 I don't see mmc@480d1000 being disabled in
> > >> omap5-uevm.dts and I can't see anything in the history of that file
> > >> either. Where did your copy come from?
> > > 
> > > I'm currently working on the "omap5-v3.11-rc3" branch from
> > > git://github.com/rogerq/linux.git, which contains a few necessary
> > > platform patches not upstreamed. In omap5-uevm.dts, there are lines like:
> > > 
> > > 253 &mmc4 {
> > > 254     status = "disabled";
> > > 255 };
> > > 256
> > > 257 &mmc5 {
> > > 258     status = "disabled";
> > > 259 };
> > > 
> > > the mmc4 refers to mmc@480d1000, which defines at omap5.dtsi:
> > > 
> > > 417         mmc4: mmc@480d1000 {
> > > 
> > > I checked Linus' mainline git tree. It is the same about disabled mmc4
> > > in omap5-uevm.dts. And the change is introduced in commit 5dd18b0 of the
> > > mainline kernel.
> > > 
> > > Anyway, I'll see what exactly happened in the dom0 kernel dealing with
> > > those "disabled" regions.
> > 
> > I looked at the Linux code. It will populate the different devices via
> > the of_platform_populate (drivers/of/platform.c).
> > 
> > This function checks in of_platform_create_pdata if the device is
> > available. So the mmc driver (driver/mmc/host/omap_hsmmc.c) should not
> > be called for mmc4.
> 
> I've discuessed this issue on linux-omap@vger.kernel.org. The TI guy says
> that "DT disabled" means that device won't be created but hwmod bus
> initially would try to initialize all supported modules by doing reset
> access in their io memory address regions. That's why dom0 have accessed
> mmc4 address even though it has been disabled.

ePAPR lists the option of "status = fail" which is "Indicates that the
device is not operational. A serious error was detected in the device,
and it is unlikely to become operational without repair." I'm not sure
that is quite right though.

Rather than whitelisting and mapping disabled devices through perhaps we
should implement them as read 0xf (or 0x0) and write ignore?

Or maybe we should just be mapping non-blacklisted disabled devices
through to dom0, for it to use or ignore as it pleases. Julien, what was
the reasoning here again?

Ian.

Re: Problems when using latest git tree to boot xen on OMAP5

[Chen Baozi]

On Oct 9, 2013, at 4:11 PM, Ian Campbell <Ian.Campbell@citrix.com> wrote:

> On Wed, 2013-10-09 at 15:46 +0800, Chen Baozi wrote:
>> On Mon, Oct 07, 2013 at 11:59:16AM +0100, Julien Grall wrote:
>>> On 10/07/2013 10:58 AM, Chen Baozi wrote:
>>>> On 10/07/2013 04:39 PM, Ian Campbell wrote:
>>>>> 
>>>>> It is certainly a bug in the kernel if it is accessing something which
>>>>> is disabled. It may also independently be a bug in the dts that this
>>>>> devices is disabled.
>>>>> 
>>>>> However in v3.12-rc4 I don't see mmc@480d1000 being disabled in
>>>>> omap5-uevm.dts and I can't see anything in the history of that file
>>>>> either. Where did your copy come from?
>>>> 
>>>> I'm currently working on the "omap5-v3.11-rc3" branch from
>>>> git://github.com/rogerq/linux.git, which contains a few necessary
>>>> platform patches not upstreamed. In omap5-uevm.dts, there are lines like:
>>>> 
>>>> 253 &mmc4 {
>>>> 254     status = "disabled";
>>>> 255 };
>>>> 256
>>>> 257 &mmc5 {
>>>> 258     status = "disabled";
>>>> 259 };
>>>> 
>>>> the mmc4 refers to mmc@480d1000, which defines at omap5.dtsi:
>>>> 
>>>> 417         mmc4: mmc@480d1000 {
>>>> 
>>>> I checked Linus' mainline git tree. It is the same about disabled mmc4
>>>> in omap5-uevm.dts. And the change is introduced in commit 5dd18b0 of the
>>>> mainline kernel.
>>>> 
>>>> Anyway, I'll see what exactly happened in the dom0 kernel dealing with
>>>> those "disabled" regions.
>>> 
>>> I looked at the Linux code. It will populate the different devices via
>>> the of_platform_populate (drivers/of/platform.c).
>>> 
>>> This function checks in of_platform_create_pdata if the device is
>>> available. So the mmc driver (driver/mmc/host/omap_hsmmc.c) should not
>>> be called for mmc4.
>> 
>> I've discuessed this issue on linux-omap@vger.kernel.org. The TI guy says
>> that "DT disabled" means that device won't be created but hwmod bus
>> initially would try to initialize all supported modules by doing reset
>> access in their io memory address regions. That's why dom0 have accessed
>> mmc4 address even though it has been disabled.
> 
> ePAPR lists the option of "status = fail" which is "Indicates that the
> device is not operational. A serious error was detected in the device,
> and it is unlikely to become operational without repair." I'm not sure
> that is quite right though.

I guess when TI implemented omap_hwmod, DT is still not widely accepted 
on ARM platform. So when omap_hwmod doing initialization, it doesn't
follow ePAPR standard strictly. Maybe we can call this a historical
issue?

The TI people said they have addressed proposals that those resets will
be removed to avoid our issue. But it seems this won't be too soon I
guess...

Cheers,

Baozi

> 
> Rather than whitelisting and mapping disabled devices through perhaps we
> should implement them as read 0xf (or 0x0) and write ignore?
> 
> Or maybe we should just be mapping non-blacklisted disabled devices
> through to dom0, for it to use or ignore as it pleases. Julien, what was
> the reasoning here again?
> 
> Ian.
> 

Re: Problems when using latest git tree to boot xen on OMAP5

[Julien Grall]

On 10/09/2013 09:11 AM, Ian Campbell wrote:
> Rather than whitelisting and mapping disabled devices through perhaps we
> should implement them as read 0xf (or 0x0) and write ignore?

We can't assume that these values (0x0 or 0xf) won't affect dom0 behaviour.

What about a new quirk to map all disabled device in dom0
(PLATFORM_QUIRK_DOM0_MAP_DISABLED_DEVICE)?

-- 
Julien Grall

Re: Problems when using latest git tree to boot xen on OMAP5

[Ian Campbell]

On Wed, 2013-10-09 at 12:15 +0100, Julien Grall wrote:
> On 10/09/2013 09:11 AM, Ian Campbell wrote:
> > Rather than whitelisting and mapping disabled devices through perhaps we
> > should implement them as read 0xf (or 0x0) and write ignore?
> 
> We can't assume that these values (0x0 or 0xf) won't affect dom0 behaviour.

0xff is what you would typically get back if there was no actual
hardware present.

> What about a new quirk to map all disabled device in dom0
> (PLATFORM_QUIRK_DOM0_MAP_DISABLED_DEVICE)?

Why is this not the default behaviour?

Re: Problems when using latest git tree to boot xen on OMAP5

[Stefano Stabellini]

On Wed, 9 Oct 2013, Ian Campbell wrote:
> On Wed, 2013-10-09 at 12:15 +0100, Julien Grall wrote:
> > On 10/09/2013 09:11 AM, Ian Campbell wrote:
> > > Rather than whitelisting and mapping disabled devices through perhaps we
> > > should implement them as read 0xf (or 0x0) and write ignore?
> > 
> > We can't assume that these values (0x0 or 0xf) won't affect dom0 behaviour.
> 
> 0xff is what you would typically get back if there was no actual
> hardware present.
> 
> > What about a new quirk to map all disabled device in dom0
> > (PLATFORM_QUIRK_DOM0_MAP_DISABLED_DEVICE)?
> 
> Why is this not the default behaviour?

I think this is not a good idea: it prevents us from assigning one
device directly to a domain other than dom0 from Xen at boot time
(without going through something like PCI passthough).
Also if we really don't want dom0 to use a device, it makes perfect
sense that we don't map the device to it.

I would rather introduce PLATFORM_QUIRK_DOM0_EMULATE_DISABLED_DEVICE
and implement it with a simple write ignore, read 0xff.

Re: Problems when using latest git tree to boot xen on OMAP5

[Julien Grall]

On 10/09/2013 12:19 PM, Ian Campbell wrote:
> On Wed, 2013-10-09 at 12:15 +0100, Julien Grall wrote:
>> On 10/09/2013 09:11 AM, Ian Campbell wrote:
>>> Rather than whitelisting and mapping disabled devices through perhaps we
>>> should implement them as read 0xf (or 0x0) and write ignore?
>>
>> We can't assume that these values (0x0 or 0xf) won't affect dom0 behaviour.
> 
> 0xff is what you would typically get back if there was no actual
> hardware present.
> 
>> What about a new quirk to map all disabled device in dom0
>> (PLATFORM_QUIRK_DOM0_MAP_DISABLED_DEVICE)?
> 
> Why is this not the default behaviour?

>From our discussion a months a ago (http://patches.linaro.org/19590/),
status = "disabled" means : "not present/wired-up on this variant of
the platform".

As Linux should cope with this status, why do we need to map these
devices in DOM0?

If we go further, with passthrough we may want remove device from the
device tree. If Linux has hardcoded device (but also in the device tree)
it will likely fail.

-- 
Julien Grall

Re: Problems when using latest git tree to boot xen on OMAP5

[Ian Campbell]

On Wed, 2013-10-09 at 12:29 +0100, Stefano Stabellini wrote:
> On Wed, 9 Oct 2013, Ian Campbell wrote:
> > On Wed, 2013-10-09 at 12:15 +0100, Julien Grall wrote:
> > > On 10/09/2013 09:11 AM, Ian Campbell wrote:
> > > > Rather than whitelisting and mapping disabled devices through perhaps we
> > > > should implement them as read 0xf (or 0x0) and write ignore?
> > > 
> > > We can't assume that these values (0x0 or 0xf) won't affect dom0 behaviour.
> > 
> > 0xff is what you would typically get back if there was no actual
> > hardware present.
> > 
> > > What about a new quirk to map all disabled device in dom0
> > > (PLATFORM_QUIRK_DOM0_MAP_DISABLED_DEVICE)?
> > 
> > Why is this not the default behaviour?
> 
> I think this is not a good idea: it prevents us from assigning one
> device directly to a domain other than dom0 from Xen at boot time
> (without going through something like PCI passthough).

No it doesn't, it would just be a default behaviour which would be
overrideable by some new command line option.
 
Note that we don't even go as far as you suggest for PCI passthrough on
x86, where dom0 still sees the device and the hide option is a dom0 one.

> Also if we really don't want dom0 to use a device, it makes perfect
> sense that we don't map the device to it.
> 
> I would rather introduce PLATFORM_QUIRK_DOM0_EMULATE_DISABLED_DEVICE
> and implement it with a simple write ignore, read 0xff.

Re: Problems when using latest git tree to boot xen on OMAP5

[Ian Campbell]

On Wed, 2013-10-09 at 12:29 +0100, Julien Grall wrote:
> On 10/09/2013 12:19 PM, Ian Campbell wrote:
> > On Wed, 2013-10-09 at 12:15 +0100, Julien Grall wrote:
> >> On 10/09/2013 09:11 AM, Ian Campbell wrote:
> >>> Rather than whitelisting and mapping disabled devices through perhaps we
> >>> should implement them as read 0xf (or 0x0) and write ignore?
> >>
> >> We can't assume that these values (0x0 or 0xf) won't affect dom0 behaviour.
> > 
> > 0xff is what you would typically get back if there was no actual
> > hardware present.
> > 
> >> What about a new quirk to map all disabled device in dom0
> >> (PLATFORM_QUIRK_DOM0_MAP_DISABLED_DEVICE)?
> > 
> > Why is this not the default behaviour?
> 
> From our discussion a months a ago (http://patches.linaro.org/19590/),
> status = "disabled" means : "not present/wired-up on this variant of
> the platform".

Actually ePAPR says:
        Indicates that the device is not presently operational, but it
        might
        become operational in the future (for example, something is not
        plugged in, or switched off).
        Refer to the device binding for details on what disabled means
        for
        a given device.

This doesn't strictly speaking say "and you must not prod any associated
MMIO".

> As Linux should cope with this status, why do we need to map these
> devices in DOM0?

Because in practice Linux doesn't actually cope like you asserted it
would.

> If we go further, with passthrough we may want remove device from the
> device tree. If Linux has hardcoded device (but also in the device tree)
> it will likely fail.

If/when we come to be passing through a device which Linux is poking
despite it being disabled then we have a kernel bug which needs fixing,
nothing more, and there is no reason for this to be a factor in deciding
what to do by default.

Ian.

Re: Problems when using latest git tree to boot xen on OMAP5

[Stefano Stabellini]

On Wed, 9 Oct 2013, Ian Campbell wrote:
> On Wed, 2013-10-09 at 12:29 +0100, Stefano Stabellini wrote:
> > On Wed, 9 Oct 2013, Ian Campbell wrote:
> > > On Wed, 2013-10-09 at 12:15 +0100, Julien Grall wrote:
> > > > On 10/09/2013 09:11 AM, Ian Campbell wrote:
> > > > > Rather than whitelisting and mapping disabled devices through perhaps we
> > > > > should implement them as read 0xf (or 0x0) and write ignore?
> > > > 
> > > > We can't assume that these values (0x0 or 0xf) won't affect dom0 behaviour.
> > > 
> > > 0xff is what you would typically get back if there was no actual
> > > hardware present.
> > > 
> > > > What about a new quirk to map all disabled device in dom0
> > > > (PLATFORM_QUIRK_DOM0_MAP_DISABLED_DEVICE)?
> > > 
> > > Why is this not the default behaviour?
> > 
> > I think this is not a good idea: it prevents us from assigning one
> > device directly to a domain other than dom0 from Xen at boot time
> > (without going through something like PCI passthough).
> 
> No it doesn't, it would just be a default behaviour which would be
> overrideable by some new command line option.

I see.
I would still keep it off by default and maybe enable the workaround
only where we actually need it. If we enable it by default we might not
know exactly which ones are the platforms that actually require it.


> Note that we don't even go as far as you suggest for PCI passthrough on
> x86, where dom0 still sees the device and the hide option is a dom0 one.

The main difference is that in that case we can hide a given device
using pciback (or pcistub), in this case we can't. Also on x86 we are
constrained by the PCI bus, while here we might be able to have a better
design for non-PCI devices.

Re: Problems when using latest git tree to boot xen on OMAP5

[Chen Baozi]

On Wed, Oct 09, 2013 at 09:11:16AM +0100, Ian Campbell wrote:
> On Wed, 2013-10-09 at 15:46 +0800, Chen Baozi wrote:
> > On Mon, Oct 07, 2013 at 11:59:16AM +0100, Julien Grall wrote:
> > > On 10/07/2013 10:58 AM, Chen Baozi wrote:
> > > > On 10/07/2013 04:39 PM, Ian Campbell wrote:
> > > >>
> > > >> It is certainly a bug in the kernel if it is accessing something which
> > > >> is disabled. It may also independently be a bug in the dts that this
> > > >> devices is disabled.
> > > >>
> > > >> However in v3.12-rc4 I don't see mmc@480d1000 being disabled in
> > > >> omap5-uevm.dts and I can't see anything in the history of that file
> > > >> either. Where did your copy come from?
> > > > 
> > > > I'm currently working on the "omap5-v3.11-rc3" branch from
> > > > git://github.com/rogerq/linux.git, which contains a few necessary
> > > > platform patches not upstreamed. In omap5-uevm.dts, there are lines like:
> > > > 
> > > > 253 &mmc4 {
> > > > 254     status = "disabled";
> > > > 255 };
> > > > 256
> > > > 257 &mmc5 {
> > > > 258     status = "disabled";
> > > > 259 };
> > > > 
> > > > the mmc4 refers to mmc@480d1000, which defines at omap5.dtsi:
> > > > 
> > > > 417         mmc4: mmc@480d1000 {
> > > > 
> > > > I checked Linus' mainline git tree. It is the same about disabled mmc4
> > > > in omap5-uevm.dts. And the change is introduced in commit 5dd18b0 of the
> > > > mainline kernel.
> > > > 
> > > > Anyway, I'll see what exactly happened in the dom0 kernel dealing with
> > > > those "disabled" regions.
> > > 
> > > I looked at the Linux code. It will populate the different devices via
> > > the of_platform_populate (drivers/of/platform.c).
> > > 
> > > This function checks in of_platform_create_pdata if the device is
> > > available. So the mmc driver (driver/mmc/host/omap_hsmmc.c) should not
> > > be called for mmc4.
> > 
> > I've discuessed this issue on linux-omap@vger.kernel.org. The TI guy says
> > that "DT disabled" means that device won't be created but hwmod bus
> > initially would try to initialize all supported modules by doing reset
> > access in their io memory address regions. That's why dom0 have accessed
> > mmc4 address even though it has been disabled.
> 
> ePAPR lists the option of "status = fail" which is "Indicates that the
> device is not operational. A serious error was detected in the device,
> and it is unlikely to become operational without repair." I'm not sure
> that is quite right though.
> 
> Rather than whitelisting and mapping disabled devices through perhaps we
> should implement them as read 0xf (or 0x0) and write ignore?
> 
> Or maybe we should just be mapping non-blacklisted disabled devices
> through to dom0, for it to use or ignore as it pleases. Julien, what was
> the reasoning here again?

If we just simple don't pass the disabled node in DT to the dom0, anything
inpropriate?

Baozi.

Re: Problems when using latest git tree to boot xen on OMAP5

[Julien Grall]

On 10/10/2013 09:05 AM, Chen Baozi wrote:
> On Wed, Oct 09, 2013 at 09:11:16AM +0100, Ian Campbell wrote:
>> On Wed, 2013-10-09 at 15:46 +0800, Chen Baozi wrote:
>>> On Mon, Oct 07, 2013 at 11:59:16AM +0100, Julien Grall wrote:
>>>> On 10/07/2013 10:58 AM, Chen Baozi wrote:
>>>>> On 10/07/2013 04:39 PM, Ian Campbell wrote:
>>>>>>
>>>>>> It is certainly a bug in the kernel if it is accessing something which
>>>>>> is disabled. It may also independently be a bug in the dts that this
>>>>>> devices is disabled.
>>>>>>
>>>>>> However in v3.12-rc4 I don't see mmc@480d1000 being disabled in
>>>>>> omap5-uevm.dts and I can't see anything in the history of that file
>>>>>> either. Where did your copy come from?
>>>>>
>>>>> I'm currently working on the "omap5-v3.11-rc3" branch from
>>>>> git://github.com/rogerq/linux.git, which contains a few necessary
>>>>> platform patches not upstreamed. In omap5-uevm.dts, there are lines like:
>>>>>
>>>>> 253 &mmc4 {
>>>>> 254     status = "disabled";
>>>>> 255 };
>>>>> 256
>>>>> 257 &mmc5 {
>>>>> 258     status = "disabled";
>>>>> 259 };
>>>>>
>>>>> the mmc4 refers to mmc@480d1000, which defines at omap5.dtsi:
>>>>>
>>>>> 417         mmc4: mmc@480d1000 {
>>>>>
>>>>> I checked Linus' mainline git tree. It is the same about disabled mmc4
>>>>> in omap5-uevm.dts. And the change is introduced in commit 5dd18b0 of the
>>>>> mainline kernel.
>>>>>
>>>>> Anyway, I'll see what exactly happened in the dom0 kernel dealing with
>>>>> those "disabled" regions.
>>>>
>>>> I looked at the Linux code. It will populate the different devices via
>>>> the of_platform_populate (drivers/of/platform.c).
>>>>
>>>> This function checks in of_platform_create_pdata if the device is
>>>> available. So the mmc driver (driver/mmc/host/omap_hsmmc.c) should not
>>>> be called for mmc4.
>>>
>>> I've discuessed this issue on linux-omap@vger.kernel.org. The TI guy says
>>> that "DT disabled" means that device won't be created but hwmod bus
>>> initially would try to initialize all supported modules by doing reset
>>> access in their io memory address regions. That's why dom0 have accessed
>>> mmc4 address even though it has been disabled.
>>
>> ePAPR lists the option of "status = fail" which is "Indicates that the
>> device is not operational. A serious error was detected in the device,
>> and it is unlikely to become operational without repair." I'm not sure
>> that is quite right though.
>>
>> Rather than whitelisting and mapping disabled devices through perhaps we
>> should implement them as read 0xf (or 0x0) and write ignore?
>>
>> Or maybe we should just be mapping non-blacklisted disabled devices
>> through to dom0, for it to use or ignore as it pleases. Julien, what was
>> the reasoning here again?
> 
> If we just simple don't pass the disabled node in DT to the dom0, anything
> inpropriate?

Few mails ago, you said there is hardcoded access in Linux for OMAP,
right? I don't think removing the node from the Device Tree will change
something with this problem.

-- 
Julien Grall

Re: Problems when using latest git tree to boot xen on OMAP5

[Chen Baozi]

On Oct 10, 2013, at 7:07 PM, Julien Grall <julien.grall@linaro.org> wrote:

> On 10/10/2013 09:05 AM, Chen Baozi wrote:
>> On Wed, Oct 09, 2013 at 09:11:16AM +0100, Ian Campbell wrote:
>>> On Wed, 2013-10-09 at 15:46 +0800, Chen Baozi wrote:
>>>> On Mon, Oct 07, 2013 at 11:59:16AM +0100, Julien Grall wrote:
>>>>> On 10/07/2013 10:58 AM, Chen Baozi wrote:
>>>>>> On 10/07/2013 04:39 PM, Ian Campbell wrote:
>>>>>>> 
>>>>>>> It is certainly a bug in the kernel if it is accessing something which
>>>>>>> is disabled. It may also independently be a bug in the dts that this
>>>>>>> devices is disabled.
>>>>>>> 
>>>>>>> However in v3.12-rc4 I don't see mmc@480d1000 being disabled in
>>>>>>> omap5-uevm.dts and I can't see anything in the history of that file
>>>>>>> either. Where did your copy come from?
>>>>>> 
>>>>>> I'm currently working on the "omap5-v3.11-rc3" branch from
>>>>>> git://github.com/rogerq/linux.git, which contains a few necessary
>>>>>> platform patches not upstreamed. In omap5-uevm.dts, there are lines like:
>>>>>> 
>>>>>> 253 &mmc4 {
>>>>>> 254     status = "disabled";
>>>>>> 255 };
>>>>>> 256
>>>>>> 257 &mmc5 {
>>>>>> 258     status = "disabled";
>>>>>> 259 };
>>>>>> 
>>>>>> the mmc4 refers to mmc@480d1000, which defines at omap5.dtsi:
>>>>>> 
>>>>>> 417         mmc4: mmc@480d1000 {
>>>>>> 
>>>>>> I checked Linus' mainline git tree. It is the same about disabled mmc4
>>>>>> in omap5-uevm.dts. And the change is introduced in commit 5dd18b0 of the
>>>>>> mainline kernel.
>>>>>> 
>>>>>> Anyway, I'll see what exactly happened in the dom0 kernel dealing with
>>>>>> those "disabled" regions.
>>>>> 
>>>>> I looked at the Linux code. It will populate the different devices via
>>>>> the of_platform_populate (drivers/of/platform.c).
>>>>> 
>>>>> This function checks in of_platform_create_pdata if the device is
>>>>> available. So the mmc driver (driver/mmc/host/omap_hsmmc.c) should not
>>>>> be called for mmc4.
>>>> 
>>>> I've discuessed this issue on linux-omap@vger.kernel.org. The TI guy says
>>>> that "DT disabled" means that device won't be created but hwmod bus
>>>> initially would try to initialize all supported modules by doing reset
>>>> access in their io memory address regions. That's why dom0 have accessed
>>>> mmc4 address even though it has been disabled.
>>> 
>>> ePAPR lists the option of "status = fail" which is "Indicates that the
>>> device is not operational. A serious error was detected in the device,
>>> and it is unlikely to become operational without repair." I'm not sure
>>> that is quite right though.
>>> 
>>> Rather than whitelisting and mapping disabled devices through perhaps we
>>> should implement them as read 0xf (or 0x0) and write ignore?
>>> 
>>> Or maybe we should just be mapping non-blacklisted disabled devices
>>> through to dom0, for it to use or ignore as it pleases. Julien, what was
>>> the reasoning here again?
>> 
>> If we just simple don't pass the disabled node in DT to the dom0, anything
>> inpropriate?
> 
> Few mails ago, you said there is hardcoded access in Linux for OMAP,
> right? I don't think removing the node from the Device Tree will change
> something with this problem.

Oh, I just had a assumption about hardcoded stuffs then. However, after discussed
with TI guys, it turns out I've just made a wrong assumption. There is no
hard-coding rather a different requirement why omap_hwmod accesses the module's
io memory space, according to their explanation. Sorry for not making it clear
when I got the new explanation.

Thanks.

Baozi

Re: Problems when using latest git tree to boot xen on OMAP5

[Vaibhav Bedia]

Hi,

On Thu, Oct 10, 2013 at 7:36 AM, Chen Baozi <baozich@gmail.com> wrote:
>
> On Oct 10, 2013, at 7:07 PM, Julien Grall <julien.grall@linaro.org> wrote:
>
>> On 10/10/2013 09:05 AM, Chen Baozi wrote:
>>> On Wed, Oct 09, 2013 at 09:11:16AM +0100, Ian Campbell wrote:
>>>> On Wed, 2013-10-09 at 15:46 +0800, Chen Baozi wrote:
>>>>> On Mon, Oct 07, 2013 at 11:59:16AM +0100, Julien Grall wrote:
>>>>>> On 10/07/2013 10:58 AM, Chen Baozi wrote:
>>>>>>> On 10/07/2013 04:39 PM, Ian Campbell wrote:
>>>>>>>>
>>>>>>>> It is certainly a bug in the kernel if it is accessing something which
>>>>>>>> is disabled. It may also independently be a bug in the dts that this
>>>>>>>> devices is disabled.
>>>>>>>>
>>>>>>>> However in v3.12-rc4 I don't see mmc@480d1000 being disabled in
>>>>>>>> omap5-uevm.dts and I can't see anything in the history of that file
>>>>>>>> either. Where did your copy come from?
>>>>>>>
>>>>>>> I'm currently working on the "omap5-v3.11-rc3" branch from
>>>>>>> git://github.com/rogerq/linux.git, which contains a few necessary
>>>>>>> platform patches not upstreamed. In omap5-uevm.dts, there are lines like:
>>>>>>>
>>>>>>> 253 &mmc4 {
>>>>>>> 254     status = "disabled";
>>>>>>> 255 };
>>>>>>> 256
>>>>>>> 257 &mmc5 {
>>>>>>> 258     status = "disabled";
>>>>>>> 259 };
>>>>>>>
>>>>>>> the mmc4 refers to mmc@480d1000, which defines at omap5.dtsi:
>>>>>>>
>>>>>>> 417         mmc4: mmc@480d1000 {
>>>>>>>
>>>>>>> I checked Linus' mainline git tree. It is the same about disabled mmc4
>>>>>>> in omap5-uevm.dts. And the change is introduced in commit 5dd18b0 of the
>>>>>>> mainline kernel.
>>>>>>>
>>>>>>> Anyway, I'll see what exactly happened in the dom0 kernel dealing with
>>>>>>> those "disabled" regions.
>>>>>>
>>>>>> I looked at the Linux code. It will populate the different devices via
>>>>>> the of_platform_populate (drivers/of/platform.c).
>>>>>>
>>>>>> This function checks in of_platform_create_pdata if the device is
>>>>>> available. So the mmc driver (driver/mmc/host/omap_hsmmc.c) should not
>>>>>> be called for mmc4.
>>>>>
>>>>> I've discuessed this issue on linux-omap@vger.kernel.org. The TI guy says
>>>>> that "DT disabled" means that device won't be created but hwmod bus
>>>>> initially would try to initialize all supported modules by doing reset
>>>>> access in their io memory address regions. That's why dom0 have accessed
>>>>> mmc4 address even though it has been disabled.
>>>>
>>>> ePAPR lists the option of "status = fail" which is "Indicates that the
>>>> device is not operational. A serious error was detected in the device,
>>>> and it is unlikely to become operational without repair." I'm not sure
>>>> that is quite right though.
>>>>
>>>> Rather than whitelisting and mapping disabled devices through perhaps we
>>>> should implement them as read 0xf (or 0x0) and write ignore?
>>>>
>>>> Or maybe we should just be mapping non-blacklisted disabled devices
>>>> through to dom0, for it to use or ignore as it pleases. Julien, what was
>>>> the reasoning here again?
>>>
>>> If we just simple don't pass the disabled node in DT to the dom0, anything
>>> inpropriate?
>>
>> Few mails ago, you said there is hardcoded access in Linux for OMAP,
>> right? I don't think removing the node from the Device Tree will change
>> something with this problem.
>
> Oh, I just had a assumption about hardcoded stuffs then. However, after discussed
> with TI guys, it turns out I've just made a wrong assumption. There is no
> hard-coding rather a different requirement why omap_hwmod accesses the module's
> io memory space, according to their explanation. Sorry for not making it clear
> when I got the new explanation.
>

I think most of the hardcoded access has been removed. Kernel would just try to
access the modules that are listed in DT and then attemtp to do a
soft-reset to get
those modules to a known state.

I am still trying to wrap my head around how Xen works but from what i
understood
so far the soft reset in both Dom0 and DomU is the problem here?

Re: Problems when using latest git tree to boot xen on OMAP5

[Chen Baozi]

On Oct 9, 2013, at 7:15 PM, Julien Grall <julien.grall@linaro.org> wrote:

> On 10/09/2013 09:11 AM, Ian Campbell wrote:
>> Rather than whitelisting and mapping disabled devices through perhaps we
>> should implement them as read 0xf (or 0x0) and write ignore?
> 
> We can't assume that these values (0x0 or 0xf) won't affect dom0 behaviour.
> 
> What about a new quirk to map all disabled device in dom0
> (PLATFORM_QUIRK_DOM0_MAP_DISABLED_DEVICE)?

I've implemented the PLATFORM_QUIRK_DOM0_MAP_DISABLED_DEVICE quirk in my
own working tree. It seems to work very well on fixing this issue. +1 
to this idea.

Then I finished new smp initialization callbacks for omap5.

However, the new xen version still has some issues to boot dom0 on OMAP5.

Right now, Dumping cpu0 guest state shows that PC stays at __loop_delay,
while cpu1 stays at cpu_relax() line of ipi_cpu_stop(), using addr2line
to translate PC to the C code location.

Any ideas?

Cheers,

Baozi