All of lore.kernel.org
 help / color / mirror / Atom feed
* 2.6.37-git17 virtual IO boot failure
@ 2011-01-18  1:31 Anton Blanchard
  2011-01-18 22:47 ` Nishanth Aravamudan
  0 siblings, 1 reply; 14+ messages in thread
From: Anton Blanchard @ 2011-01-18  1:31 UTC (permalink / raw)
  To: nacc; +Cc: linuxppc-dev

Hi,

I was testing 2.6.37-git17 on a POWER7 with virtual IO and hit this:

Trying to unpack rootfs image as initramfs...
Freeing initrd memory: 7446k freed
vio 30000000: Warning: IOMMU dma not supported: mask
0xffffffffffffffff, table unavailable
vio 4000: Warning: IOMMU dma not supported: mask 0xffffffffffffffff,
table unavailable
vio 4001: Warning: IOMMU dma not supported: mask 0xffffffffffffffff,
table unavailable
vio 4002: Warning: IOMMU dma not supported: mask 0xffffffffffffffff,
table unavailable
vio 4004: Warning: IOMMU dma not supported: mask 0xffffffffffffffff,
table unavailable
audit: initializing netlink socket (disabled)

Haven't had a chance to look closer yet.

Anton

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: 2.6.37-git17 virtual IO boot failure
  2011-01-18  1:31 2.6.37-git17 virtual IO boot failure Anton Blanchard
@ 2011-01-18 22:47 ` Nishanth Aravamudan
  2011-01-19  0:48   ` Nishanth Aravamudan
  2011-01-19  4:06   ` Benjamin Herrenschmidt
  0 siblings, 2 replies; 14+ messages in thread
From: Nishanth Aravamudan @ 2011-01-18 22:47 UTC (permalink / raw)
  To: Anton Blanchard; +Cc: sonnyrao, miltonm, linuxppc-dev

On 18.01.2011 [12:31:52 +1100], Anton Blanchard wrote:
> Hi,
> 
> I was testing 2.6.37-git17 on a POWER7 with virtual IO and hit this:
> 
> Trying to unpack rootfs image as initramfs...
> Freeing initrd memory: 7446k freed
> vio 30000000: Warning: IOMMU dma not supported: mask
> 0xffffffffffffffff, table unavailable
> vio 4000: Warning: IOMMU dma not supported: mask 0xffffffffffffffff,
> table unavailable
> vio 4001: Warning: IOMMU dma not supported: mask 0xffffffffffffffff,
> table unavailable
> vio 4002: Warning: IOMMU dma not supported: mask 0xffffffffffffffff,
> table unavailable
> vio 4004: Warning: IOMMU dma not supported: mask 0xffffffffffffffff,
> table unavailable
> audit: initializing netlink socket (disabled)
> 
> Haven't had a chance to look closer yet.

After debugging a bit, this would appear to be due to the second hunk of
b3c73856ae47d43d0d181f9de1c1c6c0820c4515.

diff --git a/arch/powerpc/kernel/vio.c b/arch/powerpc/kernel/vio.c
index b265405..1b695fd 100644
--- a/arch/powerpc/kernel/vio.c
+++ b/arch/powerpc/kernel/vio.c
@@ -1257,6 +1257,10 @@ struct vio_dev *vio_register_device_node(struct device_node *of_node)
        viodev->dev.parent = &vio_bus_device.dev;
        viodev->dev.bus = &vio_bus_type;
        viodev->dev.release = vio_dev_release;
+        /* needed to ensure proper operation of coherent allocations
+         * later, in case driver doesn't set it explicitly */
+        dma_set_mask(&viodev->dev, DMA_BIT_MASK(64));
+        dma_set_coherent_mask(&viodev->dev, DMA_BIT_MASK(64));

        /* register with generic device framework */
        if (device_register(&viodev->dev)) {

Milton, Sonny, any thoughts?

Thanks,
Nish

-- 
Nishanth Aravamudan <nacc@us.ibm.com>
IBM Linux Technology Center

^ permalink raw reply related	[flat|nested] 14+ messages in thread

* Re: 2.6.37-git17 virtual IO boot failure
  2011-01-18 22:47 ` Nishanth Aravamudan
@ 2011-01-19  0:48   ` Nishanth Aravamudan
  2011-01-19  6:06     ` Benjamin Herrenschmidt
  2011-01-19  4:06   ` Benjamin Herrenschmidt
  1 sibling, 1 reply; 14+ messages in thread
From: Nishanth Aravamudan @ 2011-01-19  0:48 UTC (permalink / raw)
  To: Anton Blanchard; +Cc: sonnyrao, miltonm, linuxppc-dev

On 18.01.2011 [14:47:18 -0800], Nishanth Aravamudan wrote:
> On 18.01.2011 [12:31:52 +1100], Anton Blanchard wrote:
> > Hi,
> > 
> > I was testing 2.6.37-git17 on a POWER7 with virtual IO and hit this:
> > 
> > Trying to unpack rootfs image as initramfs...
> > Freeing initrd memory: 7446k freed
> > vio 30000000: Warning: IOMMU dma not supported: mask
> > 0xffffffffffffffff, table unavailable
> > vio 4000: Warning: IOMMU dma not supported: mask 0xffffffffffffffff,
> > table unavailable
> > vio 4001: Warning: IOMMU dma not supported: mask 0xffffffffffffffff,
> > table unavailable
> > vio 4002: Warning: IOMMU dma not supported: mask 0xffffffffffffffff,
> > table unavailable
> > vio 4004: Warning: IOMMU dma not supported: mask 0xffffffffffffffff,
> > table unavailable
> > audit: initializing netlink socket (disabled)
> > 
> > Haven't had a chance to look closer yet.
> 
> After debugging a bit, this would appear to be due to the second hunk of
> b3c73856ae47d43d0d181f9de1c1c6c0820c4515.
> 
> diff --git a/arch/powerpc/kernel/vio.c b/arch/powerpc/kernel/vio.c
> index b265405..1b695fd 100644
> --- a/arch/powerpc/kernel/vio.c
> +++ b/arch/powerpc/kernel/vio.c
> @@ -1257,6 +1257,10 @@ struct vio_dev *vio_register_device_node(struct device_node *of_node)
>         viodev->dev.parent = &vio_bus_device.dev;
>         viodev->dev.bus = &vio_bus_type;
>         viodev->dev.release = vio_dev_release;
> +        /* needed to ensure proper operation of coherent allocations
> +         * later, in case driver doesn't set it explicitly */
> +        dma_set_mask(&viodev->dev, DMA_BIT_MASK(64));
> +        dma_set_coherent_mask(&viodev->dev, DMA_BIT_MASK(64));
> 
>         /* register with generic device framework */
>         if (device_register(&viodev->dev)) {
> 
> Milton, Sonny, any thoughts?

A bit more detail after trying a few more kernels on the box that
originally showed the error:

1) This doesn't actually prevent booting, afaict. I think it "just"
disables DMA, which is bad, but not a boot fail, technically.

2) Reverting the above commit definitely prevents those messages.

3) I'm seeing a separate issue with 2.6.37-git17 (that's not present in
2.6.37):

sd 0:4:2:0: [sda] Aborting command: 2A
sd 0:4:2:0: Abort timed out. Resetting bus.

At which point the box locks up :)

So testing fixes is a bit of a challenge right now.

Ben, if you're ok with waiting to see if Milton or Sonny have any ideas,
I'd like to hold off on asking for a revert. In the case they do, I'll
be able to test and send out any proposed fix rapidly.

Thanks,
Nish

-- 
Nishanth Aravamudan <nacc@us.ibm.com>
IBM Linux Technology Center

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: 2.6.37-git17 virtual IO boot failure
  2011-01-18 22:47 ` Nishanth Aravamudan
  2011-01-19  0:48   ` Nishanth Aravamudan
@ 2011-01-19  4:06   ` Benjamin Herrenschmidt
  2011-01-19  4:37     ` Nishanth Aravamudan
  1 sibling, 1 reply; 14+ messages in thread
From: Benjamin Herrenschmidt @ 2011-01-19  4:06 UTC (permalink / raw)
  To: Nishanth Aravamudan; +Cc: linuxppc-dev, sonnyrao, Anton Blanchard, miltonm

On Tue, 2011-01-18 at 14:47 -0800, Nishanth Aravamudan wrote:
> On 18.01.2011 [12:31:52 +1100], Anton Blanchard wrote:
> > Hi,
> > 
> > I was testing 2.6.37-git17 on a POWER7 with virtual IO and hit this:
> > 
> > Trying to unpack rootfs image as initramfs...
> > Freeing initrd memory: 7446k freed
> > vio 30000000: Warning: IOMMU dma not supported: mask
> > 0xffffffffffffffff, table unavailable
> > vio 4000: Warning: IOMMU dma not supported: mask 0xffffffffffffffff,
> > table unavailable
> > vio 4001: Warning: IOMMU dma not supported: mask 0xffffffffffffffff,
> > table unavailable
> > vio 4002: Warning: IOMMU dma not supported: mask 0xffffffffffffffff,
> > table unavailable
> > vio 4004: Warning: IOMMU dma not supported: mask 0xffffffffffffffff,
> > table unavailable
> > audit: initializing netlink socket (disabled)
> > 
> > Haven't had a chance to look closer yet.

Well, this causes messages for vdevices that don't do DMA at all (such
as vterm etc...) and don't have the necessary properties. However, it
didn't -break- anything for me in my tests so far, just spurrious
messages. Not sure what's up with Anton's setup. Anton, can you hack the
printk to display the OF path to the device so we see what devices are
complaining ? It could be a different issue that prevents booting.

Cheers,
Ben.

> After debugging a bit, this would appear to be due to the second hunk of
> b3c73856ae47d43d0d181f9de1c1c6c0820c4515.
> 
> diff --git a/arch/powerpc/kernel/vio.c b/arch/powerpc/kernel/vio.c
> index b265405..1b695fd 100644
> --- a/arch/powerpc/kernel/vio.c
> +++ b/arch/powerpc/kernel/vio.c
> @@ -1257,6 +1257,10 @@ struct vio_dev *vio_register_device_node(struct device_node *of_node)
>         viodev->dev.parent = &vio_bus_device.dev;
>         viodev->dev.bus = &vio_bus_type;
>         viodev->dev.release = vio_dev_release;
> +        /* needed to ensure proper operation of coherent allocations
> +         * later, in case driver doesn't set it explicitly */
> +        dma_set_mask(&viodev->dev, DMA_BIT_MASK(64));
> +        dma_set_coherent_mask(&viodev->dev, DMA_BIT_MASK(64));
> 
>         /* register with generic device framework */
>         if (device_register(&viodev->dev)) {
> 
> Milton, Sonny, any thoughts?
> 
> Thanks,
> Nish
> 

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: 2.6.37-git17 virtual IO boot failure
  2011-01-19  4:06   ` Benjamin Herrenschmidt
@ 2011-01-19  4:37     ` Nishanth Aravamudan
  2011-01-19  4:54       ` Benjamin Herrenschmidt
  2011-01-29 22:22       ` Anton Blanchard
  0 siblings, 2 replies; 14+ messages in thread
From: Nishanth Aravamudan @ 2011-01-19  4:37 UTC (permalink / raw)
  To: Benjamin Herrenschmidt; +Cc: linuxppc-dev, sonnyrao, Anton Blanchard, miltonm

On 19.01.2011 [15:06:20 +1100], Benjamin Herrenschmidt wrote:
> On Tue, 2011-01-18 at 14:47 -0800, Nishanth Aravamudan wrote:
> > On 18.01.2011 [12:31:52 +1100], Anton Blanchard wrote:
> > > Hi,
> > > 
> > > I was testing 2.6.37-git17 on a POWER7 with virtual IO and hit this:
> > > 
> > > Trying to unpack rootfs image as initramfs...
> > > Freeing initrd memory: 7446k freed
> > > vio 30000000: Warning: IOMMU dma not supported: mask
> > > 0xffffffffffffffff, table unavailable
> > > vio 4000: Warning: IOMMU dma not supported: mask 0xffffffffffffffff,
> > > table unavailable
> > > vio 4001: Warning: IOMMU dma not supported: mask 0xffffffffffffffff,
> > > table unavailable
> > > vio 4002: Warning: IOMMU dma not supported: mask 0xffffffffffffffff,
> > > table unavailable
> > > vio 4004: Warning: IOMMU dma not supported: mask 0xffffffffffffffff,
> > > table unavailable
> > > audit: initializing netlink socket (disabled)
> > > 
> > > Haven't had a chance to look closer yet.
> 
> Well, this causes messages for vdevices that don't do DMA at all (such
> as vterm etc...) and don't have the necessary properties. However, it
> didn't -break- anything for me in my tests so far, just spurrious
> messages. Not sure what's up with Anton's setup. Anton, can you hack the
> printk to display the OF path to the device so we see what devices are
> complaining ? It could be a different issue that prevents booting.

Is this what you were looking for?

vio 30000000: Warning: IOMMU dma not supported: mask 0xffffffffffffffff, table unavailable
vio 30000000: Path: /vdevice/vty@30000000
vio 4000: Warning: IOMMU dma not supported: mask 0xffffffffffffffff, table unavailable
vio 4000: Path: /vdevice/IBM,sp@4000
vio 4001: Warning: IOMMU dma not supported: mask 0xffffffffffffffff, table unavailable
vio 4001: Path: /vdevice/rtc@4001
vio 4002: Warning: IOMMU dma not supported: mask 0xffffffffffffffff, table unavailable
vio 4002: Path: /vdevice/nvram@4002
vio 4004: Warning: IOMMU dma not supported: mask 0xffffffffffffffff, table unavailable
vio 4004: Path: /vdevice/gscsi@4004

FWIW, I looked at Anton's logs, and I don't think the boot failed, per
se. I think it may have timed out (but not positive on that). I was able
to boot 2.6.27-git17 on the exact same box, albeit it locks up at a
later point (the sd abort I e-mailed about in a follow-up).



> 
> Cheers,
> Ben.
> 
> > After debugging a bit, this would appear to be due to the second hunk of
> > b3c73856ae47d43d0d181f9de1c1c6c0820c4515.
> > 
> > diff --git a/arch/powerpc/kernel/vio.c b/arch/powerpc/kernel/vio.c
> > index b265405..1b695fd 100644
> > --- a/arch/powerpc/kernel/vio.c
> > +++ b/arch/powerpc/kernel/vio.c
> > @@ -1257,6 +1257,10 @@ struct vio_dev *vio_register_device_node(struct device_node *of_node)
> >         viodev->dev.parent = &vio_bus_device.dev;
> >         viodev->dev.bus = &vio_bus_type;
> >         viodev->dev.release = vio_dev_release;
> > +        /* needed to ensure proper operation of coherent allocations
> > +         * later, in case driver doesn't set it explicitly */
> > +        dma_set_mask(&viodev->dev, DMA_BIT_MASK(64));
> > +        dma_set_coherent_mask(&viodev->dev, DMA_BIT_MASK(64));
> > 
> >         /* register with generic device framework */
> >         if (device_register(&viodev->dev)) {
> > 
> > Milton, Sonny, any thoughts?
> > 
> > Thanks,
> > Nish
> > 
> 
> 

-- 
Nishanth Aravamudan <nacc@us.ibm.com>
IBM Linux Technology Center

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: 2.6.37-git17 virtual IO boot failure
  2011-01-19  4:37     ` Nishanth Aravamudan
@ 2011-01-19  4:54       ` Benjamin Herrenschmidt
  2011-01-29 22:22       ` Anton Blanchard
  1 sibling, 0 replies; 14+ messages in thread
From: Benjamin Herrenschmidt @ 2011-01-19  4:54 UTC (permalink / raw)
  To: Nishanth Aravamudan; +Cc: linuxppc-dev, sonnyrao, Anton Blanchard, miltonm

On Tue, 2011-01-18 at 20:37 -0800, Nishanth Aravamudan wrote:

> Is this what you were looking for?
> 
> vio 30000000: Warning: IOMMU dma not supported: mask 0xffffffffffffffff, table unavailable
> vio 30000000: Path: /vdevice/vty@30000000
> vio 4000: Warning: IOMMU dma not supported: mask 0xffffffffffffffff, table unavailable
> vio 4000: Path: /vdevice/IBM,sp@4000
> vio 4001: Warning: IOMMU dma not supported: mask 0xffffffffffffffff, table unavailable
> vio 4001: Path: /vdevice/rtc@4001
> vio 4002: Warning: IOMMU dma not supported: mask 0xffffffffffffffff, table unavailable
> vio 4002: Path: /vdevice/nvram@4002
> vio 4004: Warning: IOMMU dma not supported: mask 0xffffffffffffffff, table unavailable
> vio 4004: Path: /vdevice/gscsi@4004

Ok, so they are all harmess (none of those device do DMA, appart maybe
gscsi, I have no idea what it is :-)

> FWIW, I looked at Anton's logs, and I don't think the boot failed, per
> se. I think it may have timed out (but not positive on that). I was able
> to boot 2.6.27-git17 on the exact same box, albeit it locks up at a
> later point (the sd abort I e-mailed about in a follow-up).

I haven't seen your email. I'll dig. Have to run now.

Cheers,
Ben.

> 
> 
> > 
> > Cheers,
> > Ben.
> > 
> > > After debugging a bit, this would appear to be due to the second hunk of
> > > b3c73856ae47d43d0d181f9de1c1c6c0820c4515.
> > > 
> > > diff --git a/arch/powerpc/kernel/vio.c b/arch/powerpc/kernel/vio.c
> > > index b265405..1b695fd 100644
> > > --- a/arch/powerpc/kernel/vio.c
> > > +++ b/arch/powerpc/kernel/vio.c
> > > @@ -1257,6 +1257,10 @@ struct vio_dev *vio_register_device_node(struct device_node *of_node)
> > >         viodev->dev.parent = &vio_bus_device.dev;
> > >         viodev->dev.bus = &vio_bus_type;
> > >         viodev->dev.release = vio_dev_release;
> > > +        /* needed to ensure proper operation of coherent allocations
> > > +         * later, in case driver doesn't set it explicitly */
> > > +        dma_set_mask(&viodev->dev, DMA_BIT_MASK(64));
> > > +        dma_set_coherent_mask(&viodev->dev, DMA_BIT_MASK(64));
> > > 
> > >         /* register with generic device framework */
> > >         if (device_register(&viodev->dev)) {
> > > 
> > > Milton, Sonny, any thoughts?
> > > 
> > > Thanks,
> > > Nish
> > > 
> > 
> > 
> 

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: 2.6.37-git17 virtual IO boot failure
  2011-01-19  0:48   ` Nishanth Aravamudan
@ 2011-01-19  6:06     ` Benjamin Herrenschmidt
  2011-01-19 22:26       ` Nishanth Aravamudan
  0 siblings, 1 reply; 14+ messages in thread
From: Benjamin Herrenschmidt @ 2011-01-19  6:06 UTC (permalink / raw)
  To: Nishanth Aravamudan; +Cc: linuxppc-dev, sonnyrao, Anton Blanchard, miltonm

On Tue, 2011-01-18 at 16:48 -0800, Nishanth Aravamudan wrote:
> 
> Ben, if you're ok with waiting to see if Milton or Sonny have any
> ideas,
> I'd like to hold off on asking for a revert. In the case they do, I'll
> be able to test and send out any proposed fix rapidly. 

I don't believe this specific error is causing the lockup, I think we
only hit a spurrious message on devices that don't have DMA capabilities
in the first place. (But I may be wrong, I'll wait for you guys to dig
more or I'll have a look myself tomorrow if I manage to get out of
meetings).

So there's another problem with SCSI tho it -could- also be a DMA issue,
hard to tell at this point.

BTW. I'm not too happy with those defaults set to 64-bit. Probably not
an issue until your other patches go in, but some devices like veth
cannot do 64-bit DMA. I think we should default to 32-bit in the VIO
base code and explicitely enable 64-bit DMA from drivers that support it
(in theory vscsi but I haven't verified the implementation).

Cheers,
Ben.

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: 2.6.37-git17 virtual IO boot failure
  2011-01-19  6:06     ` Benjamin Herrenschmidt
@ 2011-01-19 22:26       ` Nishanth Aravamudan
  0 siblings, 0 replies; 14+ messages in thread
From: Nishanth Aravamudan @ 2011-01-19 22:26 UTC (permalink / raw)
  To: Benjamin Herrenschmidt; +Cc: linuxppc-dev, sonnyrao, Anton Blanchard, miltonm

On 19.01.2011 [17:06:18 +1100], Benjamin Herrenschmidt wrote:
> On Tue, 2011-01-18 at 16:48 -0800, Nishanth Aravamudan wrote:
> > 
> > Ben, if you're ok with waiting to see if Milton or Sonny have any
> > ideas,
> > I'd like to hold off on asking for a revert. In the case they do, I'll
> > be able to test and send out any proposed fix rapidly. 
> 
> I don't believe this specific error is causing the lockup, I think we
> only hit a spurrious message on devices that don't have DMA
> capabilities in the first place. (But I may be wrong, I'll wait for
> you guys to dig more or I'll have a look myself tomorrow if I manage
> to get out of meetings).

Yes, this seems accurate. Like I mentioned elsewhere, this box came up
ok even with these messages and seemed ok (up until the disk locked up).

> So there's another problem with SCSI tho it -could- also be a DMA issue,
> hard to tell at this point.

Right, I'm not sure how to determine that. I did see the lockup, though,
with both my patches reverted (the patches for vio, I mean, after
2.6.37)

> BTW. I'm not too happy with those defaults set to 64-bit. Probably not
> an issue until your other patches go in, but some devices like veth
> cannot do 64-bit DMA. I think we should default to 32-bit in the VIO
> base code and explicitely enable 64-bit DMA from drivers that support it
> (in theory vscsi but I haven't verified the implementation).

Ok, so change the bit-mask to 32-bit? Or would it be appropriate to
attempt 64-bit, if it fails fallback to 32-bit? Seems to be a common
pattern throughout the DMA bit-setting callers.

Thanks,
Nish

-- 
Nishanth Aravamudan <nacc@us.ibm.com>
IBM Linux Technology Center

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: 2.6.37-git17 virtual IO boot failure
  2011-01-19  4:37     ` Nishanth Aravamudan
  2011-01-19  4:54       ` Benjamin Herrenschmidt
@ 2011-01-29 22:22       ` Anton Blanchard
  2011-01-29 22:24         ` [PATCH 1/5] powerpc/numa: Only use active VPHN count fields Anton Blanchard
  1 sibling, 1 reply; 14+ messages in thread
From: Anton Blanchard @ 2011-01-29 22:22 UTC (permalink / raw)
  To: Nishanth Aravamudan; +Cc: jlarrew, linuxppc-dev


Hi,

> FWIW, I looked at Anton's logs, and I don't think the boot failed, per
> se. I think it may have timed out (but not positive on that). I was
> able to boot 2.6.27-git17 on the exact same box, albeit it locks up
> at a later point (the sd abort I e-mailed about in a follow-up).

This fail bisects down to the VPHN (shared processor affinity) patch.
I've got some fixes on the way.

Anton

^ permalink raw reply	[flat|nested] 14+ messages in thread

* [PATCH 1/5] powerpc/numa: Only use active VPHN count fields
  2011-01-29 22:22       ` Anton Blanchard
@ 2011-01-29 22:24         ` Anton Blanchard
  2011-01-29 22:26           ` [PATCH 2/5] powerpc/numa: Check for all VPHN changes Anton Blanchard
                             ` (3 more replies)
  0 siblings, 4 replies; 14+ messages in thread
From: Anton Blanchard @ 2011-01-29 22:24 UTC (permalink / raw)
  To: Nishanth Aravamudan, Benjamin Herrenschmidt, jlarrew; +Cc: linuxppc-dev


VPHN supports up to 8 distance fields but the number of entries in
ibm,associativity-reference-points signifies how many are in use.
Don't look at all the VPHN counts, only distance_ref_points_depth
worth.

Since we already cap our distance metrics at MAX_DISTANCE_REF_POINTS,
use that to size the VPHN arrays and add a BUILD_BUG_ON to avoid it growing
larger than the VPHN maximum of 8.

Signed-off-by: Anton Blanchard <anton@samba.org>
---

Index: linux-2.6/arch/powerpc/mm/numa.c
===================================================================
--- linux-2.6.orig/arch/powerpc/mm/numa.c	2011-01-29 10:48:21.280075270 +1100
+++ linux-2.6/arch/powerpc/mm/numa.c	2011-01-29 10:54:24.547203014 +1100
@@ -1291,8 +1291,7 @@ u64 memory_hotplug_max(void)
 
 /* Vrtual Processor Home Node (VPHN) support */
 #ifdef CONFIG_PPC_SPLPAR
-#define VPHN_NR_CHANGE_CTRS (8)
-static u8 vphn_cpu_change_counts[NR_CPUS][VPHN_NR_CHANGE_CTRS];
+static u8 vphn_cpu_change_counts[NR_CPUS][MAX_DISTANCE_REF_POINTS];
 static cpumask_t cpu_associativity_changes_mask;
 static int vphn_enabled;
 static void set_topology_timer(void);
@@ -1305,12 +1304,15 @@ static void setup_cpu_associativity_chan
 {
 	int cpu = 0;
 
+	/* The VPHN feature supports a maximum of 8 reference points */
+	BUILD_BUG_ON(MAX_DISTANCE_REF_POINTS > 8);
+
 	for_each_possible_cpu(cpu) {
 		int i = 0;
 		u8 *counts = vphn_cpu_change_counts[cpu];
 		volatile u8 *hypervisor_counts = lppaca[cpu].vphn_assoc_counts;
 
-		for (i = 0; i < VPHN_NR_CHANGE_CTRS; i++) {
+		for (i = 0; i < distance_ref_points_depth; i++) {
 			counts[i] = hypervisor_counts[i];
 		}
 	}
@@ -1339,7 +1341,7 @@ static int update_cpu_associativity_chan
 		u8 *counts = vphn_cpu_change_counts[cpu];
 		volatile u8 *hypervisor_counts = lppaca[cpu].vphn_assoc_counts;
 
-		for (i = 0; i < VPHN_NR_CHANGE_CTRS; i++) {
+		for (i = 0; i < distance_ref_points_depth; i++) {
 			if (hypervisor_counts[i] > counts[i]) {
 				counts[i] = hypervisor_counts[i];
 				changed = 1;

^ permalink raw reply	[flat|nested] 14+ messages in thread

* [PATCH 2/5] powerpc/numa: Check for all VPHN changes
  2011-01-29 22:24         ` [PATCH 1/5] powerpc/numa: Only use active VPHN count fields Anton Blanchard
@ 2011-01-29 22:26           ` Anton Blanchard
  2011-01-29 22:28           ` [PATCH 3/5] powerpc/numa: Add length when creating OF properties via VPHN Anton Blanchard
                             ` (2 subsequent siblings)
  3 siblings, 0 replies; 14+ messages in thread
From: Anton Blanchard @ 2011-01-29 22:26 UTC (permalink / raw)
  To: Nishanth Aravamudan, Benjamin Herrenschmidt, jlarrew; +Cc: linuxppc-dev


The hypervisor uses unsigned 1 byte counters to signal topology changes to
the OS. Since they can wrap we need to check for any difference, not just if
the hypervisor count is greater than the previous count.

Signed-off-by: Anton Blanchard <anton@samba.org>
---

Index: linux-2.6/arch/powerpc/mm/numa.c
===================================================================
--- linux-2.6.orig/arch/powerpc/mm/numa.c	2011-01-29 11:16:56.741843175 +1100
+++ linux-2.6/arch/powerpc/mm/numa.c	2011-01-29 12:44:42.059356526 +1100
@@ -1342,7 +1342,7 @@ static int update_cpu_associativity_chan
 		volatile u8 *hypervisor_counts = lppaca[cpu].vphn_assoc_counts;
 
 		for (i = 0; i < distance_ref_points_depth; i++) {
-			if (hypervisor_counts[i] > counts[i]) {
+			if (hypervisor_counts[i] != counts[i]) {
 				counts[i] = hypervisor_counts[i];
 				changed = 1;
 			}

^ permalink raw reply	[flat|nested] 14+ messages in thread

* [PATCH 3/5] powerpc/numa: Add length when creating OF properties via VPHN
  2011-01-29 22:24         ` [PATCH 1/5] powerpc/numa: Only use active VPHN count fields Anton Blanchard
  2011-01-29 22:26           ` [PATCH 2/5] powerpc/numa: Check for all VPHN changes Anton Blanchard
@ 2011-01-29 22:28           ` Anton Blanchard
  2011-01-29 22:35           ` [PATCH 4/5] powerpc/numa: Disable VPHN on dedicated processor partitions Anton Blanchard
  2011-01-29 22:37           ` [PATCH 5/5] powerpc/numa: Fix bug in unmap_cpu_from_node Anton Blanchard
  3 siblings, 0 replies; 14+ messages in thread
From: Anton Blanchard @ 2011-01-29 22:28 UTC (permalink / raw)
  To: Nishanth Aravamudan, Benjamin Herrenschmidt, jlarrew; +Cc: linuxppc-dev


The rest of the NUMA code expects an OF associativity property with
the first cell containing the length. Without this fix all topology changes
cause us to misparse the property and put the cpu into node 0.

Signed-off-by: Anton Blanchard <anton@samba.org>
---

Index: linux-2.6/arch/powerpc/mm/numa.c
===================================================================
--- linux-2.6.orig/arch/powerpc/mm/numa.c	2011-01-29 12:45:09.257654450 +1100
+++ linux-2.6/arch/powerpc/mm/numa.c	2011-01-29 12:56:30.854975882 +1100
@@ -1356,8 +1356,11 @@ static int update_cpu_associativity_chan
 	return nr_cpus;
 }
 
-/* 6 64-bit registers unpacked into 12 32-bit associativity values */
-#define VPHN_ASSOC_BUFSIZE (6*sizeof(u64)/sizeof(u32))
+/*
+ * 6 64-bit registers unpacked into 12 32-bit associativity values. To form
+ * the complete property we have to add the length in the first cell.
+ */
+#define VPHN_ASSOC_BUFSIZE (6*sizeof(u64)/sizeof(u32) + 1)
 
 /*
  * Convert the associativity domain numbers returned from the hypervisor
@@ -1373,7 +1376,7 @@ static int vphn_unpack_associativity(con
 #define VPHN_FIELD_MSB		(0x8000)
 #define VPHN_FIELD_MASK		(~VPHN_FIELD_MSB)
 
-	for (i = 0; i < VPHN_ASSOC_BUFSIZE; i++) {
+	for (i = 1; i < VPHN_ASSOC_BUFSIZE; i++) {
 		if (*field == VPHN_FIELD_UNUSED) {
 			/* All significant fields processed, and remaining
 			 * fields contain the reserved value of all 1's.
@@ -1398,6 +1401,9 @@ static int vphn_unpack_associativity(con
 		}
 	}
 
+	/* The first cell contains the length of the property */
+	unpacked[0] = nr_assoc_doms;
+
 	return nr_assoc_doms;
 }
 

^ permalink raw reply	[flat|nested] 14+ messages in thread

* [PATCH 4/5] powerpc/numa: Disable VPHN on dedicated processor partitions
  2011-01-29 22:24         ` [PATCH 1/5] powerpc/numa: Only use active VPHN count fields Anton Blanchard
  2011-01-29 22:26           ` [PATCH 2/5] powerpc/numa: Check for all VPHN changes Anton Blanchard
  2011-01-29 22:28           ` [PATCH 3/5] powerpc/numa: Add length when creating OF properties via VPHN Anton Blanchard
@ 2011-01-29 22:35           ` Anton Blanchard
  2011-01-29 22:37           ` [PATCH 5/5] powerpc/numa: Fix bug in unmap_cpu_from_node Anton Blanchard
  3 siblings, 0 replies; 14+ messages in thread
From: Anton Blanchard @ 2011-01-29 22:35 UTC (permalink / raw)
  To: Nishanth Aravamudan, Benjamin Herrenschmidt, jlarrew; +Cc: linuxppc-dev


There is no need to start up the timer and monitor topology changes on a
dedicated processor partition, so disable it.

Signed-off-by: Anton Blanchard <anton@samba.org>
---

Index: linux-2.6/arch/powerpc/mm/numa.c
===================================================================
--- linux-2.6.orig/arch/powerpc/mm/numa.c	2011-01-29 12:58:01.849279835 +1100
+++ linux-2.6/arch/powerpc/mm/numa.c	2011-01-29 12:58:02.489239819 +1100
@@ -1520,7 +1520,8 @@ int start_topology_update(void)
 {
 	int rc = 0;
 
-	if (firmware_has_feature(FW_FEATURE_VPHN)) {
+	if (firmware_has_feature(FW_FEATURE_VPHN) &&
+	    get_lppaca()->shared_proc) {
 		vphn_enabled = 1;
 		setup_cpu_associativity_change_counters();
 		init_timer_deferrable(&topology_timer);

^ permalink raw reply	[flat|nested] 14+ messages in thread

* [PATCH 5/5] powerpc/numa: Fix bug in unmap_cpu_from_node
  2011-01-29 22:24         ` [PATCH 1/5] powerpc/numa: Only use active VPHN count fields Anton Blanchard
                             ` (2 preceding siblings ...)
  2011-01-29 22:35           ` [PATCH 4/5] powerpc/numa: Disable VPHN on dedicated processor partitions Anton Blanchard
@ 2011-01-29 22:37           ` Anton Blanchard
  3 siblings, 0 replies; 14+ messages in thread
From: Anton Blanchard @ 2011-01-29 22:37 UTC (permalink / raw)
  To: Nishanth Aravamudan, Benjamin Herrenschmidt, jlarrew; +Cc: linuxppc-dev


When converting to the new cpumask code I screwed up:

-       if (cpu_isset(cpu, numa_cpumask_lookup_table[node])) {
-               cpu_clear(cpu, numa_cpumask_lookup_table[node]);
+       if (cpumask_test_cpu(cpu, node_to_cpumask_map[node])) {
+               cpumask_set_cpu(cpu, node_to_cpumask_map[node]);

This was introduced in commit 25863de07af9 (powerpc/cpumask: Convert NUMA code
to new cpumask API)

Fix it.

Signed-off-by: Anton Blanchard <anton@samba.org>
Cc: <stable@kernel.org>
---

Index: linux-2.6/arch/powerpc/mm/numa.c
===================================================================
--- linux-2.6.orig/arch/powerpc/mm/numa.c	2011-01-29 13:06:05.259039081 +1100
+++ linux-2.6/arch/powerpc/mm/numa.c	2011-01-30 08:26:48.148366563 +1100
@@ -186,7 +186,7 @@ static void unmap_cpu_from_node(unsigned
 	dbg("removing cpu %lu from node %d\n", cpu, node);
 
 	if (cpumask_test_cpu(cpu, node_to_cpumask_map[node])) {
-		cpumask_set_cpu(cpu, node_to_cpumask_map[node]);
+		cpumask_clear_cpu(cpu, node_to_cpumask_map[node]);
 	} else {
 		printk(KERN_ERR "WARNING: cpu %lu not found in node %d\n",
 		       cpu, node);

^ permalink raw reply	[flat|nested] 14+ messages in thread

end of thread, other threads:[~2011-01-29 22:37 UTC | newest]

Thread overview: 14+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2011-01-18  1:31 2.6.37-git17 virtual IO boot failure Anton Blanchard
2011-01-18 22:47 ` Nishanth Aravamudan
2011-01-19  0:48   ` Nishanth Aravamudan
2011-01-19  6:06     ` Benjamin Herrenschmidt
2011-01-19 22:26       ` Nishanth Aravamudan
2011-01-19  4:06   ` Benjamin Herrenschmidt
2011-01-19  4:37     ` Nishanth Aravamudan
2011-01-19  4:54       ` Benjamin Herrenschmidt
2011-01-29 22:22       ` Anton Blanchard
2011-01-29 22:24         ` [PATCH 1/5] powerpc/numa: Only use active VPHN count fields Anton Blanchard
2011-01-29 22:26           ` [PATCH 2/5] powerpc/numa: Check for all VPHN changes Anton Blanchard
2011-01-29 22:28           ` [PATCH 3/5] powerpc/numa: Add length when creating OF properties via VPHN Anton Blanchard
2011-01-29 22:35           ` [PATCH 4/5] powerpc/numa: Disable VPHN on dedicated processor partitions Anton Blanchard
2011-01-29 22:37           ` [PATCH 5/5] powerpc/numa: Fix bug in unmap_cpu_from_node Anton Blanchard

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.