Re: Linux 2.4.5-ac14

Re: Linux 2.4.5-ac14

[Andi Kleen]

"David S. Miller" <davem@redhat.com> writes:

> Alan Cox writes:
>  > Because right now I dont consider the 2.4.6 page cache ext2 stuff safe
>  > enough to merge. I'm letting someone else be the sucide squad.. so far it
>  > looks like it is indeed fine but I want to wait and see more yet
> 
> If it means anything it has already withstanded a few
> cerebus-->fsck_check-->cerebus rounds on machines here
> in my lab.

... it also seems to make ppc not boot anymore.

-Andi

Re: Linux 2.4.5-ac14

[Alexander Viro]

On 16 Jun 2001, Andi Kleen wrote:

> "David S. Miller" <davem@redhat.com> writes:
> 
> > Alan Cox writes:
> >  > Because right now I dont consider the 2.4.6 page cache ext2 stuff safe
> >  > enough to merge. I'm letting someone else be the sucide squad.. so far it
> >  > looks like it is indeed fine but I want to wait and see more yet
> > 
> > If it means anything it has already withstanded a few
> > cerebus-->fsck_check-->cerebus rounds on machines here
> > in my lab.
> 
> ... it also seems to make ppc not boot anymore.

Huh? By the time when ext2 directories come into the game you are _very_
far into the boot sequence. At which stage does it fail?  Does it fail
with root on minixfs/reiserfs/etc.?

Re: Linux 2.4.5-ac14

[Alexander Viro]

On 16 Jun 2001, Andi Kleen wrote:

> "David S. Miller" <davem@redhat.com> writes:
> 
> > Alan Cox writes:
> >  > Because right now I dont consider the 2.4.6 page cache ext2 stuff safe
> >  > enough to merge. I'm letting someone else be the sucide squad.. so far it
> >  > looks like it is indeed fine but I want to wait and see more yet
> > 
> > If it means anything it has already withstanded a few
> > cerebus-->fsck_check-->cerebus rounds on machines here
> > in my lab.
> 
> ... it also seems to make ppc not boot anymore.

OK, after looking at the bug report things smell very strange:

	* kernel had barfed on lookup for /dev/console.
	* kernel had found /dev - right inode number, etc.
	* read_cache_page(inode->i_mapping, n, ext2_readpage) on it
gave all-zeroes for each page within first 32Kb (size of /dev on box in
question).
	* filesystem is not corrupted.
	* all that stuff had happened with cold caches.
	* kernel was 2.4.6-pre3 + some unspecified modifications.

Very odd. Could somebody try vanilla 2.4.6-pre1 on a PPC box? I _really_
doubt that it might be an architecture-specific problem in directory
code - it would simply fail the lookup for  /dev in that case.

I'll try to find a PPC nearby, but it may be tricky on weekend. So if
somebody wants to help... Notice that problem was on read-only mount,
so it can be tested without risking fs corruption - just try to boot
with init=/bin/sh and do ls -lR, etc.

Re: Linux 2.4.5-ac14

[Marc ZYNGIER]

>>>>> "Al" == Alexander Viro <viro@math.psu.edu> writes:

Al> Very odd. Could somebody try vanilla 2.4.6-pre1 on a PPC box? I _really_
Al> doubt that it might be an architecture-specific problem in directory
Al> code - it would simply fail the lookup for  /dev in that case.

I have 2.4.6-pre3 running. Machine is a PowerMac clone with a G3 CPU.
It gets loads of bogus interrupts (known problem with this machine),
but otherwise runs fine (that is, for the time being).

maz@crisis:~$ uname -a
Linux crisis 2.4.6-pre3 #1 Sat Jun 16 01:35:36 CEST 2001 ppc unknown
maz@crisis:~$ cat /proc/cpuinfo 
processor       : 0
cpu             : 750
temperature     : 0 C
clock           : 240MHz
revision        : 2.2
bogomips        : 478.91
zero pages      : total: 0 (0Kb) current: 0 (0Kb) hits: 0/0 (0%)
machine         : Power Macintosh
motherboard     : AAPL,e407 MacRISC
memory          : 144MB
l2cr override   : 0xa5000000
pmac-generation : OldWorld

Al> I'll try to find a PPC nearby, but it may be tricky on weekend. So
Al> if somebody wants to help... Notice that problem was on read-only
Al> mount, so it can be tested without risking fs corruption - just
Al> try to boot with init=/bin/sh and do ls -lR, etc.

Been there, done that. Just works.
Would 2.4.6-pre1 be a better test ? I can dig into that if you want.

	M.
-- 
Places change, faces change. Life is so very strange.

Re: Linux 2.4.5-ac14

[Alexander Viro]

On 16 Jun 2001, Marc ZYNGIER wrote:

> >>>>> "Al" == Alexander Viro <viro@math.psu.edu> writes:
> 
> Al> Very odd. Could somebody try vanilla 2.4.6-pre1 on a PPC box? I _really_
> Al> doubt that it might be an architecture-specific problem in directory
> Al> code - it would simply fail the lookup for  /dev in that case.
> 
> I have 2.4.6-pre3 running. Machine is a PowerMac clone with a G3 CPU.
> It gets loads of bogus interrupts (known problem with this machine),
> but otherwise runs fine (that is, for the time being).

[snip]
 
> Been there, done that. Just works.
> Would 2.4.6-pre1 be a better test ? I can dig into that if you want.

Nah - just that 2.4.6-pre1 was the version where ext2 patch went in.

Re: Linux 2.4.5-ac14

[David S. Miller]

Andi Kleen writes:
 > ... it also seems to make ppc not boot anymore.

Which is odd since all my machines are big-endian too.

It might smell of a compiler bug...

Later,
David S. Miller
davem@redhat.com

pci_disable_device() vs. arch

[Benjamin Herrenschmidt]

Hi !

Would it make sense to add a 

pcibios_disable_device(pci_dev*) called from the end of 
pci_disable_device() ?

I'm adding a call to it to sungem along with other pmac stuffs
so that the chip can be properly power down (actually it's not
really powered down but unclocked) after module removal.
Of course, the arch code must be able to catch it in order to
play with the various UniNorth control bits.

Note that my current gmac driver does shut the chip down when
the interface is down, which makes it a bit more useful for
laptops as most users currently compile the driver in the kernel.

I have nothing about changing the policy if you prefer so that
users will now have to rmmod the driver once done with the
interface to save power.

Ben.

Re: pci_disable_device() vs. arch

[Jeff Garzik]

Benjamin Herrenschmidt wrote:
> 
> Hi !
> 
> Would it make sense to add a
> 
> pcibios_disable_device(pci_dev*) called from the end of
> pci_disable_device() ?
> 
> I'm adding a call to it to sungem along with other pmac stuffs
> so that the chip can be properly power down (actually it's not
> really powered down but unclocked) after module removal.
> Of course, the arch code must be able to catch it in order to
> play with the various UniNorth control bits.

What arch-specific things need to be done?

arch-specific pcibios_disable_device may be a good idea... but in this
case it sounds like you need to put #ifdef CONFIG_ALL_PPC code in
sungem.c instead, if the power-down code is specific to gmacs.


> Note that my current gmac driver does shut the chip down when
> the interface is down, which makes it a bit more useful for
> laptops as most users currently compile the driver in the kernel.

Although some drivers already do this, you really need an inactivity
timer instead of unconditionally powering-down the hardware on
dev->stop().  dhcp and other applications will often bounce the
interface...


> I have nothing about changing the policy if you prefer so that
> users will now have to rmmod the driver once done with the
> interface to save power.

For power-down specifically, you should use pci-set-power-state not
pci-disable-device.  pci_disable_device is the opposite of
pci_enable_device.  pci_enable_device not only wakes up the device, but
also assigns resources.  Which implies that pci-disable-device is
allowed to un-assign resources.  There shouldn't be a problem with a net
device doing that per se, but you should be aware of the implications.

	Jeff


-- 
Jeff Garzik      | Andre the Giant has a posse.
Building 1024    |
MandrakeSoft     |

Re: pci_disable_device() vs. arch

[Benjamin Herrenschmidt]

>What arch-specific things need to be done?

I can control the gmac cell's clock. Actually, I can do the same
with the firewire clock (the firewire cell is in the same ASIC)
and with the firewire cable power.

>arch-specific pcibios_disable_device may be a good idea... but in this
>case it sounds like you need to put #ifdef CONFIG_ALL_PPC code in
>sungem.c instead, if the power-down code is specific to gmacs.

Well... I did that in gmac.c. With the arrival of sungem, I had in
mind to avoid clobbering sungem with such things, and since we
already have arch hook for pci_enable_device(), which I use to turn
ON the chip, I though about doing the same with pci_disable_device().

(Note that I do turn it ON temporarily during PCI probe so that
the pci_dev entry exist in the kernel).

It's mostly a mater of taste, I prefered avoiding calling arch
specific routines if possible in a "generic" driver. Note that I
will probably have to do some anyway as I need some arch calls
to control the PHY reset as well.

>Although some drivers already do this, you really need an inactivity
>timer instead of unconditionally powering-down the hardware on
>dev->stop().  dhcp and other applications will often bounce the
>interface...

In that case, it's not a power down, it's just a clock control. 
Which appear to be reather safe to toggle quickly. 

>For power-down specifically, you should use pci-set-power-state not
>pci-disable-device.  pci_disable_device is the opposite of
>pci_enable_device.  pci_enable_device not only wakes up the device, but
>also assigns resources.  Which implies that pci-disable-device is
>allowed to un-assign resources.  There shouldn't be a problem with a net
>device doing that per se, but you should be aware of the implications.

I'm aware of this. Actually, pci_disable_device() doesn't de-assign
any resource, but well... I guess there should be nothing wrong about
resources beeing un-assigned and/or re-assigned when the driver is
rmmod'ed or insmod'ed back.

I agree pci_set_power_state() would be a better choice, but it would
require some arch hooks as well since this is done done via normal PCI
power management.

Since I'll have to implement the suspend() and resume() callbacks
as well, maybe  we can agree on an arch hook for pci_set_power_state().
Actually, I beleive instead of an arch callback, I'd rather see a
function pointer in the pci_bus structure of the parent bridge. Our
arch would then put the proper code for it in the pci_bus associated
with that side of UniNorth. (UniNorth has 3 host bridges).

That said, a pcibios_disable_device() hook would be logical, since
there's already one in pcibios_enable_device().

Ben.

Re: pci_disable_device() vs. arch

[Jeff Garzik]

Benjamin Herrenschmidt wrote:
> 
> >What arch-specific things need to be done?
> 
> I can control the gmac cell's clock. Actually, I can do the same
> with the firewire clock (the firewire cell is in the same ASIC)
> and with the firewire cable power.
> 
> >arch-specific pcibios_disable_device may be a good idea... but in this
> >case it sounds like you need to put #ifdef CONFIG_ALL_PPC code in
> >sungem.c instead, if the power-down code is specific to gmacs.
> 
> Well... I did that in gmac.c. With the arrival of sungem, I had in
> mind to avoid clobbering sungem with such things, and since we
> already have arch hook for pci_enable_device(), which I use to turn
> ON the chip, I though about doing the same with pci_disable_device().
> 
> (Note that I do turn it ON temporarily during PCI probe so that
> the pci_dev entry exist in the kernel).
> 
> It's mostly a mater of taste, I prefered avoiding calling arch
> specific routines if possible in a "generic" driver. Note that I
> will probably have to do some anyway as I need some arch calls
> to control the PHY reset as well.

Its not clutter -- what you are doing is hiding pieces of the driver
from the driver maintainer.  pcibios_enable_device should not be
cluttered up with such mess, too.

I point out that I recently fixed a bug where Via interrupts were being
assigned incorrectly.  If I had not done a global grep for Via
irq-related code, I would have missed the spot where the PPC code was
doing a kludge for one of the four on-board Via devices, hardcoding the
USB irq number to 11.


> >Although some drivers already do this, you really need an inactivity
> >timer instead of unconditionally powering-down the hardware on
> >dev->stop().  dhcp and other applications will often bounce the
> >interface...
> 
> In that case, it's not a power down, it's just a clock control.
> Which appear to be reather safe to toggle quickly.
> 
> >For power-down specifically, you should use pci-set-power-state not
> >pci-disable-device.  pci_disable_device is the opposite of
> >pci_enable_device.  pci_enable_device not only wakes up the device, but
> >also assigns resources.  Which implies that pci-disable-device is
> >allowed to un-assign resources.  There shouldn't be a problem with a net
> >device doing that per se, but you should be aware of the implications.
> 
> I'm aware of this. Actually, pci_disable_device() doesn't de-assign
> any resource, but well... I guess there should be nothing wrong about
> resources beeing un-assigned and/or re-assigned when the driver is
> rmmod'ed or insmod'ed back.

Correct.  If your driver uses the API correctly, then when/if we want to
mess around with hotplug resource assignment, we can un-assign resources
as we like.  Since there aren't too many users of pci_disable_device so
far, I want to make sure early adopters get it right.


> I agree pci_set_power_state() would be a better choice, but it would
> require some arch hooks as well since this is done done via normal PCI
> power management.
> 
> Since I'll have to implement the suspend() and resume() callbacks
> as well, maybe  we can agree on an arch hook for pci_set_power_state().
> Actually, I beleive instead of an arch callback, I'd rather see a
> function pointer in the pci_bus structure of the parent bridge. Our
> arch would then put the proper code for it in the pci_bus associated
> with that side of UniNorth. (UniNorth has 3 host bridges).
> 
> That said, a pcibios_disable_device() hook would be logical, since
> there's already one in pcibios_enable_device().

Can you give a -specific- example of arch code that is -not- sungem
related, but needs to occur when one powers-down a sungem MAC?

If the PM code is related to sungem, it belongs in sungem.
So far I don't see a need for arch-specific hooks anywhere...

	Jeff


-- 
Jeff Garzik      | Andre the Giant has a posse.
Building 1024    |
MandrakeSoft     |

Re: pci_disable_device() vs. arch

[Benjamin Herrenschmidt]

>
>Its not clutter -- what you are doing is hiding pieces of the driver
>from the driver maintainer.  pcibios_enable_device should not be
>cluttered up with such mess, too.

Well... pcibios_enable_device() has to at least make sure the device
gets powered up as it's powered down after PCI probe. Except if we
end up calling pci_set_power_state() to power it up early in the
sungem driver.

>I point out that I recently fixed a bug where Via interrupts were being
>assigned incorrectly.  If I had not done a global grep for Via
>irq-related code, I would have missed the spot where the PPC code was
>doing a kludge for one of the four on-board Via devices, hardcoding the
>USB irq number to 11.

Hrm... interrupt routing on some PPC-based motherboard is quite a
mess, fortunately that's not the case on pmacs. The IRQ assignement
has to be part of the arch AFAIK, only the arch knows on which
interrupt line of the controller a given chip is wired and how
interrupt controllers are cascaded.

>Correct.  If your driver uses the API correctly, then when/if we want to
>mess around with hotplug resource assignment, we can un-assign resources
>as we like.  Since there aren't too many users of pci_disable_device so
>far, I want to make sure early adopters get it right.

Well... at least with sungem, there's no such risk as the entire bus
(up to the host bridge) where it lives is internal to the UniNorth
ASIC.

>Can you give a -specific- example of arch code that is -not- sungem
>related, but needs to occur when one powers-down a sungem MAC?
>
>If the PM code is related to sungem, it belongs in sungem.
>So far I don't see a need for arch-specific hooks anywhere...

Hrm... let me try again...

Powering down individual devices can be controlled by the PCI PM
capabilities, or in some cases (at least 2 cases here on UniNorth
based pmacs) by other bits in the host bridge.

What I suggest if for pci_bus to have an optional set_power_state
function that is called when a device on that bus calls
pci_set_power_state(). This function would then be able to implement
those cases where power control is possible, while not done
via PCI PM caps.

A pci_bus structure exist for both "root" busses and busses under
PCI<->PCI bridges, so effectively, there's a pci_bus structure per
bridge (beeing host or PCI<->PCI). I beleive it makes sense for
the bridge to have a way to handle the child power state. 

Ben

Re: pci_disable_device() vs. arch

[Jeff Garzik]

Benjamin Herrenschmidt wrote:
> 
> >
> >Its not clutter -- what you are doing is hiding pieces of the driver
> >from the driver maintainer.  pcibios_enable_device should not be
> >cluttered up with such mess, too.
> 
> Well... pcibios_enable_device() has to at least make sure the device
> gets powered up as it's powered down after PCI probe. Except if we
> end up calling pci_set_power_state() to power it up early in the
> sungem driver.

huh?  pci_enable_device calls pci_set_power_state.  sungem calls
pci_enable_device.

pcibios_enable_device shouldn't have to worry about power stuff.  If it
does, you need a pcibios_set_power_state, called from
pci_set_power_state, instead.


> >I point out that I recently fixed a bug where Via interrupts were being
> >assigned incorrectly.  If I had not done a global grep for Via
> >irq-related code, I would have missed the spot where the PPC code was
> >doing a kludge for one of the four on-board Via devices, hardcoding the
> >USB irq number to 11.
> 
> Hrm... interrupt routing on some PPC-based motherboard is quite a
> mess, fortunately that's not the case on pmacs. The IRQ assignement
> has to be part of the arch AFAIK, only the arch knows on which
> interrupt line of the controller a given chip is wired and how
> interrupt controllers are cascaded.

Via is an exception


> What I suggest if for pci_bus to have an optional set_power_state
> function that is called when a device on that bus calls
> pci_set_power_state(). This function would then be able to implement
> those cases where power control is possible, while not done
> via PCI PM caps.

> A pci_bus structure exist for both "root" busses and busses under
> PCI<->PCI bridges, so effectively, there's a pci_bus structure per
> bridge (beeing host or PCI<->PCI). I beleive it makes sense for
> the bridge to have a way to handle the child power state.

Ok, agreed.  There are always gonna be special case bridges, including
(for my interest) multi-port NICs whose interfaces are presented as PCI
devices downstream from a PCI-PCI bridge.  Controlling power for these
nics is sometimes done by messing around with the PM bits on the bridge,
not on the downstream devices.

	Jeff


-- 
Jeff Garzik      | Andre the Giant has a posse.
Building 1024    |
MandrakeSoft     |

Re: pci_disable_device() vs. arch

[Benjamin Herrenschmidt]

>huh?  pci_enable_device calls pci_set_power_state.  sungem calls
>pci_enable_device.
>
>pcibios_enable_device shouldn't have to worry about power stuff.  If it
>does, you need a pcibios_set_power_state, called from
>pci_set_power_state, instead.

Right, having a hook in pci_set_power_state() handles it all. Now,
there need to be some thinking about how to actually implement it
(when to call pcibios_set_power_state inside pci_set_power_state
and what result should it returns).

I see several options:

 - Use it when the device has no PM capabilities
 - Call it first, and depending on the result code, do the normal
   PM or not, or eventually just fail.
 - Call it last/first depending if entering a low power state or
   exiting.

In my case, the device don't have PM caps, so I don't really care,
and it's the same for the other devices I have in mind that would
need it.

I still like the idea of implementing this as a function pointer
inside the pci_bus. In fact, the current implementation could be
done by filling this pointer with a default value, thus allowing
the arch to do any variation it may like by overriding this
pointer.

I keep having embedded in mind, where we have all sort of weird
designs (usually to save a few wires on a PLD) and flexibility is
fine when it doesn't add bloat. In this case, I beleive having
this notion of "pci_bus ops" for PM makes sense and is not bloat.

>Via is an exception

Yes, unless it's cascaded on a master interrupt controller. But
I agree the hard coding wasn't good, PPC is just so many different
boxes, they don't all have a nice device-tree or useable BIOS
residual datas :(

>Ok, agreed.  There are always gonna be special case bridges, including
>(for my interest) multi-port NICs whose interfaces are presented as PCI
>devices downstream from a PCI-PCI bridge.  Controlling power for these
>nics is sometimes done by messing around with the PM bits on the bridge,
>not on the downstream devices.

Ah, nice you see you agree ;)

Ben.

Re: Linux 2.4.5-ac14

[Olaf Hering]

On Sat, Jun 16, Alexander Viro wrote:

> 
> 
> On 16 Jun 2001, Andi Kleen wrote:
> 
> > "David S. Miller" <davem@redhat.com> writes:
> > 
> > > Alan Cox writes:
> > >  > Because right now I dont consider the 2.4.6 page cache ext2 stuff safe
> > >  > enough to merge. I'm letting someone else be the sucide squad.. so far it
> > >  > looks like it is indeed fine but I want to wait and see more yet
> > > 
> > > If it means anything it has already withstanded a few
> > > cerebus-->fsck_check-->cerebus rounds on machines here
> > > in my lab.
> > 
> > ... it also seems to make ppc not boot anymore.
> 
> OK, after looking at the bug report things smell very strange:
> 
> 	* kernel had barfed on lookup for /dev/console.
> 	* kernel had found /dev - right inode number, etc.
> 	* read_cache_page(inode->i_mapping, n, ext2_readpage) on it
> gave all-zeroes for each page within first 32Kb (size of /dev on box in
> question).
> 	* filesystem is not corrupted.
> 	* all that stuff had happened with cold caches.
> 	* kernel was 2.4.6-pre3 + some unspecified modifications.
> 
> Very odd. Could somebody try vanilla 2.4.6-pre1 on a PPC box? I _really_
> doubt that it might be an architecture-specific problem in directory
> code - it would simply fail the lookup for  /dev in that case.
> 
> I'll try to find a PPC nearby, but it may be tricky on weekend. So if
> somebody wants to help... Notice that problem was on read-only mount,
> so it can be tested without risking fs corruption - just try to boot
> with init=/bin/sh and do ls -lR, etc.

Al,

the pre2 didnt boot on the POWER3 SMP box. pre3 does (runs in 32bit
mode).
It could be a compiler bug because we have some trouble right now with
our toolchain on ppc32.
Right now it runs pre3 fine with a binary from the same compiler etc.

I have seen this only on that box.
Is it possible that a ppc64 writes some bogus values to the disk? I had
a ppc64 binary running but I wasnt too excited about it because it
misses some essential stuff like autofs. So I booted the 2.4.2 ppc32
kernel again with no trouble for a while..



Gruss Olaf

-- 
 $ man clone

BUGS
       Main feature not yet implemented...