Integrator PCI base dilemma

Integrator PCI base dilemma

[Linus Walleij]

When trying to convert the Integrator/AP to use Device Tree
for the PCI bridge/hose I run into these two problems:

- Unable to assign vga_base early since this should be a virtual
  address assigned really early. If I sacrifice this and try to
  just remap the stuff, I still run into:

- Unable to ioremap the PCI memory and config window because
  it is 16 MiB big. (I guess this is the problem?)

Part of the code path is called from .map_io() and basically looks
like this:

int __init pci_v3_early_init(void)
{
	iotable_init(pci_v3_io_desc, ARRAY_SIZE(pci_v3_io_desc));
	vga_base = PCI_MEMORY_VADDR;
	pci_map_io_early(__phys_to_pfn(PHYS_PCI_IO_BASE));
	return 0;
}

The VGA console seems to be the sole user (unsure what
pci_map_io_early() is for), and we like to support VGA console.

The problem is that at this point at iomap the device tree is not yet
populated.

We can attempt to do it later but then the kernel hangs when
I try to remap the two 16MiB chunks for the non-prefetched
memory and the config window.

The iomem table looks like this:

/*
 * Static mappings for the PCIv3 bridge
 *
 * e8000000	40000000	PCI memory		PHYS_PCI_MEM_BASE	(max 512M)
 * ec000000	61000000	PCI config space	PHYS_PCI_CONFIG_BASE	(max 16M)
 * fee00000	60000000	PCI IO			PHYS_PCI_IO_BASE	(max 16M)
 */
static struct map_desc pci_v3_io_desc[] __initdata __maybe_unused = {
	{
		.virtual	= (unsigned long)PCI_MEMORY_VADDR,
		.pfn		= __phys_to_pfn(PHYS_PCI_MEM_BASE),
		.length		= SZ_16M,
		.type		= MT_DEVICE
	}, {
		.virtual	= (unsigned long)PCI_CONFIG_VADDR,
		.pfn		= __phys_to_pfn(PHYS_PCI_CONFIG_BASE),
		.length		= SZ_16M,
		.type		= MT_DEVICE
	}
};

Does anyone have any hints on how to get out of this need to have the
base addresses so early?

Or am I doing something wrong when I try to push in this map
later? I tried to just ioremap() them later but that doesn't
seem to work at all. Too big?

Yours,
Linus Walleij

Integrator PCI base dilemma

[Rob Herring]

On 03/20/2013 06:15 PM, Linus Walleij wrote:
> When trying to convert the Integrator/AP to use Device Tree
> for the PCI bridge/hose I run into these two problems:
> 
> - Unable to assign vga_base early since this should be a virtual
>   address assigned really early. If I sacrifice this and try to
>   just remap the stuff, I still run into:

This could be a fixed mapping like we did for the i/o space.

> - Unable to ioremap the PCI memory and config window because
>   it is 16 MiB big. (I guess this is the problem?)
> 
> Part of the code path is called from .map_io() and basically looks
> like this:
> 
> int __init pci_v3_early_init(void)
> {
> 	iotable_init(pci_v3_io_desc, ARRAY_SIZE(pci_v3_io_desc));
> 	vga_base = PCI_MEMORY_VADDR;
> 	pci_map_io_early(__phys_to_pfn(PHYS_PCI_IO_BASE));
> 	return 0;
> }
> 
> The VGA console seems to be the sole user (unsure what
> pci_map_io_early() is for), and we like to support VGA console.

It's for the same problem you have, but for the i/o space.

> 
> The problem is that at this point at iomap the device tree is not yet
> populated.
> 
> We can attempt to do it later but then the kernel hangs when
> I try to remap the two 16MiB chunks for the non-prefetched
> memory and the config window.
> 
> The iomem table looks like this:
> 
> /*
>  * Static mappings for the PCIv3 bridge
>  *
>  * e8000000	40000000	PCI memory		PHYS_PCI_MEM_BASE	(max 512M)
>  * ec000000	61000000	PCI config space	PHYS_PCI_CONFIG_BASE	(max 16M)
>  * fee00000	60000000	PCI IO			PHYS_PCI_IO_BASE	(max 16M)
>  */
> static struct map_desc pci_v3_io_desc[] __initdata __maybe_unused = {
> 	{
> 		.virtual	= (unsigned long)PCI_MEMORY_VADDR,
> 		.pfn		= __phys_to_pfn(PHYS_PCI_MEM_BASE),
> 		.length		= SZ_16M,
> 		.type		= MT_DEVICE
> 	}, {
> 		.virtual	= (unsigned long)PCI_CONFIG_VADDR,
> 		.pfn		= __phys_to_pfn(PHYS_PCI_CONFIG_BASE),
> 		.length		= SZ_16M,
> 		.type		= MT_DEVICE
> 	}
> };
> 
> Does anyone have any hints on how to get out of this need to have the
> base addresses so early?

Wouldn't the drivers' ioremap create a mapping with the the memory
space? The only user of this mapping may be VGA.

For the config space, how much do you really need? You could ioremap
this on demand. I think this issue came up before on another platform
where the config space was very sparsely accessed.

Rob

Integrator PCI base dilemma

[Linus Walleij]

On Thu, Mar 21, 2013 at 12:54 AM, Rob Herring <robherring2@gmail.com> wrote:
> On 03/20/2013 06:15 PM, Linus Walleij wrote:

>> - Unable to assign vga_base early since this should be a virtual
>>   address assigned really early. If I sacrifice this and try to
>>   just remap the stuff, I still run into:
>
> This could be a fixed mapping like we did for the i/o space.

Hm, can you point me to the relevant topics so I can read up on it?

> Wouldn't the drivers' ioremap create a mapping with the the memory
> space? The only user of this mapping may be VGA.

Well the VGA console is supposed to be as early as possible so
in some sense it's like an earlyprint dilemma. The way the mechanism
works right now is to give a raw pointer into the remapped
PCI non-prefetched memory, then the VGA console assumes that
the VGA device will be right there, att offset zero.

(Maybe there is some guarantee that PCI VGA devices will appear
first in that memory space, or it would be hard to do stunts like
this... I admit I haven't done much of PCI in my life.)

> For the config space, how much do you really need? You could ioremap
> this on demand. I think this issue came up before on another platform
> where the config space was very sparsely accessed.

Yes I think you're right here. So there may be a way to avoid the
big chunk of 16MiB remapping and avoid using a static offset for this.

So I end up only keeping one 16MiB remapping for the memory and
another static mapping for the I/O memoryspace.

And we basically have no way to get rid of these static maps :-/

Yours,
Linus Walleij

Integrator PCI base dilemma

[Arnd Bergmann]

On Wednesday 20 March 2013, Linus Walleij wrote:
> When trying to convert the Integrator/AP to use Device Tree
> for the PCI bridge/hose I run into these two problems:
> 
> - Unable to assign vga_base early since this should be a virtual
>   address assigned really early. If I sacrifice this and try to
>   just remap the stuff, I still run into:
> 
> - Unable to ioremap the PCI memory and config window because
>   it is 16 MiB big. (I guess this is the problem?)
> 
> Part of the code path is called from .map_io() and basically looks
> like this:
> 
> int __init pci_v3_early_init(void)
> {
> 	iotable_init(pci_v3_io_desc, ARRAY_SIZE(pci_v3_io_desc));
> 	vga_base = PCI_MEMORY_VADDR;
> 	pci_map_io_early(__phys_to_pfn(PHYS_PCI_IO_BASE));
> 	return 0;
> }
> 
> The VGA console seems to be the sole user (unsure what
> pci_map_io_early() is for), and we like to support VGA console.

Yes, this is certainly required only for the VGA console

> The problem is that at this point at iomap the device tree is not yet
> populated.

I think it's fine to leave the static mapping in place for this,
you don't necessarily have to get it from DT here, since it seems
like a reasonable use of map_io().

> The iomem table looks like this:
> 
> /*
>  * Static mappings for the PCIv3 bridge
>  *
>  * e8000000	40000000	PCI memory		PHYS_PCI_MEM_BASE	(max 512M)
>  * ec000000	61000000	PCI config space	PHYS_PCI_CONFIG_BASE	(max 16M)
>  * fee00000	60000000	PCI IO			PHYS_PCI_IO_BASE	(max 16M)
>  */
> static struct map_desc pci_v3_io_desc[] __initdata __maybe_unused = {
> 	{
> 		.virtual	= (unsigned long)PCI_MEMORY_VADDR,
> 		.pfn		= __phys_to_pfn(PHYS_PCI_MEM_BASE),
> 		.length		= SZ_16M,
> 		.type		= MT_DEVICE
> 	}, {
> 		.virtual	= (unsigned long)PCI_CONFIG_VADDR,
> 		.pfn		= __phys_to_pfn(PHYS_PCI_CONFIG_BASE),
> 		.length		= SZ_16M,
> 		.type		= MT_DEVICE
> 	}
> };
> 
> Does anyone have any hints on how to get out of this need to have the
> base addresses so early?
> 
> Or am I doing something wrong when I try to push in this map
> later? I tried to just ioremap() them later but that doesn't
> seem to work at all. Too big?

I think the key is the VGA_MAP_MEM() function that can be defined in a number
of ways:

arch/alpha/include/asm/vga.h:#define VGA_MAP_MEM(x,s)   ((unsigned long) ioremap(x, s))
arch/arm/include/asm/vga.h:#define VGA_MAP_MEM(x,s)     (vga_base + (x))
arch/ia64/include/asm/vga.h:#define VGA_MAP_MEM(x,s)    ((unsigned long) ioremap_nocache(vga_console_membase + (x
arch/m32r/include/asm/vga.h:#define VGA_MAP_MEM(x,s) (unsigned long)phys_to_virt(x)
arch/mips/include/asm/vga.h:#define VGA_MAP_MEM(x, s)   (0xb0000000L + (unsigned long)(x))
arch/powerpc/include/asm/vga.h:#define VGA_MAP_MEM(x,s) ((unsigned long) ioremap((x), s))
arch/powerpc/include/asm/vga.h:#define VGA_MAP_MEM(x,s) (x + vgacon_remap_base)
arch/sparc/include/asm/vga.h:#define VGA_MAP_MEM(x,s) (x)
arch/tile/include/asm/vga.h:#define VGA_MAP_MEM(x,s)    ((unsigned long) ioremap(x, s))
arch/x86/include/asm/vga.h:#define VGA_MAP_MEM(x, s) (unsigned long)phys_to_virt(x)
arch/xtensa/include/asm/vga.h:#define VGA_MAP_MEM(x,s) (unsigned long)phys_to_virt(x)

On xtensa and x86, the low PCI memory addresses are actually mapped into the
physical address space, so they already have a mapping. It seems the same is
true on sparc, which apparently also relies on a 1:1 mapping of virtual to physical
addresses in the kernel. m32r is probably a copy-paste bug since it has no PCI.

Alpha, IA64, PPC64, and Tile use ioremap there, which seems reasonable especially
given that the at least the vga16fb assumes it can iounmap the pointer returned
by VGA_MAP_MEM when unloading. Again, I'd assume that tile is incorrectly copied from
one of the others, but it might actually work. All but ia64 here rely on the VGA
registers and memory and ROM being mapped into physical addresses 0xA0000-0xC7fff
as they are on PCs.

ARM, MIPS and PPC32 map the registers at boot time into a fixed or known virtual
address, but all implementations are really suspicious:
* On ppc32, nothing is ever assigned to vgacon_remap_base, I assume the last user
  is long gone, this may have been the PREP platform that got marked as BROKEN
  in 2005. A NULL vgacon_remap_base cannot work, since that would imply that the
  VGA memory is mapped into a low virtual address, i.e. user space.
* On MIPS, this seems to work only on the legacy RM200/300/400 platform, although
  I don't actually see where it's being mapped, and none of the platform defconfigs
  enable VGA_CONSOLE.
* On ARM, we have these definitions:

arch/arm/mach-dove/pcie.c:      vga_base = DOVE_PCIE0_MEM_PHYS_BASE;
arch/arm/mach-footbridge/dc21285.c:     vga_base = PCIMEM_BASE;
arch/arm/mach-integrator/integrator_ap.c:       vga_base = PCI_MEMORY_VADDR;
arch/arm/mach-kirkwood/pcie.c:  vga_base = KIRKWOOD_PCIE_MEM_PHYS_BASE;
arch/arm/mach-mv78xx0/pcie.c:   vga_base = MV78XX0_PCIE_MEM_PHYS_BASE;
arch/arm/mach-orion5x/pci.c:    vga_base = ORION5X_PCIE_MEM_PHYS_BASE;
arch/arm/mach-shark/pci.c:      vga_base = 0xe8000000;

Dove/integrator/kirkwood/mv78xx0/orion5x all put a *physical* address in here,
which cannot work, since that is not mapped anywhere.

Footbridge and integrator get it right, presumably because Russell was actually
using that code ;-)
Shark does not map anything to the vga_base, so that does not work. It used
to map a lot of registers, but has not mapped the first megabyte of the PCI
memory space since the start of git history.

Now what does this all tell us? First of all I think you are free to change
it in any way that does not break footbridge, since everything else is already
broken. I think the most logical way forward would be to use the same method
as ia64 and use ioremap() with a platform-specific offset, but keeping the
static mapping would probably be easier to do.

	Arnd

Integrator PCI base dilemma

[Rob Herring]

On 03/21/2013 04:16 AM, Linus Walleij wrote:
> On Thu, Mar 21, 2013 at 12:54 AM, Rob Herring <robherring2@gmail.com> wrote:
>> On 03/20/2013 06:15 PM, Linus Walleij wrote:
> 
>>> - Unable to assign vga_base early since this should be a virtual
>>>   address assigned really early. If I sacrifice this and try to
>>>   just remap the stuff, I still run into:
>>
>> This could be a fixed mapping like we did for the i/o space.
> 
> Hm, can you point me to the relevant topics so I can read up on it?

Look at pci_reserve_io, pci_map_io_early and pci_ioremap_io and the
patches that added those. The whole reason pci_map_io_early was added
was to solve exactly the same problem with some platforms needing early
access to i/o space.

> 
>> Wouldn't the drivers' ioremap create a mapping with the the memory
>> space? The only user of this mapping may be VGA.
> 
> Well the VGA console is supposed to be as early as possible so
> in some sense it's like an earlyprint dilemma. The way the mechanism
> works right now is to give a raw pointer into the remapped
> PCI non-prefetched memory, then the VGA console assumes that
> the VGA device will be right there, att offset zero.
> 
> (Maybe there is some guarantee that PCI VGA devices will appear
> first in that memory space, or it would be hard to do stunts like
> this... I admit I haven't done much of PCI in my life.)

I believe it is setup by the BIOS and fixed on x86. There's probably
some bootloader setup assumptions for ARM.

Does VGA need all 16MB as well?

> 
>> For the config space, how much do you really need? You could ioremap
>> this on demand. I think this issue came up before on another platform
>> where the config space was very sparsely accessed.
> 
> Yes I think you're right here. So there may be a way to avoid the
> big chunk of 16MiB remapping and avoid using a static offset for this.
> 
> So I end up only keeping one 16MiB remapping for the memory and
> another static mapping for the I/O memoryspace.
> 
> And we basically have no way to get rid of these static maps :-/

You don't really have to get rid of the static map necessarily. It just
needs to be more generic. If you fix the virtual address across
platforms, then you just need to get the phys address out of the DT.

Rob

Integrator PCI base dilemma

[Arnd Bergmann]

On Thursday 21 March 2013, Rob Herring wrote:
> On 03/21/2013 04:16 AM, Linus Walleij wrote:
> > On Thu, Mar 21, 2013 at 12:54 AM, Rob Herring <robherring2@gmail.com> wrote:
> >> On 03/20/2013 06:15 PM, Linus Walleij wrote:
> > 
> >>> - Unable to assign vga_base early since this should be a virtual
> >>>   address assigned really early. If I sacrifice this and try to
> >>>   just remap the stuff, I still run into:
> >>
> >> This could be a fixed mapping like we did for the i/o space.
> > 
> > Hm, can you point me to the relevant topics so I can read up on it?
> 
> Look at pci_reserve_io, pci_map_io_early and pci_ioremap_io and the
> patches that added those. The whole reason pci_map_io_early was added
> was to solve exactly the same problem with some platforms needing early
> access to i/o space.

Given that we are only talking about integrator and footbridge here,
I think adding a lot of new infrastructure is overkill. This may change
if we get new systems that would like to come up with VGA console, but
I don't see that happen at this point.

> >> Wouldn't the drivers' ioremap create a mapping with the the memory
> >> space? The only user of this mapping may be VGA.
> > 
> > Well the VGA console is supposed to be as early as possible so
> > in some sense it's like an earlyprint dilemma. The way the mechanism
> > works right now is to give a raw pointer into the remapped
> > PCI non-prefetched memory, then the VGA console assumes that
> > the VGA device will be right there, att offset zero.
> > 
> > (Maybe there is some guarantee that PCI VGA devices will appear
> > first in that memory space, or it would be hard to do stunts like
> > this... I admit I haven't done much of PCI in my life.)
> 
> I believe it is setup by the BIOS and fixed on x86. There's probably
> some bootloader setup assumptions for ARM.
> 
> Does VGA need all 16MB as well?

There is some magic ISA-compatible range that normal PCI devices stay
out of. VGA really only needs the 160KB range from bus address 0xA0000
to 0xC7FFF, see /proc/iomem on your PC.

> >> For the config space, how much do you really need? You could ioremap
> >> this on demand. I think this issue came up before on another platform
> >> where the config space was very sparsely accessed.
> > 
> > Yes I think you're right here. So there may be a way to avoid the
> > big chunk of 16MiB remapping and avoid using a static offset for this.
> > 
> > So I end up only keeping one 16MiB remapping for the memory and
> > another static mapping for the I/O memoryspace.
> > 
> > And we basically have no way to get rid of these static maps :-/
> 
> You don't really have to get rid of the static map necessarily. It just
> needs to be more generic. If you fix the virtual address across
> platforms, then you just need to get the phys address out of the DT.

Again, that would work, but out of the two platforms left with VGA console,
Integrator is the only one with DT. Is that worth it?

	Arnd

Integrator PCI base dilemma

[Russell King - ARM Linux]

Gah.  Wrap failure.

On Thu, Mar 21, 2013 at 01:02:18PM +0000, Arnd Bergmann wrote:
> Alpha, IA64, PPC64, and Tile use ioremap there, which seems reasonable
> especially given that the at least the vga16fb assumes it can iounmap
> the pointer returned by VGA_MAP_MEM when unloading.

Be. Very. Careful. with that.

Look at vgacon.  vgacon gets used on ARM.  vga16fb does not.  vgacon
does not use VGA_MAP_MEM() in a way compatible with our ioremap() -
things like the font ops will end up creating multiple mappings which
will eventually full the ioremap() space if you do go down that route.

> Again, I'd assume that tile is incorrectly copied from
> one of the others, but it might actually work. All but ia64 here rely
> on the VGA registers and memory and ROM being mapped into physical
> addresses 0xA0000-0xC7fff as they are on PCs.

That's because it's pretty much built into the way VGA crud works.
VGA BIOSes hard code these addresses into themselves.  Really, so does
the kernel, but non-native architectures are given the chance to offset
this appropriately - and remember that non-native architectures will
run the VGA BIOS via an x86 emulator, which you pretty much have to do
to get VGA cards to initialize properly.

I've tried the manual initialization with a S3 VGA card.  It was
acceptable but the black level was too high.  I could never sort that
out until I switched to running the VGA BIOS instead.

> ARM, MIPS and PPC32 map the registers at boot time into a fixed or known
> virtual address, but all implementations are really suspicious:
> * On ARM, we have these definitions:
> 
> arch/arm/mach-dove/pcie.c:      vga_base = DOVE_PCIE0_MEM_PHYS_BASE;
> arch/arm/mach-footbridge/dc21285.c:     vga_base = PCIMEM_BASE;
> arch/arm/mach-integrator/integrator_ap.c:       vga_base = PCI_MEMORY_VADDR;
> arch/arm/mach-kirkwood/pcie.c:  vga_base = KIRKWOOD_PCIE_MEM_PHYS_BASE;
> arch/arm/mach-mv78xx0/pcie.c:   vga_base = MV78XX0_PCIE_MEM_PHYS_BASE;
> arch/arm/mach-orion5x/pci.c:    vga_base = ORION5X_PCIE_MEM_PHYS_BASE;
> arch/arm/mach-shark/pci.c:      vga_base = 0xe8000000;
> 
> Dove/integrator/kirkwood/mv78xx0/orion5x all put a *physical* address in here,
> which cannot work, since that is not mapped anywhere.

Erm, no.  Look again at Integrator.  PCI_MEMORY_*V*ADDR.
arch/arm/mach-integrator/include/mach/platform.h:#define PCI_MEMORY_VADDR IOMEM(0xe8000000)

And we have a static mapping for that memory space in place - and I've
had VGA cards (Permedia3) working there when I had an AP system, so it
has worked in the past.

DC21285 works (or at least used to) because I have that on the machine
over -----> there, but its running an old kernel because both IDE and
ATA subsystems were pretty fscked last time I tried to update it, and
my network doesn't work very well without that machine working!  I do
need to try again at some point to upgrade its kernel.

> Footbridge and integrator get it right, presumably because Russell
> was actually using that code ;-)

... and for Footbridge, still do.  Remember, though that this whole
vga_base thing is a relatively recent addition due to the single zImage
project, so any mistakes with the above need proper research to see
whether they're bugs introduced by that activity.

> Now what does this all tell us? First of all I think you are free to change
> it in any way that does not break footbridge, since everything else is already
> broken.

Footbridge and Integrator/AP.

> I think the most logical way forward would be to use the same method
> as ia64 and use ioremap() with a platform-specific offset, but keeping the
> static mapping would probably be easier to do.

No, our ioremap() will definitely break here.

Integrator PCI base dilemma

[Arnd Bergmann]

On Thursday 21 March 2013, Russell King - ARM Linux wrote:
> Gah.  Wrap failure.
> 
> On Thu, Mar 21, 2013 at 01:02:18PM +0000, Arnd Bergmann wrote:
> > Alpha, IA64, PPC64, and Tile use ioremap there, which seems reasonable
> > especially given that the at least the vga16fb assumes it can iounmap
> > the pointer returned by VGA_MAP_MEM when unloading.
> 
> Be. Very. Careful. with that.
> 
> Look at vgacon.  vgacon gets used on ARM.  vga16fb does not.  vgacon
> does not use VGA_MAP_MEM() in a way compatible with our ioremap() -
> things like the font ops will end up creating multiple mappings which
> will eventually full the ioremap() space if you do go down that route.

Ah, so we have two drivers using the same interface based on conflicting
assumptions, nice.

I guess ioremap would still work on ARM  if we already have a static
mapping, but in that case there is no need to use ioremap in the first
place.

To fix this properly, we could add a matching VGA_UNMAP_MEM() interface
and use it consistently in both drivers. I don't think anyone has the
energy to do that and get it merged, but I would review patches ;-)

> > Again, I'd assume that tile is incorrectly copied from
> > one of the others, but it might actually work. All but ia64 here rely
> > on the VGA registers and memory and ROM being mapped into physical
> > addresses 0xA0000-0xC7fff as they are on PCs.
> 
> That's because it's pretty much built into the way VGA crud works.
> VGA BIOSes hard code these addresses into themselves.  Really, so does
> the kernel, but non-native architectures are given the chance to offset
> this appropriately - and remember that non-native architectures will
> run the VGA BIOS via an x86 emulator, which you pretty much have to do
> to get VGA cards to initialize properly.

Unfortunately, the offsetting also means that you have a non-zero mem_offset
value for the pci host controller, which breaks code that reads the PCI BARs
and uses those as physical addresses. I believe XFree86 traditionally did
this. I normally recommend to do PCI host bridge drivers with mem_offset=0,
meaning you have a window of valid PCI memory space addresses that match
the physical address at which they are visible to the CPU. This of course
means that VGA_MAP_MEM() cannot work, because the ISA compatible memory
space addresses end up outside of the window.

> > arch/arm/mach-dove/pcie.c:      vga_base = DOVE_PCIE0_MEM_PHYS_BASE;
> > arch/arm/mach-footbridge/dc21285.c:     vga_base = PCIMEM_BASE;
> > arch/arm/mach-integrator/integrator_ap.c:       vga_base = PCI_MEMORY_VADDR;
> > arch/arm/mach-kirkwood/pcie.c:  vga_base = KIRKWOOD_PCIE_MEM_PHYS_BASE;
> > arch/arm/mach-mv78xx0/pcie.c:   vga_base = MV78XX0_PCIE_MEM_PHYS_BASE;
> > arch/arm/mach-orion5x/pci.c:    vga_base = ORION5X_PCIE_MEM_PHYS_BASE;
> > arch/arm/mach-shark/pci.c:      vga_base = 0xe8000000;
> > 
> > Dove/integrator/kirkwood/mv78xx0/orion5x all put a *physical* address in here,
> > which cannot work, since that is not mapped anywhere.
> 
> Erm, no.  Look again at Integrator.  PCI_MEMORY_*V*ADDR.
> arch/arm/mach-integrator/include/mach/platform.h:#define PCI_MEMORY_VADDR IOMEM(0xe8000000)

Right. My brain was thinking correctly, but my fingers typed it out wrong ;-)

> > Footbridge and integrator get it right, presumably because Russell
> > was actually using that code ;-)
> 
> ... and for Footbridge, still do.  Remember, though that this whole
> vga_base thing is a relatively recent addition due to the single zImage
> project, so any mistakes with the above need proper research to see
> whether they're bugs introduced by that activity.

Yes. It took me a while yesterday to go through the history for shark, as
I feared we broke VGA_MAP_MEM during the conversion, but I'm pretty sure it
was already broken before.

> > Now what does this all tell us? First of all I think you are free to change
> > it in any way that does not break footbridge, since everything else is already
> > broken.
> 
> Footbridge and Integrator/AP.

Yes, I was assuming that Linus would keep Integrator/AP working since that is
what he uses.

> > I think the most logical way forward would be to use the same method
> > as ia64 and use ioremap() with a platform-specific offset, but keeping the
> > static mapping would probably be easier to do.
> 
> No, our ioremap() will definitely break here.

Ok, so let's keep the static mapping method.

	Arnd

Integrator PCI base dilemma

[Russell King - ARM Linux]

On Fri, Mar 22, 2013 at 09:48:27AM +0000, Arnd Bergmann wrote:
> > That's because it's pretty much built into the way VGA crud works.
> > VGA BIOSes hard code these addresses into themselves.  Really, so does
> > the kernel, but non-native architectures are given the chance to offset
> > this appropriately - and remember that non-native architectures will
> > run the VGA BIOS via an x86 emulator, which you pretty much have to do
> > to get VGA cards to initialize properly.
> 
> Unfortunately, the offsetting also means that you have a non-zero mem_offset
> value for the pci host controller, which breaks code that reads the PCI BARs
> and uses those as physical addresses.

No, you're confusing two things.

Firstly, nothing in todays kernel should read PCI BARs and treat them as
physical addresses.  Userspace is a different matter, but that is entirely
unrelated to VGA_MAP_MEM().

The non-zero mem_offset occurs if your physical address space is different
from the PCI address space.  Footbridge has this.  PCI memory address 0
is physical address 0x80000000, because all the SDRAM and DC21285
peripherals, PCI IO space etc all appear below 0x80000000.  This alone
mandates mem_offset to be 0x80000000.

You _could_ tweak a bit which will set PCI bit 31 in the address for that
window.  That gets you a 1:1 mapping.  But then the VGA addresses are
inaccessible - and as a side effect of that, XF86 will hit something else
rather than the PCI bus.

Now, we don't have to map this, but if we want VGA cards to work with
vgacon, then there is no option but to do so.  So, the first 16MB of
PCI space is statically mapped.

VGA_MAP_MEM() is then defined to be the _virtual_ address of this mapping.

If PCI and physical addresses weren't translated in this way, then mem_offset
would be zero, and VGA_MAP_MEM() would still need to have the virtual
address of the mapping.

So, the two things are _entirely_ separate.

> I believe XFree86 traditionally did
> this. I normally recommend to do PCI host bridge drivers with mem_offset=0,

I wonder how many systems you've broken by telling people to do that,
because frankly you're wrong.  See the above example on Footbridge
why you're completely wrong on this point.

And XF86 may have traditionally done this, but it will be broken on
platforms like Footbridge as long as it does this behaviour, irrespective
of the setting of mem_offset.  Also, having XF86 mess around with the
PCI configuration in ways that the kernel is not aware of has long been
viewed as a bug.

Finally, the kernel has interfaces for mapping PCI BARs, and this is the
way XF86 should now deal with VGA cards, not by mapping /dev/mem.  The
old /dev/mem interface is being deprecated for that kind of use.

> > > Now what does this all tell us? First of all I think you are free to change
> > > it in any way that does not break footbridge, since everything else is already
> > > broken.
> > 
> > Footbridge and Integrator/AP.
> 
> Yes, I was assuming that Linus would keep Integrator/AP working since that is
> what he uses.

It's entirely possible that Linus doesn't use a VGA card with Integrator/AP
if Linus is not using MILO - because MILO was the only boot loader for
that platform which had an x86 emulator built in to run the VGA BIOS to
initialize the VGA card.

There are two boot loaders I'm aware of which are capable of doing this,
and those are MILO and my boot loader for Footbridge platforms (which,
incidentally, now runs ATA card BIOSes too complete with input support
and can push out the VGA output via serial and accept keypresses via
serial) - and that doesn't work on Integrator/AP.

Integrator PCI base dilemma

[Russell King - ARM Linux]

On Thu, Mar 21, 2013 at 08:22:22AM -0500, Rob Herring wrote:
> On 03/21/2013 04:16 AM, Linus Walleij wrote:
> > Well the VGA console is supposed to be as early as possible so
> > in some sense it's like an earlyprint dilemma. The way the mechanism
> > works right now is to give a raw pointer into the remapped
> > PCI non-prefetched memory, then the VGA console assumes that
> > the VGA device will be right there, att offset zero.
> > 
> > (Maybe there is some guarantee that PCI VGA devices will appear
> > first in that memory space, or it would be hard to do stunts like
> > this... I admit I haven't done much of PCI in my life.)
> 
> I believe it is setup by the BIOS and fixed on x86. There's probably
> some bootloader setup assumptions for ARM.

All VGA cards need to be initialized before use.  On x86, that happens as
a result of the BIOS loading and running an BIOSes on any cards in the
system - starting with any VGA BIOS first.  VGA BIOSes do all sorts of
different weird stuff, so running them on non-x86 is non-trivial.  VGA
BIOSes can and do poke at other bits of PC hardware.

> Does VGA need all 16MB as well?

The only real chunk that needs to be mapped is the 640K-1MB region, which
is what x86 "removes" from system RAM and uses for VGA and BIOSes.

> >> For the config space, how much do you really need? You could ioremap
> >> this on demand. I think this issue came up before on another platform
> >> where the config space was very sparsely accessed.
> > 
> > Yes I think you're right here. So there may be a way to avoid the
> > big chunk of 16MiB remapping and avoid using a static offset for this.
> > 
> > So I end up only keeping one 16MiB remapping for the memory and
> > another static mapping for the I/O memoryspace.
> > 
> > And we basically have no way to get rid of these static maps :-/
> 
> You don't really have to get rid of the static map necessarily. It just
> needs to be more generic. If you fix the virtual address across
> platforms, then you just need to get the phys address out of the DT.

Exactly.

Integrator PCI base dilemma

[Arnd Bergmann]

On Friday 22 March 2013, Russell King - ARM Linux wrote:
> On Fri, Mar 22, 2013 at 09:48:27AM +0000, Arnd Bergmann wrote:
> > > That's because it's pretty much built into the way VGA crud works.
> > > VGA BIOSes hard code these addresses into themselves.  Really, so does
> > > the kernel, but non-native architectures are given the chance to offset
> > > this appropriately - and remember that non-native architectures will
> > > run the VGA BIOS via an x86 emulator, which you pretty much have to do
> > > to get VGA cards to initialize properly.
> > 
> > Unfortunately, the offsetting also means that you have a non-zero mem_offset
> > value for the pci host controller, which breaks code that reads the PCI BARs
> > and uses those as physical addresses.
> 
> No, you're confusing two things.
> 
> Firstly, nothing in todays kernel should read PCI BARs and treat them as
> physical addresses.  Userspace is a different matter, but that is entirely
> unrelated to VGA_MAP_MEM().

I understand the part about "should", and I would hope that there is little
code left in the kernel still doing that, but I occasionally see some of
that in old or staging drivers. Try "git grep PCI_BASE_ADDRESS" in the kernel
tree.

> The non-zero mem_offset occurs if your physical address space is different
> from the PCI address space.  Footbridge has this.  PCI memory address 0
> is physical address 0x80000000, because all the SDRAM and DC21285
> peripherals, PCI IO space etc all appear below 0x80000000.  This alone
> mandates mem_offset to be 0x80000000.
> 
> You _could_ tweak a bit which will set PCI bit 31 in the address for that
> window.  That gets you a 1:1 mapping.  But then the VGA addresses are
> inaccessible - and as a side effect of that, XF86 will hit something else
> rather than the PCI bus.
> 
> Now, we don't have to map this, but if we want VGA cards to work with
> vgacon, then there is no option but to do so.  So, the first 16MB of
> PCI space is statically mapped.

Yes, that was exactly my point.

> VGA_MAP_MEM() is then defined to be the _virtual_ address of this mapping.
> 
> If PCI and physical addresses weren't translated in this way, then mem_offset
> would be zero, and VGA_MAP_MEM() would still need to have the virtual
> address of the mapping.
> 
> So, the two things are _entirely_ separate.

My point was that you cannot have a virtual address for the window if
low PCI bus addresses are not even visible in the physical address
space. You could theoretically have some of the low 16MB of physical
address space map to the PCI bus as done on x86, but I'm not aware
of any ARM platforms doing that. This leaves you the choice between
two reasonable values for mem_offset:

a) the start of the physical address space window, mapping pci
   bus address 0. This allows you to provide a meaningful VGA_MAP_MEM
   because the low bus addresses can be mapped into the virtual
   address space, but breaks any code that makes incorrect assumptions
   about mem_offset=0.

b) set mem_offset=0 and have an identity mapped PCI bus window, which
   is what people generally expect after reading the PCI specifications,
   but which breaks the VGA console.

> > I believe XFree86 traditionally did
> > this. I normally recommend to do PCI host bridge drivers with mem_offset=0,
> 
> I wonder how many systems you've broken by telling people to do that,
> because frankly you're wrong.  See the above example on Footbridge
> why you're completely wrong on this point.

It's not completely wrong, it's picking one of two evils. VGA console
is the only thing that needs the low PCI address mapping, and if you
have any more recent GPU, you can use fbcon with a hardware specific
driver that uses the normal PCI BARs.

Also, in many cases, you don't even have the choice in the kernel between
the two options, either PCI is identity mapped or mapped starting at
address zero based on whatever the soc or board designer thought, and the
only thing we can do is tell the kernel what the hardware does.

It looks like cns3xxx, tegra, versatile and ks8695 are all hardwired to a
zero mem_offset, while dc21285, integrator, it8152 and sa1100/pci-nanoengine
map the PCI bus starting at bus address 0.

plat-orion and plat-iop are both configurable and have always been
configured to use mem_offset=0 as far as I can tell. I assume ixp4xx
also falls into this category.

> And XF86 may have traditionally done this, but it will be broken on
> platforms like Footbridge as long as it does this behaviour, irrespective
> of the setting of mem_offset.  Also, having XF86 mess around with the
> PCI configuration in ways that the kernel is not aware of has long been
> viewed as a bug.
> 
> Finally, the kernel has interfaces for mapping PCI BARs, and this is the
> way XF86 should now deal with VGA cards, not by mapping /dev/mem.  The
> old /dev/mem interface is being deprecated for that kind of use.

I'm pretty sure that Xorg gets it right these days, I was just using it
as an example of code in wide use that would not work on footbridge.

	Arnd

Integrator PCI base dilemma

[Russell King - ARM Linux]

On Fri, Mar 22, 2013 at 11:52:07AM +0000, Arnd Bergmann wrote:
> On Friday 22 March 2013, Russell King - ARM Linux wrote:
> > On Fri, Mar 22, 2013 at 09:48:27AM +0000, Arnd Bergmann wrote:
> > > > That's because it's pretty much built into the way VGA crud works.
> > > > VGA BIOSes hard code these addresses into themselves.  Really, so does
> > > > the kernel, but non-native architectures are given the chance to offset
> > > > this appropriately - and remember that non-native architectures will
> > > > run the VGA BIOS via an x86 emulator, which you pretty much have to do
> > > > to get VGA cards to initialize properly.
> > > 
> > > Unfortunately, the offsetting also means that you have a non-zero mem_offset
> > > value for the pci host controller, which breaks code that reads the PCI BARs
> > > and uses those as physical addresses.
> > 
> > No, you're confusing two things.
> > 
> > Firstly, nothing in todays kernel should read PCI BARs and treat them as
> > physical addresses.  Userspace is a different matter, but that is entirely
> > unrelated to VGA_MAP_MEM().
> 
> I understand the part about "should", and I would hope that there is little
> code left in the kernel still doing that, but I occasionally see some of
> that in old or staging drivers. Try "git grep PCI_BASE_ADDRESS" in the kernel
> tree.

Well, anything which does stuff like this:

            for (i = 0xC8000; i < 0xE8000; i += 0x4000) {
                ha->brd = ioremap(i, sizeof(u16));
                pci_write_config_dword(pdev, PCI_BASE_ADDRESS_0, i);

is broken for a whole load of reasons not just because of the write to
the PCI configuration register.  The same probably applies elsewhere.

> It's not completely wrong, it's picking one of two evils. VGA console
> is the only thing that needs the low PCI address mapping, and if you
> have any more recent GPU, you can use fbcon with a hardware specific
> driver that uses the normal PCI BARs.

I'm afraid that is not entirely true.  Many VGA cards require access
to the VGA window to access MMIO registers to tell them what to do...
And remember my earlier point - you must have something to run the
VGA BIOS to setup the VGA card, and it _will_ want to initialize it
in VGA mode using the VGA window.  So if you can't access that on
your PCI bus, you can't ever have a VGA card plugged in.

> Also, in many cases, you don't even have the choice in the kernel between
> the two options, either PCI is identity mapped or mapped starting at
> address zero based on whatever the soc or board designer thought, and the
> only thing we can do is tell the kernel what the hardware does.

Exactly which is why you're wrong. :)  If the hardware enforces a
separation of the physical address space from the PCI address space
by way of an offset translation, then you have no option but to have
a non-zero mem_offset.

> It looks like cns3xxx, tegra, versatile and ks8695 are all hardwired to a
> zero mem_offset, while dc21285, integrator, it8152 and sa1100/pci-nanoengine
> map the PCI bus starting at bus address 0.

Don't read too much into the versatile stuff.  All that is very very much
untested and fragile.  It's fairly buggered at the moment as it stands,
and as far as I can tell there's absolutely no way to know what the right
solution is - because the ARM documentation is extremely poor and all the
real hardware has been stuffed through the mincer.

So that leaves cns3xxx, tegra and ks8695 being the non-plat ones which
are "different" from the majority on ARM.

Integrator PCI base dilemma

[Arnd Bergmann]

On Friday 22 March 2013, Russell King - ARM Linux wrote:
> > Also, in many cases, you don't even have the choice in the kernel between
> > the two options, either PCI is identity mapped or mapped starting at
> > address zero based on whatever the soc or board designer thought, and the
> > only thing we can do is tell the kernel what the hardware does.
> 
> Exactly which is why you're wrong. :)  If the hardware enforces a
> separation of the physical address space from the PCI address space
> by way of an offset translation, then you have no option but to have
> a non-zero mem_offset.

I still think we are in complete agreement about the facts here, at least
I was trying to say the same thing above that you replied with.

> > It looks like cns3xxx, tegra, versatile and ks8695 are all hardwired to a
> > zero mem_offset, while dc21285, integrator, it8152 and sa1100/pci-nanoengine
> > map the PCI bus starting at bus address 0.
> 
> Don't read too much into the versatile stuff.  All that is very very much
> untested and fragile.  It's fairly buggered at the moment as it stands,
> and as far as I can tell there's absolutely no way to know what the right
> solution is - because the ARM documentation is extremely poor and all the
> real hardware has been stuffed through the mincer.
> 
> So that leaves cns3xxx, tegra and ks8695 being the non-plat ones which
> are "different" from the majority on ARM.

Well, those and all the ones that chose to set up their windows in the
same way like the plat-orion based ones.

	Arnd

Integrator PCI base dilemma

[Jason Gunthorpe]

On Fri, Mar 22, 2013 at 12:34:59PM +0000, Arnd Bergmann wrote:

> > So that leaves cns3xxx, tegra and ks8695 being the non-plat ones which
> > are "different" from the majority on ARM.
> 
> Well, those and all the ones that chose to set up their windows in the
> same way like the plat-orion based ones.

The orion stuff can create a second mapping into PCI address space
that can remap and generate VGA compatible bus addresses. That
techinque would duplicate how x86 handles this stuff these days.

Things are more complex on PCI-E. There is no subtractive decode in
PCI-E so you either need the magic window like x86 uses, or the bridge
windows and apertures have to be set 'just so'.

Further, there is danger with using an aperture offset. At the PCI bus
level there is only one address space. Creating an overlap of system
DRAM addresses and PCI device BARs within the PCI bus address space
could corrupt DMAs to the overlapped region in some cases - eg systems
that support peer-peer PCI-E.

For these reasons it is probably a good guideline that PCI-E systems
should strive to have a 1:1 mapping of their MMIO aperture..

Cheers,
Jason

Integrator PCI base dilemma

[Russell King - ARM Linux]

On Fri, Mar 22, 2013 at 12:33:20PM -0600, Jason Gunthorpe wrote:
> On Fri, Mar 22, 2013 at 12:34:59PM +0000, Arnd Bergmann wrote:
> 
> > > So that leaves cns3xxx, tegra and ks8695 being the non-plat ones which
> > > are "different" from the majority on ARM.
> > 
> > Well, those and all the ones that chose to set up their windows in the
> > same way like the plat-orion based ones.
> 
> The orion stuff can create a second mapping into PCI address space
> that can remap and generate VGA compatible bus addresses. That
> techinque would duplicate how x86 handles this stuff these days.
> 
> Things are more complex on PCI-E. There is no subtractive decode in
> PCI-E so you either need the magic window like x86 uses, or the bridge
> windows and apertures have to be set 'just so'.

That is no different to platforms like the footbridge.  We have to
place system memory above the VGA window to ensure that everything
works as it should.

Integrator PCI base dilemma

[Linus Walleij]

On Fri, Mar 22, 2013 at 11:37 AM, Russell King - ARM Linux
<linux@arm.linux.org.uk> wrote:

> On Fri, Mar 22, 2013 at 09:48:27AM +0000, Arnd Bergmann wrote:
>> Yes, I was assuming that Linus would keep Integrator/AP working since that is
>> what he uses.
>
> It's entirely possible that Linus doesn't use a VGA card with Integrator/AP
> if Linus is not using MILO - because MILO was the only boot loader for
> that platform which had an x86 emulator built in to run the VGA BIOS to
> initialize the VGA card.

You are right, or, well, I have had an S3 VGA card plugged in and been
scratching my head wondering how I should get the damned thing to
work.

Now I *know* there is not really a way for me to make this work
with the kernel only.

> There are two boot loaders I'm aware of which are capable of doing this,
> and those are MILO and my boot loader for Footbridge platforms (which,
> incidentally, now runs ATA card BIOSes too complete with input support
> and can push out the VGA output via serial and accept keypresses via
> serial) - and that doesn't work on Integrator/AP.

I'm on U-boot for this thing, I wonder if that can do x86
emulation to initialize VGA card. I guess I'm about to learn.

Yours,
Linus Walleij

Integrator PCI base dilemma

[Arnd Bergmann]

On Friday 22 March 2013, Linus Walleij wrote:
> > There are two boot loaders I'm aware of which are capable of doing this,
> > and those are MILO and my boot loader for Footbridge platforms (which,
> > incidentally, now runs ATA card BIOSes too complete with input support
> > and can push out the VGA output via serial and accept keypresses via
> > serial) - and that doesn't work on Integrator/AP.
> 
> I'm on U-boot for this thing, I wonder if that can do x86
> emulation to initialize VGA card. I guess I'm about to learn.

You should be able to do it from user space using Matthew Garrett's
vbetool[1]. If I understood it right, you can initialize the VGA
card with this, and then load vga16fb or vgacon.

If the card is VESA 2.0 compatible, you should also be able to use
it with uvesafb, which calls /sbin/v86d for x86 emulation.

	Arnd

[1] http://www.codon.org.uk/~mjg59/vbetool/

Integrator PCI base dilemma

[Wolfgang Denk]

Dear Linus Walleij,

In message <CACRpkdb5Q+EtM0GGaLFNUj1A12zVrcyUxzRvfEOpcH4-pVuasA@mail.gmail.com> you wrote:
>
> I'm on U-boot for this thing, I wonder if that can do x86
> emulation to initialize VGA card. I guess I'm about to learn.

Guess what the code in  drivers/bios_emulator  might be good for...

Best regards,

Wolfgang Denk

-- 
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd at denx.de
... The things love can drive a man to -- the  ecstasies,  the  mise-
ries,  the broken rules, the desperate chances, the glorious failures
and the glorious victories.
	-- McCoy, "Requiem for Methuselah", stardate 5843.7

Integrator PCI base dilemma

[Linus Walleij]

On Fri, Mar 22, 2013 at 10:13 PM, Wolfgang Denk <wd@denx.de> wrote:
> Dear Linus Walleij,
>
> In message <CACRpkdb5Q+EtM0GGaLFNUj1A12zVrcyUxzRvfEOpcH4-pVuasA@mail.gmail.com> you wrote:
>>
>> I'm on U-boot for this thing, I wonder if that can do x86
>> emulation to initialize VGA card. I guess I'm about to learn.
>
> Guess what the code in  drivers/bios_emulator  might be good for...

Thanks Wolfgang!

I hacked around a bit and actually got it to compile for ARM.

Now I just need to write an S3 driver or find an ATI Radeon
card since that seems to be supported already.

Yours,
Linus Walleij

Integrator PCI base dilemma

[Russell King - ARM Linux]

On Fri, Mar 22, 2013 at 11:35:54PM +0100, Linus Walleij wrote:
> On Fri, Mar 22, 2013 at 10:13 PM, Wolfgang Denk <wd@denx.de> wrote:
> > Dear Linus Walleij,
> >
> > In message <CACRpkdb5Q+EtM0GGaLFNUj1A12zVrcyUxzRvfEOpcH4-pVuasA@mail.gmail.com> you wrote:
> >>
> >> I'm on U-boot for this thing, I wonder if that can do x86
> >> emulation to initialize VGA card. I guess I'm about to learn.
> >
> > Guess what the code in  drivers/bios_emulator  might be good for...
> 
> Thanks Wolfgang!
> 
> I hacked around a bit and actually got it to compile for ARM.
> 
> Now I just need to write an S3 driver or find an ATI Radeon
> card since that seems to be supported already.

Okay, what follows is based on my latest version of the Footbridge
"bios" boot loader - which is tested to run the VGA BIOS on several
VGA cards, and the ATA RAID BIOS on an IT8212 card - including the
config stuff on that ATA RAID BIOS card.

You shouldn't need an "S3" driver.  The way I do it on Footbridge, and
the way I made the Integrator Milo do it was to basically setup the PCI
bus, find if there was a VGA card there, load the VGA BIOS into the
x86 emulation address 0xc0000 - where it would be loaded on a PC.

Beware though - the S3 BIOS will try to access the i8253 timer which
needs to be emulated, address 0x61 which can be simply discarded, maybe
the PCI config IO addresses 0xcf8..0xcff (which need to generate real
PCI config cycles), read from IO port 0x80 (which can simply return 0).

It will also want to access PCI IO addresses in addition to the normal
VGA IO address range of 0x3b0..0x3e0:
 0x8180-0x8200 (Streams processor), 0xff00-0xff44 (LPB bus),
 0x100, 0x[489abe][268ae]e8

And it will want PCI memory addresses 0xa0000-0xbffff to be accessible
at that x86 emulation offset.

As I said previously, different VGA BIOS prod different bits of PC
hardware in random ways, so I don't guarantee that the above will work
for your S3 BIOS.  Mine is a S3 Trio64 V2 card:

00:09.0 VGA compatible controller: S3 Inc. 86c775/86c785 [Trio 64V2/DX or /GX] (rev 16) (prog-if 00 [VGA])
which has IDs 5333:8901.

Integrator PCI base dilemma

[Wolfgang Denk]

Dear Linus,

In message <CACRpkdY_qj7VBKBSCsK3mnx=V5DURY-g1fHGVwDBz2WUheB=Nw@mail.gmail.com> you wrote:
>
> >> I'm on U-boot for this thing, I wonder if that can do x86
> >> emulation to initialize VGA card. I guess I'm about to learn.
> >
> > Guess what the code in  drivers/bios_emulator  might be good for...
> 
> Thanks Wolfgang!
> 
> I hacked around a bit and actually got it to compile for ARM.

If you have any patches, I'd appreciate if you could share these with
us.

Thanks in advance.

Best regards,

Wolfgang Denk

-- 
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd at denx.de
"It's when they say 2 + 2 = 5 that I begin to argue."    - Eric Pepke