kbuild now support arch/$ARCH/include - time for ARCHs to convert

kbuild now support arch/$ARCH/include - time for ARCHs to convert

[Sam Ravnborg]

The traditional location of the arch specific Makefiles
has been at:

	include/asm-$ARCH

But as suggested by several people over time and lately
by Linus in the following post:
http://kerneltrap.org/mailarchive/linux-kernel/2008/5/21/1903924

is is suggested to move the arch specific files under arch/
and for the header files this is arch/$ARCH/include.

So I have hacked up support for this in kbuild so
for the simpler cases a simple move of header files as
in:
	git mv include/asm-$ARCH arch/$ARCH/include/asm

is enough.

If Linus will accept it then I think -rc1 is a perfect
time to do it as people has less stuff pending touching
the header files that are moved.
Validation is easy - does the usual configs build?

For the archs that plays games with symlinks this is a
splendid opportunity to get rid af these.
The receipe is simple:
Use 
	KBUILD_CFLAGS += -Iarch/$ARCH/include/$CPU

to select the CPU and then do a copy of the header files
like this:

	git mv include/asm-$ARCH/$CPU arch/$ARCH/include/$CPU/cpu

Then the code can continue to use:

	#include <cpu/foo.h>

I already converted sparc to use arch/sparc/include -
while I merged the header files for sparc and sparc64.
This change is already upstream.

And for SH I posted a receipe which you can read here:
http://lkml.org/lkml/2008/7/28/83

It is SH specific but anyway pretty generic.

I am limited in time and resources at the moment but I
would be glad to help out as time permits should there
be any questions.

	Sam

Re: kbuild now support arch/$ARCH/include - time for ARCHs to convert

[Kyle McMartin]

On Mon, Jul 28, 2008 at 10:05:00PM +0200, Sam Ravnborg wrote:
> I am limited in time and resources at the moment but I
> would be glad to help out as time permits should there
> be any questions.
> 

Just a quick one... I moved the files on parisc, but found we're still
using include/asm-$foo/asm-offsets.h. A quick glance at Kbuild seems to
indicate so. Is this expected?

cheers, Kyle

Re: kbuild now support arch/$ARCH/include - time for ARCHs to convert

[Sam Ravnborg]

On Mon, Jul 28, 2008 at 11:06:47PM -0400, Kyle McMartin wrote:
> On Mon, Jul 28, 2008 at 10:05:00PM +0200, Sam Ravnborg wrote:
> > I am limited in time and resources at the moment but I
> > would be glad to help out as time permits should there
> > be any questions.
> > 
> 
> Just a quick one... I moved the files on parisc, but found we're still
> using include/asm-$foo/asm-offsets.h. A quick glance at Kbuild seems to
> indicate so. Is this expected?
Yes.

asm-offsets.h is a generated file and I wanted to avoid adding
generated files to arch/$ARCH/include.

It was also a less intrusive patch to keep it at the old location.
But I recall that it troubles me a bit before I got it
correct but the kbuild changes all take care of this.

	Sam

Re: kbuild now support arch/$ARCH/include - time for ARCHs to convert

[Kyle McMartin]

On Tue, Jul 29, 2008 at 05:27:26AM +0200, Sam Ravnborg wrote:
> On Mon, Jul 28, 2008 at 11:06:47PM -0400, Kyle McMartin wrote:
> > On Mon, Jul 28, 2008 at 10:05:00PM +0200, Sam Ravnborg wrote:
> > > I am limited in time and resources at the moment but I
> > > would be glad to help out as time permits should there
> > > be any questions.
> > > 
> > 
> > Just a quick one... I moved the files on parisc, but found we're still
> > using include/asm-$foo/asm-offsets.h. A quick glance at Kbuild seems to
> > indicate so. Is this expected?
> Yes.
> 
> asm-offsets.h is a generated file and I wanted to avoid adding
> generated files to arch/$ARCH/include.
> 
> It was also a less intrusive patch to keep it at the old location.
> But I recall that it troubles me a bit before I got it
> correct but the kbuild changes all take care of this.
> 

Thanks for clarifying, Sam.

cheers, Kyle

Re: kbuild now support arch/$ARCH/include - time for ARCHs to convert

[Greg Ungerer]

Hi Sam,

Sam Ravnborg wrote:
> The traditional location of the arch specific Makefiles
> has been at:
> 
> 	include/asm-$ARCH
> 
> But as suggested by several people over time and lately
> by Linus in the following post:
> http://kerneltrap.org/mailarchive/linux-kernel/2008/5/21/1903924
> 
> is is suggested to move the arch specific files under arch/
> and for the header files this is arch/$ARCH/include.
> 
> So I have hacked up support for this in kbuild so
> for the simpler cases a simple move of header files as
> in:
> 	git mv include/asm-$ARCH arch/$ARCH/include/asm
> 
> is enough.
> 
> If Linus will accept it then I think -rc1 is a perfect
> time to do it as people has less stuff pending touching
> the header files that are moved.
> Validation is easy - does the usual configs build?
> 
> For the archs that plays games with symlinks this is a
> splendid opportunity to get rid af these.
> The receipe is simple:
> Use 
> 	KBUILD_CFLAGS += -Iarch/$ARCH/include/$CPU
> 
> to select the CPU and then do a copy of the header files
> like this:
> 
> 	git mv include/asm-$ARCH/$CPU arch/$ARCH/include/$CPU/cpu
> 
> Then the code can continue to use:
> 
> 	#include <cpu/foo.h>
> 
> I already converted sparc to use arch/sparc/include -
> while I merged the header files for sparc and sparc64.
> This change is already upstream.
> 
> And for SH I posted a receipe which you can read here:
> http://lkml.org/lkml/2008/7/28/83
> 
> It is SH specific but anyway pretty generic.
> 
> I am limited in time and resources at the moment but I
> would be glad to help out as time permits should there
> be any questions.

I just tried moving the include/asm-m68knommu directory
to arch/m68knommu/include/asm and it all worked fine.
Clean compile first go :-)  Nice.  (As noted
asm-offsets.h was still generated in incldue/asm-m68knommu
afterwards though).

So what is the status, should I prepare git/patch
for Linus?

Regards
Greg


------------------------------------------------------------------------
Greg Ungerer  --  Chief Software Dude       EMAIL:     gerg@snapgear.com
Secure Computing Corporation                PHONE:       +61 7 3435 2888
825 Stanley St,                             FAX:         +61 7 3891 3630
Woolloongabba, QLD, 4102, Australia         WEB: http://www.SnapGear.com

Re: kbuild now support arch/$ARCH/include - time for ARCHs to convert

[Arnd Bergmann]

On Wednesday 30 July 2008, Greg Ungerer wrote:
> I just tried moving the include/asm-m68knommu directory
> to arch/m68knommu/include/asm and it all worked fine.
> Clean compile first go :-)  Nice.  (As noted
> asm-offsets.h was still generated in incldue/asm-m68knommu
> afterwards though).

I think the more interesting aspect in m68knommu is what happens
when asm-m68k gets moved. Does that work as well?

I think the best way to deal with it is to remove all files
that just include an asm-m68k file with the same name, and
add -Iarch/m68k/include to your CFLAGS. Not sure about 'make
headers_install', if there is an easy way to install the
files from each of the two directories correctly.

	Arnd <><

Re: kbuild now support arch/$ARCH/include - time for ARCHs to convert

[Sam Ravnborg]

On Wed, Jul 30, 2008 at 04:49:08PM +1000, Greg Ungerer wrote:
> Hi Sam,
> 
> Sam Ravnborg wrote:
> >The traditional location of the arch specific Makefiles
> >has been at:
> >
> >	include/asm-$ARCH
> >
> >But as suggested by several people over time and lately
> >by Linus in the following post:
> >http://kerneltrap.org/mailarchive/linux-kernel/2008/5/21/1903924
> >
> >is is suggested to move the arch specific files under arch/
> >and for the header files this is arch/$ARCH/include.
> >
> >So I have hacked up support for this in kbuild so
> >for the simpler cases a simple move of header files as
> >in:
> >	git mv include/asm-$ARCH arch/$ARCH/include/asm
> >
> >is enough.
> >
> >If Linus will accept it then I think -rc1 is a perfect
> >time to do it as people has less stuff pending touching
> >the header files that are moved.
> >Validation is easy - does the usual configs build?
> >
> >For the archs that plays games with symlinks this is a
> >splendid opportunity to get rid af these.
> >The receipe is simple:
> >Use 
> >	KBUILD_CFLAGS += -Iarch/$ARCH/include/$CPU
> >
> >to select the CPU and then do a copy of the header files
> >like this:
> >
> >	git mv include/asm-$ARCH/$CPU arch/$ARCH/include/$CPU/cpu
> >
> >Then the code can continue to use:
> >
> >	#include <cpu/foo.h>
> >
> >I already converted sparc to use arch/sparc/include -
> >while I merged the header files for sparc and sparc64.
> >This change is already upstream.
> >
> >And for SH I posted a receipe which you can read here:
> >http://lkml.org/lkml/2008/7/28/83
> >
> >It is SH specific but anyway pretty generic.
> >
> >I am limited in time and resources at the moment but I
> >would be glad to help out as time permits should there
> >be any questions.
> 
> I just tried moving the include/asm-m68knommu directory
> to arch/m68knommu/include/asm and it all worked fine.
> Clean compile first go :-)  Nice.  (As noted
> asm-offsets.h was still generated in incldue/asm-m68knommu
> afterwards though).
> 
> So what is the status, should I prepare git/patch
> for Linus?
Yes please do so.

Then we can address m68k and fix the breakage that
the sparc changes introduced.

	Sam

Re: kbuild now support arch/$ARCH/include - time for ARCHs to convert

[Sam Ravnborg]

On Wed, Jul 30, 2008 at 09:07:24AM +0200, Arnd Bergmann wrote:
> On Wednesday 30 July 2008, Greg Ungerer wrote:
> > I just tried moving the include/asm-m68knommu directory
> > to arch/m68knommu/include/asm and it all worked fine.
> > Clean compile first go :-)  Nice.  (As noted
> > asm-offsets.h was still generated in incldue/asm-m68knommu
> > afterwards though).
> 
> I think the more interesting aspect in m68knommu is what happens
> when asm-m68k gets moved. Does that work as well?
> 
> I think the best way to deal with it is to remove all files
> that just include an asm-m68k file with the same name, and
> add -Iarch/m68k/include to your CFLAGS. Not sure about 'make
> headers_install', if there is an easy way to install the
> files from each of the two directories correctly.
We had ALTARCH that I just killed as sparc was the only user.
And if we need something again I will try to come up
with something better somehow.

But m68knommu does no export any files atm,
and m68k exports only a single file.
So I doubt it is actually used by gcc/glibc atm.

	Sam

Re: kbuild now support arch/$ARCH/include - time for ARCHs to convert

[David Howells]

Arnd Bergmann <arnd@arndb.de> wrote:

> I think the more interesting aspect in m68knommu is what happens
> when asm-m68k gets moved. Does that work as well?

Can m68knommu be merged into m68k and be made contingent on CONFIG_MMU?  This
is what FRV does.

David

Re: kbuild now support arch/$ARCH/include - time for ARCHs to convert

[Geert Uytterhoeven]

On Wed, 30 Jul 2008, David Howells wrote:
> Arnd Bergmann <arnd@arndb.de> wrote:
> 
> > I think the more interesting aspect in m68knommu is what happens
> > when asm-m68k gets moved. Does that work as well?
> 
> Can m68knommu be merged into m68k and be made contingent on CONFIG_MMU?  This
> is what FRV does.

Sure it can ;-)

Actually I did a bit of work to add nommu support to m68k to make it
boot on old MMU-less Amigas, but it dates back to the 2.6.8.1 era...

The bigger issue is in those parts that are done differently by m68k and
m68knommu (esp. head.S).

Who's gonna take the work? :-)

Gr{oetje,eeting}s,

						Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
							    -- Linus Torvalds

Re: kbuild now support arch/$ARCH/include - time for ARCHs to convert

[Paul Mundt]

On Wed, Jul 30, 2008 at 02:26:04PM +0200, Geert Uytterhoeven wrote:
> On Wed, 30 Jul 2008, David Howells wrote:
> > Arnd Bergmann <arnd@arndb.de> wrote:
> > 
> > > I think the more interesting aspect in m68knommu is what happens
> > > when asm-m68k gets moved. Does that work as well?
> > 
> > Can m68knommu be merged into m68k and be made contingent on CONFIG_MMU?  This
> > is what FRV does.
> 
> Sure it can ;-)
> 
> Actually I did a bit of work to add nommu support to m68k to make it
> boot on old MMU-less Amigas, but it dates back to the 2.6.8.1 era...
> 
> The bigger issue is in those parts that are done differently by m68k and
> m68knommu (esp. head.S).
> 
> Who's gonna take the work? :-)
> 
This sort of merging is fairly easy to do incrementally, at least by
someone that has both platforms available. An easy first step would be to
move the m68knommu stuff that are implemented differently over as _nommu
files, similar to how the _32/_64 renames happened, and then set about
converging the two little by little. This is what I've been doing with
sh64->sh integration for example, despite fundamental differences like
having a totally different register layout and instruction set.

Re: kbuild now support arch/$ARCH/include - time for ARCHs to convert

[Geert Uytterhoeven]

On Thu, 31 Jul 2008, Paul Mundt wrote:
> On Wed, Jul 30, 2008 at 02:26:04PM +0200, Geert Uytterhoeven wrote:
> > On Wed, 30 Jul 2008, David Howells wrote:
> > > Arnd Bergmann <arnd@arndb.de> wrote:
> > > 
> > > > I think the more interesting aspect in m68knommu is what happens
> > > > when asm-m68k gets moved. Does that work as well?
> > > 
> > > Can m68knommu be merged into m68k and be made contingent on CONFIG_MMU?  This
> > > is what FRV does.
> > 
> > Sure it can ;-)
> > 
> > Actually I did a bit of work to add nommu support to m68k to make it
> > boot on old MMU-less Amigas, but it dates back to the 2.6.8.1 era...
> > 
> > The bigger issue is in those parts that are done differently by m68k and
> > m68knommu (esp. head.S).
> > 
> > Who's gonna take the work? :-)
> > 
> This sort of merging is fairly easy to do incrementally, at least by
> someone that has both platforms available. An easy first step would be to

Indeed. I have none of the currently supported m68knommu platforms,
though.
Don't know about Greg and m68k platforms (of course he can use ARAnyM
;-).

> move the m68knommu stuff that are implemented differently over as _nommu
> files, similar to how the _32/_64 renames happened, and then set about
> converging the two little by little. This is what I've been doing with
> sh64->sh integration for example, despite fundamental differences like
> having a totally different register layout and instruction set.

IC.

Gr{oetje,eeting}s,

						Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
							    -- Linus Torvalds

Re: kbuild now support arch/$ARCH/include - time for ARCHs to convert

[Greg Ungerer]

Geert Uytterhoeven wrote:
> On Thu, 31 Jul 2008, Paul Mundt wrote:
>> On Wed, Jul 30, 2008 at 02:26:04PM +0200, Geert Uytterhoeven wrote:
>>> On Wed, 30 Jul 2008, David Howells wrote:
>>>> Arnd Bergmann <arnd@arndb.de> wrote:
>>>>
>>>>> I think the more interesting aspect in m68knommu is what happens
>>>>> when asm-m68k gets moved. Does that work as well?
>>>> Can m68knommu be merged into m68k and be made contingent on CONFIG_MMU?  This
>>>> is what FRV does.
>>> Sure it can ;-)
>>>
>>> Actually I did a bit of work to add nommu support to m68k to make it
>>> boot on old MMU-less Amigas, but it dates back to the 2.6.8.1 era...
>>>
>>> The bigger issue is in those parts that are done differently by m68k and
>>> m68knommu (esp. head.S).
>>>
>>> Who's gonna take the work? :-)
>>>
>> This sort of merging is fairly easy to do incrementally, at least by
>> someone that has both platforms available. An easy first step would be to
> 
> Indeed. I have none of the currently supported m68knommu platforms,
> though.
> Don't know about Greg and m68k platforms (of course he can use ARAnyM
> ;-).

I don't have any real m68k (with MMU) hardware ;-)
I do have a v4e ColdFire with MMU, but that is not supported
by mainline m68k code. Maybe a help...


>> move the m68knommu stuff that are implemented differently over as _nommu
>> files, similar to how the _32/_64 renames happened, and then set about
>> converging the two little by little. This is what I've been doing with
>> sh64->sh integration for example, despite fundamental differences like
>> having a totally different register layout and instruction set.
> 
> IC.

Probably not hugely difficult to merge together. Quite a few of the
m68knommu files came from m68k with only a little change and a lot
of removal. The board/platform stuff is obviously quite different -
very little (if any) commonality.

Just needs someone with time to sit down and do it and clean up
the resulting mess.

Regards
Greg


------------------------------------------------------------------------
Greg Ungerer  --  Chief Software Dude       EMAIL:     gerg@snapgear.com
SnapGear -- a Secure Computing Company      PHONE:       +61 7 3435 2888
825 Stanley St,                             FAX:         +61 7 3891 3630
Woolloongabba, QLD, 4102, Australia         WEB: http://www.SnapGear.com

Re: kbuild now support arch/$ARCH/include - time for ARCHs to convert

[Ralf Baechle]

On Mon, Jul 28, 2008 at 10:05:00PM +0200, Sam Ravnborg wrote:

> The traditional location of the arch specific Makefiles
> has been at:
> 
> 	include/asm-$ARCH
> 
> But as suggested by several people over time and lately
> by Linus in the following post:
> http://kerneltrap.org/mailarchive/linux-kernel/2008/5/21/1903924
> 
> is is suggested to move the arch specific files under arch/
> and for the header files this is arch/$ARCH/include.
> 
> So I have hacked up support for this in kbuild so
> for the simpler cases a simple move of header files as
> in:
> 	git mv include/asm-$ARCH arch/$ARCH/include/asm
> 
> is enough.
> 
> If Linus will accept it then I think -rc1 is a perfect
> time to do it as people has less stuff pending touching
> the header files that are moved.
> Validation is easy - does the usual configs build?
> 
> For the archs that plays games with symlinks this is a
> splendid opportunity to get rid af these.
> The receipe is simple:
> Use 
> 	KBUILD_CFLAGS += -Iarch/$ARCH/include/$CPU
> 
> to select the CPU and then do a copy of the header files
> like this:
> 
> 	git mv include/asm-$ARCH/$CPU arch/$ARCH/include/$CPU/cpu
> 
> Then the code can continue to use:
> 
> 	#include <cpu/foo.h>
> 
> I already converted sparc to use arch/sparc/include -
> while I merged the header files for sparc and sparc64.
> This change is already upstream.
> 
> And for SH I posted a receipe which you can read here:
> http://lkml.org/lkml/2008/7/28/83
> 
> It is SH specific but anyway pretty generic.
> 
> I am limited in time and resources at the moment but I
> would be glad to help out as time permits should there
> be any questions.

I just did the necessary changes for MIPS and the first testbuild is running
successfully so far.

One of the ugly things in the MIPS makefiles is the use of platform-
specific headers in include/asm-mips/mach-* which contain platform specific
headers.  Ideally those should be included through something like
<mach/somefile.h> or <plat/something.h> and I think a directory like
arch/$ARCH/$PLATFORM/include/plat/foobar.h could be a good place in the
new scheme?  I assume other platforms have similar issues with subplatforms,
processor sub-architectures etc.?

  Ralf

Re: kbuild now support arch/$ARCH/include - time for ARCHs to convert

[Arnd Bergmann]

On Thursday 31 July 2008, Paul Mundt wrote:
> This sort of merging is fairly easy to do incrementally, at least by
> someone that has both platforms available. An easy first step would be to
> move the m68knommu stuff that are implemented differently over as _nommu
> files, similar to how the _32/_64 renames happened, and then set about
> converging the two little by little. This is what I've been doing with
> sh64->sh integration for example, despite fundamental differences like
> having a totally different register layout and instruction set.
 
I've played around with the purely mechanical side of merging the
m68k{,nommu} header files and came up with a script to do it.
All headers that are identical are used from their m68k variant, the
others get an #include magic like x86 has it for 32/64.

This assumes that the kernel ABI is compatible and that user space
can actually use the m68k headers for m68knommu as well.

Signed-off-by: Arnd Bergmann <arnd@arndb.de>
---
TARGET=arch/m68k/include/asm
SOURCE1=include/asm-m68k
SOURCE2=include/asm-m68knommu
FILES="atomic.h bitops.h bootinfo.h bug.h bugs.h byteorder.h cache.h \
cacheflush.h checksum.h cputime.h current.h delay.h div64.h \
dma-mapping.h dma.h elf.h entry.h fb.h fpu.h hardirq.h hw_irq.h io.h \
irq.h kmap_types.h local.h machdep.h mc146818rtc.h mmu.h mmu_context.h \
module.h page.h page_offset.h param.h pci.h percpu.h pgalloc.h \
pgtable.h processor.h ptrace.h scatterlist.h sections.h segment.h \
setup.h sigcontext.h siginfo.h signal.h string.h system.h \
thread_info.h timex.h tlbflush.h traps.h uaccess.h ucontext.h \
unaligned.h unistd.h"
EXPORTS="byteorder param ptrace sigcontext siginfo signal unistd"

mergefile() {
	BASE=${1%.h}
	mv $SOURCE1/$1 $TARGET/${BASE}_mmu.h
	mv $SOURCE2/$1 $TARGET/${BASE}_nommu.h
cat << EOF > $TARGET/$1
#ifdef __KERNEL__
#ifdef CONFIG_MMU
#include "${BASE}_mmu.h"
#else
#include "${BASE}_nommu.h"
#endif
#else
#include "${BASE}_mmu.h"
#endif
EOF
}

mergefile2() {
	if [ -e ${TARGET}/$1 ] ; then
		rm $SOURCE2/$1
	else
		mv $SOURCE2/$1 ${TARGET}
	fi
}

addexport() {
	echo "unifdef-y += ${1}_mmu.h" >> $TARGET/Kbuild
}

mkdir -p ${TARGET}
# first the incompatible files
for i in $FILES ; do mergefile $i ; done
# move all the remaining m68k files
mv include/asm-m68k/* arch/m68k/include/asm/
# add a few files that are present only on nommu,
# remove the duplicate nommu files.
for i in `ls include/asm-m68knommu/` ; do mergefile2 $i ; done
# make sure the new files get exported where needed.
for i in $EXPORTS ; do addexport $i ; done

Re: kbuild now support arch/$ARCH/include - time for ARCHs to convert

[Sam Ravnborg]

On Thu, Jul 31, 2008 at 09:47:25PM +0200, Arnd Bergmann wrote:
> On Thursday 31 July 2008, Paul Mundt wrote:
> > This sort of merging is fairly easy to do incrementally, at least by
> > someone that has both platforms available. An easy first step would be to
> > move the m68knommu stuff that are implemented differently over as _nommu
> > files, similar to how the _32/_64 renames happened, and then set about
> > converging the two little by little. This is what I've been doing with
> > sh64->sh integration for example, despite fundamental differences like
> > having a totally different register layout and instruction set.
>  
> I've played around with the purely mechanical side of merging the
> m68k{,nommu} header files and came up with a script to do it.
> All headers that are identical are used from their m68k variant, the
> others get an #include magic like x86 has it for 32/64.
> 
> This assumes that the kernel ABI is compatible and that user space
> can actually use the m68k headers for m68knommu as well.
> 
> Signed-off-by: Arnd Bergmann <arnd@arndb.de>
> ---
> TARGET=arch/m68k/include/asm
> SOURCE1=include/asm-m68k
> SOURCE2=include/asm-m68knommu
> FILES="atomic.h bitops.h bootinfo.h bug.h bugs.h byteorder.h cache.h \
> cacheflush.h checksum.h cputime.h current.h delay.h div64.h \
> dma-mapping.h dma.h elf.h entry.h fb.h fpu.h hardirq.h hw_irq.h io.h \
> irq.h kmap_types.h local.h machdep.h mc146818rtc.h mmu.h mmu_context.h \
> module.h page.h page_offset.h param.h pci.h percpu.h pgalloc.h \
> pgtable.h processor.h ptrace.h scatterlist.h sections.h segment.h \
> setup.h sigcontext.h siginfo.h signal.h string.h system.h \
> thread_info.h timex.h tlbflush.h traps.h uaccess.h ucontext.h \
> unaligned.h unistd.h"
> EXPORTS="byteorder param ptrace sigcontext siginfo signal unistd"
> 
> mergefile() {
> 	BASE=${1%.h}
> 	mv $SOURCE1/$1 $TARGET/${BASE}_mmu.h
> 	mv $SOURCE2/$1 $TARGET/${BASE}_nommu.h
> cat << EOF > $TARGET/$1
> #ifdef __KERNEL__
> #ifdef CONFIG_MMU
> #include "${BASE}_mmu.h"
> #else
> #include "${BASE}_nommu.h"
> #endif
> #else
> #include "${BASE}_mmu.h"
> #endif
> EOF
> }

This will leak a CONFIG_ symbol if the header file
is exported. Can we do it with a gcc defined symbol?
[See how I did it for sparc for instance]

> 
> mergefile2() {
> 	if [ -e ${TARGET}/$1 ] ; then
> 		rm $SOURCE2/$1
> 	else
> 		mv $SOURCE2/$1 ${TARGET}
> 	fi
> }
> 
> addexport() {
> 	echo "unifdef-y += ${1}_mmu.h" >> $TARGET/Kbuild
> }
unifdef is deprecated. Plese use "header-y".
They have same effect these days.

	Sam

Re: kbuild now support arch/$ARCH/include - time for ARCHs to convert

[Sam Ravnborg]

On Thu, Jul 31, 2008 at 07:23:15PM +0100, Ralf Baechle wrote:
> On Mon, Jul 28, 2008 at 10:05:00PM +0200, Sam Ravnborg wrote:
> 
> > The traditional location of the arch specific Makefiles
> > has been at:
> > 
> > 	include/asm-$ARCH
> > 
> > But as suggested by several people over time and lately
> > by Linus in the following post:
> > http://kerneltrap.org/mailarchive/linux-kernel/2008/5/21/1903924
> > 
> > is is suggested to move the arch specific files under arch/
> > and for the header files this is arch/$ARCH/include.
> > 
> > So I have hacked up support for this in kbuild so
> > for the simpler cases a simple move of header files as
> > in:
> > 	git mv include/asm-$ARCH arch/$ARCH/include/asm
> > 
> > is enough.
> > 
> > If Linus will accept it then I think -rc1 is a perfect
> > time to do it as people has less stuff pending touching
> > the header files that are moved.
> > Validation is easy - does the usual configs build?
> > 
> > For the archs that plays games with symlinks this is a
> > splendid opportunity to get rid af these.
> > The receipe is simple:
> > Use 
> > 	KBUILD_CFLAGS += -Iarch/$ARCH/include/$CPU
> > 
> > to select the CPU and then do a copy of the header files
> > like this:
> > 
> > 	git mv include/asm-$ARCH/$CPU arch/$ARCH/include/$CPU/cpu
> > 
> > Then the code can continue to use:
> > 
> > 	#include <cpu/foo.h>
> > 
> > I already converted sparc to use arch/sparc/include -
> > while I merged the header files for sparc and sparc64.
> > This change is already upstream.
> > 
> > And for SH I posted a receipe which you can read here:
> > http://lkml.org/lkml/2008/7/28/83
> > 
> > It is SH specific but anyway pretty generic.
> > 
> > I am limited in time and resources at the moment but I
> > would be glad to help out as time permits should there
> > be any questions.
> 
> I just did the necessary changes for MIPS and the first testbuild is running
> successfully so far.
> 
> One of the ugly things in the MIPS makefiles is the use of platform-
> specific headers in include/asm-mips/mach-* which contain platform specific
> headers.  Ideally those should be included through something like
> <mach/somefile.h> or <plat/something.h> and I think a directory like
> arch/$ARCH/$PLATFORM/include/plat/foobar.h could be a good place in the
> new scheme?  I assume other platforms have similar issues with subplatforms,
> processor sub-architectures etc.?

I like it. And rmk suggested something similar so I dragged you into
that thread (on linux-arm-kernel). Let's continue there.

	Sam

Re: kbuild now support arch/$ARCH/include - time for ARCHs to convert

[Greg Ungerer]

Hi Sam,

Sam Ravnborg wrote:
> On Wed, Jul 30, 2008 at 04:49:08PM +1000, Greg Ungerer wrote:
>> Hi Sam,
>>
>> Sam Ravnborg wrote:
>>> The traditional location of the arch specific Makefiles
>>> has been at:
>>>
>>> 	include/asm-$ARCH
>>>
>>> But as suggested by several people over time and lately
>>> by Linus in the following post:
>>> http://kerneltrap.org/mailarchive/linux-kernel/2008/5/21/1903924
>>>
>>> is is suggested to move the arch specific files under arch/
>>> and for the header files this is arch/$ARCH/include.
>>>
>>> So I have hacked up support for this in kbuild so
>>> for the simpler cases a simple move of header files as
>>> in:
>>> 	git mv include/asm-$ARCH arch/$ARCH/include/asm
>>>
>>> is enough.
>>>
>>> If Linus will accept it then I think -rc1 is a perfect
>>> time to do it as people has less stuff pending touching
>>> the header files that are moved.
>>> Validation is easy - does the usual configs build?
>>>
>>> For the archs that plays games with symlinks this is a
>>> splendid opportunity to get rid af these.
>>> The receipe is simple:
>>> Use 
>>> 	KBUILD_CFLAGS += -Iarch/$ARCH/include/$CPU
>>>
>>> to select the CPU and then do a copy of the header files
>>> like this:
>>>
>>> 	git mv include/asm-$ARCH/$CPU arch/$ARCH/include/$CPU/cpu
>>>
>>> Then the code can continue to use:
>>>
>>> 	#include <cpu/foo.h>
>>>
>>> I already converted sparc to use arch/sparc/include -
>>> while I merged the header files for sparc and sparc64.
>>> This change is already upstream.
>>>
>>> And for SH I posted a receipe which you can read here:
>>> http://lkml.org/lkml/2008/7/28/83
>>>
>>> It is SH specific but anyway pretty generic.
>>>
>>> I am limited in time and resources at the moment but I
>>> would be glad to help out as time permits should there
>>> be any questions.
>> I just tried moving the include/asm-m68knommu directory
>> to arch/m68knommu/include/asm and it all worked fine.
>> Clean compile first go :-)  Nice.  (As noted
>> asm-offsets.h was still generated in incldue/asm-m68knommu
>> afterwards though).
>>
>> So what is the status, should I prepare git/patch
>> for Linus?
> Yes please do so.

The move change can be pulled from:

git://git.kernel.org/pub/scm/linux/kernel/git/gerg/m68knommu.git for-linus

If there is no complaints I'll send Linus a pull request in a
couple of days time.

Regards
Greg



------------------------------------------------------------------------
Greg Ungerer  --  Chief Software Dude       EMAIL:     gerg@snapgear.com
Secure Computing Corporation                PHONE:       +61 7 3435 2888
825 Stanley St,                             FAX:         +61 7 3891 3630
Woolloongabba, QLD, 4102, Australia         WEB: http://www.SnapGear.com

Re: kbuild now support arch/$ARCH/include - time for ARCHs to convert

[Arnd Bergmann]

On Thursday 31 July 2008, Sam Ravnborg wrote:
> > #ifdef __KERNEL__
> > #ifdef CONFIG_MMU
> > #include "${BASE}_mmu.h"
> > #else
> > #include "${BASE}_nommu.h"
> > #endif
> > #else
> > #include "${BASE}_mmu.h"
> > #endif
> > EOF
> > }
> 
> This will leak a CONFIG_ symbol if the header file
> is exported. Can we do it with a gcc defined symbol?
> [See how I did it for sparc for instance]

Is there a compiler defined symbol for this? I thought the tool
chain was identical. Do you think my #ifdef __KERNEL__ does
not do what I wanted it to do? It should make sure that
user space always sees the _mmu variant.

	Arnd <><

Re: kbuild now support arch/$ARCH/include - time for ARCHs to convert

[Martin Schwidefsky]

On Mon, 2008-07-28 at 22:05 +0200, Sam Ravnborg wrote:
> So I have hacked up support for this in kbuild so
> for the simpler cases a simple move of header files as
> in:
> 	git mv include/asm-$ARCH arch/$ARCH/include/asm
> 
> is enough.

Just tested it on s390. We need a tiny fix for the qeth driver then it
works just fine (qeth has two includes for include/asm-s390 instead of
include/asm). I'll queue the qeth fix on git390. The only question is
who will do the move? 

-- 
blue skies,
  Martin.

"Reality continues to ruin my life." - Calvin.

Re: kbuild now support arch/$ARCH/include - time for ARCHs to convert

[Sam Ravnborg]

On Fri, Aug 01, 2008 at 09:35:04AM +0200, Arnd Bergmann wrote:
> On Thursday 31 July 2008, Sam Ravnborg wrote:
> > > #ifdef __KERNEL__
> > > #ifdef CONFIG_MMU
> > > #include "${BASE}_mmu.h"
> > > #else
> > > #include "${BASE}_nommu.h"
> > > #endif
> > > #else
> > > #include "${BASE}_mmu.h"
> > > #endif
> > > EOF
> > > }
> > 
> > This will leak a CONFIG_ symbol if the header file
> > is exported. Can we do it with a gcc defined symbol?
> > [See how I did it for sparc for instance]
> 
> Is there a compiler defined symbol for this? I thought the tool
> chain was identical.

> Do you think my #ifdef __KERNEL__ does
> not do what I wanted it to do?
Was too quick here - I overlooked the ifdef so it looks good.

	Sam

Re: kbuild now support arch/$ARCH/include - time for ARCHs to convert

[Sam Ravnborg]

On Fri, Aug 01, 2008 at 10:35:12AM +0200, Martin Schwidefsky wrote:
> On Mon, 2008-07-28 at 22:05 +0200, Sam Ravnborg wrote:
> > So I have hacked up support for this in kbuild so
> > for the simpler cases a simple move of header files as
> > in:
> > 	git mv include/asm-$ARCH arch/$ARCH/include/asm
> > 
> > is enough.
> 
> Just tested it on s390. We need a tiny fix for the qeth driver then it
> works just fine (qeth has two includes for include/asm-s390 instead of
> include/asm). I'll queue the qeth fix on git390. The only question is
> who will do the move? 

You will do so and ask Linus to pull it. This most go via the
arch maintainer.
And I'm glad that no other changes were needed.

	Sam

Re: kbuild now support arch/$ARCH/include - time for ARCHs to convert

[Geert Uytterhoeven]

On Fri, 1 Aug 2008, Greg Ungerer wrote:
> Sam Ravnborg wrote:
> > On Wed, Jul 30, 2008 at 04:49:08PM +1000, Greg Ungerer wrote:
> > > Sam Ravnborg wrote:
> > > > The traditional location of the arch specific Makefiles
> > > > has been at:
> > > > 
> > > > 	include/asm-$ARCH
> > > > 
> > > > But as suggested by several people over time and lately
> > > > by Linus in the following post:
> > > > http://kerneltrap.org/mailarchive/linux-kernel/2008/5/21/1903924
> > > > 
> > > > is is suggested to move the arch specific files under arch/
> > > > and for the header files this is arch/$ARCH/include.
> > > > 
> > > > So I have hacked up support for this in kbuild so
> > > > for the simpler cases a simple move of header files as
> > > > in:
> > > > 	git mv include/asm-$ARCH arch/$ARCH/include/asm
> > > > 
> > > > is enough.
> > > > 
> > > > If Linus will accept it then I think -rc1 is a perfect
> > > > time to do it as people has less stuff pending touching
> > > > the header files that are moved.
> > > > Validation is easy - does the usual configs build?
> > > > 
> > > > For the archs that plays games with symlinks this is a
> > > > splendid opportunity to get rid af these.
> > > > The receipe is simple:
> > > > Use 	KBUILD_CFLAGS += -Iarch/$ARCH/include/$CPU
> > > > 
> > > > to select the CPU and then do a copy of the header files
> > > > like this:
> > > > 
> > > > 	git mv include/asm-$ARCH/$CPU arch/$ARCH/include/$CPU/cpu
> > > > 
> > > > Then the code can continue to use:
> > > > 
> > > > 	#include <cpu/foo.h>
> > > > 
> > > > I already converted sparc to use arch/sparc/include -
> > > > while I merged the header files for sparc and sparc64.
> > > > This change is already upstream.
> > > > 
> > > > And for SH I posted a receipe which you can read here:
> > > > http://lkml.org/lkml/2008/7/28/83
> > > > 
> > > > It is SH specific but anyway pretty generic.
> > > > 
> > > > I am limited in time and resources at the moment but I
> > > > would be glad to help out as time permits should there
> > > > be any questions.
> > > I just tried moving the include/asm-m68knommu directory
> > > to arch/m68knommu/include/asm and it all worked fine.
> > > Clean compile first go :-)  Nice.  (As noted
> > > asm-offsets.h was still generated in incldue/asm-m68knommu
> > > afterwards though).
> > > 
> > > So what is the status, should I prepare git/patch
> > > for Linus?
> > Yes please do so.
> 
> The move change can be pulled from:
> 
> git://git.kernel.org/pub/scm/linux/kernel/git/gerg/m68knommu.git for-linus
> 
> If there is no complaints I'll send Linus a pull request in a
> couple of days time.

I did:

    mkdir arch/m68k/include
    mv include/asm-m68k arch/m68k/include/asm

and m68k still builds fine (I applied Stephen's patch for the
asm-sparc inclusion problem for Sun 3/3x first, of course).

As (a) I'm not using git (yet) for merging with Linus, and (b) this will
cause some breakage in the m68knommu tree anyway, perhaps it's easiest if Greg
takes care of it in m68knommu.git?

Greg, is that OK for you?
You can add my Acked-by: Geert Uytterhoeven <geert@linux-m68k.org>

There seem to be 3 classes of m68knommu header files:
  1. Plain files (some of them may be (nearly) identical copies of the
     m68k variants,
  2. Files that just include the m68k variant,
  3. Files that include the m68k variant and do something more (pci.h and
     setup.h).

Since I don't think we want to do the m68k/m68knommu merge right now
(Sorry Arnd, I'll keep your script in mind anyway!), the simplest way is to:
  - Remove all files from class 2, and add to the Makefile:

	ccflags-y += -Iarch/m68k/include

  - Include verbatim the contents of the m68k variant for class 3, cfr.
    Stephen patch for the asm-sparc inclusion problem for Sun 3/3x).

Do you agree?

Thanks!

Gr{oetje,eeting}s,

						Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
							    -- Linus Torvalds

Re: kbuild now support arch/$ARCH/include - time for ARCHs to convert

[Greg Ungerer]

Hi Geert,

Geert Uytterhoeven wrote:
> On Fri, 1 Aug 2008, Greg Ungerer wrote:
>> Sam Ravnborg wrote:
>>> On Wed, Jul 30, 2008 at 04:49:08PM +1000, Greg Ungerer wrote:
>>>> Sam Ravnborg wrote:
>>>>> The traditional location of the arch specific Makefiles
>>>>> has been at:
>>>>>
>>>>> 	include/asm-$ARCH
>>>>>
>>>>> But as suggested by several people over time and lately
>>>>> by Linus in the following post:
>>>>> http://kerneltrap.org/mailarchive/linux-kernel/2008/5/21/1903924
>>>>>
>>>>> is is suggested to move the arch specific files under arch/
>>>>> and for the header files this is arch/$ARCH/include.
>>>>>
>>>>> So I have hacked up support for this in kbuild so
>>>>> for the simpler cases a simple move of header files as
>>>>> in:
>>>>> 	git mv include/asm-$ARCH arch/$ARCH/include/asm
>>>>>
>>>>> is enough.
>>>>>
>>>>> If Linus will accept it then I think -rc1 is a perfect
>>>>> time to do it as people has less stuff pending touching
>>>>> the header files that are moved.
>>>>> Validation is easy - does the usual configs build?
>>>>>
>>>>> For the archs that plays games with symlinks this is a
>>>>> splendid opportunity to get rid af these.
>>>>> The receipe is simple:
>>>>> Use 	KBUILD_CFLAGS += -Iarch/$ARCH/include/$CPU
>>>>>
>>>>> to select the CPU and then do a copy of the header files
>>>>> like this:
>>>>>
>>>>> 	git mv include/asm-$ARCH/$CPU arch/$ARCH/include/$CPU/cpu
>>>>>
>>>>> Then the code can continue to use:
>>>>>
>>>>> 	#include <cpu/foo.h>
>>>>>
>>>>> I already converted sparc to use arch/sparc/include -
>>>>> while I merged the header files for sparc and sparc64.
>>>>> This change is already upstream.
>>>>>
>>>>> And for SH I posted a receipe which you can read here:
>>>>> http://lkml.org/lkml/2008/7/28/83
>>>>>
>>>>> It is SH specific but anyway pretty generic.
>>>>>
>>>>> I am limited in time and resources at the moment but I
>>>>> would be glad to help out as time permits should there
>>>>> be any questions.
>>>> I just tried moving the include/asm-m68knommu directory
>>>> to arch/m68knommu/include/asm and it all worked fine.
>>>> Clean compile first go :-)  Nice.  (As noted
>>>> asm-offsets.h was still generated in incldue/asm-m68knommu
>>>> afterwards though).
>>>>
>>>> So what is the status, should I prepare git/patch
>>>> for Linus?
>>> Yes please do so.
>> The move change can be pulled from:
>>
>> git://git.kernel.org/pub/scm/linux/kernel/git/gerg/m68knommu.git for-linus
>>
>> If there is no complaints I'll send Linus a pull request in a
>> couple of days time.
> 
> I did:
> 
>     mkdir arch/m68k/include
>     mv include/asm-m68k arch/m68k/include/asm
> 
> and m68k still builds fine (I applied Stephen's patch for the
> asm-sparc inclusion problem for Sun 3/3x first, of course).
> 
> As (a) I'm not using git (yet) for merging with Linus, and (b) this will
> cause some breakage in the m68knommu tree anyway, perhaps it's easiest if Greg
> takes care of it in m68knommu.git?
> 
> Greg, is that OK for you?

Yep, I will do that.


> You can add my Acked-by: Geert Uytterhoeven <geert@linux-m68k.org>
> 
> There seem to be 3 classes of m68knommu header files:
>   1. Plain files (some of them may be (nearly) identical copies of the
>      m68k variants,
>   2. Files that just include the m68k variant,
>   3. Files that include the m68k variant and do something more (pci.h and
>      setup.h).
> 
> Since I don't think we want to do the m68k/m68knommu merge right now
> (Sorry Arnd, I'll keep your script in mind anyway!), the simplest way is to:
>   - Remove all files from class 2, and add to the Makefile:
> 
> 	ccflags-y += -Iarch/m68k/include
> 
>   - Include verbatim the contents of the m68k variant for class 3, cfr.
>     Stephen patch for the asm-sparc inclusion problem for Sun 3/3x).
> 
> Do you agree?

That sounds like a good approach.

Regards
Greg



------------------------------------------------------------------------
Greg Ungerer  --  Chief Software Dude       EMAIL:     gerg@snapgear.com
Secure Computing Corporation                PHONE:       +61 7 3435 2888
825 Stanley St,                             FAX:         +61 7 3891 3630
Woolloongabba, QLD, 4102, Australia         WEB: http://www.SnapGear.com

Re: kbuild now support arch/$ARCH/include - time for ARCHs to convert

[Sam Ravnborg]

> >
> >As (a) I'm not using git (yet) for merging with Linus, and (b) this will
> >cause some breakage in the m68knommu tree anyway, perhaps it's easiest if 
> >Greg
> >takes care of it in m68knommu.git?
> >
> >Greg, is that OK for you?
> 
> Yep, I will do that.

Good. I also think this is the only right way to do it
so we do not have a longer period with m68knommu being broken.

	Sam

Re: kbuild now support arch/$ARCH/include - time for ARCHs to convert

[Arnd Bergmann]

On Sunday 03 August 2008, Geert Uytterhoeven wrote:
> There seem to be 3 classes of m68knommu header files:
>   1. Plain files (some of them may be (nearly) identical copies of the
>      m68k variants,

For reference, I found the files auxvec.h, device.h, emergency-restart.h,
futex.h, ioctl.h, irq_regs.h, kdebug.h, mutex.h and topology.h to be
entirely identical, so you may remove the m68knommu variants with your
approach.

>   2. Files that just include the m68k variant,
>   3. Files that include the m68k variant and do something more (pci.h and
>      setup.h).
>
> Since I don't think we want to do the m68k/m68knommu merge right now
> (Sorry Arnd, I'll keep your script in mind anyway!), the simplest way is to:
>  - Remove all files from class 2, and add to the Makefile:

Does this work with the exported headers? I guess it should if you
always use the m68k exports for m68knommu as well, but if that's
possible is is a question you haven't answered yet.

Right now, you would remove the following exported headers from m68knommu:
auxvec.h, errno.h, fcntl.h, ioctl.h, ioctls.h, ipcbuf.h, mman.h, msgbuf.h,
poll.h, posix_types.h, resource.h, sembuf.h, shmbuf.h, socket.h, stat.h,
statfs.h, termbits.h.

If you don't want to use the m68k exports on m68knomu, you'd also have
to duplicate all these.

	Arnd <><

Re: kbuild now support arch/$ARCH/include - time for ARCHs to convert

[Geert Uytterhoeven]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: TEXT/PLAIN; charset=UTF-8, Size: 1436 bytes --]

On Mon, 4 Aug 2008, Arnd Bergmann wrote:
> On Sunday 03 August 2008, Geert Uytterhoeven wrote:
> > There seem to be 3 classes of m68knommu header files:
> >   1. Plain files (some of them may be (nearly) identical copies of the
> >      m68k variants,
> 
> For reference, I found the files auxvec.h, device.h, emergency-restart.h,
> futex.h, ioctl.h, irq_regs.h, kdebug.h, mutex.h and topology.h to be
> entirely identical, so you may remove the m68knommu variants with your
> approach.

Thanks for the list!

> >   2. Files that just include the m68k variant,
> >   3. Files that include the m68k variant and do something more (pci.h and
> >      setup.h).
> >
> > Since I don't think we want to do the m68k/m68knommu merge right now
> > (Sorry Arnd, I'll keep your script in mind anyway!), the simplest way is to:
> >  - Remove all files from class 2, and add to the Makefile:
> 
> Does this work with the exported headers? I guess it should if you
> always use the m68k exports for m68knommu as well, but if that's
> possible is is a question you haven't answered yet.

Oops, forgot about the exported headers... Don't know. Sam?

Gr{oetje,eeting}s,

						Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
							    -- Linus Torvalds

Re: kbuild now support arch/$ARCH/include - time for ARCHs to convert

[Sam Ravnborg]

On Mon, Aug 04, 2008 at 06:00:53PM +0200, Geert Uytterhoeven wrote:
> On Mon, 4 Aug 2008, Arnd Bergmann wrote:
> > On Sunday 03 August 2008, Geert Uytterhoeven wrote:
> > > There seem to be 3 classes of m68knommu header files:
> > > ? 1. Plain files (some of them may be (nearly) identical copies of the
> > > ? ? ?m68k variants,
> > 
> > For reference, I found the files auxvec.h, device.h, emergency-restart.h,
> > futex.h, ioctl.h, irq_regs.h, kdebug.h, mutex.h and topology.h to be
> > entirely identical, so you may remove the m68knommu variants with your
> > approach.
> 
> Thanks for the list!
> 
> > > ? 2. Files that just include the m68k variant,
> > > ? 3. Files that include the m68k variant and do something more (pci.h and
> > > ? ? ?setup.h).
> > >
> > > Since I don't think we want to do the m68k/m68knommu merge right now
> > > (Sorry Arnd, I'll keep your script in mind anyway!), the simplest way is to:
> > >  - Remove all files from class 2, and add to the Makefile:
> > 
> > Does this work with the exported headers? I guess it should if you
> > always use the m68k exports for m68knommu as well, but if that's
> > possible is is a question you haven't answered yet.
> 
> Oops, forgot about the exported headers... Don't know. Sam?

For sparc I has a similar issue and in the end I decided
to put the extra effort in and merged the header files
for sparc and sparc64. In this way we then had only a single
source to export from and despite the number of files named
foo_32.h / foo_64.h it was a net win because David long term anyway
plan to merge the two.

So to me the obvious solution for m68k / m68knommu is to
put in the additional effort required to merge the header files
for the two architectures.

In the top-level Makefile you will need this trivial patch to make it work:
diff --git a/Makefile b/Makefile
index f156f40..491ce6b 100644
--- a/Makefile
+++ b/Makefile
@@ -206,10 +206,12 @@ ifeq ($(ARCH),x86_64)
 endif

 # Where to locate arch specific headers
+hdr-arch  := $(SRCARCH)
 ifeq ($(ARCH),sparc64)
        hdr-arch  := sparc
-else
-       hdr-arch  := $(SRCARCH)
+endif
+ifeq ($(ARCH),m68knommu)
+        hdr-arch := m68k
 endif

So we have a script to help us. We know what files are identical and
kbuild support is ready.

And if we avoid it we have issues with exported headers.

So let's merge the two now - there is no reason to wait.
And do so in the git tree to make it obvious what happens.

	Sam

Re: kbuild now support arch/$ARCH/include - time for ARCHs to convert

[Rusty Russell]

On Tuesday 29 July 2008 06:05:00 Sam Ravnborg wrote:
> is is suggested to move the arch specific files under arch/
> and for the header files this is arch/$ARCH/include.

Well, it would have been nice to take the opportunity to separate user-visible 
header files and non-user-visible files.  We could have left the user-visible 
ones in place, for example: diffstat would then show when you were frobbing 
the userspace interface.

But I'm guessing noone wants even more marginal-value churn right now.
Rusty.

Re: kbuild now support arch/$ARCH/include - time for ARCHs to convert

[Geert Uytterhoeven]

On Tue, 5 Aug 2008, Rusty Russell wrote:
> On Tuesday 29 July 2008 06:05:00 Sam Ravnborg wrote:
> > is is suggested to move the arch specific files under arch/
> > and for the header files this is arch/$ARCH/include.
> 
> Well, it would have been nice to take the opportunity to separate user-visible 
> header files and non-user-visible files.  We could have left the user-visible 
> ones in place, for example: diffstat would then show when you were frobbing 
> the userspace interface.
> 
> But I'm guessing noone wants even more marginal-value churn right now.
> Rusty.

That actually sounds like a good idea to me!
Can we do the same with the <linux/*.h> ones?

Gr{oetje,eeting}s,

						Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
							    -- Linus Torvalds

Re: kbuild now support arch/$ARCH/include - time for ARCHs to convert

[Sam Ravnborg]

On Tue, Aug 05, 2008 at 02:18:05PM +1000, Rusty Russell wrote:
> On Tuesday 29 July 2008 06:05:00 Sam Ravnborg wrote:
> > is is suggested to move the arch specific files under arch/
> > and for the header files this is arch/$ARCH/include.
> 
> Well, it would have been nice to take the opportunity to separate user-visible 
> header files and non-user-visible files.  We could have left the user-visible 
> ones in place, for example: diffstat would then show when you were frobbing 
> the userspace interface.

I would like to see this change too - but intentionally I would
not try to mix it with the move of the header files. Lets get the
move done now and fix the fall-outs, then we can take this logical next step.

	Sam

Re: kbuild now support arch/$ARCH/include - time for ARCHs to convert

[Russell King]

On Tue, Aug 05, 2008 at 10:45:48PM +0200, Sam Ravnborg wrote:
> On Tue, Aug 05, 2008 at 02:18:05PM +1000, Rusty Russell wrote:
> > On Tuesday 29 July 2008 06:05:00 Sam Ravnborg wrote:
> > > is is suggested to move the arch specific files under arch/
> > > and for the header files this is arch/$ARCH/include.
> > 
> > Well, it would have been nice to take the opportunity to separate user-visible 
> > header files and non-user-visible files.  We could have left the user-visible 
> > ones in place, for example: diffstat would then show when you were frobbing 
> > the userspace interface.
> 
> I would like to see this change too - but intentionally I would
> not try to mix it with the move of the header files. Lets get the
> move done now and fix the fall-outs, then we can take this logical
> next step.

Absolutely agreed.

Some architectures (eg, ARM) have quite a large headache to solve with
these changes.

Currently, I know Linus' tree builds fine for most ARM platforms (thanks
to the ARM kautobuild project.)  However, I'm seeing unexpected build
failures that don't make sense from the linux-next project.

I've just wound my tree back to a "pristine" state for linux-next, and
plan to wait for its next build to check whether its me or something else
causing it.  What I'm referring to is:

s3c2410_defconfig's showing:
include/asm/arch/map.h:16:30: error: asm/plat-s3c/map.h: No such file or directory

which, if its including it via include/ shouldn't be happening because
plat-s3c hasn't moved from include/asm(-arm)/, the same as
include/asm-arm/arch-s3c2410 hasn't moved in linux-next yet.

-- 
Russell King
 Linux kernel    2.6 ARM Linux   - http://www.arm.linux.org.uk/
 maintainer of:

Re: kbuild now support arch/$ARCH/include - time for ARCHs to convert

[Russell King]

On Tue, Aug 05, 2008 at 11:06:15PM +0100, Russell King wrote:
> Currently, I know Linus' tree builds fine for most ARM platforms (thanks
> to the ARM kautobuild project.)  However, I'm seeing unexpected build
> failures that don't make sense from the linux-next project.

And the expression "oh bugger" comes to mind.  The breakage in linux-next
is because of 2e57d051160dd61776461637f767df19036b1186 and
ea35455e0dc17d732436a5b98bd511cab64eb10e.

During the last fortnight, I'd moved the "easy to move" ARM headers
from include/asm-arm to arch/arm/include/asm.  This means that, unknown
to me, the symlink was no longer being created.  That didn't get picked
up in my testing, because I always build using O= builds into already
existing object trees, which of course all have the symlink in place.

This doesn't affect kautobuild because that always builds trees without
using the O= facility.

The question is what to do about it.  Given the size of the task, I'm
currently of the opinion that we're moving things around in the wrong
order, and 4baa9922430662431231ac637adedddbb0cfb2d7 should be reverted
until we've moved all the other ARM headers (in asm/arch and asm/plat-*)
first.

Maybe even leaving this until the next merge window now that -rc2 is
out would be a good idea.

-- 
Russell King
 Linux kernel    2.6 ARM Linux   - http://www.arm.linux.org.uk/
 maintainer of:

Re: kbuild now support arch/$ARCH/include - time for ARCHs to convert

[Sam Ravnborg]

> 
> Currently, I know Linus' tree builds fine for most ARM platforms (thanks
> to the ARM kautobuild project.)  However, I'm seeing unexpected build
> failures that don't make sense from the linux-next project.
> 
> I've just wound my tree back to a "pristine" state for linux-next, and
> plan to wait for its next build to check whether its me or something else
> causing it.  What I'm referring to is:
> 
> s3c2410_defconfig's showing:
> include/asm/arch/map.h:16:30: error: asm/plat-s3c/map.h: No such file or directory
> 
> which, if its including it via include/ shouldn't be happening because
> plat-s3c hasn't moved from include/asm(-arm)/, the same as
> include/asm-arm/arch-s3c2410 hasn't moved in linux-next yet.

This is a kbuild bug.
The top-level Makefile creates the include2/asm symlink only
if the file$(srctree)/include/asm-$ARCH/errno.h is present.

And you moved errno.h when you committed your changes so symlink
is not created anymore thus it fails.
But this is oly with "make O=..." builds which I think kautobuild
does not use (since it did not trigger the bug).

I cannot recall the exact reason why I did this check before
creating the symlink so I will simply create it always.

	Sam

Re: kbuild now support arch/$ARCH/include - time for ARCHs to convert

[Sam Ravnborg]

On Wed, Aug 06, 2008 at 08:21:26PM +0100, Russell King wrote:
> On Tue, Aug 05, 2008 at 11:06:15PM +0100, Russell King wrote:
> > Currently, I know Linus' tree builds fine for most ARM platforms (thanks
> > to the ARM kautobuild project.)  However, I'm seeing unexpected build
> > failures that don't make sense from the linux-next project.
> 
> And the expression "oh bugger" comes to mind.  The breakage in linux-next
> is because of 2e57d051160dd61776461637f767df19036b1186 and
> ea35455e0dc17d732436a5b98bd511cab64eb10e.
> 
> During the last fortnight, I'd moved the "easy to move" ARM headers
> from include/asm-arm to arch/arm/include/asm.  This means that, unknown
> to me, the symlink was no longer being created.  That didn't get picked
> up in my testing, because I always build using O= builds into already
> existing object trees, which of course all have the symlink in place.
> 
> This doesn't affect kautobuild because that always builds trees without
> using the O= facility.
> 
> The question is what to do about it.  Given the size of the task, I'm
> currently of the opinion that we're moving things around in the wrong
> order, and 4baa9922430662431231ac637adedddbb0cfb2d7 should be reverted
> until we've moved all the other ARM headers (in asm/arch and asm/plat-*)
> first.

Our mails crossed. I will fix kbuild asap so you do not need to revert.

	Sam

Re: kbuild now support arch/$ARCH/include - time for ARCHs to convert

[Russell King]

On Wed, Aug 06, 2008 at 09:28:42PM +0200, Sam Ravnborg wrote:
> Our mails crossed. I will fix kbuild asap so you do not need to revert.

Have crossed again, sorry.  If you can get a fix for the asm/errno.h
thing to Linus ASAP, that'd be great.

-- 
Russell King
 Linux kernel    2.6 ARM Linux   - http://www.arm.linux.org.uk/
 maintainer of:

Re: kbuild now support arch/$ARCH/include - time for ARCHs to convert

[Sam Ravnborg]

On Tue, Jul 29, 2008 at 10:21:45AM -0400, Kyle McMartin wrote:
> On Tue, Jul 29, 2008 at 05:27:26AM +0200, Sam Ravnborg wrote:
> > On Mon, Jul 28, 2008 at 11:06:47PM -0400, Kyle McMartin wrote:
> > > On Mon, Jul 28, 2008 at 10:05:00PM +0200, Sam Ravnborg wrote:
> > > > I am limited in time and resources at the moment but I
> > > > would be glad to help out as time permits should there
> > > > be any questions.
> > > > 
> > > 
> > > Just a quick one... I moved the files on parisc, but found we're still
> > > using include/asm-$foo/asm-offsets.h. A quick glance at Kbuild seems to
> > > indicate so. Is this expected?
> > Yes.
> > 
> > asm-offsets.h is a generated file and I wanted to avoid adding
> > generated files to arch/$ARCH/include.
> > 
> > It was also a less intrusive patch to keep it at the old location.
> > But I recall that it troubles me a bit before I got it
> > correct but the kbuild changes all take care of this.
> > 
> 
> Thanks for clarifying, Sam.

Any chance we can have this pushed towards Linus?

It would be nice to gat as many arch converted in this cycle as possible.

	Sam

Re: kbuild now support arch/$ARCH/include - time for ARCHs to convert

[Sam Ravnborg]

On Thu, Jul 31, 2008 at 07:23:15PM +0100, Ralf Baechle wrote:
> On Mon, Jul 28, 2008 at 10:05:00PM +0200, Sam Ravnborg wrote:
> 
> > The traditional location of the arch specific Makefiles
> > has been at:
> > 
> > 	include/asm-$ARCH
> > 
> > But as suggested by several people over time and lately
> > by Linus in the following post:
> > http://kerneltrap.org/mailarchive/linux-kernel/2008/5/21/1903924
> > 
> > is is suggested to move the arch specific files under arch/
> > and for the header files this is arch/$ARCH/include.
> > 
> > So I have hacked up support for this in kbuild so
> > for the simpler cases a simple move of header files as
> > in:
> > 	git mv include/asm-$ARCH arch/$ARCH/include/asm
> > 
> > is enough.
> > 
> > If Linus will accept it then I think -rc1 is a perfect
> > time to do it as people has less stuff pending touching
> > the header files that are moved.
> > Validation is easy - does the usual configs build?
> > 
> > For the archs that plays games with symlinks this is a
> > splendid opportunity to get rid af these.
> > The receipe is simple:
> > Use 
> > 	KBUILD_CFLAGS += -Iarch/$ARCH/include/$CPU
> > 
> > to select the CPU and then do a copy of the header files
> > like this:
> > 
> > 	git mv include/asm-$ARCH/$CPU arch/$ARCH/include/$CPU/cpu
> > 
> > Then the code can continue to use:
> > 
> > 	#include <cpu/foo.h>
> > 
> > I already converted sparc to use arch/sparc/include -
> > while I merged the header files for sparc and sparc64.
> > This change is already upstream.
> > 
> > And for SH I posted a receipe which you can read here:
> > http://lkml.org/lkml/2008/7/28/83
> > 
> > It is SH specific but anyway pretty generic.
> > 
> > I am limited in time and resources at the moment but I
> > would be glad to help out as time permits should there
> > be any questions.
> 
> I just did the necessary changes for MIPS and the first testbuild is running
> successfully so far.
> 
> One of the ugly things in the MIPS makefiles is the use of platform-
> specific headers in include/asm-mips/mach-* which contain platform specific
> headers.  Ideally those should be included through something like
> <mach/somefile.h> or <plat/something.h> and I think a directory like
> arch/$ARCH/$PLATFORM/include/plat/foobar.h could be a good place in the
> new scheme?  I assume other platforms have similar issues with subplatforms,
> processor sub-architectures etc.?

Hi Ralf.

Did you follow what arm have planned?
It looks like MIPS should do things in the same way so
we get some consistency.

	Sam

Re: kbuild now support arch/$ARCH/include - time for ARCHs to convert

[Paul Mundt]

On Sat, Aug 09, 2008 at 04:52:44AM +0200, Sam Ravnborg wrote:
> On Thu, Jul 31, 2008 at 07:23:15PM +0100, Ralf Baechle wrote:
> > I just did the necessary changes for MIPS and the first testbuild is running
> > successfully so far.
> > 
> > One of the ugly things in the MIPS makefiles is the use of platform-
> > specific headers in include/asm-mips/mach-* which contain platform specific
> > headers.  Ideally those should be included through something like
> > <mach/somefile.h> or <plat/something.h> and I think a directory like
> > arch/$ARCH/$PLATFORM/include/plat/foobar.h could be a good place in the
> > new scheme?  I assume other platforms have similar issues with subplatforms,
> > processor sub-architectures etc.?
> 
> Hi Ralf.
> 
> Did you follow what arm have planned?
> It looks like MIPS should do things in the same way so
> we get some consistency.
> 
Where was this discussed? SH has the same issues obviously, but we've
already settled on a scheme.

Re: kbuild now support arch/$ARCH/include - time for ARCHs to convert

[Sam Ravnborg]

On Sat, Aug 09, 2008 at 12:09:28PM +0900, Paul Mundt wrote:
> On Sat, Aug 09, 2008 at 04:52:44AM +0200, Sam Ravnborg wrote:
> > On Thu, Jul 31, 2008 at 07:23:15PM +0100, Ralf Baechle wrote:
> > > I just did the necessary changes for MIPS and the first testbuild is running
> > > successfully so far.
> > > 
> > > One of the ugly things in the MIPS makefiles is the use of platform-
> > > specific headers in include/asm-mips/mach-* which contain platform specific
> > > headers.  Ideally those should be included through something like
> > > <mach/somefile.h> or <plat/something.h> and I think a directory like
> > > arch/$ARCH/$PLATFORM/include/plat/foobar.h could be a good place in the
> > > new scheme?  I assume other platforms have similar issues with subplatforms,
> > > processor sub-architectures etc.?
> > 
> > Hi Ralf.
> > 
> > Did you follow what arm have planned?
> > It looks like MIPS should do things in the same way so
> > we get some consistency.
> > 
> Where was this discussed? SH has the same issues obviously, but we've
> already settled on a scheme.

Looking again I realize this was only discussed on the arm kernel mailing list
and people involved was rmk, Harvaard and myself.

This was not optimal - sorry.

How much mips and arm solution differ I have not looked at closely.

	Sam

Re: kbuild now support arch/$ARCH/include - time for ARCHs to convert

[Russell King]

On Sat, Aug 09, 2008 at 09:03:44AM +0200, Sam Ravnborg wrote:
> On Sat, Aug 09, 2008 at 12:09:28PM +0900, Paul Mundt wrote:
> > Where was this discussed? SH has the same issues obviously, but we've
> > already settled on a scheme.
> 
> Looking again I realize this was only discussed on the arm kernel mailing list
> and people involved was rmk, Harvaard and myself.

... and with almost every other ARM person ignoring it entirely, except
for them posting patches against the old layout, and asking when they're
going to be merged.

It was plainly obvious that almost no one was reading Sam's "[RFC pseudo
PATCH] migrate to arch/arm/include" thread.

So it might as well have been discussed here out of their sight. :(

-- 
Russell King
 Linux kernel    2.6 ARM Linux   - http://www.arm.linux.org.uk/
 maintainer of:

Re: kbuild now support arch/$ARCH/include - time for ARCHs to convert

[Sam Ravnborg]

On Mon, Jul 28, 2008 at 10:05:00PM +0200, Sam Ravnborg wrote:
> The traditional location of the arch specific Makefiles
> has been at:
> 
> 	include/asm-$ARCH
> 
> But as suggested by several people over time and lately
> by Linus in the following post:
> http://kerneltrap.org/mailarchive/linux-kernel/2008/5/21/1903924
> 
> is is suggested to move the arch specific files under arch/
> and for the header files this is arch/$ARCH/include.

So where are we now?
alpha        Not done - but trivial (I have tested it)
blackfin     Not done - uses symlinks
h8300        Not done - but looks trivial
m32r         Not done - looks trivial
mips         Not done - but I think it is prepared
powerpc      Done
sparc        Done
x86          Not done - assume planned at end of next merge window
arm          Almost done - three plat-* directories needs to be moved
cris         Not done - uses symlinks
ia64         Done
m68k         Not done - may be combined with merging headers for m68knommu
mn10300      Not done - uses symlinks
s390         Done
sparc64      Done
xtensa       Not done - uses symlinks
avr32        Done
frv          Not done - looks trivial
m68knommu    Done
parisc       Not done - but patches are prepared
sh           Done
um           Not done - uses symlinks

So 8 out of 22 so far.

That is pretty good as we only started this at the end of the
merge window.
And of these sh, arm, avr have restructured to avoid symlinks so
it was not that simple.

And we have at six architectures where a simple:
mkdir -p arch/$ARCH/include/asm
git mv include/asm-$ARCH/* arch/$ARCH/include/asm

looks like the only effort needed {alpha, h8300, m32r, mips frv, parisc}.

	Sam

Re: kbuild now support arch/$ARCH/include - time for ARCHs to convert

[Sam Ravnborg]

On Sat, Aug 09, 2008 at 08:20:31AM +0100, Russell King wrote:
> On Sat, Aug 09, 2008 at 09:03:44AM +0200, Sam Ravnborg wrote:
> > On Sat, Aug 09, 2008 at 12:09:28PM +0900, Paul Mundt wrote:
> > > Where was this discussed? SH has the same issues obviously, but we've
> > > already settled on a scheme.
> > 
> > Looking again I realize this was only discussed on the arm kernel mailing list
> > and people involved was rmk, Harvaard and myself.
> 
> ... and with almost every other ARM person ignoring it entirely

Or they know they could trust the person dealign with it...

	Sam