kbuild news

kbuild news

[Kai Germaschewski]

Hi arch maintainers,

I submitted a kbuild update to Linus which has the potential to break 
archs other than i386.

Two changes could affect you:
o the build process, while remaining recursive, does not cd into the 
  subdirs anymore. If you're just using the standard kbuild 
  infrastructure, you should be fine - however, things like
  custom include paths may have to be adapted
  (-I../../drivers/scsi -> -Idrivers/scsi).
  This change should not affect arch/$(ARCH)/boot and other custom stuff.
  However, I converted i386/boot as well, to give an example of how it can
  be done ;)

  If something goes wrong here, you'll most likely get a build failure,
  so you'll notice. The other point is a bit more subtle.

o The final link of vmlinux is now always done as a two step process:
  link a temporary .tmp_vmlinux out of $(HEAD) init/ kernel/ mm/ drivers/
  ... exactly as before. Then, potentially objects which need the
  vmlinux image to exist already are built (currently that would be
  .tmp_kallsyms.o, and possibly sparc's btfix stuff in the future).
  As a last step, the final vmlinux is linked from the .tmp_vmlinux +
  the additional objects that have just been built.

  Even when no kallsyms or other additional objects are built, the final
  vmlinux is now built with

  	ld <usual flags> -T arch/$(ARCH)/vmlinux.lds.s .tmp_vmlinux \
		-o vmlinux

  So you need to be sure that your arch's vmlinux.lds.S does not
  reorder sections in this final step.

  i386 needed the following patch:

--- ../linux-2.5.isdn/arch/i386/vmlinux.lds.S	Fri Oct  4 12:09:10 2002
+++ arch/i386/vmlinux.lds.S	Sat Oct  5 17:21:30 2002
@@ -49,6 +49,7 @@
   __setup_end = .;
   __initcall_start = .;
   .initcall.init : {
+	*(.initcall.init)
 	*(.initcall1.init) 
 	*(.initcall2.init) 
 	*(.initcall3.init) 
@@ -72,7 +73,7 @@
   __nosave_end = .;
 
   . = ALIGN(4096);
-  .data.page_aligned : { *(.data.idt) }
+  .data.page_aligned : {  *(.data.page_aligned) *(.data.idt) }
 
   . = ALIGN(32);
   .data.cacheline_aligned : { *(.data.cacheline_aligned) }
 
since after the first link, all the initcalls are in .initcall.init, and
they're supposed to just remain there during the second one, same thing 
for data.page_aligned.

You've been warned, take a look at your vmlinux.lds.S, or just compare
objdump -h .tmp_vmlinux vs objdump -h vmlinux. You don't want to see 
differences there ;)

--Kai

Re: kbuild news

[David S. Miller]

   From: Kai Germaschewski <kai-germaschewski@uiowa.edu>
   Date: Sat, 5 Oct 2002 21:10:06 -0500 (CDT)

   o The final link of vmlinux is now always done as a two step process:

This doesn't seem to be happening "always" now in current
2.5.x, I did not see a .tmp_vmlinux get generated.

It seems the whole mechanism to do kallsyms got redone since
you sent this email.

Re: kbuild news

[Kai Germaschewski]

On Mon, 7 Oct 2002, David S. Miller wrote:

>    From: Kai Germaschewski <kai-germaschewski@uiowa.edu>
>    Date: Sat, 5 Oct 2002 21:10:06 -0500 (CDT)
> 
>    o The final link of vmlinux is now always done as a two step process:
> 
> This doesn't seem to be happening "always" now in current
> 2.5.x, I did not see a .tmp_vmlinux get generated.
> 
> It seems the whole mechanism to do kallsyms got redone since
> you sent this email.

Yes, that's true, my idea on how to do that was completely broken, so 
we're basically back to the old way.

BTW: That also means that your and everybody's vmlinux.lds.S does *not* 
need adapting.

I'll take a look on how to do sparc's btfixup in a similar way, without 
messing up the common code too much. BTW, the combination kallsyms +
btfixup, does that need a particular ordering?

--Kai

Re: kbuild news

[David S. Miller]

   From: Kai Germaschewski <kai-germaschewski@uiowa.edu>
   Date: Mon, 7 Oct 2002 09:22:51 -0500 (CDT)
   
   I'll take a look on how to do sparc's btfixup in a similar way, without 
   messing up the common code too much. BTW, the combination kallsyms +
   btfixup, does that need a particular ordering?

No, the kallsyms object file would not need to be seen by
the btfixup.o generator.  It could therefore be done validly
as:

	1) build .tmp_vmlinux
	2) build btfixup.o
	3) build kallsyms
	4) link final vmlinux image

The order of #2 and #3 could be transposed and that would be fine too.

Re: kbuild news

[Kai Germaschewski]

On Mon, 7 Oct 2002, David S. Miller wrote:

>    I'll take a look on how to do sparc's btfixup in a similar way, without 
>    messing up the common code too much. BTW, the combination kallsyms +
>    btfixup, does that need a particular ordering?
> 
> No, the kallsyms object file would not need to be seen by
> the btfixup.o generator.  It could therefore be done validly
> as:
> 
> 	1) build .tmp_vmlinux
> 	2) build btfixup.o
> 	3) build kallsyms
> 	4) link final vmlinux image
> 
> The order of #2 and #3 could be transposed and that would be fine too.

Alright, that's helpful ;)

--Kai

Re: kbuild news

[Keith Owens]

On Mon, 07 Oct 2002 07:24:26 -0700 (PDT), 
"David S. Miller" <davem@redhat.com> wrote:
>No, the kallsyms object file would not need to be seen by
>the btfixup.o generator.  It could therefore be done validly
>as:
>
>	1) build .tmp_vmlinux
>	2) build btfixup.o
>	3) build kallsyms
>	4) link final vmlinux image
>
>The order of #2 and #3 could be transposed and that would be fine too.

Wrong.  Anything that changes the address or size of any symbol or
section invalidates the kallsyms data.  kallsyms must be run on the
definitive vmlinux contents, with everything else already included and
all sizes must be stable.

Re: kbuild news

[Pete Zaitcev]

> Date: Tue, 08 Oct 2002 13:11:00 -0700 (PDT)
> From: "David S. Miller" <davem@redhat.com>

>    From: Pete Zaitcev <zaitcev@redhat.com>
>    Date: Tue, 8 Oct 2002 10:42:40 -0400
>    
>    Let's face it, both btfixup and kallsyms "want" to be the last,
>    so something has to give.
> 
> No, btfixup does not care about anything that will go into
> kallsyms.o, no BTFIXUP objects may appear in kallsyms.
> 
> So btfixup may be next to last just fine.

This is very unfortunate, because it invalidates the workaround
where I moved btfixup to "make image" stage. Unles Kai is willing
to change the way the dependencies are handled, I'm stuck.
I am talking about breaking up this:

$(sort $(vmlinux-objs)): $(SUBDIRS) ;

To make every single object (including head.o) on every other
subdirectory (even benign ones) seems a little fishy.

-- Pete

Re: kbuild news

[David S. Miller]

   From: Pete Zaitcev <zaitcev@redhat.com>
   Date: Tue, 8 Oct 2002 10:42:40 -0400
   
   Let's face it, both btfixup and kallsyms "want" to be the last,
   so something has to give.

No, btfixup does not care about anything that will go into
kallsyms.o, no BTFIXUP objects may appear in kallsyms.

So btfixup may be next to last just fine.

Re: kbuild news

[Pete Zaitcev]

> From: Keith Owens <kaos@ocs.com.au>

>>No, the kallsyms object file would not need to be seen by
>>the btfixup.o generator.  It could therefore be done validly
>>as:
>>
>>	1) build .tmp_vmlinux
>>	2) build btfixup.o
>>	3) build kallsyms
>>	4) link final vmlinux image
>>
>>The order of #2 and #3 could be transposed and that would be fine too.
> 
> Wrong.  Anything that changes the address or size of any symbol or
> section invalidates the kallsyms data.  kallsyms must be run on the
> definitive vmlinux contents, with everything else already included and
> all sizes must be stable.

Let's face it, both btfixup and kallsyms "want" to be the last,
so something has to give.

-- Pete