Finding what change broke ARM

Finding what change broke ARM

[Russell King]

When building current git for ARM, I see:

  CC      arch/arm/mm/consistent.o
arch/arm/mm/consistent.c: In function `dma_free_coherent':
arch/arm/mm/consistent.c:357: error: `mem_map' undeclared (first use in this function)
arch/arm/mm/consistent.c:357: error: (Each undeclared identifier is reported only once
arch/arm/mm/consistent.c:357: error: for each function it appears in.)
make[2]: *** [arch/arm/mm/consistent.o] Error 1

How can I find what change elsewhere in the kernel tree caused this
breakage?

With bk, you could ask for a per-file revision history of the likely
candidates, and then find the changeset to view the other related
changes.

With git... ?  We don't have per-file revision history so...

-- 
Russell King
 Linux kernel    2.6 ARM Linux   - http://www.arm.linux.org.uk/
 maintainer of:  2.6 Serial core

Re: Finding what change broke ARM

[Russell King]

On Fri, Jun 24, 2005 at 10:19:51AM +0100, Russell King wrote:
> When building current git for ARM, I see:
> 
>   CC      arch/arm/mm/consistent.o
> arch/arm/mm/consistent.c: In function `dma_free_coherent':
> arch/arm/mm/consistent.c:357: error: `mem_map' undeclared (first use in this function)
> arch/arm/mm/consistent.c:357: error: (Each undeclared identifier is reported only once
> arch/arm/mm/consistent.c:357: error: for each function it appears in.)
> make[2]: *** [arch/arm/mm/consistent.o] Error 1
> 
> How can I find what change elsewhere in the kernel tree caused this
> breakage?
> 
> With bk, you could ask for a per-file revision history of the likely
> candidates, and then find the changeset to view the other related
> changes.
> 
> With git... ?  We don't have per-file revision history so...

This particular breakage appears to have been caused by kbuild not
automatically updating .config.  That's odd because at other times
I've seen kbuild be over-insistent that it needs to update .config.

However, now I get a different error:

  CC      arch/arm/mm/discontig.o
arch/arm/mm/discontig.c:19:3: #error Fix Me Please
arch/arm/mm/discontig.c:30: warning: excess elements in array initializer
arch/arm/mm/discontig.c:30: warning: (near initialization for `discontig_node_data')
arch/arm/mm/discontig.c:31: warning: excess elements in array initializer
arch/arm/mm/discontig.c:31: warning: (near initialization for `discontig_node_data')
arch/arm/mm/discontig.c:32: warning: excess elements in array initializer
arch/arm/mm/discontig.c:32: warning: (near initialization for `discontig_node_data')
make[2]: *** [arch/arm/mm/discontig.o] Error 1

and a .config which looks like this:

CONFIG_ARCH_SA1100=y
...
# CONFIG_ARCH_DISCONTIGMEM_ENABLE is not set
CONFIG_SELECT_MEMORY_MODEL=y
CONFIG_FLATMEM_MANUAL=y
# CONFIG_DISCONTIGMEM_MANUAL is not set
# CONFIG_SPARSEMEM_MANUAL is not set
CONFIG_DISCONTIGMEM=y
CONFIG_FLATMEM=y
CONFIG_FLAT_NODE_MEM_MAP=y
CONFIG_NEED_MULTIPLE_NODES=y

At a guess, this is because we have two memory models selected - because
the Kconfig magic in mm/Kconfig isn't correct for ARM.

ARM selects CONFIG_DISCONTIGMEM for certain platforms (based on
CONFIG_ARCH_SA1100 in this case.)  mm/Kconfig decides on its own back
that it'll choose CONFIG_FLATMEM for us.  So two models get selected.

Should I remove the mm/Kconfig include and replicate what's required for
ARM, or... ?  TBH mm/Kconfig seems to be rather OTT.

Help!

-- 
Russell King
 Linux kernel    2.6 ARM Linux   - http://www.arm.linux.org.uk/
 maintainer of:  2.6 Serial core

Re: Finding what change broke ARM

[Russell King]

On Fri, Jun 24, 2005 at 10:53:28AM +0100, Russell King wrote:
> and a .config which looks like this:
> 
> CONFIG_ARCH_SA1100=y
> ...
> # CONFIG_ARCH_DISCONTIGMEM_ENABLE is not set
> CONFIG_SELECT_MEMORY_MODEL=y
> CONFIG_FLATMEM_MANUAL=y
> # CONFIG_DISCONTIGMEM_MANUAL is not set
> # CONFIG_SPARSEMEM_MANUAL is not set
> CONFIG_DISCONTIGMEM=y
> CONFIG_FLATMEM=y
> CONFIG_FLAT_NODE_MEM_MAP=y
> CONFIG_NEED_MULTIPLE_NODES=y
> 
> At a guess, this is because we have two memory models selected - because
> the Kconfig magic in mm/Kconfig isn't correct for ARM.
> 
> ARM selects CONFIG_DISCONTIGMEM for certain platforms (based on
> CONFIG_ARCH_SA1100 in this case.)  mm/Kconfig decides on its own back
> that it'll choose CONFIG_FLATMEM for us.  So two models get selected.
> 
> Should I remove the mm/Kconfig include and replicate what's required for
> ARM, or... ?  TBH mm/Kconfig seems to be rather OTT.
> 
> Help!

Well, this fixes the problem, but I doubt people will like it.

Index: mm/Kconfig
===================================================================
--- fc736377c5c7e23ee78569392ed31a6030289e44/mm/Kconfig  (mode:100644)
+++ uncommitted/mm/Kconfig  (mode:100644)
@@ -71,7 +71,7 @@
 
 config FLATMEM
 	def_bool y
-	depends on (!DISCONTIGMEM && !SPARSEMEM) || FLATMEM_MANUAL
+	depends on (!DISCONTIGMEM && !SPARSEMEM) #|| FLATMEM_MANUAL
 
 config FLAT_NODE_MEM_MAP
 	def_bool y


-- 
Russell King
 Linux kernel    2.6 ARM Linux   - http://www.arm.linux.org.uk/
 maintainer of:  2.6 Serial core

Re: Finding what change broke ARM

[Matthias Urlichs]

Hi, Russell King wrote:

> With git... ?  We don't have per-file revision history so...

That doesn't really matter -- "cg-log FILE" will show you the changes
which change FILE. Works with subdirectories too.

-- 
Matthias Urlichs   |   {M:U} IT Design @ m-u-it.de   |  smurf@smurf.noris.de
Disclaimer: The quote was selected randomly. Really. | http://smurf.noris.de
 - -
"Of all the many, many different versions of gods in the world;
 Christians seem to have sought out the most unmoving, uncompromising,
 hateful, and murderous version they could find to give their worship
 too. A sad sorry lot they be indeed."
                    [Michael Suetkamp]

Re: Finding what change broke ARM

[Alecs King]

On Fri, Jun 24, 2005 at 10:19:51AM +0100, Russell King wrote:
> When building current git for ARM, I see:
> 
>   CC      arch/arm/mm/consistent.o
> arch/arm/mm/consistent.c: In function `dma_free_coherent':
> arch/arm/mm/consistent.c:357: error: `mem_map' undeclared (first use in this function)
> arch/arm/mm/consistent.c:357: error: (Each undeclared identifier is reported only once
> arch/arm/mm/consistent.c:357: error: for each function it appears in.)
> make[2]: *** [arch/arm/mm/consistent.o] Error 1
> 
> How can I find what change elsewhere in the kernel tree caused this
> breakage?
> 
> With bk, you could ask for a per-file revision history of the likely
> candidates, and then find the changeset to view the other related
> changes.
> 
> With git... ?  We don't have per-file revision history so...

Wouldnt a 'git-whatchanged -p <candidates>' help?


-- 
Alecs King

Re: Finding what change broke ARM

[Andy Whitcroft]

> Well, this fixes the problem, but I doubt people will like it.

This looks like a problem with the way the configuration options where
changed to allow more than two memory models for SPARSMEM.  I think the
right fix is the patch below.  Russell could you try this one instead.
Dave, you did most of the work on the configuration side could you look
this over (assuming it works!).

-apw

Signed-off-by: Andy Whitcroft <apw@shadowen.org>
---
 Kconfig |    3 +--
 1 files changed, 1 insertion(+), 2 deletions(-)
diff -upN reference/arch/arm/Kconfig current/arch/arm/Kconfig
--- reference/arch/arm/Kconfig
+++ current/arch/arm/Kconfig
@@ -161,7 +161,6 @@ config ARCH_RPC
 config ARCH_SA1100
 	bool "SA1100-based"
 	select ISA
-	select DISCONTIGMEM
 
 config ARCH_S3C2410
 	bool "Samsung S3C2410"
@@ -345,7 +344,7 @@ config PREEMPT
 
 config ARCH_DISCONTIGMEM_ENABLE
 	bool
-	default (ARCH_LH7A40X && !LH7A40X_CONTIGMEM)
+	default (ARCH_LH7A40X && !LH7A40X_CONTIGMEM) || ARCH_SA1100
 	help
 	  Say Y to support efficient handling of discontiguous physical memory,
 	  for architectures which are either NUMA (Non-Uniform Memory Access)

Re: Finding what change broke ARM

[Petr Baudis]

Dear diary, on Fri, Jun 24, 2005 at 11:19:51AM CEST, I got a letter
where Russell King <rmk+lkml@arm.linux.org.uk> told me that...
> When building current git for ARM, I see:
> 
>   CC      arch/arm/mm/consistent.o
> arch/arm/mm/consistent.c: In function `dma_free_coherent':
> arch/arm/mm/consistent.c:357: error: `mem_map' undeclared (first use in this function)
> arch/arm/mm/consistent.c:357: error: (Each undeclared identifier is reported only once
> arch/arm/mm/consistent.c:357: error: for each function it appears in.)
> make[2]: *** [arch/arm/mm/consistent.o] Error 1
> 
> How can I find what change elsewhere in the kernel tree caused this
> breakage?
> 
> With bk, you could ask for a per-file revision history of the likely
> candidates, and then find the changeset to view the other related
> changes.
> 
> With git... ?  We don't have per-file revision history so...

With Cogito, you can pass cg-log list of files, and it will show only
the history of the given files.

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
<Espy> be careful, some twit might quote you out of context..

Re: Finding what change broke ARM

[Dave Hansen]

On Fri, 2005-06-24 at 13:35 +0100, Andy Whitcroft wrote:
> > Well, this fixes the problem, but I doubt people will like it.
> 
> This looks like a problem with the way the configuration options where
> changed to allow more than two memory models for SPARSMEM.  I think the
> right fix is the patch below.  Russell could you try this one instead.
> Dave, you did most of the work on the configuration side could you look
> this over (assuming it works!).

That looks like the right fix.  Trying to "select" and option that's no
longer selectable is certainly a problem.

-- Dave

Re: Finding what change broke ARM

[Linus Torvalds]

On Fri, 24 Jun 2005, Russell King wrote:
>
> When building current git for ARM, I see:
> 
>   CC      arch/arm/mm/consistent.o
> arch/arm/mm/consistent.c: In function `dma_free_coherent':
> arch/arm/mm/consistent.c:357: error: `mem_map' undeclared (first use in this function)
> arch/arm/mm/consistent.c:357: error: (Each undeclared identifier is reported only once
> arch/arm/mm/consistent.c:357: error: for each function it appears in.)
> make[2]: *** [arch/arm/mm/consistent.o] Error 1
> 
> How can I find what change elsewhere in the kernel tree caused this
> breakage?

Ahhah! A real-world example of what cool things git can do. 

Anyway, the first starting point is _exactly_ the same as under BK, except 
the syntax is very different, and git does it better, in fact:

	git-whatchanged -p arch/arm/mm/consistent.c

However, in this case nothing has changed in that file over the whole 
git history, so you get an empty answer. Let's go to phase two, but first 
a comment:

> With bk, you could ask for a per-file revision history of the likely
> candidates, and then find the changeset to view the other related
> changes.
> 
> With git... ?  We don't have per-file revision history so...

We don't _store_ changes as per-file revision histories, but we do store 
it in a way where finding out what happened is efficient even per-file. 
While a line-by-line "annotate" is not efficient, the "what changed" 
certainly is.

And git actually does better than BK (or _any_ per-file history thing),
because "git-whatchanged" actually works over directories or multiple
independent files too, and it works purely on pathnames, so you can say
"git-whatchanged" for a file that has gone away to see _why_ it went away.  
In most other systems it's really hard to see what happened to something
that isn't there any more..

Anyway, the problem clearly didn't happen because of any changes to that 
file at all, so here per-file history simply doesn't help. But never fear, 
we're not screwed yet. In particular, you will now obviously suspect that 
since it wasn't that _file_ that changed, and since you know what changed 
in the ARM code, it's going to be a generic linux header file change that 
screwed you over.

So phase #2 is to do

	git-whatchanged -p include/linux

(which shows every commit that touches include/linux, and shows that part
as a patch, thus the "-p"). That starts up a pager on the results by
default, so we just be stupid about it and do a "/mem_map" to look for
changes that mention mem_map. Maybe we'll be lucky.

Even that doesn't show a whole lot: but it does point a very suspicious
finger to the recently merged sparse-mem stuff from Andy Whitcroft,
though.

And now you have a commit to look at, namely the "sparsemem memory model" 
one, commit ID d41dee369bff3b9dcb6328d4d822926c28cc2594.

In fact, looking at it, I think it's simply config option changes, and
probably the SPARSEMEM config option that has preempted your lack of
DISCONTIGMEM support.  But now you have somebody to blame and to ask for
help from: Andy Whitcroft and Dave Hansen, whom I've cc'd.

I might start phase #3 with

	git-whatchanged -p mm/Kconfig arch/arm/Kconfig

but at this point you may already have enough of a clue that you don't 
even care any more.

		Linus