RE: [MIPS] SB1: Build fix: delete initialization of flush_icache_page pointer.

RE: [MIPS] SB1: Build fix: delete initialization of flush_icache_page pointer.

[Mark E Mason]

Hello all,

FYI:

The sb1_flash_icache_page change below breaks causes 1480 kernels to
hang after freeing memory:

[4294675.358000] Looking up port of RPC 100003/2 on 192.168.0.151

[4294675.360000] Looking up port of RPC 100005/1 on 192.168.0.151

[4294675.578000] VFS: Mounted root (nfs filesystem) readonly.

[4294675.579000] Freeing unused kernel memory: 212k freed           

The console still echos return characters, but booting doesn't progress
beyond this point.

Right before this change, everything appears to be ok.

/Mark

> -----Original Message-----
> From: linux-git-patches-bounce@linux-mips.org 
> [mailto:linux-git-patches-bounce@linux-mips.org] On Behalf Of 
> linux-mips@linux-mips.org
> Sent: Saturday, August 12, 2006 6:13 PM
> To: git-commits@linux-mips.org
> Subject: [MIPS] SB1: Build fix: delete initialization of 
> flush_icache_page pointer.
> 
> Author: Ralf Baechle <ralf@linux-mips.org> Sun Aug 13 
> 02:00:02 2006 +0100
> Commit: d95bc28e8c0bc1c39ef300ec2bca6ce99245cb50
> Gitweb: http://www.linux-mips.org/g/linux/d95bc28e
> Branch: master
> 
> Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
> 
> ---
> 
>  arch/mips/mm/c-sb1.c |    1 -
>  1 files changed, 0 insertions(+), 1 deletions(-)
> 
> diff --git a/arch/mips/mm/c-sb1.c b/arch/mips/mm/c-sb1.c
> index 5df454e..4bd9ad8 100644
> --- a/arch/mips/mm/c-sb1.c
> +++ b/arch/mips/mm/c-sb1.c
> @@ -520,7 +520,6 @@ #endif
>  
>  	/* These routines are for Icache coherence with the Dcache */
>  	flush_icache_range = sb1_flush_icache_range;
> -	flush_icache_page = sb1_flush_icache_page;
>  	__flush_icache_page = sb1_flush_icache_page;
>  	flush_icache_all = __sb1_flush_icache_all; /* local only */
>  
> 
> 
> 

Re: [MIPS] SB1: Build fix: delete initialization of flush_icache_page pointer.

[Atsushi Nemoto]

On Thu, 17 Aug 2006 14:20:07 -0700, "Mark E Mason" <mark.e.mason@broadcom.com> wrote:
> The sb1_flash_icache_page change below breaks causes 1480 kernels to
> hang after freeing memory:

Does this (untested) patch work for you?

diff --git a/arch/mips/mm/c-sb1.c b/arch/mips/mm/c-sb1.c
index 4bd9ad8..0f5691a 100644
--- a/arch/mips/mm/c-sb1.c
+++ b/arch/mips/mm/c-sb1.c
@@ -253,6 +253,17 @@ void sb1___flush_cache_all(void)
 	__attribute__((alias("local_sb1___flush_cache_all")));
 #endif
 
+static inline void local_sb1_flush_data_cache_page(void * addr)
+{
+	__sb1_writeback_inv_dcache_range((unsigned long)addr,
+					 (unsigned long)addr + PAGE_SIZE);
+}
+
+static void sb1_flush_data_cache_page(unsigned long addr)
+{
+	on_each_cpu(local_sb1_flush_data_cache_page, (void *) addr, 1, 1);
+}
+
 /*
  * When flushing a range in the icache, we have to first writeback
  * the dcache for the same range, so new ifetches will see any
@@ -527,8 +538,8 @@ #endif
 	flush_cache_page = sb1_flush_cache_page;
 
 	flush_cache_sigtramp = sb1_flush_cache_sigtramp;
-	local_flush_data_cache_page = (void *) sb1_nop;
-	flush_data_cache_page = (void *) sb1_nop;
+	local_flush_data_cache_page = local_sb1_flush_data_cache_page;
+	flush_data_cache_page = sb1_flush_data_cache_page;
 
 	/* Full flush */
 	__flush_cache_all = sb1___flush_cache_all;

RE: [MIPS] SB1: Build fix: delete initialization of flush_icache_page pointer.

[Manoj Ekbote]

The patch doesn't help. The kernel hangs in the same fashion. 



>-----Original Message-----
>From: linux-mips-bounce@linux-mips.org 
>[mailto:linux-mips-bounce@linux-mips.org] On Behalf Of Atsushi Nemoto
>Sent: Thursday, August 17, 2006 6:51 PM
>To: Mark E Mason
>Cc: linux-mips@linux-mips.org; ralf@linux-mips.org; ths@networkno.de
>Subject: Re: [MIPS] SB1: Build fix: delete initialization of 
>flush_icache_page pointer.
>
>On Thu, 17 Aug 2006 14:20:07 -0700, "Mark E Mason" 
><mark.e.mason@broadcom.com> wrote:
>> The sb1_flash_icache_page change below breaks causes 1480 kernels to 
>> hang after freeing memory:
>
>Does this (untested) patch work for you?
>
>diff --git a/arch/mips/mm/c-sb1.c b/arch/mips/mm/c-sb1.c index 
>4bd9ad8..0f5691a 100644
>--- a/arch/mips/mm/c-sb1.c
>+++ b/arch/mips/mm/c-sb1.c
>@@ -253,6 +253,17 @@ void sb1___flush_cache_all(void)
> 	__attribute__((alias("local_sb1___flush_cache_all")));
> #endif
> 
>+static inline void local_sb1_flush_data_cache_page(void * addr) {
>+	__sb1_writeback_inv_dcache_range((unsigned long)addr,
>+					 (unsigned long)addr + 
>PAGE_SIZE); }
>+
>+static void sb1_flush_data_cache_page(unsigned long addr) {
>+	on_each_cpu(local_sb1_flush_data_cache_page, (void *) 
>addr, 1, 1); }
>+
> /*
>  * When flushing a range in the icache, we have to first writeback
>  * the dcache for the same range, so new ifetches will see 
>any @@ -527,8 +538,8 @@ #endif
> 	flush_cache_page = sb1_flush_cache_page;
> 
> 	flush_cache_sigtramp = sb1_flush_cache_sigtramp;
>-	local_flush_data_cache_page = (void *) sb1_nop;
>-	flush_data_cache_page = (void *) sb1_nop;
>+	local_flush_data_cache_page = local_sb1_flush_data_cache_page;
>+	flush_data_cache_page = sb1_flush_data_cache_page;
> 
> 	/* Full flush */
> 	__flush_cache_all = sb1___flush_cache_all;
>
>
>

Re: [MIPS] SB1: Build fix: delete initialization of flush_icache_page pointer.

[Atsushi Nemoto]

On Mon, 21 Aug 2006 12:27:47 -0700, "Manoj Ekbote" <manoje@broadcom.com> wrote:
> The patch doesn't help. The kernel hangs in the same fashion. 

Thank you for testing.

Then I have no idea why the kernel hangs...


Random thoughts:

Does it still hang on init=/bin/sh?

Does enabling second and third "#if 0" blocks in arch/mips/mm/fault.c
show some useful information?

Finally, I think there is no serious reason separating c-sb1.c from
c-r4k.c.  The c-r4k.c support both vtagged-icache and pindexed-dcache,
therefore SB1 can use it too.

mm/c-r4k.c:probe_pcache()
	switch (c->cputype) {
	case CPU_20KC:
	case CPU_25KF:
	case CPU_SB1:
	case CPU_SB1A:
		c->dcache.flags |= MIPS_CACHE_PINDEX;
	case CPU_R10000:
	case CPU_R12000:
	case CPU_R14000:
		break;

kernel/cpu-probe.c:cpu_probe_sibyte()
#if 0
	c->options &= ~MIPS_CPU_4K_CACHE;
	c->options |= MIPS_CPU_SB1_CACHE;
#endif

---
Atsushi Nemoto

RE: [MIPS] SB1: Build fix: delete initialization of flush_icache_page pointer.

[Manoj Ekbote]

Hello,

The latest tip still has problems with booting on Broadcom boards.
I turned on a couple of printk's in the page fault handler and I see the
following messages:

[4294669.578000] VFS: Mounted root (nfs filesystem).
[4294669.579000] Freeing unused kernel memory: 116k freed
[4294669.585000] Cpu0[init:1:0000000010000540:1:ffffffff80214a1c]
[4294669.589000] Cpu0[init:1:000000002ab01a30:1:ffffffff80214a1c]
[4294669.594000] Cpu0[init:1:000000002aaa8b00:0:000000002aaa8b00]
[4294669.595000] Cpu0[init:1:000000002aaa9364:0:000000002aaa9364]
[4294669.596000] Cpu0[init:1:000000002aab3200:0:000000002aab3200]
[4294669.600000] Cpu0[init:1:0000000000000004:0:000000002aab32f4]
[4294669.601000] do_page_fault() #2: sending SIGSEGV to init for invalid
read access from
[4294669.601000] 0000000000000004 (epc == 000000002aab32f4, ra ==
000000002aaa8cbc)
[4294669.602000] Cpu0[init:1:0000000000000004:0:000000002aab32f4]
[4294669.603000] do_page_fault() #2: sending SIGSEGV to init for invalid
read access from
[4294669.603000] 0000000000000004 (epc == 000000002aab32f4, ra ==
000000002aaa8cbc)
[4294669.604000] Cpu0[init:1:0000000000000004:0:000000002aab32f4]
[4294669.605000] do_page_fault() #2: sending SIGSEGV to init for invalid
read access from
[4294669.605000] 0000000000000004 (epc == 000000002aab32f4, ra ==
000000002aaa8cbc)

It looks like a cache corruption issue.  Did the removal of
flush_icache_page
cause this? 

Any thoughts appreciated.

Thx,
/manoj


>-----Original Message-----
>From: Atsushi Nemoto [mailto:anemo@mba.ocn.ne.jp] 
>Sent: Tuesday, August 22, 2006 9:16 PM
>To: Manoj Ekbote
>Cc: Mark E Mason; linux-mips@linux-mips.org; 
>ralf@linux-mips.org; ths@networkno.de
>Subject: Re: [MIPS] SB1: Build fix: delete initialization of 
>flush_icache_page pointer.
>
>On Mon, 21 Aug 2006 12:27:47 -0700, "Manoj Ekbote" 
><manoje@broadcom.com> wrote:
>> The patch doesn't help. The kernel hangs in the same fashion. 
>
>Thank you for testing.
>
>Then I have no idea why the kernel hangs...
>
>
>Random thoughts:
>
>Does it still hang on init=/bin/sh?
>
>Does enabling second and third "#if 0" blocks in 
>arch/mips/mm/fault.c show some useful information?
>
>Finally, I think there is no serious reason separating c-sb1.c 
>from c-r4k.c.  The c-r4k.c support both vtagged-icache and 
>pindexed-dcache, therefore SB1 can use it too.
>
>mm/c-r4k.c:probe_pcache()
>	switch (c->cputype) {
>	case CPU_20KC:
>	case CPU_25KF:
>	case CPU_SB1:
>	case CPU_SB1A:
>		c->dcache.flags |= MIPS_CACHE_PINDEX;
>	case CPU_R10000:
>	case CPU_R12000:
>	case CPU_R14000:
>		break;
>
>kernel/cpu-probe.c:cpu_probe_sibyte()
>#if 0
>	c->options &= ~MIPS_CPU_4K_CACHE;
>	c->options |= MIPS_CPU_SB1_CACHE;
>#endif
>
>---
>Atsushi Nemoto
>
>

Re: [MIPS] SB1: Build fix: delete initialization of flush_icache_page pointer.

[Atsushi Nemoto]

On Mon, 25 Sep 2006 20:05:59 -0700, "Manoj Ekbote" <manoje@broadcom.com> wrote:
> The latest tip still has problems with booting on Broadcom boards.
> I turned on a couple of printk's in the page fault handler and I see the
> following messages:
...
> It looks like a cache corruption issue.  Did the removal of
> flush_icache_page
> cause this? 

Yes perhaps.  But flush_icache_page() itself is obsolete API and
should be gone (see Documentation/cachetlb.txt).  So there should be
fault in somewhere else.

I think 

http://www.linux-mips.org/archives/linux-mips/2006-08/msg00184.html

is still needed, but it seems not enough.

If reverting flush_icache_page() worked, then you can debug more.

1. kill part of local_sb1_flush_icache_page().

If killing __sb1_writeback_inv_dcache_phys_range() part was OK, then
it would be dcache issue, if killing drop_mmu_context() part was OK,
then it would be icache issue.

2. kill calls of flush_icache_page().

If is used in only 3 places.

mm/fremap.c:    flush_icache_page(vma, page);
mm/memory.c:    flush_icache_page(vma, page);
mm/memory.c:            flush_icache_page(vma, new_page);

Finding which lines are required might help further investigation.


Another possible approach might be trying c-r4k.c instead of c-sb1.c.

If you wanted to debug with latest git tree, note that
include/asm-mips/hazard.h in current lmo git tree seems broken.  The
fix was already posted to this ML so hopefully we can see in git tree
soon...

---
Atsushi Nemoto

RE: [MIPS] SB1: Build fix: delete initialization of flush_icache_page pointer.

[Manoj Ekbote]

Hi,

I tried the patch that you pointed to and UP kernel boots fine. Looks
like that was a icache and dcache coherency problem now that there is no
flush_icache_page implementation.

The SMP kernel still has trouble.Turning on the first printk in fault.c
makes it work..which is interesting.
I added a __sb1_flush_icache_range call with dcache invalidation..still
no help. I don't see a SIGSEGV message now.The kernel hangs with no
response to keystrokes and the soft lockup detect error shows up after
sometime.

Oh, inserting flush_icache_page caused the kernel to panic. I also see
that __flush_icache_page is not used anywhere. Any future use?

Thx.

>-----Original Message-----
>From: Atsushi Nemoto [mailto:anemo@mba.ocn.ne.jp] 
>Sent: Tuesday, September 26, 2006 2:40 AM
>To: Manoj Ekbote
>Cc: Mark E Mason; linux-mips@linux-mips.org; 
>ralf@linux-mips.org; ths@networkno.de
>Subject: Re: [MIPS] SB1: Build fix: delete initialization of 
>flush_icache_page pointer.
>
>On Mon, 25 Sep 2006 20:05:59 -0700, "Manoj Ekbote" 
><manoje@broadcom.com> wrote:
>> The latest tip still has problems with booting on Broadcom boards.
>> I turned on a couple of printk's in the page fault handler and I see 
>> the following messages:
>...
>> It looks like a cache corruption issue.  Did the removal of 
>> flush_icache_page cause this?
>
>Yes perhaps.  But flush_icache_page() itself is obsolete API 
>and should be gone (see Documentation/cachetlb.txt).  So there 
>should be fault in somewhere else.
>
>I think 
>
>http://www.linux-mips.org/archives/linux-mips/2006-08/msg00184.html
>
>is still needed, but it seems not enough.
>
>If reverting flush_icache_page() worked, then you can debug more.
>
>1. kill part of local_sb1_flush_icache_page().
>
>If killing __sb1_writeback_inv_dcache_phys_range() part was 
>OK, then it would be dcache issue, if killing 
>drop_mmu_context() part was OK, then it would be icache issue.
>
>2. kill calls of flush_icache_page().
>
>If is used in only 3 places.
>
>mm/fremap.c:    flush_icache_page(vma, page);
>mm/memory.c:    flush_icache_page(vma, page);
>mm/memory.c:            flush_icache_page(vma, new_page);
>
>Finding which lines are required might help further investigation.
>
>
>Another possible approach might be trying c-r4k.c instead of c-sb1.c.
>
>If you wanted to debug with latest git tree, note that 
>include/asm-mips/hazard.h in current lmo git tree seems 
>broken.  The fix was already posted to this ML so hopefully we 
>can see in git tree soon...
>
>---
>Atsushi Nemoto
>
>

Re: [MIPS] SB1: Build fix: delete initialization of flush_icache_page pointer.

[Atsushi Nemoto]

On Tue, 26 Sep 2006 18:54:40 -0700, "Manoj Ekbote" <manoje@broadcom.com> wrote:
> I tried the patch that you pointed to and UP kernel boots fine. Looks
> like that was a icache and dcache coherency problem now that there is no
> flush_icache_page implementation.

Hmm ... so it might be a SMP kernel problem?

> Oh, inserting flush_icache_page caused the kernel to panic. I also see
> that __flush_icache_page is not used anywhere. Any future use?

I think __flush_icache_page should go away.  Here is a patch.
http://www.linux-mips.org/archives/linux-mips/2006-09/msg00003.html

BTW, what you tried is something like this ?

include/asm-mips/cacheflush.h:
static inline void flush_icache_page(struct vm_area_struct *vma,
	struct page *page)
{
	__flush_icache_page(vma, page);
}

If this caused panic, what is the message?

---
Atsushi Nemoto

Re: [MIPS] SB1: Build fix: delete initialization of flush_icache_page pointer.

[Maciej W. Rozycki]

On Wed, 27 Sep 2006, Atsushi Nemoto wrote:

> BTW, what you tried is something like this ?
> 
> include/asm-mips/cacheflush.h:
> static inline void flush_icache_page(struct vm_area_struct *vma,
> 	struct page *page)
> {
> 	__flush_icache_page(vma, page);
> }
> 
> If this caused panic, what is the message?

 I have:

#define flush_icache_page __flush_icache_page

there for the time being while working on something else and it works just 
fine with my SWARM and 2.6.18-20060920.

  Maciej

RE: [MIPS] SB1: Build fix: delete initialization of flush_icache_page pointer.

[Manoj Ekbote]

>On Tue, 26 Sep 2006 18:54:40 -0700, "Manoj Ekbote" 
><manoje@broadcom.com> wrote:
>> I tried the patch that you pointed to and UP kernel boots 
>fine. Looks 
>> like that was a icache and dcache coherency problem now that 
>there is 
>> no flush_icache_page implementation.
>
>Hmm ... so it might be a SMP kernel problem?
>
>> Oh, inserting flush_icache_page caused the kernel to panic. 
>I also see 
>> that __flush_icache_page is not used anywhere. Any future use?
>
>I think __flush_icache_page should go away.  Here is a patch.
>http://www.linux-mips.org/archives/linux-mips/2006-09/msg00003.html
>
>BTW, what you tried is something like this ?

I added a line that initializes the flush_icache_page pointer in
sb1_cache_init.
The below method worked. The SMP kernel boots fine now. Removing parts
of local_sb1_flush_icache_page doesn't help. It looks like
flush_icache_page in mm/memory.c:do_no_page is needed. Removing it will
fail the boot process.

I am wondering if people have booted the latest tree on non-Broadcom
boards...curious to know if the removal of flush_icache_page has
affected them.

Thanks.

>include/asm-mips/cacheflush.h:
>static inline void flush_icache_page(struct vm_area_struct *vma,
>	struct page *page)
>{
>	__flush_icache_page(vma, page);
>}
>
>If this caused panic, what is the message?
>
>---
>Atsushi Nemoto
>
>

Re: [MIPS] SB1: Build fix: delete initialization of flush_icache_page pointer.

[Atsushi Nemoto]

On Wed, 27 Sep 2006 19:17:16 -0700, "Manoj Ekbote" <manoje@broadcom.com> wrote:
> I added a line that initializes the flush_icache_page pointer in
> sb1_cache_init.
> The below method worked. The SMP kernel boots fine now. Removing parts
> of local_sb1_flush_icache_page doesn't help. It looks like
> flush_icache_page in mm/memory.c:do_no_page is needed. Removing it will
> fail the boot process.

Thank you for testing.

Now I'm wondering why do_no_page() works on SMP.

	if (pte_none(*page_table)) {
		flush_icache_page(vma, new_page);
		entry = mk_pte(new_page, vma->vm_page_prot);
		if (write_access)
			entry = maybe_mkwrite(pte_mkdirty(entry), vma);
		set_pte_at(mm, address, page_table, entry);
...
	} else {
		/* One of our sibling threads was faster, back out. */
		page_cache_release(new_page);
		goto unlock;
	}

	/* no need to invalidate: a not-present page shouldn't be cached */
	update_mmu_cache(vma, address, entry);

Other CPU might be able to load new pte value just after set_ste_at(),
but dcache is not flushed until update_mmu_cache().  Maybe I missed
something...

> I am wondering if people have booted the latest tree on non-Broadcom
> boards...curious to know if the removal of flush_icache_page has
> affected them.

Yes, Ralf and Thiemo said even some SB1 boards can work fine without
flush_icache_page.

---
Atsushi Nemoto

Re: [MIPS] SB1: Build fix: delete initialization of flush_icache_page pointer.

[Ralf Baechle]

On Wed, Sep 27, 2006 at 07:17:16PM -0700, Manoj Ekbote wrote:

> I am wondering if people have booted the latest tree on non-Broadcom
> boards...curious to know if the removal of flush_icache_page has
> affected them.

I applied that patch only after some positive test feedback ...

  Ralf

Re: [MIPS] SB1: Build fix: delete initialization of flush_icache_page pointer.

[Ralf Baechle]

On Thu, Sep 28, 2006 at 01:27:12PM +0900, Atsushi Nemoto wrote:
> Date:	Thu, 28 Sep 2006 13:27:12 +0900 (JST)
> To:	manoje@broadcom.com
> Cc:	linux-mips@linux-mips.org, ralf@linux-mips.org,
> 	mark.e.mason@broadcom.com
> Subject: Re: [MIPS] SB1: Build fix: delete initialization of
>  flush_icache_page pointer.
> From:	Atsushi Nemoto <anemo@mba.ocn.ne.jp>
> Content-Type: Text/Plain; charset=us-ascii
> 
> On Wed, 27 Sep 2006 19:17:16 -0700, "Manoj Ekbote" <manoje@broadcom.com> wrote:
> > I added a line that initializes the flush_icache_page pointer in
> > sb1_cache_init.
> > The below method worked. The SMP kernel boots fine now. Removing parts
> > of local_sb1_flush_icache_page doesn't help. It looks like
> > flush_icache_page in mm/memory.c:do_no_page is needed. Removing it will
> > fail the boot process.
> 
> Thank you for testing.
> 
> Now I'm wondering why do_no_page() works on SMP.
> 
> 	if (pte_none(*page_table)) {
> 		flush_icache_page(vma, new_page);
> 		entry = mk_pte(new_page, vma->vm_page_prot);
> 		if (write_access)
> 			entry = maybe_mkwrite(pte_mkdirty(entry), vma);
> 		set_pte_at(mm, address, page_table, entry);
> ...
> 	} else {
> 		/* One of our sibling threads was faster, back out. */
> 		page_cache_release(new_page);
> 		goto unlock;
> 	}
> 
> 	/* no need to invalidate: a not-present page shouldn't be cached */
> 	update_mmu_cache(vma, address, entry);
> 
> Other CPU might be able to load new pte value just after set_ste_at(),
> but dcache is not flushed until update_mmu_cache().  Maybe I missed
> something...
> 
> > I am wondering if people have booted the latest tree on non-Broadcom
> > boards...curious to know if the removal of flush_icache_page has
> > affected them.
> 
> Yes, Ralf and Thiemo said even some SB1 boards can work fine without
> flush_icache_page.

The have to by definition - flush_icache_page is going to go away.

Note the SB1 cache code is a bizarre beast.  It totally avoids Hit-type
cacheops because those used to have problems on early silicon.  So my
favorite solution would be to dump c-sb1.c and deal with any SB1 issues
that may pop up in c-r4k.c.

  Ralf

RE: [MIPS] SB1: Build fix: delete initialization of flush_icache_page pointer.

[Manoj Ekbote]

-----Original Message-----
From: Atsushi Nemoto [mailto:anemo@mba.ocn.ne.jp]
Sent: Wed 9/27/2006 9:27 PM
To: Manoj Ekbote
Cc: linux-mips@linux-mips.org; ralf@linux-mips.org; Mark E Mason
Subject: Re: [MIPS] SB1: Build fix: delete initialization of flush_icache_page pointer.
 
On Wed, 27 Sep 2006 19:17:16 -0700, "Manoj Ekbote" <manoje@broadcom.com> wrote:
> I added a line that initializes the flush_icache_page pointer in
> sb1_cache_init.
> The below method worked. The SMP kernel boots fine now. Removing parts
> of local_sb1_flush_icache_page doesn't help. It looks like
> flush_icache_page in mm/memory.c:do_no_page is needed. Removing it will
> fail the boot process.

Thank you for testing.

Now I'm wondering why do_no_page() works on SMP.

	if (pte_none(*page_table)) {
		flush_icache_page(vma, new_page);
		entry = mk_pte(new_page, vma->vm_page_prot);
		if (write_access)
			entry = maybe_mkwrite(pte_mkdirty(entry), vma);
		set_pte_at(mm, address, page_table, entry);
...
	} else {
		/* One of our sibling threads was faster, back out. */
		page_cache_release(new_page);
		goto unlock;
	}

	/* no need to invalidate: a not-present page shouldn't be cached */
	update_mmu_cache(vma, address, entry);

Other CPU might be able to load new pte value just after set_ste_at(),
but dcache is not flushed until update_mmu_cache().  Maybe I missed
something...

It looks like the page with instructions has to be invalidated.I did a flush_cache_page in __update_cache.That seems to help too.

> I am wondering if people have booted the latest tree on non-Broadcom
> boards...curious to know if the removal of flush_icache_page has
> affected them.

Yes, Ralf and Thiemo said even some SB1 boards can work fine without
flush_icache_page.

---
Atsushi Nemoto

Re: [MIPS] SB1: Build fix: delete initialization of flush_icache_page pointer.

[Atsushi Nemoto]

Yet another build fix.

Signed-off-by: Atsushi Nemoto <anemo@mba.ocn.ne.jp>

diff --git a/arch/mips/mm/c-r3k.c b/arch/mips/mm/c-r3k.c
index ccd61f6..e1f35ef 100644
--- a/arch/mips/mm/c-r3k.c
+++ b/arch/mips/mm/c-r3k.c
@@ -335,8 +335,8 @@ void __init r3k_cache_init(void)
 	flush_cache_mm = r3k_flush_cache_mm;
 	flush_cache_range = r3k_flush_cache_range;
 	flush_cache_page = r3k_flush_cache_page;
-	flush_icache_page = r3k_flush_icache_page;
-	__flush_icache_range = r3k_flush_icache_range;
+	__flush_icache_page = r3k_flush_icache_page;
+	flush_icache_range = r3k_flush_icache_range;
 
 	flush_cache_sigtramp = r3k_flush_cache_sigtramp;
 	local_flush_data_cache_page = local_r3k_flush_data_cache_page;

Re: [MIPS] SB1: Build fix: delete initialization of flush_icache_page pointer.

[Atsushi Nemoto]

One more.

[MIPS] missing bits for "Retire flush_icache_page from mm use."

Signed-off-by: Atsushi Nemoto <anemo@mba.ocn.ne.jp>

diff --git a/arch/mips/mm/c-tx39.c b/arch/mips/mm/c-tx39.c
index cdb1942..932a09d 100644
--- a/arch/mips/mm/c-tx39.c
+++ b/arch/mips/mm/c-tx39.c
@@ -408,7 +408,7 @@ void __init tx39_cache_init(void)
 		flush_cache_mm = tx39_flush_cache_mm;
 		flush_cache_range = tx39_flush_cache_range;
 		flush_cache_page = tx39_flush_cache_page;
-		flush_icache_page = tx39_flush_icache_page;
+		__flush_icache_page = tx39_flush_icache_page;
 		flush_icache_range = tx39_flush_icache_range;
 
 		flush_cache_sigtramp = tx39_flush_cache_sigtramp;