[U-Boot] [Resend RFC PATCH 1/2] armv8: Fix dcache disable function

From: Stephen Warren <swarren@wwwdotorg.org>
To: u-boot@lists.denx.de
Subject: [U-Boot] [Resend RFC PATCH 1/2] armv8: Fix dcache disable function
Date: Fri, 28 Oct 2016 11:56:59 -0600	[thread overview]
Message-ID: <04fda8db-ee07-5469-f9e5-29af5062fbe0@wwwdotorg.org> (raw)
In-Reply-To: <AM4PR0401MB1732D8A5D18B7CF1601DD20C9AAD0@AM4PR0401MB1732.eurprd04.prod.outlook.com>

On 10/28/2016 11:38 AM, york sun wrote:
> On 10/26/2016 02:02 PM, york.sun at nxp.com wrote:
>>
>> I came back from my testing and I have more questions than answers.
>>
>> For _this_ patch, I proposed to flush cache before disabling them,
>> noting once the dcache is disabled, the staled data in dirty cache is
>> not visible to the core. My argument was if we flush L1/L2, they could
>> end up in L3 (I don't know for sure). If I want to skip flushing L3, I
>> have to fix it.
>>
>> During this discussion, I thought I made a mistake by flushing L1/L2 by
>> way/set first, then flushing by VA. Actually I didn't. I flushed by VA
>> first.
>>
>> With my today's test, the baseline (working in the sense of booting
>> Linux) is
>>
>> PATCH 1/2 armv8: Fix dcache disable function
>> PATCH 2/2 armv8: Fix flush_dcache_all function
>>
>> With these two patches, I flush the stack up to top of U-Boot by VA,
>> followed by flush by set/way. L3 is not flushed. Then d-cache is
>> disabled. I know this is not a real "flush all" procedure. With this
>> modified procedure, I can continue to boot Linux.
>>
>> If I revert patch 1, i.e. to disable dcache before flushing, I can see
>> the data is not visible from the core (debug with JTAG tool). My hope
>> was the staled data should be flushed to main memory if flushed by VA.
>> That's not the case. The main memory doesn't have the correct data. So
>> my new question is, why flushing by VA doesn't flush the data to main
>> memory? Do I need to flush the cache while cache is enabled?
>>
>
> Guys,
>
> I think I found the root cause of my data loss.
>
> Current code disables D-cache and MMU before flushing the cache. I think
> the problem is turning off MMU. MMU should stay on when we flush
> D-cache. We can turn it off after the flushing. Once I make this change,
> I can see the correct data in memory after flushing (by VA).
>
> Do you agree we should leave MMU on during flushing?

If you're "flushing" by VA, then I'm not surprised since the MMU is what 
defines the VA->PA mapping, and perhaps you have some physically tagged 
caches.

However, I believe U-Boot mainline currently "flushes" by set/way, which 
I wouldn't expect MMU status to influence at all.