Re: [PATCH v2] tools: libxc: flush data cache after loading images into guest memory

[Julien Grall <julien.grall@linaro.org>]

On 12/12/2013 02:23 PM, Ian Campbell wrote:
> On ARM guest OSes are started with MMU and Caches disables (as they are on
> native) however caching is enabled in the domain running the builder and
> therefore we must flush the cache as we load the blobs, otherwise when the
> guest starts running it may not see them. The dom0 build in the hypervisor has
> the same requirements and already does the right thing.
>
> The mechanism for performing a cache flush from userspace is OS specific, so
> implement this as a new osdep hook:
>
>   - On 32-bit ARM Linux provides a system call to flush the cache.
>   - On 64-bit ARM Linux the processor is configured to allow cache flushes
>     directly from userspace.
>   - Non-Linux platforms will need to provide their own implementation. If
>     similar mechanisms are not available then a new privcmd ioctl should be a
>     suitable alternative.
>
> No cache maintenance is required on x86, so provide a stub for all non-Linux
> platforms which returns success on x86 only and log an error otherwise.
>
> This fixes guest building on Xgene which has a very large L3 cache and so is
> particularly susceptible to this problem. It has also been observed
> sporadically on midway.

This patch doesn't solve issue on Midway.

cacheflush syscall on ARM32 is calling DCCMVAU (Data Clean Cache by MVA
to PoU), that is not enough.
As I understand the ARM ARM B2.2.6 (page B2-1275):
     - PoC means the data will be written to the RAM
     - PoU means, in a same inner shareable domain, instruction/data
cache and translation page table will see the same value for a specific
MVA. It doesn't means that the data will reach the RAM.

I did some test and indeed DCCMVAC (Data Clean Cache By MVA to PoC)
resolves the problem on Midway (and generally on ARMv7).

Unfortunately Linux doesn't provide any syscall to call this function
for ARMv7 and it's not possible to call cache instruction from
userspace. What we could do is:
     - Use the "flags" parameters of cacheflush syscall and call a
function which DCCMVAC (for instance __cpuc_flush_dcache_area)
     - Extend privcmd to have a flush cache ioctl

Both solution would mean waiting Linux 3.14 (I don't think we can get an
accepted patch for 3.13).

I have also tried to trap PoU cache instruction (via HCR.TPU). But when 
Xen call DCCIMVAC/DCCIMVAU, the processor will raise a data abort fault.

Any thoughts?

-- 
Julien Grall