Re: [PATCH v2] tools: libxc: flush data cache after loading images into guest memory

[Stefano Stabellini <stefano.stabellini@eu.citrix.com>]

On Fri, 13 Dec 2013, Ian Campbell wrote:
> On Fri, 2013-12-13 at 00:49 +0000, Julien Grall wrote:
> > 
> > On 12/12/2013 02:23 PM, Ian Campbell wrote:
> > > On ARM guest OSes are started with MMU and Caches disables (as they are on
> > > native) however caching is enabled in the domain running the builder and
> > > therefore we must flush the cache as we load the blobs, otherwise when the
> > > guest starts running it may not see them. The dom0 build in the hypervisor has
> > > the same requirements and already does the right thing.
> > >
> > > The mechanism for performing a cache flush from userspace is OS specific, so
> > > implement this as a new osdep hook:
> > >
> > >   - On 32-bit ARM Linux provides a system call to flush the cache.
> > >   - On 64-bit ARM Linux the processor is configured to allow cache flushes
> > >     directly from userspace.
> > >   - Non-Linux platforms will need to provide their own implementation. If
> > >     similar mechanisms are not available then a new privcmd ioctl should be a
> > >     suitable alternative.
> > >
> > > No cache maintenance is required on x86, so provide a stub for all non-Linux
> > > platforms which returns success on x86 only and log an error otherwise.
> > >
> > > This fixes guest building on Xgene which has a very large L3 cache and so is
> > > particularly susceptible to this problem. It has also been observed
> > > sporadically on midway.
> > 
> > This patch doesn't solve issue on Midway.
> 
> That's a shame. I think we should go ahead with this patch regardless,
> since it does fix arm64 and introduces the infrastructure for arm32. I
> think there is no harm in adding the syscall on arm32 for now.

I agree.
I wonder if QEMU (qdisk) is going to need similar cache flushes.


> > cacheflush syscall on ARM32 is calling DCCMVAU (Data Clean Cache by MVA
> > to PoU), that is not enough.
> > As I understand the ARM ARM B2.2.6 (page B2-1275):
> >      - PoC means the data will be written to the RAM
> >      - PoU means, in a same inner shareable domain, instruction/data
> > cache and translation page table will see the same value for a specific
> > MVA. It doesn't means that the data will reach the RAM.
> 
> This is essentially my understanding as well.
> 
> > I did some test and indeed DCCMVAC (Data Clean Cache By MVA to PoC)
> > resolves the problem on Midway (and generally on ARMv7).
> 
> Good.
> 
> > Unfortunately Linux doesn't provide any syscall to call this function
> > for ARMv7 and it's not possible to call cache instruction from
> > userspace. What we could do is:
> >      - Use the "flags" parameters of cacheflush syscall and call a
> > function which DCCMVAC (for instance __cpuc_flush_dcache_area)
> >      - Extend privcmd to have a flush cache ioctl
> 
> Personally I think the first is nicer, but ultimately we need input from
> l-a-k on this one and would be happy with either.

I agree. Can you try to come up with such a patch?