Re: [ARM] Native application design and discussion (I hope)

From: Julien Grall <julien.grall@arm.com>
To: Volodymyr Babchuk <vlad.babchuk@gmail.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
	Andrii Anisov <andrii_anisov@epam.com>,
	Steve Capper <Steve.Capper@arm.com>,
	Dario Faggioli <dario.faggioli@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Xen Devel <xen-devel@lists.xen.org>,
	Artem Mygaiev <joculator@gmail.com>
Subject: Re: [ARM] Native application design and discussion (I hope)
Date: Thu, 11 May 2017 11:07:10 +0100	[thread overview]
Message-ID: <c0dcdc19-248b-1910-eed8-330b4a7d844d@arm.com> (raw)
In-Reply-To: <3819da8c-dd3d-54ef-efa8-ff75dc6d89a7@arm.com>

Hello,

On 10/05/17 20:04, Julien Grall wrote:
> On 05/10/2017 06:37 PM, Volodymyr Babchuk wrote:
>> Hi Julien,
>
> Hi Volodymyr,
>
>> Returning back to Native apps, I think we can make ctx switch even
>> faster by dropping p2m code. Imagine that we already created stage 1
>> MMU for native application. Then to switch to app it we need only:
>>
>> 1. Enable TGE bit in HCR
>> 2. Disable VM bit in HCR
>> 3. Save/Program EL1_TTBR and friends
>> 3.5 (optionally) save/restore FPU state
>> 4. Save/Restore general purpose registers + SP + CSR + PC to jump to
>> an app in EL0 state.
>>
>> This can be done in "real" vcpu or in idle vcpu context. No
>> differences there.
>>
>> Exception handling in hypervisor would became tricky because of vcpu
>> absence for native app. Current implementation of entry.S always says
>> general purpose registers to a vcpu structure. Basically, we should
>> teach entry.S and traps.c about native apps.
>> Am I missing something?
>
> HCR_EL2.VM is allowed to be cached in the TLBs so for correctness you
> have to flush the TLBs everytime you change this bit (see D4.8.3 in ARM
> DDI 0487A.k_iss10775).
>
> Furthermore, as I mentioned earlier (see [1]) there are dependencies on
> the VMID even when stage-2 is disabled (see D4-1823 in ARM DDI
> 0487A.k_iss10775) so you have to program correctly VTTBR_EL2.VMID. This
> also means that if you use a different EL0 app, you have to ther use a
> different VMID or flush the TLBs.
>
> Bottom line, if you don't use stage-2 page table you have to flush the
> TLBs. Likely this will have an higher impact on the platform than using
> stage-2 page table.
>
> Virtual memory is quite tricky, someone needs to look at the ARM ARM and
> check all the behaviors when disabling either stage-1 or stage-2. There
> are memory attribute implications that may make tricky to move an EL0
> app between pCPU.

Looking again at the documentation and chatting with other ARM folks. I 
was wrong on some part, sorry for the confusion.

It turns out that if you don't need to flush the TLBs when disabling the 
HCR_EL2.VM (this is what Linux does for KVM). So disabling stage-2 for 
EL0 app would be ok.

But you still need to allocate a VMID per EL0 app as TLBs will still 
depend on it even with stage-2 disabled.

Even if we keep stage-2 enabled, we would have to create dummy page 
tables of stage-1 because the memory attribute would impact performance 
and at least not allow the EL0 app to move (see D4.2.8 in ARM DDI 
0487A.k_iss10775). In this case, 1:1 page tables with a block map (e.g 
1GB) would be sufficient and rely on stage-2 page tables.

Lastly, can you remind me with platform you are using for testing?

I hope this helps.

Cheers,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel