TEE with XEN

TEE with XEN

[Peng Fan]

Hi All,

While enabling trusty os with xen, I took same approach as OP-TEE, 
with OP-TEE running in secure world. But I am also thinking this might
introduce potential issue is that secure world OS communicate with DomU.
If there are some misbehavior in secure world OS, it might let XEN
hypervisor not work proper.

In my setup, trusty os sometimes panic in secure world, xen will not able
to control the panic core anymore.

So I am thinking whether we need to emulating secure world in a XEN VM
which is the VM running DomU. Just like what ACRN did to run trusty
os.

Any comments?

Thanks,
Peng.

Re: [Tee-dev] TEE with XEN

[Volodymyr Babchuk]

Hi Peng,

On Mon, 15 Jun 2020 at 05:07, Peng Fan <peng.fan@nxp.com> wrote:
>
> Hi All,
>
> While enabling trusty os with xen, I took same approach as OP-TEE,
> with OP-TEE running in secure world. But I am also thinking this might
> introduce potential issue is that secure world OS communicate with DomU.
> If there are some misbehavior in secure world OS, it might let XEN
> hypervisor not work proper.
>
> In my setup, trusty os sometimes panic in secure world, xen will not able
> to control the panic core anymore.
>
> So I am thinking whether we need to emulating secure world in a XEN VM
> which is the VM running DomU. Just like what ACRN did to run trusty
> os.

Well, it depends on whom you are trusting more. Both XEN and TEE are minimal
OS implementations with aim at security. I'm speaking about generic TEE OS, not
about particular OS like OP-TEE or Trusty. Problem is that, if TEE is
running inside
VM, it will be susceptible to a hypervisor misbehaviour. You need to understand
that Xen and privileged domain (dom0, mostly) can access memory of any guest.
At least, in default configuration. There are means to harden this
setup. But anyways,
Xen can't be stopped from reading TEE's secrets.

If this is okay for your needs, then you can run TEE as a VM of course.

So, this is heavilly depends on your security threats model. There
can't be universal
solution. Also, I'm proposing to check Google's requirements for
Trusty environment.
Do they allow it to run outside of TrustZone? For example, GPD TEE System
Architecture document clearly says that TEE should be separated from REE by
hardware mechanisms that are not controlled by REE (section 2.2.1). I
believe, that
should be a similar document for Trusty.

-- 
WBR Volodymyr Babchuk aka lorc [+380976646013]
mailto: vlad.babchuk@gmail.com

RE: [Tee-dev] TEE with XEN

[Peng Fan]

Hi,

> Subject: Re: [Tee-dev] TEE with XEN
> 
> Hi Peng,
> 
> On Mon, 15 Jun 2020 at 05:07, Peng Fan <peng.fan@nxp.com> wrote:
> >
> > Hi All,
> >
> > While enabling trusty os with xen, I took same approach as OP-TEE,
> > with OP-TEE running in secure world. But I am also thinking this might
> > introduce potential issue is that secure world OS communicate with DomU.
> > If there are some misbehavior in secure world OS, it might let XEN
> > hypervisor not work proper.
> >
> > In my setup, trusty os sometimes panic in secure world, xen will not
> > able to control the panic core anymore.
> >
> > So I am thinking whether we need to emulating secure world in a XEN VM
> > which is the VM running DomU. Just like what ACRN did to run trusty
> > os.
> 
> Well, it depends on whom you are trusting more. Both XEN and TEE are
> minimal OS implementations with aim at security. 

XEN is targeting safety.
TEE is targeting security.

I'm speaking about generic
> TEE OS, not about particular OS like OP-TEE or Trusty. Problem is that, if TEE is
> running inside VM, it will be susceptible to a hypervisor misbehaviour. You
> need to understand that Xen and privileged domain (dom0, mostly) can access
> memory of any guest.
> At least, in default configuration. There are means to harden this setup. But
> anyways, Xen can't be stopped from reading TEE's secrets.

Yes. Understand.

> 
> If this is okay for your needs, then you can run TEE as a VM of course.
> 
> So, this is heavilly depends on your security threats model. There can't be
> universal solution. Also, I'm proposing to check Google's requirements for
> Trusty environment.

Let me try to ask Google guys to see any feedback.

Thanks,
Peng.

> Do they allow it to run outside of TrustZone? For example, GPD TEE System
> Architecture document clearly says that TEE should be separated from REE by
> hardware mechanisms that are not controlled by REE (section 2.2.1). I believe,
> that should be a similar document for Trusty.
> 
> --
> WBR Volodymyr Babchuk aka lorc [+380976646013]
> mailto: vlad.babchuk@gmail.com

Re: [Tee-dev] TEE with XEN

[Julien Grall]

+Bertrand and Stefano

On 16/06/2020 02:24, Volodymyr Babchuk wrote:
> Hi Peng,
> 
> On Mon, 15 Jun 2020 at 05:07, Peng Fan <peng.fan@nxp.com> wrote:
>>
>> Hi All,
>>
>> While enabling trusty os with xen, I took same approach as OP-TEE,
>> with OP-TEE running in secure world. But I am also thinking this might
>> introduce potential issue is that secure world OS communicate with DomU.
>> If there are some misbehavior in secure world OS, it might let XEN
>> hypervisor not work proper.
>>
>> In my setup, trusty os sometimes panic in secure world, xen will not able
>> to control the panic core anymore.

May I ask in which case Trusty is panicking?

>>
>> So I am thinking whether we need to emulating secure world in a XEN VM
>> which is the VM running DomU. Just like what ACRN did to run trusty
>> os.
> 
> Well, it depends on whom you are trusting more. Both XEN and TEE are minimal
> OS implementations with aim at security. I'm speaking about generic TEE OS, not
> about particular OS like OP-TEE or Trusty. Problem is that, if TEE is
> running inside
> VM, it will be susceptible to a hypervisor misbehaviour. You need to understand
> that Xen and privileged domain (dom0, mostly) can access memory of any guest.
> At least, in default configuration. There are means to harden this
> setup. But anyways,
> Xen can't be stopped from reading TEE's secrets.

IIRC, we discussed this approach for OP-TEE in the past. There was other 
potential pitfalls with it. For instance, you wouldn't be able to 
directly access any secure device from that guest (it is running in 
non-secure world).

There are also issues in term of latency as you may have the following 
model:

domU -> Xen -> domU TEE -> (Xen -> host TEE -> Xen -> domU TEE) -> Xen 
-> domU.

The bit in () is if you require to call the host TEE.

One possibility would be to use Secure-EL2 for your Trusty OS. But I 
don't know whether your platform supports it.

Depending on whether you can modify Trusty OS, alternative would be to 
make itvirtualization aware as OP-TEE did. The core would need to be 
resilient and the panic only affect a given client.

Cheers,

-- 
Julien Grall

Re: [Tee-dev] TEE with XEN

[Stefano Stabellini]

On Thu, 18 Jun 2020, Julien Grall wrote:
> +Bertrand and Stefano

Thanks for CC'ing me


> On 16/06/2020 02:24, Volodymyr Babchuk wrote:
> > Hi Peng,
> > 
> > On Mon, 15 Jun 2020 at 05:07, Peng Fan <peng.fan@nxp.com> wrote:
> > > 
> > > Hi All,
> > > 
> > > While enabling trusty os with xen, I took same approach as OP-TEE,
> > > with OP-TEE running in secure world. But I am also thinking this might
> > > introduce potential issue is that secure world OS communicate with DomU.
> > > If there are some misbehavior in secure world OS, it might let XEN
> > > hypervisor not work proper.
> > > 
> > > In my setup, trusty os sometimes panic in secure world, xen will not able
> > > to control the panic core anymore.
> 
> May I ask in which case Trusty is panicking?
> 
> > > 
> > > So I am thinking whether we need to emulating secure world in a XEN VM
> > > which is the VM running DomU. Just like what ACRN did to run trusty
> > > os.
> > 
> > Well, it depends on whom you are trusting more. Both XEN and TEE are minimal
> > OS implementations with aim at security. I'm speaking about generic TEE OS,
> > not
> > about particular OS like OP-TEE or Trusty. Problem is that, if TEE is
> > running inside
> > VM, it will be susceptible to a hypervisor misbehaviour. You need to
> > understand
> > that Xen and privileged domain (dom0, mostly) can access memory of any
> > guest.
> > At least, in default configuration. There are means to harden this
> > setup. But anyways,
> > Xen can't be stopped from reading TEE's secrets.
> 
> IIRC, we discussed this approach for OP-TEE in the past. There was other
> potential pitfalls with it. For instance, you wouldn't be able to directly
> access any secure device from that guest (it is running in non-secure world).

Given that Secure World has access to Normal World but not vice versa,
it is more natural to run Trusty in one of the Secure execution levels.
The expectation is that Trusty has higher privilege, thus it is more
"trusted" than anything running in Normal World, including Xen.

Of course no client should be able to crash Trusty, so the reality
sometimes can be very different from the theory :-)

If I were you, I would consider running the TEE "inside" the VM only if
the service that the TEE provides is strongly coupled with the VM and
doesn't actually need a high level of privilege to function (i.e.
doesn't access secure devices or at least not often.)


> Depending on whether you can modify Trusty OS, alternative would be to make
> itvirtualization aware as OP-TEE did. The core would need to be resilient and
> the panic only affect a given client.

This would most probably be the best compromise.

Re: [Tee-dev] TEE with XEN

[Bertrand Marquis]

Hi,

> On 18 Jun 2020, at 19:05, Julien Grall <julien@xen.org> wrote:
> 
> +Bertrand and Stefano
> 
> On 16/06/2020 02:24, Volodymyr Babchuk wrote:
>> Hi Peng,
>> On Mon, 15 Jun 2020 at 05:07, Peng Fan <peng.fan@nxp.com> wrote:
>>> 
>>> Hi All,
>>> 
>>> While enabling trusty os with xen, I took same approach as OP-TEE,
>>> with OP-TEE running in secure world. But I am also thinking this might
>>> introduce potential issue is that secure world OS communicate with DomU.
>>> If there are some misbehavior in secure world OS, it might let XEN
>>> hypervisor not work proper.
>>> 
>>> In my setup, trusty os sometimes panic in secure world, xen will not able
>>> to control the panic core anymore.
> 
> May I ask in which case Trusty is panicking?

In any case, optee should protect itself against this and it should be considered that optee is more priviledged then Xen.
So if optee is crashing we cannot expect that Xen can recover or fix it.

I would more consider this as a bug that optee needs to be robust against.

> 
>>> 
>>> So I am thinking whether we need to emulating secure world in a XEN VM
>>> which is the VM running DomU. Just like what ACRN did to run trusty
>>> os.
>> Well, it depends on whom you are trusting more. Both XEN and TEE are minimal
>> OS implementations with aim at security. I'm speaking about generic TEE OS, not
>> about particular OS like OP-TEE or Trusty. Problem is that, if TEE is
>> running inside
>> VM, it will be susceptible to a hypervisor misbehaviour. You need to understand
>> that Xen and privileged domain (dom0, mostly) can access memory of any guest.
>> At least, in default configuration. There are means to harden this
>> setup. But anyways,
>> Xen can't be stopped from reading TEE's secrets.
> 
> IIRC, we discussed this approach for OP-TEE in the past. There was other potential pitfalls with it. For instance, you wouldn't be able to directly access any secure device from that guest (it is running in non-secure world).
> 
> There are also issues in term of latency as you may have the following model:
> 
> domU -> Xen -> domU TEE -> (Xen -> host TEE -> Xen -> domU TEE) -> Xen -> domU.
> 
> The bit in () is if you require to call the host TEE.
> 
> One possibility would be to use Secure-EL2 for your Trusty OS. But I don't know whether your platform supports it.
> 
> Depending on whether you can modify Trusty OS, alternative would be to make itvirtualization aware as OP-TEE did. The core would need to be resilient and the panic only affect a given client.

I do not have right a clear idea of what is the status of optee and xen but in theory I would see 2 possible ways to handle this:
- without optee modification, something in a guest (Dom0 or an other priviledged one) needs to have access to optee and emulate optee access for others.
- with optee modifications, optee needs to have a concept of client and Xen would need to passthrough optee requests but being responsible of adding a “client” identifier. Maybe also informing Optee when a new client is created/removed.

The second scenario could then be somehow splitted in the previous one from Julien if some parts would need to be emulated somewhere in some kind of combination of the 2 models.

In any case i would always consider that anything running on optee (or in general in the secure world) is more trusted then Xen.

Regards
Bertrand

Re: [Tee-dev] TEE with XEN

[Bertrand Marquis]

> On 18 Jun 2020, at 23:17, Stefano Stabellini <sstabellini@kernel.org> wrote:
> 
> On Thu, 18 Jun 2020, Julien Grall wrote:
>> +Bertrand and Stefano
> 
> Thanks for CC'ing me
> 
> 
>> On 16/06/2020 02:24, Volodymyr Babchuk wrote:
>>> Hi Peng,
>>> 
>>> On Mon, 15 Jun 2020 at 05:07, Peng Fan <peng.fan@nxp.com> wrote:
>>>> 
>>>> Hi All,
>>>> 
>>>> While enabling trusty os with xen, I took same approach as OP-TEE,
>>>> with OP-TEE running in secure world. But I am also thinking this might
>>>> introduce potential issue is that secure world OS communicate with DomU.
>>>> If there are some misbehavior in secure world OS, it might let XEN
>>>> hypervisor not work proper.
>>>> 
>>>> In my setup, trusty os sometimes panic in secure world, xen will not able
>>>> to control the panic core anymore.
>> 
>> May I ask in which case Trusty is panicking?
>> 
>>>> 
>>>> So I am thinking whether we need to emulating secure world in a XEN VM
>>>> which is the VM running DomU. Just like what ACRN did to run trusty
>>>> os.
>>> 
>>> Well, it depends on whom you are trusting more. Both XEN and TEE are minimal
>>> OS implementations with aim at security. I'm speaking about generic TEE OS,
>>> not
>>> about particular OS like OP-TEE or Trusty. Problem is that, if TEE is
>>> running inside
>>> VM, it will be susceptible to a hypervisor misbehaviour. You need to
>>> understand
>>> that Xen and privileged domain (dom0, mostly) can access memory of any
>>> guest.
>>> At least, in default configuration. There are means to harden this
>>> setup. But anyways,
>>> Xen can't be stopped from reading TEE's secrets.
>> 
>> IIRC, we discussed this approach for OP-TEE in the past. There was other
>> potential pitfalls with it. For instance, you wouldn't be able to directly
>> access any secure device from that guest (it is running in non-secure world).
> 
> Given that Secure World has access to Normal World but not vice versa,
> it is more natural to run Trusty in one of the Secure execution levels.
> The expectation is that Trusty has higher privilege, thus it is more
> "trusted" than anything running in Normal World, including Xen.
> 
> Of course no client should be able to crash Trusty, so the reality
> sometimes can be very different from the theory :-)
> 
> If I were you, I would consider running the TEE "inside" the VM only if
> the service that the TEE provides is strongly coupled with the VM and
> doesn't actually need a high level of privilege to function (i.e.
> doesn't access secure devices or at least not often.)
> 

I could see some scenarios where this would make sense if you trust one VM more then an other.
But this falls into a model of “virtualizing” optee using a VM where from the system point of view any usage of this VM from the functionality could not be more trusted then Xen And the VM providing the service.

> 
>> Depending on whether you can modify Trusty OS, alternative would be to make
>> itvirtualization aware as OP-TEE did. The core would need to be resilient and
>> the panic only affect a given client.
> 
> This would most probably be the best compromise.

Agree.

Bertrand

RE: [Tee-dev] TEE with XEN

[Peng Fan]

Hi Bertrand,

> Subject: Re: [Tee-dev] TEE with XEN
> 
> Hi,
> 
> > On 18 Jun 2020, at 19:05, Julien Grall <julien@xen.org> wrote:
> >
> > +Bertrand and Stefano
> >
> > On 16/06/2020 02:24, Volodymyr Babchuk wrote:
> >> Hi Peng,
> >> On Mon, 15 Jun 2020 at 05:07, Peng Fan <peng.fan@nxp.com> wrote:
> >>>
> >>> Hi All,
> >>>
> >>> While enabling trusty os with xen, I took same approach as OP-TEE,
> >>> with OP-TEE running in secure world. But I am also thinking this
> >>> might introduce potential issue is that secure world OS communicate with
> DomU.
> >>> If there are some misbehavior in secure world OS, it might let XEN
> >>> hypervisor not work proper.
> >>>
> >>> In my setup, trusty os sometimes panic in secure world, xen will not
> >>> able to control the panic core anymore.
> >
> > May I ask in which case Trusty is panicking?
> 
> In any case, optee should protect itself against this and it should be
> considered that optee is more priviledged then Xen.
> So if optee is crashing we cannot expect that Xen can recover or fix it.
> 
> I would more consider this as a bug that optee needs to be robust against.

ok. I am not using OP-TEE, currently I use google trusty OS.

I have two OS, Dom0 linux + DomU android auto.

DomU android auto needs trusty OS, Dom0 Linux not need that.

I not wanna trusty OS for DomU could bring any detect to Dom0 or xen.

One more case is if dom0 linux needs OP-TEE, DomU needs google trusty,
how we handle this in one SoC?

> 
> >
> >>>
> >>> So I am thinking whether we need to emulating secure world in a XEN
> >>> VM which is the VM running DomU. Just like what ACRN did to run
> >>> trusty os.
> >> Well, it depends on whom you are trusting more. Both XEN and TEE are
> >> minimal OS implementations with aim at security. I'm speaking about
> >> generic TEE OS, not about particular OS like OP-TEE or Trusty.
> >> Problem is that, if TEE is running inside VM, it will be susceptible
> >> to a hypervisor misbehaviour. You need to understand that Xen and
> >> privileged domain (dom0, mostly) can access memory of any guest.
> >> At least, in default configuration. There are means to harden this
> >> setup. But anyways, Xen can't be stopped from reading TEE's secrets.
> >
> > IIRC, we discussed this approach for OP-TEE in the past. There was other
> potential pitfalls with it. For instance, you wouldn't be able to directly access
> any secure device from that guest (it is running in non-secure world).
> >
> > There are also issues in term of latency as you may have the following
> model:
> >
> > domU -> Xen -> domU TEE -> (Xen -> host TEE -> Xen -> domU TEE) -> Xen ->
> domU.
> >
> > The bit in () is if you require to call the host TEE.
> >
> > One possibility would be to use Secure-EL2 for your Trusty OS. But I don't
> know whether your platform supports it.
> >
> > Depending on whether you can modify Trusty OS, alternative would be to
> make itvirtualization aware as OP-TEE did. The core would need to be
> resilient and the panic only affect a given client.
> 
> I do not have right a clear idea of what is the status of optee and xen but in
> theory I would see 2 possible ways to handle this:
> - without optee modification, something in a guest (Dom0 or an other
> priviledged one) needs to have access to optee and emulate optee access for
> others.
> - with optee modifications, optee needs to have a concept of client and Xen
> would need to passthrough optee requests but being responsible of adding a
> “client” identifier. Maybe also informing Optee when a new client is
> created/removed.
> 
> The second scenario could then be somehow splitted in the previous one from
> Julien if some parts would need to be emulated somewhere in some kind of
> combination of the 2 models.
> 
> In any case i would always consider that anything running on optee (or in
> general in the secure world) is more trusted then Xen.

Ok, this means optee runs on all cores in secure world, but this would not
work when we need to support multiple OSes with their own TEE.

Regards,
Peng.

> 
> Regards
> Bertrand

Re: [Tee-dev] TEE with XEN

[Bertrand Marquis]

> On 19 Jun 2020, at 09:52, Peng Fan <peng.fan@nxp.com> wrote:
> 
> Hi Bertrand,
> 
>> Subject: Re: [Tee-dev] TEE with XEN
>> 
>> Hi,
>> 
>>> On 18 Jun 2020, at 19:05, Julien Grall <julien@xen.org> wrote:
>>> 
>>> +Bertrand and Stefano
>>> 
>>> On 16/06/2020 02:24, Volodymyr Babchuk wrote:
>>>> Hi Peng,
>>>> On Mon, 15 Jun 2020 at 05:07, Peng Fan <peng.fan@nxp.com> wrote:
>>>>> 
>>>>> Hi All,
>>>>> 
>>>>> While enabling trusty os with xen, I took same approach as OP-TEE,
>>>>> with OP-TEE running in secure world. But I am also thinking this
>>>>> might introduce potential issue is that secure world OS communicate with
>> DomU.
>>>>> If there are some misbehavior in secure world OS, it might let XEN
>>>>> hypervisor not work proper.
>>>>> 
>>>>> In my setup, trusty os sometimes panic in secure world, xen will not
>>>>> able to control the panic core anymore.
>>> 
>>> May I ask in which case Trusty is panicking?
>> 
>> In any case, optee should protect itself against this and it should be
>> considered that optee is more priviledged then Xen.
>> So if optee is crashing we cannot expect that Xen can recover or fix it.
>> 
>> I would more consider this as a bug that optee needs to be robust against.
> 
> ok. I am not using OP-TEE, currently I use google trusty OS.

Sorry i should have been more generic.
Please read this as “Anything running in secure world”, being optee or trusty.

> 
> I have two OS, Dom0 linux + DomU android auto.
> 
> DomU android auto needs trusty OS, Dom0 Linux not need that.

But i would guess your Dom0 is more “critical” then your DomU.
In this case you must make sure that any resource given to your DomU cannot affect your Dom0.
For example: if the DomU is starting a very heavy operation in blocked in trusty, any interrupt for non-secure could be blocked, thus affecting the ability of your Dom0.

> 
> I not wanna trusty OS for DomU could bring any detect to Dom0 or xen.
> 
> One more case is if dom0 linux needs OP-TEE, DomU needs google trusty,
> how we handle this in one SoC?

You have a shared resource in this case, someone more or as trusted as the clients needs to decide how the resource can be shared.
In this case could be Dom0 or Xen or Trusty or op-Tee (if i should make an order).

> 
>> 
>>> 
>>>>> 
>>>>> So I am thinking whether we need to emulating secure world in a XEN
>>>>> VM which is the VM running DomU. Just like what ACRN did to run
>>>>> trusty os.
>>>> Well, it depends on whom you are trusting more. Both XEN and TEE are
>>>> minimal OS implementations with aim at security. I'm speaking about
>>>> generic TEE OS, not about particular OS like OP-TEE or Trusty.
>>>> Problem is that, if TEE is running inside VM, it will be susceptible
>>>> to a hypervisor misbehaviour. You need to understand that Xen and
>>>> privileged domain (dom0, mostly) can access memory of any guest.
>>>> At least, in default configuration. There are means to harden this
>>>> setup. But anyways, Xen can't be stopped from reading TEE's secrets.
>>> 
>>> IIRC, we discussed this approach for OP-TEE in the past. There was other
>> potential pitfalls with it. For instance, you wouldn't be able to directly access
>> any secure device from that guest (it is running in non-secure world).
>>> 
>>> There are also issues in term of latency as you may have the following
>> model:
>>> 
>>> domU -> Xen -> domU TEE -> (Xen -> host TEE -> Xen -> domU TEE) -> Xen ->
>> domU.
>>> 
>>> The bit in () is if you require to call the host TEE.
>>> 
>>> One possibility would be to use Secure-EL2 for your Trusty OS. But I don't
>> know whether your platform supports it.
>>> 
>>> Depending on whether you can modify Trusty OS, alternative would be to
>> make itvirtualization aware as OP-TEE did. The core would need to be
>> resilient and the panic only affect a given client.
>> 
>> I do not have right a clear idea of what is the status of optee and xen but in
>> theory I would see 2 possible ways to handle this:
>> - without optee modification, something in a guest (Dom0 or an other
>> priviledged one) needs to have access to optee and emulate optee access for
>> others.
>> - with optee modifications, optee needs to have a concept of client and Xen
>> would need to passthrough optee requests but being responsible of adding a
>> “client” identifier. Maybe also informing Optee when a new client is
>> created/removed.
>> 
>> The second scenario could then be somehow splitted in the previous one from
>> Julien if some parts would need to be emulated somewhere in some kind of
>> combination of the 2 models.
>> 
>> In any case i would always consider that anything running on optee (or in
>> general in the secure world) is more trusted then Xen.
> 
> Ok, this means optee runs on all cores in secure world, but this would not
> work when we need to support multiple OSes with their own TEE.

I would think you have one TEE running on all cores (or runnable in this case).
So the Tee needs to handle several contexts or someone needs to virtualize it.

Regards
Bertrand

RE: [Tee-dev] TEE with XEN

[Peng Fan]

> Subject: Re: [Tee-dev] TEE with XEN
> 
> 
> 
> > On 19 Jun 2020, at 09:52, Peng Fan <peng.fan@nxp.com> wrote:
> >
> > Hi Bertrand,
> >
> >> Subject: Re: [Tee-dev] TEE with XEN
> >>
> >> Hi,
> >>
> >>> On 18 Jun 2020, at 19:05, Julien Grall <julien@xen.org> wrote:
> >>>
> >>> +Bertrand and Stefano
> >>>
> >>> On 16/06/2020 02:24, Volodymyr Babchuk wrote:
> >>>> Hi Peng,
> >>>> On Mon, 15 Jun 2020 at 05:07, Peng Fan <peng.fan@nxp.com> wrote:
> >>>>>
> >>>>> Hi All,
> >>>>>
> >>>>> While enabling trusty os with xen, I took same approach as OP-TEE,
> >>>>> with OP-TEE running in secure world. But I am also thinking this
> >>>>> might introduce potential issue is that secure world OS
> >>>>> communicate with
> >> DomU.
> >>>>> If there are some misbehavior in secure world OS, it might let XEN
> >>>>> hypervisor not work proper.
> >>>>>
> >>>>> In my setup, trusty os sometimes panic in secure world, xen will
> >>>>> not able to control the panic core anymore.
> >>>
> >>> May I ask in which case Trusty is panicking?
> >>
> >> In any case, optee should protect itself against this and it should
> >> be considered that optee is more priviledged then Xen.
> >> So if optee is crashing we cannot expect that Xen can recover or fix it.
> >>
> >> I would more consider this as a bug that optee needs to be robust against.
> >
> > ok. I am not using OP-TEE, currently I use google trusty OS.
> 
> Sorry i should have been more generic.
> Please read this as “Anything running in secure world”, being optee or trusty.
> 
> >
> > I have two OS, Dom0 linux + DomU android auto.
> >
> > DomU android auto needs trusty OS, Dom0 Linux not need that.
> 
> But i would guess your Dom0 is more “critical” then your DomU.
> In this case you must make sure that any resource given to your DomU cannot
> affect your Dom0.
> For example: if the DomU is starting a very heavy operation in blocked in
> trusty, any interrupt for non-secure could be blocked, thus affecting the ability
> of your Dom0.
> 
> >
> > I not wanna trusty OS for DomU could bring any detect to Dom0 or xen.
> >
> > One more case is if dom0 linux needs OP-TEE, DomU needs google trusty,
> > how we handle this in one SoC?
> 
> You have a shared resource in this case, someone more or as trusted as the
> clients needs to decide how the resource can be shared.
> In this case could be Dom0 or Xen or Trusty or op-Tee (if i should make an
> order).
> 
> >
> >>
> >>>
> >>>>>
> >>>>> So I am thinking whether we need to emulating secure world in a
> >>>>> XEN VM which is the VM running DomU. Just like what ACRN did to
> >>>>> run trusty os.
> >>>> Well, it depends on whom you are trusting more. Both XEN and TEE
> >>>> are minimal OS implementations with aim at security. I'm speaking
> >>>> about generic TEE OS, not about particular OS like OP-TEE or Trusty.
> >>>> Problem is that, if TEE is running inside VM, it will be
> >>>> susceptible to a hypervisor misbehaviour. You need to understand
> >>>> that Xen and privileged domain (dom0, mostly) can access memory of
> any guest.
> >>>> At least, in default configuration. There are means to harden this
> >>>> setup. But anyways, Xen can't be stopped from reading TEE's secrets.
> >>>
> >>> IIRC, we discussed this approach for OP-TEE in the past. There was
> >>> other
> >> potential pitfalls with it. For instance, you wouldn't be able to
> >> directly access any secure device from that guest (it is running in
> non-secure world).
> >>>
> >>> There are also issues in term of latency as you may have the
> >>> following
> >> model:
> >>>
> >>> domU -> Xen -> domU TEE -> (Xen -> host TEE -> Xen -> domU TEE) ->
> >>> Xen ->
> >> domU.
> >>>
> >>> The bit in () is if you require to call the host TEE.
> >>>
> >>> One possibility would be to use Secure-EL2 for your Trusty OS. But I
> >>> don't
> >> know whether your platform supports it.
> >>>
> >>> Depending on whether you can modify Trusty OS, alternative would be
> >>> to
> >> make itvirtualization aware as OP-TEE did. The core would need to be
> >> resilient and the panic only affect a given client.
> >>
> >> I do not have right a clear idea of what is the status of optee and
> >> xen but in theory I would see 2 possible ways to handle this:
> >> - without optee modification, something in a guest (Dom0 or an other
> >> priviledged one) needs to have access to optee and emulate optee
> >> access for others.
> >> - with optee modifications, optee needs to have a concept of client
> >> and Xen would need to passthrough optee requests but being
> >> responsible of adding a “client” identifier. Maybe also informing
> >> Optee when a new client is created/removed.
> >>
> >> The second scenario could then be somehow splitted in the previous
> >> one from Julien if some parts would need to be emulated somewhere in
> >> some kind of combination of the 2 models.
> >>
> >> In any case i would always consider that anything running on optee
> >> (or in general in the secure world) is more trusted then Xen.
> >
> > Ok, this means optee runs on all cores in secure world, but this would
> > not work when we need to support multiple OSes with their own TEE.
> 
> I would think you have one TEE running on all cores (or runnable in this case).
> So the Tee needs to handle several contexts or someone needs to virtualize it.

This back to my original question, should I virtualize TEE in a XEN dedicated VM?
or I need to emulate secure world to let one VM could have secure and non-secure
world?

Thanks,
Peng.

> 
> Regards
> Bertrand

Re: [Tee-dev] TEE with XEN

[Volodymyr Babchuk]

Hi Peng,

On Fri, 19 Jun 2020 at 12:06, Peng Fan <peng.fan@nxp.com> wrote:
>
> > Subject: Re: [Tee-dev] TEE with XEN
> >
> >
> >
> > > On 19 Jun 2020, at 09:52, Peng Fan <peng.fan@nxp.com> wrote:
> > >
> > > Hi Bertrand,
> > >
> > >> Subject: Re: [Tee-dev] TEE with XEN
> > >>
> > >> Hi,
> > >>
> > >>> On 18 Jun 2020, at 19:05, Julien Grall <julien@xen.org> wrote:
> > >>>
> > >>> +Bertrand and Stefano
> > >>>
> > >>> On 16/06/2020 02:24, Volodymyr Babchuk wrote:
> > >>>> Hi Peng,
> > >>>> On Mon, 15 Jun 2020 at 05:07, Peng Fan <peng.fan@nxp.com> wrote:
> > >>>>>
> > >>>>> Hi All,
> > >>>>>
> > >>>>> While enabling trusty os with xen, I took same approach as OP-TEE,
> > >>>>> with OP-TEE running in secure world. But I am also thinking this
> > >>>>> might introduce potential issue is that secure world OS
> > >>>>> communicate with
> > >> DomU.
> > >>>>> If there are some misbehavior in secure world OS, it might let XEN
> > >>>>> hypervisor not work proper.
> > >>>>>
> > >>>>> In my setup, trusty os sometimes panic in secure world, xen will
> > >>>>> not able to control the panic core anymore.
> > >>>
> > >>> May I ask in which case Trusty is panicking?
> > >>
> > >> In any case, optee should protect itself against this and it should
> > >> be considered that optee is more priviledged then Xen.
> > >> So if optee is crashing we cannot expect that Xen can recover or fix it.
> > >>
> > >> I would more consider this as a bug that optee needs to be robust against.
> > >
> > > ok. I am not using OP-TEE, currently I use google trusty OS.
> >
> > Sorry i should have been more generic.
> > Please read this as “Anything running in secure world”, being optee or trusty.
> >
> > >
> > > I have two OS, Dom0 linux + DomU android auto.
> > >
> > > DomU android auto needs trusty OS, Dom0 Linux not need that.
> >
> > But i would guess your Dom0 is more “critical” then your DomU.
> > In this case you must make sure that any resource given to your DomU cannot
> > affect your Dom0.
> > For example: if the DomU is starting a very heavy operation in blocked in
> > trusty, any interrupt for non-secure could be blocked, thus affecting the ability
> > of your Dom0.
> >
> > >
> > > I not wanna trusty OS for DomU could bring any detect to Dom0 or xen.
> > >
> > > One more case is if dom0 linux needs OP-TEE, DomU needs google trusty,
> > > how we handle this in one SoC?
> >
> > You have a shared resource in this case, someone more or as trusted as the
> > clients needs to decide how the resource can be shared.
> > In this case could be Dom0 or Xen or Trusty or op-Tee (if i should make an
> > order).
> >
> > >
> > >>
> > >>>
> > >>>>>
> > >>>>> So I am thinking whether we need to emulating secure world in a
> > >>>>> XEN VM which is the VM running DomU. Just like what ACRN did to
> > >>>>> run trusty os.
> > >>>> Well, it depends on whom you are trusting more. Both XEN and TEE
> > >>>> are minimal OS implementations with aim at security. I'm speaking
> > >>>> about generic TEE OS, not about particular OS like OP-TEE or Trusty.
> > >>>> Problem is that, if TEE is running inside VM, it will be
> > >>>> susceptible to a hypervisor misbehaviour. You need to understand
> > >>>> that Xen and privileged domain (dom0, mostly) can access memory of
> > any guest.
> > >>>> At least, in default configuration. There are means to harden this
> > >>>> setup. But anyways, Xen can't be stopped from reading TEE's secrets.
> > >>>
> > >>> IIRC, we discussed this approach for OP-TEE in the past. There was
> > >>> other
> > >> potential pitfalls with it. For instance, you wouldn't be able to
> > >> directly access any secure device from that guest (it is running in
> > non-secure world).
> > >>>
> > >>> There are also issues in term of latency as you may have the
> > >>> following
> > >> model:
> > >>>
> > >>> domU -> Xen -> domU TEE -> (Xen -> host TEE -> Xen -> domU TEE) ->
> > >>> Xen ->
> > >> domU.
> > >>>
> > >>> The bit in () is if you require to call the host TEE.
> > >>>
> > >>> One possibility would be to use Secure-EL2 for your Trusty OS. But I
> > >>> don't
> > >> know whether your platform supports it.
> > >>>
> > >>> Depending on whether you can modify Trusty OS, alternative would be
> > >>> to
> > >> make itvirtualization aware as OP-TEE did. The core would need to be
> > >> resilient and the panic only affect a given client.
> > >>
> > >> I do not have right a clear idea of what is the status of optee and
> > >> xen but in theory I would see 2 possible ways to handle this:
> > >> - without optee modification, something in a guest (Dom0 or an other
> > >> priviledged one) needs to have access to optee and emulate optee
> > >> access for others.
> > >> - with optee modifications, optee needs to have a concept of client
> > >> and Xen would need to passthrough optee requests but being
> > >> responsible of adding a “client” identifier. Maybe also informing
> > >> Optee when a new client is created/removed.
> > >>
> > >> The second scenario could then be somehow splitted in the previous
> > >> one from Julien if some parts would need to be emulated somewhere in
> > >> some kind of combination of the 2 models.
> > >>
> > >> In any case i would always consider that anything running on optee
> > >> (or in general in the secure world) is more trusted then Xen.
> > >
> > > Ok, this means optee runs on all cores in secure world, but this would
> > > not work when we need to support multiple OSes with their own TEE.
> >
> > I would think you have one TEE running on all cores (or runnable in this case).
> > So the Tee needs to handle several contexts or someone needs to virtualize it.
>
> This back to my original question, should I virtualize TEE in a XEN dedicated VM?
> or I need to emulate secure world to let one VM could have secure and non-secure
> world?
>

Well, I think that the best approach is that we did in the OP-TEE: make Trusty
virtualization-aware, so it can handle multiple VMs.

Things are more funny if you want to use multiple different TEEs (like
OP-TEE and Trusty)
at the same time. If this is your case, then the best approach is to implement
something like para-virtualization in the Secure World. But this would require
quite big efforts, of course.

-- 
WBR Volodymyr Babchuk aka lorc [+380976646013]
mailto: vlad.babchuk@gmail.com

RE: [Tee-dev] TEE with XEN

[Peng Fan]

> Subject: Re: [Tee-dev] TEE with XEN
> 
> Hi Peng,
> 
> On Fri, 19 Jun 2020 at 12:06, Peng Fan <peng.fan@nxp.com> wrote:
> >
> > > Subject: Re: [Tee-dev] TEE with XEN
> > >
> > >
> > >
> > > > On 19 Jun 2020, at 09:52, Peng Fan <peng.fan@nxp.com> wrote:
> > > >
> > > > Hi Bertrand,
> > > >
> > > >> Subject: Re: [Tee-dev] TEE with XEN
> > > >>
> > > >> Hi,
> > > >>
> > > >>> On 18 Jun 2020, at 19:05, Julien Grall <julien@xen.org> wrote:
> > > >>>
> > > >>> +Bertrand and Stefano
> > > >>>
> > > >>> On 16/06/2020 02:24, Volodymyr Babchuk wrote:
> > > >>>> Hi Peng,
> > > >>>> On Mon, 15 Jun 2020 at 05:07, Peng Fan <peng.fan@nxp.com>
> wrote:
> > > >>>>>
> > > >>>>> Hi All,
> > > >>>>>
> > > >>>>> While enabling trusty os with xen, I took same approach as
> > > >>>>> OP-TEE, with OP-TEE running in secure world. But I am also
> > > >>>>> thinking this might introduce potential issue is that secure
> > > >>>>> world OS communicate with
> > > >> DomU.
> > > >>>>> If there are some misbehavior in secure world OS, it might let
> > > >>>>> XEN hypervisor not work proper.
> > > >>>>>
> > > >>>>> In my setup, trusty os sometimes panic in secure world, xen
> > > >>>>> will not able to control the panic core anymore.
> > > >>>
> > > >>> May I ask in which case Trusty is panicking?
> > > >>
> > > >> In any case, optee should protect itself against this and it
> > > >> should be considered that optee is more priviledged then Xen.
> > > >> So if optee is crashing we cannot expect that Xen can recover or fix it.
> > > >>
> > > >> I would more consider this as a bug that optee needs to be robust
> against.
> > > >
> > > > ok. I am not using OP-TEE, currently I use google trusty OS.
> > >
> > > Sorry i should have been more generic.
> > > Please read this as “Anything running in secure world”, being optee or
> trusty.
> > >
> > > >
> > > > I have two OS, Dom0 linux + DomU android auto.
> > > >
> > > > DomU android auto needs trusty OS, Dom0 Linux not need that.
> > >
> > > But i would guess your Dom0 is more “critical” then your DomU.
> > > In this case you must make sure that any resource given to your DomU
> > > cannot affect your Dom0.
> > > For example: if the DomU is starting a very heavy operation in
> > > blocked in trusty, any interrupt for non-secure could be blocked,
> > > thus affecting the ability of your Dom0.
> > >
> > > >
> > > > I not wanna trusty OS for DomU could bring any detect to Dom0 or xen.
> > > >
> > > > One more case is if dom0 linux needs OP-TEE, DomU needs google
> > > > trusty, how we handle this in one SoC?
> > >
> > > You have a shared resource in this case, someone more or as trusted
> > > as the clients needs to decide how the resource can be shared.
> > > In this case could be Dom0 or Xen or Trusty or op-Tee (if i should
> > > make an order).
> > >
> > > >
> > > >>
> > > >>>
> > > >>>>>
> > > >>>>> So I am thinking whether we need to emulating secure world in
> > > >>>>> a XEN VM which is the VM running DomU. Just like what ACRN did
> > > >>>>> to run trusty os.
> > > >>>> Well, it depends on whom you are trusting more. Both XEN and
> > > >>>> TEE are minimal OS implementations with aim at security. I'm
> > > >>>> speaking about generic TEE OS, not about particular OS like OP-TEE
> or Trusty.
> > > >>>> Problem is that, if TEE is running inside VM, it will be
> > > >>>> susceptible to a hypervisor misbehaviour. You need to
> > > >>>> understand that Xen and privileged domain (dom0, mostly) can
> > > >>>> access memory of
> > > any guest.
> > > >>>> At least, in default configuration. There are means to harden
> > > >>>> this setup. But anyways, Xen can't be stopped from reading TEE's
> secrets.
> > > >>>
> > > >>> IIRC, we discussed this approach for OP-TEE in the past. There
> > > >>> was other
> > > >> potential pitfalls with it. For instance, you wouldn't be able to
> > > >> directly access any secure device from that guest (it is running
> > > >> in
> > > non-secure world).
> > > >>>
> > > >>> There are also issues in term of latency as you may have the
> > > >>> following
> > > >> model:
> > > >>>
> > > >>> domU -> Xen -> domU TEE -> (Xen -> host TEE -> Xen -> domU TEE)
> > > >>> -> Xen ->
> > > >> domU.
> > > >>>
> > > >>> The bit in () is if you require to call the host TEE.
> > > >>>
> > > >>> One possibility would be to use Secure-EL2 for your Trusty OS.
> > > >>> But I don't
> > > >> know whether your platform supports it.
> > > >>>
> > > >>> Depending on whether you can modify Trusty OS, alternative would
> > > >>> be to
> > > >> make itvirtualization aware as OP-TEE did. The core would need to
> > > >> be resilient and the panic only affect a given client.
> > > >>
> > > >> I do not have right a clear idea of what is the status of optee
> > > >> and xen but in theory I would see 2 possible ways to handle this:
> > > >> - without optee modification, something in a guest (Dom0 or an
> > > >> other priviledged one) needs to have access to optee and emulate
> > > >> optee access for others.
> > > >> - with optee modifications, optee needs to have a concept of
> > > >> client and Xen would need to passthrough optee requests but being
> > > >> responsible of adding a “client” identifier. Maybe also informing
> > > >> Optee when a new client is created/removed.
> > > >>
> > > >> The second scenario could then be somehow splitted in the
> > > >> previous one from Julien if some parts would need to be emulated
> > > >> somewhere in some kind of combination of the 2 models.
> > > >>
> > > >> In any case i would always consider that anything running on
> > > >> optee (or in general in the secure world) is more trusted then Xen.
> > > >
> > > > Ok, this means optee runs on all cores in secure world, but this
> > > > would not work when we need to support multiple OSes with their own
> TEE.
> > >
> > > I would think you have one TEE running on all cores (or runnable in this
> case).
> > > So the Tee needs to handle several contexts or someone needs to
> virtualize it.
> >
> > This back to my original question, should I virtualize TEE in a XEN dedicated
> VM?
> > or I need to emulate secure world to let one VM could have secure and
> > non-secure world?
> >
> 
> Well, I think that the best approach is that we did in the OP-TEE: make Trusty
> virtualization-aware, so it can handle multiple VMs.

Trusty virtualization-aware, you mean android Linux trusty driver communicate
with OP-TEE or else?

> 
> Things are more funny if you want to use multiple different TEEs (like OP-TEE
> and Trusty) at the same time. If this is your case, then the best approach is to
> implement something like para-virtualization in the Secure World. But this
> would require quite big efforts, of course.

This is just a brainstorm idea, I not have the case right now. I just wanna
avoid trusty os that only work with android auto vm not break dom0 and
xen. 

Thanks,
Peng.

> 
> --
> WBR Volodymyr Babchuk aka lorc [+380976646013]
> mailto: vlad.babchuk@gmail.com

Re: [Tee-dev] TEE with XEN

[Bertrand Marquis]

> On 19 Jun 2020, at 10:12, Volodymyr Babchuk <vlad.babchuk@gmail.com> wrote:
> 
> Hi Peng,
> 
> On Fri, 19 Jun 2020 at 12:06, Peng Fan <peng.fan@nxp.com> wrote:
>> 
>>> Subject: Re: [Tee-dev] TEE with XEN
>>> 
>>> 
>>> 
>>>> On 19 Jun 2020, at 09:52, Peng Fan <peng.fan@nxp.com> wrote:
>>>> 
>>>> Hi Bertrand,
>>>> 
>>>>> Subject: Re: [Tee-dev] TEE with XEN
>>>>> 
>>>>> Hi,
>>>>> 
>>>>>> On 18 Jun 2020, at 19:05, Julien Grall <julien@xen.org> wrote:
>>>>>> 
>>>>>> +Bertrand and Stefano
>>>>>> 
>>>>>> On 16/06/2020 02:24, Volodymyr Babchuk wrote:
>>>>>>> Hi Peng,
>>>>>>> On Mon, 15 Jun 2020 at 05:07, Peng Fan <peng.fan@nxp.com> wrote:
>>>>>>>> 
>>>>>>>> Hi All,
>>>>>>>> 
>>>>>>>> While enabling trusty os with xen, I took same approach as OP-TEE,
>>>>>>>> with OP-TEE running in secure world. But I am also thinking this
>>>>>>>> might introduce potential issue is that secure world OS
>>>>>>>> communicate with
>>>>> DomU.
>>>>>>>> If there are some misbehavior in secure world OS, it might let XEN
>>>>>>>> hypervisor not work proper.
>>>>>>>> 
>>>>>>>> In my setup, trusty os sometimes panic in secure world, xen will
>>>>>>>> not able to control the panic core anymore.
>>>>>> 
>>>>>> May I ask in which case Trusty is panicking?
>>>>> 
>>>>> In any case, optee should protect itself against this and it should
>>>>> be considered that optee is more priviledged then Xen.
>>>>> So if optee is crashing we cannot expect that Xen can recover or fix it.
>>>>> 
>>>>> I would more consider this as a bug that optee needs to be robust against.
>>>> 
>>>> ok. I am not using OP-TEE, currently I use google trusty OS.
>>> 
>>> Sorry i should have been more generic.
>>> Please read this as “Anything running in secure world”, being optee or trusty.
>>> 
>>>> 
>>>> I have two OS, Dom0 linux + DomU android auto.
>>>> 
>>>> DomU android auto needs trusty OS, Dom0 Linux not need that.
>>> 
>>> But i would guess your Dom0 is more “critical” then your DomU.
>>> In this case you must make sure that any resource given to your DomU cannot
>>> affect your Dom0.
>>> For example: if the DomU is starting a very heavy operation in blocked in
>>> trusty, any interrupt for non-secure could be blocked, thus affecting the ability
>>> of your Dom0.
>>> 
>>>> 
>>>> I not wanna trusty OS for DomU could bring any detect to Dom0 or xen.
>>>> 
>>>> One more case is if dom0 linux needs OP-TEE, DomU needs google trusty,
>>>> how we handle this in one SoC?
>>> 
>>> You have a shared resource in this case, someone more or as trusted as the
>>> clients needs to decide how the resource can be shared.
>>> In this case could be Dom0 or Xen or Trusty or op-Tee (if i should make an
>>> order).
>>> 
>>>> 
>>>>> 
>>>>>> 
>>>>>>>> 
>>>>>>>> So I am thinking whether we need to emulating secure world in a
>>>>>>>> XEN VM which is the VM running DomU. Just like what ACRN did to
>>>>>>>> run trusty os.
>>>>>>> Well, it depends on whom you are trusting more. Both XEN and TEE
>>>>>>> are minimal OS implementations with aim at security. I'm speaking
>>>>>>> about generic TEE OS, not about particular OS like OP-TEE or Trusty.
>>>>>>> Problem is that, if TEE is running inside VM, it will be
>>>>>>> susceptible to a hypervisor misbehaviour. You need to understand
>>>>>>> that Xen and privileged domain (dom0, mostly) can access memory of
>>> any guest.
>>>>>>> At least, in default configuration. There are means to harden this
>>>>>>> setup. But anyways, Xen can't be stopped from reading TEE's secrets.
>>>>>> 
>>>>>> IIRC, we discussed this approach for OP-TEE in the past. There was
>>>>>> other
>>>>> potential pitfalls with it. For instance, you wouldn't be able to
>>>>> directly access any secure device from that guest (it is running in
>>> non-secure world).
>>>>>> 
>>>>>> There are also issues in term of latency as you may have the
>>>>>> following
>>>>> model:
>>>>>> 
>>>>>> domU -> Xen -> domU TEE -> (Xen -> host TEE -> Xen -> domU TEE) ->
>>>>>> Xen ->
>>>>> domU.
>>>>>> 
>>>>>> The bit in () is if you require to call the host TEE.
>>>>>> 
>>>>>> One possibility would be to use Secure-EL2 for your Trusty OS. But I
>>>>>> don't
>>>>> know whether your platform supports it.
>>>>>> 
>>>>>> Depending on whether you can modify Trusty OS, alternative would be
>>>>>> to
>>>>> make itvirtualization aware as OP-TEE did. The core would need to be
>>>>> resilient and the panic only affect a given client.
>>>>> 
>>>>> I do not have right a clear idea of what is the status of optee and
>>>>> xen but in theory I would see 2 possible ways to handle this:
>>>>> - without optee modification, something in a guest (Dom0 or an other
>>>>> priviledged one) needs to have access to optee and emulate optee
>>>>> access for others.
>>>>> - with optee modifications, optee needs to have a concept of client
>>>>> and Xen would need to passthrough optee requests but being
>>>>> responsible of adding a “client” identifier. Maybe also informing
>>>>> Optee when a new client is created/removed.
>>>>> 
>>>>> The second scenario could then be somehow splitted in the previous
>>>>> one from Julien if some parts would need to be emulated somewhere in
>>>>> some kind of combination of the 2 models.
>>>>> 
>>>>> In any case i would always consider that anything running on optee
>>>>> (or in general in the secure world) is more trusted then Xen.
>>>> 
>>>> Ok, this means optee runs on all cores in secure world, but this would
>>>> not work when we need to support multiple OSes with their own TEE.
>>> 
>>> I would think you have one TEE running on all cores (or runnable in this case).
>>> So the Tee needs to handle several contexts or someone needs to virtualize it.
>> 
>> This back to my original question, should I virtualize TEE in a XEN dedicated VM?
>> or I need to emulate secure world to let one VM could have secure and non-secure
>> world?
>> 
> 
> Well, I think that the best approach is that we did in the OP-TEE: make Trusty
> virtualization-aware, so it can handle multiple VMs.
> 
> Things are more funny if you want to use multiple different TEEs (like
> OP-TEE and Trusty)
> at the same time. If this is your case, then the best approach is to implement
> something like para-virtualization in the Secure World. But this would require
> quite big efforts, of course.

I agree this is the best approach to make secure world handle several clients.
But it might be the most complex one though.

Using a VM for it might be easier but you need to check that it achieving the level of trust that you need.
You could have a VM acting as a muxer/checker and passing down requests to the secure world

The definitive answer really depends on the amount of effort, the level of security and your general system needs.

Bertrand

Re: [Tee-dev] TEE with XEN

[Volodymyr Babchuk]

On Fri, 19 Jun 2020 at 12:50, Peng Fan <peng.fan@nxp.com> wrote:
>
> > Subject: Re: [Tee-dev] TEE with XEN
> >
> > Hi Peng,
> >
> > On Fri, 19 Jun 2020 at 12:06, Peng Fan <peng.fan@nxp.com> wrote:
> > >
> > > > Subject: Re: [Tee-dev] TEE with XEN
> > > >
> > > >
> > > >
> > > > > On 19 Jun 2020, at 09:52, Peng Fan <peng.fan@nxp.com> wrote:
> > > > >
> > > > > Hi Bertrand,
> > > > >
> > > > >> Subject: Re: [Tee-dev] TEE with XEN
> > > > >>
> > > > >> Hi,
> > > > >>
> > > > >>> On 18 Jun 2020, at 19:05, Julien Grall <julien@xen.org> wrote:
> > > > >>>
> > > > >>> +Bertrand and Stefano
> > > > >>>
> > > > >>> On 16/06/2020 02:24, Volodymyr Babchuk wrote:
> > > > >>>> Hi Peng,
> > > > >>>> On Mon, 15 Jun 2020 at 05:07, Peng Fan <peng.fan@nxp.com>
> > wrote:
> > > > >>>>>
> > > > >>>>> Hi All,
> > > > >>>>>
> > > > >>>>> While enabling trusty os with xen, I took same approach as
> > > > >>>>> OP-TEE, with OP-TEE running in secure world. But I am also
> > > > >>>>> thinking this might introduce potential issue is that secure
> > > > >>>>> world OS communicate with
> > > > >> DomU.
> > > > >>>>> If there are some misbehavior in secure world OS, it might let
> > > > >>>>> XEN hypervisor not work proper.
> > > > >>>>>
> > > > >>>>> In my setup, trusty os sometimes panic in secure world, xen
> > > > >>>>> will not able to control the panic core anymore.
> > > > >>>
> > > > >>> May I ask in which case Trusty is panicking?
> > > > >>
> > > > >> In any case, optee should protect itself against this and it
> > > > >> should be considered that optee is more priviledged then Xen.
> > > > >> So if optee is crashing we cannot expect that Xen can recover or fix it.
> > > > >>
> > > > >> I would more consider this as a bug that optee needs to be robust
> > against.
> > > > >
> > > > > ok. I am not using OP-TEE, currently I use google trusty OS.
> > > >
> > > > Sorry i should have been more generic.
> > > > Please read this as “Anything running in secure world”, being optee or
> > trusty.
> > > >
> > > > >
> > > > > I have two OS, Dom0 linux + DomU android auto.
> > > > >
> > > > > DomU android auto needs trusty OS, Dom0 Linux not need that.
> > > >
> > > > But i would guess your Dom0 is more “critical” then your DomU.
> > > > In this case you must make sure that any resource given to your DomU
> > > > cannot affect your Dom0.
> > > > For example: if the DomU is starting a very heavy operation in
> > > > blocked in trusty, any interrupt for non-secure could be blocked,
> > > > thus affecting the ability of your Dom0.
> > > >
> > > > >
> > > > > I not wanna trusty OS for DomU could bring any detect to Dom0 or xen.
> > > > >
> > > > > One more case is if dom0 linux needs OP-TEE, DomU needs google
> > > > > trusty, how we handle this in one SoC?
> > > >
> > > > You have a shared resource in this case, someone more or as trusted
> > > > as the clients needs to decide how the resource can be shared.
> > > > In this case could be Dom0 or Xen or Trusty or op-Tee (if i should
> > > > make an order).
> > > >
> > > > >
> > > > >>
> > > > >>>
> > > > >>>>>
> > > > >>>>> So I am thinking whether we need to emulating secure world in
> > > > >>>>> a XEN VM which is the VM running DomU. Just like what ACRN did
> > > > >>>>> to run trusty os.
> > > > >>>> Well, it depends on whom you are trusting more. Both XEN and
> > > > >>>> TEE are minimal OS implementations with aim at security. I'm
> > > > >>>> speaking about generic TEE OS, not about particular OS like OP-TEE
> > or Trusty.
> > > > >>>> Problem is that, if TEE is running inside VM, it will be
> > > > >>>> susceptible to a hypervisor misbehaviour. You need to
> > > > >>>> understand that Xen and privileged domain (dom0, mostly) can
> > > > >>>> access memory of
> > > > any guest.
> > > > >>>> At least, in default configuration. There are means to harden
> > > > >>>> this setup. But anyways, Xen can't be stopped from reading TEE's
> > secrets.
> > > > >>>
> > > > >>> IIRC, we discussed this approach for OP-TEE in the past. There
> > > > >>> was other
> > > > >> potential pitfalls with it. For instance, you wouldn't be able to
> > > > >> directly access any secure device from that guest (it is running
> > > > >> in
> > > > non-secure world).
> > > > >>>
> > > > >>> There are also issues in term of latency as you may have the
> > > > >>> following
> > > > >> model:
> > > > >>>
> > > > >>> domU -> Xen -> domU TEE -> (Xen -> host TEE -> Xen -> domU TEE)
> > > > >>> -> Xen ->
> > > > >> domU.
> > > > >>>
> > > > >>> The bit in () is if you require to call the host TEE.
> > > > >>>
> > > > >>> One possibility would be to use Secure-EL2 for your Trusty OS.
> > > > >>> But I don't
> > > > >> know whether your platform supports it.
> > > > >>>
> > > > >>> Depending on whether you can modify Trusty OS, alternative would
> > > > >>> be to
> > > > >> make itvirtualization aware as OP-TEE did. The core would need to
> > > > >> be resilient and the panic only affect a given client.
> > > > >>
> > > > >> I do not have right a clear idea of what is the status of optee
> > > > >> and xen but in theory I would see 2 possible ways to handle this:
> > > > >> - without optee modification, something in a guest (Dom0 or an
> > > > >> other priviledged one) needs to have access to optee and emulate
> > > > >> optee access for others.
> > > > >> - with optee modifications, optee needs to have a concept of
> > > > >> client and Xen would need to passthrough optee requests but being
> > > > >> responsible of adding a “client” identifier. Maybe also informing
> > > > >> Optee when a new client is created/removed.
> > > > >>
> > > > >> The second scenario could then be somehow splitted in the
> > > > >> previous one from Julien if some parts would need to be emulated
> > > > >> somewhere in some kind of combination of the 2 models.
> > > > >>
> > > > >> In any case i would always consider that anything running on
> > > > >> optee (or in general in the secure world) is more trusted then Xen.
> > > > >
> > > > > Ok, this means optee runs on all cores in secure world, but this
> > > > > would not work when we need to support multiple OSes with their own
> > TEE.
> > > >
> > > > I would think you have one TEE running on all cores (or runnable in this
> > case).
> > > > So the Tee needs to handle several contexts or someone needs to
> > virtualize it.
> > >
> > > This back to my original question, should I virtualize TEE in a XEN dedicated
> > VM?
> > > or I need to emulate secure world to let one VM could have secure and
> > > non-secure world?
> > >
> >
> > Well, I think that the best approach is that we did in the OP-TEE: make Trusty
> > virtualization-aware, so it can handle multiple VMs.
>
> Trusty virtualization-aware, you mean android Linux trusty driver communicate
> with OP-TEE or else?

Okay, OP-TEE is implemented right as Bertran has suggested: OP-TEE can work
with multiple VMs at the same time. It has special calls to create and
destroy VM
contextes.

So, when a new VM/guest is spawned, Xen says "Hey, OP-TEE, there is a new
VM with ID NNN". OP-TEE then initializes internal state for the new VM.
At any moment Xen can say "Oops, that VM with ID NNN is dead". OP-TEE will
immediately destroy the internal state for that VM.

Also, Xen needs to perform certain actions every time VM calls OP-TEE: translate
addresses, tell OP-TEE id of that VM, lock guest pages...
And no changes to the OP-TEE client in Linux are needed. It thinks
that it communicates
directly with the OP-TEE. OP-TEE is running in the Secure World as usual.

The same can be done for any other TEE. Trusty, for example. Downside is that
you can't run two different TEEs in the Secure world.

Actually, ARM added virtualization support in Secure mode, but AFAIK, i.MX8 does
not support it.

To sum it up:
1. If you want to run only Trusty on your system, and you want to run
it in Secure World,
 you need to make Trusty virtualization-aware, and write mediator in
Xen. This is what
I did for OP-TEE.

2. If you want to run Trusty AND OP-TEE in the Secure World, then you
need to do p1.
and then implement some paravirtualization support in Secure Monitor,
Trusty and OP-TEE

3. If your use case does not require high security, you can try to run
Trusty as a VM.
But, I am almost certain that this will not pass Google's
requirements. So no Google Pay,
no E-SIM, no DRM for this setup. But, you should clarify this with
Google. I'm not a Google
representative :)