* [PATCH v4 01/10] xen/arm: add generic TEE mediator framework
2019-03-07 21:04 [PATCH v4 00/10] TEE mediator (and OP-TEE) support in XEN Volodymyr Babchuk
@ 2019-03-07 21:04 ` Volodymyr Babchuk
2019-03-15 15:03 ` Julien Grall
2019-03-07 21:04 ` [PATCH v4 02/10] xen/arm: optee: add OP-TEE header files Volodymyr Babchuk
` (8 subsequent siblings)
9 siblings, 1 reply; 47+ messages in thread
From: Volodymyr Babchuk @ 2019-03-07 21:04 UTC (permalink / raw)
To: xen-devel, tee-dev
Cc: Stefano Stabellini, Wei Liu, Konrad Rzeszutek Wilk,
George Dunlap, Andrew Cooper, Ian Jackson, Tim Deegan,
Julien Grall, Jan Beulich, Volodymyr Babchuk
From: Volodymyr Babchuk <volodymyr_babchuk@epam.com>
This patch adds basic framework for TEE mediators. Guests can't talk
to TEE directly, we need some entity that will intercept request
and decide what to do with them. "TEE mediator" is a such entity.
This is how it works: user can build XEN with multiple TEE mediators
(see the next patches, where OP-TEE mediator is introduced).
TEE mediator register self with REGISTER_TEE_MEDIATOR() macro in the
same way, as device drivers use DT_DEVICE_START()/DT_DEVICE_END()
macros.
In run-time, during initialization, framework calls probe() function
for each available mediator driver to find which TEE is installed
on the platform. Then generic vSMC handler will call selected mediator
when it intercept SMC/HVC that belongs to TEE OS or TEE application.
Currently TEE mediator can be enabled only for Dom0 using
"dom0_tee_enabled" boot argument.
Signed-off-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com>
---
Changes from v3:
- tee_enable() renamed to tee_domain_init()
- Added tee_relinquish_resources() function along with
changes to domain_relinquish_resources()
- Added command-line parameter dom0_tee_enabled, which controls
if tee is enabled for Dom0. It is disabled by default
- Instead of boolean tee state (enabled/disabled) I introduced
enumeration with two values: none or native. It is possible
to add other types of tee in the future
Changes from v2:
- Removed empty tee/Kconfig file
Changes from v1:
- Removed tee_remove() function
- CONFIG_TEE depends on EXPERT
- tee_domain_created() converted to tee_enable()
- tee_init() is called using initcall() mechanism
- tee_handle_smc() renamed to tee_handle_call()
Changes from "RFC" version:
- renamed CONFIG_ARM_TEE to CONFIG_TEE
- changed discovery mechanism: instead of UUID mathing, TEE-specific
probing is used
---
MAINTAINERS | 6 ++
xen/arch/arm/Kconfig | 7 +++
xen/arch/arm/Makefile | 1 +
xen/arch/arm/domain.c | 14 +++++
xen/arch/arm/setup.c | 8 +++
xen/arch/arm/tee/Makefile | 1 +
xen/arch/arm/tee/tee.c | 79 +++++++++++++++++++++++++
xen/arch/arm/vsmc.c | 5 ++
xen/arch/arm/xen.lds.S | 7 +++
xen/include/asm-arm/domain.h | 1 +
xen/include/asm-arm/tee/tee.h | 106 ++++++++++++++++++++++++++++++++++
xen/include/public/arch-arm.h | 4 ++
12 files changed, 239 insertions(+)
create mode 100644 xen/arch/arm/tee/Makefile
create mode 100644 xen/arch/arm/tee/tee.c
create mode 100644 xen/include/asm-arm/tee/tee.h
diff --git a/MAINTAINERS b/MAINTAINERS
index a0cda4f7a1..54436b98f5 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -383,6 +383,12 @@ F: config/Stubdom.mk.in
F: m4/stubdom.m4
F: stubdom/
+TEE MEDIATORS
+M: Volodymyr Babchuk <volodymyr_babchuk@epam.com>
+S: Supported
+F: xen/arch/arm/tee/
+F: xen/include/asm-arm/tee
+
TOOLSTACK
M: Ian Jackson <ian.jackson@eu.citrix.com>
M: Wei Liu <wei.liu2@citrix.com>
diff --git a/xen/arch/arm/Kconfig b/xen/arch/arm/Kconfig
index 581de67b6b..e527b2f885 100644
--- a/xen/arch/arm/Kconfig
+++ b/xen/arch/arm/Kconfig
@@ -105,6 +105,13 @@ config HARDEN_BRANCH_PREDICTOR
If unsure, say Y.
+config TEE
+ bool "Enable TEE mediators support" if EXPERT = "y"
+ default n
+ help
+ This option enables generic TEE mediators support. It allows guests
+ to access real TEE via one of TEE mediators implemented in XEN.
+
endmenu
menu "ARM errata workaround via the alternative framework"
diff --git a/xen/arch/arm/Makefile b/xen/arch/arm/Makefile
index cb902cb6fe..5c2aa34557 100644
--- a/xen/arch/arm/Makefile
+++ b/xen/arch/arm/Makefile
@@ -5,6 +5,7 @@ subdir-$(CONFIG_ACPI) += acpi
ifneq ($(CONFIG_NO_PLAT),y)
subdir-y += platforms
endif
+subdir-$(CONFIG_TEE) += tee
obj-$(CONFIG_HAS_ALTERNATIVE) += alternative.o
obj-y += bootfdt.init.o
diff --git a/xen/arch/arm/domain.c b/xen/arch/arm/domain.c
index 6dc633ed50..d1e2a3979d 100644
--- a/xen/arch/arm/domain.c
+++ b/xen/arch/arm/domain.c
@@ -32,6 +32,7 @@
#include <asm/platform.h>
#include <asm/procinfo.h>
#include <asm/regs.h>
+#include <asm/tee/tee.h>
#include <asm/vfp.h>
#include <asm/vgic.h>
#include <asm/vtimer.h>
@@ -705,6 +706,10 @@ int arch_domain_create(struct domain *d,
if ( (rc = domain_vtimer_init(d, &config->arch)) != 0 )
goto fail;
+ if ( config->arch.tee_type == XEN_DOMCTL_CONFIG_TEE_NATIVE )
+ if ( (rc = tee_domain_init(d)) != 0 )
+ goto fail;
+
update_domain_wallclock_time(d);
/*
@@ -743,6 +748,7 @@ void arch_domain_destroy(struct domain *d)
* iommu_domain_destroy() before p2m_teardown().
*/
iommu_domain_destroy(d);
+ tee_domain_destroy(d);
p2m_teardown(d);
domain_vgic_free(d);
domain_vuart_free(d);
@@ -949,6 +955,14 @@ int domain_relinquish_resources(struct domain *d)
*/
domain_vpl011_deinit(d);
+ d->arch.relmem = RELMEM_tee;
+ /* Fallthrough */
+
+ case RELMEM_tee:
+ ret = tee_relinquish_resources(d);
+ if (ret )
+ return ret;
+
d->arch.relmem = RELMEM_xen;
/* Fallthrough */
diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c
index 444857a967..7602dd990c 100644
--- a/xen/arch/arm/setup.c
+++ b/xen/arch/arm/setup.c
@@ -56,6 +56,9 @@ struct bootinfo __initdata bootinfo;
struct cpuinfo_arm __read_mostly boot_cpu_data;
+static bool __initdata opt_dom0_tee_enabled;
+boolean_param("dom0_tee_enabled", opt_dom0_tee_enabled);
+
#ifdef CONFIG_ACPI
bool __read_mostly acpi_disabled;
#endif
@@ -889,6 +892,11 @@ void __init start_xen(unsigned long boot_phys_offset,
/* The vGIC for DOM0 is exactly emulating the hardware GIC */
dom0_cfg.arch.gic_version = XEN_DOMCTL_CONFIG_GIC_NATIVE;
dom0_cfg.arch.nr_spis = gic_number_lines() - 32;
+ if ( opt_dom0_tee_enabled )
+ dom0_cfg.arch.tee_type = XEN_DOMCTL_CONFIG_TEE_NATIVE;
+ else
+ dom0_cfg.arch.tee_type = XEN_DOMCTL_CONFIG_TEE_NONE;
+
dom0_cfg.max_vcpus = dom0_max_vcpus();
dom0 = domain_create(0, &dom0_cfg, true);
diff --git a/xen/arch/arm/tee/Makefile b/xen/arch/arm/tee/Makefile
new file mode 100644
index 0000000000..c54d4796ff
--- /dev/null
+++ b/xen/arch/arm/tee/Makefile
@@ -0,0 +1 @@
+obj-y += tee.o
diff --git a/xen/arch/arm/tee/tee.c b/xen/arch/arm/tee/tee.c
new file mode 100644
index 0000000000..70432306b9
--- /dev/null
+++ b/xen/arch/arm/tee/tee.c
@@ -0,0 +1,79 @@
+/*
+ * xen/arch/arm/tee/tee.c
+ *
+ * Generic part of TEE mediator subsystem
+ *
+ * Volodymyr Babchuk <volodymyr_babchuk@epam.com>
+ * Copyright (c) 2018 EPAM Systems.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#include <xen/errno.h>
+#include <xen/init.h>
+#include <xen/types.h>
+
+#include <asm/tee/tee.h>
+
+extern const struct tee_mediator_desc _steemediator[], _eteemediator[];
+static const struct tee_mediator_ops *mediator_ops;
+
+bool tee_handle_call(struct cpu_user_regs *regs)
+{
+ if ( !mediator_ops )
+ return false;
+
+ return mediator_ops->handle_call(regs);
+}
+
+int tee_domain_init(struct domain *d)
+{
+ if ( !mediator_ops )
+ return -ENODEV;
+
+ return mediator_ops->domain_init(d);
+}
+
+int tee_relinquish_resources(struct domain *d)
+{
+ if ( !mediator_ops )
+ return 0;
+
+ return mediator_ops->relinquish_resources(d);
+}
+
+void tee_domain_destroy(struct domain *d)
+{
+ if ( mediator_ops )
+ mediator_ops->domain_destroy(d);
+}
+
+static int __init tee_init(void)
+{
+ const struct tee_mediator_desc *desc;
+
+ for ( desc = _steemediator; desc != _eteemediator; desc++ )
+ {
+ if ( desc->ops->probe() )
+ {
+ printk(XENLOG_INFO "Using TEE mediator for %s\n", desc->name);
+ mediator_ops = desc->ops;
+ return 0;
+ }
+ }
+
+ return 0;
+}
+
+__initcall(tee_init);
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/arch/arm/vsmc.c b/xen/arch/arm/vsmc.c
index c72b9a04ff..f8e350311d 100644
--- a/xen/arch/arm/vsmc.c
+++ b/xen/arch/arm/vsmc.c
@@ -23,6 +23,7 @@
#include <asm/monitor.h>
#include <asm/regs.h>
#include <asm/smccc.h>
+#include <asm/tee/tee.h>
#include <asm/traps.h>
#include <asm/vpsci.h>
#include <asm/platform.h>
@@ -276,6 +277,10 @@ static bool vsmccc_handle_call(struct cpu_user_regs *regs)
case ARM_SMCCC_OWNER_SIP:
handled = platform_smc(regs);
break;
+ case ARM_SMCCC_OWNER_TRUSTED_APP ... ARM_SMCCC_OWNER_TRUSTED_APP_END:
+ case ARM_SMCCC_OWNER_TRUSTED_OS ... ARM_SMCCC_OWNER_TRUSTED_OS_END:
+ handled = tee_handle_call(regs);
+ break;
}
}
diff --git a/xen/arch/arm/xen.lds.S b/xen/arch/arm/xen.lds.S
index 1e72906477..e664c4441a 100644
--- a/xen/arch/arm/xen.lds.S
+++ b/xen/arch/arm/xen.lds.S
@@ -137,6 +137,13 @@ SECTIONS
_aedevice = .;
} :text
+ . = ALIGN(8);
+ .teemediator.info : {
+ _steemediator = .;
+ *(.teemediator.info)
+ _eteemediator = .;
+ } :text
+
. = ALIGN(PAGE_SIZE); /* Init code and data */
__init_begin = .;
.init.text : {
diff --git a/xen/include/asm-arm/domain.h b/xen/include/asm-arm/domain.h
index 312fec8932..0f15372098 100644
--- a/xen/include/asm-arm/domain.h
+++ b/xen/include/asm-arm/domain.h
@@ -58,6 +58,7 @@ struct arch_domain
/* Continuable domain_relinquish_resources(). */
enum {
RELMEM_not_started,
+ RELMEM_tee,
RELMEM_xen,
RELMEM_page,
RELMEM_mapping,
diff --git a/xen/include/asm-arm/tee/tee.h b/xen/include/asm-arm/tee/tee.h
new file mode 100644
index 0000000000..bfdeccc4ad
--- /dev/null
+++ b/xen/include/asm-arm/tee/tee.h
@@ -0,0 +1,106 @@
+/*
+ * xen/include/asm-arm/tee/tee.h
+ *
+ * Generic part of TEE mediator subsystem
+ *
+ * Volodymyr Babchuk <volodymyr_babchuk@epam.com>
+ * Copyright (c) 2018 EPAM Systems.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#ifndef __ARCH_ARM_TEE_TEE_H__
+#define __ARCH_ARM_TEE_TEE_H__
+
+#include <xen/lib.h>
+#include <xen/types.h>
+
+#include <asm/regs.h>
+
+#ifdef CONFIG_TEE
+
+struct tee_mediator_ops {
+ /*
+ * Probe for TEE. Should return true if TEE found and
+ * mediator is initialized.
+ */
+ bool (*probe)(void);
+
+ /*
+ * Called during domain construction if toolstack requests to enable
+ * TEE support so mediator can inform TEE about new
+ * guest and create own structures for the new domain.
+ */
+ int (*domain_init)(struct domain *d);
+
+ /*
+ * Called during domain destruction to relinquish resources used
+ * by mediator itself. This function can return -ERESTART to indicate
+ * that it does not finished work and should be called again.
+ */
+ int (*relinquish_resources)(struct domain *d);
+
+ /*
+ * Called during domain destruction to inform TEE that guest is
+ * now dead and it can free any resources associated with it.
+ * Mediator should also free all own state.
+ */
+ void (*domain_destroy)(struct domain *d);
+
+ /* Handle SMCCC call for current domain. */
+ bool (*handle_call)(struct cpu_user_regs *regs);
+};
+
+struct tee_mediator_desc {
+ /* Name of the TEE. Just for debugging purposes. */
+ const char *name;
+
+ /* Mediator callbacks as described above. */
+ const struct tee_mediator_ops *ops;
+};
+
+bool tee_handle_call(struct cpu_user_regs *regs);
+int tee_domain_init(struct domain *d);
+int tee_relinquish_resources(struct domain *d);
+void tee_domain_destroy(struct domain *d);
+
+#define REGISTER_TEE_MEDIATOR(_name, _namestr, _ops) \
+static const struct tee_mediator_desc __tee_desc_##_name __used \
+__section(".teemediator.info") = { \
+ .name = _namestr, \
+ .ops = _ops \
+}
+
+#else
+
+static inline bool tee_handle_call(struct cpu_user_regs *regs)
+{
+ return false;
+}
+
+static inline int tee_domain_init(struct domain *d)
+{
+ return -ENODEV;
+}
+
+static inline int tee_relinquish_resources(struct domain *d)
+{
+ return 0;
+}
+
+static inline void tee_domain_destroy(struct domain *d) {}
+
+#endif /* CONFIG_TEE */
+
+#endif /* __ARCH_ARM_TEE_TEE_H__ */
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/include/public/arch-arm.h b/xen/include/public/arch-arm.h
index eb424e8286..02aa782e8e 100644
--- a/xen/include/public/arch-arm.h
+++ b/xen/include/public/arch-arm.h
@@ -304,10 +304,14 @@ DEFINE_XEN_GUEST_HANDLE(vcpu_guest_context_t);
#define XEN_DOMCTL_CONFIG_GIC_NATIVE 0
#define XEN_DOMCTL_CONFIG_GIC_V2 1
#define XEN_DOMCTL_CONFIG_GIC_V3 2
+#define XEN_DOMCTL_CONFIG_TEE_NONE 0
+#define XEN_DOMCTL_CONFIG_TEE_NATIVE 1
struct xen_arch_domainconfig {
/* IN/OUT */
uint8_t gic_version;
/* IN */
+ uint8_t tee_type;
+ /* IN */
uint32_t nr_spis;
/*
* OUT
--
2.21.0
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply related [flat|nested] 47+ messages in thread* Re: [PATCH v4 01/10] xen/arm: add generic TEE mediator framework
2019-03-07 21:04 ` [PATCH v4 01/10] xen/arm: add generic TEE mediator framework Volodymyr Babchuk
@ 2019-03-15 15:03 ` Julien Grall
0 siblings, 0 replies; 47+ messages in thread
From: Julien Grall @ 2019-03-15 15:03 UTC (permalink / raw)
To: Volodymyr Babchuk, xen-devel, tee-dev
Cc: Stefano Stabellini, Wei Liu, Konrad Rzeszutek Wilk,
George Dunlap, Andrew Cooper, Ian Jackson, Tim Deegan,
Jan Beulich
Hi,
On 07/03/2019 21:04, Volodymyr Babchuk wrote:
> From: Volodymyr Babchuk <volodymyr_babchuk@epam.com>
>
> This patch adds basic framework for TEE mediators. Guests can't talk
> to TEE directly, we need some entity that will intercept request
> and decide what to do with them. "TEE mediator" is a such entity.
>
> This is how it works: user can build XEN with multiple TEE mediators
> (see the next patches, where OP-TEE mediator is introduced).
> TEE mediator register self with REGISTER_TEE_MEDIATOR() macro in the
> same way, as device drivers use DT_DEVICE_START()/DT_DEVICE_END()
> macros.
> In run-time, during initialization, framework calls probe() function
s/In/At/
[...]
> diff --git a/xen/arch/arm/Makefile b/xen/arch/arm/Makefile
> index cb902cb6fe..5c2aa34557 100644
> --- a/xen/arch/arm/Makefile
> +++ b/xen/arch/arm/Makefile
> @@ -5,6 +5,7 @@ subdir-$(CONFIG_ACPI) += acpi
> ifneq ($(CONFIG_NO_PLAT),y)
> subdir-y += platforms
> endif
> +subdir-$(CONFIG_TEE) += tee
>
> obj-$(CONFIG_HAS_ALTERNATIVE) += alternative.o
> obj-y += bootfdt.init.o
> diff --git a/xen/arch/arm/domain.c b/xen/arch/arm/domain.c
> index 6dc633ed50..d1e2a3979d 100644
> --- a/xen/arch/arm/domain.c
> +++ b/xen/arch/arm/domain.c
> @@ -32,6 +32,7 @@
> #include <asm/platform.h>
> #include <asm/procinfo.h>
> #include <asm/regs.h>
> +#include <asm/tee/tee.h>
> #include <asm/vfp.h>
> #include <asm/vgic.h>
> #include <asm/vtimer.h>
> @@ -705,6 +706,10 @@ int arch_domain_create(struct domain *d,
> if ( (rc = domain_vtimer_init(d, &config->arch)) != 0 )
> goto fail;
>
> + if ( config->arch.tee_type == XEN_DOMCTL_CONFIG_TEE_NATIVE )
Please sanitise tee_type in arch_sanitise_domain_config. Also, can't this check
be pushed in tee_domain_init()?
This would allow us to deal with more tee_type in the future without having to
extend the check here.
> + if ( (rc = tee_domain_init(d)) != 0 )
> + goto fail;
> +
> update_domain_wallclock_time(d);
>
> /*
[...]
> diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c
> index 444857a967..7602dd990c 100644
> --- a/xen/arch/arm/setup.c
> +++ b/xen/arch/arm/setup.c
> @@ -56,6 +56,9 @@ struct bootinfo __initdata bootinfo;
>
> struct cpuinfo_arm __read_mostly boot_cpu_data;
>
> +static bool __initdata opt_dom0_tee_enabled;
> +boolean_param("dom0_tee_enabled", opt_dom0_tee_enabled);
> +
> #ifdef CONFIG_ACPI
> bool __read_mostly acpi_disabled;
> #endif
> @@ -889,6 +892,11 @@ void __init start_xen(unsigned long boot_phys_offset,
> /* The vGIC for DOM0 is exactly emulating the hardware GIC */
> dom0_cfg.arch.gic_version = XEN_DOMCTL_CONFIG_GIC_NATIVE;
> dom0_cfg.arch.nr_spis = gic_number_lines() - 32;
> + if ( opt_dom0_tee_enabled )
> + dom0_cfg.arch.tee_type = XEN_DOMCTL_CONFIG_TEE_NATIVE;
> + else
> + dom0_cfg.arch.tee_type = XEN_DOMCTL_CONFIG_TEE_NONE;
I don't like the idea to introduce a command line to turn OP-TEE on for Dom0.
Dom0 should be able to use OP-TEE by default if mediator is present. Otherwise
you had burden on the user.
We can then decide whether there are reason to allow the user to disable OP-TEE
support for Dom0.
Also, if you disable OP-TEE to Dom0, then you need to make sure the OP-TEE node
is not present in the DT. I don't see code doing that.
Lastly, any new option should be described in docs/misc/xen-command-line.pandoc.
> +
> dom0_cfg.max_vcpus = dom0_max_vcpus();
>
> dom0 = domain_create(0, &dom0_cfg, true);
> diff --git a/xen/arch/arm/tee/Makefile b/xen/arch/arm/tee/Makefile
> new file mode 100644
> index 0000000000..c54d4796ff
> --- /dev/null
> +++ b/xen/arch/arm/tee/Makefile
> @@ -0,0 +1 @@
> +obj-y += tee.o
> diff --git a/xen/arch/arm/tee/tee.c b/xen/arch/arm/tee/tee.c
> new file mode 100644
> index 0000000000..70432306b9
> --- /dev/null
> +++ b/xen/arch/arm/tee/tee.c
> @@ -0,0 +1,79 @@
> +/*
> + * xen/arch/arm/tee/tee.c
> + *
> + * Generic part of TEE mediator subsystem
> + *
> + * Volodymyr Babchuk <volodymyr_babchuk@epam.com>
> + * Copyright (c) 2018 EPAM Systems.
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + */
> +
> +#include <xen/errno.h>
> +#include <xen/init.h>
> +#include <xen/types.h>
> +
> +#include <asm/tee/tee.h>
> +
> +extern const struct tee_mediator_desc _steemediator[], _eteemediator[];
> +static const struct tee_mediator_ops *mediator_ops;
> +
> +bool tee_handle_call(struct cpu_user_regs *regs)
> +{
> + if ( !mediator_ops )
NIT: This could probably be an unlikely as if you end up calling TEE then you
have more chance to have TEE present.
[...]
> +struct tee_mediator_desc {
> + /* Name of the TEE. Just for debugging purposes. */
I would drop "Just for debugging purposes". You use it in non-debug build to
tell what mediator has been used. This could also be useful if we decide to
support multiple mediators at the same time.
> + const char *name;
> +
> + /* Mediator callbacks as described above. */
> + const struct tee_mediator_ops *ops;
> +};
> +
> +bool tee_handle_call(struct cpu_user_regs *regs);
> +int tee_domain_init(struct domain *d);
> +int tee_relinquish_resources(struct domain *d);
> +void tee_domain_destroy(struct domain *d);
> +
> +#define REGISTER_TEE_MEDIATOR(_name, _namestr, _ops) \
Please align \ with the one below.
> +static const struct tee_mediator_desc __tee_desc_##_name __used \
> +__section(".teemediator.info") = { \
> + .name = _namestr, \
> + .ops = _ops \
> +}
> +
> +#else
> +
> +static inline bool tee_handle_call(struct cpu_user_regs *regs)
> +{
> + return false;
> +}
> +
> +static inline int tee_domain_init(struct domain *d)
> +{
> + return -ENODEV;
> +}
> +
> +static inline int tee_relinquish_resources(struct domain *d)
> +{
> + return 0;
> +}
> +
> +static inline void tee_domain_destroy(struct domain *d) {}
> +
> +#endif /* CONFIG_TEE */
> +
> +#endif /* __ARCH_ARM_TEE_TEE_H__ */
> +
> +/*
> + * Local variables:
> + * mode: C
> + * c-file-style: "BSD"
> + * c-basic-offset: 4
> + * indent-tabs-mode: nil
> + * End:
> + */
> diff --git a/xen/include/public/arch-arm.h b/xen/include/public/arch-arm.h
> index eb424e8286..02aa782e8e 100644
> --- a/xen/include/public/arch-arm.h
> +++ b/xen/include/public/arch-arm.h
> @@ -304,10 +304,14 @@ DEFINE_XEN_GUEST_HANDLE(vcpu_guest_context_t);
> #define XEN_DOMCTL_CONFIG_GIC_NATIVE 0
> #define XEN_DOMCTL_CONFIG_GIC_V2 1
> #define XEN_DOMCTL_CONFIG_GIC_V3 2
> +#define XEN_DOMCTL_CONFIG_TEE_NONE 0
> +#define XEN_DOMCTL_CONFIG_TEE_NATIVE 1
While native makes sense when you only have one TEE running on the platform.
This is a bit more difficult to understand if you have multiple TEE in place
(imagine S-EL2).
I am ok with using the word native in the DOMCTL API because it is only exposed
between the hypervisor and the tools. But we should be careful on the naming
used with the user (i.e xl interface or xen command line).
> struct xen_arch_domainconfig {
> /* IN/OUT */
> uint8_t gic_version;
> /* IN */
> + uint8_t tee_type;
> + /* IN */
> uint32_t nr_spis;
> /*
> * OUT
>
Cheers,
--
Julien Grall
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 47+ messages in thread
* [PATCH v4 02/10] xen/arm: optee: add OP-TEE header files
2019-03-07 21:04 [PATCH v4 00/10] TEE mediator (and OP-TEE) support in XEN Volodymyr Babchuk
2019-03-07 21:04 ` [PATCH v4 01/10] xen/arm: add generic TEE mediator framework Volodymyr Babchuk
@ 2019-03-07 21:04 ` Volodymyr Babchuk
2019-03-07 21:04 ` [PATCH v4 04/10] xen/arm: optee: add fast calls handling Volodymyr Babchuk
` (7 subsequent siblings)
9 siblings, 0 replies; 47+ messages in thread
From: Volodymyr Babchuk @ 2019-03-07 21:04 UTC (permalink / raw)
To: xen-devel, tee-dev; +Cc: Julien Grall, Stefano Stabellini, Volodymyr Babchuk
From: Volodymyr Babchuk <volodymyr_babchuk@epam.com>
This header files describes protocol between OP-TEE and OP-TEE client
driver in Linux. They are needed for upcoming OP-TEE mediator, which
is added in the next patch.
Reason to add those headers in separate patch is to ease up review.
Those files were taken from linux tree (drivers/tee/optee/) and mangled
a bit to compile with XEN.
Signed-off-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com>
---
Changes from v3:
- Updated to latest OP-TEE version (3.4.0) because virtualization
support to OP-TEE was merged into mainline.
---
xen/include/asm-arm/tee/optee_msg.h | 444 ++++++++++++++++++++++
xen/include/asm-arm/tee/optee_smc.h | 569 ++++++++++++++++++++++++++++
2 files changed, 1013 insertions(+)
create mode 100644 xen/include/asm-arm/tee/optee_msg.h
create mode 100644 xen/include/asm-arm/tee/optee_smc.h
diff --git a/xen/include/asm-arm/tee/optee_msg.h b/xen/include/asm-arm/tee/optee_msg.h
new file mode 100644
index 0000000000..10747b2aa8
--- /dev/null
+++ b/xen/include/asm-arm/tee/optee_msg.h
@@ -0,0 +1,444 @@
+/*
+ * Copyright (c) 2015-2016, Linaro Limited
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions are met:
+ *
+ * 1. Redistributions of source code must retain the above copyright notice,
+ * this list of conditions and the following disclaimer.
+ *
+ * 2. Redistributions in binary form must reproduce the above copyright notice,
+ * this list of conditions and the following disclaimer in the documentation
+ * and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE
+ * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
+ * POSSIBILITY OF SUCH DAMAGE.
+ */
+#ifndef _OPTEE_MSG_H
+#define _OPTEE_MSG_H
+
+#include <xen/bitops.h>
+#include <xen/types.h>
+
+/*
+ * This file defines the OP-TEE message protocol used to communicate
+ * with an instance of OP-TEE running in secure world.
+ *
+ * This file is divided into three sections.
+ * 1. Formatting of messages.
+ * 2. Requests from normal world
+ * 3. Requests from secure world, Remote Procedure Call (RPC), handled by
+ * tee-supplicant.
+ */
+
+/*****************************************************************************
+ * Part 1 - formatting of messages
+ *****************************************************************************/
+
+#define OPTEE_MSG_ATTR_TYPE_NONE 0x0
+#define OPTEE_MSG_ATTR_TYPE_VALUE_INPUT 0x1
+#define OPTEE_MSG_ATTR_TYPE_VALUE_OUTPUT 0x2
+#define OPTEE_MSG_ATTR_TYPE_VALUE_INOUT 0x3
+#define OPTEE_MSG_ATTR_TYPE_RMEM_INPUT 0x5
+#define OPTEE_MSG_ATTR_TYPE_RMEM_OUTPUT 0x6
+#define OPTEE_MSG_ATTR_TYPE_RMEM_INOUT 0x7
+#define OPTEE_MSG_ATTR_TYPE_TMEM_INPUT 0x9
+#define OPTEE_MSG_ATTR_TYPE_TMEM_OUTPUT 0xa
+#define OPTEE_MSG_ATTR_TYPE_TMEM_INOUT 0xb
+
+#define OPTEE_MSG_ATTR_TYPE_MASK GENMASK(7, 0)
+
+/*
+ * Meta parameter to be absorbed by the Secure OS and not passed
+ * to the Trusted Application.
+ *
+ * Currently only used with OPTEE_MSG_CMD_OPEN_SESSION.
+ */
+#define OPTEE_MSG_ATTR_META BIT(8)
+
+/*
+ * Pointer to a list of pages used to register user-defined SHM buffer.
+ * Used with OPTEE_MSG_ATTR_TYPE_TMEM_*.
+ * buf_ptr should point to the beginning of the buffer. Buffer will contain
+ * list of page addresses. OP-TEE core can reconstruct contiguous buffer from
+ * that page addresses list. Page addresses are stored as 64 bit values.
+ * Last entry on a page should point to the next page of buffer.
+ * Every entry in buffer should point to a 4k page beginning (12 least
+ * significant bits must be equal to zero).
+ *
+ * 12 least significant bints of optee_msg_param.u.tmem.buf_ptr should hold page
+ * offset of the user buffer.
+ *
+ * So, entries should be placed like members of this structure:
+ *
+ * struct page_data {
+ * uint64_t pages_array[OPTEE_MSG_NONCONTIG_PAGE_SIZE/sizeof(uint64_t) - 1];
+ * uint64_t next_page_data;
+ * };
+ *
+ * Structure is designed to exactly fit into the page size
+ * OPTEE_MSG_NONCONTIG_PAGE_SIZE which is a standard 4KB page.
+ *
+ * The size of 4KB is chosen because this is the smallest page size for ARM
+ * architectures. If REE uses larger pages, it should divide them to 4KB ones.
+ */
+#define OPTEE_MSG_ATTR_NONCONTIG BIT(9)
+
+/*
+ * Memory attributes for caching passed with temp memrefs. The actual value
+ * used is defined outside the message protocol with the exception of
+ * OPTEE_MSG_ATTR_CACHE_PREDEFINED which means the attributes already
+ * defined for the memory range should be used. If optee_smc.h is used as
+ * bearer of this protocol OPTEE_SMC_SHM_* is used for values.
+ */
+#define OPTEE_MSG_ATTR_CACHE_SHIFT 16
+#define OPTEE_MSG_ATTR_CACHE_MASK GENMASK(2, 0)
+#define OPTEE_MSG_ATTR_CACHE_PREDEFINED 0
+
+/*
+ * Same values as TEE_LOGIN_* from TEE Internal API
+ */
+#define OPTEE_MSG_LOGIN_PUBLIC 0x00000000
+#define OPTEE_MSG_LOGIN_USER 0x00000001
+#define OPTEE_MSG_LOGIN_GROUP 0x00000002
+#define OPTEE_MSG_LOGIN_APPLICATION 0x00000004
+#define OPTEE_MSG_LOGIN_APPLICATION_USER 0x00000005
+#define OPTEE_MSG_LOGIN_APPLICATION_GROUP 0x00000006
+
+/*
+ * Page size used in non-contiguous buffer entries
+ */
+#define OPTEE_MSG_NONCONTIG_PAGE_SIZE 4096
+
+/**
+ * struct optee_msg_param_tmem - temporary memory reference parameter
+ * @buf_ptr: Address of the buffer
+ * @size: Size of the buffer
+ * @shm_ref: Temporary shared memory reference, pointer to a struct tee_shm
+ *
+ * Secure and normal world communicates pointers as physical address
+ * instead of the virtual address. This is because secure and normal world
+ * have completely independent memory mapping. Normal world can even have a
+ * hypervisor which need to translate the guest physical address (AKA IPA
+ * in ARM documentation) to a real physical address before passing the
+ * structure to secure world.
+ */
+struct optee_msg_param_tmem {
+ u64 buf_ptr;
+ u64 size;
+ u64 shm_ref;
+};
+
+/**
+ * struct optee_msg_param_rmem - registered memory reference parameter
+ * @offs: Offset into shared memory reference
+ * @size: Size of the buffer
+ * @shm_ref: Shared memory reference, pointer to a struct tee_shm
+ */
+struct optee_msg_param_rmem {
+ u64 offs;
+ u64 size;
+ u64 shm_ref;
+};
+
+/**
+ * struct optee_msg_param_value - opaque value parameter
+ *
+ * Value parameters are passed unchecked between normal and secure world.
+ */
+struct optee_msg_param_value {
+ u64 a;
+ u64 b;
+ u64 c;
+};
+
+/**
+ * struct optee_msg_param - parameter used together with struct optee_msg_arg
+ * @attr: attributes
+ * @tmem: parameter by temporary memory reference
+ * @rmem: parameter by registered memory reference
+ * @value: parameter by opaque value
+ *
+ * @attr & OPTEE_MSG_ATTR_TYPE_MASK indicates if tmem, rmem or value is used in
+ * the union. OPTEE_MSG_ATTR_TYPE_VALUE_* indicates value,
+ * OPTEE_MSG_ATTR_TYPE_TMEM_* indicates @tmem and
+ * OPTEE_MSG_ATTR_TYPE_RMEM_* indicates @rmem,
+ * OPTEE_MSG_ATTR_TYPE_NONE indicates that none of the members are used.
+ */
+struct optee_msg_param {
+ u64 attr;
+ union {
+ struct optee_msg_param_tmem tmem;
+ struct optee_msg_param_rmem rmem;
+ struct optee_msg_param_value value;
+ } u;
+};
+
+/**
+ * struct optee_msg_arg - call argument
+ * @cmd: Command, one of OPTEE_MSG_CMD_* or OPTEE_MSG_RPC_CMD_*
+ * @func: Trusted Application function, specific to the Trusted Application,
+ * used if cmd == OPTEE_MSG_CMD_INVOKE_COMMAND
+ * @session: In parameter for all OPTEE_MSG_CMD_* except
+ * OPTEE_MSG_CMD_OPEN_SESSION where it's an output parameter instead
+ * @cancel_id: Cancellation id, a unique value to identify this request
+ * @ret: return value
+ * @ret_origin: origin of the return value
+ * @num_params: number of parameters supplied to the OS Command
+ * @params: the parameters supplied to the OS Command
+ *
+ * All normal calls to Trusted OS uses this struct. If cmd requires further
+ * information than what these field holds it can be passed as a parameter
+ * tagged as meta (setting the OPTEE_MSG_ATTR_META bit in corresponding
+ * attrs field). All parameters tagged as meta has to come first.
+ *
+ * Temp memref parameters can be fragmented if supported by the Trusted OS
+ * (when optee_smc.h is bearer of this protocol this is indicated with
+ * OPTEE_SMC_SEC_CAP_UNREGISTERED_SHM). If a logical memref parameter is
+ * fragmented then has all but the last fragment the
+ * OPTEE_MSG_ATTR_FRAGMENT bit set in attrs. Even if a memref is fragmented
+ * it will still be presented as a single logical memref to the Trusted
+ * Application.
+ */
+struct optee_msg_arg {
+ u32 cmd;
+ u32 func;
+ u32 session;
+ u32 cancel_id;
+ u32 pad;
+ u32 ret;
+ u32 ret_origin;
+ u32 num_params;
+
+ /* num_params tells the actual number of element in params */
+ struct optee_msg_param params[0];
+};
+
+/**
+ * OPTEE_MSG_GET_ARG_SIZE - return size of struct optee_msg_arg
+ *
+ * @num_params: Number of parameters embedded in the struct optee_msg_arg
+ *
+ * Returns the size of the struct optee_msg_arg together with the number
+ * of embedded parameters.
+ */
+#define OPTEE_MSG_GET_ARG_SIZE(num_params) \
+ (sizeof(struct optee_msg_arg) + \
+ sizeof(struct optee_msg_param) * (num_params))
+
+/*****************************************************************************
+ * Part 2 - requests from normal world
+ *****************************************************************************/
+
+/*
+ * Return the following UID if using API specified in this file without
+ * further extensions:
+ * 384fb3e0-e7f8-11e3-af63-0002a5d5c51b.
+ * Represented in 4 32-bit words in OPTEE_MSG_UID_0, OPTEE_MSG_UID_1,
+ * OPTEE_MSG_UID_2, OPTEE_MSG_UID_3.
+ */
+#define OPTEE_MSG_UID_0 0x384fb3e0
+#define OPTEE_MSG_UID_1 0xe7f811e3
+#define OPTEE_MSG_UID_2 0xaf630002
+#define OPTEE_MSG_UID_3 0xa5d5c51b
+#define OPTEE_MSG_FUNCID_CALLS_UID 0xFF01
+
+/*
+ * Returns 2.0 if using API specified in this file without further
+ * extensions. Represented in 2 32-bit words in OPTEE_MSG_REVISION_MAJOR
+ * and OPTEE_MSG_REVISION_MINOR
+ */
+#define OPTEE_MSG_REVISION_MAJOR 2
+#define OPTEE_MSG_REVISION_MINOR 0
+#define OPTEE_MSG_FUNCID_CALLS_REVISION 0xFF03
+
+/*
+ * Get UUID of Trusted OS.
+ *
+ * Used by non-secure world to figure out which Trusted OS is installed.
+ * Note that returned UUID is the UUID of the Trusted OS, not of the API.
+ *
+ * Returns UUID in 4 32-bit words in the same way as
+ * OPTEE_MSG_FUNCID_CALLS_UID described above.
+ */
+#define OPTEE_MSG_OS_OPTEE_UUID_0 0x486178e0
+#define OPTEE_MSG_OS_OPTEE_UUID_1 0xe7f811e3
+#define OPTEE_MSG_OS_OPTEE_UUID_2 0xbc5e0002
+#define OPTEE_MSG_OS_OPTEE_UUID_3 0xa5d5c51b
+#define OPTEE_MSG_FUNCID_GET_OS_UUID 0x0000
+
+/*
+ * Get revision of Trusted OS.
+ *
+ * Used by non-secure world to figure out which version of the Trusted OS
+ * is installed. Note that the returned revision is the revision of the
+ * Trusted OS, not of the API.
+ *
+ * Returns revision in 2 32-bit words in the same way as
+ * OPTEE_MSG_CALLS_REVISION described above.
+ */
+#define OPTEE_MSG_FUNCID_GET_OS_REVISION 0x0001
+
+/*
+ * Do a secure call with struct optee_msg_arg as argument
+ * The OPTEE_MSG_CMD_* below defines what goes in struct optee_msg_arg::cmd
+ *
+ * OPTEE_MSG_CMD_OPEN_SESSION opens a session to a Trusted Application.
+ * The first two parameters are tagged as meta, holding two value
+ * parameters to pass the following information:
+ * param[0].u.value.a-b uuid of Trusted Application
+ * param[1].u.value.a-b uuid of Client
+ * param[1].u.value.c Login class of client OPTEE_MSG_LOGIN_*
+ *
+ * OPTEE_MSG_CMD_INVOKE_COMMAND invokes a command a previously opened
+ * session to a Trusted Application. struct optee_msg_arg::func is Trusted
+ * Application function, specific to the Trusted Application.
+ *
+ * OPTEE_MSG_CMD_CLOSE_SESSION closes a previously opened session to
+ * Trusted Application.
+ *
+ * OPTEE_MSG_CMD_CANCEL cancels a currently invoked command.
+ *
+ * OPTEE_MSG_CMD_REGISTER_SHM registers a shared memory reference. The
+ * information is passed as:
+ * [in] param[0].attr OPTEE_MSG_ATTR_TYPE_TMEM_INPUT
+ * [| OPTEE_MSG_ATTR_FRAGMENT]
+ * [in] param[0].u.tmem.buf_ptr physical address (of first fragment)
+ * [in] param[0].u.tmem.size size (of first fragment)
+ * [in] param[0].u.tmem.shm_ref holds shared memory reference
+ * ...
+ * The shared memory can optionally be fragmented, temp memrefs can follow
+ * each other with all but the last with the OPTEE_MSG_ATTR_FRAGMENT bit set.
+ *
+ * OPTEE_MSG_CMD_UNREGISTER_SHM unregisteres a previously registered shared
+ * memory reference. The information is passed as:
+ * [in] param[0].attr OPTEE_MSG_ATTR_TYPE_RMEM_INPUT
+ * [in] param[0].u.rmem.shm_ref holds shared memory reference
+ * [in] param[0].u.rmem.offs 0
+ * [in] param[0].u.rmem.size 0
+ */
+#define OPTEE_MSG_CMD_OPEN_SESSION 0
+#define OPTEE_MSG_CMD_INVOKE_COMMAND 1
+#define OPTEE_MSG_CMD_CLOSE_SESSION 2
+#define OPTEE_MSG_CMD_CANCEL 3
+#define OPTEE_MSG_CMD_REGISTER_SHM 4
+#define OPTEE_MSG_CMD_UNREGISTER_SHM 5
+#define OPTEE_MSG_FUNCID_CALL_WITH_ARG 0x0004
+
+/*****************************************************************************
+ * Part 3 - Requests from secure world, RPC
+ *****************************************************************************/
+
+/*
+ * All RPC is done with a struct optee_msg_arg as bearer of information,
+ * struct optee_msg_arg::arg holds values defined by OPTEE_MSG_RPC_CMD_* below
+ *
+ * RPC communication with tee-supplicant is reversed compared to normal
+ * client communication desribed above. The supplicant receives requests
+ * and sends responses.
+ */
+
+/*
+ * Load a TA into memory, defined in tee-supplicant
+ */
+#define OPTEE_MSG_RPC_CMD_LOAD_TA 0
+
+/*
+ * Reserved
+ */
+#define OPTEE_MSG_RPC_CMD_RPMB 1
+
+/*
+ * File system access, defined in tee-supplicant
+ */
+#define OPTEE_MSG_RPC_CMD_FS 2
+
+/*
+ * Get time
+ *
+ * Returns number of seconds and nano seconds since the Epoch,
+ * 1970-01-01 00:00:00 +0000 (UTC).
+ *
+ * [out] param[0].u.value.a Number of seconds
+ * [out] param[0].u.value.b Number of nano seconds.
+ */
+#define OPTEE_MSG_RPC_CMD_GET_TIME 3
+
+/*
+ * Wait queue primitive, helper for secure world to implement a wait queue.
+ *
+ * If secure world need to wait for a secure world mutex it issues a sleep
+ * request instead of spinning in secure world. Conversely is a wakeup
+ * request issued when a secure world mutex with a thread waiting thread is
+ * unlocked.
+ *
+ * Waiting on a key
+ * [in] param[0].u.value.a OPTEE_MSG_RPC_WAIT_QUEUE_SLEEP
+ * [in] param[0].u.value.b wait key
+ *
+ * Waking up a key
+ * [in] param[0].u.value.a OPTEE_MSG_RPC_WAIT_QUEUE_WAKEUP
+ * [in] param[0].u.value.b wakeup key
+ */
+#define OPTEE_MSG_RPC_CMD_WAIT_QUEUE 4
+#define OPTEE_MSG_RPC_WAIT_QUEUE_SLEEP 0
+#define OPTEE_MSG_RPC_WAIT_QUEUE_WAKEUP 1
+
+/*
+ * Suspend execution
+ *
+ * [in] param[0].value .a number of milliseconds to suspend
+ */
+#define OPTEE_MSG_RPC_CMD_SUSPEND 5
+
+/*
+ * Allocate a piece of shared memory
+ *
+ * Shared memory can optionally be fragmented, to support that additional
+ * spare param entries are allocated to make room for eventual fragments.
+ * The spare param entries has .attr = OPTEE_MSG_ATTR_TYPE_NONE when
+ * unused. All returned temp memrefs except the last should have the
+ * OPTEE_MSG_ATTR_FRAGMENT bit set in the attr field.
+ *
+ * [in] param[0].u.value.a type of memory one of
+ * OPTEE_MSG_RPC_SHM_TYPE_* below
+ * [in] param[0].u.value.b requested size
+ * [in] param[0].u.value.c required alignment
+ *
+ * [out] param[0].u.tmem.buf_ptr physical address (of first fragment)
+ * [out] param[0].u.tmem.size size (of first fragment)
+ * [out] param[0].u.tmem.shm_ref shared memory reference
+ * ...
+ * [out] param[n].u.tmem.buf_ptr physical address
+ * [out] param[n].u.tmem.size size
+ * [out] param[n].u.tmem.shm_ref shared memory reference (same value
+ * as in param[n-1].u.tmem.shm_ref)
+ */
+#define OPTEE_MSG_RPC_CMD_SHM_ALLOC 6
+/* Memory that can be shared with a non-secure user space application */
+#define OPTEE_MSG_RPC_SHM_TYPE_APPL 0
+/* Memory only shared with non-secure kernel */
+#define OPTEE_MSG_RPC_SHM_TYPE_KERNEL 1
+
+/*
+ * Free shared memory previously allocated with OPTEE_MSG_RPC_CMD_SHM_ALLOC
+ *
+ * [in] param[0].u.value.a type of memory one of
+ * OPTEE_MSG_RPC_SHM_TYPE_* above
+ * [in] param[0].u.value.b value of shared memory reference
+ * returned in param[0].u.tmem.shm_ref
+ * above
+ */
+#define OPTEE_MSG_RPC_CMD_SHM_FREE 7
+
+#endif /* _OPTEE_MSG_H */
diff --git a/xen/include/asm-arm/tee/optee_smc.h b/xen/include/asm-arm/tee/optee_smc.h
new file mode 100644
index 0000000000..5a6494384e
--- /dev/null
+++ b/xen/include/asm-arm/tee/optee_smc.h
@@ -0,0 +1,569 @@
+/* SPDX-License-Identifier: BSD-2-Clause */
+/*
+ * Copyright (c) 2015, Linaro Limited
+ */
+#ifndef OPTEE_SMC_H
+#define OPTEE_SMC_H
+
+#include <asm/smccc.h>
+#include <xen/bitops.h>
+
+#define OPTEE_SMC_STD_CALL_VAL(func_num) \
+ ARM_SMCCC_CALL_VAL(ARM_SMCCC_STD_CALL, ARM_SMCCC_CONV_32, \
+ ARM_SMCCC_OWNER_TRUSTED_OS, (func_num))
+#define OPTEE_SMC_FAST_CALL_VAL(func_num) \
+ ARM_SMCCC_CALL_VAL(ARM_SMCCC_FAST_CALL, ARM_SMCCC_CONV_32, \
+ ARM_SMCCC_OWNER_TRUSTED_OS, (func_num))
+
+/*
+ * Function specified by SMC Calling convention.
+ */
+#define OPTEE_SMC_FUNCID_CALLS_COUNT 0xFF00
+#define OPTEE_SMC_CALLS_COUNT \
+ ARM_SMCCC_CALL_VAL(ARM_SMCCC_FAST_CALL, ARM_SMCCC_CONV_32, \
+ ARM_SMCCC_OWNER_TRUSTED_OS_END, \
+ OPTEE_SMC_FUNCID_CALLS_COUNT)
+
+/*
+ * Normal cached memory (write-back), shareable for SMP systems and not
+ * shareable for UP systems.
+ */
+#define OPTEE_SMC_SHM_CACHED 1
+
+/*
+ * a0..a7 is used as register names in the descriptions below, on arm32
+ * that translates to r0..r7 and on arm64 to w0..w7. In both cases it's
+ * 32-bit registers.
+ */
+
+/*
+ * Function specified by SMC Calling convention
+ *
+ * Return the following UID if using API specified in this file
+ * without further extensions:
+ * 384fb3e0-e7f8-11e3-af63-0002a5d5c51b.
+ * see also OPTEE_MSG_UID_* in optee_msg.h
+ */
+#define OPTEE_SMC_FUNCID_CALLS_UID OPTEE_MSG_FUNCID_CALLS_UID
+#define OPTEE_SMC_CALLS_UID \
+ ARM_SMCCC_CALL_VAL(ARM_SMCCC_FAST_CALL, ARM_SMCCC_CONV_32, \
+ ARM_SMCCC_OWNER_TRUSTED_OS_END, \
+ OPTEE_SMC_FUNCID_CALLS_UID)
+
+/*
+ * Function specified by SMC Calling convention
+ *
+ * Returns 2.0 if using API specified in this file without further extensions.
+ * see also OPTEE_MSG_REVISION_* in optee_msg.h
+ */
+#define OPTEE_SMC_FUNCID_CALLS_REVISION OPTEE_MSG_FUNCID_CALLS_REVISION
+#define OPTEE_SMC_CALLS_REVISION \
+ ARM_SMCCC_CALL_VAL(ARM_SMCCC_FAST_CALL, ARM_SMCCC_CONV_32, \
+ ARM_SMCCC_OWNER_TRUSTED_OS_END, \
+ OPTEE_SMC_FUNCID_CALLS_REVISION)
+
+/*
+ * Get UUID of Trusted OS.
+ *
+ * Used by non-secure world to figure out which Trusted OS is installed.
+ * Note that returned UUID is the UUID of the Trusted OS, not of the API.
+ *
+ * Returns UUID in a0-4 in the same way as OPTEE_SMC_CALLS_UID
+ * described above.
+ */
+#define OPTEE_SMC_FUNCID_GET_OS_UUID OPTEE_MSG_FUNCID_GET_OS_UUID
+#define OPTEE_SMC_CALL_GET_OS_UUID \
+ OPTEE_SMC_FAST_CALL_VAL(OPTEE_SMC_FUNCID_GET_OS_UUID)
+
+/*
+ * Get revision of Trusted OS.
+ *
+ * Used by non-secure world to figure out which version of the Trusted OS
+ * is installed. Note that the returned revision is the revision of the
+ * Trusted OS, not of the API.
+ *
+ * Returns revision in a0-1 in the same way as OPTEE_SMC_CALLS_REVISION
+ * described above. May optionally return a 32-bit build identifier in a2,
+ * with zero meaning unspecified.
+ */
+#define OPTEE_SMC_FUNCID_GET_OS_REVISION OPTEE_MSG_FUNCID_GET_OS_REVISION
+#define OPTEE_SMC_CALL_GET_OS_REVISION \
+ OPTEE_SMC_FAST_CALL_VAL(OPTEE_SMC_FUNCID_GET_OS_REVISION)
+
+/*
+ * Call with struct optee_msg_arg as argument
+ *
+ * Call register usage:
+ * a0 SMC Function ID, OPTEE_SMC*CALL_WITH_ARG
+ * a1 Upper 32 bits of a 64-bit physical pointer to a struct optee_msg_arg
+ * a2 Lower 32 bits of a 64-bit physical pointer to a struct optee_msg_arg
+ * a3 Cache settings, not used if physical pointer is in a predefined shared
+ * memory area else per OPTEE_SMC_SHM_*
+ * a4-6 Not used
+ * a7 Hypervisor Client ID register
+ *
+ * Normal return register usage:
+ * a0 Return value, OPTEE_SMC_RETURN_*
+ * a1-3 Not used
+ * a4-7 Preserved
+ *
+ * OPTEE_SMC_RETURN_ETHREAD_LIMIT return register usage:
+ * a0 Return value, OPTEE_SMC_RETURN_ETHREAD_LIMIT
+ * a1-3 Preserved
+ * a4-7 Preserved
+ *
+ * RPC return register usage:
+ * a0 Return value, OPTEE_SMC_RETURN_IS_RPC(val)
+ * a1-2 RPC parameters
+ * a3-7 Resume information, must be preserved
+ *
+ * Possible return values:
+ * OPTEE_SMC_RETURN_UNKNOWN_FUNCTION Trusted OS does not recognize this
+ * function.
+ * OPTEE_SMC_RETURN_OK Call completed, result updated in
+ * the previously supplied struct
+ * optee_msg_arg.
+ * OPTEE_SMC_RETURN_ETHREAD_LIMIT Number of Trusted OS threads exceeded,
+ * try again later.
+ * OPTEE_SMC_RETURN_EBADADDR Bad physical pointer to struct
+ * optee_msg_arg.
+ * OPTEE_SMC_RETURN_EBADCMD Bad/unknown cmd in struct optee_msg_arg
+ * OPTEE_SMC_RETURN_IS_RPC() Call suspended by RPC call to normal
+ * world.
+ */
+#define OPTEE_SMC_FUNCID_CALL_WITH_ARG OPTEE_MSG_FUNCID_CALL_WITH_ARG
+#define OPTEE_SMC_CALL_WITH_ARG \
+ OPTEE_SMC_STD_CALL_VAL(OPTEE_SMC_FUNCID_CALL_WITH_ARG)
+
+/*
+ * Get Shared Memory Config
+ *
+ * Returns the Secure/Non-secure shared memory config.
+ *
+ * Call register usage:
+ * a0 SMC Function ID, OPTEE_SMC_GET_SHM_CONFIG
+ * a1-6 Not used
+ * a7 Hypervisor Client ID register
+ *
+ * Have config return register usage:
+ * a0 OPTEE_SMC_RETURN_OK
+ * a1 Physical address of start of SHM
+ * a2 Size of of SHM
+ * a3 Cache settings of memory, as defined by the
+ * OPTEE_SMC_SHM_* values above
+ * a4-7 Preserved
+ *
+ * Not available register usage:
+ * a0 OPTEE_SMC_RETURN_ENOTAVAIL
+ * a1-3 Not used
+ * a4-7 Preserved
+ */
+#define OPTEE_SMC_FUNCID_GET_SHM_CONFIG 7
+#define OPTEE_SMC_GET_SHM_CONFIG \
+ OPTEE_SMC_FAST_CALL_VAL(OPTEE_SMC_FUNCID_GET_SHM_CONFIG)
+
+/*
+ * Configures L2CC mutex
+ *
+ * Disables, enables usage of L2CC mutex. Returns or sets physical address
+ * of L2CC mutex.
+ *
+ * Call register usage:
+ * a0 SMC Function ID, OPTEE_SMC_L2CC_MUTEX
+ * a1 OPTEE_SMC_L2CC_MUTEX_GET_ADDR Get physical address of mutex
+ * OPTEE_SMC_L2CC_MUTEX_SET_ADDR Set physical address of mutex
+ * OPTEE_SMC_L2CC_MUTEX_ENABLE Enable usage of mutex
+ * OPTEE_SMC_L2CC_MUTEX_DISABLE Disable usage of mutex
+ * a2 if a1 == OPTEE_SMC_L2CC_MUTEX_SET_ADDR, upper 32bit of a 64bit
+ * physical address of mutex
+ * a3 if a1 == OPTEE_SMC_L2CC_MUTEX_SET_ADDR, lower 32bit of a 64bit
+ * physical address of mutex
+ * a3-6 Not used
+ * a7 Hypervisor Client ID register
+ *
+ * Have config return register usage:
+ * a0 OPTEE_SMC_RETURN_OK
+ * a1 Preserved
+ * a2 if a1 == OPTEE_SMC_L2CC_MUTEX_GET_ADDR, upper 32bit of a 64bit
+ * physical address of mutex
+ * a3 if a1 == OPTEE_SMC_L2CC_MUTEX_GET_ADDR, lower 32bit of a 64bit
+ * physical address of mutex
+ * a3-7 Preserved
+ *
+ * Error return register usage:
+ * a0 OPTEE_SMC_RETURN_ENOTAVAIL Physical address not available
+ * OPTEE_SMC_RETURN_EBADADDR Bad supplied physical address
+ * OPTEE_SMC_RETURN_EBADCMD Unsupported value in a1
+ * a1-7 Preserved
+ */
+#define OPTEE_SMC_L2CC_MUTEX_GET_ADDR 0
+#define OPTEE_SMC_L2CC_MUTEX_SET_ADDR 1
+#define OPTEE_SMC_L2CC_MUTEX_ENABLE 2
+#define OPTEE_SMC_L2CC_MUTEX_DISABLE 3
+#define OPTEE_SMC_FUNCID_L2CC_MUTEX 8
+#define OPTEE_SMC_L2CC_MUTEX \
+ OPTEE_SMC_FAST_CALL_VAL(OPTEE_SMC_FUNCID_L2CC_MUTEX)
+
+/*
+ * Exchanges capabilities between normal world and secure world
+ *
+ * Call register usage:
+ * a0 SMC Function ID, OPTEE_SMC_EXCHANGE_CAPABILITIES
+ * a1 bitfield of normal world capabilities OPTEE_SMC_NSEC_CAP_*
+ * a2-6 Not used
+ * a7 Hypervisor Client ID register
+ *
+ * Normal return register usage:
+ * a0 OPTEE_SMC_RETURN_OK
+ * a1 bitfield of secure world capabilities OPTEE_SMC_SEC_CAP_*
+ * a2-7 Preserved
+ *
+ * Error return register usage:
+ * a0 OPTEE_SMC_RETURN_ENOTAVAIL, can't use the capabilities from normal world
+ * a1 bitfield of secure world capabilities OPTEE_SMC_SEC_CAP_*
+ * a2-7 Preserved
+ */
+/* Normal world works as a uniprocessor system */
+#define OPTEE_SMC_NSEC_CAP_UNIPROCESSOR (1 << 0)
+/* Secure world has reserved shared memory for normal world to use */
+#define OPTEE_SMC_SEC_CAP_HAVE_RESERVED_SHM (1 << 0)
+/* Secure world can communicate via previously unregistered shared memory */
+#define OPTEE_SMC_SEC_CAP_UNREGISTERED_SHM (1 << 1)
+
+/*
+ * Secure world supports commands "register/unregister shared memory",
+ * secure world accepts command buffers located in any parts of non-secure RAM
+ */
+#define OPTEE_SMC_SEC_CAP_DYNAMIC_SHM (1 << 2)
+
+#define OPTEE_SMC_FUNCID_EXCHANGE_CAPABILITIES 9
+#define OPTEE_SMC_EXCHANGE_CAPABILITIES \
+ OPTEE_SMC_FAST_CALL_VAL(OPTEE_SMC_FUNCID_EXCHANGE_CAPABILITIES)
+
+/*
+ * Disable and empties cache of shared memory objects
+ *
+ * Secure world can cache frequently used shared memory objects, for
+ * example objects used as RPC arguments. When secure world is idle this
+ * function returns one shared memory reference to free. To disable the
+ * cache and free all cached objects this function has to be called until
+ * it returns OPTEE_SMC_RETURN_ENOTAVAIL.
+ *
+ * Call register usage:
+ * a0 SMC Function ID, OPTEE_SMC_DISABLE_SHM_CACHE
+ * a1-6 Not used
+ * a7 Hypervisor Client ID register
+ *
+ * Normal return register usage:
+ * a0 OPTEE_SMC_RETURN_OK
+ * a1 Upper 32 bits of a 64-bit Shared memory cookie
+ * a2 Lower 32 bits of a 64-bit Shared memory cookie
+ * a3-7 Preserved
+ *
+ * Cache empty return register usage:
+ * a0 OPTEE_SMC_RETURN_ENOTAVAIL
+ * a1-7 Preserved
+ *
+ * Not idle return register usage:
+ * a0 OPTEE_SMC_RETURN_EBUSY
+ * a1-7 Preserved
+ */
+#define OPTEE_SMC_FUNCID_DISABLE_SHM_CACHE 10
+#define OPTEE_SMC_DISABLE_SHM_CACHE \
+ OPTEE_SMC_FAST_CALL_VAL(OPTEE_SMC_FUNCID_DISABLE_SHM_CACHE)
+
+/*
+ * Enable cache of shared memory objects
+ *
+ * Secure world can cache frequently used shared memory objects, for
+ * example objects used as RPC arguments. When secure world is idle this
+ * function returns OPTEE_SMC_RETURN_OK and the cache is enabled. If
+ * secure world isn't idle OPTEE_SMC_RETURN_EBUSY is returned.
+ *
+ * Call register usage:
+ * a0 SMC Function ID, OPTEE_SMC_ENABLE_SHM_CACHE
+ * a1-6 Not used
+ * a7 Hypervisor Client ID register
+ *
+ * Normal return register usage:
+ * a0 OPTEE_SMC_RETURN_OK
+ * a1-7 Preserved
+ *
+ * Not idle return register usage:
+ * a0 OPTEE_SMC_RETURN_EBUSY
+ * a1-7 Preserved
+ */
+#define OPTEE_SMC_FUNCID_ENABLE_SHM_CACHE 11
+#define OPTEE_SMC_ENABLE_SHM_CACHE \
+ OPTEE_SMC_FAST_CALL_VAL(OPTEE_SMC_FUNCID_ENABLE_SHM_CACHE)
+
+/*
+ * Release of secondary cores
+ *
+ * OP-TEE in secure world is in charge of the release process of secondary
+ * cores. The Rich OS issue the this request to ask OP-TEE to boot up the
+ * secondary cores, go through the OP-TEE per-core initialization, and then
+ * switch to the Non-seCure world with the Rich OS provided entry address.
+ * The secondary cores enter Non-Secure world in SVC mode, with Thumb, FIQ,
+ * IRQ and Abort bits disabled.
+ *
+ * Call register usage:
+ * a0 SMC Function ID, OPTEE_SMC_BOOT_SECONDARY
+ * a1 Index of secondary core to boot
+ * a2 Upper 32 bits of a 64-bit Non-Secure world entry physical address
+ * a3 Lower 32 bits of a 64-bit Non-Secure world entry physical address
+ * a4-7 Not used
+ *
+ * Normal return register usage:
+ * a0 OPTEE_SMC_RETURN_OK
+ * a1-7 Preserved
+ *
+ * Error return:
+ * a0 OPTEE_SMC_RETURN_EBADCMD Core index out of range
+ * a1-7 Preserved
+ *
+ * Not idle return register usage:
+ * a0 OPTEE_SMC_RETURN_EBUSY
+ * a1-7 Preserved
+ */
+#define OPTEE_SMC_FUNCID_BOOT_SECONDARY 12
+#define OPTEE_SMC_BOOT_SECONDARY \
+ OPTEE_SMC_FAST_CALL_VAL(OPTEE_SMC_FUNCID_BOOT_SECONDARY)
+
+/*
+ * Inform OP-TEE about a new virtual machine
+ *
+ * Hypervisor issues this call during virtual machine (guest) creation.
+ * OP-TEE records client id of new virtual machine and prepares
+ * to receive requests from it. This call is available only if OP-TEE
+ * was built with virtualization support.
+ *
+ * Call requests usage:
+ * a0 SMC Function ID, OPTEE_SMC_VM_CREATED
+ * a1 Hypervisor Client ID of newly created virtual machine
+ * a2-6 Not used
+ * a7 Hypervisor Client ID register. Must be 0, because only hypervisor
+ * can issue this call
+ *
+ * Normal return register usage:
+ * a0 OPTEE_SMC_RETURN_OK
+ * a1-7 Preserved
+ *
+ * Error return:
+ * a0 OPTEE_SMC_RETURN_ENOTAVAIL OP-TEE have no resources for
+ * another VM
+ * a1-7 Preserved
+ *
+ */
+#define OPTEE_SMC_FUNCID_VM_CREATED 13
+#define OPTEE_SMC_VM_CREATED \
+ OPTEE_SMC_FAST_CALL_VAL(OPTEE_SMC_FUNCID_VM_CREATED)
+
+/*
+ * Inform OP-TEE about shutdown of a virtual machine
+ *
+ * Hypervisor issues this call during virtual machine (guest) destruction.
+ * OP-TEE will clean up all resources associated with this VM. This call is
+ * available only if OP-TEE was built with virtualization support.
+ *
+ * Call requests usage:
+ * a0 SMC Function ID, OPTEE_SMC_VM_DESTROYED
+ * a1 Hypervisor Client ID of virtual machine being shut down
+ * a2-6 Not used
+ * a7 Hypervisor Client ID register. Must be 0, because only hypervisor
+ * can issue this call
+ *
+ * Normal return register usage:
+ * a0 OPTEE_SMC_RETURN_OK
+ * a1-7 Preserved
+ *
+ */
+#define OPTEE_SMC_FUNCID_VM_DESTROYED 14
+#define OPTEE_SMC_VM_DESTROYED \
+ OPTEE_SMC_FAST_CALL_VAL(OPTEE_SMC_FUNCID_VM_DESTROYED)
+
+/*
+ * Query OP-TEE about its build configuration
+ *
+ * Normal World OS or Hypervisor issues this call to find out certain OP-TEE
+ * build options, like number of supported threads or if virtualization support
+ * is enabled.
+ *
+ * Call requests usage:
+ * a0 SMC Function ID, OPTEE_SMC_GET_CONFIG
+ * a1 Identifier of configuration option
+ * a2-6 Not used
+ * a7 Hypervisor Client ID register
+ *
+ * Normal return register usage:
+ * a0 OPTEE_SMC_RETURN_OK
+ * a1-6 Requested information
+ * a7 Preserved
+ *
+ * Error return:
+ * a0 OPTEE_SMC_RETURN_ENOTAVAIL Unknown configuration option was
+ * supplied in a1 register
+ * a1-7 Preserved
+ */
+#define OPTEE_SMC_FUNCID_GET_CONFIG 15
+#define OPTEE_SMC_GET_CONFIG \
+ OPTEE_SMC_FAST_CALL_VAL(OPTEE_SMC_FUNCID_GET_CONFIG)
+
+/* Possible configuration options for OPTEE_SMC_FUNCID_GET_CONFIG call: */
+
+/* a1 will contain 0. Used to check of OPTEE_SMC_FUNCID_GET_CONFIG is supported */
+#define OPTEE_SMC_CONFIG_NONE 0
+
+/* a1 will contain maximum number of threads supported by OP-TEE */
+#define OPTEE_SMC_CONFIG_NUM_THREADS 1
+
+/* a1 will contain 1 if virtualization support is enabled, 0 otherwise */
+#define OPTEE_SMC_CONFIG_VIRTUALIZATION 2
+
+/*
+ * Resume from RPC (for example after processing a foreign interrupt)
+ *
+ * Call register usage:
+ * a0 SMC Function ID, OPTEE_SMC_CALL_RETURN_FROM_RPC
+ * a1-3 Value of a1-3 when OPTEE_SMC_CALL_WITH_ARG returned
+ * OPTEE_SMC_RETURN_RPC in a0
+ *
+ * Return register usage is the same as for OPTEE_SMC_*CALL_WITH_ARG above.
+ *
+ * Possible return values
+ * OPTEE_SMC_RETURN_UNKNOWN_FUNCTION Trusted OS does not recognize this
+ * function.
+ * OPTEE_SMC_RETURN_OK Original call completed, result
+ * updated in the previously supplied.
+ * struct optee_msg_arg
+ * OPTEE_SMC_RETURN_RPC Call suspended by RPC call to normal
+ * world.
+ * OPTEE_SMC_RETURN_ERESUME Resume failed, the opaque resume
+ * information was corrupt.
+ */
+#define OPTEE_SMC_FUNCID_RETURN_FROM_RPC 3
+#define OPTEE_SMC_CALL_RETURN_FROM_RPC \
+ OPTEE_SMC_STD_CALL_VAL(OPTEE_SMC_FUNCID_RETURN_FROM_RPC)
+
+#define OPTEE_SMC_RETURN_RPC_PREFIX_MASK 0xFFFF0000
+#define OPTEE_SMC_RETURN_RPC_PREFIX 0xFFFF0000
+#define OPTEE_SMC_RETURN_RPC_FUNC_MASK 0x0000FFFF
+
+#define OPTEE_SMC_RETURN_GET_RPC_FUNC(ret) \
+ ((ret) & OPTEE_SMC_RETURN_RPC_FUNC_MASK)
+
+#define OPTEE_SMC_RPC_VAL(func) ((func) | OPTEE_SMC_RETURN_RPC_PREFIX)
+
+/*
+ * Allocate memory for RPC parameter passing. The memory is used to hold a
+ * struct optee_msg_arg.
+ *
+ * "Call" register usage:
+ * a0 This value, OPTEE_SMC_RETURN_RPC_ALLOC
+ * a1 Size in bytes of required argument memory
+ * a2 Not used
+ * a3 Resume information, must be preserved
+ * a4-5 Not used
+ * a6-7 Resume information, must be preserved
+ *
+ * "Return" register usage:
+ * a0 SMC Function ID, OPTEE_SMC_CALL_RETURN_FROM_RPC.
+ * a1 Upper 32 bits of 64-bit physical pointer to allocated
+ * memory, (a1 == 0 && a2 == 0) if size was 0 or if memory can't
+ * be allocated.
+ * a2 Lower 32 bits of 64-bit physical pointer to allocated
+ * memory, (a1 == 0 && a2 == 0) if size was 0 or if memory can't
+ * be allocated
+ * a3 Preserved
+ * a4 Upper 32 bits of 64-bit Shared memory cookie used when freeing
+ * the memory or doing an RPC
+ * a5 Lower 32 bits of 64-bit Shared memory cookie used when freeing
+ * the memory or doing an RPC
+ * a6-7 Preserved
+ */
+#define OPTEE_SMC_RPC_FUNC_ALLOC 0
+#define OPTEE_SMC_RETURN_RPC_ALLOC \
+ OPTEE_SMC_RPC_VAL(OPTEE_SMC_RPC_FUNC_ALLOC)
+
+/*
+ * Free memory previously allocated by OPTEE_SMC_RETURN_RPC_ALLOC
+ *
+ * "Call" register usage:
+ * a0 This value, OPTEE_SMC_RETURN_RPC_FREE
+ * a1 Upper 32 bits of 64-bit shared memory cookie belonging to this
+ * argument memory
+ * a2 Lower 32 bits of 64-bit shared memory cookie belonging to this
+ * argument memory
+ * a3-7 Resume information, must be preserved
+ *
+ * "Return" register usage:
+ * a0 SMC Function ID, OPTEE_SMC_CALL_RETURN_FROM_RPC.
+ * a1-2 Not used
+ * a3-7 Preserved
+ */
+#define OPTEE_SMC_RPC_FUNC_FREE 2
+#define OPTEE_SMC_RETURN_RPC_FREE \
+ OPTEE_SMC_RPC_VAL(OPTEE_SMC_RPC_FUNC_FREE)
+
+/*
+ * Deliver a foreign interrupt in normal world.
+ *
+ * "Call" register usage:
+ * a0 OPTEE_SMC_RETURN_RPC_FOREIGN_INTR
+ * a1-7 Resume information, must be preserved
+ *
+ * "Return" register usage:
+ * a0 SMC Function ID, OPTEE_SMC_CALL_RETURN_FROM_RPC.
+ * a1-7 Preserved
+ */
+#define OPTEE_SMC_RPC_FUNC_FOREIGN_INTR 4
+#define OPTEE_SMC_RETURN_RPC_FOREIGN_INTR \
+ OPTEE_SMC_RPC_VAL(OPTEE_SMC_RPC_FUNC_FOREIGN_INTR)
+
+/*
+ * Do an RPC request. The supplied struct optee_msg_arg tells which
+ * request to do and the parameters for the request. The following fields
+ * are used (the rest are unused):
+ * - cmd the Request ID
+ * - ret return value of the request, filled in by normal world
+ * - num_params number of parameters for the request
+ * - params the parameters
+ * - param_attrs attributes of the parameters
+ *
+ * "Call" register usage:
+ * a0 OPTEE_SMC_RETURN_RPC_CMD
+ * a1 Upper 32 bits of a 64-bit Shared memory cookie holding a
+ * struct optee_msg_arg, must be preserved, only the data should
+ * be updated
+ * a2 Lower 32 bits of a 64-bit Shared memory cookie holding a
+ * struct optee_msg_arg, must be preserved, only the data should
+ * be updated
+ * a3-7 Resume information, must be preserved
+ *
+ * "Return" register usage:
+ * a0 SMC Function ID, OPTEE_SMC_CALL_RETURN_FROM_RPC.
+ * a1-2 Not used
+ * a3-7 Preserved
+ */
+#define OPTEE_SMC_RPC_FUNC_CMD 5
+#define OPTEE_SMC_RETURN_RPC_CMD \
+ OPTEE_SMC_RPC_VAL(OPTEE_SMC_RPC_FUNC_CMD)
+
+/* Returned in a0 */
+#define OPTEE_SMC_RETURN_UNKNOWN_FUNCTION 0xFFFFFFFF
+
+/* Returned in a0 only from Trusted OS functions */
+#define OPTEE_SMC_RETURN_OK 0x0
+#define OPTEE_SMC_RETURN_ETHREAD_LIMIT 0x1
+#define OPTEE_SMC_RETURN_EBUSY 0x2
+#define OPTEE_SMC_RETURN_ERESUME 0x3
+#define OPTEE_SMC_RETURN_EBADADDR 0x4
+#define OPTEE_SMC_RETURN_EBADCMD 0x5
+#define OPTEE_SMC_RETURN_ENOMEM 0x6
+#define OPTEE_SMC_RETURN_ENOTAVAIL 0x7
+#define OPTEE_SMC_RETURN_IS_RPC(ret) \
+ (((ret) != OPTEE_SMC_RETURN_UNKNOWN_FUNCTION) && \
+ ((((ret) & OPTEE_SMC_RETURN_RPC_PREFIX_MASK) == \
+ OPTEE_SMC_RETURN_RPC_PREFIX)))
+
+#endif /* OPTEE_SMC_H */
--
2.21.0
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply related [flat|nested] 47+ messages in thread* [PATCH v4 04/10] xen/arm: optee: add fast calls handling
2019-03-07 21:04 [PATCH v4 00/10] TEE mediator (and OP-TEE) support in XEN Volodymyr Babchuk
2019-03-07 21:04 ` [PATCH v4 01/10] xen/arm: add generic TEE mediator framework Volodymyr Babchuk
2019-03-07 21:04 ` [PATCH v4 02/10] xen/arm: optee: add OP-TEE header files Volodymyr Babchuk
@ 2019-03-07 21:04 ` Volodymyr Babchuk
2019-03-15 15:46 ` Julien Grall
2019-03-07 21:04 ` [PATCH v4 03/10] xen/arm: optee: add OP-TEE mediator skeleton Volodymyr Babchuk
` (6 subsequent siblings)
9 siblings, 1 reply; 47+ messages in thread
From: Volodymyr Babchuk @ 2019-03-07 21:04 UTC (permalink / raw)
To: xen-devel, tee-dev
Cc: Volodymyr Babchuk, Julien Grall, Stefano Stabellini, Volodymyr Babchuk
From: Volodymyr Babchuk <vlad.babchuk@gmail.com>
Some fast SMCCC calls to OP-TEE should be handled in a special way.
Capabilities exchange should be filtered out, so only caps
known to mediator are used. Also mediator disables static SHM
memory capability, because it can't share OP-TEE memory with a domain.
Only domain can share memory with OP-TEE, so it ensures that OP-TEE
supports dynamic SHM.
Basically, static SHM is a reserved memory region which is always
mapped into OP-TEE address space. It belongs to OP-TEE. Normally,
NW is allowed to access there, so it can communicate with OP-TEE.
On other hand, dynamic SHM is NW's own memory, which it can share
with OP-TEE. OP-TEE maps this memory dynamically, when it wants to
access it.
Because mediator can't share one static SHM region with all guests, it
just disables it for all of them. It is possible to make exception for
Dom0, and it will be done in a separate patch.
Signed-off-by: Volodymyr Babchuk <vlad.babchuk@gmail.com>
---
All the patches to optee.c should be merged together. They were
split to ease up review. But they depend heavily on each other.
Changes from v2:
- Defined known capabilities explicitly
- Fixed code style
---
xen/arch/arm/tee/optee.c | 57 ++++++++++++++++++++++++++++++++++++++--
1 file changed, 55 insertions(+), 2 deletions(-)
diff --git a/xen/arch/arm/tee/optee.c b/xen/arch/arm/tee/optee.c
index b50c581aaf..61554ea191 100644
--- a/xen/arch/arm/tee/optee.c
+++ b/xen/arch/arm/tee/optee.c
@@ -24,6 +24,11 @@
/* Client ID 0 is reserved for hypervisor itself */
#define OPTEE_CLIENT_ID(domain) ((domain)->domain_id + 1)
+#define OPTEE_KNOWN_NSEC_CAPS OPTEE_SMC_NSEC_CAP_UNIPROCESSOR
+#define OPTEE_KNOWN_SEC_CAPS (OPTEE_SMC_SEC_CAP_HAVE_RESERVED_SHM | \
+ OPTEE_SMC_SEC_CAP_UNREGISTERED_SHM | \
+ OPTEE_SMC_SEC_CAP_DYNAMIC_SHM)
+
static bool optee_probe(void)
{
struct dt_device_node *node;
@@ -99,6 +104,18 @@ static void forward_call(struct cpu_user_regs *regs)
set_user_reg(regs, 7, 0);
}
+static void set_return(struct cpu_user_regs *regs, uint32_t ret)
+{
+ set_user_reg(regs, 0, ret);
+ set_user_reg(regs, 1, 0);
+ set_user_reg(regs, 2, 0);
+ set_user_reg(regs, 3, 0);
+ set_user_reg(regs, 4, 0);
+ set_user_reg(regs, 5, 0);
+ set_user_reg(regs, 6, 0);
+ set_user_reg(regs, 7, 0);
+}
+
static int optee_relinquish_resources(struct domain *d)
{
return 0;
@@ -124,6 +141,37 @@ static void optee_domain_destroy(struct domain *d)
&resp);
}
+static void handle_exchange_capabilities(struct cpu_user_regs *regs)
+{
+ uint32_t caps;
+
+ /* Filter out unknown guest caps */
+ caps = get_user_reg(regs, 1);
+ caps &= OPTEE_KNOWN_NSEC_CAPS;
+ set_user_reg(regs, 1, caps);
+
+ forward_call(regs);
+ if ( get_user_reg(regs, 0) != OPTEE_SMC_RETURN_OK )
+ return;
+
+ caps = get_user_reg(regs, 1);
+
+ /* Filter out unknown OP-TEE caps */
+ caps &= OPTEE_KNOWN_SEC_CAPS;
+
+ /* Drop static SHM_RPC cap */
+ caps &= ~OPTEE_SMC_SEC_CAP_HAVE_RESERVED_SHM;
+
+ /* Don't allow guests to work without dynamic SHM */
+ if ( !(caps & OPTEE_SMC_SEC_CAP_DYNAMIC_SHM) )
+ {
+ set_return(regs, OPTEE_SMC_RETURN_ENOTAVAIL);
+ return;
+ }
+
+ set_user_reg(regs, 1, caps);
+}
+
static bool optee_handle_call(struct cpu_user_regs *regs)
{
if ( !current->domain->arch.tee )
@@ -138,12 +186,17 @@ static bool optee_handle_call(struct cpu_user_regs *regs)
case OPTEE_SMC_FUNCID_GET_OS_REVISION:
case OPTEE_SMC_ENABLE_SHM_CACHE:
case OPTEE_SMC_DISABLE_SHM_CACHE:
- case OPTEE_SMC_GET_SHM_CONFIG:
- case OPTEE_SMC_EXCHANGE_CAPABILITIES:
case OPTEE_SMC_CALL_WITH_ARG:
case OPTEE_SMC_CALL_RETURN_FROM_RPC:
forward_call(regs);
return true;
+ case OPTEE_SMC_GET_SHM_CONFIG:
+ /* No static SHM available for guests */
+ set_return(regs, OPTEE_SMC_RETURN_ENOTAVAIL);
+ return true;
+ case OPTEE_SMC_EXCHANGE_CAPABILITIES:
+ handle_exchange_capabilities(regs);
+ return true;
default:
return false;
}
--
2.21.0
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply related [flat|nested] 47+ messages in thread* Re: [PATCH v4 04/10] xen/arm: optee: add fast calls handling
2019-03-07 21:04 ` [PATCH v4 04/10] xen/arm: optee: add fast calls handling Volodymyr Babchuk
@ 2019-03-15 15:46 ` Julien Grall
0 siblings, 0 replies; 47+ messages in thread
From: Julien Grall @ 2019-03-15 15:46 UTC (permalink / raw)
To: Volodymyr Babchuk, xen-devel, tee-dev
Cc: Volodymyr Babchuk, Stefano Stabellini
Hi Volodymyr,
On 07/03/2019 21:04, Volodymyr Babchuk wrote:
> +static void set_return(struct cpu_user_regs *regs, uint32_t ret)
> +{
> + set_user_reg(regs, 0, ret);
> + set_user_reg(regs, 1, 0);
> + set_user_reg(regs, 2, 0);
> + set_user_reg(regs, 3, 0);
> + set_user_reg(regs, 4, 0);
> + set_user_reg(regs, 5, 0);
> + set_user_reg(regs, 6, 0);
> + set_user_reg(regs, 7, 0);
This does not look correct to me. From patch #2, registers 4-7 are always
preserved. registers 1-3 may be preserved as well depending on the call.
More importantly, Xen is now using SMCCC interface v1.1. This means, register
0-3 are used for return values, registers 4-7 are preserved.
Cheers,
--
Julien Grall
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 47+ messages in thread
* [PATCH v4 03/10] xen/arm: optee: add OP-TEE mediator skeleton
2019-03-07 21:04 [PATCH v4 00/10] TEE mediator (and OP-TEE) support in XEN Volodymyr Babchuk
` (2 preceding siblings ...)
2019-03-07 21:04 ` [PATCH v4 04/10] xen/arm: optee: add fast calls handling Volodymyr Babchuk
@ 2019-03-07 21:04 ` Volodymyr Babchuk
2019-03-15 15:24 ` Julien Grall
2019-03-15 15:47 ` Julien Grall
2019-03-07 21:04 ` [PATCH v4 05/10] xen/arm: optee: add std call handling Volodymyr Babchuk
` (5 subsequent siblings)
9 siblings, 2 replies; 47+ messages in thread
From: Volodymyr Babchuk @ 2019-03-07 21:04 UTC (permalink / raw)
To: xen-devel, tee-dev
Cc: Volodymyr Babchuk, Julien Grall, Stefano Stabellini, Volodymyr Babchuk
From: Volodymyr Babchuk <vlad.babchuk@gmail.com>
Add very basic OP-TEE mediator. It can probe for OP-TEE presence,
tell it about domain creation/destruction and forward all known
calls.
This is all what is needed for Dom0 to work with OP-TEE as long as
Dom0 shares 1:1 mapped pages with OP-TEE. Any attempt to call OP-TEE
from DomU will fail and will lead to random memory corruption. But
this is impossible, because there is no means to enable TEE support
for DomUs right now. Also, problems can arise if Dom0 uses pages
mapped from other domains. This will be fixed in the following
patches, as more of the mediator functionality will be added.
This code issues two non-preemptible calls to OP-TEE: to create and
to destroy client context. They can't block in OP-TEE, as they are
considered "fast calls" in terms of ARM SMCCC.
Signed-off-by: Volodymyr Babchuk <vlad.babchuk@gmail.com>
---
All the patches to optee.c should be merged together. They were
split to ease up review. But they depend heavily on each other.
Changes from v3:
- Introduced optee_relinquish_resources() function to free
mediator resources in a more controllable way
Changes from v2:
- Fixed coding style
- Introduced tee/Kconfig
- Fixed error messages
---
xen/arch/arm/Kconfig | 2 +
xen/arch/arm/tee/Kconfig | 4 +
xen/arch/arm/tee/Makefile | 1 +
xen/arch/arm/tee/optee.c | 170 +++++++++++++++++++++++++++++++++++
xen/include/asm-arm/domain.h | 3 +
5 files changed, 180 insertions(+)
create mode 100644 xen/arch/arm/tee/Kconfig
create mode 100644 xen/arch/arm/tee/optee.c
diff --git a/xen/arch/arm/Kconfig b/xen/arch/arm/Kconfig
index e527b2f885..99e6f0ebb2 100644
--- a/xen/arch/arm/Kconfig
+++ b/xen/arch/arm/Kconfig
@@ -237,3 +237,5 @@ source "arch/arm/platforms/Kconfig"
source "common/Kconfig"
source "drivers/Kconfig"
+
+source "arch/arm/tee/Kconfig"
diff --git a/xen/arch/arm/tee/Kconfig b/xen/arch/arm/tee/Kconfig
new file mode 100644
index 0000000000..5b829db2e9
--- /dev/null
+++ b/xen/arch/arm/tee/Kconfig
@@ -0,0 +1,4 @@
+config OPTEE
+ bool "Enable OP-TEE mediator"
+ default n
+ depends on TEE
diff --git a/xen/arch/arm/tee/Makefile b/xen/arch/arm/tee/Makefile
index c54d4796ff..982c879684 100644
--- a/xen/arch/arm/tee/Makefile
+++ b/xen/arch/arm/tee/Makefile
@@ -1 +1,2 @@
obj-y += tee.o
+obj-$(CONFIG_OPTEE) += optee.o
diff --git a/xen/arch/arm/tee/optee.c b/xen/arch/arm/tee/optee.c
new file mode 100644
index 0000000000..b50c581aaf
--- /dev/null
+++ b/xen/arch/arm/tee/optee.c
@@ -0,0 +1,170 @@
+/*
+ * xen/arch/arm/tee/optee.c
+ *
+ * OP-TEE mediator
+ *
+ * Volodymyr Babchuk <volodymyr_babchuk@epam.com>
+ * Copyright (c) 2018 EPAM Systems.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#include <xen/device_tree.h>
+#include <xen/sched.h>
+
+#include <asm/smccc.h>
+#include <asm/tee/tee.h>
+#include <asm/tee/optee_msg.h>
+#include <asm/tee/optee_smc.h>
+
+#define OPTEE_ENABLED ((void*)0x1)
+
+/* Client ID 0 is reserved for hypervisor itself */
+#define OPTEE_CLIENT_ID(domain) ((domain)->domain_id + 1)
+
+static bool optee_probe(void)
+{
+ struct dt_device_node *node;
+ struct arm_smccc_res resp;
+
+ /* Check for entry in dtb */
+ node = dt_find_compatible_node(NULL, NULL, "linaro,optee-tz");
+ if ( !node )
+ return false;
+
+ /* Check UID */
+ arm_smccc_smc(ARM_SMCCC_CALL_UID_FID(TRUSTED_OS_END), &resp);
+
+ if ( (uint32_t)resp.a0 != OPTEE_MSG_UID_0 ||
+ (uint32_t)resp.a1 != OPTEE_MSG_UID_1 ||
+ (uint32_t)resp.a2 != OPTEE_MSG_UID_2 ||
+ (uint32_t)resp.a3 != OPTEE_MSG_UID_3 )
+ return false;
+
+ return true;
+}
+
+static int optee_domain_init(struct domain *d)
+{
+ struct arm_smccc_res resp;
+
+ /*
+ * Inform OP-TEE about a new guest.
+ * This is a "Fast" call in terms of OP-TEE. This basically
+ * means that it can't be preempted, because there is no
+ * thread allocated for it in OP-TEE. It is close to atomic
+ * context in linux kernel: E.g. no blocking calls can be issued.
+ * Also, interrupts are disabled.
+ *
+ * a7 should be 0, so we can't skip last 6 parameters of arm_smccc_smc()
+ */
+ arm_smccc_smc(OPTEE_SMC_VM_CREATED, OPTEE_CLIENT_ID(d), 0, 0, 0, 0, 0, 0,
+ &resp);
+ if ( resp.a0 != OPTEE_SMC_RETURN_OK )
+ {
+ gprintk(XENLOG_WARNING, "Unable to create OPTEE client: rc = 0x%X\n",
+ (uint32_t)resp.a0);
+
+ return -ENODEV;
+ }
+
+ d->arch.tee = OPTEE_ENABLED;
+
+ return 0;
+}
+
+static void forward_call(struct cpu_user_regs *regs)
+{
+ struct arm_smccc_res resp;
+
+ arm_smccc_smc(get_user_reg(regs, 0),
+ get_user_reg(regs, 1),
+ get_user_reg(regs, 2),
+ get_user_reg(regs, 3),
+ get_user_reg(regs, 4),
+ get_user_reg(regs, 5),
+ get_user_reg(regs, 6),
+ OPTEE_CLIENT_ID(current->domain),
+ &resp);
+
+ set_user_reg(regs, 0, resp.a0);
+ set_user_reg(regs, 1, resp.a1);
+ set_user_reg(regs, 2, resp.a2);
+ set_user_reg(regs, 3, resp.a3);
+ set_user_reg(regs, 4, 0);
+ set_user_reg(regs, 5, 0);
+ set_user_reg(regs, 6, 0);
+ set_user_reg(regs, 7, 0);
+}
+
+static int optee_relinquish_resources(struct domain *d)
+{
+ return 0;
+}
+
+static void optee_domain_destroy(struct domain *d)
+{
+ struct arm_smccc_res resp;
+
+ if ( !d->arch.tee )
+ return;
+
+ /*
+ * Inform OP-TEE that domain is shutting down. This is
+ * also a fast SMC call, like OPTEE_SMC_VM_CREATED, so
+ * it is also non-preemptible.
+ * At this time all domain VCPUs should be stopped. OP-TEE
+ * relies on this.
+ *
+ * a7 should be 0, so we can't skip last 6 parameters od arm_smccc_smc()
+ */
+ arm_smccc_smc(OPTEE_SMC_VM_DESTROYED, OPTEE_CLIENT_ID(d), 0, 0, 0, 0, 0, 0,
+ &resp);
+}
+
+static bool optee_handle_call(struct cpu_user_regs *regs)
+{
+ if ( !current->domain->arch.tee )
+ return false;
+
+ switch ( get_user_reg(regs, 0) )
+ {
+ case OPTEE_SMC_CALLS_COUNT:
+ case OPTEE_SMC_CALLS_UID:
+ case OPTEE_SMC_CALLS_REVISION:
+ case OPTEE_SMC_CALL_GET_OS_UUID:
+ case OPTEE_SMC_FUNCID_GET_OS_REVISION:
+ case OPTEE_SMC_ENABLE_SHM_CACHE:
+ case OPTEE_SMC_DISABLE_SHM_CACHE:
+ case OPTEE_SMC_GET_SHM_CONFIG:
+ case OPTEE_SMC_EXCHANGE_CAPABILITIES:
+ case OPTEE_SMC_CALL_WITH_ARG:
+ case OPTEE_SMC_CALL_RETURN_FROM_RPC:
+ forward_call(regs);
+ return true;
+ default:
+ return false;
+ }
+}
+
+static const struct tee_mediator_ops optee_ops =
+{
+ .probe = optee_probe,
+ .domain_init = optee_domain_init,
+ .domain_destroy = optee_domain_destroy,
+ .relinquish_resources = optee_relinquish_resources,
+ .handle_call = optee_handle_call,
+};
+
+REGISTER_TEE_MEDIATOR(optee, "OP-TEE", &optee_ops);
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/include/asm-arm/domain.h b/xen/include/asm-arm/domain.h
index 0f15372098..2960a53e69 100644
--- a/xen/include/asm-arm/domain.h
+++ b/xen/include/asm-arm/domain.h
@@ -98,6 +98,9 @@ struct arch_domain
struct vpl011 vpl011;
#endif
+#ifdef CONFIG_TEE
+ void *tee;
+#endif
} __cacheline_aligned;
struct arch_vcpu
--
2.21.0
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply related [flat|nested] 47+ messages in thread* Re: [PATCH v4 03/10] xen/arm: optee: add OP-TEE mediator skeleton
2019-03-07 21:04 ` [PATCH v4 03/10] xen/arm: optee: add OP-TEE mediator skeleton Volodymyr Babchuk
@ 2019-03-15 15:24 ` Julien Grall
2019-03-15 19:00 ` Volodymyr Babchuk
2019-03-15 15:47 ` Julien Grall
1 sibling, 1 reply; 47+ messages in thread
From: Julien Grall @ 2019-03-15 15:24 UTC (permalink / raw)
To: Volodymyr Babchuk, xen-devel, tee-dev
Cc: Volodymyr Babchuk, Stefano Stabellini
Hi,
On 07/03/2019 21:04, Volodymyr Babchuk wrote:
> From: Volodymyr Babchuk <vlad.babchuk@gmail.com>
>
> Add very basic OP-TEE mediator. It can probe for OP-TEE presence,
> tell it about domain creation/destruction and forward all known
> calls.
>
> This is all what is needed for Dom0 to work with OP-TEE as long as
> Dom0 shares 1:1 mapped pages with OP-TEE. Any attempt to call OP-TEE
> from DomU will fail and will lead to random memory corruption. But
> this is impossible, because there is no means to enable TEE support
> for DomUs right now. Also, problems can arise if Dom0 uses pages
> mapped from other domains. This will be fixed in the following
> patches, as more of the mediator functionality will be added.
>
> This code issues two non-preemptible calls to OP-TEE: to create and
> to destroy client context. They can't block in OP-TEE, as they are
> considered "fast calls" in terms of ARM SMCCC.
>
> Signed-off-by: Volodymyr Babchuk <vlad.babchuk@gmail.com>
>
> ---
>
> All the patches to optee.c should be merged together. They were
> split to ease up review. But they depend heavily on each other.
>
> Changes from v3:
> - Introduced optee_relinquish_resources() function to free
> mediator resources in a more controllable way
>
> Changes from v2:
> - Fixed coding style
> - Introduced tee/Kconfig
> - Fixed error messages
> ---
> xen/arch/arm/Kconfig | 2 +
> xen/arch/arm/tee/Kconfig | 4 +
> xen/arch/arm/tee/Makefile | 1 +
> xen/arch/arm/tee/optee.c | 170 +++++++++++++++++++++++++++++++++++
> xen/include/asm-arm/domain.h | 3 +
> 5 files changed, 180 insertions(+)
> create mode 100644 xen/arch/arm/tee/Kconfig
> create mode 100644 xen/arch/arm/tee/optee.c
>
> diff --git a/xen/arch/arm/Kconfig b/xen/arch/arm/Kconfig
> index e527b2f885..99e6f0ebb2 100644
> --- a/xen/arch/arm/Kconfig
> +++ b/xen/arch/arm/Kconfig
> @@ -237,3 +237,5 @@ source "arch/arm/platforms/Kconfig"
> source "common/Kconfig"
>
> source "drivers/Kconfig"
> +
> +source "arch/arm/tee/Kconfig"
> diff --git a/xen/arch/arm/tee/Kconfig b/xen/arch/arm/tee/Kconfig
> new file mode 100644
> index 0000000000..5b829db2e9
> --- /dev/null
> +++ b/xen/arch/arm/tee/Kconfig
> @@ -0,0 +1,4 @@
> +config OPTEE
> + bool "Enable OP-TEE mediator"
> + default n
> + depends on TEE
> diff --git a/xen/arch/arm/tee/Makefile b/xen/arch/arm/tee/Makefile
> index c54d4796ff..982c879684 100644
> --- a/xen/arch/arm/tee/Makefile
> +++ b/xen/arch/arm/tee/Makefile
> @@ -1 +1,2 @@
> obj-y += tee.o
> +obj-$(CONFIG_OPTEE) += optee.o
> diff --git a/xen/arch/arm/tee/optee.c b/xen/arch/arm/tee/optee.c
> new file mode 100644
> index 0000000000..b50c581aaf
> --- /dev/null
> +++ b/xen/arch/arm/tee/optee.c
> @@ -0,0 +1,170 @@
> +/*
> + * xen/arch/arm/tee/optee.c
> + *
> + * OP-TEE mediator
> + *
> + * Volodymyr Babchuk <volodymyr_babchuk@epam.com>
> + * Copyright (c) 2018 EPAM Systems.
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
I think you are missing parts of the GPLv2 here:
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
> + */
> +
> +#include <xen/device_tree.h>
> +#include <xen/sched.h>
> +
> +#include <asm/smccc.h>
> +#include <asm/tee/tee.h>
> +#include <asm/tee/optee_msg.h>
> +#include <asm/tee/optee_smc.h>
> +
> +#define OPTEE_ENABLED ((void*)0x1)
Please don't do that. Introduce a dummy structure instead and fill it up when
needed.
> +
> +/* Client ID 0 is reserved for hypervisor itself */
s/for/for the/
> +#define OPTEE_CLIENT_ID(domain) ((domain)->domain_id + 1)
> +
> +static bool optee_probe(void)
> +{
> + struct dt_device_node *node;
> + struct arm_smccc_res resp;
> +
> + /* Check for entry in dtb */
> + node = dt_find_compatible_node(NULL, NULL, "linaro,optee-tz");
> + if ( !node )
> + return false;
> +
> + /* Check UID */
> + arm_smccc_smc(ARM_SMCCC_CALL_UID_FID(TRUSTED_OS_END), &resp);
> +
> + if ( (uint32_t)resp.a0 != OPTEE_MSG_UID_0 ||
> + (uint32_t)resp.a1 != OPTEE_MSG_UID_1 ||
> + (uint32_t)resp.a2 != OPTEE_MSG_UID_2 ||
> + (uint32_t)resp.a3 != OPTEE_MSG_UID_3 )
> + return false;
> +
> + return true;
> +}
> +
> +static int optee_domain_init(struct domain *d)
> +{
> + struct arm_smccc_res resp;
> +
> + /*
> + * Inform OP-TEE about a new guest.
> + * This is a "Fast" call in terms of OP-TEE. This basically
> + * means that it can't be preempted, because there is no
> + * thread allocated for it in OP-TEE. It is close to atomic
> + * context in linux kernel: E.g. no blocking calls can be issued.
This does not really make sense to describe Linux here. Can't you just make the
wording OS agnostic?
> + * Also, interrupts are disabled.
> + *
> + * a7 should be 0, so we can't skip last 6 parameters of arm_smccc_smc()
> + */
> + arm_smccc_smc(OPTEE_SMC_VM_CREATED, OPTEE_CLIENT_ID(d), 0, 0, 0, 0, 0, 0,
> + &resp);
> + if ( resp.a0 != OPTEE_SMC_RETURN_OK )
> + {
> + gprintk(XENLOG_WARNING, "Unable to create OPTEE client: rc = 0x%X\n",
gprintk will print the current vCPU and not the domain created. This is not very
useful to know the current domain. So it would be better to use:
printk(XENLOG_G_WARNING, "%pd: Unable to create OPTEE client: rc ...", d);
> + (uint32_t)resp.a0);
> +
> + return -ENODEV;
> + }
> +
> + d->arch.tee = OPTEE_ENABLED;
> +
> + return 0;
> +}
> +
> +static void forward_call(struct cpu_user_regs *regs)
> +{
> + struct arm_smccc_res resp;
> +
> + arm_smccc_smc(get_user_reg(regs, 0),
> + get_user_reg(regs, 1),
> + get_user_reg(regs, 2),
> + get_user_reg(regs, 3),
> + get_user_reg(regs, 4),
> + get_user_reg(regs, 5),
> + get_user_reg(regs, 6),
> + OPTEE_CLIENT_ID(current->domain),
> + &resp);
> +
> + set_user_reg(regs, 0, resp.a0);
> + set_user_reg(regs, 1, resp.a1);
> + set_user_reg(regs, 2, resp.a2);
> + set_user_reg(regs, 3, resp.a3);
> + set_user_reg(regs, 4, 0);
> + set_user_reg(regs, 5, 0);
> + set_user_reg(regs, 6, 0);
> + set_user_reg(regs, 7, 0);
> +}
> +
> +static int optee_relinquish_resources(struct domain *d)
> +{
> + return 0;
> +}
> +
> +static void optee_domain_destroy(struct domain *d)
> +{
> + struct arm_smccc_res resp;
> +
> + if ( !d->arch.tee )
> + return;
> +
> + /*
> + * Inform OP-TEE that domain is shutting down. This is
> + * also a fast SMC call, like OPTEE_SMC_VM_CREATED, so
> + * it is also non-preemptible.
> + * At this time all domain VCPUs should be stopped. OP-TEE
> + * relies on this.
> + *
> + * a7 should be 0, so we can't skip last 6 parameters od arm_smccc_smc()
NIT: s/od/of/
> + */
> + arm_smccc_smc(OPTEE_SMC_VM_DESTROYED, OPTEE_CLIENT_ID(d), 0, 0, 0, 0, 0, 0,
> + &resp);
Your split between domain_destroy and relinquish_resources looks wrong. If you
relinquish resources before telling OP-TEE then you are at risk that OP-TEE will
use those resources.
Instead you should first tell OP-TEE the domain is shutting down, then release
the resources.
> +}
> +
> +static bool optee_handle_call(struct cpu_user_regs *regs)
> +{
> + if ( !current->domain->arch.tee )
> + return false;
> +
> + switch ( get_user_reg(regs, 0) )
> + {
> + case OPTEE_SMC_CALLS_COUNT:
> + case OPTEE_SMC_CALLS_UID:
> + case OPTEE_SMC_CALLS_REVISION:
> + case OPTEE_SMC_CALL_GET_OS_UUID:
> + case OPTEE_SMC_FUNCID_GET_OS_REVISION:
> + case OPTEE_SMC_ENABLE_SHM_CACHE:
> + case OPTEE_SMC_DISABLE_SHM_CACHE:
> + case OPTEE_SMC_GET_SHM_CONFIG:
> + case OPTEE_SMC_EXCHANGE_CAPABILITIES:
> + case OPTEE_SMC_CALL_WITH_ARG:
> + case OPTEE_SMC_CALL_RETURN_FROM_RPC:
> + forward_call(regs);
> + return true;
> + default:
> + return false;
> + }
> +}
> +
> +static const struct tee_mediator_ops optee_ops =
> +{
> + .probe = optee_probe,
> + .domain_init = optee_domain_init,
> + .domain_destroy = optee_domain_destroy,
> + .relinquish_resources = optee_relinquish_resources,
> + .handle_call = optee_handle_call,
> +};
> +
> +REGISTER_TEE_MEDIATOR(optee, "OP-TEE", &optee_ops);
> +
> +/*
> + * Local variables:
> + * mode: C
> + * c-file-style: "BSD"
> + * c-basic-offset: 4
> + * indent-tabs-mode: nil
> + * End:
> + */
> diff --git a/xen/include/asm-arm/domain.h b/xen/include/asm-arm/domain.h
> index 0f15372098..2960a53e69 100644
> --- a/xen/include/asm-arm/domain.h
> +++ b/xen/include/asm-arm/domain.h
> @@ -98,6 +98,9 @@ struct arch_domain
> struct vpl011 vpl011;
> #endif
>
> +#ifdef CONFIG_TEE
> + void *tee;
> +#endif
> } __cacheline_aligned;
>
> struct arch_vcpu
>
Cheers,
--
Julien Grall
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 47+ messages in thread* Re: [PATCH v4 03/10] xen/arm: optee: add OP-TEE mediator skeleton
2019-03-15 15:24 ` Julien Grall
@ 2019-03-15 19:00 ` Volodymyr Babchuk
2019-03-15 20:18 ` Julien Grall
0 siblings, 1 reply; 47+ messages in thread
From: Volodymyr Babchuk @ 2019-03-15 19:00 UTC (permalink / raw)
To: Julien Grall
Cc: tee-dev, xen-devel, Stefano Stabellini, Volodymyr Babchuk,
Volodymyr Babchuk
Hello Julien,
Thank you for the review.
Julien Grall writes:
>> + */
>> +
>> +#include <xen/device_tree.h>
>> +#include <xen/sched.h>
>> +
>> +#include <asm/smccc.h>
>> +#include <asm/tee/tee.h>
>> +#include <asm/tee/optee_msg.h>
>> +#include <asm/tee/optee_smc.h>
>> +
>> +#define OPTEE_ENABLED ((void*)0x1)
>
> Please don't do that. Introduce a dummy structure instead and fill it
> up when needed.
>
It will be removed in the patch #5. But probably yes, I can
create empty optee_domain structure in this patch.
>> +
>> +/* Client ID 0 is reserved for hypervisor itself */
>
> s/for/for the/
>
>> +#define OPTEE_CLIENT_ID(domain) ((domain)->domain_id + 1)
>> +
>> +static bool optee_probe(void)
>> +{
>> + struct dt_device_node *node;
>> + struct arm_smccc_res resp;
>> +
>> + /* Check for entry in dtb */
>> + node = dt_find_compatible_node(NULL, NULL, "linaro,optee-tz");
>> + if ( !node )
>> + return false;
>> +
>> + /* Check UID */
>> + arm_smccc_smc(ARM_SMCCC_CALL_UID_FID(TRUSTED_OS_END), &resp);
>> +
>> + if ( (uint32_t)resp.a0 != OPTEE_MSG_UID_0 ||
>> + (uint32_t)resp.a1 != OPTEE_MSG_UID_1 ||
>> + (uint32_t)resp.a2 != OPTEE_MSG_UID_2 ||
>> + (uint32_t)resp.a3 != OPTEE_MSG_UID_3 )
>> + return false;
>> +
>> + return true;
>> +}
>> +
>> +static int optee_domain_init(struct domain *d)
>> +{
>> + struct arm_smccc_res resp;
>> +
>> + /*
>> + * Inform OP-TEE about a new guest.
>> + * This is a "Fast" call in terms of OP-TEE. This basically
>> + * means that it can't be preempted, because there is no
>> + * thread allocated for it in OP-TEE. It is close to atomic
>> + * context in linux kernel: E.g. no blocking calls can be issued.
>
> This does not really make sense to describe Linux here. Can't you just
> make the wording OS agnostic?
>
It was just as an example. But okay, I'll remove mention of Linux.
>> + * Also, interrupts are disabled.
>> + *
>> + * a7 should be 0, so we can't skip last 6 parameters of arm_smccc_smc()
>> + */
>> + arm_smccc_smc(OPTEE_SMC_VM_CREATED, OPTEE_CLIENT_ID(d), 0, 0, 0, 0, 0, 0,
>> + &resp);
>> + if ( resp.a0 != OPTEE_SMC_RETURN_OK )
>> + {
>> + gprintk(XENLOG_WARNING, "Unable to create OPTEE client: rc = 0x%X\n",
>
> gprintk will print the current vCPU and not the domain created. This
> is not very useful to know the current domain. So it would be better
> to use:
>
> printk(XENLOG_G_WARNING, "%pd: Unable to create OPTEE client: rc ...", d);
>
Good point, thank you.
>> + (uint32_t)resp.a0);
>> +
>> + return -ENODEV;
>> + }
>> +
>> + d->arch.tee = OPTEE_ENABLED;
>> +
>> + return 0;
>> +}
>> +
>> +static void forward_call(struct cpu_user_regs *regs)
>> +{
>> + struct arm_smccc_res resp;
>> +
>> + arm_smccc_smc(get_user_reg(regs, 0),
>> + get_user_reg(regs, 1),
>> + get_user_reg(regs, 2),
>> + get_user_reg(regs, 3),
>> + get_user_reg(regs, 4),
>> + get_user_reg(regs, 5),
>> + get_user_reg(regs, 6),
>> + OPTEE_CLIENT_ID(current->domain),
>> + &resp);
>> +
>> + set_user_reg(regs, 0, resp.a0);
>> + set_user_reg(regs, 1, resp.a1);
>> + set_user_reg(regs, 2, resp.a2);
>> + set_user_reg(regs, 3, resp.a3);
>> + set_user_reg(regs, 4, 0);
>> + set_user_reg(regs, 5, 0);
>> + set_user_reg(regs, 6, 0);
>> + set_user_reg(regs, 7, 0);
>> +}
>> +
>> +static int optee_relinquish_resources(struct domain *d)
>> +{
>> + return 0;
>> +}
>> +
>> +static void optee_domain_destroy(struct domain *d)
>> +{
>> + struct arm_smccc_res resp;
>> +
>> + if ( !d->arch.tee )
>> + return;
>> +
>> + /*
>> + * Inform OP-TEE that domain is shutting down. This is
>> + * also a fast SMC call, like OPTEE_SMC_VM_CREATED, so
>> + * it is also non-preemptible.
>> + * At this time all domain VCPUs should be stopped. OP-TEE
>> + * relies on this.
>> + *
>> + * a7 should be 0, so we can't skip last 6 parameters od arm_smccc_smc()
>
> NIT: s/od/of/
>
>> + */
>> + arm_smccc_smc(OPTEE_SMC_VM_DESTROYED, OPTEE_CLIENT_ID(d), 0, 0, 0, 0, 0, 0,
>> + &resp);
>
> Your split between domain_destroy and relinquish_resources looks
> wrong. If you relinquish resources before telling OP-TEE then you are
> at risk that OP-TEE will use those resources.
>
> Instead you should first tell OP-TEE the domain is shutting down, then
> release the resources.
Both this calls are supposed to be called after all guest's VCPUs are
stopped, so OP-TEE will be not able to use any resources allocated to
the domain, because OP-TEE is schedule by the Normal World. I assume,
this is true for any other TEE.
[...]
--
Best regards,Volodymyr Babchuk
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 47+ messages in thread* Re: [PATCH v4 03/10] xen/arm: optee: add OP-TEE mediator skeleton
2019-03-15 19:00 ` Volodymyr Babchuk
@ 2019-03-15 20:18 ` Julien Grall
0 siblings, 0 replies; 47+ messages in thread
From: Julien Grall @ 2019-03-15 20:18 UTC (permalink / raw)
To: Volodymyr Babchuk
Cc: tee-dev, xen-devel, Stefano Stabellini, Volodymyr Babchuk
Hi Volodymyr,
On 3/15/19 7:00 PM, Volodymyr Babchuk wrote:
>>> + */
>>> +
>>> +#include <xen/device_tree.h>
>>> +#include <xen/sched.h>
>>> +
>>> +#include <asm/smccc.h>
>>> +#include <asm/tee/tee.h>
>>> +#include <asm/tee/optee_msg.h>
>>> +#include <asm/tee/optee_smc.h>
>>> +
>>> +#define OPTEE_ENABLED ((void*)0x1)
>>
>> Please don't do that. Introduce a dummy structure instead and fill it
>> up when needed.
>>
>
> It will be removed in the patch #5. But probably yes, I can
> create empty optee_domain structure in this patch.
I know it will be removed in patch #5. This is exactly why I don't want
to see such things at the first place.
>
>>> +
>>> +/* Client ID 0 is reserved for hypervisor itself */
>>
>> s/for/for the/
>>
>>> +#define OPTEE_CLIENT_ID(domain) ((domain)->domain_id + 1)
>>> +
>>> +static bool optee_probe(void)
>>> +{
>>> + struct dt_device_node *node;
>>> + struct arm_smccc_res resp;
>>> +
>>> + /* Check for entry in dtb */
>>> + node = dt_find_compatible_node(NULL, NULL, "linaro,optee-tz");
>>> + if ( !node )
>>> + return false;
>>> +
>>> + /* Check UID */
>>> + arm_smccc_smc(ARM_SMCCC_CALL_UID_FID(TRUSTED_OS_END), &resp);
>>> +
>>> + if ( (uint32_t)resp.a0 != OPTEE_MSG_UID_0 ||
>>> + (uint32_t)resp.a1 != OPTEE_MSG_UID_1 ||
>>> + (uint32_t)resp.a2 != OPTEE_MSG_UID_2 ||
>>> + (uint32_t)resp.a3 != OPTEE_MSG_UID_3 )
>>> + return false;
>>> +
>>> + return true;
>>> +}
>>> +
>>> +static int optee_domain_init(struct domain *d)
>>> +{
>>> + struct arm_smccc_res resp;
>>> +
>>> + /*
>>> + * Inform OP-TEE about a new guest.
>>> + * This is a "Fast" call in terms of OP-TEE. This basically
>>> + * means that it can't be preempted, because there is no
>>> + * thread allocated for it in OP-TEE. It is close to atomic
>>> + * context in linux kernel: E.g. no blocking calls can be issued.
>>
>> This does not really make sense to describe Linux here. Can't you just
>> make the wording OS agnostic?
>>
> It was just as an example. But okay, I'll remove mention of Linux.
Xen developer may not know the Linux internals.
>
>
>>> + * Also, interrupts are disabled.
>>> + *
>>> + * a7 should be 0, so we can't skip last 6 parameters of arm_smccc_smc()
>>> + */
>>> + arm_smccc_smc(OPTEE_SMC_VM_CREATED, OPTEE_CLIENT_ID(d), 0, 0, 0, 0, 0, 0,
>>> + &resp);
>>> + if ( resp.a0 != OPTEE_SMC_RETURN_OK )
>>> + {
>>> + gprintk(XENLOG_WARNING, "Unable to create OPTEE client: rc = 0x%X\n",
>>
>> gprintk will print the current vCPU and not the domain created. This
>> is not very useful to know the current domain. So it would be better
>> to use:
>>
>> printk(XENLOG_G_WARNING, "%pd: Unable to create OPTEE client: rc ...", d);
>>
> Good point, thank you.
>
>>> + (uint32_t)resp.a0);
>>> +
>>> + return -ENODEV;
>>> + }
>>> +
>>> + d->arch.tee = OPTEE_ENABLED;
>>> +
>>> + return 0;
>>> +}
>>> +
>>> +static void forward_call(struct cpu_user_regs *regs)
>>> +{
>>> + struct arm_smccc_res resp;
>>> +
>>> + arm_smccc_smc(get_user_reg(regs, 0),
>>> + get_user_reg(regs, 1),
>>> + get_user_reg(regs, 2),
>>> + get_user_reg(regs, 3),
>>> + get_user_reg(regs, 4),
>>> + get_user_reg(regs, 5),
>>> + get_user_reg(regs, 6),
>>> + OPTEE_CLIENT_ID(current->domain),
>>> + &resp);
>>> +
>>> + set_user_reg(regs, 0, resp.a0);
>>> + set_user_reg(regs, 1, resp.a1);
>>> + set_user_reg(regs, 2, resp.a2);
>>> + set_user_reg(regs, 3, resp.a3);
>>> + set_user_reg(regs, 4, 0);
>>> + set_user_reg(regs, 5, 0);
>>> + set_user_reg(regs, 6, 0);
>>> + set_user_reg(regs, 7, 0);
>>> +}
>>> +
>>> +static int optee_relinquish_resources(struct domain *d)
>>> +{
>>> + return 0;
>>> +}
>>> +
>>> +static void optee_domain_destroy(struct domain *d)
>>> +{
>>> + struct arm_smccc_res resp;
>>> +
>>> + if ( !d->arch.tee )
>>> + return;
>>> +
>>> + /*
>>> + * Inform OP-TEE that domain is shutting down. This is
>>> + * also a fast SMC call, like OPTEE_SMC_VM_CREATED, so
>>> + * it is also non-preemptible.
>>> + * At this time all domain VCPUs should be stopped. OP-TEE
>>> + * relies on this.
>>> + *
>>> + * a7 should be 0, so we can't skip last 6 parameters od arm_smccc_smc()
>>
>> NIT: s/od/of/
>>
>>> + */
>>> + arm_smccc_smc(OPTEE_SMC_VM_DESTROYED, OPTEE_CLIENT_ID(d), 0, 0, 0, 0, 0, 0,
>>> + &resp);
>>
>> Your split between domain_destroy and relinquish_resources looks
>> wrong. If you relinquish resources before telling OP-TEE then you are
>> at risk that OP-TEE will use those resources.
>>
>> Instead you should first tell OP-TEE the domain is shutting down, then
>> release the resources.
>
> Both this calls are supposed to be called after all guest's VCPUs are
> stopped, so OP-TEE will be not able to use any resources allocated to
> the domain, because OP-TEE is schedule by the Normal World. I assume,
> this is true for any other TEE.
I don't know enough OP-TEE in general to be able to say how it works.
However, you can only be sure the buffers will not be used after
VM_DESTROYED is called. Before, this may be the case, but this a pretty
fragile assumption. If it does not hold, then you may end up to corrupt
the hypervisor (or another guest).
It is not a position I would like to be in. The more that it should not
require too much work to re-order the code.
Cheers,
--
Julien Grall
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 47+ messages in thread
* Re: [PATCH v4 03/10] xen/arm: optee: add OP-TEE mediator skeleton
2019-03-07 21:04 ` [PATCH v4 03/10] xen/arm: optee: add OP-TEE mediator skeleton Volodymyr Babchuk
2019-03-15 15:24 ` Julien Grall
@ 2019-03-15 15:47 ` Julien Grall
1 sibling, 0 replies; 47+ messages in thread
From: Julien Grall @ 2019-03-15 15:47 UTC (permalink / raw)
To: Volodymyr Babchuk, xen-devel, tee-dev
Cc: Volodymyr Babchuk, Stefano Stabellini
Hi Volodymyr,
On 07/03/2019 21:04, Volodymyr Babchuk wrote:
> +static void forward_call(struct cpu_user_regs *regs)
> +{
> + struct arm_smccc_res resp;
> +
> + arm_smccc_smc(get_user_reg(regs, 0),
> + get_user_reg(regs, 1),
> + get_user_reg(regs, 2),
> + get_user_reg(regs, 3),
> + get_user_reg(regs, 4),
> + get_user_reg(regs, 5),
> + get_user_reg(regs, 6),
> + OPTEE_CLIENT_ID(current->domain),
> + &resp);
> +
> + set_user_reg(regs, 0, resp.a0);
> + set_user_reg(regs, 1, resp.a1);
> + set_user_reg(regs, 2, resp.a2);
> + set_user_reg(regs, 3, resp.a3);
> + set_user_reg(regs, 4, 0);
> + set_user_reg(regs, 5, 0);
> + set_user_reg(regs, 6, 0);
> + set_user_reg(regs, 7, 0);
Related to what I pointed on patch #4, this does not look right to me.
Cheers,
--
Julien Grall
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 47+ messages in thread
* [PATCH v4 05/10] xen/arm: optee: add std call handling
2019-03-07 21:04 [PATCH v4 00/10] TEE mediator (and OP-TEE) support in XEN Volodymyr Babchuk
` (3 preceding siblings ...)
2019-03-07 21:04 ` [PATCH v4 03/10] xen/arm: optee: add OP-TEE mediator skeleton Volodymyr Babchuk
@ 2019-03-07 21:04 ` Volodymyr Babchuk
2019-03-18 13:50 ` Julien Grall
2019-03-07 21:04 ` [PATCH v4 07/10] xen/arm: optee: add support for arbitrary shared memory Volodymyr Babchuk
` (4 subsequent siblings)
9 siblings, 1 reply; 47+ messages in thread
From: Volodymyr Babchuk @ 2019-03-07 21:04 UTC (permalink / raw)
To: xen-devel, tee-dev
Cc: Volodymyr Babchuk, Julien Grall, Stefano Stabellini, Volodymyr Babchuk
From: Volodymyr Babchuk <vlad.babchuk@gmail.com>
The main way to communicate with OP-TEE is to issue standard SMCCC
call. "Standard" is a SMCCC term and it means that call can be
interrupted and OP-TEE can return control to NW before completing
the call.
In contrast with fast calls, where arguments and return values
are passed in registers, standard calls use shared memory. Register
pair a1,a2 holds 64-bit PA of command buffer, where all arguments
are stored and which is used to return data. OP-TEE internally
copies contents of this buffer into own secure memory before accessing
and validating any data in command buffer. This is done to make sure
that NW will not change contents of the validated parameters.
Mediator needs to do the same for number of reasons:
1. To make sure that guest will not change data after validation.
2. To translate IPAs to PAs in the command buffer (this is not done
in this patch).
3. To hide translated address from guest, so it will not be able
to do IPA->PA translation by misusing mediator.
During standard call OP-TEE can issue multiple "RPC returns", asking
NW to do some work for OP-TEE. NW then issues special call
OPTEE_SMC_CALL_RETURN_FROM_RPC to resume handling of the original call.
Thus, mediator needs to maintain context for original standard call
during multiple SMCCC calls.
Standard call is considered complete, when returned value is
not a RPC request.
Signed-off-by: Volodymyr Babchuk <vlad.babchuk@gmail.com>
---
All the patches to optee.c should be merged together. They were
split to ease up review. But they depend heavily on each other.
Changes from v3:
- Added ability to read number of threads from OP-TEE, if it
supports this feature
- Pages are allocated from domheap, instead of xenheap
- Added comments for complex code
Changes from v2:
- renamed struct domain_ctx to struct optee_domain
- fixed coding style
- Now I use access_guest_memory_by_ipa() instead of mappings
to read command buffer
- Added tracking for in flight calls, so guest can't resume
the same call from two CPUs simultaniously
---
xen/arch/arm/tee/optee.c | 447 ++++++++++++++++++++++++++++++++++++++-
1 file changed, 440 insertions(+), 7 deletions(-)
diff --git a/xen/arch/arm/tee/optee.c b/xen/arch/arm/tee/optee.c
index 61554ea191..66517ebf6f 100644
--- a/xen/arch/arm/tee/optee.c
+++ b/xen/arch/arm/tee/optee.c
@@ -12,6 +12,10 @@
*/
#include <xen/device_tree.h>
+#include <xen/domain_page.h>
+#include <xen/err.h>
+#include <xen/guest_access.h>
+#include <xen/mm.h>
#include <xen/sched.h>
#include <asm/smccc.h>
@@ -19,16 +23,46 @@
#include <asm/tee/optee_msg.h>
#include <asm/tee/optee_smc.h>
-#define OPTEE_ENABLED ((void*)0x1)
-
/* Client ID 0 is reserved for hypervisor itself */
#define OPTEE_CLIENT_ID(domain) ((domain)->domain_id + 1)
+/*
+ * Default maximal number concurrent threads that OP-TEE supports.
+ * This limits number of standard calls that guest can have.
+ */
+#define DEF_MAX_OPTEE_THREADS 16
+
#define OPTEE_KNOWN_NSEC_CAPS OPTEE_SMC_NSEC_CAP_UNIPROCESSOR
#define OPTEE_KNOWN_SEC_CAPS (OPTEE_SMC_SEC_CAP_HAVE_RESERVED_SHM | \
OPTEE_SMC_SEC_CAP_UNREGISTERED_SHM | \
OPTEE_SMC_SEC_CAP_DYNAMIC_SHM)
+static unsigned int max_optee_threads = DEF_MAX_OPTEE_THREADS;
+
+/*
+ * Call context. OP-TEE can issue multiple RPC returns during one call.
+ * We need to preserve context during them.
+ */
+struct optee_std_call {
+ struct list_head list;
+ /* Page where shadowed copy of call arguments is stored */
+ struct page_info *xen_arg_pg;
+ /* Above page mapped into XEN */
+ struct optee_msg_arg *xen_arg;
+ /* Address of original call arguments */
+ paddr_t guest_arg_ipa;
+ int optee_thread_id;
+ int rpc_op;
+ bool in_flight;
+};
+
+/* Domain context */
+struct optee_domain {
+ struct list_head call_list;
+ atomic_t call_count;
+ spinlock_t lock;
+};
+
static bool optee_probe(void)
{
struct dt_device_node *node;
@@ -48,12 +82,25 @@ static bool optee_probe(void)
(uint32_t)resp.a3 != OPTEE_MSG_UID_3 )
return false;
+ /* Read number of threads */
+ arm_smccc_smc(OPTEE_SMC_GET_CONFIG, OPTEE_SMC_CONFIG_NUM_THREADS, &resp);
+ if ( resp.a0 == OPTEE_SMC_RETURN_OK )
+ {
+ max_optee_threads = resp.a1;
+ printk(XENLOG_DEBUG "OP-TEE supports %u threads.\n", max_optee_threads);
+ }
+
return true;
}
static int optee_domain_init(struct domain *d)
{
struct arm_smccc_res resp;
+ struct optee_domain *ctx;
+
+ ctx = xzalloc(struct optee_domain);
+ if ( !ctx )
+ return -ENOMEM;
/*
* Inform OP-TEE about a new guest.
@@ -72,10 +119,16 @@ static int optee_domain_init(struct domain *d)
gprintk(XENLOG_WARNING, "Unable to create OPTEE client: rc = 0x%X\n",
(uint32_t)resp.a0);
+ xfree(ctx);
+
return -ENODEV;
}
- d->arch.tee = OPTEE_ENABLED;
+ INIT_LIST_HEAD(&ctx->call_list);
+ atomic_set(&ctx->call_count, 0);
+ spin_lock_init(&ctx->lock);
+
+ d->arch.tee = ctx;
return 0;
}
@@ -116,16 +169,138 @@ static void set_return(struct cpu_user_regs *regs, uint32_t ret)
set_user_reg(regs, 7, 0);
}
+static uint64_t regpair_to_uint64(struct cpu_user_regs *regs, unsigned int idx)
+{
+ return (uint64_t)get_user_reg(regs, idx) << 32 |
+ (uint32_t)get_user_reg(regs, idx + 1);
+}
+
+static void uint64_to_regpair(struct cpu_user_regs *regs, unsigned int idx,
+ uint64_t val)
+{
+ set_user_reg(regs, idx, val >> 32);
+ set_user_reg(regs, idx + 1, (uint32_t)val);
+}
+
+static struct optee_std_call *allocate_std_call(struct optee_domain *ctx)
+{
+ struct optee_std_call *call;
+ int count;
+
+ /* Make sure that guest does not execute more than max_optee_threads */
+ count = atomic_add_unless(&ctx->call_count, 1, max_optee_threads);
+ if ( count == max_optee_threads )
+ return ERR_PTR(-ENOSPC);
+
+ call = xzalloc(struct optee_std_call);
+ if ( !call )
+ {
+ atomic_dec(&ctx->call_count);
+ return ERR_PTR(-ENOMEM);
+ }
+
+ call->optee_thread_id = -1;
+ call->in_flight = true;
+
+ spin_lock(&ctx->lock);
+ list_add_tail(&call->list, &ctx->call_list);
+ spin_unlock(&ctx->lock);
+
+ return call;
+}
+
+static void free_std_call(struct optee_domain *ctx,
+ struct optee_std_call *call)
+{
+ atomic_dec(&ctx->call_count);
+
+ spin_lock(&ctx->lock);
+ list_del(&call->list);
+ spin_unlock(&ctx->lock);
+
+ ASSERT(!call->in_flight);
+ ASSERT(!call->xen_arg);
+
+ if ( call->xen_arg_pg )
+ free_domheap_page(call->xen_arg_pg);
+
+ xfree(call);
+}
+
+static void map_xen_arg(struct optee_std_call *call)
+{
+ ASSERT(!call->xen_arg);
+
+ call->xen_arg = __map_domain_page(call->xen_arg_pg);
+}
+
+static void unmap_xen_arg(struct optee_std_call *call)
+{
+ if ( !call->xen_arg )
+ return;
+
+ unmap_domain_page(call->xen_arg);
+ call->xen_arg = NULL;
+}
+
+static struct optee_std_call *get_std_call(struct optee_domain *ctx,
+ int thread_id)
+{
+ struct optee_std_call *call;
+
+ spin_lock(&ctx->lock);
+ list_for_each_entry( call, &ctx->call_list, list )
+ {
+ if ( call->optee_thread_id == thread_id )
+ {
+ if ( call->in_flight )
+ {
+ gdprintk(XENLOG_WARNING, "Guest tries to execute call which is already in flight\n");
+ goto out;
+ }
+ call->in_flight = true;
+ map_xen_arg(call);
+ spin_unlock(&ctx->lock);
+
+ return call;
+ }
+ }
+
+out:
+ spin_unlock(&ctx->lock);
+
+ return NULL;
+}
+
+static void put_std_call(struct optee_domain *ctx, struct optee_std_call *call)
+{
+ spin_lock(&ctx->lock);
+ ASSERT(call->in_flight);
+ unmap_xen_arg(call);
+ call->in_flight = false;
+ spin_unlock(&ctx->lock);
+}
+
static int optee_relinquish_resources(struct domain *d)
{
+ struct optee_std_call *call, *call_tmp;
+ struct optee_domain *ctx = d->arch.tee;
+
+ if ( !ctx )
+ return 0;
+
+ list_for_each_entry_safe( call, call_tmp, &ctx->call_list, list )
+ free_std_call(ctx, call);
+
return 0;
}
static void optee_domain_destroy(struct domain *d)
{
struct arm_smccc_res resp;
+ struct optee_domain *ctx = d->arch.tee;
- if ( !d->arch.tee )
+ if ( !ctx )
return;
/*
@@ -139,6 +314,258 @@ static void optee_domain_destroy(struct domain *d)
*/
arm_smccc_smc(OPTEE_SMC_VM_DESTROYED, OPTEE_CLIENT_ID(d), 0, 0, 0, 0, 0, 0,
&resp);
+
+ ASSERT(!spin_is_locked(&ctx->lock));
+ ASSERT(!atomic_read(&ctx->call_count));
+
+ XFREE(d->arch.tee);
+}
+
+/*
+ * Copy command buffer into domheap memory to:
+ * 1) Hide translated addresses from guest
+ * 2) Make sure that guest wouldn't change data in command buffer during call
+ */
+static bool copy_std_request(struct cpu_user_regs *regs,
+ struct optee_std_call *call)
+{
+ paddr_t xen_addr;
+
+ call->guest_arg_ipa = regpair_to_uint64(regs, 1);
+
+ /*
+ * Command buffer should start at page boundary.
+ * This is OP-TEE ABI requirement.
+ */
+ if ( call->guest_arg_ipa & (OPTEE_MSG_NONCONTIG_PAGE_SIZE - 1) )
+ {
+ set_return(regs, OPTEE_SMC_RETURN_EBADADDR);
+ return false;
+ }
+
+ BUILD_BUG_ON(OPTEE_MSG_NONCONTIG_PAGE_SIZE > PAGE_SIZE);
+
+ call->xen_arg_pg = alloc_domheap_page(current->domain, 0);
+ if ( !call->xen_arg_pg )
+ {
+ set_return(regs, OPTEE_SMC_RETURN_ENOMEM);
+ return false;
+ }
+
+ map_xen_arg(call);
+
+ if ( access_guest_memory_by_ipa(current->domain, call->guest_arg_ipa,
+ call->xen_arg,
+ OPTEE_MSG_NONCONTIG_PAGE_SIZE, false) )
+ {
+ set_return(regs, OPTEE_SMC_RETURN_EBADADDR);
+ return false;
+ }
+
+ /* Send to OP-TEE maddr of the shadow buffer */
+ xen_addr = page_to_maddr(call->xen_arg_pg);
+ uint64_to_regpair(regs, 1, xen_addr);
+
+ return true;
+}
+
+/*
+ * Copy result of completed request back to guest's buffer.
+ * We are copying only values that subjected to change to minimize
+ * possible information leak.
+ */
+static void copy_std_request_back(struct optee_domain *ctx,
+ struct cpu_user_regs *regs,
+ struct optee_std_call *call)
+{
+ struct optee_msg_arg *guest_arg;
+ struct page_info *page;
+ p2m_type_t t;
+ unsigned int i;
+ uint32_t attr;
+
+ page = get_page_from_gfn(current->domain,
+ gfn_x(gaddr_to_gfn(call->guest_arg_ipa)),
+ &t, P2M_ALLOC);
+ if ( !page || t != p2m_ram_rw )
+ {
+ if ( page )
+ put_page(page);
+
+ /*
+ * Guest did something to own command buffer during the call.
+ * Now we even can't write error code to the command
+ * buffer. Let's try to return generic error via
+ * register. Problem is that OP-TEE does not know that guest
+ * didn't received valid response. But at least guest will
+ * know that something bad happened.
+ */
+ set_return(regs, OPTEE_SMC_RETURN_EBADADDR);
+
+ return;
+ }
+
+ guest_arg = __map_domain_page(page);
+
+ guest_arg->ret = call->xen_arg->ret;
+ guest_arg->ret_origin = call->xen_arg->ret_origin;
+ guest_arg->session = call->xen_arg->session;
+
+ for ( i = 0; i < call->xen_arg->num_params; i++ )
+ {
+ attr = call->xen_arg->params[i].attr;
+
+ switch ( attr & OPTEE_MSG_ATTR_TYPE_MASK )
+ {
+ case OPTEE_MSG_ATTR_TYPE_TMEM_OUTPUT:
+ case OPTEE_MSG_ATTR_TYPE_TMEM_INOUT:
+ guest_arg->params[i].u.tmem.size =
+ call->xen_arg->params[i].u.tmem.size;
+ continue;
+ case OPTEE_MSG_ATTR_TYPE_RMEM_OUTPUT:
+ case OPTEE_MSG_ATTR_TYPE_RMEM_INOUT:
+ guest_arg->params[i].u.rmem.size =
+ call->xen_arg->params[i].u.rmem.size;
+ continue;
+ case OPTEE_MSG_ATTR_TYPE_VALUE_OUTPUT:
+ case OPTEE_MSG_ATTR_TYPE_VALUE_INOUT:
+ guest_arg->params[i].u.value.a =
+ call->xen_arg->params[i].u.value.a;
+ guest_arg->params[i].u.value.b =
+ call->xen_arg->params[i].u.value.b;
+ guest_arg->params[i].u.value.c =
+ call->xen_arg->params[i].u.value.c;
+ continue;
+ case OPTEE_MSG_ATTR_TYPE_NONE:
+ case OPTEE_MSG_ATTR_TYPE_RMEM_INPUT:
+ case OPTEE_MSG_ATTR_TYPE_TMEM_INPUT:
+ continue;
+ }
+ }
+
+ unmap_domain_page(guest_arg);
+ put_page(page);
+}
+
+/* Handle RPC return from OP-TEE */
+static void handle_rpc_return(struct cpu_user_regs *regs,
+ struct optee_std_call *call)
+{
+ call->optee_thread_id = get_user_reg(regs, 3);
+ call->rpc_op = OPTEE_SMC_RETURN_GET_RPC_FUNC(get_user_reg(regs, 0));
+}
+
+/*
+ * (Re)start standard call. This function will be called in two cases:
+ * 1. Guest initiates new standard call
+ * 2. Guest finished RPC handling and asks OP-TEE to resume the call
+ *
+ * In any case OP-TEE can either complete call or issue another RPC.
+ * If this is RPC - we need to store call context and return back to guest.
+ * If call is complete - we need to return results with copy_std_request_back()
+ * and then we will destroy the call context as it is not needed anymore.
+ */
+static void execute_std_call(struct optee_domain *ctx,
+ struct cpu_user_regs *regs,
+ struct optee_std_call *call)
+{
+ register_t optee_ret;
+
+ forward_call(regs);
+
+ optee_ret = get_user_reg(regs, 0);
+ if ( OPTEE_SMC_RETURN_IS_RPC(optee_ret) )
+ {
+ handle_rpc_return(regs, call);
+ put_std_call(ctx, call);
+
+ return;
+ }
+
+ copy_std_request_back(ctx, regs, call);
+
+ put_std_call(ctx, call);
+ free_std_call(ctx, call);
+}
+
+static void handle_std_call(struct optee_domain *ctx,
+ struct cpu_user_regs *regs)
+{
+ struct optee_std_call *call = allocate_std_call(ctx);
+
+ if ( IS_ERR(call) )
+ {
+ if ( PTR_ERR(call) == -ENOMEM )
+ set_return(regs, OPTEE_SMC_RETURN_ENOMEM);
+ else
+ set_return(regs, OPTEE_SMC_RETURN_ETHREAD_LIMIT);
+
+ return;
+ }
+
+ if ( !copy_std_request(regs, call) )
+ goto err;
+
+ switch ( call->xen_arg->cmd )
+ {
+ case OPTEE_MSG_CMD_OPEN_SESSION:
+ case OPTEE_MSG_CMD_CLOSE_SESSION:
+ case OPTEE_MSG_CMD_INVOKE_COMMAND:
+ case OPTEE_MSG_CMD_CANCEL:
+ case OPTEE_MSG_CMD_REGISTER_SHM:
+ case OPTEE_MSG_CMD_UNREGISTER_SHM:
+ execute_std_call(ctx, regs, call);
+ return;
+ default:
+ set_return(regs, OPTEE_SMC_RETURN_EBADCMD);
+ break;
+ }
+
+err:
+ put_std_call(ctx, call);
+ free_std_call(ctx, call);
+
+ return;
+}
+
+static void handle_rpc(struct optee_domain *ctx, struct cpu_user_regs *regs)
+{
+ struct optee_std_call *call;
+ int optee_thread_id = get_user_reg(regs, 3);
+
+ call = get_std_call(ctx, optee_thread_id);
+
+ if ( !call )
+ {
+ set_return(regs, OPTEE_SMC_RETURN_ERESUME);
+ return;
+ }
+
+ /*
+ * This is to prevent race between new call with the same thread id.
+ * OP-TEE can reuse thread id right after it finished handling the call,
+ * before XEN had chance to free old call context.
+ */
+ call->optee_thread_id = -1;
+
+ switch ( call->rpc_op )
+ {
+ case OPTEE_SMC_RPC_FUNC_ALLOC:
+ /* TODO: Add handling */
+ break;
+ case OPTEE_SMC_RPC_FUNC_FREE:
+ /* TODO: Add handling */
+ break;
+ case OPTEE_SMC_RPC_FUNC_FOREIGN_INTR:
+ break;
+ case OPTEE_SMC_RPC_FUNC_CMD:
+ /* TODO: Add handling */
+ break;
+ }
+
+ execute_std_call(ctx, regs, call);
+
+ return;
}
static void handle_exchange_capabilities(struct cpu_user_regs *regs)
@@ -174,7 +601,9 @@ static void handle_exchange_capabilities(struct cpu_user_regs *regs)
static bool optee_handle_call(struct cpu_user_regs *regs)
{
- if ( !current->domain->arch.tee )
+ struct optee_domain *ctx = current->domain->arch.tee;
+
+ if ( !ctx )
return false;
switch ( get_user_reg(regs, 0) )
@@ -186,8 +615,6 @@ static bool optee_handle_call(struct cpu_user_regs *regs)
case OPTEE_SMC_FUNCID_GET_OS_REVISION:
case OPTEE_SMC_ENABLE_SHM_CACHE:
case OPTEE_SMC_DISABLE_SHM_CACHE:
- case OPTEE_SMC_CALL_WITH_ARG:
- case OPTEE_SMC_CALL_RETURN_FROM_RPC:
forward_call(regs);
return true;
case OPTEE_SMC_GET_SHM_CONFIG:
@@ -197,6 +624,12 @@ static bool optee_handle_call(struct cpu_user_regs *regs)
case OPTEE_SMC_EXCHANGE_CAPABILITIES:
handle_exchange_capabilities(regs);
return true;
+ case OPTEE_SMC_CALL_WITH_ARG:
+ handle_std_call(ctx, regs);
+ return true;
+ case OPTEE_SMC_CALL_RETURN_FROM_RPC:
+ handle_rpc(ctx, regs);
+ return true;
default:
return false;
}
--
2.21.0
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply related [flat|nested] 47+ messages in thread* Re: [PATCH v4 05/10] xen/arm: optee: add std call handling
2019-03-07 21:04 ` [PATCH v4 05/10] xen/arm: optee: add std call handling Volodymyr Babchuk
@ 2019-03-18 13:50 ` Julien Grall
2019-03-20 16:14 ` Volodymyr Babchuk
0 siblings, 1 reply; 47+ messages in thread
From: Julien Grall @ 2019-03-18 13:50 UTC (permalink / raw)
To: Volodymyr Babchuk, xen-devel, tee-dev
Cc: Volodymyr Babchuk, Stefano Stabellini
Hi Volodymyr,
Only few NITs in this patch. See below:
On 07/03/2019 21:04, Volodymyr Babchuk wrote:
> +/*
> + * Default maximal number concurrent threads that OP-TEE supports.
> + * This limits number of standard calls that guest can have.
> + */
> +#define DEF_MAX_OPTEE_THREADS 16
> +
> #define OPTEE_KNOWN_NSEC_CAPS OPTEE_SMC_NSEC_CAP_UNIPROCESSOR
> #define OPTEE_KNOWN_SEC_CAPS (OPTEE_SMC_SEC_CAP_HAVE_RESERVED_SHM | \
> OPTEE_SMC_SEC_CAP_UNREGISTERED_SHM | \
> OPTEE_SMC_SEC_CAP_DYNAMIC_SHM)
>
> +static unsigned int max_optee_threads = DEF_MAX_OPTEE_THREADS;
NIT: This could be __read_mostly.
> +
> +/*
> + * Call context. OP-TEE can issue multiple RPC returns during one call.
> + * We need to preserve context during them.
> + */
> +struct optee_std_call {
> + struct list_head list;
> + /* Page where shadowed copy of call arguments is stored */
> + struct page_info *xen_arg_pg;
> + /* Above page mapped into XEN */
> + struct optee_msg_arg *xen_arg;
> + /* Address of original call arguments */
> + paddr_t guest_arg_ipa;
> + int optee_thread_id;
> + int rpc_op;
> + bool in_flight;
> +};
> +
> +/* Domain context */
> +struct optee_domain {
> + struct list_head call_list;
> + atomic_t call_count;
> + spinlock_t lock;
> +};
> +
> static bool optee_probe(void)
> {
> struct dt_device_node *node;
> @@ -48,12 +82,25 @@ static bool optee_probe(void)
> (uint32_t)resp.a3 != OPTEE_MSG_UID_3 )
> return false;
>
> + /* Read number of threads */
> + arm_smccc_smc(OPTEE_SMC_GET_CONFIG, OPTEE_SMC_CONFIG_NUM_THREADS, &resp);
> + if ( resp.a0 == OPTEE_SMC_RETURN_OK )
Out of interest, when was this call added?
> + {
> + max_optee_threads = resp.a1;
> + printk(XENLOG_DEBUG "OP-TEE supports %u threads.\n", max_optee_threads);
extra NIT: I would use XENLOG_INFO rather than XENLOG_DEBUG. This is a useful
information to have in bug report.
Regarding the message, what matters is the number of threads for guest. So I
would rework it to make it more meaning full for a user that does not know the
internal.
You might also want to move the message out of the if. So you have the message
even when OP-TEE does not support the SMC.
[...]
> +static struct optee_std_call *get_std_call(struct optee_domain *ctx,
> + int thread_id)
> +{
> + struct optee_std_call *call;
> +
> + spin_lock(&ctx->lock);
> + list_for_each_entry( call, &ctx->call_list, list )
> + {
> + if ( call->optee_thread_id == thread_id )
> + {
> + if ( call->in_flight )
> + {
> + gdprintk(XENLOG_WARNING, "Guest tries to execute call which is already in flight\n");
NIT: Missing full stop.
Also, the line is over 80 characters. While we don't want to split in the middle
of the message (so ack/grep can be used easily), you will want to split the line
after the comma.
> + goto out;
> + }
> + call->in_flight = true;
> + map_xen_arg(call);
NIT: Does this need to be done with the lock taken?
> + spin_unlock(&ctx->lock);
> +
> + return call;
> + }
> + }
> +
> +out:
> + spin_unlock(&ctx->lock);
> +
> + return NULL;
> +}
> +
> +static void put_std_call(struct optee_domain *ctx, struct optee_std_call *call)
> +{
> + spin_lock(&ctx->lock);
> + ASSERT(call->in_flight);
> + unmap_xen_arg(call);
Same question for the unmap.
> + call->in_flight = false;
> + spin_unlock(&ctx->lock);
> +}
Cheers,
--
Julien Grall
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 47+ messages in thread* Re: [PATCH v4 05/10] xen/arm: optee: add std call handling
2019-03-18 13:50 ` Julien Grall
@ 2019-03-20 16:14 ` Volodymyr Babchuk
2019-03-20 16:48 ` Julien Grall
0 siblings, 1 reply; 47+ messages in thread
From: Volodymyr Babchuk @ 2019-03-20 16:14 UTC (permalink / raw)
To: Julien Grall
Cc: tee-dev, xen-devel, Stefano Stabellini, Volodymyr Babchuk,
Volodymyr Babchuk
Hi Julien,
Julien Grall writes:
[...]
>> struct dt_device_node *node;
>> @@ -48,12 +82,25 @@ static bool optee_probe(void)
>> (uint32_t)resp.a3 != OPTEE_MSG_UID_3 )
>> return false;
>> + /* Read number of threads */
>> + arm_smccc_smc(OPTEE_SMC_GET_CONFIG, OPTEE_SMC_CONFIG_NUM_THREADS, &resp);
>> + if ( resp.a0 == OPTEE_SMC_RETURN_OK )
>
> Out of interest, when was this call added?
It is on review right now. We have achieved agreement on that this call
is needed. I believe this will be merged into OP-TEE before I'll send v5
of this series.
>> + {
>> + max_optee_threads = resp.a1;
>> + printk(XENLOG_DEBUG "OP-TEE supports %u threads.\n", max_optee_threads);
>
> extra NIT: I would use XENLOG_INFO rather than XENLOG_DEBUG. This is a
> useful information to have in bug report.
>
> Regarding the message, what matters is the number of threads for
> guest. So I would rework it to make it more meaning full for a user
> that does not know the internal.
>
> You might also want to move the message out of the if. So you have the
> message even when OP-TEE does not support the SMC.
In that other case I don't know how much treads OP-TEE really
supports... I think, I will need to rephrase that message. Or, better,
I'll add another message like "Suggesting that OP-TEE supports %d
threads per guest".
> [...]
>
>> +static struct optee_std_call *get_std_call(struct optee_domain *ctx,
>> + int thread_id)
>> +{
>> + struct optee_std_call *call;
>> +
>> + spin_lock(&ctx->lock);
>> + list_for_each_entry( call, &ctx->call_list, list )
>> + {
>> + if ( call->optee_thread_id == thread_id )
>> + {
>> + if ( call->in_flight )
>> + {
>> + gdprintk(XENLOG_WARNING, "Guest tries to execute call which is already in flight\n");
>
> NIT: Missing full stop.
>
> Also, the line is over 80 characters. While we don't want to split in
> the middle of the message (so ack/grep can be used easily), you will
> want to split the line after the comma.
>
>> + goto out;
>> + }
>> + call->in_flight = true;
>> + map_xen_arg(call);
>
> NIT: Does this need to be done with the lock taken?
unmap_xen_arg() is also called while holding the lock. Otherwise there
can be race with a other CPU.
>> + spin_unlock(&ctx->lock);
>> +
>> + return call;
>> + }
>> + }
>> +
>> +out:
>> + spin_unlock(&ctx->lock);
>> +
>> + return NULL;
>> +}
>> +
>> +static void put_std_call(struct optee_domain *ctx, struct optee_std_call *call)
>> +{
>> + spin_lock(&ctx->lock);
>> + ASSERT(call->in_flight);
>> + unmap_xen_arg(call);
>
> Same question for the unmap.
Yeah, in normal circumstances guest should not try to resume call on
another vCPU, because we didn't returned from the original call on
current vCPU. But what if gust will try to do this? There are chances,
that current CPU will unmap buffer that was mapped by other CPU an
instance ago.
>> + call->in_flight = false;
>> + spin_unlock(&ctx->lock);
>> +}
>
> Cheers,
--
Best regards,Volodymyr Babchuk
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 47+ messages in thread* Re: [PATCH v4 05/10] xen/arm: optee: add std call handling
2019-03-20 16:14 ` Volodymyr Babchuk
@ 2019-03-20 16:48 ` Julien Grall
2019-03-20 17:42 ` Volodymyr Babchuk
0 siblings, 1 reply; 47+ messages in thread
From: Julien Grall @ 2019-03-20 16:48 UTC (permalink / raw)
To: Volodymyr Babchuk
Cc: tee-dev, xen-devel, Stefano Stabellini, Volodymyr Babchuk
On 20/03/2019 16:14, Volodymyr Babchuk wrote:
>
> Hi Julien,
Hi Volodymyr,
> Julien Grall writes:
>
> [...]
>>> struct dt_device_node *node;
>>> @@ -48,12 +82,25 @@ static bool optee_probe(void)
>>> (uint32_t)resp.a3 != OPTEE_MSG_UID_3 )
>>> return false;
>>> + /* Read number of threads */
>>> + arm_smccc_smc(OPTEE_SMC_GET_CONFIG, OPTEE_SMC_CONFIG_NUM_THREADS, &resp);
>>> + if ( resp.a0 == OPTEE_SMC_RETURN_OK )
>>
>> Out of interest, when was this call added?
>
> It is on review right now. We have achieved agreement on that this call
> is needed. I believe this will be merged into OP-TEE before I'll send v5
> of this series.
Please mention it after --- so we don't forget to check the state of the new
call before merging to Xen.
>
>>> + {
>>> + max_optee_threads = resp.a1;
>>> + printk(XENLOG_DEBUG "OP-TEE supports %u threads.\n", max_optee_threads);
>>
>> extra NIT: I would use XENLOG_INFO rather than XENLOG_DEBUG. This is a
>> useful information to have in bug report.
>>
>> Regarding the message, what matters is the number of threads for
>> guest. So I would rework it to make it more meaning full for a user
>> that does not know the internal.
>>
>> You might also want to move the message out of the if. So you have the
>> message even when OP-TEE does not support the SMC.
> In that other case I don't know how much treads OP-TEE really
> supports... I think, I will need to rephrase that message. Or, better,
> I'll add another message like "Suggesting that OP-TEE supports %d
> threads per guest".
Was there any OP-TEE released containing your Virtualization patches?
Depending on the answer, I would consider to mandate
OPTEE_SMC_CONFIG_NUM_THREADS so we don't have to deal with a default value in Xen.
Also, should we expose this call to the guest as well?
>
>> [...]
>>
>>> +static struct optee_std_call *get_std_call(struct optee_domain *ctx,
>>> + int thread_id)
>>> +{
>>> + struct optee_std_call *call;
>>> +
>>> + spin_lock(&ctx->lock);
>>> + list_for_each_entry( call, &ctx->call_list, list )
>>> + {
>>> + if ( call->optee_thread_id == thread_id )
>>> + {
>>> + if ( call->in_flight )
>>> + {
>>> + gdprintk(XENLOG_WARNING, "Guest tries to execute call which is already in flight\n");
>>
>> NIT: Missing full stop.
>>
>> Also, the line is over 80 characters. While we don't want to split in
>> the middle of the message (so ack/grep can be used easily), you will
>> want to split the line after the comma.
>>
>>> + goto out;
>>> + }
>>> + call->in_flight = true;
>>> + map_xen_arg(call);
>>
>> NIT: Does this need to be done with the lock taken?
> unmap_xen_arg() is also called while holding the lock. Otherwise there
> can be race with a other CPU.
[...]
>
>>> + spin_unlock(&ctx->lock);
>>> +
>>> + return call;
>>> + }
>>> + }
>>> +
>>> +out:
>>> + spin_unlock(&ctx->lock);
>>> +
>>> + return NULL;
>>> +}
>>> +
>>> +static void put_std_call(struct optee_domain *ctx, struct optee_std_call *call)
>>> +{
>>> + spin_lock(&ctx->lock);
>>> + ASSERT(call->in_flight);
>>> + unmap_xen_arg(call);
>>
>> Same question for the unmap.
> Yeah, in normal circumstances guest should not try to resume call on
> another vCPU, because we didn't returned from the original call on
> current vCPU. But what if gust will try to do this? There are chances,
> that current CPU will unmap buffer that was mapped by other CPU an
> instance ago.
Wasn't it the point to have the in_flight field? As soon as you set in_flight to
true, then only one CPU can have the optee_std_call structure in hand.
So, as long as you map/unmap within the section protected by "in_flight", you
protected against any race. The lock is only here to protect the field in_flight
and the list. Did I miss anything?
Cheers,
--
Julien Grall
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 47+ messages in thread* Re: [PATCH v4 05/10] xen/arm: optee: add std call handling
2019-03-20 16:48 ` Julien Grall
@ 2019-03-20 17:42 ` Volodymyr Babchuk
2019-03-20 18:08 ` Julien Grall
0 siblings, 1 reply; 47+ messages in thread
From: Volodymyr Babchuk @ 2019-03-20 17:42 UTC (permalink / raw)
To: Julien Grall
Cc: tee-dev, xen-devel, Stefano Stabellini, Volodymyr Babchuk,
Volodymyr Babchuk
Julien Grall writes:
> On 20/03/2019 16:14, Volodymyr Babchuk wrote:
>>
>> Hi Julien,
>
> Hi Volodymyr,
>
>> Julien Grall writes:
>>
>> [...]
>>>> struct dt_device_node *node;
>>>> @@ -48,12 +82,25 @@ static bool optee_probe(void)
>>>> (uint32_t)resp.a3 != OPTEE_MSG_UID_3 )
>>>> return false;
>>>> + /* Read number of threads */
>>>> + arm_smccc_smc(OPTEE_SMC_GET_CONFIG, OPTEE_SMC_CONFIG_NUM_THREADS, &resp);
>>>> + if ( resp.a0 == OPTEE_SMC_RETURN_OK )
>>>
>>> Out of interest, when was this call added?
>>
>> It is on review right now. We have achieved agreement on that this call
>> is needed. I believe this will be merged into OP-TEE before I'll send v5
>> of this series.
>
> Please mention it after --- so we don't forget to check the state of
> the new call before merging to Xen.
Okay, sure
>
>>
>>>> + {
>>>> + max_optee_threads = resp.a1;
>>>> + printk(XENLOG_DEBUG "OP-TEE supports %u threads.\n", max_optee_threads);
>>>
>>> extra NIT: I would use XENLOG_INFO rather than XENLOG_DEBUG. This is a
>>> useful information to have in bug report.
>>>
>>> Regarding the message, what matters is the number of threads for
>>> guest. So I would rework it to make it more meaning full for a user
>>> that does not know the internal.
>>>
>>> You might also want to move the message out of the if. So you have the
>>> message even when OP-TEE does not support the SMC.
>> In that other case I don't know how much treads OP-TEE really
>> supports... I think, I will need to rephrase that message. Or, better,
>> I'll add another message like "Suggesting that OP-TEE supports %d
>> threads per guest".
>
> Was there any OP-TEE released containing your Virtualization patches?
No, there was no releases with virtualization support.
> Depending on the answer, I would consider to mandate
> OPTEE_SMC_CONFIG_NUM_THREADS so we don't have to deal with a default
> value in Xen.
Makes sense, I think.
> Also, should we expose this call to the guest as well?
Yes, this a good point.
[...]
>>>> +static void put_std_call(struct optee_domain *ctx, struct optee_std_call *call)
>>>> +{
>>>> + spin_lock(&ctx->lock);
>>>> + ASSERT(call->in_flight);
>>>> + unmap_xen_arg(call);
>>>
>>> Same question for the unmap.
>> Yeah, in normal circumstances guest should not try to resume call on
>> another vCPU, because we didn't returned from the original call on
>> current vCPU. But what if gust will try to do this? There are chances,
>> that current CPU will unmap buffer that was mapped by other CPU an
>> instance ago.
>
> Wasn't it the point to have the in_flight field? As soon as you set
> in_flight to true, then only one CPU can have the optee_std_call
> structure in hand.
>
> So, as long as you map/unmap within the section protected by
> "in_flight", you protected against any race. The lock is only here to
> protect the field in_flight and the list. Did I miss anything?
This is partially true. I can call map_xen_arg() after spin_unlock(), yes.
But then I need to call unmap_xen_arg() before spin_lock() in
put_std_call() function. If there were no ASSERT() that would be okay. But with
ASSERT() it is looking weird: firstly we are unmapping buffer and the we are
asserting that we had a right to do this. This is sort of "use before check"
And obviously, I can't call ASSERT() without holding the the spinlock.
On other hand, ASSERT() is a debugging feature, so we can pretend that is
not there... If you okay with this, I'll move mapping/unmapping out of
the spinlocks.
--
Best regards,Volodymyr Babchuk
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 47+ messages in thread* Re: [PATCH v4 05/10] xen/arm: optee: add std call handling
2019-03-20 17:42 ` Volodymyr Babchuk
@ 2019-03-20 18:08 ` Julien Grall
0 siblings, 0 replies; 47+ messages in thread
From: Julien Grall @ 2019-03-20 18:08 UTC (permalink / raw)
To: Volodymyr Babchuk
Cc: tee-dev, xen-devel, Stefano Stabellini, Volodymyr Babchuk
Hi Volodymyr,
On 20/03/2019 17:42, Volodymyr Babchuk wrote:
> [...]
>
>>>>> +static void put_std_call(struct optee_domain *ctx, struct optee_std_call *call)
>>>>> +{
>>>>> + spin_lock(&ctx->lock);
>>>>> + ASSERT(call->in_flight);
>>>>> + unmap_xen_arg(call);
>>>>
>>>> Same question for the unmap.
>>> Yeah, in normal circumstances guest should not try to resume call on
>>> another vCPU, because we didn't returned from the original call on
>>> current vCPU. But what if gust will try to do this? There are chances,
>>> that current CPU will unmap buffer that was mapped by other CPU an
>>> instance ago.
>>
>> Wasn't it the point to have the in_flight field? As soon as you set
>> in_flight to true, then only one CPU can have the optee_std_call
>> structure in hand.
>>
>> So, as long as you map/unmap within the section protected by
>> "in_flight", you protected against any race. The lock is only here to
>> protect the field in_flight and the list. Did I miss anything?
>
> This is partially true. I can call map_xen_arg() after spin_unlock(), yes.
>
> But then I need to call unmap_xen_arg() before spin_lock() in
> put_std_call() function. If there were no ASSERT() that would be okay. But with
> ASSERT() it is looking weird: firstly we are unmapping buffer and the we are
> asserting that we had a right to do this. This is sort of "use before check"
If you hit the ASSERT (or if it where a BUG_ON) then something has already got
horribly wrong. You will crash in any case, it is not going to matter much
whether you crash before unmapping or after unmapping.
>
> And obviously, I can't call ASSERT() without holding the the spinlock. >
> On other hand, ASSERT() is a debugging feature, so we can pretend that is
> not there... If you okay with this, I'll move mapping/unmapping out of
> the spinlocks.
ASSERT/BUG_ON are only here to catch against anything would be horribly wrong if
we continue (the severity will drive the choice between ASSERT and BUG_ON).
In normal case, this should never be hit. So I would prefer if lock section are
kept to minimal.
Cheers,
--
Julien Grall
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 47+ messages in thread
* [PATCH v4 07/10] xen/arm: optee: add support for arbitrary shared memory
2019-03-07 21:04 [PATCH v4 00/10] TEE mediator (and OP-TEE) support in XEN Volodymyr Babchuk
` (4 preceding siblings ...)
2019-03-07 21:04 ` [PATCH v4 05/10] xen/arm: optee: add std call handling Volodymyr Babchuk
@ 2019-03-07 21:04 ` Volodymyr Babchuk
2019-03-18 15:27 ` Julien Grall
2019-03-07 21:04 ` [PATCH v4 06/10] xen/arm: optee: add support for RPC SHM buffers Volodymyr Babchuk
` (3 subsequent siblings)
9 siblings, 1 reply; 47+ messages in thread
From: Volodymyr Babchuk @ 2019-03-07 21:04 UTC (permalink / raw)
To: xen-devel, tee-dev
Cc: Volodymyr Babchuk, Julien Grall, Stefano Stabellini, Volodymyr Babchuk
From: Volodymyr Babchuk <vlad.babchuk@gmail.com>
Shared memory is widely used by NW (Normal World) to communicate with
TAs (Trusted Applications) in OP-TEE. NW can share part of own memory
with TA or with OP-TEE core, by registering it in OP-TEE, or by
providing a temporal reference. Anyways, information about such memory
buffers are sent to OP-TEE as a list of pages. This mechanism is
described in optee_msg.h.
Mediator should step in when NW tries to share memory with
OP-TEE for two reasons:
1. Do address translation from IPA to PA.
2. Pin domain pages while they are mapped into OP-TEE or TA
address space, so domain can't transfer this pages to
other domain or balloon out them.
Address translation is done by translate_noncontig(...) function.
It allocates new buffer from domheap and then walks on guest
provided list of pages, translates addresses and stores PAs into
newly allocated buffer. This buffer will be provided to OP-TEE
instead of original buffer from the guest. This buffer will
be freed at the end of standard call.
In the same time this function pins pages and stores them in
struct optee_shm_buf object. This object will live all the time,
when given SHM buffer is known to OP-TEE. It will be freed
after guest unregisters shared buffer. At this time pages
will be unpinned.
Guest can share buffer with OP-TEE for duration for one call,
or permanently, using OPTEE_MSG_CMD_REGISTER_SHM call. We need
to handle both options.
Also we want to limit total size of shared buffers. As it is not
possible to get limit from OP-TEE, we need to choose some arbitrary
value. Currently limit is 16384 of 4K pages.
Signed-off-by: Volodymyr Babchuk <vlad.babchuk@gmail.com>
---
All the patches to optee.c should be merged together. They were
split to ease up review. But they depend heavily on each other.
Changes from v3:
- Reworked pagelists storage - there is no more static storage for
5 buffers, instead structure with all data is allocated dynamically
- Now this code uses domheap instead of xenheap
- Various style fixes
- gdprintk() fixes
Changes from v2:
- Made sure that guest does not tries to register shared buffer with
the same cookie twice
- Fixed coding style
- Use access_guest_memory_by_ipa() instead of direct memory mapping
---
xen/arch/arm/tee/optee.c | 404 +++++++++++++++++++++++++++++++++++++++
1 file changed, 404 insertions(+)
diff --git a/xen/arch/arm/tee/optee.c b/xen/arch/arm/tee/optee.c
index 291ed2fe25..14e295a422 100644
--- a/xen/arch/arm/tee/optee.c
+++ b/xen/arch/arm/tee/optee.c
@@ -24,6 +24,22 @@
#include <asm/tee/optee_msg.h>
#include <asm/tee/optee_smc.h>
+/*
+ * "The return code is an error that originated within the underlying
+ * communications stack linking the rich OS with the TEE" as described
+ * in GP TEE Client API Specification.
+ */
+#define TEEC_ORIGIN_COMMS 0x00000002
+
+/*
+ * "Input parameters were invalid" as described
+ * in GP TEE Client API Specification.
+ */
+#define TEEC_ERROR_BAD_PARAMETERS 0xFFFF0006
+
+/* "System ran out of resources" */
+#define TEEC_ERROR_OUT_OF_MEMORY 0xFFFF000C
+
/* Client ID 0 is reserved for hypervisor itself */
#define OPTEE_CLIENT_ID(domain) ((domain)->domain_id + 1)
@@ -33,6 +49,15 @@
*/
#define DEF_MAX_OPTEE_THREADS 16
+/*
+ * Maximum total number of pages that guest can share with
+ * OP-TEE. Currently value is selected arbitrary. Actual number of
+ * pages depends on free heap in OP-TEE. As we can't do any
+ * assumptions about OP-TEE heap usage, we limit number of pages
+ * arbitrary.
+ */
+#define MAX_TOTAL_SMH_BUF_PG 16384
+
#define OPTEE_KNOWN_NSEC_CAPS OPTEE_SMC_NSEC_CAP_UNIPROCESSOR
#define OPTEE_KNOWN_SEC_CAPS (OPTEE_SMC_SEC_CAP_HAVE_RESERVED_SHM | \
OPTEE_SMC_SEC_CAP_UNREGISTERED_SHM | \
@@ -65,11 +90,31 @@ struct shm_rpc {
uint64_t cookie;
};
+/* Shared memory buffer for arbitrary data */
+struct optee_shm_buf {
+ struct list_head list;
+ uint64_t cookie;
+ unsigned int page_cnt;
+ /*
+ * Shadowed container for list of pages that guest tries to share
+ * with OP-TEE. This is not the list of pages that guest shared
+ * with OP-TEE, but container for list of those pages. Check
+ * OPTEE_MSG_ATTR_NONCONTIG definition in optee_msg.h for more
+ * information.
+ */
+ struct page_info *pg_list;
+ unsigned int pg_list_order;
+ /* Pinned guest pages that are shared with OP-TEE */
+ struct page_info *pages[];
+};
+
/* Domain context */
struct optee_domain {
struct list_head call_list;
struct list_head shm_rpc_list;
+ struct list_head optee_shm_buf_list;
atomic_t call_count;
+ atomic_t optee_shm_buf_pages;
spinlock_t lock;
};
@@ -136,7 +181,9 @@ static int optee_domain_init(struct domain *d)
INIT_LIST_HEAD(&ctx->call_list);
INIT_LIST_HEAD(&ctx->shm_rpc_list);
+ INIT_LIST_HEAD(&ctx->optee_shm_buf_list);
atomic_set(&ctx->call_count, 0);
+ atomic_set(&ctx->optee_shm_buf_pages, 0);
spin_lock_init(&ctx->lock);
d->arch.tee = ctx;
@@ -363,10 +410,134 @@ static void free_shm_rpc(struct optee_domain *ctx, uint64_t cookie)
xfree(shm_rpc);
}
+static struct optee_shm_buf *allocate_optee_shm_buf(struct optee_domain *ctx,
+ uint64_t cookie,
+ unsigned int pages_cnt,
+ struct page_info *pg_list,
+ unsigned int pg_list_order)
+{
+ struct optee_shm_buf *optee_shm_buf, *optee_shm_buf_tmp;
+ int old, new;
+ int err_code;
+
+ do
+ {
+ old = atomic_read(&ctx->optee_shm_buf_pages);
+ new = old + pages_cnt;
+ if ( new >= MAX_TOTAL_SMH_BUF_PG )
+ return ERR_PTR(-ENOMEM);
+ }
+ while ( unlikely(old != atomic_cmpxchg(&ctx->optee_shm_buf_pages,
+ old, new)) );
+
+ optee_shm_buf = xzalloc_bytes(sizeof(struct optee_shm_buf) +
+ pages_cnt * sizeof(struct page *));
+ if ( !optee_shm_buf )
+ {
+ err_code = -ENOMEM;
+ goto err;
+ }
+
+ optee_shm_buf->cookie = cookie;
+ optee_shm_buf->pg_list = pg_list;
+ optee_shm_buf->pg_list_order = pg_list_order;
+
+ spin_lock(&ctx->lock);
+ /* Check if there is already SHM with the same cookie */
+ list_for_each_entry( optee_shm_buf_tmp, &ctx->optee_shm_buf_list, list )
+ {
+ if ( optee_shm_buf_tmp->cookie == cookie )
+ {
+ spin_unlock(&ctx->lock);
+ gdprintk(XENLOG_WARNING, "Guest tries to use the same SHM buffer cookie %lx\n",
+ cookie);
+ err_code = -EINVAL;
+ goto err;
+ }
+ }
+
+ list_add_tail(&optee_shm_buf->list, &ctx->optee_shm_buf_list);
+ spin_unlock(&ctx->lock);
+
+ return optee_shm_buf;
+
+err:
+ xfree(optee_shm_buf);
+ atomic_sub(pages_cnt, &ctx->optee_shm_buf_pages);
+
+ return ERR_PTR(err_code);
+}
+
+static void free_pg_list(struct optee_shm_buf *optee_shm_buf)
+{
+ if ( optee_shm_buf->pg_list )
+ {
+ free_domheap_pages(optee_shm_buf->pg_list,
+ optee_shm_buf->pg_list_order);
+ optee_shm_buf->pg_list = NULL;
+ }
+}
+
+static void free_optee_shm_buf(struct optee_domain *ctx, uint64_t cookie)
+{
+ struct optee_shm_buf *optee_shm_buf;
+ bool found = false;
+
+ spin_lock(&ctx->lock);
+ list_for_each_entry( optee_shm_buf, &ctx->optee_shm_buf_list, list )
+ {
+ if ( optee_shm_buf->cookie == cookie )
+ {
+ found = true;
+ list_del(&optee_shm_buf->list);
+ break;
+ }
+ }
+ spin_unlock(&ctx->lock);
+
+ if ( !found )
+ return;
+
+ for ( int i = 0; i < optee_shm_buf->page_cnt; i++ )
+ if ( optee_shm_buf->pages[i] )
+ put_page(optee_shm_buf->pages[i]);
+
+ free_pg_list(optee_shm_buf);
+
+ atomic_sub(optee_shm_buf->page_cnt, &ctx->optee_shm_buf_pages);
+
+ xfree(optee_shm_buf);
+}
+
+static void free_optee_shm_buf_pg_list(struct optee_domain *ctx,
+ uint64_t cookie)
+{
+ struct optee_shm_buf *optee_shm_buf;
+ bool found = false;
+
+ spin_lock(&ctx->lock);
+ list_for_each_entry( optee_shm_buf, &ctx->optee_shm_buf_list, list )
+ {
+ if ( optee_shm_buf->cookie == cookie )
+ {
+ found = true;
+ break;
+ }
+ }
+ spin_unlock(&ctx->lock);
+
+ if ( found )
+ free_pg_list(optee_shm_buf);
+ else
+ gdprintk(XENLOG_ERR, "Can't find pagelist for SHM buffer with cookie %lx to free it\n",
+ cookie);
+}
+
static int optee_relinquish_resources(struct domain *d)
{
struct optee_std_call *call, *call_tmp;
struct shm_rpc *shm_rpc, *shm_rpc_tmp;
+ struct optee_shm_buf *optee_shm_buf, *optee_shm_buf_tmp;
struct optee_domain *ctx = d->arch.tee;
if ( !ctx )
@@ -381,6 +552,13 @@ static int optee_relinquish_resources(struct domain *d)
list_for_each_entry_safe( shm_rpc, shm_rpc_tmp, &ctx->shm_rpc_list, list )
free_shm_rpc(ctx, shm_rpc->cookie);
+ if ( hypercall_preempt_check() )
+ return -ERESTART;
+
+ list_for_each_entry_safe( optee_shm_buf, optee_shm_buf_tmp,
+ &ctx->optee_shm_buf_list, list )
+ free_optee_shm_buf(ctx, optee_shm_buf->cookie);
+
return 0;
}
@@ -406,11 +584,186 @@ static void optee_domain_destroy(struct domain *d)
ASSERT(!spin_is_locked(&ctx->lock));
ASSERT(!atomic_read(&ctx->call_count));
+ ASSERT(!atomic_read(&ctx->optee_shm_buf_pages));
ASSERT(list_empty(&ctx->shm_rpc_list));
XFREE(d->arch.tee);
}
+#define PAGELIST_ENTRIES_PER_PAGE \
+ ((OPTEE_MSG_NONCONTIG_PAGE_SIZE / sizeof(u64)) - 1)
+
+static size_t get_pages_list_size(size_t num_entries)
+{
+ int pages = DIV_ROUND_UP(num_entries, PAGELIST_ENTRIES_PER_PAGE);
+
+ return pages * OPTEE_MSG_NONCONTIG_PAGE_SIZE;
+}
+
+static int translate_noncontig(struct optee_domain *ctx,
+ struct optee_std_call *call,
+ struct optee_msg_param *param)
+{
+ uint64_t size;
+ unsigned int page_offset;
+ unsigned int num_pages;
+ unsigned int order;
+ unsigned int entries_on_page = 0;
+ gfn_t gfn;
+ p2m_type_t p2m;
+ struct page_info *guest_page, *xen_pages;
+ struct optee_shm_buf *optee_shm_buf;
+ /*
+ * This is memory layout for page list. Basically list consists of 4k pages,
+ * every page store 511 page addresses of user buffer and page address of
+ * the next page of list.
+ *
+ * Refer to OPTEE_MSG_ATTR_NONCONTIG description in optee_msg.h for details.
+ */
+ struct {
+ uint64_t pages_list[PAGELIST_ENTRIES_PER_PAGE];
+ uint64_t next_page_data;
+ } *pages_data_guest, *pages_data_xen;
+
+ /* Offset of user buffer withing page */
+ page_offset = param->u.tmem.buf_ptr & (OPTEE_MSG_NONCONTIG_PAGE_SIZE - 1);
+
+ /* Size of the user buffer in bytes */
+ size = ROUNDUP(param->u.tmem.size + page_offset,
+ OPTEE_MSG_NONCONTIG_PAGE_SIZE);
+
+ num_pages = DIV_ROUND_UP(size, OPTEE_MSG_NONCONTIG_PAGE_SIZE);
+
+ order = get_order_from_bytes(get_pages_list_size(num_pages));
+
+ xen_pages = alloc_domheap_pages(current->domain, order, 0);
+ if ( !xen_pages )
+ return -ENOMEM;
+
+ optee_shm_buf = allocate_optee_shm_buf(ctx, param->u.tmem.shm_ref,
+ num_pages, xen_pages, order);
+ if ( IS_ERR(optee_shm_buf) )
+ return PTR_ERR(optee_shm_buf);
+
+ gfn = gaddr_to_gfn(param->u.tmem.buf_ptr &
+ ~(OPTEE_MSG_NONCONTIG_PAGE_SIZE - 1));
+
+ guest_page = get_page_from_gfn(current->domain, gfn_x(gfn), &p2m, P2M_ALLOC);
+ if ( !guest_page || p2m != p2m_ram_rw )
+ return -EINVAL;
+
+ pages_data_guest = __map_domain_page(guest_page);
+ pages_data_xen = __map_domain_page(xen_pages);
+
+ while ( num_pages )
+ {
+ struct page_info *page;
+ page = get_page_from_gfn(current->domain,
+ paddr_to_pfn(pages_data_guest->pages_list[entries_on_page]),
+ &p2m, P2M_ALLOC);
+
+ if ( !page || p2m != p2m_ram_rw )
+ goto err_unmap;
+
+ optee_shm_buf->pages[optee_shm_buf->page_cnt++] = page;
+ pages_data_xen->pages_list[entries_on_page] = page_to_maddr(page);
+ entries_on_page++;
+
+ if ( entries_on_page == PAGELIST_ENTRIES_PER_PAGE )
+ {
+ pages_data_xen->next_page_data = page_to_maddr(xen_pages + 1);
+ unmap_domain_page(pages_data_xen);
+ xen_pages++;
+
+ gfn = gaddr_to_gfn(pages_data_guest->next_page_data);
+
+ unmap_domain_page(pages_data_guest);
+ put_page(guest_page);
+
+ guest_page = get_page_from_gfn(current->domain, gfn_x(gfn), &p2m,
+ P2M_ALLOC);
+ if ( !guest_page || p2m != p2m_ram_rw )
+ return -EINVAL;
+
+ pages_data_guest = __map_domain_page(guest_page);
+ pages_data_xen = __map_domain_page(xen_pages);
+ /* Roll over to the next page */
+ entries_on_page = 0;
+ }
+ num_pages--;
+ }
+
+ unmap_domain_page(pages_data_guest);
+ unmap_domain_page(pages_data_xen);
+ put_page(guest_page);
+
+ param->u.tmem.buf_ptr = page_to_maddr(optee_shm_buf->pg_list) |
+ page_offset;
+
+ return 0;
+
+err_unmap:
+ unmap_domain_page(pages_data_guest);
+ unmap_domain_page(pages_data_xen);
+ put_page(guest_page);
+ free_optee_shm_buf(ctx, optee_shm_buf->cookie);
+
+ return -EINVAL;
+}
+
+static int translate_params(struct optee_domain *ctx,
+ struct optee_std_call *call)
+{
+ unsigned int i;
+ uint32_t attr;
+ int ret = 0;
+
+ for ( i = 0; i < call->xen_arg->num_params; i++ )
+ {
+ attr = call->xen_arg->params[i].attr;
+
+ switch ( attr & OPTEE_MSG_ATTR_TYPE_MASK )
+ {
+ case OPTEE_MSG_ATTR_TYPE_TMEM_INPUT:
+ case OPTEE_MSG_ATTR_TYPE_TMEM_OUTPUT:
+ case OPTEE_MSG_ATTR_TYPE_TMEM_INOUT:
+ if ( attr & OPTEE_MSG_ATTR_NONCONTIG )
+ {
+ ret = translate_noncontig(ctx, call, call->xen_arg->params + i);
+ if ( ret )
+ goto out;
+ }
+ else
+ {
+ gdprintk(XENLOG_WARNING, "Guest tries to use old tmem arg\n");
+ ret = -EINVAL;
+ goto out;
+ }
+ break;
+ case OPTEE_MSG_ATTR_TYPE_NONE:
+ case OPTEE_MSG_ATTR_TYPE_VALUE_INPUT:
+ case OPTEE_MSG_ATTR_TYPE_VALUE_OUTPUT:
+ case OPTEE_MSG_ATTR_TYPE_VALUE_INOUT:
+ case OPTEE_MSG_ATTR_TYPE_RMEM_INPUT:
+ case OPTEE_MSG_ATTR_TYPE_RMEM_OUTPUT:
+ case OPTEE_MSG_ATTR_TYPE_RMEM_INOUT:
+ continue;
+ }
+ }
+
+out:
+ if ( ret )
+ {
+ call->xen_arg->ret_origin = TEEC_ORIGIN_COMMS;
+ if ( ret == -ENOMEM )
+ call->xen_arg->ret = TEEC_ERROR_OUT_OF_MEMORY;
+ else
+ call->xen_arg->ret = TEEC_ERROR_BAD_PARAMETERS;
+ }
+
+ return ret;
+}
+
/*
* Copy command buffer into domheap memory to:
* 1) Hide translated addresses from guest
@@ -537,6 +890,27 @@ static void copy_std_request_back(struct optee_domain *ctx,
put_page(page);
}
+
+static void free_shm_buffers(struct optee_domain *ctx,
+ struct optee_msg_arg *arg)
+{
+ unsigned int i;
+
+ for ( i = 0; i < arg->num_params; i ++ )
+ {
+ switch ( arg->params[i].attr & OPTEE_MSG_ATTR_TYPE_MASK )
+ {
+ case OPTEE_MSG_ATTR_TYPE_TMEM_INPUT:
+ case OPTEE_MSG_ATTR_TYPE_TMEM_OUTPUT:
+ case OPTEE_MSG_ATTR_TYPE_TMEM_INOUT:
+ free_optee_shm_buf(ctx, arg->params[i].u.tmem.shm_ref);
+ break;
+ default:
+ break;
+ }
+ }
+}
+
/* Handle RPC return from OP-TEE */
static void handle_rpc_return(struct cpu_user_regs *regs,
struct optee_std_call *call)
@@ -556,6 +930,8 @@ static void handle_rpc_return(struct cpu_user_regs *regs,
* If this is RPC - we need to store call context and return back to guest.
* If call is complete - we need to return results with copy_std_request_back()
* and then we will destroy the call context as it is not needed anymore.
+ *
+ * Shared buffers should be handled in a special way.
*/
static void execute_std_call(struct optee_domain *ctx,
struct cpu_user_regs *regs,
@@ -576,6 +952,23 @@ static void execute_std_call(struct optee_domain *ctx,
copy_std_request_back(ctx, regs, call);
+ switch ( call->xen_arg->cmd )
+ {
+ case OPTEE_MSG_CMD_REGISTER_SHM:
+ if ( call->xen_arg->ret == 0 )
+ free_optee_shm_buf_pg_list(ctx,
+ call->xen_arg->params[0].u.tmem.shm_ref);
+ else
+ free_optee_shm_buf(ctx, call->xen_arg->params[0].u.tmem.shm_ref);
+ break;
+ case OPTEE_MSG_CMD_UNREGISTER_SHM:
+ if ( call->xen_arg->ret == 0 )
+ free_optee_shm_buf(ctx, call->xen_arg->params[0].u.rmem.shm_ref);
+ break;
+ default:
+ free_shm_buffers(ctx, call->xen_arg);
+ }
+
put_std_call(ctx, call);
free_std_call(ctx, call);
}
@@ -606,6 +999,17 @@ static void handle_std_call(struct optee_domain *ctx,
case OPTEE_MSG_CMD_CANCEL:
case OPTEE_MSG_CMD_REGISTER_SHM:
case OPTEE_MSG_CMD_UNREGISTER_SHM:
+ if( translate_params(ctx, call) )
+ {
+ /*
+ * translate_params() sets xen_arg->ret value to non-zero.
+ * So, technically, SMC was successful, but there was an error
+ * during handling standard call encapsulated into this SMC.
+ */
+ copy_std_request_back(ctx, regs, call);
+ set_return(regs, OPTEE_SMC_RETURN_OK);
+ goto err;
+ }
execute_std_call(ctx, regs, call);
return;
default:
--
2.21.0
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply related [flat|nested] 47+ messages in thread* Re: [PATCH v4 07/10] xen/arm: optee: add support for arbitrary shared memory
2019-03-07 21:04 ` [PATCH v4 07/10] xen/arm: optee: add support for arbitrary shared memory Volodymyr Babchuk
@ 2019-03-18 15:27 ` Julien Grall
2019-03-20 16:39 ` Volodymyr Babchuk
0 siblings, 1 reply; 47+ messages in thread
From: Julien Grall @ 2019-03-18 15:27 UTC (permalink / raw)
To: Volodymyr Babchuk, xen-devel, tee-dev
Cc: Volodymyr Babchuk, Stefano Stabellini
On 07/03/2019 21:04, Volodymyr Babchuk wrote:
> From: Volodymyr Babchuk <vlad.babchuk@gmail.com>
Some of the patches are using your EPAM e-mail addresss. Other are using your
gmail address. Could you confirm this is expected?
>
> Shared memory is widely used by NW (Normal World) to communicate with
> TAs (Trusted Applications) in OP-TEE. NW can share part of own memory
> with TA or with OP-TEE core, by registering it in OP-TEE, or by
> providing a temporal reference. Anyways, information about such memory
> buffers are sent to OP-TEE as a list of pages. This mechanism is
> described in optee_msg.h.
>
> Mediator should step in when NW tries to share memory with
> OP-TEE for two reasons:
>
> 1. Do address translation from IPA to PA.
> 2. Pin domain pages while they are mapped into OP-TEE or TA
> address space, so domain can't transfer this pages to
> other domain or balloon out them.
>
> Address translation is done by translate_noncontig(...) function.
> It allocates new buffer from domheap and then walks on guest
> provided list of pages, translates addresses and stores PAs into
> newly allocated buffer. This buffer will be provided to OP-TEE
> instead of original buffer from the guest. This buffer will
> be freed at the end of standard call.
>
> In the same time this function pins pages and stores them in
> struct optee_shm_buf object. This object will live all the time,
> when given SHM buffer is known to OP-TEE. It will be freed
> after guest unregisters shared buffer. At this time pages
> will be unpinned.
>
> Guest can share buffer with OP-TEE for duration for one call,
> or permanently, using OPTEE_MSG_CMD_REGISTER_SHM call. We need
> to handle both options.
>
> Also we want to limit total size of shared buffers. As it is not
> possible to get limit from OP-TEE, we need to choose some arbitrary
> value. Currently limit is 16384 of 4K pages.
I can't promise Xen will only be 4K only. So it would be better to make the
number agnostic. Or at least writing clearly on top of the definition that it is
assumed 4KB (maybe with a BUILD_BUG_ON(PAGE_SIZE != 4096) if not already in place).
>
> Signed-off-by: Volodymyr Babchuk <vlad.babchuk@gmail.com>
> ---
> All the patches to optee.c should be merged together. They were
> split to ease up review. But they depend heavily on each other.
>
> Changes from v3:
> - Reworked pagelists storage - there is no more static storage for
> 5 buffers, instead structure with all data is allocated dynamically
> - Now this code uses domheap instead of xenheap
> - Various style fixes
> - gdprintk() fixes
>
> Changes from v2:
> - Made sure that guest does not tries to register shared buffer with
> the same cookie twice
> - Fixed coding style
> - Use access_guest_memory_by_ipa() instead of direct memory mapping
> ---
> xen/arch/arm/tee/optee.c | 404 +++++++++++++++++++++++++++++++++++++++
> 1 file changed, 404 insertions(+)
>
> diff --git a/xen/arch/arm/tee/optee.c b/xen/arch/arm/tee/optee.c
> index 291ed2fe25..14e295a422 100644
> --- a/xen/arch/arm/tee/optee.c
> +++ b/xen/arch/arm/tee/optee.c
> @@ -24,6 +24,22 @@
> #include <asm/tee/optee_msg.h>
> #include <asm/tee/optee_smc.h>
>
> +/*
> + * "The return code is an error that originated within the underlying
> + * communications stack linking the rich OS with the TEE" as described
> + * in GP TEE Client API Specification.
> + */
> +#define TEEC_ORIGIN_COMMS 0x00000002
> +
> +/*
> + * "Input parameters were invalid" as described
> + * in GP TEE Client API Specification.
> + */
> +#define TEEC_ERROR_BAD_PARAMETERS 0xFFFF0006
> +
> +/* "System ran out of resources" */
> +#define TEEC_ERROR_OUT_OF_MEMORY 0xFFFF000C
> +
> /* Client ID 0 is reserved for hypervisor itself */
> #define OPTEE_CLIENT_ID(domain) ((domain)->domain_id + 1)
>
> @@ -33,6 +49,15 @@
> */
> #define DEF_MAX_OPTEE_THREADS 16
>
> +/*
> + * Maximum total number of pages that guest can share with
> + * OP-TEE. Currently value is selected arbitrary. Actual number of
> + * pages depends on free heap in OP-TEE. As we can't do any
> + * assumptions about OP-TEE heap usage, we limit number of pages
> + * arbitrary.
> + */
> +#define MAX_TOTAL_SMH_BUF_PG 16384
> +
> #define OPTEE_KNOWN_NSEC_CAPS OPTEE_SMC_NSEC_CAP_UNIPROCESSOR
> #define OPTEE_KNOWN_SEC_CAPS (OPTEE_SMC_SEC_CAP_HAVE_RESERVED_SHM | \
> OPTEE_SMC_SEC_CAP_UNREGISTERED_SHM | \
> @@ -65,11 +90,31 @@ struct shm_rpc {
> uint64_t cookie;
> };
>
> +/* Shared memory buffer for arbitrary data */
> +struct optee_shm_buf {
> + struct list_head list;
> + uint64_t cookie;
> + unsigned int page_cnt;
> + /*
> + * Shadowed container for list of pages that guest tries to share
> + * with OP-TEE. This is not the list of pages that guest shared
> + * with OP-TEE, but container for list of those pages. Check
> + * OPTEE_MSG_ATTR_NONCONTIG definition in optee_msg.h for more
> + * information.
> + */
> + struct page_info *pg_list;
> + unsigned int pg_list_order;
> + /* Pinned guest pages that are shared with OP-TEE */
> + struct page_info *pages[];
> +};
> +
> /* Domain context */
> struct optee_domain {
> struct list_head call_list;
> struct list_head shm_rpc_list;
> + struct list_head optee_shm_buf_list;
> atomic_t call_count;
> + atomic_t optee_shm_buf_pages;
> spinlock_t lock;
> };
>
> @@ -136,7 +181,9 @@ static int optee_domain_init(struct domain *d)
>
> INIT_LIST_HEAD(&ctx->call_list);
> INIT_LIST_HEAD(&ctx->shm_rpc_list);
> + INIT_LIST_HEAD(&ctx->optee_shm_buf_list);
> atomic_set(&ctx->call_count, 0);
> + atomic_set(&ctx->optee_shm_buf_pages, 0);
> spin_lock_init(&ctx->lock);
>
> d->arch.tee = ctx;
> @@ -363,10 +410,134 @@ static void free_shm_rpc(struct optee_domain *ctx, uint64_t cookie)
> xfree(shm_rpc);
> }
>
> +static struct optee_shm_buf *allocate_optee_shm_buf(struct optee_domain *ctx,
> + uint64_t cookie,
> + unsigned int pages_cnt,
> + struct page_info *pg_list,
> + unsigned int pg_list_order)
> +{
> + struct optee_shm_buf *optee_shm_buf, *optee_shm_buf_tmp;
> + int old, new;
> + int err_code;
> +
> + do
> + {
> + old = atomic_read(&ctx->optee_shm_buf_pages);
> + new = old + pages_cnt;
> + if ( new >= MAX_TOTAL_SMH_BUF_PG )
Again, the limitation is in number of page and quite high. What would prevent a
guest to register shared memory page by page? If nothing, then I think you can
end up to interesting issues in Xen because of the growing list and memory used.
> + return ERR_PTR(-ENOMEM);
> + }
> + while ( unlikely(old != atomic_cmpxchg(&ctx->optee_shm_buf_pages,
> + old, new)) );
> +
> + optee_shm_buf = xzalloc_bytes(sizeof(struct optee_shm_buf) +
> + pages_cnt * sizeof(struct page *));
> + if ( !optee_shm_buf )
> + {
> + err_code = -ENOMEM;
> + goto err;
> + }
> +
> + optee_shm_buf->cookie = cookie;
> + optee_shm_buf->pg_list = pg_list;
> + optee_shm_buf->pg_list_order = pg_list_order;
> +
> + spin_lock(&ctx->lock);
> + /* Check if there is already SHM with the same cookie */
> + list_for_each_entry( optee_shm_buf_tmp, &ctx->optee_shm_buf_list, list )
> + {
> + if ( optee_shm_buf_tmp->cookie == cookie )
> + {
> + spin_unlock(&ctx->lock);
> + gdprintk(XENLOG_WARNING, "Guest tries to use the same SHM buffer cookie %lx\n",
The line looks too long. Please split after the first comma.
> + cookie);
> + err_code = -EINVAL;
> + goto err;
> + }
> + }
> +
> + list_add_tail(&optee_shm_buf->list, &ctx->optee_shm_buf_list);
> + spin_unlock(&ctx->lock);
> +
> + return optee_shm_buf;
> +
> +err:
> + xfree(optee_shm_buf);
> + atomic_sub(pages_cnt, &ctx->optee_shm_buf_pages);
> +
> + return ERR_PTR(err_code);
> +}
> +
> +static void free_pg_list(struct optee_shm_buf *optee_shm_buf)
> +{
> + if ( optee_shm_buf->pg_list )
> + {
> + free_domheap_pages(optee_shm_buf->pg_list,
> + optee_shm_buf->pg_list_order);
> + optee_shm_buf->pg_list = NULL;
> + }
> +}
> +
> +static void free_optee_shm_buf(struct optee_domain *ctx, uint64_t cookie)
> +{
> + struct optee_shm_buf *optee_shm_buf;
> + bool found = false;
> +
> + spin_lock(&ctx->lock);
> + list_for_each_entry( optee_shm_buf, &ctx->optee_shm_buf_list, list )
> + {
> + if ( optee_shm_buf->cookie == cookie )
> + {
> + found = true;
> + list_del(&optee_shm_buf->list);
> + break;
> + }
> + }
> + spin_unlock(&ctx->lock);
> +
> + if ( !found )
> + return;
> +
> + for ( int i = 0; i < optee_shm_buf->page_cnt; i++ )
As already request in the previous version, please define the variable i outside
of the for loop. Also, the variable should be unsigned int.
> + if ( optee_shm_buf->pages[i] )
> + put_page(optee_shm_buf->pages[i]);
> +
> + free_pg_list(optee_shm_buf);
> +
> + atomic_sub(optee_shm_buf->page_cnt, &ctx->optee_shm_buf_pages);
> +
> + xfree(optee_shm_buf);
> +}
> +
> +static void free_optee_shm_buf_pg_list(struct optee_domain *ctx,
> + uint64_t cookie)
> +{
> + struct optee_shm_buf *optee_shm_buf;
> + bool found = false;
> +
> + spin_lock(&ctx->lock);
> + list_for_each_entry( optee_shm_buf, &ctx->optee_shm_buf_list, list )
> + {
> + if ( optee_shm_buf->cookie == cookie )
> + {
> + found = true;
> + break;
> + }
> + }
> + spin_unlock(&ctx->lock);
> +
> + if ( found )
> + free_pg_list(optee_shm_buf);
> + else
> + gdprintk(XENLOG_ERR, "Can't find pagelist for SHM buffer with cookie %lx to free it\n",
> + cookie);
> +}
> +
> static int optee_relinquish_resources(struct domain *d)
> {
> struct optee_std_call *call, *call_tmp;
> struct shm_rpc *shm_rpc, *shm_rpc_tmp;
> + struct optee_shm_buf *optee_shm_buf, *optee_shm_buf_tmp;
> struct optee_domain *ctx = d->arch.tee;
>
> if ( !ctx )
> @@ -381,6 +552,13 @@ static int optee_relinquish_resources(struct domain *d)
> list_for_each_entry_safe( shm_rpc, shm_rpc_tmp, &ctx->shm_rpc_list, list )
> free_shm_rpc(ctx, shm_rpc->cookie);
>
> + if ( hypercall_preempt_check() )
> + return -ERESTART;
Same question as the other hypercall_preempt_check().
> +
> + list_for_each_entry_safe( optee_shm_buf, optee_shm_buf_tmp,
> + &ctx->optee_shm_buf_list, list )
> + free_optee_shm_buf(ctx, optee_shm_buf->cookie);
> +
> return 0;
> }
>
> @@ -406,11 +584,186 @@ static void optee_domain_destroy(struct domain *d)
>
> ASSERT(!spin_is_locked(&ctx->lock));
> ASSERT(!atomic_read(&ctx->call_count));
> + ASSERT(!atomic_read(&ctx->optee_shm_buf_pages));
> ASSERT(list_empty(&ctx->shm_rpc_list));
>
> XFREE(d->arch.tee);
> }
>
> +#define PAGELIST_ENTRIES_PER_PAGE \
> + ((OPTEE_MSG_NONCONTIG_PAGE_SIZE / sizeof(u64)) - 1)
> +
> +static size_t get_pages_list_size(size_t num_entries)
> +{
> + int pages = DIV_ROUND_UP(num_entries, PAGELIST_ENTRIES_PER_PAGE);
> +
> + return pages * OPTEE_MSG_NONCONTIG_PAGE_SIZE;
> +}
> +
> +static int translate_noncontig(struct optee_domain *ctx,
> + struct optee_std_call *call,
> + struct optee_msg_param *param)
> +{
> + uint64_t size;
> + unsigned int page_offset;
> + unsigned int num_pages;
> + unsigned int order;
> + unsigned int entries_on_page = 0;
> + gfn_t gfn;
> + p2m_type_t p2m;
> + struct page_info *guest_page, *xen_pages;
> + struct optee_shm_buf *optee_shm_buf;
> + /*
> + * This is memory layout for page list. Basically list consists of 4k pages,
> + * every page store 511 page addresses of user buffer and page address of
> + * the next page of list.
> + *
> + * Refer to OPTEE_MSG_ATTR_NONCONTIG description in optee_msg.h for details.
> + */
> + struct {
> + uint64_t pages_list[PAGELIST_ENTRIES_PER_PAGE];
> + uint64_t next_page_data;
> + } *pages_data_guest, *pages_data_xen;
> +
> + /* Offset of user buffer withing page */
Offset of the buffer within the OP-TEE page
> + page_offset = param->u.tmem.buf_ptr & (OPTEE_MSG_NONCONTIG_PAGE_SIZE - 1);
> +
> + /* Size of the user buffer in bytes */
> + size = ROUNDUP(param->u.tmem.size + page_offset,
> + OPTEE_MSG_NONCONTIG_PAGE_SIZE);
> +
> + num_pages = DIV_ROUND_UP(size, OPTEE_MSG_NONCONTIG_PAGE_SIZE);
> +
> + order = get_order_from_bytes(get_pages_list_size(num_pages));
> +
> + xen_pages = alloc_domheap_pages(current->domain, order, 0);
> + if ( !xen_pages )
> + return -ENOMEM;
> +
> + optee_shm_buf = allocate_optee_shm_buf(ctx, param->u.tmem.shm_ref,
> + num_pages, xen_pages, order);
In an earlier version, I pointed out that you will allow to allocate up to 64MB
per call. This is a potential security issue on Xen. While I said I would be
happy to get this code merged as it, I would expect to at least see a TODO in
the code explaining potential problem. So we know the problem exist and can't
security support until this is fixed.
I may have missed other places while reviewing this version. Please go back in
the review I have made in the past and document all the potential security holes.
> + if ( IS_ERR(optee_shm_buf) )
> + return PTR_ERR(optee_shm_buf);
> +
> + gfn = gaddr_to_gfn(param->u.tmem.buf_ptr &
> + ~(OPTEE_MSG_NONCONTIG_PAGE_SIZE - 1));
> +
> + guest_page = get_page_from_gfn(current->domain, gfn_x(gfn), &p2m, P2M_ALLOC);
> + if ( !guest_page || p2m != p2m_ram_rw )
> + return -EINVAL;
> +
> + pages_data_guest = __map_domain_page(guest_page);
> + pages_data_xen = __map_domain_page(xen_pages);
> +
> + while ( num_pages )
> + {
> + struct page_info *page;
Newline here please.
> + page = get_page_from_gfn(current->domain,
> + paddr_to_pfn(pages_data_guest->pages_list[entries_on_page]),
> + &p2m, P2M_ALLOC);
The indentation is wrong here. But the problem is due to the long name. For
instance, you have 3 times the word "page" on the same line. Is that necessary?
> +
> + if ( !page || p2m != p2m_ram_rw )
> + goto err_unmap;
> +
> + optee_shm_buf->pages[optee_shm_buf->page_cnt++] = page;
> + pages_data_xen->pages_list[entries_on_page] = page_to_maddr(page);
> + entries_on_page++;
> +
> + if ( entries_on_page == PAGELIST_ENTRIES_PER_PAGE )
> + {
> + pages_data_xen->next_page_data = page_to_maddr(xen_pages + 1);
> + unmap_domain_page(pages_data_xen);
> + xen_pages++;
> +
> + gfn = gaddr_to_gfn(pages_data_guest->next_page_data);
> +
> + unmap_domain_page(pages_data_guest);
> + put_page(guest_page);
> +
> + guest_page = get_page_from_gfn(current->domain, gfn_x(gfn), &p2m,
> + P2M_ALLOC);
> + if ( !guest_page || p2m != p2m_ram_rw )
> + return -EINVAL;
> +
> + pages_data_guest = __map_domain_page(guest_page);
> + pages_data_xen = __map_domain_page(xen_pages);
> + /* Roll over to the next page */
> + entries_on_page = 0;
> + }
> + num_pages--;
> + }
> +
> + unmap_domain_page(pages_data_guest);
> + unmap_domain_page(pages_data_xen);
> + put_page(guest_page);
> +
> + param->u.tmem.buf_ptr = page_to_maddr(optee_shm_buf->pg_list) |
> + page_offset;
> +
> + return 0;
> +
> +err_unmap:
> + unmap_domain_page(pages_data_guest);
> + unmap_domain_page(pages_data_xen);
> + put_page(guest_page);
> + free_optee_shm_buf(ctx, optee_shm_buf->cookie);
> +
> + return -EINVAL;
> +}
> +
> +static int translate_params(struct optee_domain *ctx,
> + struct optee_std_call *call)
> +{
> + unsigned int i;
> + uint32_t attr;
> + int ret = 0;
> +
> + for ( i = 0; i < call->xen_arg->num_params; i++ )
> + {
> + attr = call->xen_arg->params[i].attr;
> +
> + switch ( attr & OPTEE_MSG_ATTR_TYPE_MASK )
> + {
> + case OPTEE_MSG_ATTR_TYPE_TMEM_INPUT:
> + case OPTEE_MSG_ATTR_TYPE_TMEM_OUTPUT:
> + case OPTEE_MSG_ATTR_TYPE_TMEM_INOUT:
> + if ( attr & OPTEE_MSG_ATTR_NONCONTIG )
> + {
> + ret = translate_noncontig(ctx, call, call->xen_arg->params + i);
> + if ( ret )
> + goto out;
> + }
> + else
> + {
> + gdprintk(XENLOG_WARNING, "Guest tries to use old tmem arg\n");
> + ret = -EINVAL;
> + goto out;
> + }
> + break;
> + case OPTEE_MSG_ATTR_TYPE_NONE:
> + case OPTEE_MSG_ATTR_TYPE_VALUE_INPUT:
> + case OPTEE_MSG_ATTR_TYPE_VALUE_OUTPUT:
> + case OPTEE_MSG_ATTR_TYPE_VALUE_INOUT:
> + case OPTEE_MSG_ATTR_TYPE_RMEM_INPUT:
> + case OPTEE_MSG_ATTR_TYPE_RMEM_OUTPUT:
> + case OPTEE_MSG_ATTR_TYPE_RMEM_INOUT:
> + continue;
> + }
> + }
> +
> +out:
> + if ( ret )
> + {
> + call->xen_arg->ret_origin = TEEC_ORIGIN_COMMS;
> + if ( ret == -ENOMEM )
> + call->xen_arg->ret = TEEC_ERROR_OUT_OF_MEMORY;
> + else
> + call->xen_arg->ret = TEEC_ERROR_BAD_PARAMETERS;
> + }
> +
> + return ret;
> +}
> +
> /*
> * Copy command buffer into domheap memory to:
> * 1) Hide translated addresses from guest
> @@ -537,6 +890,27 @@ static void copy_std_request_back(struct optee_domain *ctx,
> put_page(page);
> }
>
> +
> +static void free_shm_buffers(struct optee_domain *ctx,
> + struct optee_msg_arg *arg)
> +{
> + unsigned int i;
> +
> + for ( i = 0; i < arg->num_params; i ++ )
> + {
> + switch ( arg->params[i].attr & OPTEE_MSG_ATTR_TYPE_MASK )
> + {
> + case OPTEE_MSG_ATTR_TYPE_TMEM_INPUT:
> + case OPTEE_MSG_ATTR_TYPE_TMEM_OUTPUT:
> + case OPTEE_MSG_ATTR_TYPE_TMEM_INOUT:
> + free_optee_shm_buf(ctx, arg->params[i].u.tmem.shm_ref);
> + break;
> + default:
> + break;
> + }
> + }
> +}
> +
> /* Handle RPC return from OP-TEE */
> static void handle_rpc_return(struct cpu_user_regs *regs,
> struct optee_std_call *call)
> @@ -556,6 +930,8 @@ static void handle_rpc_return(struct cpu_user_regs *regs,
> * If this is RPC - we need to store call context and return back to guest.
> * If call is complete - we need to return results with copy_std_request_back()
> * and then we will destroy the call context as it is not needed anymore.
> + *
> + * Shared buffers should be handled in a special way.
> */
> static void execute_std_call(struct optee_domain *ctx,
> struct cpu_user_regs *regs,
> @@ -576,6 +952,23 @@ static void execute_std_call(struct optee_domain *ctx,
>
> copy_std_request_back(ctx, regs, call);
>
> + switch ( call->xen_arg->cmd )
> + {
> + case OPTEE_MSG_CMD_REGISTER_SHM:
> + if ( call->xen_arg->ret == 0 )
> + free_optee_shm_buf_pg_list(ctx,
> + call->xen_arg->params[0].u.tmem.shm_ref);
> + else
> + free_optee_shm_buf(ctx, call->xen_arg->params[0].u.tmem.shm_ref);
> + break;
> + case OPTEE_MSG_CMD_UNREGISTER_SHM:
> + if ( call->xen_arg->ret == 0 )
> + free_optee_shm_buf(ctx, call->xen_arg->params[0].u.rmem.shm_ref);
> + break;
> + default:
> + free_shm_buffers(ctx, call->xen_arg);
> + }
This switch seems to be new. Could you explain in a comment what they are
supposed to do? For instance, why do you need to make UNREGISTER_SHM and
REGISTER_SHM speific by specifying the exact buffer?
> +
> put_std_call(ctx, call);
> free_std_call(ctx, call);
> }
> @@ -606,6 +999,17 @@ static void handle_std_call(struct optee_domain *ctx,
> case OPTEE_MSG_CMD_CANCEL:
> case OPTEE_MSG_CMD_REGISTER_SHM:
> case OPTEE_MSG_CMD_UNREGISTER_SHM:
> + if( translate_params(ctx, call) )
> + {
> + /*
> + * translate_params() sets xen_arg->ret value to non-zero.
> + * So, technically, SMC was successful, but there was an error
> + * during handling standard call encapsulated into this SMC.
> + */
> + copy_std_request_back(ctx, regs, call);
> + set_return(regs, OPTEE_SMC_RETURN_OK);
> + goto err;
> + }
> execute_std_call(ctx, regs, call);
> return;
> default:
>
Cheers,
--
Julien Grall
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 47+ messages in thread* Re: [PATCH v4 07/10] xen/arm: optee: add support for arbitrary shared memory
2019-03-18 15:27 ` Julien Grall
@ 2019-03-20 16:39 ` Volodymyr Babchuk
2019-03-20 17:47 ` Julien Grall
0 siblings, 1 reply; 47+ messages in thread
From: Volodymyr Babchuk @ 2019-03-20 16:39 UTC (permalink / raw)
To: Julien Grall
Cc: tee-dev, xen-devel, Stefano Stabellini, Volodymyr Babchuk,
Volodymyr Babchuk
Julien Grall writes:
> On 07/03/2019 21:04, Volodymyr Babchuk wrote:
>> From: Volodymyr Babchuk <vlad.babchuk@gmail.com>
>
> Some of the patches are using your EPAM e-mail addresss. Other are
> using your gmail address. Could you confirm this is expected?
No, I'll make sure that all patches are authored by
<volodymyr_babchuk@epam.com>
>>
>> Shared memory is widely used by NW (Normal World) to communicate with
>> TAs (Trusted Applications) in OP-TEE. NW can share part of own memory
>> with TA or with OP-TEE core, by registering it in OP-TEE, or by
>> providing a temporal reference. Anyways, information about such memory
>> buffers are sent to OP-TEE as a list of pages. This mechanism is
>> described in optee_msg.h.
>>
>> Mediator should step in when NW tries to share memory with
>> OP-TEE for two reasons:
>>
>> 1. Do address translation from IPA to PA.
>> 2. Pin domain pages while they are mapped into OP-TEE or TA
>> address space, so domain can't transfer this pages to
>> other domain or balloon out them.
>>
>> Address translation is done by translate_noncontig(...) function.
>> It allocates new buffer from domheap and then walks on guest
>> provided list of pages, translates addresses and stores PAs into
>> newly allocated buffer. This buffer will be provided to OP-TEE
>> instead of original buffer from the guest. This buffer will
>> be freed at the end of standard call.
>>
>> In the same time this function pins pages and stores them in
>> struct optee_shm_buf object. This object will live all the time,
>> when given SHM buffer is known to OP-TEE. It will be freed
>> after guest unregisters shared buffer. At this time pages
>> will be unpinned.
>>
>> Guest can share buffer with OP-TEE for duration for one call,
>> or permanently, using OPTEE_MSG_CMD_REGISTER_SHM call. We need
>> to handle both options.
>>
>> Also we want to limit total size of shared buffers. As it is not
>> possible to get limit from OP-TEE, we need to choose some arbitrary
>> value. Currently limit is 16384 of 4K pages.
>
> I can't promise Xen will only be 4K only. So it would be better to
> make the number agnostic. Or at least writing clearly on top of the
> definition that it is assumed 4KB (maybe with a BUILD_BUG_ON(PAGE_SIZE
> != 4096) if not already in place).
I can replace define with something like (67108864 / PAGE_SIZE). With
appropriate comment, of course.
>
>>
>> Signed-off-by: Volodymyr Babchuk <vlad.babchuk@gmail.com>
>> ---
>> All the patches to optee.c should be merged together. They were
>> split to ease up review. But they depend heavily on each other.
>>
>> Changes from v3:
>> - Reworked pagelists storage - there is no more static storage for
>> 5 buffers, instead structure with all data is allocated dynamically
>> - Now this code uses domheap instead of xenheap
>> - Various style fixes
>> - gdprintk() fixes
>>
>> Changes from v2:
>> - Made sure that guest does not tries to register shared buffer with
>> the same cookie twice
>> - Fixed coding style
>> - Use access_guest_memory_by_ipa() instead of direct memory mapping
>> ---
>> xen/arch/arm/tee/optee.c | 404 +++++++++++++++++++++++++++++++++++++++
>> 1 file changed, 404 insertions(+)
>>
>> diff --git a/xen/arch/arm/tee/optee.c b/xen/arch/arm/tee/optee.c
>> index 291ed2fe25..14e295a422 100644
>> --- a/xen/arch/arm/tee/optee.c
>> +++ b/xen/arch/arm/tee/optee.c
>> @@ -24,6 +24,22 @@
>> #include <asm/tee/optee_msg.h>
>> #include <asm/tee/optee_smc.h>
>> +/*
>> + * "The return code is an error that originated within the underlying
>> + * communications stack linking the rich OS with the TEE" as described
>> + * in GP TEE Client API Specification.
>> + */
>> +#define TEEC_ORIGIN_COMMS 0x00000002
>> +
>> +/*
>> + * "Input parameters were invalid" as described
>> + * in GP TEE Client API Specification.
>> + */
>> +#define TEEC_ERROR_BAD_PARAMETERS 0xFFFF0006
>> +
>> +/* "System ran out of resources" */
>> +#define TEEC_ERROR_OUT_OF_MEMORY 0xFFFF000C
>> +
>> /* Client ID 0 is reserved for hypervisor itself */
>> #define OPTEE_CLIENT_ID(domain) ((domain)->domain_id + 1)
>> @@ -33,6 +49,15 @@
>> */
>> #define DEF_MAX_OPTEE_THREADS 16
>> +/*
>> + * Maximum total number of pages that guest can share with
>> + * OP-TEE. Currently value is selected arbitrary. Actual number of
>> + * pages depends on free heap in OP-TEE. As we can't do any
>> + * assumptions about OP-TEE heap usage, we limit number of pages
>> + * arbitrary.
>> + */
>> +#define MAX_TOTAL_SMH_BUF_PG 16384
>> +
>> #define OPTEE_KNOWN_NSEC_CAPS OPTEE_SMC_NSEC_CAP_UNIPROCESSOR
>> #define OPTEE_KNOWN_SEC_CAPS (OPTEE_SMC_SEC_CAP_HAVE_RESERVED_SHM | \
>> OPTEE_SMC_SEC_CAP_UNREGISTERED_SHM | \
>> @@ -65,11 +90,31 @@ struct shm_rpc {
>> uint64_t cookie;
>> };
>> +/* Shared memory buffer for arbitrary data */
>> +struct optee_shm_buf {
>> + struct list_head list;
>> + uint64_t cookie;
>> + unsigned int page_cnt;
>> + /*
>> + * Shadowed container for list of pages that guest tries to share
>> + * with OP-TEE. This is not the list of pages that guest shared
>> + * with OP-TEE, but container for list of those pages. Check
>> + * OPTEE_MSG_ATTR_NONCONTIG definition in optee_msg.h for more
>> + * information.
>> + */
>> + struct page_info *pg_list;
>> + unsigned int pg_list_order;
>> + /* Pinned guest pages that are shared with OP-TEE */
>> + struct page_info *pages[];
>> +};
>> +
>> /* Domain context */
>> struct optee_domain {
>> struct list_head call_list;
>> struct list_head shm_rpc_list;
>> + struct list_head optee_shm_buf_list;
>> atomic_t call_count;
>> + atomic_t optee_shm_buf_pages;
>> spinlock_t lock;
>> };
>> @@ -136,7 +181,9 @@ static int optee_domain_init(struct domain *d)
>> INIT_LIST_HEAD(&ctx->call_list);
>> INIT_LIST_HEAD(&ctx->shm_rpc_list);
>> + INIT_LIST_HEAD(&ctx->optee_shm_buf_list);
>> atomic_set(&ctx->call_count, 0);
>> + atomic_set(&ctx->optee_shm_buf_pages, 0);
>> spin_lock_init(&ctx->lock);
>> d->arch.tee = ctx;
>> @@ -363,10 +410,134 @@ static void free_shm_rpc(struct optee_domain *ctx, uint64_t cookie)
>> xfree(shm_rpc);
>> }
>> +static struct optee_shm_buf *allocate_optee_shm_buf(struct
>> optee_domain *ctx,
>> + uint64_t cookie,
>> + unsigned int pages_cnt,
>> + struct page_info *pg_list,
>> + unsigned int pg_list_order)
>> +{
>> + struct optee_shm_buf *optee_shm_buf, *optee_shm_buf_tmp;
>> + int old, new;
>> + int err_code;
>> +
>> + do
>> + {
>> + old = atomic_read(&ctx->optee_shm_buf_pages);
>> + new = old + pages_cnt;
>> + if ( new >= MAX_TOTAL_SMH_BUF_PG )
>
> Again, the limitation is in number of page and quite high. What would
> prevent a guest to register shared memory page by page? If nothing,
> then I think you can end up to interesting issues in Xen because of
> the growing list and memory used.
OP-TEE will limit this on it's side. In most cases Xen have much bigger
heap than OP-TEE :-)
Do you want me to limit this both in memory size and in number of buffers?
>
>> + return ERR_PTR(-ENOMEM);
>> + }
>> + while ( unlikely(old != atomic_cmpxchg(&ctx->optee_shm_buf_pages,
>> + old, new)) );
>> +
>> + optee_shm_buf = xzalloc_bytes(sizeof(struct optee_shm_buf) +
>> + pages_cnt * sizeof(struct page *));
>> + if ( !optee_shm_buf )
>> + {
>> + err_code = -ENOMEM;
>> + goto err;
>> + }
>> +
>> + optee_shm_buf->cookie = cookie;
>> + optee_shm_buf->pg_list = pg_list;
>> + optee_shm_buf->pg_list_order = pg_list_order;
>> +
>> + spin_lock(&ctx->lock);
>> + /* Check if there is already SHM with the same cookie */
>> + list_for_each_entry( optee_shm_buf_tmp, &ctx->optee_shm_buf_list, list )
>> + {
>> + if ( optee_shm_buf_tmp->cookie == cookie )
>> + {
>> + spin_unlock(&ctx->lock);
>> + gdprintk(XENLOG_WARNING, "Guest tries to use the same SHM buffer cookie %lx\n",
>
> The line looks too long. Please split after the first comma.
Ah, okay. I seen both variant in the code and was unsure which one is
right. Will do for all long messages.
[...]
>> static int optee_relinquish_resources(struct domain *d)
>> {
>> struct optee_std_call *call, *call_tmp;
>> struct shm_rpc *shm_rpc, *shm_rpc_tmp;
>> + struct optee_shm_buf *optee_shm_buf, *optee_shm_buf_tmp;
>> struct optee_domain *ctx = d->arch.tee;
>> if ( !ctx )
>> @@ -381,6 +552,13 @@ static int optee_relinquish_resources(struct domain *d)
>> list_for_each_entry_safe( shm_rpc, shm_rpc_tmp, &ctx->shm_rpc_list, list )
>> free_shm_rpc(ctx, shm_rpc->cookie);
>> + if ( hypercall_preempt_check() )
>> + return -ERESTART;
>
> Same question as the other hypercall_preempt_check().
Excuse me, but what question? I looked thru all your emails and can't
find one.
>> +
>> + list_for_each_entry_safe( optee_shm_buf, optee_shm_buf_tmp,
>> + &ctx->optee_shm_buf_list, list )
>> + free_optee_shm_buf(ctx, optee_shm_buf->cookie);
>> +
>> return 0;
>> }
>> @@ -406,11 +584,186 @@ static void optee_domain_destroy(struct
>> domain *d)
>> ASSERT(!spin_is_locked(&ctx->lock));
>> ASSERT(!atomic_read(&ctx->call_count));
>> + ASSERT(!atomic_read(&ctx->optee_shm_buf_pages));
>> ASSERT(list_empty(&ctx->shm_rpc_list));
>> XFREE(d->arch.tee);
>> }
>> +#define PAGELIST_ENTRIES_PER_PAGE \
>> + ((OPTEE_MSG_NONCONTIG_PAGE_SIZE / sizeof(u64)) - 1)
>> +
>> +static size_t get_pages_list_size(size_t num_entries)
>> +{
>> + int pages = DIV_ROUND_UP(num_entries, PAGELIST_ENTRIES_PER_PAGE);
>> +
>> + return pages * OPTEE_MSG_NONCONTIG_PAGE_SIZE;
>> +}
>> +
>> +static int translate_noncontig(struct optee_domain *ctx,
>> + struct optee_std_call *call,
>> + struct optee_msg_param *param)
>> +{
>> + uint64_t size;
>> + unsigned int page_offset;
>> + unsigned int num_pages;
>> + unsigned int order;
>> + unsigned int entries_on_page = 0;
>> + gfn_t gfn;
>> + p2m_type_t p2m;
>> + struct page_info *guest_page, *xen_pages;
>> + struct optee_shm_buf *optee_shm_buf;
>> + /*
>> + * This is memory layout for page list. Basically list consists of 4k pages,
>> + * every page store 511 page addresses of user buffer and page address of
>> + * the next page of list.
>> + *
>> + * Refer to OPTEE_MSG_ATTR_NONCONTIG description in optee_msg.h for details.
>> + */
>> + struct {
>> + uint64_t pages_list[PAGELIST_ENTRIES_PER_PAGE];
>> + uint64_t next_page_data;
>> + } *pages_data_guest, *pages_data_xen;
>> +
>> + /* Offset of user buffer withing page */
>
> Offset of the buffer within the OP-TEE page
>
>> + page_offset = param->u.tmem.buf_ptr & (OPTEE_MSG_NONCONTIG_PAGE_SIZE - 1);
>> +
>> + /* Size of the user buffer in bytes */
>> + size = ROUNDUP(param->u.tmem.size + page_offset,
>> + OPTEE_MSG_NONCONTIG_PAGE_SIZE);
>> +
>> + num_pages = DIV_ROUND_UP(size, OPTEE_MSG_NONCONTIG_PAGE_SIZE);
>> +
>> + order = get_order_from_bytes(get_pages_list_size(num_pages));
>> +
>> + xen_pages = alloc_domheap_pages(current->domain, order, 0);
>> + if ( !xen_pages )
>> + return -ENOMEM;
>> +
>> + optee_shm_buf = allocate_optee_shm_buf(ctx, param->u.tmem.shm_ref,
>> + num_pages, xen_pages, order);
>
> In an earlier version, I pointed out that you will allow to allocate
> up to 64MB per call. This is a potential security issue on Xen. While
> I said I would be happy to get this code merged as it, I would expect
> to at least see a TODO in the code explaining potential problem. So we
> know the problem exist and can't security support until this is fixed.
Sure. You want me to add this TODO there or inside
allocate_optee_shm_buf() function, where it actually does allocation?
> I may have missed other places while reviewing this version. Please go
> back in the review I have made in the past and document all the
> potential security holes.
Okay, will do.
>> + if ( IS_ERR(optee_shm_buf) )
>> + return PTR_ERR(optee_shm_buf);
>> +
>> + gfn = gaddr_to_gfn(param->u.tmem.buf_ptr &
>> + ~(OPTEE_MSG_NONCONTIG_PAGE_SIZE - 1));
>> +
>> + guest_page = get_page_from_gfn(current->domain, gfn_x(gfn), &p2m, P2M_ALLOC);
>> + if ( !guest_page || p2m != p2m_ram_rw )
>> + return -EINVAL;
>> +
>> + pages_data_guest = __map_domain_page(guest_page);
>> + pages_data_xen = __map_domain_page(xen_pages);
>> +
>> + while ( num_pages )
>> + {
>> + struct page_info *page;
>
> Newline here please.
>
>> + page = get_page_from_gfn(current->domain,
>> + paddr_to_pfn(pages_data_guest->pages_list[entries_on_page]),
>> + &p2m, P2M_ALLOC);
>
> The indentation is wrong here. But the problem is due to the long
> name. For instance, you have 3 times the word "page" on the same
> line. Is that necessary?
Code is quite complex. I want every variable to be as descriptive as
possible. In some cases it leads to issues like this :-)
I'll see what can be done there.
>> +
>> + if ( !page || p2m != p2m_ram_rw )
>> + goto err_unmap;
>> +
>> + optee_shm_buf->pages[optee_shm_buf->page_cnt++] = page;
>> + pages_data_xen->pages_list[entries_on_page] = page_to_maddr(page);
>> + entries_on_page++;
>> +
>> + if ( entries_on_page == PAGELIST_ENTRIES_PER_PAGE )
>> + {
>> + pages_data_xen->next_page_data = page_to_maddr(xen_pages + 1);
>> + unmap_domain_page(pages_data_xen);
>> + xen_pages++;
>> +
>> + gfn = gaddr_to_gfn(pages_data_guest->next_page_data);
>> +
>> + unmap_domain_page(pages_data_guest);
>> + put_page(guest_page);
>> +
>> + guest_page = get_page_from_gfn(current->domain, gfn_x(gfn), &p2m,
>> + P2M_ALLOC);
>> + if ( !guest_page || p2m != p2m_ram_rw )
>> + return -EINVAL;
>> +
>> + pages_data_guest = __map_domain_page(guest_page);
>> + pages_data_xen = __map_domain_page(xen_pages);
>> + /* Roll over to the next page */
>> + entries_on_page = 0;
>> + }
>> + num_pages--;
>> + }
>> +
>> + unmap_domain_page(pages_data_guest);
>> + unmap_domain_page(pages_data_xen);
>> + put_page(guest_page);
>> +
>> + param->u.tmem.buf_ptr = page_to_maddr(optee_shm_buf->pg_list) |
>> + page_offset;
>> +
>> + return 0;
>> +
>> +err_unmap:
>> + unmap_domain_page(pages_data_guest);
>> + unmap_domain_page(pages_data_xen);
>> + put_page(guest_page);
>> + free_optee_shm_buf(ctx, optee_shm_buf->cookie);
>> +
>> + return -EINVAL;
>> +}
>> +
>> +static int translate_params(struct optee_domain *ctx,
>> + struct optee_std_call *call)
>> +{
>> + unsigned int i;
>> + uint32_t attr;
>> + int ret = 0;
>> +
>> + for ( i = 0; i < call->xen_arg->num_params; i++ )
>> + {
>> + attr = call->xen_arg->params[i].attr;
>> +
>> + switch ( attr & OPTEE_MSG_ATTR_TYPE_MASK )
>> + {
>> + case OPTEE_MSG_ATTR_TYPE_TMEM_INPUT:
>> + case OPTEE_MSG_ATTR_TYPE_TMEM_OUTPUT:
>> + case OPTEE_MSG_ATTR_TYPE_TMEM_INOUT:
>> + if ( attr & OPTEE_MSG_ATTR_NONCONTIG )
>> + {
>> + ret = translate_noncontig(ctx, call, call->xen_arg->params + i);
>> + if ( ret )
>> + goto out;
>> + }
>> + else
>> + {
>> + gdprintk(XENLOG_WARNING, "Guest tries to use old tmem arg\n");
>> + ret = -EINVAL;
>> + goto out;
>> + }
>> + break;
>> + case OPTEE_MSG_ATTR_TYPE_NONE:
>> + case OPTEE_MSG_ATTR_TYPE_VALUE_INPUT:
>> + case OPTEE_MSG_ATTR_TYPE_VALUE_OUTPUT:
>> + case OPTEE_MSG_ATTR_TYPE_VALUE_INOUT:
>> + case OPTEE_MSG_ATTR_TYPE_RMEM_INPUT:
>> + case OPTEE_MSG_ATTR_TYPE_RMEM_OUTPUT:
>> + case OPTEE_MSG_ATTR_TYPE_RMEM_INOUT:
>> + continue;
>> + }
>> + }
>> +
>> +out:
>> + if ( ret )
>> + {
>> + call->xen_arg->ret_origin = TEEC_ORIGIN_COMMS;
>> + if ( ret == -ENOMEM )
>> + call->xen_arg->ret = TEEC_ERROR_OUT_OF_MEMORY;
>> + else
>> + call->xen_arg->ret = TEEC_ERROR_BAD_PARAMETERS;
>> + }
>> +
>> + return ret;
>> +}
>> +
>> /*
>> * Copy command buffer into domheap memory to:
>> * 1) Hide translated addresses from guest
>> @@ -537,6 +890,27 @@ static void copy_std_request_back(struct optee_domain *ctx,
>> put_page(page);
>> }
>> +
>> +static void free_shm_buffers(struct optee_domain *ctx,
>> + struct optee_msg_arg *arg)
>> +{
>> + unsigned int i;
>> +
>> + for ( i = 0; i < arg->num_params; i ++ )
>> + {
>> + switch ( arg->params[i].attr & OPTEE_MSG_ATTR_TYPE_MASK )
>> + {
>> + case OPTEE_MSG_ATTR_TYPE_TMEM_INPUT:
>> + case OPTEE_MSG_ATTR_TYPE_TMEM_OUTPUT:
>> + case OPTEE_MSG_ATTR_TYPE_TMEM_INOUT:
>> + free_optee_shm_buf(ctx, arg->params[i].u.tmem.shm_ref);
>> + break;
>> + default:
>> + break;
>> + }
>> + }
>> +}
>> +
>> /* Handle RPC return from OP-TEE */
>> static void handle_rpc_return(struct cpu_user_regs *regs,
>> struct optee_std_call *call)
>> @@ -556,6 +930,8 @@ static void handle_rpc_return(struct cpu_user_regs *regs,
>> * If this is RPC - we need to store call context and return back to guest.
>> * If call is complete - we need to return results with copy_std_request_back()
>> * and then we will destroy the call context as it is not needed anymore.
>> + *
>> + * Shared buffers should be handled in a special way.
>> */
>> static void execute_std_call(struct optee_domain *ctx,
>> struct cpu_user_regs *regs,
>> @@ -576,6 +952,23 @@ static void execute_std_call(struct optee_domain *ctx,
>> copy_std_request_back(ctx, regs, call);
>> + switch ( call->xen_arg->cmd )
>> + {
>> + case OPTEE_MSG_CMD_REGISTER_SHM:
>> + if ( call->xen_arg->ret == 0 )
>> + free_optee_shm_buf_pg_list(ctx,
>> + call->xen_arg->params[0].u.tmem.shm_ref);
>> + else
>> + free_optee_shm_buf(ctx, call->xen_arg->params[0].u.tmem.shm_ref);
>> + break;
>> + case OPTEE_MSG_CMD_UNREGISTER_SHM:
>> + if ( call->xen_arg->ret == 0 )
>> + free_optee_shm_buf(ctx, call->xen_arg->params[0].u.rmem.shm_ref);
>> + break;
>> + default:
>> + free_shm_buffers(ctx, call->xen_arg);
>> + }
>
> This switch seems to be new. Could you explain in a comment what they
> are supposed to do? For instance, why do you need to make
> UNREGISTER_SHM and REGISTER_SHM speific by specifying the exact
> buffer?
This is partially covered in comment above execute_std_call() function
definition. Basically, there are cases when guest shares memory temporary, for
duration of one call. And there is a case when guest registers shared
memory buffer on OP-TEE, so then the guest can refer the buffer in
subsequent calls. I'll put this in the comment.
>> +
>> put_std_call(ctx, call);
>> free_std_call(ctx, call);
>> }
>> @@ -606,6 +999,17 @@ static void handle_std_call(struct optee_domain *ctx,
>> case OPTEE_MSG_CMD_CANCEL:
>> case OPTEE_MSG_CMD_REGISTER_SHM:
>> case OPTEE_MSG_CMD_UNREGISTER_SHM:
>> + if( translate_params(ctx, call) )
>> + {
>> + /*
>> + * translate_params() sets xen_arg->ret value to non-zero.
>> + * So, technically, SMC was successful, but there was an error
>> + * during handling standard call encapsulated into this SMC.
>> + */
>> + copy_std_request_back(ctx, regs, call);
>> + set_return(regs, OPTEE_SMC_RETURN_OK);
>> + goto err;
>> + }
>> execute_std_call(ctx, regs, call);
>> return;
>> default:
>>
>
> Cheers,
--
Best regards,Volodymyr Babchuk
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 47+ messages in thread* Re: [PATCH v4 07/10] xen/arm: optee: add support for arbitrary shared memory
2019-03-20 16:39 ` Volodymyr Babchuk
@ 2019-03-20 17:47 ` Julien Grall
2019-03-20 19:37 ` Volodymyr Babchuk
0 siblings, 1 reply; 47+ messages in thread
From: Julien Grall @ 2019-03-20 17:47 UTC (permalink / raw)
To: Volodymyr Babchuk
Cc: tee-dev, xen-devel, Stefano Stabellini, Volodymyr Babchuk
Hi,
On 20/03/2019 16:39, Volodymyr Babchuk wrote:
>
> Julien Grall writes:
>
>> On 07/03/2019 21:04, Volodymyr Babchuk wrote:
>>> From: Volodymyr Babchuk <vlad.babchuk@gmail.com>
>>
>> Some of the patches are using your EPAM e-mail addresss. Other are
>> using your gmail address. Could you confirm this is expected?
>
> No, I'll make sure that all patches are authored by
> <volodymyr_babchuk@epam.com>
And your signed-off-by as well :).
[...]
>> I can't promise Xen will only be 4K only. So it would be better to
>> make the number agnostic. Or at least writing clearly on top of the
>> definition that it is assumed 4KB (maybe with a BUILD_BUG_ON(PAGE_SIZE
>> != 4096) if not already in place).
>
> I can replace define with something like (67108864 / PAGE_SIZE). With
> appropriate comment, of course.
Well, none of your code is ready for another PAGE_SIZE than 4KB. So the
BUILD_BUG_ON(PAGE_SIZE != 4096) is probably more suitable.
[...]
>>> +static struct optee_shm_buf *allocate_optee_shm_buf(struct
>>> optee_domain *ctx,
>>> + uint64_t cookie,
>>> + unsigned int pages_cnt,
>>> + struct page_info *pg_list,
>>> + unsigned int pg_list_order)
>>> +{
>>> + struct optee_shm_buf *optee_shm_buf, *optee_shm_buf_tmp;
>>> + int old, new;
>>> + int err_code;
>>> +
>>> + do
>>> + {
>>> + old = atomic_read(&ctx->optee_shm_buf_pages);
>>> + new = old + pages_cnt;
>>> + if ( new >= MAX_TOTAL_SMH_BUF_PG )
>>
>> Again, the limitation is in number of page and quite high. What would
>> prevent a guest to register shared memory page by page? If nothing,
>> then I think you can end up to interesting issues in Xen because of
>> the growing list and memory used.
>
> OP-TEE will limit this on it's side. In most cases Xen have much bigger
> heap than OP-TEE :-)
The main problem is not the heap. The problem is the size of the list to browse.
>
> Do you want me to limit this both in memory size and in number of buffers?
I am not necessarily asking to put a limitation. What I ask is documenting what
can happen. So we can take proper action in the future (such as when deciding
whether to security support it).
[...]
>
> [...]
>>> static int optee_relinquish_resources(struct domain *d)
>>> {
>>> struct optee_std_call *call, *call_tmp;
>>> struct shm_rpc *shm_rpc, *shm_rpc_tmp;
>>> + struct optee_shm_buf *optee_shm_buf, *optee_shm_buf_tmp;
>>> struct optee_domain *ctx = d->arch.tee;
>>> if ( !ctx )
>>> @@ -381,6 +552,13 @@ static int optee_relinquish_resources(struct domain *d)
>>> list_for_each_entry_safe( shm_rpc, shm_rpc_tmp, &ctx->shm_rpc_list, list )
>>> free_shm_rpc(ctx, shm_rpc->cookie);
>>> + if ( hypercall_preempt_check() )
>>> + return -ERESTART;
>>
>> Same question as the other hypercall_preempt_check().
>
> Excuse me, but what question? I looked thru all your emails and can't
> find one.
It looks like I have by mistake trimmed my question on the other patch :/.
Anyway, you should explain how you decide the placement of each
hypercall_preempt_check(). For instance, if the lists are quite big, then you
may want add a preempt check in the loop.
The key point here is to document the choice even if you wrote it is "random".
This will help in the future if we need to revise preemption choice.
>
>>> +
>>> + list_for_each_entry_safe( optee_shm_buf, optee_shm_buf_tmp,
>>> + &ctx->optee_shm_buf_list, list )
>>> + free_optee_shm_buf(ctx, optee_shm_buf->cookie);
>>> +
>>> return 0;
>>> }
>>> @@ -406,11 +584,186 @@ static void optee_domain_destroy(struct
>>> domain *d)
>>> ASSERT(!spin_is_locked(&ctx->lock));
>>> ASSERT(!atomic_read(&ctx->call_count));
>>> + ASSERT(!atomic_read(&ctx->optee_shm_buf_pages));
>>> ASSERT(list_empty(&ctx->shm_rpc_list));
>>> XFREE(d->arch.tee);
>>> }
>>> +#define PAGELIST_ENTRIES_PER_PAGE \
>>> + ((OPTEE_MSG_NONCONTIG_PAGE_SIZE / sizeof(u64)) - 1)
>>> +
>>> +static size_t get_pages_list_size(size_t num_entries)
>>> +{
>>> + int pages = DIV_ROUND_UP(num_entries, PAGELIST_ENTRIES_PER_PAGE);
>>> +
>>> + return pages * OPTEE_MSG_NONCONTIG_PAGE_SIZE;
>>> +}
>>> +
>>> +static int translate_noncontig(struct optee_domain *ctx,
>>> + struct optee_std_call *call,
>>> + struct optee_msg_param *param)
>>> +{
>>> + uint64_t size;
>>> + unsigned int page_offset;
>>> + unsigned int num_pages;
>>> + unsigned int order;
>>> + unsigned int entries_on_page = 0;
>>> + gfn_t gfn;
>>> + p2m_type_t p2m;
>>> + struct page_info *guest_page, *xen_pages;
>>> + struct optee_shm_buf *optee_shm_buf;
>>> + /*
>>> + * This is memory layout for page list. Basically list consists of 4k pages,
>>> + * every page store 511 page addresses of user buffer and page address of
>>> + * the next page of list.
>>> + *
>>> + * Refer to OPTEE_MSG_ATTR_NONCONTIG description in optee_msg.h for details.
>>> + */
>>> + struct {
>>> + uint64_t pages_list[PAGELIST_ENTRIES_PER_PAGE];
>>> + uint64_t next_page_data;
>>> + } *pages_data_guest, *pages_data_xen;
>>> +
>>> + /* Offset of user buffer withing page */
>>
>> Offset of the buffer within the OP-TEE page
>>
>>> + page_offset = param->u.tmem.buf_ptr & (OPTEE_MSG_NONCONTIG_PAGE_SIZE - 1);
>>> +
>>> + /* Size of the user buffer in bytes */
>>> + size = ROUNDUP(param->u.tmem.size + page_offset,
>>> + OPTEE_MSG_NONCONTIG_PAGE_SIZE);
>>> +
>>> + num_pages = DIV_ROUND_UP(size, OPTEE_MSG_NONCONTIG_PAGE_SIZE);
>>> +
>>> + order = get_order_from_bytes(get_pages_list_size(num_pages));
>>> +
>>> + xen_pages = alloc_domheap_pages(current->domain, order, 0);
>>> + if ( !xen_pages )
>>> + return -ENOMEM;
>>> +
>>> + optee_shm_buf = allocate_optee_shm_buf(ctx, param->u.tmem.shm_ref,
>>> + num_pages, xen_pages, order);
>>
>> In an earlier version, I pointed out that you will allow to allocate
>> up to 64MB per call. This is a potential security issue on Xen. While
>> I said I would be happy to get this code merged as it, I would expect
>> to at least see a TODO in the code explaining potential problem. So we
>> know the problem exist and can't security support until this is fixed.
> Sure. You want me to add this TODO there or inside
> allocate_optee_shm_buf() function, where it actually does allocation?
I don't mind the placement as long as the TODO is there.
>
>> I may have missed other places while reviewing this version. Please go
>> back in the review I have made in the past and document all the
>> potential security holes.
> Okay, will do.
>
>>> + if ( IS_ERR(optee_shm_buf) )
>>> + return PTR_ERR(optee_shm_buf);
>>> +
>>> + gfn = gaddr_to_gfn(param->u.tmem.buf_ptr &
>>> + ~(OPTEE_MSG_NONCONTIG_PAGE_SIZE - 1));
>>> +
>>> + guest_page = get_page_from_gfn(current->domain, gfn_x(gfn), &p2m, P2M_ALLOC);
>>> + if ( !guest_page || p2m != p2m_ram_rw )
>>> + return -EINVAL;
>>> +
>>> + pages_data_guest = __map_domain_page(guest_page);quite
>>> + pages_data_xen = __map_domain_page(xen_pages);
>>> +
>>> + while ( num_pages )
>>> + {
>>> + struct page_info *page;
>>
>> Newline here please.
>>
>>> + page = get_page_from_gfn(current->domain,
>>> + paddr_to_pfn(pages_data_guest->pages_list[entries_on_page]),
>>> + &p2m, P2M_ALLOC);
>>
>> The indentation is wrong here. But the problem is due to the long
>> name. For instance, you have 3 times the word "page" on the same
>> line. Is that necessary?
> Code is quite complex. I want every variable to be as descriptive as
> possible. In some cases it leads to issues like this :-)
There are way to simplify this code.
1) The OP-TEE code contains the following pattern a few time.
page = get_page_from_gfn(....)
if ( !page || p2mt != p2m_ram_rw )
...
You can create an helper to encapsulate that. I think you have one case where
the p2mt check is different. So potentially you want to patch the type check
in parameter.
2) You probably move __map_domain_page(guest_page/xen_pages) within the loop.
And just deal with guest_page/xen_pages outside.
With that and the renaming, then the code will become suddenly a bit less complex.
Cheers,
--
Julien Grall
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 47+ messages in thread* Re: [PATCH v4 07/10] xen/arm: optee: add support for arbitrary shared memory
2019-03-20 17:47 ` Julien Grall
@ 2019-03-20 19:37 ` Volodymyr Babchuk
2019-03-21 10:39 ` Julien Grall
0 siblings, 1 reply; 47+ messages in thread
From: Volodymyr Babchuk @ 2019-03-20 19:37 UTC (permalink / raw)
To: Julien Grall
Cc: tee-dev, xen-devel, Stefano Stabellini, Volodymyr Babchuk,
Volodymyr Babchuk
Julien Grall writes:
>>> Some of the patches are using your EPAM e-mail addresss. Other are
>>> using your gmail address. Could you confirm this is expected?
>>
>> No, I'll make sure that all patches are authored by
>> <volodymyr_babchuk@epam.com>
>
> And your signed-off-by as well :).
Sure :)
>
> [...]
>
>>> I can't promise Xen will only be 4K only. So it would be better to
>>> make the number agnostic. Or at least writing clearly on top of the
>>> definition that it is assumed 4KB (maybe with a BUILD_BUG_ON(PAGE_SIZE
>>> != 4096) if not already in place).
>>
>> I can replace define with something like (67108864 / PAGE_SIZE). With
>> appropriate comment, of course.
>
> Well, none of your code is ready for another PAGE_SIZE than 4KB. So
> the BUILD_BUG_ON(PAGE_SIZE != 4096) is probably more suitable.
Yes, makes sense.
> [...]
>
>>>> +static struct optee_shm_buf *allocate_optee_shm_buf(struct
>>>> optee_domain *ctx,
>>>> + uint64_t cookie,
>>>> + unsigned int pages_cnt,
>>>> + struct page_info *pg_list,
>>>> + unsigned int pg_list_order)
>>>> +{
>>>> + struct optee_shm_buf *optee_shm_buf, *optee_shm_buf_tmp;
>>>> + int old, new;
>>>> + int err_code;
>>>> +
>>>> + do
>>>> + {
>>>> + old = atomic_read(&ctx->optee_shm_buf_pages);
>>>> + new = old + pages_cnt;
>>>> + if ( new >= MAX_TOTAL_SMH_BUF_PG )
>>>
>>> Again, the limitation is in number of page and quite high. What would
>>> prevent a guest to register shared memory page by page? If nothing,
>>> then I think you can end up to interesting issues in Xen because of
>>> the growing list and memory used.
>>
>> OP-TEE will limit this on it's side. In most cases Xen have much bigger
>> heap than OP-TEE :-)
>
> The main problem is not the heap. The problem is the size of the list to browse.
>>
>> Do you want me to limit this both in memory size and in number of buffers?
>
> I am not necessarily asking to put a limitation. What I ask is
> documenting what can happen. So we can take proper action in the
> future (such as when deciding whether to security support it).
>
Ah, okay, got it.
> [...]
>
>>
>> [...]
>>>> static int optee_relinquish_resources(struct domain *d)
>>>> {
>>>> struct optee_std_call *call, *call_tmp;
>>>> struct shm_rpc *shm_rpc, *shm_rpc_tmp;
>>>> + struct optee_shm_buf *optee_shm_buf, *optee_shm_buf_tmp;
>>>> struct optee_domain *ctx = d->arch.tee;
>>>> if ( !ctx )
>>>> @@ -381,6 +552,13 @@ static int optee_relinquish_resources(struct domain *d)
>>>> list_for_each_entry_safe( shm_rpc, shm_rpc_tmp, &ctx->shm_rpc_list, list )
>>>> free_shm_rpc(ctx, shm_rpc->cookie);
>>>> + if ( hypercall_preempt_check() )
>>>> + return -ERESTART;
>>>
>>> Same question as the other hypercall_preempt_check().
>>
>> Excuse me, but what question? I looked thru all your emails and can't
>> find one.
>
> It looks like I have by mistake trimmed my question on the other patch :/.
>
> Anyway, you should explain how you decide the placement of each
> hypercall_preempt_check(). For instance, if the lists are quite big,
> then you may want add a preempt check in the loop.
I see your point there. Check in the loop would require additional
logic. Is there any best practices? E.g. how often I should check for
preemption? Every N iterations of loop, where N is...
> The key point here is to document the choice even if you wrote it is
> "random". This will help in the future if we need to revise preemption
> choice.
I'd prefer to do proper fix, if it does not require a big amount of
work. Otherwise I'll add document the current state.
[...]
>>>> + page = get_page_from_gfn(current->domain,
>>>> + paddr_to_pfn(pages_data_guest->pages_list[entries_on_page]),
>>>> + &p2m, P2M_ALLOC);
>>>
>>> The indentation is wrong here. But the problem is due to the long
>>> name. For instance, you have 3 times the word "page" on the same
>>> line. Is that necessary?
>> Code is quite complex. I want every variable to be as descriptive as
>> possible. In some cases it leads to issues like this :-)
>
> There are way to simplify this code.
>
> 1) The OP-TEE code contains the following pattern a few time.
>
> page = get_page_from_gfn(....)
> if ( !page || p2mt != p2m_ram_rw )
> ...
>
> You can create an helper to encapsulate that. I think you have one case where
> the p2mt check is different. So potentially you want to patch the type check
> in parameter.
Yes, this a good idea.
> 2) You probably move __map_domain_page(guest_page/xen_pages) within
> the loop. And just deal with guest_page/xen_pages outside.
>
> With that and the renaming, then the code will become suddenly a bit less complex.
Thank you for recommendations. I'll try to do in this way.
--
Best regards,Volodymyr Babchuk
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 47+ messages in thread* Re: [PATCH v4 07/10] xen/arm: optee: add support for arbitrary shared memory
2019-03-20 19:37 ` Volodymyr Babchuk
@ 2019-03-21 10:39 ` Julien Grall
0 siblings, 0 replies; 47+ messages in thread
From: Julien Grall @ 2019-03-21 10:39 UTC (permalink / raw)
To: Volodymyr Babchuk
Cc: tee-dev, xen-devel, Stefano Stabellini, Volodymyr Babchuk
Hi,
On 3/20/19 7:37 PM, Volodymyr Babchuk wrote:
>
> Julien Grall writes:
>> Anyway, you should explain how you decide the placement of each
>> hypercall_preempt_check(). For instance, if the lists are quite big,
>> then you may want add a preempt check in the loop.
> I see your point there. Check in the loop would require additional
> logic. Is there any best practices? E.g. how often I should check for
> preemption? Every N iterations of loop, where N is...
It depends on how complex are each iteration of the loop. For instance,
in the p2m code, we check preemption every 512 mappings freed (see
relinquish_p2m_mapping).
>
>> The key point here is to document the choice even if you wrote it is
>> "random". This will help in the future if we need to revise preemption
>> choice.
> I'd prefer to do proper fix, if it does not require a big amount of
> work. Otherwise I'll add document the current state.
proper fix and preemption cannot be in the same sentence :). This is a
judgment call based on the complexity of the code.
In the example I gave above, we decided arbitrarily to preempt every 512
iterations. We didn't have any number but though 512 iterations was a
good number.
I don't expect you to find the best place right now. I am only asking to
document just we know the choice was "arbitrary".
Cheers,
--
Julien Grall
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 47+ messages in thread
* [PATCH v4 06/10] xen/arm: optee: add support for RPC SHM buffers
2019-03-07 21:04 [PATCH v4 00/10] TEE mediator (and OP-TEE) support in XEN Volodymyr Babchuk
` (5 preceding siblings ...)
2019-03-07 21:04 ` [PATCH v4 07/10] xen/arm: optee: add support for arbitrary shared memory Volodymyr Babchuk
@ 2019-03-07 21:04 ` Volodymyr Babchuk
2019-03-18 14:21 ` Julien Grall
2019-03-07 21:04 ` [PATCH v4 09/10] tools/arm: tee: add "tee" option for xl.cfg Volodymyr Babchuk
` (2 subsequent siblings)
9 siblings, 1 reply; 47+ messages in thread
From: Volodymyr Babchuk @ 2019-03-07 21:04 UTC (permalink / raw)
To: xen-devel, tee-dev
Cc: Volodymyr Babchuk, Julien Grall, Stefano Stabellini, Volodymyr Babchuk
From: Volodymyr Babchuk <vlad.babchuk@gmail.com>
OP-TEE usually uses the same idea with command buffers (see
previous commit) to issue RPC requests. Problem is that initially
it has no buffer, where it can write request. So the first RPC
request it makes is special: it requests NW to allocate shared
buffer for other RPC requests. Usually this buffer is allocated
only once for every OP-TEE thread and it remains allocated all
the time until guest shuts down. Guest can ask OP-TEE to disable
RPC buffers caching, in this case OP-TEE will ask guest to
allocate/free buffer for the each RPC.
Mediator needs to pin this buffer to make sure that domain can't
transfer it to someone else.
Life cycle of this buffer is controlled by OP-TEE. It asks guest
to create buffer and it asks it to free it. So it there is no much
sense to limit number of those buffers, as we limited number of
concurrent standard calls, because this can impair functionality of
OP-TEE.
OP-TEE expects that this RPC buffer have size of
OPTEE_MSG_NONCONTIG_PAGE_SIZE, which equals to 4096 and is aligned
with the same size. So, basically it expects one 4k page from the
guest. This is the same as Xen's PAGE_SIZE.
Signed-off-by: Volodymyr Babchuk <vlad.babchuk@gmail.com>
---
All the patches to optee.c should be merged together. They were
split to ease up review. But they depend heavily on each other.
Changes from v3:
- Removed MAX_RPC_SHMS constant. Now this value depends on
number of OP-TEE threads
- Various formatting fixes
- Added checks for guest memory type
Changes from v2:
- Added check to ensure that guests does not return two SHM buffers
with the same cookie
- Fixed coding style
- Storing RPC parameters during RPC return to make sure, that guest
will not change them during call continuation
---
xen/arch/arm/tee/optee.c | 133 ++++++++++++++++++++++++++++++++++++++-
1 file changed, 131 insertions(+), 2 deletions(-)
diff --git a/xen/arch/arm/tee/optee.c b/xen/arch/arm/tee/optee.c
index 66517ebf6f..291ed2fe25 100644
--- a/xen/arch/arm/tee/optee.c
+++ b/xen/arch/arm/tee/optee.c
@@ -18,6 +18,7 @@
#include <xen/mm.h>
#include <xen/sched.h>
+#include <asm/event.h>
#include <asm/smccc.h>
#include <asm/tee/tee.h>
#include <asm/tee/optee_msg.h>
@@ -54,11 +55,20 @@ struct optee_std_call {
int optee_thread_id;
int rpc_op;
bool in_flight;
+ register_t rpc_params[2];
+};
+
+/* Pre-allocated SHM buffer for RPC commands */
+struct shm_rpc {
+ struct list_head list;
+ struct page_info *guest_page;
+ uint64_t cookie;
};
/* Domain context */
struct optee_domain {
struct list_head call_list;
+ struct list_head shm_rpc_list;
atomic_t call_count;
spinlock_t lock;
};
@@ -125,6 +135,7 @@ static int optee_domain_init(struct domain *d)
}
INIT_LIST_HEAD(&ctx->call_list);
+ INIT_LIST_HEAD(&ctx->shm_rpc_list);
atomic_set(&ctx->call_count, 0);
spin_lock_init(&ctx->lock);
@@ -281,9 +292,81 @@ static void put_std_call(struct optee_domain *ctx, struct optee_std_call *call)
spin_unlock(&ctx->lock);
}
+static struct shm_rpc *allocate_and_pin_shm_rpc(struct optee_domain *ctx,
+ gfn_t gfn, uint64_t cookie)
+{
+ struct shm_rpc *shm_rpc, *shm_rpc_tmp;
+ p2m_type_t t;
+
+ shm_rpc = xzalloc(struct shm_rpc);
+ if ( !shm_rpc )
+ return ERR_PTR(-ENOMEM);
+
+ /* This page will be shared with OP-TEE, so we need to pin it. */
+ shm_rpc->guest_page = get_page_from_gfn(current->domain, gfn_x(gfn), &t,
+ P2M_ALLOC);
+ if ( !shm_rpc->guest_page || t != p2m_ram_rw )
+ goto err;
+
+ shm_rpc->cookie = cookie;
+
+ spin_lock(&ctx->lock);
+ /* Check if there is existing SHM with the same cookie. */
+ list_for_each_entry( shm_rpc_tmp, &ctx->shm_rpc_list, list )
+ {
+ if ( shm_rpc_tmp->cookie == cookie )
+ {
+ spin_unlock(&ctx->lock);
+ gdprintk(XENLOG_WARNING, "Guest tries to use the same RPC SHM cookie %lx\n",
+ cookie);
+ goto err;
+ }
+ }
+
+ list_add_tail(&shm_rpc->list, &ctx->shm_rpc_list);
+ spin_unlock(&ctx->lock);
+
+ return shm_rpc;
+
+err:
+ if ( shm_rpc->guest_page )
+ put_page(shm_rpc->guest_page);
+ xfree(shm_rpc);
+
+ return ERR_PTR(-EINVAL);
+}
+
+static void free_shm_rpc(struct optee_domain *ctx, uint64_t cookie)
+{
+ struct shm_rpc *shm_rpc;
+ bool found = false;
+
+ spin_lock(&ctx->lock);
+
+ list_for_each_entry( shm_rpc, &ctx->shm_rpc_list, list )
+ {
+ if ( shm_rpc->cookie == cookie )
+ {
+ found = true;
+ list_del(&shm_rpc->list);
+ break;
+ }
+ }
+ spin_unlock(&ctx->lock);
+
+ if ( !found )
+ return;
+
+ ASSERT(shm_rpc->guest_page);
+ put_page(shm_rpc->guest_page);
+
+ xfree(shm_rpc);
+}
+
static int optee_relinquish_resources(struct domain *d)
{
struct optee_std_call *call, *call_tmp;
+ struct shm_rpc *shm_rpc, *shm_rpc_tmp;
struct optee_domain *ctx = d->arch.tee;
if ( !ctx )
@@ -292,6 +375,12 @@ static int optee_relinquish_resources(struct domain *d)
list_for_each_entry_safe( call, call_tmp, &ctx->call_list, list )
free_std_call(ctx, call);
+ if ( hypercall_preempt_check() )
+ return -ERESTART;
+
+ list_for_each_entry_safe( shm_rpc, shm_rpc_tmp, &ctx->shm_rpc_list, list )
+ free_shm_rpc(ctx, shm_rpc->cookie);
+
return 0;
}
@@ -317,6 +406,7 @@ static void optee_domain_destroy(struct domain *d)
ASSERT(!spin_is_locked(&ctx->lock));
ASSERT(!atomic_read(&ctx->call_count));
+ ASSERT(list_empty(&ctx->shm_rpc_list));
XFREE(d->arch.tee);
}
@@ -451,6 +541,8 @@ static void copy_std_request_back(struct optee_domain *ctx,
static void handle_rpc_return(struct cpu_user_regs *regs,
struct optee_std_call *call)
{
+ call->rpc_params[0] = get_user_reg(regs, 1);
+ call->rpc_params[1] = get_user_reg(regs, 2);
call->optee_thread_id = get_user_reg(regs, 3);
call->rpc_op = OPTEE_SMC_RETURN_GET_RPC_FUNC(get_user_reg(regs, 0));
}
@@ -528,6 +620,39 @@ err:
return;
}
+static void handle_rpc_func_alloc(struct optee_domain *ctx,
+ struct cpu_user_regs *regs)
+{
+ struct shm_rpc *shm_rpc;
+ paddr_t ptr = regpair_to_uint64(regs, 1);
+ uint64_t cookie = regpair_to_uint64(regs, 4);
+
+ if ( ptr & (OPTEE_MSG_NONCONTIG_PAGE_SIZE - 1) )
+ {
+ gdprintk(XENLOG_WARNING, "Domain returned invalid RPC command buffer\n");
+ /*
+ * OP-TEE is waiting for a response to the RPC. We can't just
+ * return error to the guest. We need to provide some invalid
+ * value to OP-TEE, so it can handle error on its side.
+ */
+ ptr = 0;
+ goto out;
+ }
+
+ shm_rpc = allocate_and_pin_shm_rpc(ctx, gaddr_to_gfn(ptr), cookie);
+ if ( IS_ERR(shm_rpc) )
+ {
+ gdprintk(XENLOG_WARNING, "Failed to allocate shm_rpc object: %ld\n",
+ PTR_ERR(shm_rpc));
+ ptr = 0;
+ }
+ else
+ ptr = page_to_maddr(shm_rpc->guest_page);
+
+out:
+ uint64_to_regpair(regs, 1, ptr);
+}
+
static void handle_rpc(struct optee_domain *ctx, struct cpu_user_regs *regs)
{
struct optee_std_call *call;
@@ -551,11 +676,15 @@ static void handle_rpc(struct optee_domain *ctx, struct cpu_user_regs *regs)
switch ( call->rpc_op )
{
case OPTEE_SMC_RPC_FUNC_ALLOC:
- /* TODO: Add handling */
+ handle_rpc_func_alloc(ctx, regs);
break;
case OPTEE_SMC_RPC_FUNC_FREE:
- /* TODO: Add handling */
+ {
+ uint64_t cookie = (uint64_t)call->rpc_params[0] << 32 |
+ (uint32_t)call->rpc_params[1];
+ free_shm_rpc(ctx, cookie);
break;
+ }
case OPTEE_SMC_RPC_FUNC_FOREIGN_INTR:
break;
case OPTEE_SMC_RPC_FUNC_CMD:
--
2.21.0
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply related [flat|nested] 47+ messages in thread* Re: [PATCH v4 06/10] xen/arm: optee: add support for RPC SHM buffers
2019-03-07 21:04 ` [PATCH v4 06/10] xen/arm: optee: add support for RPC SHM buffers Volodymyr Babchuk
@ 2019-03-18 14:21 ` Julien Grall
2019-03-20 16:21 ` Volodymyr Babchuk
0 siblings, 1 reply; 47+ messages in thread
From: Julien Grall @ 2019-03-18 14:21 UTC (permalink / raw)
To: Volodymyr Babchuk, xen-devel, tee-dev
Cc: Volodymyr Babchuk, Stefano Stabellini
Hi Volodymyr,
On 07/03/2019 21:04, Volodymyr Babchuk wrote:
> From: Volodymyr Babchuk <vlad.babchuk@gmail.com>
>
> OP-TEE usually uses the same idea with command buffers (see
> previous commit) to issue RPC requests. Problem is that initially
> it has no buffer, where it can write request. So the first RPC
> request it makes is special: it requests NW to allocate shared
> buffer for other RPC requests. Usually this buffer is allocated
> only once for every OP-TEE thread and it remains allocated all
> the time until guest shuts down. Guest can ask OP-TEE to disable
> RPC buffers caching, in this case OP-TEE will ask guest to
> allocate/free buffer for the each RPC.
>
> Mediator needs to pin this buffer to make sure that domain can't
> transfer it to someone else.
At the moment, Xen on Arm doesn't support transfer of a page between domain (see
steal_page). What we want to prevent here is the domain to free the page (via
XENMEM_decrease_reservation). If the reference drop to 0, the page will be freed
and could potentially be allocated for Xen usage or another domain. Taking the
reference here, will prevent it to free until the reference is dropped.
So I would reword this sentence. Something like:
"Mediator needs to pin the buffer to make sure the page will not be freed while
it is shared with OP-TEE".
>
> Life cycle of this buffer is controlled by OP-TEE. It asks guest
> to create buffer and it asks it to free it. So it there is no much
NIT: s/no/not/
> sense to limit number of those buffers, as we limited number of
NIT: s/limited/already limit the/
> concurrent standard calls, because this can impair functionality of
> OP-TEE.
Could you add a similar comment on top of call_count?
The code looks good to me.
Cheers,
--
Julien Grall
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 47+ messages in thread
* Re: [PATCH v4 06/10] xen/arm: optee: add support for RPC SHM buffers
2019-03-18 14:21 ` Julien Grall
@ 2019-03-20 16:21 ` Volodymyr Babchuk
2019-03-20 16:52 ` Julien Grall
0 siblings, 1 reply; 47+ messages in thread
From: Volodymyr Babchuk @ 2019-03-20 16:21 UTC (permalink / raw)
To: Julien Grall
Cc: tee-dev, xen-devel, Stefano Stabellini, Volodymyr Babchuk,
Volodymyr Babchuk
Julien Grall writes:
Hi Julien,
> Hi Volodymyr,
>
> On 07/03/2019 21:04, Volodymyr Babchuk wrote:
>> From: Volodymyr Babchuk <vlad.babchuk@gmail.com>
>>
>> OP-TEE usually uses the same idea with command buffers (see
>> previous commit) to issue RPC requests. Problem is that initially
>> it has no buffer, where it can write request. So the first RPC
>> request it makes is special: it requests NW to allocate shared
>> buffer for other RPC requests. Usually this buffer is allocated
>> only once for every OP-TEE thread and it remains allocated all
>> the time until guest shuts down. Guest can ask OP-TEE to disable
>> RPC buffers caching, in this case OP-TEE will ask guest to
>> allocate/free buffer for the each RPC.
>>
>> Mediator needs to pin this buffer to make sure that domain can't
>> transfer it to someone else.
>
> At the moment, Xen on Arm doesn't support transfer of a page between
> domain (see steal_page). What we want to prevent here is the domain to
> free the page (via XENMEM_decrease_reservation). If the reference drop
> to 0, the page will be freed and could potentially be allocated for
> Xen usage or another domain. Taking the reference here, will prevent
> it to free until the reference is dropped.
>
> So I would reword this sentence. Something like:
>
> "Mediator needs to pin the buffer to make sure the page will not be
> freed while it is shared with OP-TEE".
When I wrote that, I kept in mind grant mappings as well. But, I
think, I was mistaken there, because grant mappings will have p2m type
p2m_grant_map_rw/p2m_grant_map_ro. So yeah, your variant is better. Thanks.
[...]
>> concurrent standard calls, because this can impair functionality of
>> OP-TEE.
>
> Could you add a similar comment on top of call_count?
Sure, will do.
--
Best regards,Volodymyr Babchuk
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 47+ messages in thread
* Re: [PATCH v4 06/10] xen/arm: optee: add support for RPC SHM buffers
2019-03-20 16:21 ` Volodymyr Babchuk
@ 2019-03-20 16:52 ` Julien Grall
2019-03-20 17:09 ` Volodymyr Babchuk
0 siblings, 1 reply; 47+ messages in thread
From: Julien Grall @ 2019-03-20 16:52 UTC (permalink / raw)
To: Volodymyr Babchuk
Cc: tee-dev, xen-devel, Stefano Stabellini, Volodymyr Babchuk
Hi,
On 20/03/2019 16:21, Volodymyr Babchuk wrote:
>
> Julien Grall writes:
>> On 07/03/2019 21:04, Volodymyr Babchuk wrote:
>>> From: Volodymyr Babchuk <vlad.babchuk@gmail.com>
>>>
>>> OP-TEE usually uses the same idea with command buffers (see
>>> previous commit) to issue RPC requests. Problem is that initially
>>> it has no buffer, where it can write request. So the first RPC
>>> request it makes is special: it requests NW to allocate shared
>>> buffer for other RPC requests. Usually this buffer is allocated
>>> only once for every OP-TEE thread and it remains allocated all
>>> the time until guest shuts down. Guest can ask OP-TEE to disable
>>> RPC buffers caching, in this case OP-TEE will ask guest to
>>> allocate/free buffer for the each RPC.
>>>
>>> Mediator needs to pin this buffer to make sure that domain can't
>>> transfer it to someone else.
>>
>> At the moment, Xen on Arm doesn't support transfer of a page between
>> domain (see steal_page). What we want to prevent here is the domain to
>> free the page (via XENMEM_decrease_reservation). If the reference drop
>> to 0, the page will be freed and could potentially be allocated for
>> Xen usage or another domain. Taking the reference here, will prevent
>> it to free until the reference is dropped.
>>
>> So I would reword this sentence. Something like:
>>
>> "Mediator needs to pin the buffer to make sure the page will not be
>> freed while it is shared with OP-TEE".
> When I wrote that, I kept in mind grant mappings as well. But, I
> think, I was mistaken there, because grant mappings will have p2m type
> p2m_grant_map_rw/p2m_grant_map_ro. So yeah, your variant is better. Thanks.
I don't understand how grant mappings are related to this patch. What did you
have in mind about them?
Cheers,
--
Julien Grall
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 47+ messages in thread
* Re: [PATCH v4 06/10] xen/arm: optee: add support for RPC SHM buffers
2019-03-20 16:52 ` Julien Grall
@ 2019-03-20 17:09 ` Volodymyr Babchuk
0 siblings, 0 replies; 47+ messages in thread
From: Volodymyr Babchuk @ 2019-03-20 17:09 UTC (permalink / raw)
To: Julien Grall
Cc: tee-dev, xen-devel, Stefano Stabellini, Volodymyr Babchuk,
Volodymyr Babchuk
Julien Grall writes:
> Hi,
>
> On 20/03/2019 16:21, Volodymyr Babchuk wrote:
>>
>> Julien Grall writes:
>>> On 07/03/2019 21:04, Volodymyr Babchuk wrote:
>>>> From: Volodymyr Babchuk <vlad.babchuk@gmail.com>
>>>>
>>>> OP-TEE usually uses the same idea with command buffers (see
>>>> previous commit) to issue RPC requests. Problem is that initially
>>>> it has no buffer, where it can write request. So the first RPC
>>>> request it makes is special: it requests NW to allocate shared
>>>> buffer for other RPC requests. Usually this buffer is allocated
>>>> only once for every OP-TEE thread and it remains allocated all
>>>> the time until guest shuts down. Guest can ask OP-TEE to disable
>>>> RPC buffers caching, in this case OP-TEE will ask guest to
>>>> allocate/free buffer for the each RPC.
>>>>
>>>> Mediator needs to pin this buffer to make sure that domain can't
>>>> transfer it to someone else.
>>>
>>> At the moment, Xen on Arm doesn't support transfer of a page between
>>> domain (see steal_page). What we want to prevent here is the domain to
>>> free the page (via XENMEM_decrease_reservation). If the reference drop
>>> to 0, the page will be freed and could potentially be allocated for
>>> Xen usage or another domain. Taking the reference here, will prevent
>>> it to free until the reference is dropped.
>>>
>>> So I would reword this sentence. Something like:
>>>
>>> "Mediator needs to pin the buffer to make sure the page will not be
>>> freed while it is shared with OP-TEE".
>> When I wrote that, I kept in mind grant mappings as well. But, I
>> think, I was mistaken there, because grant mappings will have p2m type
>> p2m_grant_map_rw/p2m_grant_map_ro. So yeah, your variant is better. Thanks.
>
> I don't understand how grant mappings are related to this patch. What
> did you have in mind about them?
For some reason I assumed that following scenario would be possible:
Guest A grants access to own page to guest B. Guest B maps this page and
then shares it with OP-TEE.
My concern was that Guest B than can unmap granted page and release
grant, but it still be able to alter data on that page via OP-TEE.
But, looks I was mistaken there, because currently I'm checking p2m type to
be strictly p2m_ram_rw before sharing the page with OP-TEE. So mentioned
scenario is impossible in the first place.
--
Best regards,Volodymyr Babchuk
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 47+ messages in thread
* [PATCH v4 09/10] tools/arm: tee: add "tee" option for xl.cfg
2019-03-07 21:04 [PATCH v4 00/10] TEE mediator (and OP-TEE) support in XEN Volodymyr Babchuk
` (6 preceding siblings ...)
2019-03-07 21:04 ` [PATCH v4 06/10] xen/arm: optee: add support for RPC SHM buffers Volodymyr Babchuk
@ 2019-03-07 21:04 ` Volodymyr Babchuk
2019-03-18 15:49 ` Julien Grall
2019-03-07 21:04 ` [PATCH v4 08/10] xen/arm: optee: add support for RPC commands Volodymyr Babchuk
2019-03-07 21:04 ` [PATCH v4 10/10] tools/arm: optee: create optee firmware node in DT if tee=native Volodymyr Babchuk
9 siblings, 1 reply; 47+ messages in thread
From: Volodymyr Babchuk @ 2019-03-07 21:04 UTC (permalink / raw)
To: xen-devel, tee-dev; +Cc: Volodymyr Babchuk, Wei Liu, Ian Jackson
From: Volodymyr Babchuk <vlad.babchuk@gmail.com>
This enumeration controls TEE type for a domain. Currently there is
two possible options: either 'none' or 'native'.
'none' is the default value and it basically disables TEE support at
all.
'native' enables access to a "real" TEE installed on a platform.
It is possible to add another types in the future.
Signed-off-by: Volodymyr Babchuk <vlad.babchuk@gmail.com>
---
All the patches to optee.c should be merged together. They were
split to ease up review. But they depend heavily on each other.
Changes from v3:
- tee_enabled renamed to tee_type. Currently two types are supported
as described in the commit message
- Add LIBXL_HAVE_BUILDINFO_ARCH_ARM_TEE definition
Changes from v2:
- Use arch.tee_enabled instead of separate domctl
---
docs/man/xl.cfg.5.pod.in | 12 ++++++++++++
tools/libxl/libxl.h | 5 +++++
tools/libxl/libxl_arm.c | 13 +++++++++++++
tools/libxl/libxl_types.idl | 6 ++++++
tools/xl/xl_parse.c | 9 +++++++++
5 files changed, 45 insertions(+)
diff --git a/docs/man/xl.cfg.5.pod.in b/docs/man/xl.cfg.5.pod.in
index ad81af1ed8..e15981882b 100644
--- a/docs/man/xl.cfg.5.pod.in
+++ b/docs/man/xl.cfg.5.pod.in
@@ -2702,6 +2702,18 @@ Currently, only the "sbsa_uart" model is supported for ARM.
=back
+=over 4
+
+=item B<tee=["none", "native"]>
+
+Set TEE type for the guest. Currently only OP-TEE is supported. If
+this option is set to "native", xl will create guest, which can access
+native TEE on your system (just make sure that you are using OP-TEE
+with virtualization support endabled). Also OP-TEE node will be
+emitted into guest's device tree.
+
+=back
+
=head3 x86
=over 4
diff --git a/tools/libxl/libxl.h b/tools/libxl/libxl.h
index a38e5cdba2..b24e4141b1 100644
--- a/tools/libxl/libxl.h
+++ b/tools/libxl/libxl.h
@@ -273,6 +273,11 @@
*/
#define LIBXL_HAVE_BUILDINFO_ARM_GIC_VERSION 1
+/*
+ * libxl_domain_build_info has the arch_arm.tee field.
+ */
+#define LIBXL_HAVE_BUILDINFO_ARCH_ARM_TEE 1
+
/*
* LIBXL_HAVE_SOFT_RESET indicates that libxl supports performing
* 'soft reset' for domains and there is 'soft_reset' shutdown reason
diff --git a/tools/libxl/libxl_arm.c b/tools/libxl/libxl_arm.c
index 141e159043..6930d0ab3b 100644
--- a/tools/libxl/libxl_arm.c
+++ b/tools/libxl/libxl_arm.c
@@ -89,6 +89,19 @@ int libxl__arch_domain_prepare_config(libxl__gc *gc,
return ERROR_FAIL;
}
+ switch (d_config->b_info.arch_arm.tee) {
+ case LIBXL_TEE_TYPE_NONE:
+ config->arch.tee_type = XEN_DOMCTL_CONFIG_TEE_NONE;
+ break;
+ case LIBXL_TEE_TYPE_NATIVE:
+ config->arch.tee_type = XEN_DOMCTL_CONFIG_TEE_NATIVE;
+ break;
+ default:
+ LOG(ERROR, "Unknown TEE type %d",
+ d_config->b_info.arch_arm.tee);
+ return ERROR_FAIL;
+ }
+
return 0;
}
diff --git a/tools/libxl/libxl_types.idl b/tools/libxl/libxl_types.idl
index b685ac47ac..4f1eb229b8 100644
--- a/tools/libxl/libxl_types.idl
+++ b/tools/libxl/libxl_types.idl
@@ -457,6 +457,11 @@ libxl_gic_version = Enumeration("gic_version", [
(0x30, "v3")
], init_val = "LIBXL_GIC_VERSION_DEFAULT")
+libxl_tee_type = Enumeration("tee_type", [
+ (0, "none"),
+ (1, "native")
+ ], init_val = "LIBXL_TEE_TYPE_NONE")
+
libxl_rdm_reserve = Struct("rdm_reserve", [
("strategy", libxl_rdm_reserve_strategy),
("policy", libxl_rdm_reserve_policy),
@@ -615,6 +620,7 @@ libxl_domain_build_info = Struct("domain_build_info",[
("arch_arm", Struct(None, [("gic_version", libxl_gic_version),
("vuart", libxl_vuart_type),
+ ("tee", libxl_tee_type),
])),
# Alternate p2m is not bound to any architecture or guest type, as it is
# supported by x86 HVM and ARM support is planned.
diff --git a/tools/xl/xl_parse.c b/tools/xl/xl_parse.c
index 352cd214dd..c27e7ae1f0 100644
--- a/tools/xl/xl_parse.c
+++ b/tools/xl/xl_parse.c
@@ -2690,6 +2690,15 @@ skip_usbdev:
}
}
+ if (!xlu_cfg_get_string (config, "tee", &buf, 1)) {
+ e = libxl_tee_type_from_string(buf, &b_info->arch_arm.tee);
+ if (e) {
+ fprintf(stderr,
+ "Unknown tee \"%s\" specified\n", buf);
+ exit(-ERROR_FAIL);
+ }
+ }
+
parse_vkb_list(config, d_config);
xlu_cfg_destroy(config);
--
2.21.0
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply related [flat|nested] 47+ messages in thread* Re: [PATCH v4 09/10] tools/arm: tee: add "tee" option for xl.cfg
2019-03-07 21:04 ` [PATCH v4 09/10] tools/arm: tee: add "tee" option for xl.cfg Volodymyr Babchuk
@ 2019-03-18 15:49 ` Julien Grall
2019-03-18 21:04 ` Achin Gupta
2019-03-20 15:27 ` Volodymyr Babchuk
0 siblings, 2 replies; 47+ messages in thread
From: Julien Grall @ 2019-03-18 15:49 UTC (permalink / raw)
To: Volodymyr Babchuk, xen-devel, tee-dev
Cc: Volodymyr Babchuk, Ian Jackson, Achin Gupta, Wei Liu
(+ Achin)
On 07/03/2019 21:04, Volodymyr Babchuk wrote:
> From: Volodymyr Babchuk <vlad.babchuk@gmail.com>
>
> This enumeration controls TEE type for a domain. Currently there is
> two possible options: either 'none' or 'native'.
>
> 'none' is the default value and it basically disables TEE support at
> all.
>
> 'native' enables access to a "real" TEE installed on a platform.
I am aware I made that suggestion. But I think the naming is not ideal between
the user and the toolstack. The question is how this is going to fit with the
S-EL2 feature where multiple TEE can run together?
I have CCed Achin to see he has any vision how this could be interfaced.
>
> It is possible to add another types in the future.
>
> Signed-off-by: Volodymyr Babchuk <vlad.babchuk@gmail.com>
> ---
>
> All the patches to optee.c should be merged together. They were
> split to ease up review. But they depend heavily on each other.
>
> Changes from v3:
> - tee_enabled renamed to tee_type. Currently two types are supported
> as described in the commit message
> - Add LIBXL_HAVE_BUILDINFO_ARCH_ARM_TEE definition
>
> Changes from v2:
> - Use arch.tee_enabled instead of separate domctl
> ---
> docs/man/xl.cfg.5.pod.in | 12 ++++++++++++
> tools/libxl/libxl.h | 5 +++++
> tools/libxl/libxl_arm.c | 13 +++++++++++++
> tools/libxl/libxl_types.idl | 6 ++++++
> tools/xl/xl_parse.c | 9 +++++++++
> 5 files changed, 45 insertions(+)
>
> diff --git a/docs/man/xl.cfg.5.pod.in b/docs/man/xl.cfg.5.pod.in
> index ad81af1ed8..e15981882b 100644
> --- a/docs/man/xl.cfg.5.pod.in
> +++ b/docs/man/xl.cfg.5.pod.in
> @@ -2702,6 +2702,18 @@ Currently, only the "sbsa_uart" model is supported for ARM.
>
> =back
>
> +=over 4
> +
> +=item B<tee=["none", "native"]>
> +
> +Set TEE type for the guest. Currently only OP-TEE is supported. If
> +this option is set to "native", xl will create guest, which can access
> +native TEE on your system (just make sure that you are using OP-TEE
> +with virtualization support endabled). Also OP-TEE node will be
> +emitted into guest's device tree.
> +
> +=back
> +
> =head3 x86
>
> =over 4
> diff --git a/tools/libxl/libxl.h b/tools/libxl/libxl.h
> index a38e5cdba2..b24e4141b1 100644
> --- a/tools/libxl/libxl.h
> +++ b/tools/libxl/libxl.h
> @@ -273,6 +273,11 @@
> */
> #define LIBXL_HAVE_BUILDINFO_ARM_GIC_VERSION 1
>
> +/*
> + * libxl_domain_build_info has the arch_arm.tee field.
> + */
> +#define LIBXL_HAVE_BUILDINFO_ARCH_ARM_TEE 1
> +
> /*
> * LIBXL_HAVE_SOFT_RESET indicates that libxl supports performing
> * 'soft reset' for domains and there is 'soft_reset' shutdown reason
> diff --git a/tools/libxl/libxl_arm.c b/tools/libxl/libxl_arm.c
> index 141e159043..6930d0ab3b 100644
> --- a/tools/libxl/libxl_arm.c
> +++ b/tools/libxl/libxl_arm.c
> @@ -89,6 +89,19 @@ int libxl__arch_domain_prepare_config(libxl__gc *gc,
> return ERROR_FAIL;
> }
>
> + switch (d_config->b_info.arch_arm.tee) {
> + case LIBXL_TEE_TYPE_NONE:
> + config->arch.tee_type = XEN_DOMCTL_CONFIG_TEE_NONE;
> + break;
> + case LIBXL_TEE_TYPE_NATIVE:
> + config->arch.tee_type = XEN_DOMCTL_CONFIG_TEE_NATIVE;
> + break;
> + default:
> + LOG(ERROR, "Unknown TEE type %d",
> + d_config->b_info.arch_arm.tee);
> + return ERROR_FAIL;
> + }
> +
> return 0;
> }
>
> diff --git a/tools/libxl/libxl_types.idl b/tools/libxl/libxl_types.idl
> index b685ac47ac..4f1eb229b8 100644
> --- a/tools/libxl/libxl_types.idl
> +++ b/tools/libxl/libxl_types.idl
> @@ -457,6 +457,11 @@ libxl_gic_version = Enumeration("gic_version", [
> (0x30, "v3")
> ], init_val = "LIBXL_GIC_VERSION_DEFAULT")
>
> +libxl_tee_type = Enumeration("tee_type", [
> + (0, "none"),
> + (1, "native")
> + ], init_val = "LIBXL_TEE_TYPE_NONE")
> +
> libxl_rdm_reserve = Struct("rdm_reserve", [
> ("strategy", libxl_rdm_reserve_strategy),
> ("policy", libxl_rdm_reserve_policy),
> @@ -615,6 +620,7 @@ libxl_domain_build_info = Struct("domain_build_info",[
>
> ("arch_arm", Struct(None, [("gic_version", libxl_gic_version),
> ("vuart", libxl_vuart_type),
> + ("tee", libxl_tee_type),
AFAICT, TEE also exists on other architecture. So I am wondering whether this
field should be moved out of arch_arm?
Cheers,
--
Julien Grall
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 47+ messages in thread* Re: [PATCH v4 09/10] tools/arm: tee: add "tee" option for xl.cfg
2019-03-18 15:49 ` Julien Grall
@ 2019-03-18 21:04 ` Achin Gupta
2019-03-20 16:18 ` Julien Grall
2019-03-20 15:27 ` Volodymyr Babchuk
1 sibling, 1 reply; 47+ messages in thread
From: Achin Gupta @ 2019-03-18 21:04 UTC (permalink / raw)
To: Julien Grall
Cc: Wei Liu, Volodymyr Babchuk, Ian Jackson, tee-dev, xen-devel, nd,
Volodymyr Babchuk
Hi Julien,
On Mon, Mar 18, 2019 at 03:49:12PM +0000, Julien Grall wrote:
> (+ Achin)
>
> On 07/03/2019 21:04, Volodymyr Babchuk wrote:
> > From: Volodymyr Babchuk <vlad.babchuk@gmail.com>
> >
> > This enumeration controls TEE type for a domain. Currently there is
> > two possible options: either 'none' or 'native'.
> >
> > 'none' is the default value and it basically disables TEE support at
> > all.
> >
> > 'native' enables access to a "real" TEE installed on a platform.
>
> I am aware I made that suggestion. But I think the naming is not ideal
> between the user and the toolstack. The question is how this is going to fit
> with the S-EL2 feature where multiple TEE can run together?
>
> I have CCed Achin to see he has any vision how this could be interfaced.
Thanks.
Multiple TEEs (or rather Trusted OSs) can coexist on Armv8.3 and earlier. They
will not be isolated but play along nicely.
The intent is that prior to S-EL2 and multiple TOSs, each TOS will migrate to
using the SPCI spec. At this stage, there should be no need for a TOS specific
mediator in the Hypervisor. IOW, there should be a "generic" SPCI
mediator. Maybe, we can add a TEE type 'generic' later to enable access to any
TEE through this generic interface?
Support for multiple TOSs has raised other questions that we are trying to
address e.g. dependencies between them or on guests in Nwd, impact on scheduling
decisions made by Nwd etc. Support for OP-TEE in this patch stack does not need
to answer these just yet it seems. It is more likely that we will have to tackle
support for multiple TEEs afresh rather than treating it as an extension of
support for a specific TOS.
Happy to discuss further and I hope this helps in some way.
cheers,
Achin
>
> >
> > It is possible to add another types in the future.
> >
> > Signed-off-by: Volodymyr Babchuk <vlad.babchuk@gmail.com>
> > ---
> >
> > All the patches to optee.c should be merged together. They were
> > split to ease up review. But they depend heavily on each other.
> >
> > Changes from v3:
> > - tee_enabled renamed to tee_type. Currently two types are supported
> > as described in the commit message
> > - Add LIBXL_HAVE_BUILDINFO_ARCH_ARM_TEE definition
> >
> > Changes from v2:
> > - Use arch.tee_enabled instead of separate domctl
> > ---
> > docs/man/xl.cfg.5.pod.in | 12 ++++++++++++
> > tools/libxl/libxl.h | 5 +++++
> > tools/libxl/libxl_arm.c | 13 +++++++++++++
> > tools/libxl/libxl_types.idl | 6 ++++++
> > tools/xl/xl_parse.c | 9 +++++++++
> > 5 files changed, 45 insertions(+)
> >
> > diff --git a/docs/man/xl.cfg.5.pod.in b/docs/man/xl.cfg.5.pod.in
> > index ad81af1ed8..e15981882b 100644
> > --- a/docs/man/xl.cfg.5.pod.in
> > +++ b/docs/man/xl.cfg.5.pod.in
> > @@ -2702,6 +2702,18 @@ Currently, only the "sbsa_uart" model is supported for ARM.
> > =back
> > +=over 4
> > +
> > +=item B<tee=["none", "native"]>
> > +
> > +Set TEE type for the guest. Currently only OP-TEE is supported. If
> > +this option is set to "native", xl will create guest, which can access
> > +native TEE on your system (just make sure that you are using OP-TEE
> > +with virtualization support endabled). Also OP-TEE node will be
> > +emitted into guest's device tree.
> > +
> > +=back
> > +
> > =head3 x86
> > =over 4
> > diff --git a/tools/libxl/libxl.h b/tools/libxl/libxl.h
> > index a38e5cdba2..b24e4141b1 100644
> > --- a/tools/libxl/libxl.h
> > +++ b/tools/libxl/libxl.h
> > @@ -273,6 +273,11 @@
> > */
> > #define LIBXL_HAVE_BUILDINFO_ARM_GIC_VERSION 1
> > +/*
> > + * libxl_domain_build_info has the arch_arm.tee field.
> > + */
> > +#define LIBXL_HAVE_BUILDINFO_ARCH_ARM_TEE 1
> > +
> > /*
> > * LIBXL_HAVE_SOFT_RESET indicates that libxl supports performing
> > * 'soft reset' for domains and there is 'soft_reset' shutdown reason
> > diff --git a/tools/libxl/libxl_arm.c b/tools/libxl/libxl_arm.c
> > index 141e159043..6930d0ab3b 100644
> > --- a/tools/libxl/libxl_arm.c
> > +++ b/tools/libxl/libxl_arm.c
> > @@ -89,6 +89,19 @@ int libxl__arch_domain_prepare_config(libxl__gc *gc,
> > return ERROR_FAIL;
> > }
> > + switch (d_config->b_info.arch_arm.tee) {
> > + case LIBXL_TEE_TYPE_NONE:
> > + config->arch.tee_type = XEN_DOMCTL_CONFIG_TEE_NONE;
> > + break;
> > + case LIBXL_TEE_TYPE_NATIVE:
> > + config->arch.tee_type = XEN_DOMCTL_CONFIG_TEE_NATIVE;
> > + break;
> > + default:
> > + LOG(ERROR, "Unknown TEE type %d",
> > + d_config->b_info.arch_arm.tee);
> > + return ERROR_FAIL;
> > + }
> > +
> > return 0;
> > }
> > diff --git a/tools/libxl/libxl_types.idl b/tools/libxl/libxl_types.idl
> > index b685ac47ac..4f1eb229b8 100644
> > --- a/tools/libxl/libxl_types.idl
> > +++ b/tools/libxl/libxl_types.idl
> > @@ -457,6 +457,11 @@ libxl_gic_version = Enumeration("gic_version", [
> > (0x30, "v3")
> > ], init_val = "LIBXL_GIC_VERSION_DEFAULT")
> > +libxl_tee_type = Enumeration("tee_type", [
> > + (0, "none"),
> > + (1, "native")
> > + ], init_val = "LIBXL_TEE_TYPE_NONE")
> > +
> > libxl_rdm_reserve = Struct("rdm_reserve", [
> > ("strategy", libxl_rdm_reserve_strategy),
> > ("policy", libxl_rdm_reserve_policy),
> > @@ -615,6 +620,7 @@ libxl_domain_build_info = Struct("domain_build_info",[
> > ("arch_arm", Struct(None, [("gic_version", libxl_gic_version),
> > ("vuart", libxl_vuart_type),
> > + ("tee", libxl_tee_type),
>
> AFAICT, TEE also exists on other architecture. So I am wondering whether
> this field should be moved out of arch_arm?
>
> Cheers,
>
> --
> Julien Grall
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 47+ messages in thread* Re: [PATCH v4 09/10] tools/arm: tee: add "tee" option for xl.cfg
2019-03-18 21:04 ` Achin Gupta
@ 2019-03-20 16:18 ` Julien Grall
0 siblings, 0 replies; 47+ messages in thread
From: Julien Grall @ 2019-03-20 16:18 UTC (permalink / raw)
To: Achin Gupta
Cc: Wei Liu, Volodymyr Babchuk, Ian Jackson, tee-dev, xen-devel, nd,
Volodymyr Babchuk
Hi Achin,
On 18/03/2019 21:04, Achin Gupta wrote:
> On Mon, Mar 18, 2019 at 03:49:12PM +0000, Julien Grall wrote:
>> (+ Achin)
>>
>> On 07/03/2019 21:04, Volodymyr Babchuk wrote:
>>> From: Volodymyr Babchuk <vlad.babchuk@gmail.com>
>>>
>>> This enumeration controls TEE type for a domain. Currently there is
>>> two possible options: either 'none' or 'native'.
>>>
>>> 'none' is the default value and it basically disables TEE support at
>>> all.
>>>
>>> 'native' enables access to a "real" TEE installed on a platform.
>>
>> I am aware I made that suggestion. But I think the naming is not ideal
>> between the user and the toolstack. The question is how this is going to fit
>> with the S-EL2 feature where multiple TEE can run together?
>>
>> I have CCed Achin to see he has any vision how this could be interfaced.
>
> Thanks.
>
> Multiple TEEs (or rather Trusted OSs) can coexist on Armv8.3 and earlier. They
> will not be isolated but play along nicely.
That's interesting. So, in the current case (i.e without SPCI), how do you
communicate with a specific Trusted OS?
>
> The intent is that prior to S-EL2 and multiple TOSs, each TOS will migrate to
> using the SPCI spec. At this stage, there should be no need for a TOS specific
> mediator in the Hypervisor. IOW, there should be a "generic" SPCI
> mediator. Maybe, we can add a TEE type 'generic' later to enable access to any
> TEE through this generic interface?
Yes, that's probably a good option to add later. So maybe renaming 'native' to
'optee'(or 'op-tee') would be more suitable here. This avoid the ambiguity of
'native'.
>
> Support for multiple TOSs has raised other questions that we are trying to
> address e.g. dependencies between them or on guests in Nwd, impact on scheduling
> decisions made by Nwd etc. Support for OP-TEE in this patch stack does not need
> to answer these just yet it seems. It is more likely that we will have to tackle
> support for multiple TEEs afresh rather than treating it as an extension of
> support for a specific TOS.
I totally agree. SPCI is a separate topic, although I wanted to get a more
suitable name for the config so we can avoid introduce a new option later on.
>
> Happy to discuss further and I hope this helps in some way.
Thank you for the feedback!
Cheers,
--
Julien Grall
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 47+ messages in thread
* Re: [PATCH v4 09/10] tools/arm: tee: add "tee" option for xl.cfg
2019-03-18 15:49 ` Julien Grall
2019-03-18 21:04 ` Achin Gupta
@ 2019-03-20 15:27 ` Volodymyr Babchuk
2019-03-20 16:06 ` Julien Grall
1 sibling, 1 reply; 47+ messages in thread
From: Volodymyr Babchuk @ 2019-03-20 15:27 UTC (permalink / raw)
To: Julien Grall
Cc: Wei Liu, Volodymyr Babchuk, Achin Gupta, Ian Jackson, tee-dev,
xen-devel, Volodymyr Babchuk
Hello Julien,
Julien Grall writes:
> On 07/03/2019 21:04, Volodymyr Babchuk wrote:
>> From: Volodymyr Babchuk <vlad.babchuk@gmail.com>
>>
>> This enumeration controls TEE type for a domain. Currently there is
>> two possible options: either 'none' or 'native'.
>>
>> 'none' is the default value and it basically disables TEE support at
>> all.
>>
>> 'native' enables access to a "real" TEE installed on a platform.
>
> I am aware I made that suggestion. But I think the naming is not ideal
> between the user and the toolstack. The question is how this is going
> to fit with the S-EL2 feature where multiple TEE can run together?
You see, support for S-EL2 or support for multiple TEEs is a much broader
topic. For me, naming for configuration option is the least important
thing in this case :-)
Right there is no clear vision how it will ever work. I scanned through
SPCI spec and I already have a couple of questions. I need to study it
harder to make serious statements, but I already see that current
mediator framework will hardly fit into SPCI stuff. Frankly, I have
concerns that OP-TEE (or any other existing TEE) will be compatible
with SPCI-enabled systems without major rework. So, we will need to do
big overhaul anyways, when there will appear first SPCI-compatible TEEs and
secure hypervisors.
AFAIK, there is no ARMv8.4 platforms, no S-EL2 hypervisors and so on. It
will take at least couple of years before S-EL2 will hit the market. So
in my opinion it is too early to make any assumptions on how to support
all this in Xen. Lets stick to the current matters.
I'm not insisting on "native". But I can't invent something better right
now. Probably, SPCI-enabled TEE also will be considered "native" as
opposed to, say, "emulated".
As I said, I can't come with anything better than "native". But I'm open
to any suggestions.
> I have CCed Achin to see he has any vision how this could be interfaced.
>
>>
>> It is possible to add another types in the future.
>>
>> Signed-off-by: Volodymyr Babchuk <vlad.babchuk@gmail.com>
>> ---
>>
>> All the patches to optee.c should be merged together. They were
>> split to ease up review. But they depend heavily on each other.
>>
>> Changes from v3:
>> - tee_enabled renamed to tee_type. Currently two types are supported
>> as described in the commit message
>> - Add LIBXL_HAVE_BUILDINFO_ARCH_ARM_TEE definition
>>
>> Changes from v2:
>> - Use arch.tee_enabled instead of separate domctl
>> ---
>> docs/man/xl.cfg.5.pod.in | 12 ++++++++++++
>> tools/libxl/libxl.h | 5 +++++
>> tools/libxl/libxl_arm.c | 13 +++++++++++++
>> tools/libxl/libxl_types.idl | 6 ++++++
>> tools/xl/xl_parse.c | 9 +++++++++
>> 5 files changed, 45 insertions(+)
>>
>> diff --git a/docs/man/xl.cfg.5.pod.in b/docs/man/xl.cfg.5.pod.in
>> index ad81af1ed8..e15981882b 100644
>> --- a/docs/man/xl.cfg.5.pod.in
>> +++ b/docs/man/xl.cfg.5.pod.in
>> @@ -2702,6 +2702,18 @@ Currently, only the "sbsa_uart" model is supported for ARM.
>> =back
>> +=over 4
>> +
>> +=item B<tee=["none", "native"]>
>> +
>> +Set TEE type for the guest. Currently only OP-TEE is supported. If
>> +this option is set to "native", xl will create guest, which can access
>> +native TEE on your system (just make sure that you are using OP-TEE
>> +with virtualization support endabled). Also OP-TEE node will be
>> +emitted into guest's device tree.
>> +
>> +=back
>> +
>> =head3 x86
>> =over 4
>> diff --git a/tools/libxl/libxl.h b/tools/libxl/libxl.h
>> index a38e5cdba2..b24e4141b1 100644
>> --- a/tools/libxl/libxl.h
>> +++ b/tools/libxl/libxl.h
>> @@ -273,6 +273,11 @@
>> */
>> #define LIBXL_HAVE_BUILDINFO_ARM_GIC_VERSION 1
>> +/*
>> + * libxl_domain_build_info has the arch_arm.tee field.
>> + */
>> +#define LIBXL_HAVE_BUILDINFO_ARCH_ARM_TEE 1
>> +
>> /*
>> * LIBXL_HAVE_SOFT_RESET indicates that libxl supports performing
>> * 'soft reset' for domains and there is 'soft_reset' shutdown reason
>> diff --git a/tools/libxl/libxl_arm.c b/tools/libxl/libxl_arm.c
>> index 141e159043..6930d0ab3b 100644
>> --- a/tools/libxl/libxl_arm.c
>> +++ b/tools/libxl/libxl_arm.c
>> @@ -89,6 +89,19 @@ int libxl__arch_domain_prepare_config(libxl__gc *gc,
>> return ERROR_FAIL;
>> }
>> + switch (d_config->b_info.arch_arm.tee) {
>> + case LIBXL_TEE_TYPE_NONE:
>> + config->arch.tee_type = XEN_DOMCTL_CONFIG_TEE_NONE;
>> + break;
>> + case LIBXL_TEE_TYPE_NATIVE:
>> + config->arch.tee_type = XEN_DOMCTL_CONFIG_TEE_NATIVE;
>> + break;
>> + default:
>> + LOG(ERROR, "Unknown TEE type %d",
>> + d_config->b_info.arch_arm.tee);
>> + return ERROR_FAIL;
>> + }
>> +
>> return 0;
>> }
>> diff --git a/tools/libxl/libxl_types.idl
>> b/tools/libxl/libxl_types.idl
>> index b685ac47ac..4f1eb229b8 100644
>> --- a/tools/libxl/libxl_types.idl
>> +++ b/tools/libxl/libxl_types.idl
>> @@ -457,6 +457,11 @@ libxl_gic_version = Enumeration("gic_version", [
>> (0x30, "v3")
>> ], init_val = "LIBXL_GIC_VERSION_DEFAULT")
>> +libxl_tee_type = Enumeration("tee_type", [
>> + (0, "none"),
>> + (1, "native")
>> + ], init_val = "LIBXL_TEE_TYPE_NONE")
>> +
>> libxl_rdm_reserve = Struct("rdm_reserve", [
>> ("strategy", libxl_rdm_reserve_strategy),
>> ("policy", libxl_rdm_reserve_policy),
>> @@ -615,6 +620,7 @@ libxl_domain_build_info = Struct("domain_build_info",[
>> ("arch_arm", Struct(None, [("gic_version",
>> libxl_gic_version),
>> ("vuart", libxl_vuart_type),
>> + ("tee", libxl_tee_type),
>
> AFAICT, TEE also exists on other architecture. So I am wondering
> whether this field should be moved out of arch_arm?
In v3 thread you asked if I see any use for x86. Because in that version
this option was in common section. Honestly, I don't see any use there,
because I have no idea how this can be implemented on x86. I checked
Open-TEE implementation as an example, but it appeared that it works in
userpsace.
So, I would stick to ARM-only. But I have no strong objections, if x86
folks are willing to support TEEs in the common code.
--
Best regards,Volodymyr Babchuk
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 47+ messages in thread* Re: [PATCH v4 09/10] tools/arm: tee: add "tee" option for xl.cfg
2019-03-20 15:27 ` Volodymyr Babchuk
@ 2019-03-20 16:06 ` Julien Grall
2019-03-20 17:01 ` Volodymyr Babchuk
0 siblings, 1 reply; 47+ messages in thread
From: Julien Grall @ 2019-03-20 16:06 UTC (permalink / raw)
To: Volodymyr Babchuk
Cc: Wei Liu, Volodymyr Babchuk, Achin Gupta, Ian Jackson, tee-dev, xen-devel
On 20/03/2019 15:27, Volodymyr Babchuk wrote:
>
> Hello Julien,
>
> Julien Grall writes:
>
>> On 07/03/2019 21:04, Volodymyr Babchuk wrote:
>>> From: Volodymyr Babchuk <vlad.babchuk@gmail.com>
>>>
>>> This enumeration controls TEE type for a domain. Currently there is
>>> two possible options: either 'none' or 'native'.
>>>
>>> 'none' is the default value and it basically disables TEE support at
>>> all.
>>>
>>> 'native' enables access to a "real" TEE installed on a platform.
>>
>> I am aware I made that suggestion. But I think the naming is not ideal
>> between the user and the toolstack. The question is how this is going
>> to fit with the S-EL2 feature where multiple TEE can run together?
> You see, support for S-EL2 or support for multiple TEEs is a much broader
> topic. For me, naming for configuration option is the least important
> thing in this case :-)
Naming exposed to users are hard to remove. If the naming is too ambiguous, then
we will have to introduce a new option later on. This is not very ideal, so it
would be better if we can find something different.
>
> Right there is no clear vision how it will ever work. I scanned through
> SPCI spec and I already have a couple of questions. I need to study it
> harder to make serious statements, but I already see that current
> mediator framework will hardly fit into SPCI stuff. Frankly, I have
> concerns that OP-TEE (or any other existing TEE) will be compatible
> with SPCI-enabled systems without major rework. So, we will need to do
> big overhaul anyways, when there will appear first SPCI-compatible TEEs and
> secure hypervisors.
>
> AFAIK, there is no ARMv8.4 platforms, no S-EL2 hypervisors and so on. It
> will take at least couple of years before S-EL2 will hit the market. So
> in my opinion it is too early to make any assumptions on how to support
> all this in Xen. Lets stick to the current matters.
I am fully aware that more work will need to be done with S-EL2. I am not
expecting you (or anyone else here) to come with the implementation now. My
point is the naming should be chosen so it prevents ambiguity with whatever we
know will come up in the future.
>
> I'm not insisting on "native". But I can't invent something better right
> now. Probably, SPCI-enabled TEE also will be considered "native" as
> opposed to, say, "emulated". >
> As I said, I can't come with anything better than "native". But I'm open
> to any suggestions.
Well one solution is to ditch "native" and name it "optee". By giving the name
you avoid ambiguity. If we ever have multiple op-tee instance running, then it
could easily be extend with a comma. So you allow backward compatibility.
>> AFAICT, TEE also exists on other architecture. So I am wondering
>> whether this field should be moved out of arch_arm?
> In v3 thread you asked if I see any use for x86. Because in that version
> this option was in common section. Honestly, I don't see any use there,
> because I have no idea how this can be implemented on x86.
On the v3, I pointed out that the documentation was in Arm section but the code
was in common. When I asked a question, it does not mean I am not happy with it.
It only means I am trying to understand your choice so I can make my mind on it.
Sadly, you left it unanswered until now.
In this context, I am not asking you how this is going to be implemented but
whether this option could potentially be used in the future for other
architecture (e.g x86, RISC-V...). For instance, the option "device_tree" is
part for common options despite it is only implemented on Arm.
Cheers,
--
Julien Grall
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 47+ messages in thread
* Re: [PATCH v4 09/10] tools/arm: tee: add "tee" option for xl.cfg
2019-03-20 16:06 ` Julien Grall
@ 2019-03-20 17:01 ` Volodymyr Babchuk
2019-03-20 18:35 ` Julien Grall
0 siblings, 1 reply; 47+ messages in thread
From: Volodymyr Babchuk @ 2019-03-20 17:01 UTC (permalink / raw)
To: Julien Grall
Cc: Wei Liu, Volodymyr Babchuk, Achin Gupta, Ian Jackson, tee-dev,
xen-devel, Volodymyr Babchuk
Julien Grall writes:
> On 20/03/2019 15:27, Volodymyr Babchuk wrote:
>>
>> Hello Julien,
>>
>> Julien Grall writes:
>>
>>> On 07/03/2019 21:04, Volodymyr Babchuk wrote:
>>>> From: Volodymyr Babchuk <vlad.babchuk@gmail.com>
>>>>
>>>> This enumeration controls TEE type for a domain. Currently there is
>>>> two possible options: either 'none' or 'native'.
>>>>
>>>> 'none' is the default value and it basically disables TEE support at
>>>> all.
>>>>
>>>> 'native' enables access to a "real" TEE installed on a platform.
>>>
>>> I am aware I made that suggestion. But I think the naming is not ideal
>>> between the user and the toolstack. The question is how this is going
>>> to fit with the S-EL2 feature where multiple TEE can run together?
>> You see, support for S-EL2 or support for multiple TEEs is a much broader
>> topic. For me, naming for configuration option is the least important
>> thing in this case :-)
>
> Naming exposed to users are hard to remove. If the naming is too
> ambiguous, then we will have to introduce a new option later on. This
> is not very ideal, so it would be better if we can find something
> different.
>
>>
>> Right there is no clear vision how it will ever work. I scanned through
>> SPCI spec and I already have a couple of questions. I need to study it
>> harder to make serious statements, but I already see that current
>> mediator framework will hardly fit into SPCI stuff. Frankly, I have
>> concerns that OP-TEE (or any other existing TEE) will be compatible
>> with SPCI-enabled systems without major rework. So, we will need to do
>> big overhaul anyways, when there will appear first SPCI-compatible TEEs and
>> secure hypervisors.
>>
>> AFAIK, there is no ARMv8.4 platforms, no S-EL2 hypervisors and so on. It
>> will take at least couple of years before S-EL2 will hit the market. So
>> in my opinion it is too early to make any assumptions on how to support
>> all this in Xen. Lets stick to the current matters.
>
> I am fully aware that more work will need to be done with S-EL2. I am
> not expecting you (or anyone else here) to come with the
> implementation now. My point is the naming should be chosen so it
> prevents ambiguity with whatever we know will come up in the future.
>
>>
>> I'm not insisting on "native". But I can't invent something better right
>> now. Probably, SPCI-enabled TEE also will be considered "native" as
>> opposed to, say, "emulated". >
>> As I said, I can't come with anything better than "native". But I'm open
>> to any suggestions.
>
> Well one solution is to ditch "native" and name it "optee". By giving
> the name you avoid ambiguity. If we ever have multiple op-tee instance
> running, then it could easily be extend with a comma. So you allow
> backward compatibility.
I considered this. But If I remember right, idea was to query Xen about
available TEE, and configure guest in the appropriate way. So "native"
(or some other generic way) could be used for OP-TEE, Google Trusty or
any other TEE, without changing guest configuration.
Using "optee" will cause explicit configuration for OP-TEE only. I can't
say that this is good or bad. It just different. Do you think that would
be better approach?
>
>>> AFAICT, TEE also exists on other architecture. So I am wondering
>>> whether this field should be moved out of arch_arm?
>> In v3 thread you asked if I see any use for x86. Because in that version
>> this option was in common section. Honestly, I don't see any use there,
>> because I have no idea how this can be implemented on x86.
>
> On the v3, I pointed out that the documentation was in Arm section but
> the code was in common. When I asked a question, it does not mean I am
> not happy with it. It only means I am trying to understand your choice
> so I can make my mind on it. Sadly, you left it unanswered until now.
Oh, I see. My intention was to put this into ARM section. So, I
documented it in the right way, but did mistake, putting it into wrong
place in the xl/libxl code. It was my first patch to toolstack, so I
just didn't knew the right place.
> In this context, I am not asking you how this is going to be
> implemented but whether this option could potentially be used in the
> future for other architecture (e.g x86, RISC-V...). For instance, the
> option "device_tree" is part for common options despite it is only
> implemented on Arm.
I believe to your experience there. If you think it is better to move
this into common code - I'll do this.
--
Best regards,Volodymyr Babchuk
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 47+ messages in thread
* Re: [PATCH v4 09/10] tools/arm: tee: add "tee" option for xl.cfg
2019-03-20 17:01 ` Volodymyr Babchuk
@ 2019-03-20 18:35 ` Julien Grall
2019-04-05 10:25 ` [Xen-devel] " Volodymyr Babchuk
0 siblings, 1 reply; 47+ messages in thread
From: Julien Grall @ 2019-03-20 18:35 UTC (permalink / raw)
To: Volodymyr Babchuk
Cc: Wei Liu, Volodymyr Babchuk, Achin Gupta, Ian Jackson, tee-dev, xen-devel
Hi,
On 20/03/2019 17:01, Volodymyr Babchuk wrote:
>
> Julien Grall writes:
>
>> On 20/03/2019 15:27, Volodymyr Babchuk wrote:
>>>
>>> Hello Julien,
>>>
>>> Julien Grall writes:
>>>
>>>> On 07/03/2019 21:04, Volodymyr Babchuk wrote:
>>>>> From: Volodymyr Babchuk <vlad.babchuk@gmail.com>
>>>>>
>>>>> This enumeration controls TEE type for a domain. Currently there is
>>>>> two possible options: either 'none' or 'native'.
>>>>>
>>>>> 'none' is the default value and it basically disables TEE support at
>>>>> all.
>>>>>
>>>>> 'native' enables access to a "real" TEE installed on a platform.
>>>>
>>>> I am aware I made that suggestion. But I think the naming is not ideal
>>>> between the user and the toolstack. The question is how this is going
>>>> to fit with the S-EL2 feature where multiple TEE can run together?
>>> You see, support for S-EL2 or support for multiple TEEs is a much broader
>>> topic. For me, naming for configuration option is the least important
>>> thing in this case :-)
>>
>> Naming exposed to users are hard to remove. If the naming is too
>> ambiguous, then we will have to introduce a new option later on. This
>> is not very ideal, so it would be better if we can find something
>> different.
>>
>>>
>>> Right there is no clear vision how it will ever work. I scanned through
>>> SPCI spec and I already have a couple of questions. I need to study it
>>> harder to make serious statements, but I already see that current
>>> mediator framework will hardly fit into SPCI stuff. Frankly, I have
>>> concerns that OP-TEE (or any other existing TEE) will be compatible
>>> with SPCI-enabled systems without major rework. So, we will need to do
>>> big overhaul anyways, when there will appear first SPCI-compatible TEEs and
>>> secure hypervisors.
>>>
>>> AFAIK, there is no ARMv8.4 platforms, no S-EL2 hypervisors and so on. It
>>> will take at least couple of years before S-EL2 will hit the market. So
>>> in my opinion it is too early to make any assumptions on how to support
>>> all this in Xen. Lets stick to the current matters.
>>
>> I am fully aware that more work will need to be done with S-EL2. I am
>> not expecting you (or anyone else here) to come with the
>> implementation now. My point is the naming should be chosen so it
>> prevents ambiguity with whatever we know will come up in the future.
>>
>>>
>>> I'm not insisting on "native". But I can't invent something better right
>>> now. Probably, SPCI-enabled TEE also will be considered "native" as
>>> opposed to, say, "emulated". >
>>> As I said, I can't come with anything better than "native". But I'm open
>>> to any suggestions.
>>
>> Well one solution is to ditch "native" and name it "optee". By giving
>> the name you avoid ambiguity. If we ever have multiple op-tee instance
>> running, then it could easily be extend with a comma. So you allow
>> backward compatibility.
>
> I considered this. But If I remember right, idea was to query Xen about
> available TEE, and configure guest in the appropriate way. So "native"
> (or some other generic way) could be used for OP-TEE, Google Trusty or
> any other TEE, without changing guest configuration.
>
> Using "optee" will cause explicit configuration for OP-TEE only. I can't
> say that this is good or bad. It just different. Do you think that would
> be better approach?
You have 2 different cases to take into account:
- A guest is able to deal with all supported Trusted OS. So "native" will
let the toolstack query the current Trusted OS and expose it to the guest.
- A guest can only deal with a specific Trusted OS. In that case, the user
might want to specify in the configuration the expected Trusted OS so it can't
boot if not available.
What I suggest is deferring the former case until we have another TEE in hand.
Maybe at that time, we will have a better naming :).
>
>>
>>>> AFAICT, TEE also exists on other architecture. So I am wondering
>>>> whether this field should be moved out of arch_arm?
>>> In v3 thread you asked if I see any use for x86. Because in that version
>>> this option was in common section. Honestly, I don't see any use there,
>>> because I have no idea how this can be implemented on x86.
>>
>> On the v3, I pointed out that the documentation was in Arm section but
>> the code was in common. When I asked a question, it does not mean I am
>> not happy with it. It only means I am trying to understand your choice
>> so I can make my mind on it. Sadly, you left it unanswered until now.
>
> Oh, I see. My intention was to put this into ARM section. So, I
> documented it in the right way, but did mistake, putting it into wrong
> place in the xl/libxl code. It was my first patch to toolstack, so I
> just didn't knew the right place.
>
>> In this context, I am not asking you how this is going to be
>> implemented but whether this option could potentially be used in the
>> future for other architecture (e.g x86, RISC-V...). For instance, the
>> option "device_tree" is part for common options despite it is only
>> implemented on Arm.
>
> I believe to your experience there. If you think it is better to move
> this into common code - I'll do this.
From my understanding, TEE [1] seems to be a rather generic name with some
supports on various architectures. I can't rule out that other architectures
(current or future) will not want to provide guest TEE.
So I think the common option is the best here.
Cheers,
[1] https://en.wikipedia.org/wiki/Trusted_execution_environment
--
Julien Grall
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 47+ messages in thread* Re: [PATCH v4 09/10] tools/arm: tee: add "tee" option for xl.cfg
@ 2019-04-05 10:25 ` Volodymyr Babchuk
0 siblings, 0 replies; 47+ messages in thread
From: Volodymyr Babchuk @ 2019-04-05 10:25 UTC (permalink / raw)
To: Julien Grall
Cc: Wei Liu, Volodymyr Babchuk, Achin Gupta, Ian Jackson, tee-dev,
xen-devel, Volodymyr Babchuk
Hi Julien,
Julien Grall writes:
> Hi,
>
> On 20/03/2019 17:01, Volodymyr Babchuk wrote:
>>
>> Julien Grall writes:
>>
>>> On 20/03/2019 15:27, Volodymyr Babchuk wrote:
>>>>
>>>> Hello Julien,
>>>>
>>>> Julien Grall writes:
>>>>
>>>>> On 07/03/2019 21:04, Volodymyr Babchuk wrote:
>>>>>> From: Volodymyr Babchuk <vlad.babchuk@gmail.com>
>>>>>>
>>>>>> This enumeration controls TEE type for a domain. Currently there is
>>>>>> two possible options: either 'none' or 'native'.
>>>>>>
>>>>>> 'none' is the default value and it basically disables TEE support at
>>>>>> all.
>>>>>>
>>>>>> 'native' enables access to a "real" TEE installed on a platform.
>>>>>
>>>>> I am aware I made that suggestion. But I think the naming is not ideal
>>>>> between the user and the toolstack. The question is how this is going
>>>>> to fit with the S-EL2 feature where multiple TEE can run together?
>>>> You see, support for S-EL2 or support for multiple TEEs is a much broader
>>>> topic. For me, naming for configuration option is the least important
>>>> thing in this case :-)
>>>
>>> Naming exposed to users are hard to remove. If the naming is too
>>> ambiguous, then we will have to introduce a new option later on. This
>>> is not very ideal, so it would be better if we can find something
>>> different.
>>>
>>>>
>>>> Right there is no clear vision how it will ever work. I scanned through
>>>> SPCI spec and I already have a couple of questions. I need to study it
>>>> harder to make serious statements, but I already see that current
>>>> mediator framework will hardly fit into SPCI stuff. Frankly, I have
>>>> concerns that OP-TEE (or any other existing TEE) will be compatible
>>>> with SPCI-enabled systems without major rework. So, we will need to do
>>>> big overhaul anyways, when there will appear first SPCI-compatible TEEs and
>>>> secure hypervisors.
>>>>
>>>> AFAIK, there is no ARMv8.4 platforms, no S-EL2 hypervisors and so on. It
>>>> will take at least couple of years before S-EL2 will hit the market. So
>>>> in my opinion it is too early to make any assumptions on how to support
>>>> all this in Xen. Lets stick to the current matters.
>>>
>>> I am fully aware that more work will need to be done with S-EL2. I am
>>> not expecting you (or anyone else here) to come with the
>>> implementation now. My point is the naming should be chosen so it
>>> prevents ambiguity with whatever we know will come up in the future.
>>>
>>>>
>>>> I'm not insisting on "native". But I can't invent something better right
>>>> now. Probably, SPCI-enabled TEE also will be considered "native" as
>>>> opposed to, say, "emulated". >
>>>> As I said, I can't come with anything better than "native". But I'm open
>>>> to any suggestions.
>>>
>>> Well one solution is to ditch "native" and name it "optee". By giving
>>> the name you avoid ambiguity. If we ever have multiple op-tee instance
>>> running, then it could easily be extend with a comma. So you allow
>>> backward compatibility.
>>
>> I considered this. But If I remember right, idea was to query Xen about
>> available TEE, and configure guest in the appropriate way. So "native"
>> (or some other generic way) could be used for OP-TEE, Google Trusty or
>> any other TEE, without changing guest configuration.
>>
>> Using "optee" will cause explicit configuration for OP-TEE only. I can't
>> say that this is good or bad. It just different. Do you think that would
>> be better approach?
>
> You have 2 different cases to take into account:
> - A guest is able to deal with all supported Trusted OS. So
> "native" will let the toolstack query the current Trusted OS and
> expose it to the guest.
> - A guest can only deal with a specific Trusted OS. In that case,
> the user might want to specify in the configuration the expected
> Trusted OS so it can't boot if not available.
>
> What I suggest is deferring the former case until we have another TEE
> in hand. Maybe at that time, we will have a better naming :).
Okay, so summarizing this up, you are proposing to use "optee". Am I got
you right?
--
Best regards,Volodymyr Babchuk
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 47+ messages in thread
* Re: [Xen-devel] [PATCH v4 09/10] tools/arm: tee: add "tee" option for xl.cfg
@ 2019-04-05 10:25 ` Volodymyr Babchuk
0 siblings, 0 replies; 47+ messages in thread
From: Volodymyr Babchuk @ 2019-04-05 10:25 UTC (permalink / raw)
To: Julien Grall
Cc: Wei Liu, Volodymyr Babchuk, Achin Gupta, Ian Jackson, tee-dev,
xen-devel, Volodymyr Babchuk
Hi Julien,
Julien Grall writes:
> Hi,
>
> On 20/03/2019 17:01, Volodymyr Babchuk wrote:
>>
>> Julien Grall writes:
>>
>>> On 20/03/2019 15:27, Volodymyr Babchuk wrote:
>>>>
>>>> Hello Julien,
>>>>
>>>> Julien Grall writes:
>>>>
>>>>> On 07/03/2019 21:04, Volodymyr Babchuk wrote:
>>>>>> From: Volodymyr Babchuk <vlad.babchuk@gmail.com>
>>>>>>
>>>>>> This enumeration controls TEE type for a domain. Currently there is
>>>>>> two possible options: either 'none' or 'native'.
>>>>>>
>>>>>> 'none' is the default value and it basically disables TEE support at
>>>>>> all.
>>>>>>
>>>>>> 'native' enables access to a "real" TEE installed on a platform.
>>>>>
>>>>> I am aware I made that suggestion. But I think the naming is not ideal
>>>>> between the user and the toolstack. The question is how this is going
>>>>> to fit with the S-EL2 feature where multiple TEE can run together?
>>>> You see, support for S-EL2 or support for multiple TEEs is a much broader
>>>> topic. For me, naming for configuration option is the least important
>>>> thing in this case :-)
>>>
>>> Naming exposed to users are hard to remove. If the naming is too
>>> ambiguous, then we will have to introduce a new option later on. This
>>> is not very ideal, so it would be better if we can find something
>>> different.
>>>
>>>>
>>>> Right there is no clear vision how it will ever work. I scanned through
>>>> SPCI spec and I already have a couple of questions. I need to study it
>>>> harder to make serious statements, but I already see that current
>>>> mediator framework will hardly fit into SPCI stuff. Frankly, I have
>>>> concerns that OP-TEE (or any other existing TEE) will be compatible
>>>> with SPCI-enabled systems without major rework. So, we will need to do
>>>> big overhaul anyways, when there will appear first SPCI-compatible TEEs and
>>>> secure hypervisors.
>>>>
>>>> AFAIK, there is no ARMv8.4 platforms, no S-EL2 hypervisors and so on. It
>>>> will take at least couple of years before S-EL2 will hit the market. So
>>>> in my opinion it is too early to make any assumptions on how to support
>>>> all this in Xen. Lets stick to the current matters.
>>>
>>> I am fully aware that more work will need to be done with S-EL2. I am
>>> not expecting you (or anyone else here) to come with the
>>> implementation now. My point is the naming should be chosen so it
>>> prevents ambiguity with whatever we know will come up in the future.
>>>
>>>>
>>>> I'm not insisting on "native". But I can't invent something better right
>>>> now. Probably, SPCI-enabled TEE also will be considered "native" as
>>>> opposed to, say, "emulated". >
>>>> As I said, I can't come with anything better than "native". But I'm open
>>>> to any suggestions.
>>>
>>> Well one solution is to ditch "native" and name it "optee". By giving
>>> the name you avoid ambiguity. If we ever have multiple op-tee instance
>>> running, then it could easily be extend with a comma. So you allow
>>> backward compatibility.
>>
>> I considered this. But If I remember right, idea was to query Xen about
>> available TEE, and configure guest in the appropriate way. So "native"
>> (or some other generic way) could be used for OP-TEE, Google Trusty or
>> any other TEE, without changing guest configuration.
>>
>> Using "optee" will cause explicit configuration for OP-TEE only. I can't
>> say that this is good or bad. It just different. Do you think that would
>> be better approach?
>
> You have 2 different cases to take into account:
> - A guest is able to deal with all supported Trusted OS. So
> "native" will let the toolstack query the current Trusted OS and
> expose it to the guest.
> - A guest can only deal with a specific Trusted OS. In that case,
> the user might want to specify in the configuration the expected
> Trusted OS so it can't boot if not available.
>
> What I suggest is deferring the former case until we have another TEE
> in hand. Maybe at that time, we will have a better naming :).
Okay, so summarizing this up, you are proposing to use "optee". Am I got
you right?
--
Best regards,Volodymyr Babchuk
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 47+ messages in thread
* Re: [PATCH v4 09/10] tools/arm: tee: add "tee" option for xl.cfg
@ 2019-04-08 10:47 ` Julien Grall
0 siblings, 0 replies; 47+ messages in thread
From: Julien Grall @ 2019-04-08 10:47 UTC (permalink / raw)
To: Volodymyr Babchuk
Cc: Wei Liu, Volodymyr Babchuk, Achin Gupta, Ian Jackson, tee-dev, xen-devel
Hi,
On 4/5/19 11:25 AM, Volodymyr Babchuk wrote:
>
> Hi Julien,
>
> Julien Grall writes:
>
>> Hi,
>>
>> On 20/03/2019 17:01, Volodymyr Babchuk wrote:
>>>
>>> Julien Grall writes:
>>>
>>>> On 20/03/2019 15:27, Volodymyr Babchuk wrote:
>>>>>
>>>>> Hello Julien,
>>>>>
>>>>> Julien Grall writes:
>>>>>
>>>>>> On 07/03/2019 21:04, Volodymyr Babchuk wrote:
>>>>>>> From: Volodymyr Babchuk <vlad.babchuk@gmail.com>
>>>>>>>
>>>>>>> This enumeration controls TEE type for a domain. Currently there is
>>>>>>> two possible options: either 'none' or 'native'.
>>>>>>>
>>>>>>> 'none' is the default value and it basically disables TEE support at
>>>>>>> all.
>>>>>>>
>>>>>>> 'native' enables access to a "real" TEE installed on a platform.
>>>>>>
>>>>>> I am aware I made that suggestion. But I think the naming is not ideal
>>>>>> between the user and the toolstack. The question is how this is going
>>>>>> to fit with the S-EL2 feature where multiple TEE can run together?
>>>>> You see, support for S-EL2 or support for multiple TEEs is a much broader
>>>>> topic. For me, naming for configuration option is the least important
>>>>> thing in this case :-)
>>>>
>>>> Naming exposed to users are hard to remove. If the naming is too
>>>> ambiguous, then we will have to introduce a new option later on. This
>>>> is not very ideal, so it would be better if we can find something
>>>> different.
>>>>
>>>>>
>>>>> Right there is no clear vision how it will ever work. I scanned through
>>>>> SPCI spec and I already have a couple of questions. I need to study it
>>>>> harder to make serious statements, but I already see that current
>>>>> mediator framework will hardly fit into SPCI stuff. Frankly, I have
>>>>> concerns that OP-TEE (or any other existing TEE) will be compatible
>>>>> with SPCI-enabled systems without major rework. So, we will need to do
>>>>> big overhaul anyways, when there will appear first SPCI-compatible TEEs and
>>>>> secure hypervisors.
>>>>>
>>>>> AFAIK, there is no ARMv8.4 platforms, no S-EL2 hypervisors and so on. It
>>>>> will take at least couple of years before S-EL2 will hit the market. So
>>>>> in my opinion it is too early to make any assumptions on how to support
>>>>> all this in Xen. Lets stick to the current matters.
>>>>
>>>> I am fully aware that more work will need to be done with S-EL2. I am
>>>> not expecting you (or anyone else here) to come with the
>>>> implementation now. My point is the naming should be chosen so it
>>>> prevents ambiguity with whatever we know will come up in the future.
>>>>
>>>>>
>>>>> I'm not insisting on "native". But I can't invent something better right
>>>>> now. Probably, SPCI-enabled TEE also will be considered "native" as
>>>>> opposed to, say, "emulated". >
>>>>> As I said, I can't come with anything better than "native". But I'm open
>>>>> to any suggestions.
>>>>
>>>> Well one solution is to ditch "native" and name it "optee". By giving
>>>> the name you avoid ambiguity. If we ever have multiple op-tee instance
>>>> running, then it could easily be extend with a comma. So you allow
>>>> backward compatibility.
>>>
>>> I considered this. But If I remember right, idea was to query Xen about
>>> available TEE, and configure guest in the appropriate way. So "native"
>>> (or some other generic way) could be used for OP-TEE, Google Trusty or
>>> any other TEE, without changing guest configuration.
>>>
>>> Using "optee" will cause explicit configuration for OP-TEE only. I can't
>>> say that this is good or bad. It just different. Do you think that would
>>> be better approach?
>>
>> You have 2 different cases to take into account:
>> - A guest is able to deal with all supported Trusted OS. So
>> "native" will let the toolstack query the current Trusted OS and
>> expose it to the guest.
>> - A guest can only deal with a specific Trusted OS. In that case,
>> the user might want to specify in the configuration the expected
>> Trusted OS so it can't boot if not available.
>>
>> What I suggest is deferring the former case until we have another TEE
>> in hand. Maybe at that time, we will have a better naming :).
>
> Okay, so summarizing this up, you are proposing to use "optee". Am I got
> you right?
I am suggesting 'tee = "optee"' for now.
Cheers,
--
Julien Grall
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 47+ messages in thread
* Re: [Xen-devel] [PATCH v4 09/10] tools/arm: tee: add "tee" option for xl.cfg
@ 2019-04-08 10:47 ` Julien Grall
0 siblings, 0 replies; 47+ messages in thread
From: Julien Grall @ 2019-04-08 10:47 UTC (permalink / raw)
To: Volodymyr Babchuk
Cc: Wei Liu, Volodymyr Babchuk, Achin Gupta, Ian Jackson, tee-dev, xen-devel
Hi,
On 4/5/19 11:25 AM, Volodymyr Babchuk wrote:
>
> Hi Julien,
>
> Julien Grall writes:
>
>> Hi,
>>
>> On 20/03/2019 17:01, Volodymyr Babchuk wrote:
>>>
>>> Julien Grall writes:
>>>
>>>> On 20/03/2019 15:27, Volodymyr Babchuk wrote:
>>>>>
>>>>> Hello Julien,
>>>>>
>>>>> Julien Grall writes:
>>>>>
>>>>>> On 07/03/2019 21:04, Volodymyr Babchuk wrote:
>>>>>>> From: Volodymyr Babchuk <vlad.babchuk@gmail.com>
>>>>>>>
>>>>>>> This enumeration controls TEE type for a domain. Currently there is
>>>>>>> two possible options: either 'none' or 'native'.
>>>>>>>
>>>>>>> 'none' is the default value and it basically disables TEE support at
>>>>>>> all.
>>>>>>>
>>>>>>> 'native' enables access to a "real" TEE installed on a platform.
>>>>>>
>>>>>> I am aware I made that suggestion. But I think the naming is not ideal
>>>>>> between the user and the toolstack. The question is how this is going
>>>>>> to fit with the S-EL2 feature where multiple TEE can run together?
>>>>> You see, support for S-EL2 or support for multiple TEEs is a much broader
>>>>> topic. For me, naming for configuration option is the least important
>>>>> thing in this case :-)
>>>>
>>>> Naming exposed to users are hard to remove. If the naming is too
>>>> ambiguous, then we will have to introduce a new option later on. This
>>>> is not very ideal, so it would be better if we can find something
>>>> different.
>>>>
>>>>>
>>>>> Right there is no clear vision how it will ever work. I scanned through
>>>>> SPCI spec and I already have a couple of questions. I need to study it
>>>>> harder to make serious statements, but I already see that current
>>>>> mediator framework will hardly fit into SPCI stuff. Frankly, I have
>>>>> concerns that OP-TEE (or any other existing TEE) will be compatible
>>>>> with SPCI-enabled systems without major rework. So, we will need to do
>>>>> big overhaul anyways, when there will appear first SPCI-compatible TEEs and
>>>>> secure hypervisors.
>>>>>
>>>>> AFAIK, there is no ARMv8.4 platforms, no S-EL2 hypervisors and so on. It
>>>>> will take at least couple of years before S-EL2 will hit the market. So
>>>>> in my opinion it is too early to make any assumptions on how to support
>>>>> all this in Xen. Lets stick to the current matters.
>>>>
>>>> I am fully aware that more work will need to be done with S-EL2. I am
>>>> not expecting you (or anyone else here) to come with the
>>>> implementation now. My point is the naming should be chosen so it
>>>> prevents ambiguity with whatever we know will come up in the future.
>>>>
>>>>>
>>>>> I'm not insisting on "native". But I can't invent something better right
>>>>> now. Probably, SPCI-enabled TEE also will be considered "native" as
>>>>> opposed to, say, "emulated". >
>>>>> As I said, I can't come with anything better than "native". But I'm open
>>>>> to any suggestions.
>>>>
>>>> Well one solution is to ditch "native" and name it "optee". By giving
>>>> the name you avoid ambiguity. If we ever have multiple op-tee instance
>>>> running, then it could easily be extend with a comma. So you allow
>>>> backward compatibility.
>>>
>>> I considered this. But If I remember right, idea was to query Xen about
>>> available TEE, and configure guest in the appropriate way. So "native"
>>> (or some other generic way) could be used for OP-TEE, Google Trusty or
>>> any other TEE, without changing guest configuration.
>>>
>>> Using "optee" will cause explicit configuration for OP-TEE only. I can't
>>> say that this is good or bad. It just different. Do you think that would
>>> be better approach?
>>
>> You have 2 different cases to take into account:
>> - A guest is able to deal with all supported Trusted OS. So
>> "native" will let the toolstack query the current Trusted OS and
>> expose it to the guest.
>> - A guest can only deal with a specific Trusted OS. In that case,
>> the user might want to specify in the configuration the expected
>> Trusted OS so it can't boot if not available.
>>
>> What I suggest is deferring the former case until we have another TEE
>> in hand. Maybe at that time, we will have a better naming :).
>
> Okay, so summarizing this up, you are proposing to use "optee". Am I got
> you right?
I am suggesting 'tee = "optee"' for now.
Cheers,
--
Julien Grall
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 47+ messages in thread
* [PATCH v4 08/10] xen/arm: optee: add support for RPC commands
2019-03-07 21:04 [PATCH v4 00/10] TEE mediator (and OP-TEE) support in XEN Volodymyr Babchuk
` (7 preceding siblings ...)
2019-03-07 21:04 ` [PATCH v4 09/10] tools/arm: tee: add "tee" option for xl.cfg Volodymyr Babchuk
@ 2019-03-07 21:04 ` Volodymyr Babchuk
2019-03-18 15:38 ` Julien Grall
2019-03-07 21:04 ` [PATCH v4 10/10] tools/arm: optee: create optee firmware node in DT if tee=native Volodymyr Babchuk
9 siblings, 1 reply; 47+ messages in thread
From: Volodymyr Babchuk @ 2019-03-07 21:04 UTC (permalink / raw)
To: xen-devel, tee-dev
Cc: Volodymyr Babchuk, Julien Grall, Stefano Stabellini, Volodymyr Babchuk
From: Volodymyr Babchuk <vlad.babchuk@gmail.com>
OP-TEE can issue multiple RPC requests. We are interested mostly in
request that asks NW to allocate/free shared memory for OP-TEE
needs, because mediator needs to do address translation in the same
way as it was done for shared buffers registered by NW.
OP-TEE can ask NW to allocate multiple buffers during the call. We
know that if OP-TEE asks for another buffer, we can free pglist for
the previous one.
As mediator now accesses shared command buffer, we need to shadow
it in the same way, as we shadow request buffers for STD calls.
Earlier, we just passed address of this buffer to OP-TEE, but
now we need to read and write to it, so it should be shadowed.
Signed-off-by: Volodymyr Babchuk <vlad.babchuk@gmail.com>
---
All the patches to optee.c should be merged together. They were
split to ease up review. But they depend heavily on each other.
Changes from v3:
- return value of access_guest_memory_by_ipa() now checked
- changed how information about shared buffer is stored in call
context
- domheap now used instead of xenheap
- various coding style fixes
Changes from v2:
- Use access_guest_memory_by_ipa() instead of direct mapping
---
xen/arch/arm/tee/optee.c | 228 +++++++++++++++++++++++++++++++++++++--
1 file changed, 217 insertions(+), 11 deletions(-)
diff --git a/xen/arch/arm/tee/optee.c b/xen/arch/arm/tee/optee.c
index 14e295a422..c176597500 100644
--- a/xen/arch/arm/tee/optee.c
+++ b/xen/arch/arm/tee/optee.c
@@ -31,6 +31,9 @@
*/
#define TEEC_ORIGIN_COMMS 0x00000002
+/* "Non-specific cause" */
+#define TEEC_ERROR_GENERIC 0xFFFF0000
+
/*
* "Input parameters were invalid" as described
* in GP TEE Client API Specification.
@@ -79,6 +82,7 @@ struct optee_std_call {
paddr_t guest_arg_ipa;
int optee_thread_id;
int rpc_op;
+ uint64_t rpc_data_cookie;
bool in_flight;
register_t rpc_params[2];
};
@@ -87,6 +91,9 @@ struct optee_std_call {
struct shm_rpc {
struct list_head list;
struct page_info *guest_page;
+ struct page_info *xen_arg_pg;
+ struct optee_msg_arg *xen_arg;
+ gfn_t gfn;
uint64_t cookie;
};
@@ -349,11 +356,19 @@ static struct shm_rpc *allocate_and_pin_shm_rpc(struct optee_domain *ctx,
if ( !shm_rpc )
return ERR_PTR(-ENOMEM);
+ shm_rpc->xen_arg_pg = alloc_domheap_page(current->domain, 0);
+ if ( !shm_rpc->xen_arg_pg )
+ {
+ xfree(shm_rpc);
+ return ERR_PTR(-ENOMEM);
+ }
+
/* This page will be shared with OP-TEE, so we need to pin it. */
shm_rpc->guest_page = get_page_from_gfn(current->domain, gfn_x(gfn), &t,
P2M_ALLOC);
if ( !shm_rpc->guest_page || t != p2m_ram_rw )
goto err;
+ shm_rpc->gfn = gfn;
shm_rpc->cookie = cookie;
@@ -376,8 +391,11 @@ static struct shm_rpc *allocate_and_pin_shm_rpc(struct optee_domain *ctx,
return shm_rpc;
err:
+ free_domheap_page(shm_rpc->xen_arg_pg);
+
if ( shm_rpc->guest_page )
put_page(shm_rpc->guest_page);
+
xfree(shm_rpc);
return ERR_PTR(-EINVAL);
@@ -404,12 +422,32 @@ static void free_shm_rpc(struct optee_domain *ctx, uint64_t cookie)
if ( !found )
return;
+ free_domheap_page(shm_rpc->xen_arg_pg);
+
ASSERT(shm_rpc->guest_page);
put_page(shm_rpc->guest_page);
xfree(shm_rpc);
}
+static struct shm_rpc *find_shm_rpc(struct optee_domain *ctx, uint64_t cookie)
+{
+ struct shm_rpc *shm_rpc;
+
+ spin_lock(&ctx->lock);
+ list_for_each_entry( shm_rpc, &ctx->shm_rpc_list, list )
+ {
+ if ( shm_rpc->cookie == cookie )
+ {
+ spin_unlock(&ctx->lock);
+ return shm_rpc;
+ }
+ }
+ spin_unlock(&ctx->lock);
+
+ return NULL;
+}
+
static struct optee_shm_buf *allocate_optee_shm_buf(struct optee_domain *ctx,
uint64_t cookie,
unsigned int pages_cnt,
@@ -912,13 +950,59 @@ static void free_shm_buffers(struct optee_domain *ctx,
}
/* Handle RPC return from OP-TEE */
-static void handle_rpc_return(struct cpu_user_regs *regs,
- struct optee_std_call *call)
+static int handle_rpc_return(struct optee_domain *ctx,
+ struct cpu_user_regs *regs,
+ struct optee_std_call *call)
{
+ int ret = 0;
+
call->rpc_params[0] = get_user_reg(regs, 1);
call->rpc_params[1] = get_user_reg(regs, 2);
call->optee_thread_id = get_user_reg(regs, 3);
call->rpc_op = OPTEE_SMC_RETURN_GET_RPC_FUNC(get_user_reg(regs, 0));
+
+ if ( call->rpc_op == OPTEE_SMC_RPC_FUNC_CMD )
+ {
+ /* Copy RPC request from shadowed buffer to guest */
+ uint64_t cookie = regpair_to_uint64(regs, 1);
+ struct shm_rpc *shm_rpc = find_shm_rpc(ctx, cookie);
+ if ( !shm_rpc )
+ {
+ /*
+ * This is a very exceptional situation: OP-TEE used
+ * cookie for unknown shared buffer. Something is very
+ * wrong there. We can't even report error back to OP-TEE,
+ * because there is no buffer where we can write return
+ * code. Luckily, OP-TEE sets default error code into that
+ * buffer before the call, expecting that normal world
+ * will overwrite it with actual result. So we can just
+ * continue the call.
+ */
+ gprintk(XENLOG_ERR, "Can't find SHM-RPC with cookie %lx\n", cookie);
+
+ return -ERESTART;
+ }
+
+ shm_rpc->xen_arg = __map_domain_page(shm_rpc->xen_arg_pg);
+
+ if ( access_guest_memory_by_ipa(current->domain,
+ gfn_to_gaddr(shm_rpc->gfn),
+ shm_rpc->xen_arg,
+ OPTEE_MSG_GET_ARG_SIZE(shm_rpc->xen_arg->num_params),
+ true) )
+ {
+ /*
+ * We were unable to propagate request to guest, so let's return
+ * back to OP-TEE.
+ */
+ shm_rpc->xen_arg->ret = TEEC_ERROR_GENERIC;
+ ret = -ERESTART;
+ }
+
+ unmap_domain_page(shm_rpc->xen_arg);
+ }
+
+ return ret;
}
/*
@@ -931,6 +1015,9 @@ static void handle_rpc_return(struct cpu_user_regs *regs,
* If call is complete - we need to return results with copy_std_request_back()
* and then we will destroy the call context as it is not needed anymore.
*
+ * In some rare cases we can't propagate RPC request back to guest, so we will
+ * restart the call, telling OP-TEE that request had failed.
+ *
* Shared buffers should be handled in a special way.
*/
static void execute_std_call(struct optee_domain *ctx,
@@ -939,15 +1026,23 @@ static void execute_std_call(struct optee_domain *ctx,
{
register_t optee_ret;
- forward_call(regs);
-
- optee_ret = get_user_reg(regs, 0);
- if ( OPTEE_SMC_RETURN_IS_RPC(optee_ret) )
+ while ( true )
{
- handle_rpc_return(regs, call);
- put_std_call(ctx, call);
+ forward_call(regs);
- return;
+ optee_ret = get_user_reg(regs, 0);
+ if ( OPTEE_SMC_RETURN_IS_RPC(optee_ret) )
+ {
+ if ( handle_rpc_return(ctx, regs, call) == -ERESTART )
+ {
+ set_user_reg(regs, 0, OPTEE_SMC_CALL_RETURN_FROM_RPC);
+ continue;
+ }
+
+ put_std_call(ctx, call);
+ return;
+ }
+ break;
}
copy_std_request_back(ctx, regs, call);
@@ -1024,6 +1119,117 @@ err:
return;
}
+static void handle_rpc_cmd_alloc(struct optee_domain *ctx,
+ struct cpu_user_regs *regs,
+ struct optee_std_call *call,
+ struct shm_rpc *shm_rpc)
+{
+ if ( shm_rpc->xen_arg->ret || shm_rpc->xen_arg->num_params != 1 )
+ return;
+
+ if ( shm_rpc->xen_arg->params[0].attr != (OPTEE_MSG_ATTR_TYPE_TMEM_OUTPUT |
+ OPTEE_MSG_ATTR_NONCONTIG) )
+ {
+ gdprintk(XENLOG_WARNING, "Invalid attrs for shared mem buffer: %lx\n",
+ shm_rpc->xen_arg->params[0].attr);
+ return;
+ }
+
+ /* Free pg list for buffer */
+ if ( call->rpc_data_cookie )
+ free_optee_shm_buf_pg_list(ctx, call->rpc_data_cookie);
+
+ if ( !translate_noncontig(ctx, call, &shm_rpc->xen_arg->params[0]) )
+ {
+ call->rpc_data_cookie =
+ shm_rpc->xen_arg->params[0].u.tmem.shm_ref;
+ }
+ else
+ {
+ call->rpc_data_cookie = 0;
+ /*
+ * Okay, so there was problem with guest's buffer and we need
+ * to tell about this to OP-TEE.
+ */
+ shm_rpc->xen_arg->ret = TEEC_ERROR_GENERIC;
+ shm_rpc->xen_arg->num_params = 0;
+ /*
+ * TODO: With current implementation, OP-TEE will not issue
+ * RPC to free this buffer. Guest and OP-TEE will be out of
+ * sync: guest believes that it provided buffer to OP-TEE,
+ * while OP-TEE thinks of opposite. Ideally, we need to
+ * emulate RPC with OPTEE_MSG_RPC_CMD_SHM_FREE command.
+ */
+ }
+}
+
+static void handle_rpc_cmd(struct optee_domain *ctx, struct cpu_user_regs *regs,
+ struct optee_std_call *call)
+{
+ struct shm_rpc *shm_rpc;
+ uint64_t cookie;
+ size_t arg_size;
+
+ cookie = regpair_to_uint64(regs, 1);
+
+ shm_rpc = find_shm_rpc(ctx, cookie);
+
+ if ( !shm_rpc )
+ {
+ gdprintk(XENLOG_ERR, "Can't find SHM-RPC with cookie %lx\n", cookie);
+ return;
+ }
+
+ shm_rpc->xen_arg = __map_domain_page(shm_rpc->xen_arg_pg);
+
+ /* First, copy only header to read number of arguments */
+ if ( access_guest_memory_by_ipa(current->domain,
+ gfn_to_gaddr(shm_rpc->gfn),
+ shm_rpc->xen_arg,
+ sizeof(struct optee_msg_arg),
+ false) )
+ {
+ shm_rpc->xen_arg->ret = TEEC_ERROR_GENERIC;
+ goto out;
+ }
+
+ arg_size = OPTEE_MSG_GET_ARG_SIZE(shm_rpc->xen_arg->num_params);
+ if ( arg_size > OPTEE_MSG_NONCONTIG_PAGE_SIZE )
+ {
+ shm_rpc->xen_arg->ret = TEEC_ERROR_GENERIC;
+ goto out;
+ }
+
+ /* Read the whole command structure */
+ if ( access_guest_memory_by_ipa(current->domain, gfn_to_gaddr(shm_rpc->gfn),
+ shm_rpc->xen_arg, arg_size, false) )
+ {
+ shm_rpc->xen_arg->ret = TEEC_ERROR_GENERIC;
+ goto out;
+ }
+
+ switch (shm_rpc->xen_arg->cmd)
+ {
+ case OPTEE_MSG_RPC_CMD_GET_TIME:
+ case OPTEE_MSG_RPC_CMD_WAIT_QUEUE:
+ case OPTEE_MSG_RPC_CMD_SUSPEND:
+ break;
+ case OPTEE_MSG_RPC_CMD_SHM_ALLOC:
+ handle_rpc_cmd_alloc(ctx, regs, call, shm_rpc);
+ break;
+ case OPTEE_MSG_RPC_CMD_SHM_FREE:
+ free_optee_shm_buf(ctx, shm_rpc->xen_arg->params[0].u.value.b);
+ if ( call->rpc_data_cookie == shm_rpc->xen_arg->params[0].u.value.b )
+ call->rpc_data_cookie = 0;
+ break;
+ default:
+ break;
+ }
+
+out:
+ unmap_domain_page(shm_rpc->xen_arg);
+}
+
static void handle_rpc_func_alloc(struct optee_domain *ctx,
struct cpu_user_regs *regs)
{
@@ -1051,7 +1257,7 @@ static void handle_rpc_func_alloc(struct optee_domain *ctx,
ptr = 0;
}
else
- ptr = page_to_maddr(shm_rpc->guest_page);
+ ptr = page_to_maddr(shm_rpc->xen_arg_pg);
out:
uint64_to_regpair(regs, 1, ptr);
@@ -1092,7 +1298,7 @@ static void handle_rpc(struct optee_domain *ctx, struct cpu_user_regs *regs)
case OPTEE_SMC_RPC_FUNC_FOREIGN_INTR:
break;
case OPTEE_SMC_RPC_FUNC_CMD:
- /* TODO: Add handling */
+ handle_rpc_cmd(ctx, regs, call);
break;
}
--
2.21.0
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply related [flat|nested] 47+ messages in thread* Re: [PATCH v4 08/10] xen/arm: optee: add support for RPC commands
2019-03-07 21:04 ` [PATCH v4 08/10] xen/arm: optee: add support for RPC commands Volodymyr Babchuk
@ 2019-03-18 15:38 ` Julien Grall
2019-03-20 15:36 ` Volodymyr Babchuk
0 siblings, 1 reply; 47+ messages in thread
From: Julien Grall @ 2019-03-18 15:38 UTC (permalink / raw)
To: Volodymyr Babchuk, xen-devel, tee-dev
Cc: Volodymyr Babchuk, Stefano Stabellini
Hi Volodymyr,
On 07/03/2019 21:04, Volodymyr Babchuk wrote:
> @@ -376,8 +391,11 @@ static struct shm_rpc *allocate_and_pin_shm_rpc(struct optee_domain *ctx,
> return shm_rpc;
>
> err:
> + free_domheap_page(shm_rpc->xen_arg_pg);
> +
> if ( shm_rpc->guest_page )
> put_page(shm_rpc->guest_page);
> +
Spurious change.
[...]
> + /*
> + * TODO: With current implementation, OP-TEE will not issue
> + * RPC to free this buffer. Guest and OP-TEE will be out of
> + * sync: guest believes that it provided buffer to OP-TEE,
> + * while OP-TEE thinks of opposite. Ideally, we need to
> + * emulate RPC with OPTEE_MSG_RPC_CMD_SHM_FREE command.
> + */
Can this condition happen if Xen runs out of memory?
Cheers,
--
Julien Grall
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 47+ messages in thread
* Re: [PATCH v4 08/10] xen/arm: optee: add support for RPC commands
2019-03-18 15:38 ` Julien Grall
@ 2019-03-20 15:36 ` Volodymyr Babchuk
2019-03-20 16:27 ` Julien Grall
0 siblings, 1 reply; 47+ messages in thread
From: Volodymyr Babchuk @ 2019-03-20 15:36 UTC (permalink / raw)
To: Julien Grall
Cc: tee-dev, xen-devel, Stefano Stabellini, Volodymyr Babchuk,
Volodymyr Babchuk
Hi Julien,
Julien Grall writes:
>> if ( shm_rpc->guest_page )
>> put_page(shm_rpc->guest_page);
>> +
>
> Spurious change.
Good catch. Thank you.
>
>> + /*
>> + * TODO: With current implementation, OP-TEE will not issue
>> + * RPC to free this buffer. Guest and OP-TEE will be out of
>> + * sync: guest believes that it provided buffer to OP-TEE,
>> + * while OP-TEE thinks of opposite. Ideally, we need to
>> + * emulate RPC with OPTEE_MSG_RPC_CMD_SHM_FREE command.
>> + */
> Can this condition happen if Xen runs out of memory?
Yes, this is one of reasons why translate_noncontig() might fail. It
uses both xenheap to allocate data structure and domheap to allocate
pagelists buffers. Any of those can fail, resulting in inconsistency.
You gave me idea to put gdprintk() with big fat warning there.
--
Best regards,Volodymyr Babchuk
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 47+ messages in thread
* Re: [PATCH v4 08/10] xen/arm: optee: add support for RPC commands
2019-03-20 15:36 ` Volodymyr Babchuk
@ 2019-03-20 16:27 ` Julien Grall
2019-03-20 16:47 ` Volodymyr Babchuk
0 siblings, 1 reply; 47+ messages in thread
From: Julien Grall @ 2019-03-20 16:27 UTC (permalink / raw)
To: Volodymyr Babchuk
Cc: tee-dev, xen-devel, Stefano Stabellini, Volodymyr Babchuk
On 20/03/2019 15:36, Volodymyr Babchuk wrote:
>
> Hi Julien,
Hi Volodymyr,
>
> Julien Grall writes:
>
>>> if ( shm_rpc->guest_page )
>>> put_page(shm_rpc->guest_page);
>>> +
>>
>> Spurious change.
>
> Good catch. Thank you.
>
>>
>>> + /*
>>> + * TODO: With current implementation, OP-TEE will not issue
>>> + * RPC to free this buffer. Guest and OP-TEE will be out of
>>> + * sync: guest believes that it provided buffer to OP-TEE,
>>> + * while OP-TEE thinks of opposite. Ideally, we need to
>>> + * emulate RPC with OPTEE_MSG_RPC_CMD_SHM_FREE command.
>>> + */
>> Can this condition happen if Xen runs out of memory?
>
> Yes, this is one of reasons why translate_noncontig() might fail. It
> uses both xenheap to allocate data structure and domheap to allocate
> pagelists buffers. Any of those can fail, resulting in inconsistency.
That's not very ideal. This mean a well-behaving guest can get out of sync
because of memory pressure (could be cause by a misbehaving guest on the
platform). What will be the consequence for the guest? Can it continue safely?
This feature is under EXPERT mode so I would be ok to get as this in Xen.
Although, I would like to hear Stefano's opinion here.
In any case, this should be fixed because we consider removing EXPERT mode and
security support it.
>
> You gave me idea to put gdprintk() with big fat warning there.
You clearly want to gprintk here because if this happen in non-debug build you
want to know that something has gone really wrong as soon as possible.
Cheers,
--
Julien Grall
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 47+ messages in thread
* Re: [PATCH v4 08/10] xen/arm: optee: add support for RPC commands
2019-03-20 16:27 ` Julien Grall
@ 2019-03-20 16:47 ` Volodymyr Babchuk
0 siblings, 0 replies; 47+ messages in thread
From: Volodymyr Babchuk @ 2019-03-20 16:47 UTC (permalink / raw)
To: Julien Grall
Cc: tee-dev, xen-devel, Stefano Stabellini, Volodymyr Babchuk,
Volodymyr Babchuk
Julien Grall writes:
[...]
>>>> + /*
>>>> + * TODO: With current implementation, OP-TEE will not issue
>>>> + * RPC to free this buffer. Guest and OP-TEE will be out of
>>>> + * sync: guest believes that it provided buffer to OP-TEE,
>>>> + * while OP-TEE thinks of opposite. Ideally, we need to
>>>> + * emulate RPC with OPTEE_MSG_RPC_CMD_SHM_FREE command.
>>>> + */
>>> Can this condition happen if Xen runs out of memory?
>>
>> Yes, this is one of reasons why translate_noncontig() might fail. It
>> uses both xenheap to allocate data structure and domheap to allocate
>> pagelists buffers. Any of those can fail, resulting in inconsistency.
>
> That's not very ideal. This mean a well-behaving guest can get out of
> sync because of memory pressure (could be cause by a misbehaving guest
> on the platform). What will be the consequence for the guest? Can it
> continue safely?
There will be memory leak at guest side. It will not break things right
away, subsequent calls would work as expected. But, anyways, it is
not good. I'll see how hard to emulate RPC request from the Xen to
properly fix this issue.
> This feature is under EXPERT mode so I would be ok to get as this in
> Xen. Although, I would like to hear Stefano's opinion here.
>
> In any case, this should be fixed because we consider removing EXPERT
> mode and security support it.
>
>>
>> You gave me idea to put gdprintk() with big fat warning there.
>
> You clearly want to gprintk here because if this happen in non-debug
> build you want to know that something has gone really wrong as soon as
> possible.
Yes, right. Thank you.
--
Best regards,Volodymyr Babchuk
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 47+ messages in thread
* [PATCH v4 10/10] tools/arm: optee: create optee firmware node in DT if tee=native
2019-03-07 21:04 [PATCH v4 00/10] TEE mediator (and OP-TEE) support in XEN Volodymyr Babchuk
` (8 preceding siblings ...)
2019-03-07 21:04 ` [PATCH v4 08/10] xen/arm: optee: add support for RPC commands Volodymyr Babchuk
@ 2019-03-07 21:04 ` Volodymyr Babchuk
2019-03-18 15:50 ` Julien Grall
9 siblings, 1 reply; 47+ messages in thread
From: Volodymyr Babchuk @ 2019-03-07 21:04 UTC (permalink / raw)
To: xen-devel, tee-dev; +Cc: Volodymyr Babchuk, Wei Liu, Ian Jackson
From: Volodymyr Babchuk <vlad.babchuk@gmail.com>
If TEE support is enabled with "tee=native" option in xl.cfg,
then we need to inform guest about available TEE.
Currently only OP-TEE is supported, so we'll create DT
node in a way that is expected by optee driver in linux.
Signed-off-by: Volodymyr Babchuk <vlad.babchuk@gmail.com>
---
This patch depends on patches to optee.c.
Changes from v3:
- "smc" method replaced with "hvc"
- Coding style fixes
---
tools/libxl/libxl_arm.c | 29 +++++++++++++++++++++++++++++
1 file changed, 29 insertions(+)
diff --git a/tools/libxl/libxl_arm.c b/tools/libxl/libxl_arm.c
index 6930d0ab3b..9c342a3b0d 100644
--- a/tools/libxl/libxl_arm.c
+++ b/tools/libxl/libxl_arm.c
@@ -420,6 +420,32 @@ static int make_psci_node(libxl__gc *gc, void *fdt)
return 0;
}
+static int make_optee_node(libxl__gc *gc, void *fdt)
+{
+ int res;
+ LOG(DEBUG, "Creating OP-TEE node in dtb");
+
+ res = fdt_begin_node(fdt, "firmware");
+ if (res) return res;
+
+ res = fdt_begin_node(fdt, "optee");
+ if (res) return res;
+
+ res = fdt_property_compat(gc, fdt, 1, "linaro,optee-tz");
+ if (res) return res;
+
+ res = fdt_property_string(fdt, "method", "hvc");
+ if (res) return res;
+
+ res = fdt_end_node(fdt);
+ if (res) return res;
+
+ res = fdt_end_node(fdt);
+ if (res) return res;
+
+ return 0;
+}
+
static int make_memory_nodes(libxl__gc *gc, void *fdt,
const struct xc_dom_image *dom)
{
@@ -933,6 +959,9 @@ next_resize:
if (info->arch_arm.vuart == LIBXL_VUART_TYPE_SBSA_UART)
FDT( make_vpl011_uart_node(gc, fdt, ainfo, dom) );
+ if (info->arch_arm.tee == LIBXL_TEE_TYPE_NATIVE)
+ FDT( make_optee_node(gc, fdt) );
+
if (pfdt)
FDT( copy_partial_fdt(gc, fdt, pfdt) );
--
2.21.0
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply related [flat|nested] 47+ messages in thread* Re: [PATCH v4 10/10] tools/arm: optee: create optee firmware node in DT if tee=native
2019-03-07 21:04 ` [PATCH v4 10/10] tools/arm: optee: create optee firmware node in DT if tee=native Volodymyr Babchuk
@ 2019-03-18 15:50 ` Julien Grall
0 siblings, 0 replies; 47+ messages in thread
From: Julien Grall @ 2019-03-18 15:50 UTC (permalink / raw)
To: Volodymyr Babchuk, xen-devel, tee-dev
Cc: Volodymyr Babchuk, Ian Jackson, Wei Liu
Hi,
On 07/03/2019 21:04, Volodymyr Babchuk wrote:
> static int make_memory_nodes(libxl__gc *gc, void *fdt,
> const struct xc_dom_image *dom)
> {
> @@ -933,6 +959,9 @@ next_resize:
> if (info->arch_arm.vuart == LIBXL_VUART_TYPE_SBSA_UART)
> FDT( make_vpl011_uart_node(gc, fdt, ainfo, dom) );
>
> + if (info->arch_arm.tee == LIBXL_TEE_TYPE_NATIVE)
Pending the discussion on the name:
Reviewed-by: Julien Grall <julien.grall@arm.com>
> + FDT( make_optee_node(gc, fdt) );
> +
> if (pfdt)
> FDT( copy_partial_fdt(gc, fdt, pfdt) );
>
>
Cheers,
--
Julien Grall
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 47+ messages in thread