Re: [PATCH v19 8/9] arm_ffa: efi: introduce FF-A MM communication

[Ilias Apalodimas <ilias.apalodimas@linaro.org>]

On Fri, Aug 04, 2023 at 02:33:44PM +0100, Abdellatif El Khlifi wrote:
> Add MM communication support using FF-A transport
> 
> This feature allows accessing MM partitions services through
> EFI MM communication protocol. MM partitions such as StandAlonneMM
> or smm-gateway secure partitions which reside in secure world.
> 
> An MM shared buffer and a door bell event are used to exchange
> the data.
> 
> The data is used by EFI services such as GetVariable()/SetVariable()
> and copied from the communication buffer to the MM shared buffer.
> 
> The secure partition is notified about availability of data in the
> MM shared buffer by an FF-A message (door bell).
> 
> On such event, MM SP can read the data and updates the MM shared
> buffer with the response data.
> 
> The response data is copied back to the communication buffer and
> consumed by the EFI subsystem.
> 
> MM communication protocol supports FF-A 64-bit direct messaging.
> 
> We tested the FF-A MM communication on the Corstone-1000 platform.
> 
> We ran the UEFI SCT test suite containing EFI setVariable, getVariable and
> getNextVariable tests which involve FF-A MM communication and all tests
> are passing with the current changes.
> 
> We made the SCT test reports (part of the ACS results) public following the
> latest Corstone-1000 platform software release. Please find the test
> reports at [1].
> 
> [1]: https://gitlab.arm.com/arm-reference-solutions/arm-reference-solutions-test-report/-/tree/master/embedded-a/corstone1000/CORSTONE1000-2023.06/acs_results_fpga.zip
> 
> Signed-off-by: Abdellatif El Khlifi <abdellatif.elkhlifi@arm.com>
> Tested-by: Gowtham Suresh Kumar <gowtham.sureshkumar@arm.com>
> Reviewed-by: Simon Glass <sjg@chromium.org>
> Cc: Tom Rini <trini@konsulko.com>
> Cc: Ilias Apalodimas <ilias.apalodimas@linaro.org>
> Cc: Jens Wiklander <jens.wiklander@linaro.org>
> 
> ---
> 
> Changelog:
> ===============
> 
> v19:
> 
> Tom:
> 
> * use CONFIG_FFA_SHARED_MM_BUF_* in place of macros
> 
> v18:
> 
> Ilias, Tom:
> 
> * drop the use of configs for the shared MM buffer, put back #ifdefs instead
> * add test information to the commit log
> 

Thanks for the quick rework.  I like the fact you have the SMC as a
fallback to FF-A in case it's not discovered.  Once and if we get dynamic 
buffer allocation for the NS-world we can easily switch this to an
automatically scanable feature again.

Tom, I am fine with the series now.  I've reviewed the relevant EFI and asm
bits, feel free to pull it in if the rest of the people are happy. 

Reviewed-by: Ilias Apalodimas <ilias.apalodimas@linaro.org>



> v17:
> 
> * show a debug message rather than an error when FF-A is not detected
> 
> v16:
> 
> * lib/efi_loader/Kconfig:
>    rather than automatically selecting OPTEE and ARM_FFA_TRANSPORT configs by
>    EFI_MM_COMM_TEE, set them as dependencies (Otherwise FF-A will be automatically
>    enabled for boards that don't need it).
> 
> v15:
> 
> Simon:
> 
> * replace FFA_SHARED_MM_BUFFER_* defines with configs
> 
> v14:
> 
> Ilias:
> 
> * drop truncating var_payload->size when using FF-A
> * map the MM SP return codes to errnos
> 
> v13:
> 
> * remove FF-A and Optee ifdefs
> 
> v12:
> 
> * drop use of calloc when querying SPs
> * address nits
> 
> v11:
> 
> * rename select_ffa_mm_comms() to select_mm_comms()
> * improve the logic of MM transport selection in mm_communicate()
> * addressing nits
> 
> v10:
> 
> * use the FF-A driver Uclass operations
> * use uclass_first_device()
> * addressing nits
> 
> v9: align how FF-A is used with FF-A discovery through DM
> 
> v8:
> 
> * isolate the compilation choices between FF-A and OP-TEE
> * update partition_info_get() second argument to be an SP count
> * pass NULL device pointer to the FF-A bus discovery and operations
> 
> v7:
> 
> * set the MM door bell event to use 64-bit direct messaging
> * issue a compile time error when one of these macros are not found :
>   FFA_SHARED_MM_BUFFER_SIZE, FFA_SHARED_MM_BUFFER_OFFSET, FFA_SHARED_MM_BUFFER_ADDR
> * make mm_sp_svc_uuid static
> * replace EINVAL with ENOMEM in ffa_discover_mm_sp_id() when calloc() fails
> * improve use of unmap_sysmem() in ffa_mm_communicate()
> 
> v6:
> 
> * add FF-A runtime discovery at MM communication level
> * drop EFI runtime support for FF-A MM communication
> * revert the changes in include/mm_communication.h for
>   efi_mm_communicate_header and smm_variable_access structures
> 
> v4:
> 
> * use the new FF-A driver interfaces
> * discover MM partitions at runtime
> * copy FF-A driver private data to EFI runtime section at
>   ExitBootServices()
> * drop use of FFA_ERR_STAT_SUCCESS error code
> * replace EFI_BUFFER_TOO_SMALL with EFI_OUT_OF_RESOURCES
>   in ffa_mm_communicate(). No need for efi_memcpy_runtime() anymore
> * revert the error log in mm_communicate() in case of failure
> * remove packed attribute from efi_mm_communicate_header and
>   smm_variable_communicate_header
> 
> v2:
> 
> * set default values to 0 for FFA_SHARED_MM_BUFFER_SIZE, FFA_SHARED_MM_BUFFER_ADDR and MM_SP_UUID_DATA and add warnings
> 
> v1:
> 
> * introduce FF-A MM communication
> 
> include/mm_communication.h        |  17 ++
>  lib/efi_loader/Kconfig            |  39 ++++-
>  lib/efi_loader/efi_variable_tee.c | 270 +++++++++++++++++++++++++++++-
>  3 files changed, 320 insertions(+), 6 deletions(-)
> 
> diff --git a/include/mm_communication.h b/include/mm_communication.h
> index e65fbde60d..f38f1a5344 100644
> --- a/include/mm_communication.h
> +++ b/include/mm_communication.h
> @@ -6,6 +6,9 @@
>   *  Copyright (c) 2017, Intel Corporation. All rights reserved.
>   *  Copyright (C) 2020 Linaro Ltd. <sughosh.ganu@linaro.org>
>   *  Copyright (C) 2020 Linaro Ltd. <ilias.apalodimas@linaro.org>
> + *  Copyright 2022-2023 Arm Limited and/or its affiliates <open-source-office@arm.com>
> + *    Authors:
> + *      Abdellatif El Khlifi <abdellatif.elkhlifi@arm.com>
>   */
>  
>  #ifndef _MM_COMMUNICATION_H_
> @@ -13,6 +16,11 @@
>  
>  #include <part_efi.h>
>  
> +#if CONFIG_IS_ENABLED(ARM_FFA_TRANSPORT)
> +/* MM service UUID string (big-endian format). This UUID is  common across all MM SPs */
> +#define MM_SP_UUID	"33d532ed-e699-0942-c09c-a798d9cd722d"
> +#endif
> +
>  /*
>   * Interface to the pseudo Trusted Application (TA), which provides a
>   * communication channel with the Standalone MM (Management Mode)
> @@ -248,4 +256,13 @@ struct smm_variable_var_check_property {
>  	u16                       name[];
>  };
>  
> +#if CONFIG_IS_ENABLED(ARM_FFA_TRANSPORT)
> +/* supported MM transports */
> +enum mm_comms_select {
> +	MM_COMMS_UNDEFINED,
> +	MM_COMMS_FFA,
> +	MM_COMMS_OPTEE
> +};
> +#endif
> +
>  #endif /* _MM_COMMUNICATION_H_ */
> diff --git a/lib/efi_loader/Kconfig b/lib/efi_loader/Kconfig
> index c5835e6ef6..852f9671d8 100644
> --- a/lib/efi_loader/Kconfig
> +++ b/lib/efi_loader/Kconfig
> @@ -55,13 +55,50 @@ config EFI_VARIABLE_FILE_STORE
>  	  stored as file /ubootefi.var on the EFI system partition.
>  
>  config EFI_MM_COMM_TEE
> -	bool "UEFI variables storage service via OP-TEE"
> +	bool "UEFI variables storage service via the trusted world"
>  	depends on OPTEE
>  	help
> +	  Allowing access to the MM SP services (SPs such as  StandAlonneMM, smm-gateway).
> +	  When using the u-boot OP-TEE driver, StandAlonneMM is supported.
> +	  When using the u-boot FF-A  driver any MM SP is supported.
> +
>  	  If OP-TEE is present and running StandAloneMM, dispatch all UEFI
>  	  variable related operations to that. The application will verify,
>  	  authenticate and store the variables on an RPMB.
>  
> +	  When ARM_FFA_TRANSPORT is used, dispatch all UEFI variable related
> +	  operations to the MM SP running in the secure world.
> +	  A door bell mechanism is used to notify the SP when there is data in the shared
> +	  MM buffer. The data is copied by u-boot to the shared buffer before issuing
> +	  the door bell event.
> +
> +config FFA_SHARED_MM_BUF_SIZE
> +	int "Memory size of the shared MM communication buffer"
> +	depends on EFI_MM_COMM_TEE && ARM_FFA_TRANSPORT
> +	help
> +	  This defines the size in bytes of the memory area reserved for the shared
> +	  buffer used for communication between the MM feature in U-Boot and
> +	  the MM SP in secure world.
> +	  The size of the memory region must be a multiple of the size of the maximum
> +	  translation granule size that is specified in the ID_AA64MMFR0_EL1 System register.
> +	  It is assumed that the MM SP knows the size of the shared MM communication buffer.
> +
> +config FFA_SHARED_MM_BUF_OFFSET
> +	int "Data offset in the shared MM communication buffer"
> +	depends on EFI_MM_COMM_TEE && ARM_FFA_TRANSPORT
> +	help
> +	  This defines the offset in bytes of the data read or written to in the shared
> +	  buffer by the MM SP.
> +
> +config FFA_SHARED_MM_BUF_ADDR
> +	hex "Define the address of the shared MM communication buffer"
> +	depends on EFI_MM_COMM_TEE && ARM_FFA_TRANSPORT
> +	help
> +	  This defines the address of the shared MM communication buffer
> +	  used for communication between the MM feature in U-Boot and
> +	  the MM SP in secure world.
> +	  It is assumed that the MM SP knows the address of the shared MM communication buffer.
> +
>  config EFI_VARIABLE_NO_STORE
>  	bool "Don't persist non-volatile UEFI variables"
>  	help
> diff --git a/lib/efi_loader/efi_variable_tee.c b/lib/efi_loader/efi_variable_tee.c
> index dfef18435d..09d03c0eee 100644
> --- a/lib/efi_loader/efi_variable_tee.c
> +++ b/lib/efi_loader/efi_variable_tee.c
> @@ -4,16 +4,38 @@
>   *
>   *  Copyright (C) 2019 Linaro Ltd. <sughosh.ganu@linaro.org>
>   *  Copyright (C) 2019 Linaro Ltd. <ilias.apalodimas@linaro.org>
> + *  Copyright 2022-2023 Arm Limited and/or its affiliates <open-source-office@arm.com>
> + *
> + *  Authors:
> + *    Abdellatif El Khlifi <abdellatif.elkhlifi@arm.com>
>   */
>  
>  #include <common.h>
> +#if CONFIG_IS_ENABLED(ARM_FFA_TRANSPORT)
> +#include <arm_ffa.h>
> +#endif
> +#include <cpu_func.h>
> +#include <dm.h>
>  #include <efi.h>
>  #include <efi_api.h>
>  #include <efi_loader.h>
>  #include <efi_variable.h>
> -#include <tee.h>
>  #include <malloc.h>
> +#include <mapmem.h>
>  #include <mm_communication.h>
> +#include <tee.h>
> +
> +#if CONFIG_IS_ENABLED(ARM_FFA_TRANSPORT)
> +/* MM return codes */
> +#define MM_SUCCESS (0)
> +#define MM_NOT_SUPPORTED (-1)
> +#define MM_INVALID_PARAMETER (-2)
> +#define MM_DENIED (-3)
> +#define MM_NO_MEMORY (-5)
> +
> +static const char *mm_sp_svc_uuid = MM_SP_UUID;
> +static u16 mm_sp_id;
> +#endif
>  
>  extern struct efi_var_file __efi_runtime_data *efi_var_buf;
>  static efi_uintn_t max_buffer_size;	/* comm + var + func + data */
> @@ -144,12 +166,238 @@ static efi_status_t optee_mm_communicate(void *comm_buf, ulong dsize)
>  	return ret;
>  }
>  
> +#if CONFIG_IS_ENABLED(ARM_FFA_TRANSPORT)
> +/**
> + * ffa_notify_mm_sp() - Announce there is data in the shared buffer
> + *
> + * Notify the MM partition in the trusted world that
> + * data is available in the shared buffer.
> + * This is a blocking call during which trusted world has exclusive access
> + * to the MM shared buffer.
> + *
> + * Return:
> + *
> + * 0 on success
> + */
> +static int ffa_notify_mm_sp(void)
> +{
> +	struct ffa_send_direct_data msg = {0};
> +	int ret;
> +	int sp_event_ret;
> +	struct udevice *dev;
> +
> +	ret = uclass_first_device_err(UCLASS_FFA, &dev);
> +	if (ret) {
> +		log_err("EFI: Cannot find FF-A bus device, notify MM SP failure\n");
> +		return ret;
> +	}
> +
> +	msg.data0 = CONFIG_FFA_SHARED_MM_BUF_OFFSET; /* x3 */
> +
> +	ret = ffa_sync_send_receive(dev, mm_sp_id, &msg, 1);
> +	if (ret)
> +		return ret;
> +
> +	sp_event_ret = msg.data0; /* x3 */
> +
> +	switch (sp_event_ret) {
> +	case MM_SUCCESS:
> +		ret = 0;
> +		break;
> +	case MM_NOT_SUPPORTED:
> +		ret = -EINVAL;
> +		break;
> +	case MM_INVALID_PARAMETER:
> +		ret = -EPERM;
> +		break;
> +	case MM_DENIED:
> +		ret = -EACCES;
> +		break;
> +	case MM_NO_MEMORY:
> +		ret = -EBUSY;
> +		break;
> +	default:
> +		ret = -EACCES;
> +	}
> +
> +	return ret;
> +}
> +
> +/**
> + * ffa_discover_mm_sp_id() - Query the MM partition ID
> + *
> + * Use the FF-A driver to get the MM partition ID.
> + * If multiple partitions are found, use the first one.
> + * This is a boot time function.
> + *
> + * Return:
> + *
> + * 0 on success
> + */
> +static int ffa_discover_mm_sp_id(void)
> +{
> +	u32 count = 0;
> +	int ret;
> +	struct ffa_partition_desc *descs;
> +	struct udevice *dev;
> +
> +	ret = uclass_first_device_err(UCLASS_FFA, &dev);
> +	if (ret) {
> +		log_err("EFI: Cannot find FF-A bus device, MM SP discovery failure\n");
> +		return ret;
> +	}
> +
> +	/* Ask the driver to fill the buffer with the SPs info */
> +	ret = ffa_partition_info_get(dev, mm_sp_svc_uuid, &count, &descs);
> +	if (ret) {
> +		log_err("EFI: Failure in querying SPs info (%d), MM SP discovery failure\n", ret);
> +		return ret;
> +	}
> +
> +	/* MM SPs found , use the first one */
> +
> +	mm_sp_id = descs[0].info.id;
> +
> +	log_info("EFI: MM partition ID 0x%x\n", mm_sp_id);
> +
> +	return 0;
> +}
> +
>  /**
> - * mm_communicate() - Adjust the cmonnucation buffer to StandAlonneMM and send
> + * ffa_mm_communicate() - Exchange EFI services data with  the MM partition using FF-A
> + * @comm_buf:		locally allocated communication buffer used for rx/tx
> + * @dsize:				communication buffer size
> + *
> + * Issue a door bell event to notify the MM partition (SP) running in OP-TEE
> + * that there is data to read from the shared buffer.
> + * Communication with the MM SP is performed using FF-A transport.
> + * On the event, MM SP can read the data from the buffer and
> + * update the MM shared buffer with response data.
> + * The response data is copied back to the communication buffer.
> + *
> + * Return:
> + *
> + * EFI status code
> + */
> +static efi_status_t ffa_mm_communicate(void *comm_buf, ulong comm_buf_size)
> +{
> +	ulong tx_data_size;
> +	int ffa_ret;
> +	efi_status_t efi_ret;
> +	struct efi_mm_communicate_header *mm_hdr;
> +	void *virt_shared_buf;
> +
> +	if (!comm_buf)
> +		return EFI_INVALID_PARAMETER;
> +
> +	/* Discover MM partition ID at boot time */
> +	if (!mm_sp_id && ffa_discover_mm_sp_id()) {
> +		log_err("EFI: Failure to discover MM SP ID at boot time, FF-A MM comms failure\n");
> +		return EFI_UNSUPPORTED;
> +	}
> +
> +	mm_hdr = (struct efi_mm_communicate_header *)comm_buf;
> +	tx_data_size = mm_hdr->message_len + sizeof(efi_guid_t) + sizeof(size_t);
> +
> +	if (comm_buf_size != tx_data_size || tx_data_size > CONFIG_FFA_SHARED_MM_BUF_SIZE)
> +		return EFI_INVALID_PARAMETER;
> +
> +	/* Copy the data to the shared buffer */
> +
> +	virt_shared_buf = map_sysmem((phys_addr_t)CONFIG_FFA_SHARED_MM_BUF_ADDR, 0);
> +	memcpy(virt_shared_buf, comm_buf, tx_data_size);
> +
> +	/*
> +	 * The secure world might have cache disabled for
> +	 * the device region used for shared buffer (which is the case for Optee).
> +	 * In this case, the secure world reads the data from DRAM.
> +	 * Let's flush the cache so the DRAM is updated with the latest data.
> +	 */
> +#ifdef CONFIG_ARM64
> +	invalidate_dcache_all();
> +#endif
> +
> +	/* Announce there is data in the shared buffer */
> +
> +	ffa_ret = ffa_notify_mm_sp();
> +
> +	switch (ffa_ret) {
> +	case 0: {
> +		ulong rx_data_size;
> +		/* Copy the MM SP response from the shared buffer to the communication buffer */
> +		rx_data_size = ((struct efi_mm_communicate_header *)virt_shared_buf)->message_len +
> +			sizeof(efi_guid_t) +
> +			sizeof(size_t);
> +
> +		if (rx_data_size > comm_buf_size) {
> +			efi_ret = EFI_OUT_OF_RESOURCES;
> +			break;
> +		}
> +
> +		memcpy(comm_buf, virt_shared_buf, rx_data_size);
> +		efi_ret = EFI_SUCCESS;
> +		break;
> +	}
> +	case -EINVAL:
> +		efi_ret = EFI_DEVICE_ERROR;
> +		break;
> +	case -EPERM:
> +		efi_ret = EFI_INVALID_PARAMETER;
> +		break;
> +	case -EACCES:
> +		efi_ret = EFI_ACCESS_DENIED;
> +		break;
> +	case -EBUSY:
> +		efi_ret = EFI_OUT_OF_RESOURCES;
> +		break;
> +	default:
> +		efi_ret = EFI_ACCESS_DENIED;
> +	}
> +
> +	unmap_sysmem(virt_shared_buf);
> +	return efi_ret;
> +}
> +
> +/**
> + * get_mm_comms() - detect the available MM transport
> + *
> + * Make sure the FF-A bus is probed successfully
> + * which means FF-A communication with secure world works and ready
> + * for use.
> + *
> + * If FF-A bus is not ready, use OPTEE comms.
> + *
> + * Return:
> + *
> + * MM_COMMS_FFA or MM_COMMS_OPTEE
> + */
> +static enum mm_comms_select get_mm_comms(void)
> +{
> +	struct udevice *dev;
> +	int ret;
> +
> +	ret = uclass_first_device_err(UCLASS_FFA, &dev);
> +	if (ret) {
> +		log_debug("EFI: Cannot find FF-A bus device, trying Optee comms\n");
> +		return MM_COMMS_OPTEE;
> +	}
> +
> +	return MM_COMMS_FFA;
> +}
> +#endif
> +
> +/**
> + * mm_communicate() - Adjust the communication buffer to the MM SP and send
>   * it to OP-TEE
>   *
> - * @comm_buf:		locally allocted communcation buffer
> + * @comm_buf:		locally allocated communication buffer
>   * @dsize:		buffer size
> + *
> + * The SP (also called partition) can be any MM SP such as  StandAlonneMM or smm-gateway.
> + * The comm_buf format is the same for both partitions.
> + * When using the u-boot OP-TEE driver, StandAlonneMM is supported.
> + * When using the u-boot FF-A  driver, any MM SP is supported.
> + *
>   * Return:		status code
>   */
>  static efi_status_t mm_communicate(u8 *comm_buf, efi_uintn_t dsize)
> @@ -157,12 +405,24 @@ static efi_status_t mm_communicate(u8 *comm_buf, efi_uintn_t dsize)
>  	efi_status_t ret;
>  	struct efi_mm_communicate_header *mm_hdr;
>  	struct smm_variable_communicate_header *var_hdr;
> +#if CONFIG_IS_ENABLED(ARM_FFA_TRANSPORT)
> +	enum mm_comms_select mm_comms;
> +#endif
>  
>  	dsize += MM_COMMUNICATE_HEADER_SIZE + MM_VARIABLE_COMMUNICATE_SIZE;
>  	mm_hdr = (struct efi_mm_communicate_header *)comm_buf;
>  	var_hdr = (struct smm_variable_communicate_header *)mm_hdr->data;
>  
> -	ret = optee_mm_communicate(comm_buf, dsize);
> +#if CONFIG_IS_ENABLED(ARM_FFA_TRANSPORT)
> +	mm_comms = get_mm_comms();
> +	if (mm_comms == MM_COMMS_FFA)
> +		ret = ffa_mm_communicate(comm_buf, dsize);
> +	else
> +		ret = optee_mm_communicate(comm_buf, dsize);
> +#else
> +		ret = optee_mm_communicate(comm_buf, dsize);
> +#endif
> +
>  	if (ret != EFI_SUCCESS) {
>  		log_err("%s failed!\n", __func__);
>  		return ret;
> @@ -697,7 +957,7 @@ void efi_variables_boot_exit_notify(void)
>  		ret = EFI_NOT_FOUND;
>  
>  	if (ret != EFI_SUCCESS)
> -		log_err("Unable to notify StMM for ExitBootServices\n");
> +		log_err("Unable to notify the MM partition for ExitBootServices\n");
>  	free(comm_buf);
>  
>  	/*
> -- 
> 2.25.1
>