[U-Boot] [PATCH v7 3/7] ARM: socfpga: Add FPGA drivers for Arria 10 FPGA bitstream loading

[Marek Vasut <marex@denx.de>]

On 2/2/19 4:27 AM, Chee, Tien Fong wrote:
> On Fri, 2019-02-01 at 12:12 -0800, Dalon L Westergreen wrote:
>> On Thu, 2019-01-31 at 22:51 +0800, tien.fong.chee at intel.com wrote:
>>>
>>> From: Tien Fong Chee <tien.fong.chee@intel.com>
>>>
>>> Add FPGA driver to support program FPGA with FPGA bitstream loading
>>> from
>>> filesystem. The driver are designed based on generic firmware
>>> loader
>>> framework. The driver can handle FPGA program operation from
>>> loading FPGA
>>> bitstream in flash to memory and then to program FPGA.
>>>
>>> Signed-off-by: Tien Fong Chee <tien.fong.chee@intel.com>
>>>
>>> ---
>>>
>>> changes for v7
>>> - Restructure the FPGA driver to support both peripheral bitstream
>>> and core
>>>   bitstream bundled into FIT image.
>>> - Support loadable property for core bitstream. User can set
>>> loadable
>>>   in DDR for better performance. This loading would be done in one
>>> large
>>>   chunk instead of chunk by chunk loading with small memory buffer.
>>> ---
>>>  arch/arm/dts/socfpga_arria10_socdk_sdmmc.dts       |  18 +
>>>  .../include/mach/fpga_manager_arria10.h            |  39 +-
>>>  drivers/fpga/socfpga_arria10.c                     | 417
>>> ++++++++++++++++++++-
>>>  3 files changed, 457 insertions(+), 17 deletions(-)
>>>
>>> diff --git a/arch/arm/dts/socfpga_arria10_socdk_sdmmc.dts
>>> b/arch/arm/dts/socfpga_arria10_socdk_sdmmc.dts
>>> index 998d811..dc55618 100644
>>> --- a/arch/arm/dts/socfpga_arria10_socdk_sdmmc.dts
>>> +++ b/arch/arm/dts/socfpga_arria10_socdk_sdmmc.dts
>>> @@ -18,6 +18,24 @@
>>>  /dts-v1/;
>>>  #include "socfpga_arria10_socdk.dtsi"
>>>  
>>> +/ {
>>> +	chosen {
>>> +		firmware-loader = &fs_loader0;
>>> +	};
>>> +
>>> +	fs_loader0: fs-loader at 0 {
>>> +		u-boot,dm-pre-reloc;
>>> +		compatible = "u-boot,fs-loader";
>>> +		phandlepart = <&mmc 1>;
>>> +	};
>>> +};
>>> +
>>> +&fpga_mgr {
>>> +	u-boot,dm-pre-reloc;
>>> +	altr,bitstream = "fit_spl_fpga.itb";
>>> +	altr,bitstream-core = "fit_spl_fpga.itb";
>>> +};
>>> +
>>>  &mmc {
>>>  	u-boot,dm-pre-reloc;
>>>  	status = "okay";
>>> diff --git a/arch/arm/mach-
>>> socfpga/include/mach/fpga_manager_arria10.h
>>> b/arch/arm/mach-socfpga/include/mach/fpga_manager_arria10.h
>>> index 09d13f6..683c84c 100644
>>> --- a/arch/arm/mach-socfpga/include/mach/fpga_manager_arria10.h
>>> +++ b/arch/arm/mach-socfpga/include/mach/fpga_manager_arria10.h
>>> @@ -1,9 +1,13 @@
>>>  /* SPDX-License-Identifier: GPL-2.0 */
>>>  /*
>>> - * Copyright (C) 2017 Intel Corporation <www.intel.com>
>>> + * Copyright (C) 2017-2019 Intel Corporation <www.intel.com>
>>>   * All rights reserved.
>>>   */
>>>  
>>>
>> [...]	}
>>>
>>> +
>>> +	if (!is_fpgamgr_early_user_mode() ||
>>> is_fpgamgr_user_mode()) {
>>> +		fpga_node_name = "fpga-1";
>>> +		printf("FPGA: Start to program peripheral
>>> bitstream ...\n");
>>> +	} else if (!is_fpgamgr_user_mode()) {
>>> +		fpga_node_name = "fpga-2";
>>> +		printf("FPGA: Start to program core bitstream
>>> ...\n");
>>> +	}
>> Why are you relying on the node name to define the core / periph RBF?
>> Would
>> it not be better to define this in a property?
> I need to ensure correct sequence of programming the rbf images at
> different FPGA mode. fpga with index 1 always means the 1st to program.
> 
>>  how would one have multiple
>> core and periph rbfs in a single fit image?
> I want strict control to avoid the performance penalty, when this
> happen, performance is almost same as programming full image.
> 
> So, in a single FIT image, only have one core rbf and periph rbf, and
> core rbf must always be the 1st and with the right absolute data
> position to avoid the buffer unaligned. Strict sequence control is also
> one of the reason having one core rbf and one periph rbf in a single
> FIT image.

I thought we established that you cannot depend on the placement and
alignment of files within fitImage ?

Also, you should be able to use fitImage configurations to select from
multiple core and periph RBFs, there's no point in limiting ourselves to
1 core and 1 periph RBF with flexible format such as the fitImage.

> You can create multiple FIT image for multiple core and periph rbfs.
> There are properties such as "altr,bitstream" and "altr,bitstream-core" 
> for supporting different FIT image.
> 
> You can change those properties in U-Boot env(for runtime), or using
> different DTS for differnt configuration.
> 
>>
>> --dalon
>>
>>>
>>> +
>>> +	node_offset = fit_image_get_node(buffer_p,
>>> fpga_node_name);
>>> +	if (node_offset < 0) {
>>> +		debug("FPGA: Could not find node '%s' in FIT\n",
>>> +		     fpga_node_name);
>>> +		return -ENOENT;
>>> +	}
>>> +
>>> +	const char *uname = fit_get_name(buffer_p, node_offset,
>>> NULL);
>>> +
>>> +	if (uname)
>>> +		debug("FPGA: %s\n", uname);
>>> +
>>> +	ret = fit_image_get_data_position(buffer_p, node_offset,
>>> &rbf_offset);
>>> +	if (ret < 0) {
>>> +		debug("FPGA: Could not find data position
>>> (err=%d)\n", ret);
>>> +		return -ENOENT;
>>> +	}
>>> +
>>> +	ret = fit_image_get_data_size(buffer_p, node_offset,
>>> &rbf_size);
>>> +	if (ret < 0) {
>>> +		debug("FPGA: Could not find data size (err=%d)\n",
>>> ret);
>>> +		return -ENOENT;
>>> +	}
>>> +
>>> +	ret = fit_image_get_load(buffer_p, node_offset, (ulong
>>> *)loadable);
>>> +	if (ret < 0) {
>>> +		debug("FPGA: Could not find loadable (err=%d)\n",
>>> ret);
>>> +		debug("FPGA: Using default buffer and size\n");
>>> +	} else {
>>> +		buffer_p = (u32 *)*loadable;
>>> +		buffer_size = rbf_size;
>>> +		debug("FPGA: Found loadable address = 0x%x\n",
>>> *loadable);
>>> +	}
>>> +
>>> +	debug("FPGA: External data: offset = 0x%x, size = 0x%x\n",
>>> +	      rbf_offset, rbf_size);
>>> +
>>> +	fpga_loadfs->remaining = rbf_size;
>>> +
>>> +	/*
>>> +	 * Determine buffer size vs bitstream size, and
>>> calculating number of
>>> +	 * chunk by chunk transfer is required due to smaller
>>> buffer size
>>> +	 * compare to bitstream
>>> +	 */
>>> +	if (rbf_size <= buffer_size) {
>>> +		/* Loading whole bitstream into buffer */
>>> +		buffer_size = rbf_size;
>>> +		fpga_loadfs->remaining = 0;
>>> +	} else {
>>> +		fpga_loadfs->remaining -= buffer_size;
>>> +	}
>>> +
>>> +	fpga_loadfs->offset = rbf_offset;
>>> +	/* Loading bitstream into buffer */
>>> +	ret = request_firmware_into_buf(dev,
>>> +					fpga_loadfs->fpga_fsinfo-
>>>> filename,
>>> +					buffer_p,
>>> +					buffer_size,
>>> +					fpga_loadfs->offset);
>>> +	if (ret < 0) {
>>> +		debug("FPGA: Failed to read bitstream from
>>> flash.\n");
>>> +		return -ENOENT;
>>> +	}
>>> +
>>> +	/* Update next reading bitstream offset */
>>> +	fpga_loadfs->offset += buffer_size;
>>> +
>>> +	/* Getting info about bitstream types */
>>> +	get_rbf_image_info(&fpga_loadfs->rbfinfo, (u16
>>> *)buffer_p);
>>> +
>>> +	/* Update the final addr for bitstream */
>>> +	*buffer = (u32)buffer_p;
>>> +
>>> +	/* Update the size of bitstream to be programmed into FPGA
>>> */
>>> +	*buffer_bsize = buffer_size;
>>> +
>>> +	return 0;
>>> +}
>>> +
>>> +static int subsequent_loading_rbf_to_buffer(struct udevice *dev,
>>> +					struct fpga_loadfs_info
>>> *fpga_loadfs,
>>> +					u32 *buffer, size_t
>>> *buffer_bsize)
>>> +{
>>> +	int ret = 0;
>>> +	u32 *buffer_p = (u32 *)*buffer;
>>> +
>>> +	/* Read the bitstream chunk by chunk. */
>>> +	if (fpga_loadfs->remaining > *buffer_bsize) {
>>> +		fpga_loadfs->remaining -= *buffer_bsize;
>>> +	} else {
>>> +		*buffer_bsize = fpga_loadfs->remaining;
>>> +		fpga_loadfs->remaining = 0;
>>> +	}
>>> +
>>> +	ret = request_firmware_into_buf(dev,
>>> +					fpga_loadfs->fpga_fsinfo-
>>>> filename,
>>> +					buffer_p,
>>> +					*buffer_bsize,
>>> +					fpga_loadfs->offset);
>>> +	if (ret < 0) {
>>> +		debug("FPGA: Failed to read bitstream from
>>> flash.\n");
>>> +		return -ENOENT;
>>> +	}
>>> +
>>> +	/* Update next reading bitstream offset */
>>> +	fpga_loadfs->offset += *buffer_bsize;
>>> +
>>> +	return 0;
>>> +}
>>> +
>>> +int socfpga_loadfs(fpga_fs_info *fpga_fsinfo, const void *buf,
>>> size_t bsize,
>>> +			u32 offset)
>>> +{
>>> +	struct fpga_loadfs_info fpga_loadfs;
>>> +	int status = 0;
>>> +	int ret = 0;
>>> +	u32 buffer = (u32)buf;
>>> +	size_t buffer_sizebytes = bsize;
>>> +	size_t buffer_sizebytes_ori = bsize;
>>> +	size_t total_sizeof_image = 0;
>>> +	struct udevice *dev;
>>> +
>>> +	ret = uclass_get_device(UCLASS_FS_FIRMWARE_LOADER, 0,
>>> &dev);
>>> +	if (ret)
>>> +		return ret;
>>> +
>>> +	memset(&fpga_loadfs, 0, sizeof(fpga_loadfs));
>>> +
>>> +	WATCHDOG_RESET();
>>> +
>>> +	fpga_loadfs.fpga_fsinfo = fpga_fsinfo;
>>> +	fpga_loadfs.offset = offset;
>>> +
>>> +	printf("FPGA: Start to program ...\n");
>>> +
>>> +	/*
>>> +	 * Note: Both buffer and buffer_sizebytes values can be
>>> altered by
>>> +	 * function below.
>>> +	 */
>>> +	ret = first_loading_rbf_to_buffer(dev, &fpga_loadfs,
>>> &buffer,
>>> +					   &buffer_sizebytes);
>>> +	if (ret)
>>> +		return ret;
>>> +
>>> +	if (fpga_loadfs.rbfinfo.section == core_section &&
>>> +		!(is_fpgamgr_early_user_mode() &&
>>> !is_fpgamgr_user_mode())) {
>>> +		debug("FPGA : Must be in Early Release mode to
>>> program ");
>>> +		debug("core bitstream.\n");
>>> +		return 0;
>>> +	}
>>> +
>>> +	/* Disable all signals from HPS peripheral controller to
>>> FPGA */
>>> +	writel(0, &system_manager_base->fpgaintf_en_global);
>>> +
>>> +	/* Disable all axi bridges (hps2fpga, lwhps2fpga &
>>> fpga2hps) */
>>> +	socfpga_bridges_reset();
>>> +
>>> +	if (fpga_loadfs.rbfinfo.section == periph_section) {
>>> +		/* Initialize the FPGA Manager */
>>> +		status = fpgamgr_program_init((u32 *)buffer,
>>> buffer_sizebytes);
>>> +		if (status) {
>>> +			debug("FPGA: Init with peripheral
>>> bitstream failed.\n");
>>> +			return -EPERM;
>>> +		}
>>> +	}
>>> +
>>> +	WATCHDOG_RESET();
>>> +
>>> +	/* Transfer bitstream to FPGA Manager */
>>> +	fpgamgr_program_write((void *)buffer, buffer_sizebytes);
>>> +
>>> +	total_sizeof_image += buffer_sizebytes;
>>> +
>>> +	WATCHDOG_RESET();
>>> +
>>> +	while (fpga_loadfs.remaining) {
>>> +		ret = subsequent_loading_rbf_to_buffer(dev,
>>> +							&fpga_load
>>> fs,
>>> +							&buffer,
>>> +							&buffer_si
>>> zebytes_ori);
>>> +
>>> +		if (ret)
>>> +			return ret;
>>> +
>>> +		/* Transfer data to FPGA Manager */
>>> +		fpgamgr_program_write((void *)buffer,
>>> +					buffer_sizebytes_ori);
>>> +
>>> +		total_sizeof_image += buffer_sizebytes_ori;
>>> +
>>> +		WATCHDOG_RESET();
>>> +	}
>>> +
>>> +	if (fpga_loadfs.rbfinfo.section == periph_section) {
>>> +		if (fpgamgr_wait_early_user_mode() != -ETIMEDOUT)
>>> {
>>> +			config_pins(gd->fdt_blob, "shared");
>>> +			puts("FPGA: Early Release Succeeded.\n");
>>> +		} else {
>>> +			debug("FPGA: Failed to see Early
>>> Release.\n");
>>> +			return -EIO;
>>> +		}
>>> +
>>> +		/* For monolithic bitstream */
>>> +		if (is_fpgamgr_user_mode()) {
>>> +			/* Ensure the FPGA entering config done */
>>> +			status = fpgamgr_program_finish();
>>> +			if (status)
>>> +				return status;
>>> +
>>> +			config_pins(gd->fdt_blob, "fpga");
>>> +			puts("FPGA: Enter user mode.\n");
>>> +		}
>>> +	} else if (fpga_loadfs.rbfinfo.section == core_section) {
>>> +		/* Ensure the FPGA entering config done */
>>> +		status = fpgamgr_program_finish();
>>> +		if (status)
>>> +			return status;
>>> +
>>> +		config_pins(gd->fdt_blob, "fpga");
>>> +		puts("FPGA: Enter user mode.\n");
>>> +	} else {
>>> +		debug("FPGA: Config Error: Unsupported bitstream
>>> type.\n");
>>> +		return -ENOEXEC;
>>> +	}
>>> +
>>> +	return (int)total_sizeof_image;
>>> +}
>>> +#endif
>>> +
>>> +/* This function is used to load the core bitstream from the
>>> OCRAM. */
>>>  int socfpga_load(Altera_desc *desc, const void *rbf_data, size_t
>>> rbf_size)
>>>  {
>>> -	int status;
>>> +	unsigned long status;
>>> +	struct rbf_info rbfinfo;
>>>  
>>> -	/* disable all signals from hps peripheral controller to
>>> fpga */
>>> +	memset(&rbfinfo, 0, sizeof(rbfinfo));
>>> +
>>> +	/* Disable all signals from hps peripheral controller to
>>> fpga */
>>>  	writel(0, &system_manager_base->fpgaintf_en_global);
>>>  
>>> -	/* disable all axi bridge (hps2fpga, lwhps2fpga &
>>> fpga2hps) */
>>> +	/* Disable all axi bridge (hps2fpga, lwhps2fpga &
>>> fpga2hps) */
>>>  	socfpga_bridges_reset();
>>>  
>>> -	/* Initialize the FPGA Manager */
>>> -	status = fpgamgr_program_init((u32 *)rbf_data, rbf_size);
>>> -	if (status)
>>> -		return status;
>>> +	/* Getting info about bitstream types */
>>> +	get_rbf_image_info(&rbfinfo, (u16 *)rbf_data);
>>> +
>>> +	if (rbfinfo.section == periph_section) {
>>> +		/* Initialize the FPGA Manager */
>>> +		status = fpgamgr_program_init((u32 *)rbf_data,
>>> rbf_size);
>>> +		if (status)
>>> +			return status;
>>> +	}
>>> +
>>> +	if (rbfinfo.section == core_section &&
>>> +		!(is_fpgamgr_early_user_mode() &&
>>> !is_fpgamgr_user_mode())) {
>>> +		debug("FPGA : Must be in early release mode to
>>> program ");
>>> +		debug("core bitstream.\n");
>>> +		return 0;
>>> +	}
>>>  
>>> -	/* Write the RBF data to FPGA Manager */
>>> +	/* Write the bitstream to FPGA Manager */
>>>  	fpgamgr_program_write(rbf_data, rbf_size);
>>>  
>>> -	return fpgamgr_program_finish();
>>> +	status = fpgamgr_program_finish();
>>> +	if (status) {
>>> +		config_pins(gd->fdt_blob, "fpga");
>>> +		puts("FPGA: Enter user mode.\n");
>>> +	}
>>> +
>>> +	return status;
>>>  }


-- 
Best regards,
Marek Vasut