[Buildroot] [PATCH v3 1/1] package/mfgtools: bump to version 1.3.154

[Buildroot] [PATCH v3 1/1] package/mfgtools: bump to version 1.3.154

[Jörg Krause]

The version 0.02 was a pre-release and is dated from Nov 20, 2017.

Meanwhile:
 * the repo owner switch to NXPmicro
 * latest version is 1.3.154
 * the build system is CMake
 * the license is BSD-3 only
 * update the license hash as the copyright year and the owner has
   changed
 * drop the readme.txt file as is outdated
 * drop patch, which is not needed with the new version
 * update dependencies

Note, that mfgtools uses git to define a version string `GIT_VERSION`.
It does so even when building from a source tarball (automatically
generated by github). The problem is, that git provides the version
information of Buildroot and mfgtools uses this version information to
do a runtime check to detect outdated command list scripts.

We use a hook which the version generation script provides to write the
correct version string.

Signed-off-by: J?rg Krause <joerg.krause@embedded.rocks>
---
v3:
 * bump to latest version 1.3.154
 * adapt version string hook to changes done upstream
 * add optional host OpenSSL dependency to add support for https
   download (introduced in 1.3.102)

v2:
 * fix URL in Config.in.host
 * overwrite gen_ver.sh instead of patching it and creating gitversion.h
   [suggested by Arnout]
---
 ...onflicting-declaration-of-__time64_t.patch | 27 --------
 package/mfgtools/Config.in.host               | 17 +++--
 package/mfgtools/mfgtools.hash                |  5 +-
 package/mfgtools/mfgtools.mk                  | 40 +++++------
 package/mfgtools/readme.txt                   | 69 -------------------
 5 files changed, 31 insertions(+), 127 deletions(-)
 delete mode 100644 package/mfgtools/0001-lnx_def.h-fix-conflicting-declaration-of-__time64_t.patch
 delete mode 100644 package/mfgtools/readme.txt

diff --git a/package/mfgtools/0001-lnx_def.h-fix-conflicting-declaration-of-__time64_t.patch b/package/mfgtools/0001-lnx_def.h-fix-conflicting-declaration-of-__time64_t.patch
deleted file mode 100644
index 7d0d3974a2..0000000000
--- a/package/mfgtools/0001-lnx_def.h-fix-conflicting-declaration-of-__time64_t.patch
+++ /dev/null
@@ -1,27 +0,0 @@
-From 1af7bc09451e0b779f7bfb0dcc58ffb3abe9c584 Mon Sep 17 00:00:00 2001
-From: Gary Bisson <bisson.gary@gmail.com>
-Date: Tue, 11 Feb 2020 15:30:12 +0100
-Subject: [PATCH] lnx_def.h: fix conflicting declaration of __time64_t
-
-Signed-off-by: Gary Bisson <bisson.gary@gmail.com>
----
- MfgToolLib/lnx_def.h | 2 ++
- 1 file changed, 2 insertions(+)
-
-diff --git a/MfgToolLib/lnx_def.h b/MfgToolLib/lnx_def.h
-index c6869dd..a38bb99 100644
---- a/MfgToolLib/lnx_def.h
-+++ b/MfgToolLib/lnx_def.h
-@@ -124,7 +124,9 @@ typedef unsigned long * ULONG_PTR;
- typedef char * LPOLESTR;
- typedef void * HDEVINFO;
- typedef unsigned long long ULONGLONG;
-+#ifndef __time64_t
- typedef long long __time64_t;
-+#endif
- typedef long long __int64;
- typedef unsigned long DWORD_PTR;
- typedef unsigned int UINT;
--- 
-2.25.0
-
diff --git a/package/mfgtools/Config.in.host b/package/mfgtools/Config.in.host
index 6f2f3a3268..061a2f5238 100644
--- a/package/mfgtools/Config.in.host
+++ b/package/mfgtools/Config.in.host
@@ -2,9 +2,16 @@ config BR2_PACKAGE_HOST_MFGTOOLS
 	bool "host mfgtools"
 	depends on BR2_arm
 	help
-	  This package contains the Freescale manufacturing tool.
-	  It is designed to program firmware to i.MX boards during
-	  production. The communication is done over USB using the
-	  Freescale UTP protocol.
+	  Freescale/NXP i.MX Chip image deploy tools.
 
-	  https://github.com/codeauroraforum/mfgtools
+	  https://github.com/NXPmicro/mfgtools
+
+if BR2_PACKAGE_HOST_MFGTOOLS
+
+config BR2_PACKAGE_HOST_MFGTOOLS_SSL
+	bool "https support"
+	select BR2_PACKAGE_HOST_OPENSSL
+	help
+	  Enable https support.
+
+endif
diff --git a/package/mfgtools/mfgtools.hash b/package/mfgtools/mfgtools.hash
index 0b215f5295..754a5782da 100644
--- a/package/mfgtools/mfgtools.hash
+++ b/package/mfgtools/mfgtools.hash
@@ -1,4 +1,3 @@
 # locally computed
-sha256  055d71227d18883d6e8bc9e854c076015f9a7749820a94272e19071bf0b25c89  mfgtools-0.02.tar.gz
-sha256  2655559a6bb1179eae514f5c7166f4ede4f2453efa9cf4dc3c045cab5d57dede  LICENSE
-sha256  0963b6e5086bf454265b0f57821a02b681d1211e40ad74c310231cb4d94815c9  README.txt
+sha256  19f34b6152786b662c7ba755ad1b05421803ebb4d87b2aaccde8e1cf25efd987  mfgtools-1.3.154.tar.gz
+sha256  cc8d47f7b9260f6669ecd41c24554c552f17581d81ee8fc602c6d23edb8bf495  LICENSE
diff --git a/package/mfgtools/mfgtools.mk b/package/mfgtools/mfgtools.mk
index 48ce5dda28..04165c35da 100644
--- a/package/mfgtools/mfgtools.mk
+++ b/package/mfgtools/mfgtools.mk
@@ -4,31 +4,25 @@
 #
 ################################################################################
 
-MFGTOOLS_VERSION = 0.02
-MFGTOOLS_SITE = $(call github,codeauroraforum,mfgtools,v$(MFGTOOLS_VERSION))
-MFGTOOLS_SUBDIR = MfgToolLib
-MFGTOOLS_LICENSE = BSD-3-Clause or CPOL
-MFGTOOLS_LICENSE_FILES = LICENSE README.txt
-HOST_MFGTOOLS_DEPENDENCIES = host-libusb
+MFGTOOLS_VERSION = 1.3.154
+MFGTOOLS_SITE = $(call github,NXPmicro,mfgtools,uuu_$(MFGTOOLS_VERSION))
+MFGTOOLS_LICENSE = BSD-3-Clause
+MFGTOOLS_LICENSE_FILES = LICENSE
+HOST_MFGTOOLS_DEPENDENCIES = host-libusb host-libzip host-zlib
 
-HOST_MFGTOOLS_CFLAGS = \
-	$(HOST_CFLAGS) $(HOST_LDFLAGS) -std=c++11 -lpthread \
-	-L$(@D)/MfgToolLib -lMfgToolLib -I$(@D)/MfgToolLib \
-	-lusb-1.0 -I$(HOST_DIR)/include/libusb-1.0 \
-	-fpermissive -Wno-write-strings
+ifeq ($(BR2_PACKAGE_HOST_MFGTOOLS_SSL),y)
+HOST_MFGTOOLS_DEPENDENCIES += host-openssl
+endif
 
-define HOST_MFGTOOLS_CLI_BUILD
-	$(HOST_CONFIGURE_OPTS) $(MAKE) CC="$(HOSTCXX)" \
-		CFLAGS="$(HOST_MFGTOOLS_CFLAGS)" -C $(@D)/TestPrgm
-endef
-
-HOST_MFGTOOLS_POST_BUILD_HOOKS += HOST_MFGTOOLS_CLI_BUILD
-
-define HOST_MFGTOOLS_INSTALL_CMDS
-	$(INSTALL) -D -m 755 $(@D)/MfgToolLib/libMfgToolLib.so \
-		$(HOST_DIR)/lib/libMfgToolLib.so
-	$(INSTALL) -D -m 755 $(@D)/TestPrgm/mfgtoolcli \
-		$(HOST_DIR)/bin/mfgtoolcli
+# Version string generation is broken in mfgtools as it relies on git, even when
+# building from a source tarball. The version string is used by mfgtools to do a
+# runtime check to detect outdated command list scripts. The script used to
+# create the version string allows to set a custom version string by creating a
+# file named `.tarball-version`. Use this hook to create this file, so mfgtools
+# runs with the correct version string (preceeded by `uuu_` as done by default).
+define HOST_MFGTOOLS_CREATE_VERSION_FILE
+	echo "uuu_$(MFGTOOLS_VERSION)" > $(@D)/.tarball-version
 endef
+HOST_MFGTOOLS_POST_PATCH_HOOKS += HOST_MFGTOOLS_CREATE_VERSION_FILE
 
 $(eval $(host-cmake-package))
diff --git a/package/mfgtools/readme.txt b/package/mfgtools/readme.txt
deleted file mode 100644
index 320e6ec493..0000000000
--- a/package/mfgtools/readme.txt
+++ /dev/null
@@ -1,69 +0,0 @@
-MfgTools Howto
-==============
-
-1. Build your mfgtool image
-
-Make sure to enable the following Buildroot options:
-
-BR2_PACKAGE_FREESCALE_IMX=y
-BR2_PACKAGE_IMX_UUC=y
-BR2_TARGET_ROOTFS_CPIO=y
-BR2_TARGET_ROOTFS_CPIO_GZIP=y
-BR2_TARGET_ROOTFS_CPIO_UIMAGE=y
-
-Also modify your kernel configuration to have:
-
-CONFIG_USB_GADGET=y
-CONFIG_USB_MASS_STORAGE=y
-CONFIG_FSL_UTP=y
-CONFIG_MMC_BLOCK_MINORS=16
-
-2. Go into the output and create the necessary folders
-
-$ cd output
-$ mkdir -p "Profiles/Linux/OS Firmware/firmware"
-
-3. Create your XML update script named ucl2.xml
-
-You can find a sample XML at:
-
-$ wget https://storage.googleapis.com/boundarydevices.com/ucl2.xml \
-  -O Profiles/Linux/OS\ Firmware/ucl2.xml
-
-4. Copy the U-Boot, Kernel and initramfs images to the appropriate
-folder
-
-$ cp images/u-boot.imx images/zImage images/imx6q-sabrelite.dtb \
-  images/rootfs.cpio.uboot Profiles/Linux/OS\ Firmware/firmware/
-
-5. Copy the prebuilt binaries to be flashed
-
-Depending on your ucl2.xml file, the sample doesn't flash anything.
-
-6. Run the MfgTools client:
-
-$ ./host/bin/mfgtoolcli -l mmc -s uboot_defconfig=imx \
-  -s dtbname=imx6q-sabrelite.dtb -s initramfs=rootfs.cpio.uboot \
-  -s mmc=1 -p 1
-
-For more information about the tools options, please read the
-"Manufacturing Tool V2 Quick Start Guide.docx" documentation contained
-in every mfgtools package from NXP website[1].
-
-Note: All the above commands require your Linux host user to have
-permissions to access the USB devices. Please make sure to have udev
-rules that allow the user to communicate with the BootROM IDs
-(Freescale USB recovery) as well as the one used for the UTP Linux
-image (0x066F:0x37FF).  Using 'sudo' in front of the mfgtoolcli
-command would also grant you the necessary permission but it is *not*
-recommended.
-
-Also, if your U-Boot environment doesn't include mfgtools bootargs,
-make sure to set the following:
-
-setenv bootargs "console=${console},${baudrate} g_mass_storage.stall=0 \
-	g_mass_storage.removable=1 g_mass_storage.idVendor=0x066F \
-	g_mass_storage.idProduct=0x37FF g_mass_storage.iSerialNumber=\"\" \
-	g_mass_storage.file=/fat"
-
-[1] http://www.nxp.com/products/software-and-tools/software-development-tools/i.mx-software-and-tools/i.mx-6-series-software-and-development-tool-resources:IMX6_SW
-- 
2.26.0

[Buildroot] [PATCH v3 1/1] package/mfgtools: bump to version 1.3.154

[Thomas Petazzoni]

Hello,

On Mon, 13 Apr 2020 23:42:49 +0200
J?rg Krause <joerg.krause@embedded.rocks> wrote:

> +MFGTOOLS_VERSION = 1.3.154
> +MFGTOOLS_SITE = $(call github,NXPmicro,mfgtools,uuu_$(MFGTOOLS_VERSION))
> +MFGTOOLS_LICENSE = BSD-3-Clause
> +MFGTOOLS_LICENSE_FILES = LICENSE
> +HOST_MFGTOOLS_DEPENDENCIES = host-libusb host-libzip host-zlib

This is missing host-bzip2, otherwise the build fails with:

CMake Error at /home/br-user/output/host/share/cmake-3.15/Modules/FindPackageHandleStandardArgs.cmake:137 (message):
  Could NOT find BZip2 (missing: BZIP2_LIBRARIES BZIP2_INCLUDE_DIR)
Call Stack (most recent call first):
  /home/br-user/output/host/share/cmake-3.15/Modules/FindPackageHandleStandardArgs.cmake:378 (_FPHSA_FAILURE_MESSAGE)
  /home/br-user/output/host/share/cmake-3.15/Modules/FindBZip2.cmake:63 (FIND_PACKAGE_HANDLE_STANDARD_ARGS)
  libuuu/CMakeLists.txt:6 (find_package)

It also fails to build without host-openssl:

-- Could NOT find OpenSSL, try to set the path to OpenSSL root folder in the system variable OPENSSL_ROOT_DIR (missing: OPENSSL_CRYPTO_LIBRARY OPENSSL_INCLUDE_DIR) 
CMake Error: The following variables are used in this project, but they are set to NOTFOUND.
Please set them or make sure they are set and tested correctly in the CMake files:
OPENSSL_CRYPTO_LIBRARY (ADVANCED)
    linked by target "uuu" in directory /home/br-user/output/build/host-mfgtools-1.3.154/uuu
OPENSSL_SSL_LIBRARY (ADVANCED)
    linked by target "uuu" in directory /home/br-user/output/build/host-mfgtools-1.3.154/uuu

But looking more at the source code, uuu/CMakeLists.txt is checking for
openssl, but the code in uuu/ does not use it. libuuu/ is using it, and
makes it conditional.

This all seems to be pretty broken :-/

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

[Buildroot] [PATCH v3 1/1] package/mfgtools: bump to version 1.3.154

[Thomas Petazzoni]

On Tue, 19 May 2020 23:08:45 +0200
Thomas Petazzoni <thomas.petazzoni@bootlin.com> wrote:

> This is missing host-bzip2, otherwise the build fails with:
> 
> CMake Error at /home/br-user/output/host/share/cmake-3.15/Modules/FindPackageHandleStandardArgs.cmake:137 (message):
>   Could NOT find BZip2 (missing: BZIP2_LIBRARIES BZIP2_INCLUDE_DIR)
> Call Stack (most recent call first):
>   /home/br-user/output/host/share/cmake-3.15/Modules/FindPackageHandleStandardArgs.cmake:378 (_FPHSA_FAILURE_MESSAGE)
>   /home/br-user/output/host/share/cmake-3.15/Modules/FindBZip2.cmake:63 (FIND_PACKAGE_HANDLE_STANDARD_ARGS)
>   libuuu/CMakeLists.txt:6 (find_package)
> 
> It also fails to build without host-openssl:
> 
> -- Could NOT find OpenSSL, try to set the path to OpenSSL root folder in the system variable OPENSSL_ROOT_DIR (missing: OPENSSL_CRYPTO_LIBRARY OPENSSL_INCLUDE_DIR) 
> CMake Error: The following variables are used in this project, but they are set to NOTFOUND.
> Please set them or make sure they are set and tested correctly in the CMake files:
> OPENSSL_CRYPTO_LIBRARY (ADVANCED)
>     linked by target "uuu" in directory /home/br-user/output/build/host-mfgtools-1.3.154/uuu
> OPENSSL_SSL_LIBRARY (ADVANCED)
>     linked by target "uuu" in directory /home/br-user/output/build/host-mfgtools-1.3.154/uuu
> 
> But looking more at the source code, uuu/CMakeLists.txt is checking for
> openssl, but the code in uuu/ does not use it. libuuu/ is using it, and
> makes it conditional.
> 
> This all seems to be pretty broken :-/

Another issue is that mfgtools overwrites CMAKE_CXX_FLAGS:

libuuu/CMakeLists.txt:set(CMAKE_CXX_FLAGS "-O2 ${UUUSSL}")

due to this, we lose the CMAKE_CXX_FLAGS passed on the command line by
Buildroot, which includes -I$(HOST_DIR)/include, and the build of
host-mfgtools does not find the libraries installed in $(HOST_DIR).

I would recommend you to build in a more minimal environment to detect
such issues. Indeed, you have not noticed any of the issue I have
reported because you have bzip2 and openssl installed system-wide.

Best regards,

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

[Buildroot] [PATCH v3 1/1] package/mfgtools: bump to version 1.3.154

[Gary Bisson]

Hi Jorg,

On Mon, Apr 13, 2020 at 11:42:49PM +0200, J?rg Krause wrote:
> The version 0.02 was a pre-release and is dated from Nov 20, 2017.
> 
> Meanwhile:
>  * the repo owner switch to NXPmicro
>  * latest version is 1.3.154
>  * the build system is CMake
>  * the license is BSD-3 only
>  * update the license hash as the copyright year and the owner has
>    changed
>  * drop the readme.txt file as is outdated
>  * drop patch, which is not needed with the new version
>  * update dependencies
> 
> Note, that mfgtools uses git to define a version string `GIT_VERSION`.
> It does so even when building from a source tarball (automatically
> generated by github). The problem is, that git provides the version
> information of Buildroot and mfgtools uses this version information to
> do a runtime check to detect outdated command list scripts.

Actually in latest release, the tarball generated includes a
.tarball-version file that includes the proper revision.
It is supported since 1.3.169, which is considered "pre-release" as it
isn't used in a GA BSP yet, but I think that feature is interesting
enough to bump the package already.

> We use a hook which the version generation script provides to write the
> correct version string.
> 
> Signed-off-by: J?rg Krause <joerg.krause@embedded.rocks>
> ---
> v3:
>  * bump to latest version 1.3.154
>  * adapt version string hook to changes done upstream
>  * add optional host OpenSSL dependency to add support for https
>    download (introduced in 1.3.102)

I'm confused I thought we agreed that it'd be best to keep mgftools
as-is and only create a new "uuu" or "imx-uuu" package not to break
compatibility with people using the mfgtools v2? [1]
Has there been a new development in that discussion?

Regards,
Gary

[1] http://patchwork.ozlabs.org/project/buildroot/patch/20200109191020.1282319-2-joerg.krause at embedded.rocks/

[Buildroot] [PATCH v3 1/1] package/mfgtools: bump to version 1.3.154

[Jörg Krause]

Hi Gary,

On Wed, 2020-05-20 at 14:53 +0200, Gary Bisson wrote:
> Hi Jorg,
> 
> On Mon, Apr 13, 2020 at 11:42:49PM +0200, J?rg Krause wrote:
> > The version 0.02 was a pre-release and is dated from Nov 20, 2017.
> > 
> > Meanwhile:
> >  * the repo owner switch to NXPmicro
> >  * latest version is 1.3.154
> >  * the build system is CMake
> >  * the license is BSD-3 only
> >  * update the license hash as the copyright year and the owner has
> >    changed
> >  * drop the readme.txt file as is outdated
> >  * drop patch, which is not needed with the new version
> >  * update dependencies
> > 
> > Note, that mfgtools uses git to define a version string `GIT_VERSION`.
> > It does so even when building from a source tarball (automatically
> > generated by github). The problem is, that git provides the version
> > information of Buildroot and mfgtools uses this version information to
> > do a runtime check to detect outdated command list scripts.
> 
> Actually in latest release, the tarball generated includes a
> .tarball-version file that includes the proper revision.
> It is supported since 1.3.169, which is considered "pre-release" as it
> isn't used in a GA BSP yet, but I think that feature is interesting
> enough to bump the package already.

I've tested latest version 1.3.191 and the hook is not necessary
anymore.

> > We use a hook which the version generation script provides to write the
> > correct version string.
> > 
> > Signed-off-by: J?rg Krause <joerg.krause@embedded.rocks>
> > ---
> > v3:
> >  * bump to latest version 1.3.154
> >  * adapt version string hook to changes done upstream
> >  * add optional host OpenSSL dependency to add support for https
> >    download (introduced in 1.3.102)
> 
> I'm confused I thought we agreed that it'd be best to keep mgftools
> as-is and only create a new "uuu" or "imx-uuu" package not to break
> compatibility with people using the mfgtools v2? [1]
> Has there been a new development in that discussion?

Right, we both agreed on that. I proposed this patch before we agreed.
I didn't removed it from patchwork after that because I was hoping for
further discussion on the subject.

> 
> [1] http://patchwork.ozlabs.org/project/buildroot/patch/20200109191020.1282319-2-joerg.krause at embedded.rocks/

Best regards
J?rg Krause