[U-Boot] [PATCH v2 0/8] tools: Support building U-Boot host tools for Windows via MSYS2

[U-Boot] [PATCH v2 0/8] tools: Support building U-Boot host tools for Windows via MSYS2

[Bin Meng]

Per current U-Boot README, building Windows versions of the utilities
in the tools directory is done via the MinGW toolchain. But testing
shows that it is broken and actually it must have been broken for
quite a long time.

Fixing MinGW build seems quite amount of work as developers of
U-Boot normally work on Linux boxes hence codes written are mainly
for Linux or POSIX OSes. We must investigate another way of building
host tools for Windows, and now we have MSYS2, a software distro and
building platform for Windows, to build POSIX compliant software on
Windows using an emulation layer.

This small series fixes several issues in current U-Boot tools codes,
that only assume a Linux host is used. Cases are using standard C
typedefs whenever possbile, or using compiler builtin functions to
improve portability, etc.

A reST document for how to build U-Boot host tools for both platforms
is added.

A new CI pipeline configuration for Microsoft Azure Pipelines is added
for U-Boot. We rely on it to ensure future host tools for Windows build
does not break. I've also investigated putting what we have for now on
GitLab-CI and Travis-CI to Azure Pipelines, but buildman hangs forever
and the root cause is not figured out yet.

Changes in v2:
- new patch: Add .gitattributes for line endings
- new patch: tools: Avoid creating symbolic links for tools/version.h
- new patch: Add Micirosoft Azure pipelines configuration

Bin Meng (8):
  tools: image.h: Use portable uint32_t instead of linux-specific __be32
  tools: mtk_image.h: Use portable uintXX_t instead of linux-specific
    __leXX
  tools: zynqmpbif: Use compiler builtin instead of linux-specific
    __swab32
  linux/types.h: Surround 'struct ustat' with __linux__
  doc: Add documentation for how to build U-Boot host tools
  Add .gitattributes for line endings
  tools: Avoid creating symbolic links for tools/version.h
  Add Micirosoft Azure pipelines configuration

 .gitattributes        |  2 ++
 Makefile              |  9 ++++--
 azure-pipelines.yml   | 35 +++++++++++++++++++++
 doc/build/index.rst   |  9 ++++++
 doc/build/tools.rst   | 47 ++++++++++++++++++++++++++++
 doc/index.rst         | 11 +++++++
 include/image.h       | 14 ++++-----
 include/linux/types.h |  2 ++
 tools/.gitignore      |  1 +
 tools/mtk_image.h     | 86 +++++++++++++++++++++++++--------------------------
 tools/version.h       |  1 -
 tools/zynqmpbif.c     |  2 +-
 12 files changed, 164 insertions(+), 55 deletions(-)
 create mode 100644 .gitattributes
 create mode 100644 azure-pipelines.yml
 create mode 100644 doc/build/index.rst
 create mode 100644 doc/build/tools.rst
 delete mode 120000 tools/version.h

-- 
2.7.4

[U-Boot] [PATCH v2 1/8] tools: image.h: Use portable uint32_t instead of linux-specific __be32

[Bin Meng]

__be32 has Linux kernel specific __attribute__((bitwise)) which is
not portable. Use uint32_t instead.

Signed-off-by: Bin Meng <bmeng.cn@gmail.com>
---

Changes in v2: None

 include/image.h | 14 +++++++-------
 1 file changed, 7 insertions(+), 7 deletions(-)

diff --git a/include/image.h b/include/image.h
index c1065c0..f4d2aaf 100644
--- a/include/image.h
+++ b/include/image.h
@@ -319,13 +319,13 @@ enum {
  * all data in network byte order (aka natural aka bigendian).
  */
 typedef struct image_header {
-	__be32		ih_magic;	/* Image Header Magic Number	*/
-	__be32		ih_hcrc;	/* Image Header CRC Checksum	*/
-	__be32		ih_time;	/* Image Creation Timestamp	*/
-	__be32		ih_size;	/* Image Data Size		*/
-	__be32		ih_load;	/* Data	 Load  Address		*/
-	__be32		ih_ep;		/* Entry Point Address		*/
-	__be32		ih_dcrc;	/* Image Data CRC Checksum	*/
+	uint32_t	ih_magic;	/* Image Header Magic Number	*/
+	uint32_t	ih_hcrc;	/* Image Header CRC Checksum	*/
+	uint32_t	ih_time;	/* Image Creation Timestamp	*/
+	uint32_t	ih_size;	/* Image Data Size		*/
+	uint32_t	ih_load;	/* Data	 Load  Address		*/
+	uint32_t	ih_ep;		/* Entry Point Address		*/
+	uint32_t	ih_dcrc;	/* Image Data CRC Checksum	*/
 	uint8_t		ih_os;		/* Operating System		*/
 	uint8_t		ih_arch;	/* CPU architecture		*/
 	uint8_t		ih_type;	/* Image Type			*/
-- 
2.7.4

[U-Boot] [PATCH v2 2/8] tools: mtk_image.h: Use portable uintXX_t instead of linux-specific __leXX

[Bin Meng]

__leXX has Linux kernel specific __attribute__((bitwise)) which is
not portable. Use corresponding uintXX_t instead.

Signed-off-by: Bin Meng <bmeng.cn@gmail.com>
---

Changes in v2: None

 tools/mtk_image.h | 86 +++++++++++++++++++++++++++----------------------------
 1 file changed, 43 insertions(+), 43 deletions(-)

diff --git a/tools/mtk_image.h b/tools/mtk_image.h
index 0a9eab3..4e78b3d 100644
--- a/tools/mtk_image.h
+++ b/tools/mtk_image.h
@@ -9,14 +9,14 @@
 #ifndef _MTK_IMAGE_H
 #define _MTK_IMAGE_H
 
-/* Device header definitions */
+/* Device header definitions, all fields are little-endian */
 
 /* Header for NOR/SD/eMMC */
 union gen_boot_header {
 	struct {
 		char name[12];
-		__le32 version;
-		__le32 size;
+		uint32_t version;
+		uint32_t size;
 	};
 
 	uint8_t pad[0x200];
@@ -32,14 +32,14 @@ union nand_boot_header {
 		char name[12];
 		char version[4];
 		char id[8];
-		__le16 ioif;
-		__le16 pagesize;
-		__le16 addrcycles;
-		__le16 oobsize;
-		__le16 pages_of_block;
-		__le16 numblocks;
-		__le16 writesize_shift;
-		__le16 erasesize_shift;
+		uint16_t ioif;
+		uint16_t pagesize;
+		uint16_t addrcycles;
+		uint16_t oobsize;
+		uint16_t pages_of_block;
+		uint16_t numblocks;
+		uint16_t writesize_shift;
+		uint16_t erasesize_shift;
 		uint8_t dummy[60];
 		uint8_t ecc_parity[28];
 	};
@@ -54,14 +54,14 @@ union nand_boot_header {
 /* BootROM layout header */
 struct brom_layout_header {
 	char name[8];
-	__le32 version;
-	__le32 header_size;
-	__le32 total_size;
-	__le32 magic;
-	__le32 type;
-	__le32 header_size_2;
-	__le32 total_size_2;
-	__le32 unused;
+	uint32_t version;
+	uint32_t header_size;
+	uint32_t total_size;
+	uint32_t magic;
+	uint32_t type;
+	uint32_t header_size_2;
+	uint32_t total_size_2;
+	uint32_t unused;
 };
 
 #define BRLYT_NAME		"BRLYT"
@@ -90,8 +90,8 @@ struct gen_device_header {
 struct gfh_common_header {
 	uint8_t magic[3];
 	uint8_t version;
-	__le16 size;
-	__le16 type;
+	uint16_t size;
+	uint16_t type;
 };
 
 #define GFH_HEADER_MAGIC	"MMM"
@@ -106,17 +106,17 @@ struct gfh_common_header {
 struct gfh_file_info {
 	struct gfh_common_header gfh;
 	char name[12];
-	__le32 unused;
-	__le16 file_type;
+	uint32_t unused;
+	uint16_t file_type;
 	uint8_t flash_type;
 	uint8_t sig_type;
-	__le32 load_addr;
-	__le32 total_size;
-	__le32 max_size;
-	__le32 hdr_size;
-	__le32 sig_size;
-	__le32 jump_offset;
-	__le32 processed;
+	uint32_t load_addr;
+	uint32_t total_size;
+	uint32_t max_size;
+	uint32_t hdr_size;
+	uint32_t sig_size;
+	uint32_t jump_offset;
+	uint32_t processed;
 };
 
 #define GFH_FILE_INFO_NAME	"FILE_INFO"
@@ -129,16 +129,16 @@ struct gfh_file_info {
 
 struct gfh_bl_info {
 	struct gfh_common_header gfh;
-	__le32 attr;
+	uint32_t attr;
 };
 
 struct gfh_brom_cfg {
 	struct gfh_common_header gfh;
-	__le32 cfg_bits;
-	__le32 usbdl_by_auto_detect_timeout_ms;
+	uint32_t cfg_bits;
+	uint32_t usbdl_by_auto_detect_timeout_ms;
 	uint8_t unused[0x48];
-	__le32 usbdl_by_kcol0_timeout_ms;
-	__le32 usbdl_by_flag_timeout_ms;
+	uint32_t usbdl_by_kcol0_timeout_ms;
+	uint32_t usbdl_by_flag_timeout_ms;
 	uint32_t pad;
 };
 
@@ -157,15 +157,15 @@ struct gfh_anti_clone {
 	uint8_t ac_b2k;
 	uint8_t ac_b2c;
 	uint16_t pad;
-	__le32 ac_offset;
-	__le32 ac_len;
+	uint32_t ac_offset;
+	uint32_t ac_len;
 };
 
 struct gfh_brom_sec_cfg {
 	struct gfh_common_header gfh;
-	__le32 cfg_bits;
+	uint32_t cfg_bits;
 	char customer_name[0x20];
-	__le32 pad;
+	uint32_t pad;
 };
 
 #define BROM_SEC_CFG_JTAG_EN	1
@@ -184,11 +184,11 @@ struct gfh_header {
 
 union lk_hdr {
 	struct {
-		__le32 magic;
-		__le32 size;
+		uint32_t magic;
+		uint32_t size;
 		char name[32];
-		__le32 loadaddr;
-		__le32 mode;
+		uint32_t loadaddr;
+		uint32_t mode;
 	};
 
 	uint8_t data[512];
-- 
2.7.4

[U-Boot] [PATCH v2 3/8] tools: zynqmpbif: Use compiler builtin instead of linux-specific __swab32

[Bin Meng]

__swab32() is a Linux specific macro defined in linux/swab.h. Let's
use the compiler equivalent builtin function __builtin_bswap32() for
better portability.

Signed-off-by: Bin Meng <bmeng.cn@gmail.com>
---

Changes in v2: None

 tools/zynqmpbif.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/zynqmpbif.c b/tools/zynqmpbif.c
index 8c47107..82ce0ac 100644
--- a/tools/zynqmpbif.c
+++ b/tools/zynqmpbif.c
@@ -517,7 +517,7 @@ static int bif_add_bit(struct bif_entry *bf)
 	debug("Bitstream Length: 0x%x\n", bitlen);
 	for (i = 0; i < bitlen; i += sizeof(uint32_t)) {
 		uint32_t *bitbin32 = (uint32_t *)&bitbin[i];
-		*bitbin32 = __swab32(*bitbin32);
+		*bitbin32 = __builtin_bswap32(*bitbin32);
 	}
 
 	if (!bf->dest_dev)
-- 
2.7.4

[U-Boot] [PATCH v2 4/8] linux/types.h: Surround 'struct ustat' with __linux__

[Bin Meng]

'struct ustat' uses linux-specific typedefs to declare its memebers:
__kernel_daddr_t and __kernel_ino_t. It is currently not used by any
U-Boot codes, but when we build U-Boot tools for other platform like
Windows, this becomes a problem.

Let's surround it with __linux__.

Signed-off-by: Bin Meng <bmeng.cn@gmail.com>
---

Changes in v2: None

 include/linux/types.h | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/include/linux/types.h b/include/linux/types.h
index cc6f7cb..51cb284 100644
--- a/include/linux/types.h
+++ b/include/linux/types.h
@@ -151,12 +151,14 @@ typedef __u32 __bitwise __wsum;
 
 typedef unsigned __bitwise__	gfp_t;
 
+#ifdef __linux__
 struct ustat {
 	__kernel_daddr_t	f_tfree;
 	__kernel_ino_t		f_tinode;
 	char			f_fname[6];
 	char			f_fpack[6];
 };
+#endif
 
 #define DECLARE_BITMAP(name, bits) \
 	unsigned long name[BITS_TO_LONGS(bits)]
-- 
2.7.4

[U-Boot] [PATCH v2 5/8] doc: Add documentation for how to build U-Boot host tools

[Bin Meng]

This adds a reST document for how to build U-Boot host tools,
including information for both Linux and Windows.

Signed-off-by: Bin Meng <bmeng.cn@gmail.com>

---

Changes in v2: None

 doc/build/index.rst |  9 +++++++++
 doc/build/tools.rst | 47 +++++++++++++++++++++++++++++++++++++++++++++++
 doc/index.rst       | 11 +++++++++++
 3 files changed, 67 insertions(+)
 create mode 100644 doc/build/index.rst
 create mode 100644 doc/build/tools.rst

diff --git a/doc/build/index.rst b/doc/build/index.rst
new file mode 100644
index 0000000..e4e3411
--- /dev/null
+++ b/doc/build/index.rst
@@ -0,0 +1,9 @@
+.. SPDX-License-Identifier: GPL-2.0+
+
+Build U-Boot
+============
+
+.. toctree::
+   :maxdepth: 2
+
+   tools
diff --git a/doc/build/tools.rst b/doc/build/tools.rst
new file mode 100644
index 0000000..c06f915
--- /dev/null
+++ b/doc/build/tools.rst
@@ -0,0 +1,47 @@
+.. SPDX-License-Identifier: GPL-2.0+
+.. sectionauthor:: Bin Meng <bmeng.cn@gmail.com>
+
+Host tools
+==========
+
+Building tools for Linux
+------------------------
+
+To allow distributions to distribute all possible tools in a generic way,
+avoiding the need of specific tools building for each machine, a tools only
+defconfig file is provided.
+
+Using this, we can build the tools by doing::
+
+   $ make tools-only_defconfig
+   $ make tools-only
+
+Building tools for Windows
+--------------------------
+If you wish to generate Windows versions of the utilities in the tools directory
+you can use MSYS2, a software distro and building platform for Windows.
+
+Download the MSYS2 installer from https://www.msys2.org. Make sure you have
+installed all required packages below in order to build these host tools::
+
+   * gcc (9.1.0)
+   * make (4.2.1)
+   * bison (3.4.2)
+   * diffutils (3.7)
+   * openssl-devel (1.1.1.d)
+
+Note the version numbers in these parentheses above are the package versions
+at the time being when writing this document. The MSYS2 installer tested is
+http://repo.msys2.org/distrib/x86_64/msys2-x86_64-20190524.exe.
+
+There are 3 MSYS subsystems installed: MSYS2, MinGW32 and MinGW64. Each
+subsystem provides an environment to build Windows applications. The MSYS2
+environment is for building POSIX compliant software on Windows using an
+emulation layer. The MinGW32/64 subsystems are for building native Windows
+applications using a linux toolchain (gcc, bash, etc), targeting respectively
+32 and 64 bit Windows.
+
+Launch the MSYS2 shell of the MSYS2 environment, and do the following::
+
+   $ make tools-only_defconfig
+   $ make tools-only NO_SDL=1
diff --git a/doc/index.rst b/doc/index.rst
index 458f0d2..206a045 100644
--- a/doc/index.rst
+++ b/doc/index.rst
@@ -15,6 +15,17 @@ if you want to help out.
 .. toctree::
    :maxdepth: 2
 
+User-oriented documentation
+---------------------------
+
+The following manuals are written for *users* of the U-Boot - those who are
+trying to get it to work optimally on a given system.
+
+.. toctree::
+   :maxdepth: 2
+
+   build/index
+
 Unified Extensible Firmware (UEFI)
 ----------------------------------
 
-- 
2.7.4

[U-Boot] [PATCH v2 6/8] Add .gitattributes for line endings

[Bin Meng]

When building U-Boot host tools for Windows from Microsoft Azure
Pipelines, we see tons of weird warnings and errors emitted from
every Kconfig files:

  Kconfig:6:warning: ignoring unsupported character ''
  Kconfig:6:warning: ignoring unsupported character ''
  Kconfig:8:warning: ignoring unsupported character ''
  Kconfig:9:warning: ignoring unsupported character ''
  Kconfig:10:warning: ignoring unsupported character ''
  Kconfig:10:warning: ignoring unsupported character ''
  Kconfig:13:warning: ignoring unsupported character ''
  arch/Kconfig:1:warning: ignoring unsupported character ''
  arch/Kconfig:2:warning: ignoring unsupported character ''
  arch/Kconfig:2:warning: ignoring unsupported character ''
  arch/Kconfig:4:warning: ignoring unsupported character ''
  ...

After several rounds of experiments, it turns out this is caused
by line endings. Historically, Linux and macOS used linefeed (LF)
characters while Windows used a carriage return plus a linefeed
(CRLF). When Azure Pipelines checks out the U-Boot repo, Git tries
to compensate for the difference by automatically making lines end
in CRLF in the working directory on Windows, which confuses the
Kconfig file parsing logic.

Fortunately Git provides a way for repos to tell Git not to do such
automatical line endings conversion via .gitattributes file below:

* text eol=lf

Signed-off-by: Bin Meng <bmeng.cn@gmail.com>

---

Changes in v2:
- new patch: Add .gitattributes for line endings

 .gitattributes | 2 ++
 1 file changed, 2 insertions(+)
 create mode 100644 .gitattributes

diff --git a/.gitattributes b/.gitattributes
new file mode 100644
index 0000000..8560b79
--- /dev/null
+++ b/.gitattributes
@@ -0,0 +1,2 @@
+# Declare files that always have LF line endings on checkout
+* text eol=lf
-- 
2.7.4

[U-Boot] [PATCH v2 7/8] tools: Avoid creating symbolic links for tools/version.h

[Bin Meng]

When building U-Boot host tools for Windows from Microsoft Azure
Pipelines, the following errors were seen:

  HOSTCC  tools/mkenvimage.o
  In file included from tools/mkenvimage.c:25:
  ./tools/version.h:1:1: error: expected identifier or ‘(’ before ‘.’ token
     1 | ../include/version.h
       | ^
  tools/mkenvimage.c: In function ‘main’:
  tools/mkenvimage.c:117:4: warning: implicit declaration of function ‘usage’ [-Wimplicit-function-declaration]
   117 |    usage(prg);
       |    ^~~~~
  tools/mkenvimage.c:120:35: error: ‘PLAIN_VERSION’ undeclared (first use in this function)
   120 |    printf("%s version %s\n", prg, PLAIN_VERSION);
       |                                   ^~~~~~~~~~~~~
  tools/mkenvimage.c:120:35: note: each undeclared identifier is reported only once for each function it appears in
  make[2]: *** [scripts/Makefile.host:114: tools/mkenvimage.o] Error 1

It turns out tools/version.h is a symbolic link and with Windows
default settings it is unsupported hence the actual content of
tools/version.h is not what file include/version.h has, but the
the linked file path, which breaks the build.

To fix this, remove the symbolic links for tools/version.h. Instead
we perform a copy from include/version.h during the build.

Signed-off-by: Bin Meng <bmeng.cn@gmail.com>

---

Changes in v2:
- new patch: tools: Avoid creating symbolic links for tools/version.h

 Makefile         | 9 ++++++---
 tools/.gitignore | 1 +
 tools/version.h  | 1 -
 3 files changed, 7 insertions(+), 4 deletions(-)
 delete mode 120000 tools/version.h

diff --git a/Makefile b/Makefile
index cbacf1c..e78b317 100644
--- a/Makefile
+++ b/Makefile
@@ -1837,11 +1837,14 @@ checkarmreloc: u-boot
 		false; \
 	fi
 
-envtools: scripts_basic $(version_h) $(timestamp_h)
+tools/version.h: include/version.h
+	$(call if_changed,copy)
+
+envtools: scripts_basic $(version_h) $(timestamp_h) tools/version.h
 	$(Q)$(MAKE) $(build)=tools/env
 
 tools-only: export TOOLS_ONLY=y
-tools-only: scripts_basic $(version_h) $(timestamp_h)
+tools-only: scripts_basic $(version_h) $(timestamp_h) tools/version.h
 	$(Q)$(MAKE) $(build)=tools
 
 tools-all: export HOST_TOOLS_ALL=y
@@ -1869,7 +1872,7 @@ CLEAN_DIRS  += $(MODVERDIR) \
 	       $(foreach d, spl tpl, $(patsubst %,$d/%, \
 			$(filter-out include, $(shell ls -1 $d 2>/dev/null))))
 
-CLEAN_FILES += include/bmp_logo.h include/bmp_logo_data.h \
+CLEAN_FILES += include/bmp_logo.h include/bmp_logo_data.h tools/version.h \
 	       boot* u-boot* MLO* SPL System.map fit-dtb.blob*
 
 # Directories & files removed with 'make mrproper'
diff --git a/tools/.gitignore b/tools/.gitignore
index bd03d32..d0176a7 100644
--- a/tools/.gitignore
+++ b/tools/.gitignore
@@ -31,4 +31,5 @@
 /spl_size_limit
 /sunxi-spl-image-builder
 /ubsha1
+/version.h
 /xway-swap-bytes
diff --git a/tools/version.h b/tools/version.h
deleted file mode 120000
index bb57607..0000000
--- a/tools/version.h
+++ /dev/null
@@ -1 +0,0 @@
-../include/version.h
\ No newline at end of file
-- 
2.7.4

[U-Boot] [PATCH v2 8/8] Add Micirosoft Azure pipelines configuration

[Bin Meng]

Microsoft Azure Pipelines provides unlimited CI/CD minutes and 10
parallel jobs to every open source project for free [1].

This adds a configuration file for Azure Pipelines to utilize the
free Windows VM hosted by Microsoft to ensure no build broken in
building U-Boot host tools for Windows.

[1] https://azure.microsoft.com/en-us/blog/announcing-azure-pipelines-with-unlimited-ci-cd-minutes-for-open-source/

Signed-off-by: Bin Meng <bmeng.cn@gmail.com>

---
See the build result at:
https://dev.azure.com/bmeng/GitHub/_build/results?buildId=53

Changes in v2:
- new patch: Add Micirosoft Azure pipelines configuration

 azure-pipelines.yml | 35 +++++++++++++++++++++++++++++++++++
 1 file changed, 35 insertions(+)
 create mode 100644 azure-pipelines.yml

diff --git a/azure-pipelines.yml b/azure-pipelines.yml
new file mode 100644
index 0000000..cc0514b
--- /dev/null
+++ b/azure-pipelines.yml
@@ -0,0 +1,35 @@
+jobs:
+  - job: tools_only_windows
+    displayName: 'Ensure host tools build for Windows'
+    pool:
+      vmImage: vs2015-win2012r2
+    strategy:
+      matrix:
+        i686:
+          MSYS_DIR: msys32
+          BASE_REPO: msys2-ci-base-i686
+        x86_64:
+          MSYS_DIR: msys64
+          BASE_REPO: msys2-ci-base
+    steps:
+      - script: |
+          git clone https://github.com/msys2/$(BASE_REPO).git %CD:~0,2%\$(MSYS_DIR)
+        displayName: 'Install MSYS2'
+      - script: |
+          set PATH=%CD:~0,2%\$(MSYS_DIR)\usr\bin;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem
+          %CD:~0,2%\$(MSYS_DIR)\usr\bin\pacman --noconfirm -Syyuu
+        displayName: 'Update MSYS2'
+      - script: |
+          set PATH=%CD:~0,2%\$(MSYS_DIR)\usr\bin;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem
+          %CD:~0,2%\$(MSYS_DIR)\usr\bin\pacman --noconfirm --needed -S make gcc bison diffutils openssl-devel
+        displayName: 'Install Toolchain'
+      - script: |
+          set PATH=C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem
+          echo make tools-only_defconfig tools-only NO_SDL=1 > build-tools.sh
+          %CD:~0,2%\$(MSYS_DIR)\usr\bin\bash -lc "bash build-tools.sh"
+        displayName: 'Build Host Tools'
+        env:
+          # Tell MSYS2 we need a POSIX emulation layer
+          MSYSTEM: MSYS
+          # Tell MSYS2 not to ‘cd’ our startup directory to HOME
+          CHERE_INVOKING: yes
-- 
2.7.4

[U-Boot] [PATCH v2 8/8] Add Micirosoft Azure pipelines configuration

[Tom Rini]

On Wed, Oct 23, 2019 at 08:11:52PM -0700, Bin Meng wrote:

> Microsoft Azure Pipelines provides unlimited CI/CD minutes and 10
> parallel jobs to every open source project for free [1].
> 
> This adds a configuration file for Azure Pipelines to utilize the
> free Windows VM hosted by Microsoft to ensure no build broken in
> building U-Boot host tools for Windows.
> 
> [1] https://azure.microsoft.com/en-us/blog/announcing-azure-pipelines-with-unlimited-ci-cd-minutes-for-open-source/
> 
> Signed-off-by: Bin Meng <bmeng.cn@gmail.com>
> 
> ---
> See the build result at:
> https://dev.azure.com/bmeng/GitHub/_build/results?buildId=53
> 
> Changes in v2:
> - new patch: Add Micirosoft Azure pipelines configuration
> 
>  azure-pipelines.yml | 35 +++++++++++++++++++++++++++++++++++
>  1 file changed, 35 insertions(+)
>  create mode 100644 azure-pipelines.yml

Thanks a lot for doing this.  I'm starting to look at what's needed so
that I can also run this automatically and perhaps evaluate it for other
uses in U-Boot as well.  One thing I would like to change is that it
looks like under pipeline settings we can specify the file and I'd like
to call this ".azure-pipelines.yml" instead to match travis/gitlab
files.

> diff --git a/azure-pipelines.yml b/azure-pipelines.yml
> new file mode 100644
> index 0000000..cc0514b
> --- /dev/null
> +++ b/azure-pipelines.yml
> @@ -0,0 +1,35 @@
> +jobs:
> +  - job: tools_only_windows
> +    displayName: 'Ensure host tools build for Windows'
> +    pool:
> +      vmImage: vs2015-win2012r2
> +    strategy:
> +      matrix:
> +        i686:
> +          MSYS_DIR: msys32
> +          BASE_REPO: msys2-ci-base-i686
> +        x86_64:
> +          MSYS_DIR: msys64
> +          BASE_REPO: msys2-ci-base
> +    steps:
> +      - script: |
> +          git clone https://github.com/msys2/$(BASE_REPO).git %CD:~0,2%\$(MSYS_DIR)
> +        displayName: 'Install MSYS2'
> +      - script: |
> +          set PATH=%CD:~0,2%\$(MSYS_DIR)\usr\bin;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem
> +          %CD:~0,2%\$(MSYS_DIR)\usr\bin\pacman --noconfirm -Syyuu
> +        displayName: 'Update MSYS2'
> +      - script: |
> +          set PATH=%CD:~0,2%\$(MSYS_DIR)\usr\bin;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem
> +          %CD:~0,2%\$(MSYS_DIR)\usr\bin\pacman --noconfirm --needed -S make gcc bison diffutils openssl-devel
> +        displayName: 'Install Toolchain'
> +      - script: |
> +          set PATH=C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem
> +          echo make tools-only_defconfig tools-only NO_SDL=1 > build-tools.sh
> +          %CD:~0,2%\$(MSYS_DIR)\usr\bin\bash -lc "bash build-tools.sh"
> +        displayName: 'Build Host Tools'
> +        env:
> +          # Tell MSYS2 we need a POSIX emulation layer
> +          MSYSTEM: MSYS
> +          # Tell MSYS2 not to ‘cd’ our startup directory to HOME
> +          CHERE_INVOKING: yes

While I start understanding overall syntax, is this particular style one
that would allow us to add in more jobs, like say all of the ones we do
in GitLab, and in that staged fashion (do all of the testsuites, then
test.py runs then world build)?  Thanks!

-- 
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20191024/ea4071e4/attachment.sig>

[U-Boot] [PATCH v2 8/8] Add Micirosoft Azure pipelines configuration

[Bin Meng]

Hi Tom,

On Thu, Oct 24, 2019 at 11:01 PM Tom Rini <trini@konsulko.com> wrote:
>
> On Wed, Oct 23, 2019 at 08:11:52PM -0700, Bin Meng wrote:
>
> > Microsoft Azure Pipelines provides unlimited CI/CD minutes and 10
> > parallel jobs to every open source project for free [1].
> >
> > This adds a configuration file for Azure Pipelines to utilize the
> > free Windows VM hosted by Microsoft to ensure no build broken in
> > building U-Boot host tools for Windows.
> >
> > [1] https://azure.microsoft.com/en-us/blog/announcing-azure-pipelines-with-unlimited-ci-cd-minutes-for-open-source/
> >
> > Signed-off-by: Bin Meng <bmeng.cn@gmail.com>
> >
> > ---
> > See the build result at:
> > https://dev.azure.com/bmeng/GitHub/_build/results?buildId=53
> >
> > Changes in v2:
> > - new patch: Add Micirosoft Azure pipelines configuration
> >
> >  azure-pipelines.yml | 35 +++++++++++++++++++++++++++++++++++
> >  1 file changed, 35 insertions(+)
> >  create mode 100644 azure-pipelines.yml
>
> Thanks a lot for doing this.  I'm starting to look at what's needed so
> that I can also run this automatically and perhaps evaluate it for other
> uses in U-Boot as well.  One thing I would like to change is that it
> looks like under pipeline settings we can specify the file and I'd like
> to call this ".azure-pipelines.yml" instead to match travis/gitlab
> files.

I named this as I see examples from other open source projects have
such a name. Let me try ".azure-pipelines.yml".

>
> > diff --git a/azure-pipelines.yml b/azure-pipelines.yml
> > new file mode 100644
> > index 0000000..cc0514b
> > --- /dev/null
> > +++ b/azure-pipelines.yml
> > @@ -0,0 +1,35 @@
> > +jobs:
> > +  - job: tools_only_windows
> > +    displayName: 'Ensure host tools build for Windows'
> > +    pool:
> > +      vmImage: vs2015-win2012r2
> > +    strategy:
> > +      matrix:
> > +        i686:
> > +          MSYS_DIR: msys32
> > +          BASE_REPO: msys2-ci-base-i686
> > +        x86_64:
> > +          MSYS_DIR: msys64
> > +          BASE_REPO: msys2-ci-base
> > +    steps:
> > +      - script: |
> > +          git clone https://github.com/msys2/$(BASE_REPO).git %CD:~0,2%\$(MSYS_DIR)
> > +        displayName: 'Install MSYS2'
> > +      - script: |
> > +          set PATH=%CD:~0,2%\$(MSYS_DIR)\usr\bin;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem
> > +          %CD:~0,2%\$(MSYS_DIR)\usr\bin\pacman --noconfirm -Syyuu
> > +        displayName: 'Update MSYS2'
> > +      - script: |
> > +          set PATH=%CD:~0,2%\$(MSYS_DIR)\usr\bin;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem
> > +          %CD:~0,2%\$(MSYS_DIR)\usr\bin\pacman --noconfirm --needed -S make gcc bison diffutils openssl-devel
> > +        displayName: 'Install Toolchain'
> > +      - script: |
> > +          set PATH=C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem
> > +          echo make tools-only_defconfig tools-only NO_SDL=1 > build-tools.sh
> > +          %CD:~0,2%\$(MSYS_DIR)\usr\bin\bash -lc "bash build-tools.sh"
> > +        displayName: 'Build Host Tools'
> > +        env:
> > +          # Tell MSYS2 we need a POSIX emulation layer
> > +          MSYSTEM: MSYS
> > +          # Tell MSYS2 not to ‘cd’ our startup directory to HOME
> > +          CHERE_INVOKING: yes
>
> While I start understanding overall syntax, is this particular style one
> that would allow us to add in more jobs, like say all of the ones we do
> in GitLab, and in that staged fashion (do all of the testsuites, then
> test.py runs then world build)?  Thanks!

Yes, it supports concepts like pipeline / stage / job. A pipeline is
composed of stages, and only when the first stage finishes can the
second stage start. A stage is composed of jobs.
Please see https://docs.microsoft.com/en-us/azure/devops/pipelines/yaml-schema?view=azure-devops&tabs=schema

Regards,
Bin

[U-Boot] [PATCH v2 8/8] Add Micirosoft Azure pipelines configuration

[Bin Meng]

On Thu, Oct 24, 2019 at 11:10 PM Bin Meng <bmeng.cn@gmail.com> wrote:
>
> Hi Tom,
>
> On Thu, Oct 24, 2019 at 11:01 PM Tom Rini <trini@konsulko.com> wrote:
> >
> > On Wed, Oct 23, 2019 at 08:11:52PM -0700, Bin Meng wrote:
> >

Oops, just noticed a typo in the commit summary. Will correct it in v3.

> > > Microsoft Azure Pipelines provides unlimited CI/CD minutes and 10
> > > parallel jobs to every open source project for free [1].
> > >
> > > This adds a configuration file for Azure Pipelines to utilize the
> > > free Windows VM hosted by Microsoft to ensure no build broken in
> > > building U-Boot host tools for Windows.
> > >
> > > [1] https://azure.microsoft.com/en-us/blog/announcing-azure-pipelines-with-unlimited-ci-cd-minutes-for-open-source/
> > >
> > > Signed-off-by: Bin Meng <bmeng.cn@gmail.com>
> > >
> > > ---
> > > See the build result at:
> > > https://dev.azure.com/bmeng/GitHub/_build/results?buildId=53
> > >
> > > Changes in v2:
> > > - new patch: Add Micirosoft Azure pipelines configuration
> > >
> > >  azure-pipelines.yml | 35 +++++++++++++++++++++++++++++++++++
> > >  1 file changed, 35 insertions(+)
> > >  create mode 100644 azure-pipelines.yml
> >
> > Thanks a lot for doing this.  I'm starting to look at what's needed so
> > that I can also run this automatically and perhaps evaluate it for other
> > uses in U-Boot as well.  One thing I would like to change is that it
> > looks like under pipeline settings we can specify the file and I'd like
> > to call this ".azure-pipelines.yml" instead to match travis/gitlab
> > files.
>
> I named this as I see examples from other open source projects have
> such a name. Let me try ".azure-pipelines.yml".

I tried to rename it to ".azure-pipelines.yml", but the Azure
pipelines does not recognize it.

>
> >
> > > diff --git a/azure-pipelines.yml b/azure-pipelines.yml
> > > new file mode 100644
> > > index 0000000..cc0514b
> > > --- /dev/null
> > > +++ b/azure-pipelines.yml
> > > @@ -0,0 +1,35 @@
> > > +jobs:
> > > +  - job: tools_only_windows
> > > +    displayName: 'Ensure host tools build for Windows'
> > > +    pool:
> > > +      vmImage: vs2015-win2012r2
> > > +    strategy:
> > > +      matrix:
> > > +        i686:
> > > +          MSYS_DIR: msys32
> > > +          BASE_REPO: msys2-ci-base-i686
> > > +        x86_64:
> > > +          MSYS_DIR: msys64
> > > +          BASE_REPO: msys2-ci-base
> > > +    steps:
> > > +      - script: |
> > > +          git clone https://github.com/msys2/$(BASE_REPO).git %CD:~0,2%\$(MSYS_DIR)
> > > +        displayName: 'Install MSYS2'
> > > +      - script: |
> > > +          set PATH=%CD:~0,2%\$(MSYS_DIR)\usr\bin;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem
> > > +          %CD:~0,2%\$(MSYS_DIR)\usr\bin\pacman --noconfirm -Syyuu
> > > +        displayName: 'Update MSYS2'
> > > +      - script: |
> > > +          set PATH=%CD:~0,2%\$(MSYS_DIR)\usr\bin;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem
> > > +          %CD:~0,2%\$(MSYS_DIR)\usr\bin\pacman --noconfirm --needed -S make gcc bison diffutils openssl-devel
> > > +        displayName: 'Install Toolchain'
> > > +      - script: |
> > > +          set PATH=C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem
> > > +          echo make tools-only_defconfig tools-only NO_SDL=1 > build-tools.sh
> > > +          %CD:~0,2%\$(MSYS_DIR)\usr\bin\bash -lc "bash build-tools.sh"
> > > +        displayName: 'Build Host Tools'
> > > +        env:
> > > +          # Tell MSYS2 we need a POSIX emulation layer
> > > +          MSYSTEM: MSYS
> > > +          # Tell MSYS2 not to ‘cd’ our startup directory to HOME
> > > +          CHERE_INVOKING: yes
> >
> > While I start understanding overall syntax, is this particular style one
> > that would allow us to add in more jobs, like say all of the ones we do
> > in GitLab, and in that staged fashion (do all of the testsuites, then
> > test.py runs then world build)?  Thanks!
>
> Yes, it supports concepts like pipeline / stage / job. A pipeline is
> composed of stages, and only when the first stage finishes can the
> second stage start. A stage is composed of jobs.

Jobs can be paralleled executed. Up to 10 jobs for free.

> Please see https://docs.microsoft.com/en-us/azure/devops/pipelines/yaml-schema?view=azure-devops&tabs=schema

Regards,
Bin

[U-Boot] [PATCH v2 8/8] Add Micirosoft Azure pipelines configuration

[Tom Rini]

On Thu, Oct 24, 2019 at 11:31:30PM +0800, Bin Meng wrote:
> On Thu, Oct 24, 2019 at 11:10 PM Bin Meng <bmeng.cn@gmail.com> wrote:
> >
> > Hi Tom,
> >
> > On Thu, Oct 24, 2019 at 11:01 PM Tom Rini <trini@konsulko.com> wrote:
> > >
> > > On Wed, Oct 23, 2019 at 08:11:52PM -0700, Bin Meng wrote:
> > >
> 
> Oops, just noticed a typo in the commit summary. Will correct it in v3.
> 
> > > > Microsoft Azure Pipelines provides unlimited CI/CD minutes and 10
> > > > parallel jobs to every open source project for free [1].
> > > >
> > > > This adds a configuration file for Azure Pipelines to utilize the
> > > > free Windows VM hosted by Microsoft to ensure no build broken in
> > > > building U-Boot host tools for Windows.
> > > >
> > > > [1] https://azure.microsoft.com/en-us/blog/announcing-azure-pipelines-with-unlimited-ci-cd-minutes-for-open-source/
> > > >
> > > > Signed-off-by: Bin Meng <bmeng.cn@gmail.com>
> > > >
> > > > ---
> > > > See the build result at:
> > > > https://dev.azure.com/bmeng/GitHub/_build/results?buildId=53
> > > >
> > > > Changes in v2:
> > > > - new patch: Add Micirosoft Azure pipelines configuration
> > > >
> > > >  azure-pipelines.yml | 35 +++++++++++++++++++++++++++++++++++
> > > >  1 file changed, 35 insertions(+)
> > > >  create mode 100644 azure-pipelines.yml
> > >
> > > Thanks a lot for doing this.  I'm starting to look at what's needed so
> > > that I can also run this automatically and perhaps evaluate it for other
> > > uses in U-Boot as well.  One thing I would like to change is that it
> > > looks like under pipeline settings we can specify the file and I'd like
> > > to call this ".azure-pipelines.yml" instead to match travis/gitlab
> > > files.
> >
> > I named this as I see examples from other open source projects have
> > such a name. Let me try ".azure-pipelines.yml".
> 
> I tried to rename it to ".azure-pipelines.yml", but the Azure
> pipelines does not recognize it.

Did you find the place in the pipeline settings page to change where it
looks?
https://stackoverflow.com/questions/55614307/can-you-change-the-location-of-azure-pipelines-yaml-in-azure-devops
is what I found and why I asked.

> > > > diff --git a/azure-pipelines.yml b/azure-pipelines.yml
> > > > new file mode 100644
> > > > index 0000000..cc0514b
> > > > --- /dev/null
> > > > +++ b/azure-pipelines.yml
> > > > @@ -0,0 +1,35 @@
> > > > +jobs:
> > > > +  - job: tools_only_windows
> > > > +    displayName: 'Ensure host tools build for Windows'
> > > > +    pool:
> > > > +      vmImage: vs2015-win2012r2
> > > > +    strategy:
> > > > +      matrix:
> > > > +        i686:
> > > > +          MSYS_DIR: msys32
> > > > +          BASE_REPO: msys2-ci-base-i686
> > > > +        x86_64:
> > > > +          MSYS_DIR: msys64
> > > > +          BASE_REPO: msys2-ci-base
> > > > +    steps:
> > > > +      - script: |
> > > > +          git clone https://github.com/msys2/$(BASE_REPO).git %CD:~0,2%\$(MSYS_DIR)
> > > > +        displayName: 'Install MSYS2'
> > > > +      - script: |
> > > > +          set PATH=%CD:~0,2%\$(MSYS_DIR)\usr\bin;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem
> > > > +          %CD:~0,2%\$(MSYS_DIR)\usr\bin\pacman --noconfirm -Syyuu
> > > > +        displayName: 'Update MSYS2'
> > > > +      - script: |
> > > > +          set PATH=%CD:~0,2%\$(MSYS_DIR)\usr\bin;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem
> > > > +          %CD:~0,2%\$(MSYS_DIR)\usr\bin\pacman --noconfirm --needed -S make gcc bison diffutils openssl-devel
> > > > +        displayName: 'Install Toolchain'
> > > > +      - script: |
> > > > +          set PATH=C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem
> > > > +          echo make tools-only_defconfig tools-only NO_SDL=1 > build-tools.sh
> > > > +          %CD:~0,2%\$(MSYS_DIR)\usr\bin\bash -lc "bash build-tools.sh"
> > > > +        displayName: 'Build Host Tools'
> > > > +        env:
> > > > +          # Tell MSYS2 we need a POSIX emulation layer
> > > > +          MSYSTEM: MSYS
> > > > +          # Tell MSYS2 not to ‘cd’ our startup directory to HOME
> > > > +          CHERE_INVOKING: yes
> > >
> > > While I start understanding overall syntax, is this particular style one
> > > that would allow us to add in more jobs, like say all of the ones we do
> > > in GitLab, and in that staged fashion (do all of the testsuites, then
> > > test.py runs then world build)?  Thanks!
> >
> > Yes, it supports concepts like pipeline / stage / job. A pipeline is
> > composed of stages, and only when the first stage finishes can the
> > second stage start. A stage is composed of jobs.
> 
> Jobs can be paralleled executed. Up to 10 jobs for free.
> 
> > Please see https://docs.microsoft.com/en-us/azure/devops/pipelines/yaml-schema?view=azure-devops&tabs=schema

Right, OK.  I'll pencil in some time to find out if that means we can
really only have 10 job stanzas and if so, figure out what if anything
makes sense to have as an attempt to have another free CI builder for
everything and not just this use-case.

-- 
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20191024/3b69f3e3/attachment.sig>

[U-Boot] [PATCH v2 8/8] Add Micirosoft Azure pipelines configuration

[Bin Meng]

Hi Tom,

On Thu, Oct 24, 2019 at 11:38 PM Tom Rini <trini@konsulko.com> wrote:
>
> On Thu, Oct 24, 2019 at 11:31:30PM +0800, Bin Meng wrote:
> > On Thu, Oct 24, 2019 at 11:10 PM Bin Meng <bmeng.cn@gmail.com> wrote:
> > >
> > > Hi Tom,
> > >
> > > On Thu, Oct 24, 2019 at 11:01 PM Tom Rini <trini@konsulko.com> wrote:
> > > >
> > > > On Wed, Oct 23, 2019 at 08:11:52PM -0700, Bin Meng wrote:
> > > >
> >
> > Oops, just noticed a typo in the commit summary. Will correct it in v3.
> >
> > > > > Microsoft Azure Pipelines provides unlimited CI/CD minutes and 10
> > > > > parallel jobs to every open source project for free [1].
> > > > >
> > > > > This adds a configuration file for Azure Pipelines to utilize the
> > > > > free Windows VM hosted by Microsoft to ensure no build broken in
> > > > > building U-Boot host tools for Windows.
> > > > >
> > > > > [1] https://azure.microsoft.com/en-us/blog/announcing-azure-pipelines-with-unlimited-ci-cd-minutes-for-open-source/
> > > > >
> > > > > Signed-off-by: Bin Meng <bmeng.cn@gmail.com>
> > > > >
> > > > > ---
> > > > > See the build result at:
> > > > > https://dev.azure.com/bmeng/GitHub/_build/results?buildId=53
> > > > >
> > > > > Changes in v2:
> > > > > - new patch: Add Micirosoft Azure pipelines configuration
> > > > >
> > > > >  azure-pipelines.yml | 35 +++++++++++++++++++++++++++++++++++
> > > > >  1 file changed, 35 insertions(+)
> > > > >  create mode 100644 azure-pipelines.yml
> > > >
> > > > Thanks a lot for doing this.  I'm starting to look at what's needed so
> > > > that I can also run this automatically and perhaps evaluate it for other
> > > > uses in U-Boot as well.  One thing I would like to change is that it
> > > > looks like under pipeline settings we can specify the file and I'd like
> > > > to call this ".azure-pipelines.yml" instead to match travis/gitlab
> > > > files.
> > >
> > > I named this as I see examples from other open source projects have
> > > such a name. Let me try ".azure-pipelines.yml".
> >
> > I tried to rename it to ".azure-pipelines.yml", but the Azure
> > pipelines does not recognize it.
>
> Did you find the place in the pipeline settings page to change where it
> looks?
> https://stackoverflow.com/questions/55614307/can-you-change-the-location-of-azure-pipelines-yaml-in-azure-devops
> is what I found and why I asked.

Thanks. I followed this link and now it is able to accept
".azure-pipelines.yml".

>
> > > > > diff --git a/azure-pipelines.yml b/azure-pipelines.yml
> > > > > new file mode 100644
> > > > > index 0000000..cc0514b
> > > > > --- /dev/null
> > > > > +++ b/azure-pipelines.yml
> > > > > @@ -0,0 +1,35 @@
> > > > > +jobs:
> > > > > +  - job: tools_only_windows
> > > > > +    displayName: 'Ensure host tools build for Windows'
> > > > > +    pool:
> > > > > +      vmImage: vs2015-win2012r2
> > > > > +    strategy:
> > > > > +      matrix:
> > > > > +        i686:
> > > > > +          MSYS_DIR: msys32
> > > > > +          BASE_REPO: msys2-ci-base-i686
> > > > > +        x86_64:
> > > > > +          MSYS_DIR: msys64
> > > > > +          BASE_REPO: msys2-ci-base
> > > > > +    steps:
> > > > > +      - script: |
> > > > > +          git clone https://github.com/msys2/$(BASE_REPO).git %CD:~0,2%\$(MSYS_DIR)
> > > > > +        displayName: 'Install MSYS2'
> > > > > +      - script: |
> > > > > +          set PATH=%CD:~0,2%\$(MSYS_DIR)\usr\bin;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem
> > > > > +          %CD:~0,2%\$(MSYS_DIR)\usr\bin\pacman --noconfirm -Syyuu
> > > > > +        displayName: 'Update MSYS2'
> > > > > +      - script: |
> > > > > +          set PATH=%CD:~0,2%\$(MSYS_DIR)\usr\bin;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem
> > > > > +          %CD:~0,2%\$(MSYS_DIR)\usr\bin\pacman --noconfirm --needed -S make gcc bison diffutils openssl-devel
> > > > > +        displayName: 'Install Toolchain'
> > > > > +      - script: |
> > > > > +          set PATH=C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem
> > > > > +          echo make tools-only_defconfig tools-only NO_SDL=1 > build-tools.sh
> > > > > +          %CD:~0,2%\$(MSYS_DIR)\usr\bin\bash -lc "bash build-tools.sh"
> > > > > +        displayName: 'Build Host Tools'
> > > > > +        env:
> > > > > +          # Tell MSYS2 we need a POSIX emulation layer
> > > > > +          MSYSTEM: MSYS
> > > > > +          # Tell MSYS2 not to ‘cd’ our startup directory to HOME
> > > > > +          CHERE_INVOKING: yes
> > > >
> > > > While I start understanding overall syntax, is this particular style one
> > > > that would allow us to add in more jobs, like say all of the ones we do
> > > > in GitLab, and in that staged fashion (do all of the testsuites, then
> > > > test.py runs then world build)?  Thanks!
> > >
> > > Yes, it supports concepts like pipeline / stage / job. A pipeline is
> > > composed of stages, and only when the first stage finishes can the
> > > second stage start. A stage is composed of jobs.
> >
> > Jobs can be paralleled executed. Up to 10 jobs for free.
> >
> > > Please see https://docs.microsoft.com/en-us/azure/devops/pipelines/yaml-schema?view=azure-devops&tabs=schema
>
> Right, OK.  I'll pencil in some time to find out if that means we can
> really only have 10 job stanzas and if so, figure out what if anything
> makes sense to have as an attempt to have another free CI builder for
> everything and not just this use-case.

Regards,
Bin

[U-Boot] [PATCH v2 8/8] Add Micirosoft Azure pipelines configuration

[Bin Meng]

Hi Tom,

On Fri, Oct 25, 2019 at 1:52 PM Bin Meng <bmeng.cn@gmail.com> wrote:
>
> Hi Tom,
>
> On Thu, Oct 24, 2019 at 11:38 PM Tom Rini <trini@konsulko.com> wrote:
> >
> > On Thu, Oct 24, 2019 at 11:31:30PM +0800, Bin Meng wrote:
> > > On Thu, Oct 24, 2019 at 11:10 PM Bin Meng <bmeng.cn@gmail.com> wrote:
> > > >
> > > > Hi Tom,
> > > >
> > > > On Thu, Oct 24, 2019 at 11:01 PM Tom Rini <trini@konsulko.com> wrote:
> > > > >
> > > > > On Wed, Oct 23, 2019 at 08:11:52PM -0700, Bin Meng wrote:
> > > > >
> > >
> > > Oops, just noticed a typo in the commit summary. Will correct it in v3.
> > >
> > > > > > Microsoft Azure Pipelines provides unlimited CI/CD minutes and 10
> > > > > > parallel jobs to every open source project for free [1].
> > > > > >
> > > > > > This adds a configuration file for Azure Pipelines to utilize the
> > > > > > free Windows VM hosted by Microsoft to ensure no build broken in
> > > > > > building U-Boot host tools for Windows.
> > > > > >
> > > > > > [1] https://azure.microsoft.com/en-us/blog/announcing-azure-pipelines-with-unlimited-ci-cd-minutes-for-open-source/
> > > > > >
> > > > > > Signed-off-by: Bin Meng <bmeng.cn@gmail.com>
> > > > > >
> > > > > > ---
> > > > > > See the build result at:
> > > > > > https://dev.azure.com/bmeng/GitHub/_build/results?buildId=53
> > > > > >
> > > > > > Changes in v2:
> > > > > > - new patch: Add Micirosoft Azure pipelines configuration
> > > > > >
> > > > > >  azure-pipelines.yml | 35 +++++++++++++++++++++++++++++++++++
> > > > > >  1 file changed, 35 insertions(+)
> > > > > >  create mode 100644 azure-pipelines.yml
> > > > >
> > > > > Thanks a lot for doing this.  I'm starting to look at what's needed so
> > > > > that I can also run this automatically and perhaps evaluate it for other
> > > > > uses in U-Boot as well.  One thing I would like to change is that it
> > > > > looks like under pipeline settings we can specify the file and I'd like
> > > > > to call this ".azure-pipelines.yml" instead to match travis/gitlab
> > > > > files.
> > > >
> > > > I named this as I see examples from other open source projects have
> > > > such a name. Let me try ".azure-pipelines.yml".
> > >
> > > I tried to rename it to ".azure-pipelines.yml", but the Azure
> > > pipelines does not recognize it.
> >
> > Did you find the place in the pipeline settings page to change where it
> > looks?
> > https://stackoverflow.com/questions/55614307/can-you-change-the-location-of-azure-pipelines-yaml-in-azure-devops
> > is what I found and why I asked.
>
> Thanks. I followed this link and now it is able to accept
> ".azure-pipelines.yml".
>
> >
> > > > > > diff --git a/azure-pipelines.yml b/azure-pipelines.yml
> > > > > > new file mode 100644
> > > > > > index 0000000..cc0514b
> > > > > > --- /dev/null
> > > > > > +++ b/azure-pipelines.yml
> > > > > > @@ -0,0 +1,35 @@
> > > > > > +jobs:
> > > > > > +  - job: tools_only_windows
> > > > > > +    displayName: 'Ensure host tools build for Windows'
> > > > > > +    pool:
> > > > > > +      vmImage: vs2015-win2012r2
> > > > > > +    strategy:
> > > > > > +      matrix:
> > > > > > +        i686:
> > > > > > +          MSYS_DIR: msys32
> > > > > > +          BASE_REPO: msys2-ci-base-i686
> > > > > > +        x86_64:
> > > > > > +          MSYS_DIR: msys64
> > > > > > +          BASE_REPO: msys2-ci-base
> > > > > > +    steps:
> > > > > > +      - script: |
> > > > > > +          git clone https://github.com/msys2/$(BASE_REPO).git %CD:~0,2%\$(MSYS_DIR)
> > > > > > +        displayName: 'Install MSYS2'
> > > > > > +      - script: |
> > > > > > +          set PATH=%CD:~0,2%\$(MSYS_DIR)\usr\bin;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem
> > > > > > +          %CD:~0,2%\$(MSYS_DIR)\usr\bin\pacman --noconfirm -Syyuu
> > > > > > +        displayName: 'Update MSYS2'
> > > > > > +      - script: |
> > > > > > +          set PATH=%CD:~0,2%\$(MSYS_DIR)\usr\bin;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem
> > > > > > +          %CD:~0,2%\$(MSYS_DIR)\usr\bin\pacman --noconfirm --needed -S make gcc bison diffutils openssl-devel
> > > > > > +        displayName: 'Install Toolchain'
> > > > > > +      - script: |
> > > > > > +          set PATH=C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem
> > > > > > +          echo make tools-only_defconfig tools-only NO_SDL=1 > build-tools.sh
> > > > > > +          %CD:~0,2%\$(MSYS_DIR)\usr\bin\bash -lc "bash build-tools.sh"
> > > > > > +        displayName: 'Build Host Tools'
> > > > > > +        env:
> > > > > > +          # Tell MSYS2 we need a POSIX emulation layer
> > > > > > +          MSYSTEM: MSYS
> > > > > > +          # Tell MSYS2 not to ‘cd’ our startup directory to HOME
> > > > > > +          CHERE_INVOKING: yes
> > > > >
> > > > > While I start understanding overall syntax, is this particular style one
> > > > > that would allow us to add in more jobs, like say all of the ones we do
> > > > > in GitLab, and in that staged fashion (do all of the testsuites, then
> > > > > test.py runs then world build)?  Thanks!
> > > >
> > > > Yes, it supports concepts like pipeline / stage / job. A pipeline is
> > > > composed of stages, and only when the first stage finishes can the
> > > > second stage start. A stage is composed of jobs.
> > >
> > > Jobs can be paralleled executed. Up to 10 jobs for free.
> > >
> > > > Please see https://docs.microsoft.com/en-us/azure/devops/pipelines/yaml-schema?view=azure-devops&tabs=schema
> >
> > Right, OK.  I'll pencil in some time to find out if that means we can
> > really only have 10 job stanzas and if so, figure out what if anything
> > makes sense to have as an attempt to have another free CI builder for
> > everything and not just this use-case.

I enabled all CI testing in the Azure Pipeline. It took about 2 hour
and 15 minutes for a complete run. I think it can be used as a backup.

See https://dev.azure.com/bmeng/GitHub/_build/results?buildId=104

There is still one build error I need fix. Please have a look.

Regards,
Bin

[U-Boot] [PATCH v2 8/8] Add Micirosoft Azure pipelines configuration

[Tom Rini]

On Sat, Oct 26, 2019 at 08:12:22PM +0800, Bin Meng wrote:
> Hi Tom,
> 
> On Fri, Oct 25, 2019 at 1:52 PM Bin Meng <bmeng.cn@gmail.com> wrote:
> >
> > Hi Tom,
> >
> > On Thu, Oct 24, 2019 at 11:38 PM Tom Rini <trini@konsulko.com> wrote:
> > >
> > > On Thu, Oct 24, 2019 at 11:31:30PM +0800, Bin Meng wrote:
> > > > On Thu, Oct 24, 2019 at 11:10 PM Bin Meng <bmeng.cn@gmail.com> wrote:
> > > > >
> > > > > Hi Tom,
> > > > >
> > > > > On Thu, Oct 24, 2019 at 11:01 PM Tom Rini <trini@konsulko.com> wrote:
> > > > > >
> > > > > > On Wed, Oct 23, 2019 at 08:11:52PM -0700, Bin Meng wrote:
> > > > > >
> > > >
> > > > Oops, just noticed a typo in the commit summary. Will correct it in v3.
> > > >
> > > > > > > Microsoft Azure Pipelines provides unlimited CI/CD minutes and 10
> > > > > > > parallel jobs to every open source project for free [1].
> > > > > > >
> > > > > > > This adds a configuration file for Azure Pipelines to utilize the
> > > > > > > free Windows VM hosted by Microsoft to ensure no build broken in
> > > > > > > building U-Boot host tools for Windows.
> > > > > > >
> > > > > > > [1] https://azure.microsoft.com/en-us/blog/announcing-azure-pipelines-with-unlimited-ci-cd-minutes-for-open-source/
> > > > > > >
> > > > > > > Signed-off-by: Bin Meng <bmeng.cn@gmail.com>
> > > > > > >
> > > > > > > ---
> > > > > > > See the build result at:
> > > > > > > https://dev.azure.com/bmeng/GitHub/_build/results?buildId=53
> > > > > > >
> > > > > > > Changes in v2:
> > > > > > > - new patch: Add Micirosoft Azure pipelines configuration
> > > > > > >
> > > > > > >  azure-pipelines.yml | 35 +++++++++++++++++++++++++++++++++++
> > > > > > >  1 file changed, 35 insertions(+)
> > > > > > >  create mode 100644 azure-pipelines.yml
> > > > > >
> > > > > > Thanks a lot for doing this.  I'm starting to look at what's needed so
> > > > > > that I can also run this automatically and perhaps evaluate it for other
> > > > > > uses in U-Boot as well.  One thing I would like to change is that it
> > > > > > looks like under pipeline settings we can specify the file and I'd like
> > > > > > to call this ".azure-pipelines.yml" instead to match travis/gitlab
> > > > > > files.
> > > > >
> > > > > I named this as I see examples from other open source projects have
> > > > > such a name. Let me try ".azure-pipelines.yml".
> > > >
> > > > I tried to rename it to ".azure-pipelines.yml", but the Azure
> > > > pipelines does not recognize it.
> > >
> > > Did you find the place in the pipeline settings page to change where it
> > > looks?
> > > https://stackoverflow.com/questions/55614307/can-you-change-the-location-of-azure-pipelines-yaml-in-azure-devops
> > > is what I found and why I asked.
> >
> > Thanks. I followed this link and now it is able to accept
> > ".azure-pipelines.yml".
> >
> > >
> > > > > > > diff --git a/azure-pipelines.yml b/azure-pipelines.yml
> > > > > > > new file mode 100644
> > > > > > > index 0000000..cc0514b
> > > > > > > --- /dev/null
> > > > > > > +++ b/azure-pipelines.yml
> > > > > > > @@ -0,0 +1,35 @@
> > > > > > > +jobs:
> > > > > > > +  - job: tools_only_windows
> > > > > > > +    displayName: 'Ensure host tools build for Windows'
> > > > > > > +    pool:
> > > > > > > +      vmImage: vs2015-win2012r2
> > > > > > > +    strategy:
> > > > > > > +      matrix:
> > > > > > > +        i686:
> > > > > > > +          MSYS_DIR: msys32
> > > > > > > +          BASE_REPO: msys2-ci-base-i686
> > > > > > > +        x86_64:
> > > > > > > +          MSYS_DIR: msys64
> > > > > > > +          BASE_REPO: msys2-ci-base
> > > > > > > +    steps:
> > > > > > > +      - script: |
> > > > > > > +          git clone https://github.com/msys2/$(BASE_REPO).git %CD:~0,2%\$(MSYS_DIR)
> > > > > > > +        displayName: 'Install MSYS2'
> > > > > > > +      - script: |
> > > > > > > +          set PATH=%CD:~0,2%\$(MSYS_DIR)\usr\bin;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem
> > > > > > > +          %CD:~0,2%\$(MSYS_DIR)\usr\bin\pacman --noconfirm -Syyuu
> > > > > > > +        displayName: 'Update MSYS2'
> > > > > > > +      - script: |
> > > > > > > +          set PATH=%CD:~0,2%\$(MSYS_DIR)\usr\bin;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem
> > > > > > > +          %CD:~0,2%\$(MSYS_DIR)\usr\bin\pacman --noconfirm --needed -S make gcc bison diffutils openssl-devel
> > > > > > > +        displayName: 'Install Toolchain'
> > > > > > > +      - script: |
> > > > > > > +          set PATH=C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem
> > > > > > > +          echo make tools-only_defconfig tools-only NO_SDL=1 > build-tools.sh
> > > > > > > +          %CD:~0,2%\$(MSYS_DIR)\usr\bin\bash -lc "bash build-tools.sh"
> > > > > > > +        displayName: 'Build Host Tools'
> > > > > > > +        env:
> > > > > > > +          # Tell MSYS2 we need a POSIX emulation layer
> > > > > > > +          MSYSTEM: MSYS
> > > > > > > +          # Tell MSYS2 not to ‘cd’ our startup directory to HOME
> > > > > > > +          CHERE_INVOKING: yes
> > > > > >
> > > > > > While I start understanding overall syntax, is this particular style one
> > > > > > that would allow us to add in more jobs, like say all of the ones we do
> > > > > > in GitLab, and in that staged fashion (do all of the testsuites, then
> > > > > > test.py runs then world build)?  Thanks!
> > > > >
> > > > > Yes, it supports concepts like pipeline / stage / job. A pipeline is
> > > > > composed of stages, and only when the first stage finishes can the
> > > > > second stage start. A stage is composed of jobs.
> > > >
> > > > Jobs can be paralleled executed. Up to 10 jobs for free.
> > > >
> > > > > Please see https://docs.microsoft.com/en-us/azure/devops/pipelines/yaml-schema?view=azure-devops&tabs=schema
> > >
> > > Right, OK.  I'll pencil in some time to find out if that means we can
> > > really only have 10 job stanzas and if so, figure out what if anything
> > > makes sense to have as an attempt to have another free CI builder for
> > > everything and not just this use-case.
> 
> I enabled all CI testing in the Azure Pipeline. It took about 2 hour
> and 15 minutes for a complete run. I think it can be used as a backup.
> 
> See https://dev.azure.com/bmeng/GitHub/_build/results?buildId=104
> 
> There is still one build error I need fix. Please have a look.

Can you please post the full pipeline yaml file?  Thanks!

-- 
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20191026/4fe1cc04/attachment.sig>