* [Buildroot] [PATCHv3 2/6] configs/rock_pi_4_defconfig: remove BR2_TARGET_UBOOT_NEEDS_PYTHON
2020-07-27 15:51 [Buildroot] [PATCHv3 1/6] configs/imx6ullevk_defconfig: remove typo Romain Naour
@ 2020-07-27 15:51 ` Romain Naour
2020-07-27 15:51 ` [Buildroot] [PATCHv3 3/6] configs/rock_pi_n10_defconfig: " Romain Naour
` (4 subsequent siblings)
5 siblings, 0 replies; 8+ messages in thread
From: Romain Naour @ 2020-07-27 15:51 UTC (permalink / raw)
To: buildroot
BR2_TARGET_UBOOT_NEEDS_PYTHON3 is already selected for
this defconfig, BR2_TARGET_UBOOT_NEEDS_PYTHON and
BR2_TARGET_UBOOT_NEEDS_PYTHON3 can't be selected at the
same time.
Reported by check-dotconfig.py.
Fixes:
https://gitlab.com/kubu93/buildroot/-/jobs/657885313
Signed-off-by: Romain Naour <romain.naour@gmail.com>
---
configs/rock_pi_4_defconfig | 1 -
1 file changed, 1 deletion(-)
diff --git a/configs/rock_pi_4_defconfig b/configs/rock_pi_4_defconfig
index 22ef74be82..401a59bac4 100644
--- a/configs/rock_pi_4_defconfig
+++ b/configs/rock_pi_4_defconfig
@@ -23,7 +23,6 @@ BR2_TARGET_UBOOT_CUSTOM_REPO_URL="https://github.com/amarula/u-boot-amarula.git"
BR2_TARGET_UBOOT_CUSTOM_REPO_VERSION="cf2747360fdd1eb0d51dd01a984c7c62c7f714fe"
BR2_TARGET_UBOOT_BOARD_DEFCONFIG="rock-pi-4-rk3399"
BR2_TARGET_UBOOT_NEEDS_DTC=y
-BR2_TARGET_UBOOT_NEEDS_PYTHON=y
BR2_TARGET_UBOOT_NEEDS_PYTHON3=y
BR2_TARGET_UBOOT_NEEDS_ATF_BL31=y
BR2_TARGET_UBOOT_NEEDS_ATF_BL31_ELF=y
--
2.25.4
^ permalink raw reply related [flat|nested] 8+ messages in thread
* [Buildroot] [PATCHv3 3/6] configs/rock_pi_n10_defconfig: remove BR2_TARGET_UBOOT_NEEDS_PYTHON
2020-07-27 15:51 [Buildroot] [PATCHv3 1/6] configs/imx6ullevk_defconfig: remove typo Romain Naour
2020-07-27 15:51 ` [Buildroot] [PATCHv3 2/6] configs/rock_pi_4_defconfig: remove BR2_TARGET_UBOOT_NEEDS_PYTHON Romain Naour
@ 2020-07-27 15:51 ` Romain Naour
2020-07-27 15:51 ` [Buildroot] [PATCHv3 4/6] support/scripts: add check-dotconfig.py Romain Naour
` (3 subsequent siblings)
5 siblings, 0 replies; 8+ messages in thread
From: Romain Naour @ 2020-07-27 15:51 UTC (permalink / raw)
To: buildroot
BR2_TARGET_UBOOT_NEEDS_PYTHON3 is already selected for
this defconfig, BR2_TARGET_UBOOT_NEEDS_PYTHON and
BR2_TARGET_UBOOT_NEEDS_PYTHON3 can't be selected at the
same time.
Reported by check-dotconfig.py.
Fixes:
https://gitlab.com/kubu93/buildroot/-/jobs/657885314
Signed-off-by: Romain Naour <romain.naour@gmail.com>
---
configs/rock_pi_n10_defconfig | 1 -
1 file changed, 1 deletion(-)
diff --git a/configs/rock_pi_n10_defconfig b/configs/rock_pi_n10_defconfig
index bfaa489a20..c622839c89 100644
--- a/configs/rock_pi_n10_defconfig
+++ b/configs/rock_pi_n10_defconfig
@@ -23,7 +23,6 @@ BR2_TARGET_UBOOT_CUSTOM_REPO_URL="https://github.com/amarula/u-boot-amarula.git"
BR2_TARGET_UBOOT_CUSTOM_REPO_VERSION="cf2747360fdd1eb0d51dd01a984c7c62c7f714fe"
BR2_TARGET_UBOOT_BOARD_DEFCONFIG="rock-pi-n10-rk3399pro"
BR2_TARGET_UBOOT_NEEDS_DTC=y
-BR2_TARGET_UBOOT_NEEDS_PYTHON=y
BR2_TARGET_UBOOT_NEEDS_PYTHON3=y
BR2_TARGET_UBOOT_NEEDS_ATF_BL31=y
BR2_TARGET_UBOOT_NEEDS_ATF_BL31_ELF=y
--
2.25.4
^ permalink raw reply related [flat|nested] 8+ messages in thread
* [Buildroot] [PATCHv3 4/6] support/scripts: add check-dotconfig.py
2020-07-27 15:51 [Buildroot] [PATCHv3 1/6] configs/imx6ullevk_defconfig: remove typo Romain Naour
2020-07-27 15:51 ` [Buildroot] [PATCHv3 2/6] configs/rock_pi_4_defconfig: remove BR2_TARGET_UBOOT_NEEDS_PYTHON Romain Naour
2020-07-27 15:51 ` [Buildroot] [PATCHv3 3/6] configs/rock_pi_n10_defconfig: " Romain Naour
@ 2020-07-27 15:51 ` Romain Naour
2020-07-27 16:18 ` Yann E. MORIN
2020-07-27 15:51 ` [Buildroot] [PATCHv3 5/6] gitlab-ci: check generated config files Romain Naour
` (2 subsequent siblings)
5 siblings, 1 reply; 8+ messages in thread
From: Romain Naour @ 2020-07-27 15:51 UTC (permalink / raw)
To: buildroot
For the same reason as for 50b747f212be2c9c0f7cf10c674ed488d042715c,
we need to check if the generated configuration file (.config)
contains all symbols present in the defconfig file.
If not there is an issue with the defconfig.
This script will be used in .gitlab-ci.yml.
Inspired by is_toolchain_usable() function from genrandconfig:
https://git.busybox.net/buildroot/tree/utils/genrandconfig?h=2020.02#n164
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Yann E. MORIN <yann.morin.1998@free.fr>
---
v2: update with the improved patch suggested by Yann.
http://lists.busybox.net/pipermail/buildroot/2020-April/280679.html
---
support/scripts/check-dotconfig.py | 44 ++++++++++++++++++++++++++++++
1 file changed, 44 insertions(+)
create mode 100755 support/scripts/check-dotconfig.py
diff --git a/support/scripts/check-dotconfig.py b/support/scripts/check-dotconfig.py
new file mode 100755
index 0000000000..81c83a8481
--- /dev/null
+++ b/support/scripts/check-dotconfig.py
@@ -0,0 +1,44 @@
+#!/usr/bin/env python3
+
+# This scripts check that all lines present in the defconfig are
+# still present in the .config
+
+import sys
+
+
+def main():
+ if not (len(sys.argv) == 3):
+ print("Error: incorrect number of arguments")
+ print("""Usage: check-dotconfig <configfile> <defconfig>""")
+ sys.exit(1)
+
+ configfile = sys.argv[1]
+ defconfig = sys.argv[2]
+
+ # strip() to get rid of trailing \n
+ with open(configfile) as configf:
+ configlines = [l.strip() for l in configf.readlines()]
+
+ defconfiglines = []
+ with open(defconfig) as defconfigf:
+ for line in defconfigf.readlines():
+ # strip() to get rid of trailing \n
+ line = line.strip()
+ if line.startswith("BR2_"):
+ defconfiglines.append(line)
+ elif line.startswith('# BR2_') and line.endswith(' is not set'):
+ defconfiglines.append(line)
+
+ # Check that all the defconfig lines are still present
+ missing = [defconfigline.strip() for defconfigline in defconfiglines
+ if defconfigline not in configlines]
+
+ if len(missing):
+ print("WARN: defconfig {} can't be used:".format(defconfig))
+ for m in missing:
+ print(" Missing: {}".format(m))
+ sys.exit(1 if len(missing) else 0)
+
+
+if __name__ == "__main__":
+ main()
--
2.25.4
^ permalink raw reply related [flat|nested] 8+ messages in thread
* [Buildroot] [PATCHv3 4/6] support/scripts: add check-dotconfig.py
2020-07-27 15:51 ` [Buildroot] [PATCHv3 4/6] support/scripts: add check-dotconfig.py Romain Naour
@ 2020-07-27 16:18 ` Yann E. MORIN
0 siblings, 0 replies; 8+ messages in thread
From: Yann E. MORIN @ 2020-07-27 16:18 UTC (permalink / raw)
To: buildroot
Romain, All,
On 2020-07-27 17:51 +0200, Romain Naour spake thusly:
> For the same reason as for 50b747f212be2c9c0f7cf10c674ed488d042715c,
> we need to check if the generated configuration file (.config)
> contains all symbols present in the defconfig file.
>
> If not there is an issue with the defconfig.
>
> This script will be used in .gitlab-ci.yml.
>
> Inspired by is_toolchain_usable() function from genrandconfig:
> https://git.busybox.net/buildroot/tree/utils/genrandconfig?h=2020.02#n164
>
> Signed-off-by: Romain Naour <romain.naour@gmail.com>
> Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
> Cc: Yann E. MORIN <yann.morin.1998@free.fr>
> ---
> v2: update with the improved patch suggested by Yann.
> http://lists.busybox.net/pipermail/buildroot/2020-April/280679.html
> ---
> support/scripts/check-dotconfig.py | 44 ++++++++++++++++++++++++++++++
> 1 file changed, 44 insertions(+)
> create mode 100755 support/scripts/check-dotconfig.py
>
> diff --git a/support/scripts/check-dotconfig.py b/support/scripts/check-dotconfig.py
> new file mode 100755
> index 0000000000..81c83a8481
> --- /dev/null
> +++ b/support/scripts/check-dotconfig.py
> @@ -0,0 +1,44 @@
> +#!/usr/bin/env python3
> +
> +# This scripts check that all lines present in the defconfig are
> +# still present in the .config
> +
> +import sys
> +
> +
> +def main():
> + if not (len(sys.argv) == 3):
> + print("Error: incorrect number of arguments")
> + print("""Usage: check-dotconfig <configfile> <defconfig>""")
> + sys.exit(1)
> +
> + configfile = sys.argv[1]
> + defconfig = sys.argv[2]
> +
> + # strip() to get rid of trailing \n
> + with open(configfile) as configf:
> + configlines = [l.strip() for l in configf.readlines()]
> +
> + defconfiglines = []
> + with open(defconfig) as defconfigf:
> + for line in defconfigf.readlines():
> + # strip() to get rid of trailing \n
> + line = line.strip()
I've used a generator to strip the lines when reading:
for line in (line.strip() for line in defconfigf.readlines()):
> + if line.startswith("BR2_"):
> + defconfiglines.append(line)
> + elif line.startswith('# BR2_') and line.endswith(' is not set'):
> + defconfiglines.append(line)
> +
> + # Check that all the defconfig lines are still present
> + missing = [defconfigline.strip() for defconfigline in defconfiglines
No need to strip() here, lines are already stripped. I also shortened
the variable from defconfigline to simply line, so that it fits on a
single line. (muahaha!)
> + if defconfigline not in configlines]
> +
> + if len(missing):
Testing is a list ie empty is simply done by testing the list;
if missing:
> + print("WARN: defconfig {} can't be used:".format(defconfig))
> + for m in missing:
> + print(" Missing: {}".format(m))
> + sys.exit(1 if len(missing) else 0)
I moved the sys.exit(1) inside the conditional, it's s much simpler.
Applied to master with those fixes, thanks!
Regards,
Yann E. MORIN.
> +
> +
> +if __name__ == "__main__":
> + main()
> --
> 2.25.4
>
--
.-----------------.--------------------.------------------.--------------------.
| Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ |
| +33 561 099 427 `------------.-------: X AGAINST | \e/ There is no |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. |
'------------------------------^-------^------------------^--------------------'
^ permalink raw reply [flat|nested] 8+ messages in thread
* [Buildroot] [PATCHv3 5/6] gitlab-ci: check generated config files
2020-07-27 15:51 [Buildroot] [PATCHv3 1/6] configs/imx6ullevk_defconfig: remove typo Romain Naour
` (2 preceding siblings ...)
2020-07-27 15:51 ` [Buildroot] [PATCHv3 4/6] support/scripts: add check-dotconfig.py Romain Naour
@ 2020-07-27 15:51 ` Romain Naour
2020-07-27 15:51 ` [Buildroot] [PATCHv3 6/6] gitlab-ci: check all defconfigs on every push Romain Naour
2020-07-27 16:19 ` [Buildroot] [PATCHv3 1/6] configs/imx6ullevk_defconfig: remove typo Yann E. MORIN
5 siblings, 0 replies; 8+ messages in thread
From: Romain Naour @ 2020-07-27 15:51 UTC (permalink / raw)
To: buildroot
Use the script added by the previous patch to check
generated config files.
Tested on gitlab:
https://gitlab.com/kubu93/buildroot/pipelines/137597966
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
---
support/misc/gitlab-ci.yml.in | 1 +
1 file changed, 1 insertion(+)
diff --git a/support/misc/gitlab-ci.yml.in b/support/misc/gitlab-ci.yml.in
index cf9bf5f9cf..01592f7155 100644
--- a/support/misc/gitlab-ci.yml.in
+++ b/support/misc/gitlab-ci.yml.in
@@ -38,6 +38,7 @@ check-package:
script:
- echo "Configure Buildroot for ${DEFCONFIG_NAME}"
- make ${DEFCONFIG_NAME}
+ - ./support/scripts/check-dotconfig.py .config ./configs/${DEFCONFIG_NAME}
- echo 'Build buildroot'
- |
make > >(tee build.log |grep '>>>') 2>&1 || {
--
2.25.4
^ permalink raw reply related [flat|nested] 8+ messages in thread
* [Buildroot] [PATCHv3 6/6] gitlab-ci: check all defconfigs on every push
2020-07-27 15:51 [Buildroot] [PATCHv3 1/6] configs/imx6ullevk_defconfig: remove typo Romain Naour
` (3 preceding siblings ...)
2020-07-27 15:51 ` [Buildroot] [PATCHv3 5/6] gitlab-ci: check generated config files Romain Naour
@ 2020-07-27 15:51 ` Romain Naour
2020-07-27 16:19 ` [Buildroot] [PATCHv3 1/6] configs/imx6ullevk_defconfig: remove typo Yann E. MORIN
5 siblings, 0 replies; 8+ messages in thread
From: Romain Naour @ 2020-07-27 15:51 UTC (permalink / raw)
To: buildroot
The defconfig check has been introduced by the previous
patch before the building each defconfig but those builds
are done every week or more.
Checking if a defconfig is valid can be done on every
push in the repository since it take few seconds.
This would allow to detect as soon as possible a problem
in a defconfig and eventually avoid breaking the build
while build testing all defconfig.
Introduce a new job template ".defconfig_check" in
gitlab-ci.yml.in and modify the generate-gitlab-ci-yml
to create a job for each defconfig to run the test.
Although, we could have used only one job to do all
tests, using one job per defconfig allow to identify
easily in gitlab which defconfig is falling.
Tested:
https://gitlab.com/kubu93/buildroot/pipelines/138331069
https://gitlab.com/kubu93/buildroot/pipelines/171223758
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
---
support/misc/gitlab-ci.yml.in | 13 +++++++++++++
support/scripts/generate-gitlab-ci-yml | 2 +-
2 files changed, 14 insertions(+), 1 deletion(-)
diff --git a/support/misc/gitlab-ci.yml.in b/support/misc/gitlab-ci.yml.in
index 01592f7155..7218ea027e 100644
--- a/support/misc/gitlab-ci.yml.in
+++ b/support/misc/gitlab-ci.yml.in
@@ -34,6 +34,19 @@ check-package:
script:
- make check-package
+.defconfig_check:
+ script:
+ - echo "Configure Buildroot for ${DEFCONFIG_NAME}"
+ - make ${DEFCONFIG_NAME}
+ - support/scripts/check-dotconfig.py .config configs/${DEFCONFIG_NAME}
+ artifacts:
+ when: on_failure
+ expire_in: 2 weeks
+ paths:
+ - .config
+ before_script:
+ - DEFCONFIG_NAME=$(echo ${CI_JOB_NAME} | sed -e 's,_check$,,g')
+
.defconfig_base:
script:
- echo "Configure Buildroot for ${DEFCONFIG_NAME}"
diff --git a/support/scripts/generate-gitlab-ci-yml b/support/scripts/generate-gitlab-ci-yml
index 262a7649b2..5cef1146e2 100755
--- a/support/scripts/generate-gitlab-ci-yml
+++ b/support/scripts/generate-gitlab-ci-yml
@@ -10,7 +10,7 @@ cat "${input}"
cd configs
LC_ALL=C ls -1 *_defconfig
) \
- | sed 's/$/: { extends: .defconfig }/'
+ | sed -r -e 's/^(.+)$/\1: { extends: .defconfig }\n\1_check: { extends: .defconfig_check }/'
./support/testing/run-tests -l 2>&1 \
| sed -r -e '/^test_run \((.*)\).*/!d; s//\1: { extends: .runtime_test }/' \
--
2.25.4
^ permalink raw reply related [flat|nested] 8+ messages in thread
* [Buildroot] [PATCHv3 1/6] configs/imx6ullevk_defconfig: remove typo
2020-07-27 15:51 [Buildroot] [PATCHv3 1/6] configs/imx6ullevk_defconfig: remove typo Romain Naour
` (4 preceding siblings ...)
2020-07-27 15:51 ` [Buildroot] [PATCHv3 6/6] gitlab-ci: check all defconfigs on every push Romain Naour
@ 2020-07-27 16:19 ` Yann E. MORIN
5 siblings, 0 replies; 8+ messages in thread
From: Yann E. MORIN @ 2020-07-27 16:19 UTC (permalink / raw)
To: buildroot
Romain, All,
On 2020-07-27 17:51 +0200, Romain Naour spake thusly:
> Reported by check-dotconfig.py.
>
> Fixes:
> https://gitlab.com/kubu93/buildroot/-/jobs/657885169
>
> Signed-off-by: Romain Naour <romain.naour@gmail.com>
Whole series applied to master, thanks.
Please see my reply to patch 4 for specific comments on that one.
Regards,
Yann E. MORIN.
> ---
> configs/imx6ullevk_defconfig | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/configs/imx6ullevk_defconfig b/configs/imx6ullevk_defconfig
> index 4a5558dd6c..81db66268a 100644
> --- a/configs/imx6ullevk_defconfig
> +++ b/configs/imx6ullevk_defconfig
> @@ -23,7 +23,7 @@ BR2_TARGET_UBOOT=y
> BR2_TARGET_UBOOT_BOARDNAME="mx6ull_14x14_evk"
> BR2_TARGET_UBOOT_CUSTOM_VERSION=y
> BR2_TARGET_UBOOT_CUSTOM_VERSION_VALUE="2020.07"
> -BR2_TARGET_UBOOT_FORMAT_DTB_IMX=y"
> +BR2_TARGET_UBOOT_FORMAT_DTB_IMX=y
> BR2_TARGET_UBOOT_NEEDS_DTC=y
>
> # required tools to create the SD card image
> --
> 2.25.4
>
> _______________________________________________
> buildroot mailing list
> buildroot at busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot
--
.-----------------.--------------------.------------------.--------------------.
| Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ |
| +33 561 099 427 `------------.-------: X AGAINST | \e/ There is no |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. |
'------------------------------^-------^------------------^--------------------'
^ permalink raw reply [flat|nested] 8+ messages in thread