[Buildroot] [PATCH][autobuild] autobuild-run: introduce "--tc-cfg-uri" option for toolchain configs URI

[Buildroot] [PATCH][autobuild] autobuild-run: introduce "--tc-cfg-uri" option for toolchain configs URI

[Vlad Zakharov]

This commit introduces "--tc-cfg-uri" option (short "-t").
Described option is used to pass URI of file that contain
toolchains configurations to be used in autobuilds.

Default value refers to standard buildroot autobuild toolchain
configuration file:
http://autobuild.buildroot.org/toolchains/configs/toolchain-configs.csv

Such update allows users to add their own toolchain configurations
to use in buildroot autobuilds.

Signed-off-by: Vlad Zakharov <vzakhar@synopsys.com>
---
 scripts/autobuild-run | 9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)

diff --git a/scripts/autobuild-run b/scripts/autobuild-run
index 33a0481..4b0d7c2 100755
--- a/scripts/autobuild-run
+++ b/scripts/autobuild-run
@@ -66,6 +66,7 @@ defaults = {
     '--make-opts': '',
     '--nice': 0,
     '--pid-file': '/tmp/buildroot-autobuild.pid',
+    '--tc-cfg-uri': 'http://autobuild.buildroot.org/toolchains/configs/toolchain-configs.csv',
 }
 
 doc = """autobuild-run - run Buildroot autobuilder
@@ -97,6 +98,7 @@ Options:
                                  Defaults to %(--pid-file)s.
   -c, --config CONFIG            path to configuration file
                                  Not set by default.
+  -t, --tc-cfg-uri URI           URI of toolchain configuration file
 
 Format of the configuration file:
 
@@ -243,7 +245,7 @@ class SystemInfo:
 
         return not missing_requirements
 
-def get_toolchain_configs():
+def get_toolchain_configs(**kwargs):
     """Fetch and return the possible toolchain configurations
 
     This function returns an array of dictionaries, with for each toolchain:
@@ -252,7 +254,7 @@ def get_toolchain_configs():
         - hostarch: the host architecture for which the toolchain is built
         - contents: an array of lines of the defconfig
     """
-    tc_cfg_uri = 'http://autobuild.buildroot.org/toolchains/configs/toolchain-configs.csv'
+    tc_cfg_uri = kwargs['tc_cfg_uri']
 
     with urlopen_closing(tc_cfg_uri) as r:
         l = decode_byte_list(r.readlines())
@@ -514,7 +516,7 @@ def gen_config(**kwargs):
     log_write(log, "INFO: generate the configuration")
 
     # Select a random toolchain configuration
-    configs = get_toolchain_configs()
+    configs = get_toolchain_configs(**kwargs)
     i = randint(0, len(configs) - 1)
     config = configs[i]
 
@@ -905,6 +907,7 @@ def main():
                 submitter = args['--submitter'],
                 make_opts = (args['--make-opts'] or ''),
                 nice = (args['--nice'] or 0),
+                tc_cfg_uri = args['--tc-cfg-uri'],
                 upload = upload,
                 buildpid = buildpid
             ))
-- 
2.5.5

[Buildroot] [PATCH][autobuild] autobuild-run: introduce "--tc-cfg-uri" option for toolchain configs URI

[Thomas Petazzoni]

Hello,

On Fri, 21 Oct 2016 13:24:14 +0300, Vlad Zakharov wrote:
> This commit introduces "--tc-cfg-uri" option (short "-t").
> Described option is used to pass URI of file that contain
> toolchains configurations to be used in autobuilds.
> 
> Default value refers to standard buildroot autobuild toolchain
> configuration file:
> http://autobuild.buildroot.org/toolchains/configs/toolchain-configs.csv
> 
> Such update allows users to add their own toolchain configurations
> to use in buildroot autobuilds.
> 
> Signed-off-by: Vlad Zakharov <vzakhar@synopsys.com>

I am not sure I like this approach very much. It means people can start
using a non-default set of toolchains, and then submit results to
autobuild.buildroot.org. Such results would possibly not be
reproducible by others because we may not have access to those
toolchains.

Is this intended to run tests only the ARC architecture builds on your
autobuilder?

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux and Kernel engineering
http://free-electrons.com

[Buildroot] [PATCH][autobuild] autobuild-run: introduce "--tc-cfg-uri" option for toolchain configs URI

[Vlad Zakharov]

Hi Thomas,

Yes, we wan't to use?

On Fri, 2016-10-21 at 13:34 +0200, Thomas Petazzoni wrote:
> Hello,
> 
> On Fri, 21 Oct 2016 13:24:14 +0300, Vlad Zakharov wrote:
> > 
> > This commit introduces "--tc-cfg-uri" option (short "-t").
> > Described option is used to pass URI of file that contain
> > toolchains configurations to be used in autobuilds.
> > 
> > Default value refers to standard buildroot autobuild toolchain
> > configuration file:
> > http://autobuild.buildroot.org/toolchains/configs/toolchain-configs.csv
> > 
> > Such update allows users to add their own toolchain configurations
> > to use in buildroot autobuilds.
> > 
> > Signed-off-by: Vlad Zakharov <vzakhar@synopsys.com>
> 
> I am not sure I like this approach very much. It means people can start
> using a non-default set of toolchains, and then submit results to
> autobuild.buildroot.org. Such results would possibly not be
> reproducible by others because we may not have access to those
> toolchains.
> 

Of course people can misuse the script and run autobuilds with mysterious external toolchain and nobody except
themselves will be able to reproduce such builds. However if people use internal buildroot toolchain information of how
toolchain has been configured is always accessible in ".config" file that is provided with other build results.?

> Is this intended to run tests only the ARC architecture builds on your
> autobuilder?
> 

In our case we want at least to run more autobuilds with internal up-to-date toolchain to get relevant visibility of
what is happening with ARC latest tools. Suggested approach will give us flexible way to set up such configurations.?
In default configs there is only one configuration for arc that uses internal tools.?
?

> Thomas
-- 
Best regards,
Vlad Zakharov <vzakhar@synopsys.com>

[Buildroot] [PATCH][autobuild] autobuild-run: introduce "--tc-cfg-uri" option for toolchain configs URI

[Alexey Brodkin]

Hi Thomas,

On Fri, 2016-10-21 at 11:56 +0000, Vlad Zakharov wrote:
> Hi Thomas,
> 
> Yes, we wan't to use?
> 
> On Fri, 2016-10-21 at 13:34 +0200, Thomas Petazzoni wrote:
> > 
> > Hello,
> > 
> > On Fri, 21 Oct 2016 13:24:14 +0300, Vlad Zakharov wrote:
> > > 
> > > 
> > > This commit introduces "--tc-cfg-uri" option (short "-t").
> > > Described option is used to pass URI of file that contain
> > > toolchains configurations to be used in autobuilds.
> > > 
> > > Default value refers to standard buildroot autobuild toolchain
> > > configuration file:
> > > http://autobuild.buildroot.org/toolchains/configs/toolchain-configs.csv
> > > 
> > > Such update allows users to add their own toolchain configurations
> > > to use in buildroot autobuilds.
> > > 
> > > Signed-off-by: Vlad Zakharov <vzakhar@synopsys.com>
> > 
> > I am not sure I like this approach very much. It means people can start
> > using a non-default set of toolchains, and then submit results to
> > autobuild.buildroot.org. Such results would possibly not be
> > reproducible by others because we may not have access to those
> > toolchains.

Well as long as you have control over stuff that gets added in your
autobuilder web-server through distribution of credentials
IMHO there should be no problems. In worst case one day somebody
will catch unexpected path to external toolchain in .config from
one report and that will be a reason to ask owner of corresponding
build server to not play that games.

Still I might be missing something.

-Alexey

[Buildroot] [PATCH][autobuild] autobuild-run: introduce "--tc-cfg-uri" option for toolchain configs URI

[Arnout Vandecappelle]

On 21-10-16 13:34, Thomas Petazzoni wrote:
> Hello,
> 
> On Fri, 21 Oct 2016 13:24:14 +0300, Vlad Zakharov wrote:
>> This commit introduces "--tc-cfg-uri" option (short "-t").
>> Described option is used to pass URI of file that contain
>> toolchains configurations to be used in autobuilds.
>>
>> Default value refers to standard buildroot autobuild toolchain
>> configuration file:
>> http://autobuild.buildroot.org/toolchains/configs/toolchain-configs.csv
>>
>> Such update allows users to add their own toolchain configurations
>> to use in buildroot autobuilds.
>>
>> Signed-off-by: Vlad Zakharov <vzakhar@synopsys.com>
> 
> I am not sure I like this approach very much. It means people can start
> using a non-default set of toolchains, and then submit results to
> autobuild.buildroot.org. Such results would possibly not be
> reproducible by others because we may not have access to those
> toolchains.

 Maybe just make this option mutually exclusive --upload? I.e., automatically
disable upload if this option is set. I guess that's OK for the ARC use case?

 Regards,
 Arnout

> 
> Is this intended to run tests only the ARC architecture builds on your
> autobuilder?
> 
> Thomas
> 

-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF

[Buildroot] [PATCH][autobuild] autobuild-run: introduce "--tc-cfg-uri" option for toolchain configs URI

[Alexey Brodkin]

Hi Arnout,

> -----Original Message-----
> From: Arnout Vandecappelle [mailto:arnout at mind.be]
> Sent: 21 ??????? 2016 ?. 21:14
> To: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>; Vlad Zakharov <Vladislav.Zakharov@synopsys.com>
> Cc: buildroot at busybox.net; Alexey Brodkin <Alexey.Brodkin@synopsys.com>; Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
> Subject: Re: [Buildroot] [PATCH][autobuild] autobuild-run: introduce "--tc-cfg-uri" option for toolchain configs URI

[snip]

> >
> > I am not sure I like this approach very much. It means people can start
> > using a non-default set of toolchains, and then submit results to
> > autobuild.buildroot.org. Such results would possibly not be
> > reproducible by others because we may not have access to those
> > toolchains.
> 
>  Maybe just make this option mutually exclusive --upload? I.e., automatically
> disable upload if this option is set. I guess that's OK for the ARC use case?

Well we may indeed implement something like this.

But frankly the idea was to give more testing to "non-standard" configurations
especially with Buildroot-built toolchain (on every build so no prebuilt tools at all).
I.e. to get better coverage when BR's toolchain is configured with different set of features.

-Alexey

[Buildroot] [PATCH][autobuild] autobuild-run: introduce "--tc-cfg-uri" option for toolchain configs URI

[Arnout Vandecappelle]

On 21-10-16 21:18, Alexey Brodkin wrote:
> Hi Arnout,
> 
>> -----Original Message-----
>> From: Arnout Vandecappelle [mailto:arnout at mind.be]
>> Sent: 21 ??????? 2016 ?. 21:14
>> To: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>; Vlad Zakharov <Vladislav.Zakharov@synopsys.com>
>> Cc: buildroot at busybox.net; Alexey Brodkin <Alexey.Brodkin@synopsys.com>; Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
>> Subject: Re: [Buildroot] [PATCH][autobuild] autobuild-run: introduce "--tc-cfg-uri" option for toolchain configs URI
> 
> [snip]
> 
>>>
>>> I am not sure I like this approach very much. It means people can start
>>> using a non-default set of toolchains, and then submit results to
>>> autobuild.buildroot.org. Such results would possibly not be
>>> reproducible by others because we may not have access to those
>>> toolchains.
>>
>>  Maybe just make this option mutually exclusive --upload? I.e., automatically
>> disable upload if this option is set. I guess that's OK for the ARC use case?
> 
> Well we may indeed implement something like this.
> 
> But frankly the idea was to give more testing to "non-standard" configurations
> especially with Buildroot-built toolchain (on every build so no prebuilt tools at all).
> I.e. to get better coverage when BR's toolchain is configured with different set of features.

 If the intention is that they are registered in a.b.o, why not just add the
configurations to the official list?

 Regards,
 Arnout

-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF

[Buildroot] [PATCH][autobuild] autobuild-run: introduce "--tc-cfg-uri" option for toolchain configs URI

[Alexey Brodkin]

Hi Arnout,

> -----Original Message-----
> From: Arnout Vandecappelle [mailto:arnout at mind.be]
> Sent: 21 ??????? 2016 ?. 22:29
> To: Alexey Brodkin <Alexey.Brodkin@synopsys.com>; Vlad Zakharov <Vladislav.Zakharov@synopsys.com>
> Cc: buildroot at busybox.net; Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
> Subject: Re: [Buildroot] [PATCH][autobuild] autobuild-run: introduce "--tc-cfg-uri" option for toolchain configs URI

[snip]

> >>
> >>  Maybe just make this option mutually exclusive --upload? I.e., automatically
> >> disable upload if this option is set. I guess that's OK for the ARC use case?
> >
> > Well we may indeed implement something like this.
> >
> > But frankly the idea was to give more testing to "non-standard" configurations
> > especially with Buildroot-built toolchain (on every build so no prebuilt tools at all).
> > I.e. to get better coverage when BR's toolchain is configured with different set of features.
> 
>  If the intention is that they are registered in a.b.o, why not just add the
> configurations to the official list?

Well we may do it.
But as I understood autobuilder script randomly selects config from this list
http://autobuild.buildroot.org/toolchains/configs/toolchain-configs.csv

Which means if we add more entries for ARC we'll have ARC build happening more often
But I'd like to keep balance as it is because obviously all targets should get equal testing.

-Alexey

[Buildroot] [PATCH][autobuild] autobuild-run: introduce "--tc-cfg-uri" option for toolchain configs URI

[Thomas Petazzoni]

Hello,

On Fri, 21 Oct 2016 13:24:14 +0300, Vlad Zakharov wrote:
> This commit introduces "--tc-cfg-uri" option (short "-t").
> Described option is used to pass URI of file that contain
> toolchains configurations to be used in autobuilds.
> 
> Default value refers to standard buildroot autobuild toolchain
> configuration file:
> http://autobuild.buildroot.org/toolchains/configs/toolchain-configs.csv
> 
> Such update allows users to add their own toolchain configurations
> to use in buildroot autobuilds.
> 
> Signed-off-by: Vlad Zakharov <vzakhar@synopsys.com>
> ---
>  scripts/autobuild-run | 9 ++++++---
>  1 file changed, 6 insertions(+), 3 deletions(-)

I've changed the option to --toolchains-url and applied. Thanks!

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com