* [U-Boot] [PATCH 0/3] buildman updates
@ 2018-11-21 8:31 Trevor Woerner
2018-11-21 8:31 ` [U-Boot] [PATCH 1/3] buildman/toolchain.py: update versions Trevor Woerner
` (2 more replies)
0 siblings, 3 replies; 16+ messages in thread
From: Trevor Woerner @ 2018-11-21 8:31 UTC (permalink / raw)
To: u-boot
Hi,
This patchset was created to update buildman, and fix a couple small issues
that were discovered. kernel.org has a new set of toolchains (8.1.0), and I
ran into some issues trying to download them.
I'm not a python guru, so if there are better ways to code these fixes, please
let me know.
Best regards,
Trevor
Trevor Woerner (3):
buildman/toolchain.py: update versions
buildman/toolchain.py: fix toolchain directory
buildman/toolchain.py: handle inconsistent tarball names
tools/buildman/toolchain.py | 10 ++++++----
1 file changed, 6 insertions(+), 4 deletions(-)
--
2.17.0.582.gccdcbd54c
^ permalink raw reply [flat|nested] 16+ messages in thread
* [U-Boot] [PATCH 1/3] buildman/toolchain.py: update versions
2018-11-21 8:31 [U-Boot] [PATCH 0/3] buildman updates Trevor Woerner
@ 2018-11-21 8:31 ` Trevor Woerner
2018-11-27 1:02 ` Simon Glass
2018-11-29 1:35 ` Daniel Schwierzeck
2018-11-21 8:31 ` [U-Boot] [PATCH 2/3] buildman/toolchain.py: fix toolchain directory Trevor Woerner
2018-11-21 8:31 ` [U-Boot] [PATCH 3/3] buildman/toolchain.py: handle inconsistent tarball names Trevor Woerner
2 siblings, 2 replies; 16+ messages in thread
From: Trevor Woerner @ 2018-11-21 8:31 UTC (permalink / raw)
To: u-boot
On kernel.org, a newer 8.1.0 version of the toolchains is available. Also,
update the kernel.org version list to include some of the older versions so
more architectures can be supported.
Signed-off-by: Trevor Woerner <trevor@toganlabs.com>
---
tools/buildman/toolchain.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/tools/buildman/toolchain.py b/tools/buildman/toolchain.py
index 4b35f400e9..442a7f977a 100644
--- a/tools/buildman/toolchain.py
+++ b/tools/buildman/toolchain.py
@@ -430,7 +430,7 @@ class Toolchains:
"""
arch = command.OutputOneLine('uname', '-m')
base = 'https://www.kernel.org/pub/tools/crosstool/files/bin'
- versions = ['7.3.0', '6.4.0', '4.9.4']
+ versions = ['8.1.0', '7.3.0', '6.4.0', '5.5.0', '4.9.4', '4.9.0', '4.8.5', '4.8.0', '4.7.3', '4.6.3', '4.6.2', '4.6.1', '4.5.1', '4.2.4']
links = []
for version in versions:
url = '%s/%s/%s/' % (base, arch, version)
--
2.17.0.582.gccdcbd54c
^ permalink raw reply related [flat|nested] 16+ messages in thread
* [U-Boot] [PATCH 2/3] buildman/toolchain.py: fix toolchain directory
2018-11-21 8:31 [U-Boot] [PATCH 0/3] buildman updates Trevor Woerner
2018-11-21 8:31 ` [U-Boot] [PATCH 1/3] buildman/toolchain.py: update versions Trevor Woerner
@ 2018-11-21 8:31 ` Trevor Woerner
2018-11-27 1:02 ` Simon Glass
2018-11-21 8:31 ` [U-Boot] [PATCH 3/3] buildman/toolchain.py: handle inconsistent tarball names Trevor Woerner
2 siblings, 1 reply; 16+ messages in thread
From: Trevor Woerner @ 2018-11-21 8:31 UTC (permalink / raw)
To: u-boot
The hexagon toolchain (4.6.1) from kernel.org, for example, was packaged in
a way that is different from most toolchains. The first entry when unpacking
most toolchain tarballs is:
gcc-<version>-nolib/<targetarch>-<system>
e.g.:
gcc-8.1.0-nolibc/aarch64-linux/
The first entry of the hexagon toolchain, however, is:
gcc-4.6.1-nolibc/
This causes the buildman logic in toolchain.py::ScanPath() to not be able to
find the "*gcc" executable since it looks in gcc-4.6.1-nolib/{.|bin|usr/bin}
instead of gcc-4.6.1/hexagon-linux/{.|bin|usr/bin}. Therefore when buildman
tries to download a set of toolchains that includes hexagon, the script fails.
This update takes the second line of the tarball unpacking (which works for
all the toolchains I've tested from kernel.org) and parses it to take the
first two elements, separated by '/'. It makes this logic a bit more robust.
Signed-off-by: Trevor Woerner <trevor@toganlabs.com>
---
tools/buildman/toolchain.py | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/tools/buildman/toolchain.py b/tools/buildman/toolchain.py
index 442a7f977a..326c609a41 100644
--- a/tools/buildman/toolchain.py
+++ b/tools/buildman/toolchain.py
@@ -502,7 +502,8 @@ class Toolchains:
trailing /
"""
stdout = command.Output('tar', 'xvfJ', fname, '-C', dest)
- return stdout.splitlines()[0][:-1]
+ dirs = stdout.splitlines()[1].split('/')[:2]
+ return '/'.join(dirs)
def TestSettingsHasPath(self, path):
"""Check if buildman will find this toolchain
--
2.17.0.582.gccdcbd54c
^ permalink raw reply related [flat|nested] 16+ messages in thread
* [U-Boot] [PATCH 3/3] buildman/toolchain.py: handle inconsistent tarball names
2018-11-21 8:31 [U-Boot] [PATCH 0/3] buildman updates Trevor Woerner
2018-11-21 8:31 ` [U-Boot] [PATCH 1/3] buildman/toolchain.py: update versions Trevor Woerner
2018-11-21 8:31 ` [U-Boot] [PATCH 2/3] buildman/toolchain.py: fix toolchain directory Trevor Woerner
@ 2018-11-21 8:31 ` Trevor Woerner
2018-11-27 1:02 ` Simon Glass
2 siblings, 1 reply; 16+ messages in thread
From: Trevor Woerner @ 2018-11-21 8:31 UTC (permalink / raw)
To: u-boot
Unfortunately, for some releases the kernel.org toolchain tarball names adhere
to the following pattern:
<hostarch>-gcc-<ver>-nolib-<targetarch>-<type>.tar.xz
e.g.:
x86_64-gcc-8.1.0-nolibc-aarch64-linux.tar.xz
while others use the following pattern:
<hostarch>-gcc-<ver>-nolib_<targetarch>-<type>.tar.xz
e.g.:
x86_64-gcc-7.3.0-nolibc_aarch64-linux.tar.xz
Notice that the first pattern has dashes throughout, while the second has
dashes throughout except just before the target architecture which has an
underscore.
The "dash throughout" versions from kernel.org are:
8.1.0, 6.4.0, 5.5.0, 4.9.4, 4.8.5, 4.6.1
while the "dash and underscore" versions from kernel.org are:
7.3.0, 4.9.0, 4.8.0, 4.7.3, 4.6.3, 4.6.2, 4.5.1, 4.2.4
This tweak allows the code to handle both versions. Note that this tweak also
causes the architecture parsing to get confused and find the following two
bogus architectures, "2.0" and "64", which are explicitly checked for, and
removed.
Signed-off-by: Trevor Woerner <trevor@toganlabs.com>
---
tools/buildman/toolchain.py | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
diff --git a/tools/buildman/toolchain.py b/tools/buildman/toolchain.py
index 326c609a41..4db70572e1 100644
--- a/tools/buildman/toolchain.py
+++ b/tools/buildman/toolchain.py
@@ -517,13 +517,14 @@ class Toolchains:
def ListArchs(self):
"""List architectures with available toolchains to download"""
host_arch, archives = self.LocateArchUrl('list')
- re_arch = re.compile('[-a-z0-9.]*_([^-]*)-.*')
+ re_arch = re.compile('[-a-z0-9.]*[-_]([^-]*)-.*')
arch_set = set()
for archive in archives:
# Remove the host architecture from the start
arch = re_arch.match(archive[len(host_arch):])
if arch:
- arch_set.add(arch.group(1))
+ if arch.group(1) != "2.0" and arch.group(1) != "64":
+ arch_set.add(arch.group(1))
return sorted(arch_set)
def FetchAndInstall(self, arch):
--
2.17.0.582.gccdcbd54c
^ permalink raw reply related [flat|nested] 16+ messages in thread
* [U-Boot] [PATCH 1/3] buildman/toolchain.py: update versions
2018-11-21 8:31 ` [U-Boot] [PATCH 1/3] buildman/toolchain.py: update versions Trevor Woerner
@ 2018-11-27 1:02 ` Simon Glass
2018-11-29 1:35 ` Daniel Schwierzeck
1 sibling, 0 replies; 16+ messages in thread
From: Simon Glass @ 2018-11-27 1:02 UTC (permalink / raw)
To: u-boot
On Wed, 21 Nov 2018 at 01:32, Trevor Woerner <trevor@toganlabs.com> wrote:
>
> On kernel.org, a newer 8.1.0 version of the toolchains is available. Also,
> update the kernel.org version list to include some of the older versions so
> more architectures can be supported.
>
> Signed-off-by: Trevor Woerner <trevor@toganlabs.com>
> ---
> tools/buildman/toolchain.py | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
Reviewed-by: Simon Glass <sjg@chromium.org>
^ permalink raw reply [flat|nested] 16+ messages in thread
* [U-Boot] [PATCH 2/3] buildman/toolchain.py: fix toolchain directory
2018-11-21 8:31 ` [U-Boot] [PATCH 2/3] buildman/toolchain.py: fix toolchain directory Trevor Woerner
@ 2018-11-27 1:02 ` Simon Glass
0 siblings, 0 replies; 16+ messages in thread
From: Simon Glass @ 2018-11-27 1:02 UTC (permalink / raw)
To: u-boot
On Wed, 21 Nov 2018 at 01:32, Trevor Woerner <trevor@toganlabs.com> wrote:
>
> The hexagon toolchain (4.6.1) from kernel.org, for example, was packaged in
> a way that is different from most toolchains. The first entry when unpacking
> most toolchain tarballs is:
>
> gcc-<version>-nolib/<targetarch>-<system>
>
> e.g.:
>
> gcc-8.1.0-nolibc/aarch64-linux/
>
> The first entry of the hexagon toolchain, however, is:
>
> gcc-4.6.1-nolibc/
>
> This causes the buildman logic in toolchain.py::ScanPath() to not be able to
> find the "*gcc" executable since it looks in gcc-4.6.1-nolib/{.|bin|usr/bin}
> instead of gcc-4.6.1/hexagon-linux/{.|bin|usr/bin}. Therefore when buildman
> tries to download a set of toolchains that includes hexagon, the script fails.
>
> This update takes the second line of the tarball unpacking (which works for
> all the toolchains I've tested from kernel.org) and parses it to take the
> first two elements, separated by '/'. It makes this logic a bit more robust.
>
> Signed-off-by: Trevor Woerner <trevor@toganlabs.com>
> ---
> tools/buildman/toolchain.py | 3 ++-
> 1 file changed, 2 insertions(+), 1 deletion(-)
Reviewed-by: Simon Glass <sjg@chromium.org>
BTW I really appreciate your great commit msg.
- Simon
^ permalink raw reply [flat|nested] 16+ messages in thread
* [U-Boot] [PATCH 3/3] buildman/toolchain.py: handle inconsistent tarball names
2018-11-21 8:31 ` [U-Boot] [PATCH 3/3] buildman/toolchain.py: handle inconsistent tarball names Trevor Woerner
@ 2018-11-27 1:02 ` Simon Glass
2018-11-28 0:51 ` Trevor Woerner
0 siblings, 1 reply; 16+ messages in thread
From: Simon Glass @ 2018-11-27 1:02 UTC (permalink / raw)
To: u-boot
On Wed, 21 Nov 2018 at 01:32, Trevor Woerner <trevor@toganlabs.com> wrote:
>
> Unfortunately, for some releases the kernel.org toolchain tarball names adhere
> to the following pattern:
>
> <hostarch>-gcc-<ver>-nolib-<targetarch>-<type>.tar.xz
>
> e.g.:
> x86_64-gcc-8.1.0-nolibc-aarch64-linux.tar.xz
>
> while others use the following pattern:
>
> <hostarch>-gcc-<ver>-nolib_<targetarch>-<type>.tar.xz
>
> e.g.:
>
> x86_64-gcc-7.3.0-nolibc_aarch64-linux.tar.xz
>
> Notice that the first pattern has dashes throughout, while the second has
> dashes throughout except just before the target architecture which has an
> underscore.
>
> The "dash throughout" versions from kernel.org are:
>
> 8.1.0, 6.4.0, 5.5.0, 4.9.4, 4.8.5, 4.6.1
>
> while the "dash and underscore" versions from kernel.org are:
>
> 7.3.0, 4.9.0, 4.8.0, 4.7.3, 4.6.3, 4.6.2, 4.5.1, 4.2.4
>
> This tweak allows the code to handle both versions. Note that this tweak also
> causes the architecture parsing to get confused and find the following two
> bogus architectures, "2.0" and "64", which are explicitly checked for, and
> removed.
>
> Signed-off-by: Trevor Woerner <trevor@toganlabs.com>
> ---
> tools/buildman/toolchain.py | 5 +++--
> 1 file changed, 3 insertions(+), 2 deletions(-)
Reviewed-by: Simon Glass <sjg@chromium.org>
But please use single quotes for strings.
^ permalink raw reply [flat|nested] 16+ messages in thread
* [U-Boot] [PATCH 3/3] buildman/toolchain.py: handle inconsistent tarball names
2018-11-27 1:02 ` Simon Glass
@ 2018-11-28 0:51 ` Trevor Woerner
2018-11-28 19:38 ` Simon Glass
0 siblings, 1 reply; 16+ messages in thread
From: Trevor Woerner @ 2018-11-28 0:51 UTC (permalink / raw)
To: u-boot
On Mon, Nov 26, 2018 at 8:06 PM Simon Glass <sjg@chromium.org> wrote:
> Reviewed-by: Simon Glass <sjg@chromium.org>
> But please use single quotes for strings.
>
I can send a v2 if you like?
^ permalink raw reply [flat|nested] 16+ messages in thread
* [U-Boot] [PATCH 3/3] buildman/toolchain.py: handle inconsistent tarball names
2018-11-28 0:51 ` Trevor Woerner
@ 2018-11-28 19:38 ` Simon Glass
2018-11-28 19:54 ` Trevor Woerner
2018-12-05 23:11 ` sjg at google.com
0 siblings, 2 replies; 16+ messages in thread
From: Simon Glass @ 2018-11-28 19:38 UTC (permalink / raw)
To: u-boot
Hi Trevor,
On Tue, 27 Nov 2018 at 17:51, Trevor Woerner <twoerner@gmail.com> wrote:
>
> On Mon, Nov 26, 2018 at 8:06 PM Simon Glass <sjg@chromium.org> wrote:
>>
>> Reviewed-by: Simon Glass <sjg@chromium.org>
>> But please use single quotes for strings.
>
>
> I can send a v2 if you like?
If you like, but I can fix it up when applying.
- Simon
^ permalink raw reply [flat|nested] 16+ messages in thread
* [U-Boot] [PATCH 3/3] buildman/toolchain.py: handle inconsistent tarball names
2018-11-28 19:38 ` Simon Glass
@ 2018-11-28 19:54 ` Trevor Woerner
2018-12-05 23:11 ` sjg at google.com
1 sibling, 0 replies; 16+ messages in thread
From: Trevor Woerner @ 2018-11-28 19:54 UTC (permalink / raw)
To: u-boot
On Wed, Nov 28, 2018 at 2:38 PM Simon Glass <sjg@chromium.org> wrote:
> Hi Trevor,
>
> On Tue, 27 Nov 2018 at 17:51, Trevor Woerner <twoerner@gmail.com> wrote:
> >
> > On Mon, Nov 26, 2018 at 8:06 PM Simon Glass <sjg@chromium.org> wrote:
> >>
> >> Reviewed-by: Simon Glass <sjg@chromium.org>
> >> But please use single quotes for strings.
> >
> >
> > I can send a v2 if you like?
>
> If you like, but I can fix it up when applying.
>
>
Yes, please go ahead and fix it up.
^ permalink raw reply [flat|nested] 16+ messages in thread
* [U-Boot] [PATCH 1/3] buildman/toolchain.py: update versions
2018-11-21 8:31 ` [U-Boot] [PATCH 1/3] buildman/toolchain.py: update versions Trevor Woerner
2018-11-27 1:02 ` Simon Glass
@ 2018-11-29 1:35 ` Daniel Schwierzeck
2018-11-29 19:03 ` Tom Rini
1 sibling, 1 reply; 16+ messages in thread
From: Daniel Schwierzeck @ 2018-11-29 1:35 UTC (permalink / raw)
To: u-boot
Am 21.11.18 um 09:31 schrieb Trevor Woerner:
> On kernel.org, a newer 8.1.0 version of the toolchains is available. Also,
> update the kernel.org version list to include some of the older versions so
> more architectures can be supported.
>
> Signed-off-by: Trevor Woerner <trevor@toganlabs.com>
> ---
> tools/buildman/toolchain.py | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/tools/buildman/toolchain.py b/tools/buildman/toolchain.py
> index 4b35f400e9..442a7f977a 100644
> --- a/tools/buildman/toolchain.py
> +++ b/tools/buildman/toolchain.py
> @@ -430,7 +430,7 @@ class Toolchains:
> """
> arch = command.OutputOneLine('uname', '-m')
> base = 'https://www.kernel.org/pub/tools/crosstool/files/bin'
> - versions = ['7.3.0', '6.4.0', '4.9.4']
> + versions = ['8.1.0', '7.3.0', '6.4.0', '5.5.0', '4.9.4', '4.9.0', '4.8.5', '4.8.0', '4.7.3', '4.6.3', '4.6.2', '4.6.1', '4.5.1', '4.2.4']
with 4c58d273e0f2cc21821c1de32494abd5de411d9b we removed the older and
unneeded versions. So it doesn't make sense to add them again.
Also did you a test run on Travis CI to verify that all archs still
compile where a 8.1.0 toolchain is available? Travis CI will
automatically use the latest avaiable toolchain for each arch. Possible
new warnings or errors need to be fixed before doing the switch to 8.1.0.
> links = []
> for version in versions:
> url = '%s/%s/%s/' % (base, arch, version)
>
--
- Daniel
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20181129/76f94910/attachment.sig>
^ permalink raw reply [flat|nested] 16+ messages in thread
* [U-Boot] [PATCH 1/3] buildman/toolchain.py: update versions
2018-11-29 1:35 ` Daniel Schwierzeck
@ 2018-11-29 19:03 ` Tom Rini
2018-11-30 11:48 ` Trevor Woerner
0 siblings, 1 reply; 16+ messages in thread
From: Tom Rini @ 2018-11-29 19:03 UTC (permalink / raw)
To: u-boot
On Thu, Nov 29, 2018 at 02:35:56AM +0100, Daniel Schwierzeck wrote:
>
>
> Am 21.11.18 um 09:31 schrieb Trevor Woerner:
> > On kernel.org, a newer 8.1.0 version of the toolchains is available. Also,
> > update the kernel.org version list to include some of the older versions so
> > more architectures can be supported.
> >
> > Signed-off-by: Trevor Woerner <trevor@toganlabs.com>
> > ---
> > tools/buildman/toolchain.py | 2 +-
> > 1 file changed, 1 insertion(+), 1 deletion(-)
> >
> > diff --git a/tools/buildman/toolchain.py b/tools/buildman/toolchain.py
> > index 4b35f400e9..442a7f977a 100644
> > --- a/tools/buildman/toolchain.py
> > +++ b/tools/buildman/toolchain.py
> > @@ -430,7 +430,7 @@ class Toolchains:
> > """
> > arch = command.OutputOneLine('uname', '-m')
> > base = 'https://www.kernel.org/pub/tools/crosstool/files/bin'
> > - versions = ['7.3.0', '6.4.0', '4.9.4']
> > + versions = ['8.1.0', '7.3.0', '6.4.0', '5.5.0', '4.9.4', '4.9.0', '4.8.5', '4.8.0', '4.7.3', '4.6.3', '4.6.2', '4.6.1', '4.5.1', '4.2.4']
>
> with 4c58d273e0f2cc21821c1de32494abd5de411d9b we removed the older and
> unneeded versions. So it doesn't make sense to add them again.
>
> Also did you a test run on Travis CI to verify that all archs still
> compile where a 8.1.0 toolchain is available? Travis CI will
> automatically use the latest avaiable toolchain for each arch. Possible
> new warnings or errors need to be fixed before doing the switch to 8.1.0.
Agreed. Are you using buildman outside of U-Boot where you have a
usecase for adding in old versions / other architectures? I do want to
move up to 8.1.0 but I'm sure we have warnings to fix before we can do
that. I _think_ I even posted a link to what was failing a while ago
and fixed a few things, but there's some harder to solve problems left.
--
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: not available
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20181129/7c3d1b85/attachment.sig>
^ permalink raw reply [flat|nested] 16+ messages in thread
* [U-Boot] [PATCH 1/3] buildman/toolchain.py: update versions
2018-11-29 19:03 ` Tom Rini
@ 2018-11-30 11:48 ` Trevor Woerner
2018-11-30 12:18 ` Tom Rini
2018-12-01 1:41 ` Trevor Woerner
0 siblings, 2 replies; 16+ messages in thread
From: Trevor Woerner @ 2018-11-30 11:48 UTC (permalink / raw)
To: u-boot
On Thu, Nov 29, 2018 at 2:03 PM Tom Rini <trini@konsulko.com> wrote:
> On Thu, Nov 29, 2018 at 02:35:56AM +0100, Daniel Schwierzeck wrote:
> >
> >
> > Am 21.11.18 um 09:31 schrieb Trevor Woerner:
> > > On kernel.org, a newer 8.1.0 version of the toolchains is available.
> Also,
> > > update the kernel.org version list to include some of the older
> versions so
> > > more architectures can be supported.
> > >
> > > Signed-off-by: Trevor Woerner <trevor@toganlabs.com>
> > > ---
> > > tools/buildman/toolchain.py | 2 +-
> > > 1 file changed, 1 insertion(+), 1 deletion(-)
> > >
> > > diff --git a/tools/buildman/toolchain.py b/tools/buildman/toolchain.py
> > > index 4b35f400e9..442a7f977a 100644
> > > --- a/tools/buildman/toolchain.py
> > > +++ b/tools/buildman/toolchain.py
> > > @@ -430,7 +430,7 @@ class Toolchains:
> > > """
> > > arch = command.OutputOneLine('uname', '-m')
> > > base = 'https://www.kernel.org/pub/tools/crosstool/files/bin'
> > > - versions = ['7.3.0', '6.4.0', '4.9.4']
> > > + versions = ['8.1.0', '7.3.0', '6.4.0', '5.5.0', '4.9.4',
> '4.9.0', '4.8.5', '4.8.0', '4.7.3', '4.6.3', '4.6.2', '4.6.1', '4.5.1',
> '4.2.4']
> >
> > with 4c58d273e0f2cc21821c1de32494abd5de411d9b we removed the older and
> > unneeded versions. So it doesn't make sense to add them again.
> >
> > Also did you a test run on Travis CI to verify that all archs still
> > compile where a 8.1.0 toolchain is available? Travis CI will
> > automatically use the latest avaiable toolchain for each arch. Possible
> > new warnings or errors need to be fixed before doing the switch to 8.1.0.
>
> Agreed. Are you using buildman outside of U-Boot where you have a
> usecase for adding in old versions / other architectures?
Yes. I was using it as a general-purpose toolchain-downloading tool.
> I do want to
> move up to 8.1.0 but I'm sure we have warnings to fix before we can do
> that. I _think_ I even posted a link to what was failing a while ago
> and fixed a few things, but there's some harder to solve problems left.
>
Okay, no problem.
In any case I'm partway through doing a test with and without my patches.
I'll post my results for the sake of completeness.
I'm also interested in exploring whether it might be possible to have
buildman download pre-built toolchains from bootlin. Maybe that would be of
some interest?
^ permalink raw reply [flat|nested] 16+ messages in thread
* [U-Boot] [PATCH 1/3] buildman/toolchain.py: update versions
2018-11-30 11:48 ` Trevor Woerner
@ 2018-11-30 12:18 ` Tom Rini
2018-12-01 1:41 ` Trevor Woerner
1 sibling, 0 replies; 16+ messages in thread
From: Tom Rini @ 2018-11-30 12:18 UTC (permalink / raw)
To: u-boot
On Fri, Nov 30, 2018 at 06:48:35AM -0500, Trevor Woerner wrote:
> On Thu, Nov 29, 2018 at 2:03 PM Tom Rini <trini@konsulko.com> wrote:
>
> > On Thu, Nov 29, 2018 at 02:35:56AM +0100, Daniel Schwierzeck wrote:
> > >
> > >
> > > Am 21.11.18 um 09:31 schrieb Trevor Woerner:
> > > > On kernel.org, a newer 8.1.0 version of the toolchains is available.
> > Also,
> > > > update the kernel.org version list to include some of the older
> > versions so
> > > > more architectures can be supported.
> > > >
> > > > Signed-off-by: Trevor Woerner <trevor@toganlabs.com>
> > > > ---
> > > > tools/buildman/toolchain.py | 2 +-
> > > > 1 file changed, 1 insertion(+), 1 deletion(-)
> > > >
> > > > diff --git a/tools/buildman/toolchain.py b/tools/buildman/toolchain.py
> > > > index 4b35f400e9..442a7f977a 100644
> > > > --- a/tools/buildman/toolchain.py
> > > > +++ b/tools/buildman/toolchain.py
> > > > @@ -430,7 +430,7 @@ class Toolchains:
> > > > """
> > > > arch = command.OutputOneLine('uname', '-m')
> > > > base = 'https://www.kernel.org/pub/tools/crosstool/files/bin'
> > > > - versions = ['7.3.0', '6.4.0', '4.9.4']
> > > > + versions = ['8.1.0', '7.3.0', '6.4.0', '5.5.0', '4.9.4',
> > '4.9.0', '4.8.5', '4.8.0', '4.7.3', '4.6.3', '4.6.2', '4.6.1', '4.5.1',
> > '4.2.4']
> > >
> > > with 4c58d273e0f2cc21821c1de32494abd5de411d9b we removed the older and
> > > unneeded versions. So it doesn't make sense to add them again.
> > >
> > > Also did you a test run on Travis CI to verify that all archs still
> > > compile where a 8.1.0 toolchain is available? Travis CI will
> > > automatically use the latest avaiable toolchain for each arch. Possible
> > > new warnings or errors need to be fixed before doing the switch to 8.1.0.
> >
> > Agreed. Are you using buildman outside of U-Boot where you have a
> > usecase for adding in old versions / other architectures?
>
>
> Yes. I was using it as a general-purpose toolchain-downloading tool.
>
>
> > I do want to
> > move up to 8.1.0 but I'm sure we have warnings to fix before we can do
> > that. I _think_ I even posted a link to what was failing a while ago
> > and fixed a few things, but there's some harder to solve problems left.
> >
>
> Okay, no problem.
>
> In any case I'm partway through doing a test with and without my patches.
> I'll post my results for the sake of completeness.
>
> I'm also interested in exploring whether it might be possible to have
> buildman download pre-built toolchains from bootlin. Maybe that would be of
> some interest?
We had thought about that for a bit, for a while, but Arnd is keeping
the kernel.org ones more up to date than Bootlin was, when we had need
of easy access to a specific new version. Things may have changed since
then since it was 1-2 years ago, at this point. But yes, a patch to add
an option to download from elsewhere (perhaps by base URL?) would be
useful I think. I can see cases of pointing buildman at an internal
mirror instead for example.
--
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: not available
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20181130/cec37fb7/attachment.sig>
^ permalink raw reply [flat|nested] 16+ messages in thread
* [U-Boot] [PATCH 1/3] buildman/toolchain.py: update versions
2018-11-30 11:48 ` Trevor Woerner
2018-11-30 12:18 ` Tom Rini
@ 2018-12-01 1:41 ` Trevor Woerner
1 sibling, 0 replies; 16+ messages in thread
From: Trevor Woerner @ 2018-12-01 1:41 UTC (permalink / raw)
To: u-boot
On Fri 2018-11-30 @ 06:48:35 AM, Trevor Woerner wrote:
> In any case I'm partway through doing a test with and without my patches.
> I'll post my results for the sake of completeness.
Using a relatively recent pull, I have the "master" test, and the "twoerner"
test which simply adds my patches on top of "master":
$ git show-branch --sha1-name master contrib/twoerner/buildman
! [master] Merge branch '2018-11-28-master-imports'
* [contrib/twoerner/buildman] buildman/toolchain.py: handle inconsistent tarball names
--
* [11028a5d2a] buildman/toolchain.py: handle inconsistent tarball names
* [ab4e8c6d2c] buildman/toolchain.py: fix toolchain directory
* [e8b8be4d32] buildman/toolchain.py: update versions
-- [e16c888fab] Merge branch '2018-11-28-master-imports'
Cleaning out ~/.buildman and ~/.buildman-toolchains between each test, here
are my results after fetching the toolchains:
$ ./tools/buildman/buildman --fetch-arch all
$ diff -u master/toolchains twoerner/toolchains
--- master/toolchains 2018-11-30 20:32:33.593062110 -0500
+++ twoerner/toolchains 2018-11-30 20:33:20.025307799 -0500
@@ -1,14 +1,17 @@
-List of available toolchains (35):
+List of available toolchains (41):
aarch64
alpha
am33_2.0
arc
arm
+avr32
bfin
c6x
cris
+crisv32
frv
h8300
+hexagon
hppa
hppa64
i386
@@ -18,14 +21,17 @@
microblaze
mips
mips64
+nds32le
nios2
or1k
+or32
powerpc
powerpc64
ppc64le
riscv32
riscv64
s390
+s390x
sh2
sh4
sparc
including version information:
$ diff -u master/toolchains-versions twoerner/toolchains-versions
--- master/toolchains-versions 2018-11-30 20:31:56.328864954 -0500
+++ twoerner/toolchains-versions 2018-11-30 20:31:51.088837231 -0500
@@ -1,36 +1,42 @@
-List of available toolchains (35):
-aarch64 : /home/trevor/.buildman-toolchains/gcc-7.3.0-nolibc/aarch64-linux/bin/aarch64-linux-gcc
-alpha : /home/trevor/.buildman-toolchains/gcc-7.3.0-nolibc/alpha-linux/bin/alpha-linux-gcc
+List of available toolchains (41):
+aarch64 : /home/trevor/.buildman-toolchains/gcc-8.1.0-nolibc/aarch64-linux/bin/aarch64-linux-gcc
+alpha : /home/trevor/.buildman-toolchains/gcc-8.1.0-nolibc/alpha-linux/bin/alpha-linux-gcc
am33_2.0 : /home/trevor/.buildman-toolchains/gcc-7.3.0-nolibc/am33_2.0-linux/bin/am33_2.0-linux-gcc
-arc : /home/trevor/.buildman-toolchains/gcc-7.3.0-nolibc/arc-elf/bin/arc-elf-gcc
-arm : /home/trevor/.buildman-toolchains/gcc-7.3.0-nolibc/arm-linux-gnueabi/bin/arm-linux-gnueabi-gcc
+arc : /home/trevor/.buildman-toolchains/gcc-8.1.0-nolibc/arc-linux/bin/arc-linux-gcc
+arm : /home/trevor/.buildman-toolchains/gcc-8.1.0-nolibc/arm-linux-gnueabi/bin/arm-linux-gnueabi-gcc
+avr32 : /home/trevor/.buildman-toolchains/gcc-4.2.4-nolibc/avr32-linux/bin/avr32-linux-gcc
bfin : /home/trevor/.buildman-toolchains/gcc-7.3.0-nolibc/bfin-uclinux/bin/bfin-uclinux-gcc
-c6x : /home/trevor/.buildman-toolchains/gcc-7.3.0-nolibc/c6x-elf/bin/c6x-elf-gcc
+c6x : /home/trevor/.buildman-toolchains/gcc-8.1.0-nolibc/c6x-elf/bin/c6x-elf-gcc
cris : /home/trevor/.buildman-toolchains/gcc-7.3.0-nolibc/cris-linux/bin/cris-linux-gcc
+crisv32 : /home/trevor/.buildman-toolchains/gcc-4.6.3-nolibc/crisv32-linux/bin/crisv32-linux-gcc
frv : /home/trevor/.buildman-toolchains/gcc-7.3.0-nolibc/frv-linux/bin/frv-linux-gcc
-h8300 : /home/trevor/.buildman-toolchains/gcc-7.3.0-nolibc/h8300-linux/bin/h8300-linux-gcc
-hppa : /home/trevor/.buildman-toolchains/gcc-7.3.0-nolibc/hppa-linux/bin/hppa-linux-gcc
-hppa64 : /home/trevor/.buildman-toolchains/gcc-7.3.0-nolibc/hppa64-linux/bin/hppa64-linux-gcc
-i386 : /home/trevor/.buildman-toolchains/gcc-7.3.0-nolibc/i386-linux/bin/i386-linux-gcc
-ia64 : /home/trevor/.buildman-toolchains/gcc-7.3.0-nolibc/ia64-linux/bin/ia64-linux-gcc
+h8300 : /home/trevor/.buildman-toolchains/gcc-8.1.0-nolibc/h8300-linux/bin/h8300-linux-gcc
+hexagon : /home/trevor/.buildman-toolchains/gcc-4.6.1-nolibc/hexagon-linux/bin/hexagon-linux-gcc
+hppa : /home/trevor/.buildman-toolchains/gcc-8.1.0-nolibc/hppa-linux/bin/hppa-linux-gcc
+hppa64 : /home/trevor/.buildman-toolchains/gcc-8.1.0-nolibc/hppa64-linux/bin/hppa64-linux-gcc
+i386 : /home/trevor/.buildman-toolchains/gcc-8.1.0-nolibc/i386-linux/bin/i386-linux-gcc
+ia64 : /home/trevor/.buildman-toolchains/gcc-8.1.0-nolibc/ia64-linux/bin/ia64-linux-gcc
m32r : /home/trevor/.buildman-toolchains/gcc-7.3.0-nolibc/m32r-linux/bin/m32r-linux-gcc
-m68k : /home/trevor/.buildman-toolchains/gcc-7.3.0-nolibc/m68k-linux/bin/m68k-linux-gcc
-microblaze: /home/trevor/.buildman-toolchains/gcc-7.3.0-nolibc/microblaze-linux/bin/microblaze-linux-gcc
-mips : /home/trevor/.buildman-toolchains/gcc-7.3.0-nolibc/mips-linux/bin/mips-linux-gcc
-mips64 : /home/trevor/.buildman-toolchains/gcc-7.3.0-nolibc/mips64-linux/bin/mips64-linux-gcc
-nios2 : /home/trevor/.buildman-toolchains/gcc-7.3.0-nolibc/nios2-linux/bin/nios2-linux-gcc
+m68k : /home/trevor/.buildman-toolchains/gcc-8.1.0-nolibc/m68k-linux/bin/m68k-linux-gcc
+microblaze: /home/trevor/.buildman-toolchains/gcc-8.1.0-nolibc/microblaze-linux/bin/microblaze-linux-gcc
+mips : /home/trevor/.buildman-toolchains/gcc-8.1.0-nolibc/mips-linux/bin/mips-linux-gcc
+mips64 : /home/trevor/.buildman-toolchains/gcc-8.1.0-nolibc/mips64-linux/bin/mips64-linux-gcc
+nds32le : /home/trevor/.buildman-toolchains/gcc-7.3.0-nolibc/nds32le-elf/bin/nds32le-elf-gcc
+nios2 : /home/trevor/.buildman-toolchains/gcc-8.1.0-nolibc/nios2-linux/bin/nios2-linux-gcc
or1k : /home/trevor/.buildman-toolchains/gcc-7.3.0-nolibc/or1k-linux/bin/or1k-linux-gcc
-powerpc : /home/trevor/.buildman-toolchains/gcc-7.3.0-nolibc/powerpc-linux/bin/powerpc-linux-gcc
-powerpc64 : /home/trevor/.buildman-toolchains/gcc-7.3.0-nolibc/powerpc64-linux/bin/powerpc64-linux-gcc
+or32 : /home/trevor/.buildman-toolchains/gcc-4.5.1-nolibc/or32-linux/bin/or32-linux-gcc
+powerpc : /home/trevor/.buildman-toolchains/gcc-8.1.0-nolibc/powerpc-linux/bin/powerpc-linux-gcc
+powerpc64 : /home/trevor/.buildman-toolchains/gcc-8.1.0-nolibc/powerpc64-linux/bin/powerpc64-linux-gcc
ppc64le : /home/trevor/.buildman-toolchains/gcc-7.3.0-nolibc/ppc64le-linux/bin/ppc64le-linux-gcc
-riscv32 : /home/trevor/.buildman-toolchains/gcc-7.3.0-nolibc/riscv32-linux/bin/riscv32-linux-gcc
-riscv64 : /home/trevor/.buildman-toolchains/gcc-7.3.0-nolibc/riscv64-linux/bin/riscv64-linux-gcc
-s390 : /home/trevor/.buildman-toolchains/gcc-7.3.0-nolibc/s390-linux/bin/s390-linux-gcc
-sh2 : /home/trevor/.buildman-toolchains/gcc-7.3.0-nolibc/sh2-linux/bin/sh2-linux-gcc
-sh4 : /home/trevor/.buildman-toolchains/gcc-7.3.0-nolibc/sh4-linux/bin/sh4-linux-gcc
-sparc : /home/trevor/.buildman-toolchains/gcc-7.3.0-nolibc/sparc-linux/bin/sparc-linux-gcc
-sparc64 : /home/trevor/.buildman-toolchains/gcc-7.3.0-nolibc/sparc64-linux/bin/sparc64-linux-gcc
+riscv32 : /home/trevor/.buildman-toolchains/gcc-8.1.0-nolibc/riscv32-linux/bin/riscv32-linux-gcc
+riscv64 : /home/trevor/.buildman-toolchains/gcc-8.1.0-nolibc/riscv64-linux/bin/riscv64-linux-gcc
+s390 : /home/trevor/.buildman-toolchains/gcc-8.1.0-nolibc/s390-linux/bin/s390-linux-gcc
+s390x : /home/trevor/.buildman-toolchains/gcc-4.9.0-nolibc/s390x-linux/bin/s390x-linux-gcc
+sh2 : /home/trevor/.buildman-toolchains/gcc-8.1.0-nolibc/sh2-linux/bin/sh2-linux-gcc
+sh4 : /home/trevor/.buildman-toolchains/gcc-8.1.0-nolibc/sh4-linux/bin/sh4-linux-gcc
+sparc : /home/trevor/.buildman-toolchains/gcc-8.1.0-nolibc/sparc-linux/bin/sparc-linux-gcc
+sparc64 : /home/trevor/.buildman-toolchains/gcc-8.1.0-nolibc/sparc64-linux/bin/sparc64-linux-gcc
tilegx : /home/trevor/.buildman-toolchains/gcc-7.3.0-nolibc/tilegx-linux/bin/tilegx-linux-gcc
tilepro : /home/trevor/.buildman-toolchains/gcc-7.3.0-nolibc/tilepro-linux/bin/tilepro-linux-gcc
-x86_64 : /home/trevor/.buildman-toolchains/gcc-7.3.0-nolibc/x86_64-linux/bin/x86_64-linux-gcc
-xtensa : /home/trevor/.buildman-toolchains/gcc-7.3.0-nolibc/xtensa-linux/bin/xtensa-linux-gcc
+x86_64 : /home/trevor/.buildman-toolchains/gcc-8.1.0-nolibc/x86_64-linux/bin/x86_64-linux-gcc
+xtensa : /home/trevor/.buildman-toolchains/gcc-8.1.0-nolibc/xtensa-linux/bin/xtensa-linux-gcc
Getting buildman to build the head revision for everything that it can, the
command I used is:
$ ./tools/buildman/buildman --output-dir ~/tmp/u-boot-buildman/master/output-dir 2>&1 | tee ~/tmp/u-boot-buildman/master/build.log
and
$ ./tools/buildman/buildman --output-dir ~/tmp/u-boot-buildman/twoerner/output-dir 2>&1 | tee ~/tmp/u-boot-buildman/twoerner/build.log
Then, for results:
$ ./tools/buildman/buildman --output-dir ~/tmp/u-boot-buildman/master/output-dir -s > ~/tmp/u-boot-buildman/master/build-summary
and
$ ./tools/buildman/buildman --output-dir ~/tmp/u-boot-buildman/twoerner/output-dir -s > ~/tmp/u-boot-buildman/twoerner/build-summary
I editied these result files so that each build target was listed on its own
line (which makes the comparison easier). Comparing between these two builds
yields:
$ diff -u master/build-summary.clean twoerner/build-summary.clean
--- master/build-summary.clean 2018-11-30 20:10:33.590069536 -0500
+++ twoerner/build-summary.clean 2018-11-30 20:09:16.325652908 -0500
@@ -1,5 +1,6 @@
boards.cfg is up to date. Nothing to do.
Summary of current source for 1305 boards (20 threads, 1 job per thread)
+ x86: + efi-x86_app
aarch64: w+ khadas-vim
pine_h64
pinebook
@@ -34,7 +35,13 @@
sopine_baseboard
orangepi_zero_plus2
mvebu_db-88f3720
- arc: w+ hsdk
+ powerpc: w+ MPC8641HPCN_36BIT
+ MPC8610HPCD
+ MPC8641HPCN
+ xpedite517x
+ MCR3000
+ sbc8641d
+ arc: + iot_devkit
nios2: w+ 10m50
mips: w+ imgtec_xilfpga
bcm968380gerg_ram
@@ -65,6 +72,7 @@
mk802_a10s
am43xx_hs_evm
ls1021aqds_nor_lpuart
+ h2200
mx7ulp_evk_plugin
Linksprite_pcDuino
display5
@@ -101,6 +109,7 @@
q8_a23_tablet_800x480
A20-OLinuXino-Lime
imx6qdl_icore_mipi
+ zipitz2
ls1021atwr_nor_lpuart
k2e_hs_evm
ls1021atwr_sdcard_qspi
@@ -139,6 +148,7 @@
Bananapro
Cubieboard4
ls1021aiot_sdcard
+ colibri_pxa270
ls1021aiot_qspi
Yones_Toptech_BS1078_V2
imx6qdl_icore_nand
@@ -272,3 +282,5 @@
q8_a13_tablet
A13-OLinuXinoM
+ openrd_base openrd_client openrd_ultimate
+ nds32: + adp-ag101p
+ adp-ae3xx
So we see 3 who new architectures are added: x86, powerpc, and nds32. And even
with the existing architectures, 3 new targets are built, but arc switches
from "hsdk" to "iot_devkit".
Is this meaningful?
^ permalink raw reply [flat|nested] 16+ messages in thread
* [U-Boot] [PATCH 3/3] buildman/toolchain.py: handle inconsistent tarball names
2018-11-28 19:38 ` Simon Glass
2018-11-28 19:54 ` Trevor Woerner
@ 2018-12-05 23:11 ` sjg at google.com
1 sibling, 0 replies; 16+ messages in thread
From: sjg at google.com @ 2018-12-05 23:11 UTC (permalink / raw)
To: u-boot
On Wed, Nov 28, 2018 at 2:38 PM Simon Glass <sjg@chromium.org> wrote:
> Hi Trevor,
>
> On Tue, 27 Nov 2018 at 17:51, Trevor Woerner <twoerner@gmail.com> wrote:
> >
> > On Mon, Nov 26, 2018 at 8:06 PM Simon Glass <sjg@chromium.org> wrote:
> >>
> >> Reviewed-by: Simon Glass <sjg@chromium.org>
> >> But please use single quotes for strings.
> >
> >
> > I can send a v2 if you like?
>
> If you like, but I can fix it up when applying.
>
>
Yes, please go ahead and fix it up.
Applied to u-boot-dm/master, thanks!
^ permalink raw reply [flat|nested] 16+ messages in thread
end of thread, other threads:[~2018-12-05 23:11 UTC | newest]
Thread overview: 16+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-11-21 8:31 [U-Boot] [PATCH 0/3] buildman updates Trevor Woerner
2018-11-21 8:31 ` [U-Boot] [PATCH 1/3] buildman/toolchain.py: update versions Trevor Woerner
2018-11-27 1:02 ` Simon Glass
2018-11-29 1:35 ` Daniel Schwierzeck
2018-11-29 19:03 ` Tom Rini
2018-11-30 11:48 ` Trevor Woerner
2018-11-30 12:18 ` Tom Rini
2018-12-01 1:41 ` Trevor Woerner
2018-11-21 8:31 ` [U-Boot] [PATCH 2/3] buildman/toolchain.py: fix toolchain directory Trevor Woerner
2018-11-27 1:02 ` Simon Glass
2018-11-21 8:31 ` [U-Boot] [PATCH 3/3] buildman/toolchain.py: handle inconsistent tarball names Trevor Woerner
2018-11-27 1:02 ` Simon Glass
2018-11-28 0:51 ` Trevor Woerner
2018-11-28 19:38 ` Simon Glass
2018-11-28 19:54 ` Trevor Woerner
2018-12-05 23:11 ` sjg at google.com
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.