[PATCH 1/3 v2] sanity.bbclass: Check for the known broken version of make

[PATCH 1/3 v2] sanity.bbclass: Check for the known broken version of make

[Mark Hatle]

See GNU Savannah bug 30612 -- make 3.82 is known to be broken.

A number of vendors are providing a modified version, so checking
for just the version string is not enough.  We also need to check
if the patch for the issue has been applied.  We use a modified
version of the reproduced to check for the issue.

Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
---
 meta/classes/sanity.bbclass | 37 +++++++++++++++++++++++++++++++++++++
 1 file changed, 37 insertions(+)

diff --git a/meta/classes/sanity.bbclass b/meta/classes/sanity.bbclass
index e21cb6b..f2ed91e 100644
--- a/meta/classes/sanity.bbclass
+++ b/meta/classes/sanity.bbclass
@@ -306,6 +306,42 @@ def check_gcc_march(sanity_data):
 
     return result
 
+# Unpatched versions of make 3.82 are known to be broken.  See GNU Savannah Bug 30612.
+# Use a modified reproducer from http://savannah.gnu.org/bugs/?30612 to validate.
+def check_make_version(sanity_data):
+    from distutils.version import LooseVersion
+    status, result = oe.utils.getstatusoutput("make --version")
+    if status != 0:
+        return "Unable to execute make --version, exit code %s\n" % status
+    version = result.split()[2]
+    if LooseVersion(version) == LooseVersion("3.82"):
+        # Construct a test file
+        f = open("makefile_test", "w")
+        f.write("makefile_test.a: makefile_test_a.c makefile_test_b.c makefile_test.a( makefile_test_a.c makefile_test_b.c)\n")
+        f.write("\n")
+        f.write("makefile_test_a.c:\n")
+        f.write("	touch $@\n")
+        f.write("\n")
+        f.write("makefile_test_b.c:\n")
+        f.write("	touch $@\n")
+        f.close()
+
+        # Check if make 3.82 has been patched
+        status,result = oe.utils.getstatusoutput("make -f makefile_test")
+
+        os.remove("makefile_test")
+        if os.path.exists("makefile_test_a.c"):
+            os.remove("makefile_test_a.c")
+        if os.path.exists("makefile_test_b.c"):
+            os.remove("makefile_test_b.c")
+        if os.path.exists("makefile_test.a"):
+            os.remove("makefile_test.a")
+
+        if status != 0:
+            return "Your version of make 3.82 is broken. Please revert to 3.81 or install a patched version.\n"
+    return None
+
+
 # Tar version 1.24 and onwards handle overwriting symlinks correctly
 # but earlier versions do not; this needs to work properly for sstate
 def check_tar_version(sanity_data):
@@ -436,6 +472,7 @@ def check_sanity_version_change(status, d):
     except ImportError:
         status.addresult('Your python is not a full install. Please install the module xml.parsers.expat (python-xml on openSUSE and SUSE Linux).\n')
 
+    status.addresult(check_make_version(d))
     status.addresult(check_tar_version(d))
     status.addresult(check_git_version(d))
 
-- 
1.8.3.1

Re: [PATCH 1/3 v2] sanity.bbclass: Check for the known broken version of make

[Burton, Ross]

On 27 June 2013 15:08, Mark Hatle <mark.hatle@windriver.com> wrote:
> See GNU Savannah bug 30612 -- make 3.82 is known to be broken.
>
> A number of vendors are providing a modified version, so checking
> for just the version string is not enough.  We also need to check
> if the patch for the issue has been applied.  We use a modified
> version of the reproduced to check for the issue.
>
> Signed-off-by: Mark Hatle <mark.hatle@windriver.com>

Verified that this detects the 3.82 in Debian's experimental
repository that is well and truly broken.

Signed-off-by: Ross Burton <ross.burton@intel.com>

Ross

Re: [PATCH 1/3 v2] sanity.bbclass: Check for the known broken version of make

[Paul Barker]

On 27 June 2013 15:21, Burton, Ross <ross.burton@intel.com> wrote:
> On 27 June 2013 15:08, Mark Hatle <mark.hatle@windriver.com> wrote:
>> See GNU Savannah bug 30612 -- make 3.82 is known to be broken.
>>
>> A number of vendors are providing a modified version, so checking
>> for just the version string is not enough.  We also need to check
>> if the patch for the issue has been applied.  We use a modified
>> version of the reproduced to check for the issue.
>>
>> Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
>
> Verified that this detects the 3.82 in Debian's experimental
> repository that is well and truly broken.
>
> Signed-off-by: Ross Burton <ross.burton@intel.com>
>

This picks out the current version of make in Arch Linux as broken
despite it having a patch for bug #30612. I'm not sure if this is
intended, could be that the applied patch isn't a complete fix. I
don't have time to dig into this, just thought it'd be useful to you
to know.

If you know what fix you're looking for, the applied patches and build
script can be seen at
https://projects.archlinux.org/svntogit/packages.git/tree/trunk?h=packages/make

Thanks,

--
Paul Barker

Email: paul@paulbarker.me.uk
http://www.paulbarker.me.uk

Re: [PATCH 1/3 v2] sanity.bbclass: Check for the known broken version of make

[Mark Hatle]

On 6/28/13 5:20 AM, Paul Barker wrote:
> On 27 June 2013 15:21, Burton, Ross <ross.burton@intel.com> wrote:
>> On 27 June 2013 15:08, Mark Hatle <mark.hatle@windriver.com> wrote:
>>> See GNU Savannah bug 30612 -- make 3.82 is known to be broken.
>>>
>>> A number of vendors are providing a modified version, so checking
>>> for just the version string is not enough.  We also need to check
>>> if the patch for the issue has been applied.  We use a modified
>>> version of the reproduced to check for the issue.
>>>
>>> Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
>>
>> Verified that this detects the 3.82 in Debian's experimental
>> repository that is well and truly broken.
>>
>> Signed-off-by: Ross Burton <ross.burton@intel.com>
>>
>
> This picks out the current version of make in Arch Linux as broken
> despite it having a patch for bug #30612. I'm not sure if this is
> intended, could be that the applied patch isn't a complete fix. I
> don't have time to dig into this, just thought it'd be useful to you
> to know.

This is exactly why the check is being done.  From what I found there are a 
number of make versions on various distros that only have partial fixes for bug 
30612.

For the RedHat/Fedora bug I filed, see:
  https://bugzilla.redhat.com/show_bug.cgi?id=975597

The test case, and a patch to the missing commit are listed.

 
http://git.savannah.gnu.org/cgit/make.git/commit/?id=b06b8c64a29a5ba3a8daecd829fa2f98d42cb285

> If you know what fix you're looking for, the applied patches and build
> script can be seen at
> https://projects.archlinux.org/svntogit/packages.git/tree/trunk?h=packages/make

Looking at that list, it appears the commit referenced above is indeed the one 
missing.

--Mark

> Thanks,
>
> --
> Paul Barker
>
> Email: paul@paulbarker.me.uk
> http://www.paulbarker.me.uk
>

Re: [PATCH 1/3 v2] sanity.bbclass: Check for the known broken version of make

[Paul Barker]

[-- Attachment #1: Type: text/plain, Size: 1359 bytes --]

On 28 Jun 2013 14:17, "Mark Hatle" <mark.hatle@windriver.com> wrote:
>
> On 6/28/13 5:20 AM, Paul Barker wrote:
>>
>> This picks out the current version of make in Arch Linux as broken
>> despite it having a patch for bug #30612. I'm not sure if this is
>> intended, could be that the applied patch isn't a complete fix. I
>> don't have time to dig into this, just thought it'd be useful to you
>> to know.
>
>
> This is exactly why the check is being done.  From what I found there are
a number of make versions on various distros that only have partial fixes
for bug 30612.
>
> For the RedHat/Fedora bug I filed, see:
>  https://bugzilla.redhat.com/show_bug.cgi?id=975597
>
> The test case, and a patch to the missing commit are listed.
>
>
>
http://git.savannah.gnu.org/cgit/make.git/commit/?id=b06b8c64a29a5ba3a8daecd829fa2f98d42cb285
>
>
>> If you know what fix you're looking for, the applied patches and build
>> script can be seen at
>>
https://projects.archlinux.org/svntogit/packages.git/tree/trunk?h=packages/make
>
>
> Looking at that list, it appears the commit referenced above is indeed
the one missing.
>

I'll email the relevant people within arch linux later. Most of my
important builds are done on Ubuntu server 12.04 anyway.

Paul Barker

Email: paul@paulbarker.me.uk
http://www.paulbarker.me.uk

[-- Attachment #2: Type: text/html, Size: 2086 bytes --]

Re: [PATCH 1/3 v2] sanity.bbclass: Check for the known broken version of make

[Paul Barker]

On 28 June 2013 16:11, Paul Barker <paul@paulbarker.me.uk> wrote:
>
> On 28 Jun 2013 14:17, "Mark Hatle" <mark.hatle@windriver.com> wrote:
>>
>> On 6/28/13 5:20 AM, Paul Barker wrote:
>>> If you know what fix you're looking for, the applied patches and build
>>> script can be seen at
>>>
>>> https://projects.archlinux.org/svntogit/packages.git/tree/trunk?h=packages/make
>>
>>
>> Looking at that list, it appears the commit referenced above is indeed the
>> one missing.
>>
>
> I'll email the relevant people within arch linux later. Most of my important
> builds are done on Ubuntu server 12.04 anyway.
>

Bug filed: https://bugs.archlinux.org/task/35968

--
Paul Barker

Email: paul@paulbarker.me.uk
http://www.paulbarker.me.uk

Re: [PATCH 1/3 v2] sanity.bbclass: Check for the known broken version of make

[Trevor Woerner]

On 27 June 2013 10:08, Mark Hatle <mark.hatle@windriver.com> wrote:
> See GNU Savannah bug 30612 -- make 3.82 is known to be broken.
>
> A number of vendors are providing a modified version, so checking
> for just the version string is not enough.  We also need to check
> if the patch for the issue has been applied.  We use a modified
> version of the reproduced to check for the issue.
>
> Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
> ---
> +
> +        if status != 0:
> +            return "Your version of make 3.82 is broken. Please revert to 3.81 or install a patched version.\n"


Instead of returning an error and asking the user to manually update
their own 'make', wouldn't it be better if bitbake simply built its
own known-to-be-working -native version instead? In this way a good,
working version of 'make' could be installed in a potential SDK's
sysroot as well?

Re: [PATCH 1/3 v2] sanity.bbclass: Check for the known broken version of make

[Philip Balister]

On 06/30/2013 11:56 AM, Trevor Woerner wrote:
> On 27 June 2013 10:08, Mark Hatle <mark.hatle@windriver.com> wrote:
>> See GNU Savannah bug 30612 -- make 3.82 is known to be broken.
>>
>> A number of vendors are providing a modified version, so checking
>> for just the version string is not enough.  We also need to check
>> if the patch for the issue has been applied.  We use a modified
>> version of the reproduced to check for the issue.
>>
>> Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
>> ---
>> +
>> +        if status != 0:
>> +            return "Your version of make 3.82 is broken. Please revert to 3.81 or install a patched version.\n"
> 
> 
> Instead of returning an error and asking the user to manually update
> their own 'make', wouldn't it be better if bitbake simply built its
> own known-to-be-working -native version instead? In this way a good,
> working version of 'make' could be installed in a potential SDK's
> sysroot as well?

Is the broken version good enough to build a working version?

Philip


> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core
> 
> 

Re: [PATCH 1/3 v2] sanity.bbclass: Check for the known broken version of make

[Trevor Woerner]

On 30 June 2013 12:02, Philip Balister <philip@balister.org> wrote:
> On 06/30/2013 11:56 AM, Trevor Woerner wrote:
>> On 27 June 2013 10:08, Mark Hatle <mark.hatle@windriver.com> wrote:
>>> See GNU Savannah bug 30612 -- make 3.82 is known to be broken.
>>>
>>> A number of vendors are providing a modified version, so checking
>>> for just the version string is not enough.  We also need to check
>>> if the patch for the issue has been applied.  We use a modified
>>> version of the reproduced to check for the issue.
>>>
>>> Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
>>> ---
>>> +
>>> +        if status != 0:
>>> +            return "Your version of make 3.82 is broken. Please revert to 3.81 or install a patched version.\n"
>>
>>
>> Instead of returning an error and asking the user to manually update
>> their own 'make', wouldn't it be better if bitbake simply built its
>> own known-to-be-working -native version instead? In this way a good,
>> working version of 'make' could be installed in a potential SDK's
>> sysroot as well?
>
> Is the broken version good enough to build a working version?


In my case that's what I did (i.e. I used make-3.82 to build and
install early in my $PATH make-3.81). I was then able to bitbake again
which went to completion successfully.

Re: [PATCH 1/3 v2] sanity.bbclass: Check for the known broken version of make

[Mark Hatle]

On 6/30/13 11:02 AM, Philip Balister wrote:
> On 06/30/2013 11:56 AM, Trevor Woerner wrote:
>> On 27 June 2013 10:08, Mark Hatle <mark.hatle@windriver.com> wrote:
>>> See GNU Savannah bug 30612 -- make 3.82 is known to be broken.
>>>
>>> A number of vendors are providing a modified version, so checking
>>> for just the version string is not enough.  We also need to check
>>> if the patch for the issue has been applied.  We use a modified
>>> version of the reproduced to check for the issue.
>>>
>>> Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
>>> ---
>>> +
>>> +        if status != 0:
>>> +            return "Your version of make 3.82 is broken. Please revert to 3.81 or install a patched version.\n"
>>
>>
>> Instead of returning an error and asking the user to manually update
>> their own 'make', wouldn't it be better if bitbake simply built its
>> own known-to-be-working -native version instead? In this way a good,
>> working version of 'make' could be installed in a potential SDK's
>> sysroot as well?
>
> Is the broken version good enough to build a working version?

(Catching up on email after my vacation...)

That is the primary issue, can we even trust the host system to work well enough 
to build a working version.

Second, there was recently a change to oe-core that got rid of all of the 
"bootstrap" builds, in favor of the 'buildtools'.  For anyone with an old or 
broken system, they will need to download (or build) the buildtools.. install it 
and have it in their path prior to running oe-core.

This will provide the basic set of python, tar, git, and make that is needed for 
the build system.

On a "working" host, you can use bitbake buildtools-tarball to generate it.

--Mark

> Philip
>
>
>> _______________________________________________
>> Openembedded-core mailing list
>> Openembedded-core@lists.openembedded.org
>> http://lists.openembedded.org/mailman/listinfo/openembedded-core
>>
>>

Re: [PATCH 1/3 v2] sanity.bbclass: Check for the known broken version of make

[Trevor Woerner]

On 8 July 2013 17:55, Mark Hatle <mark.hatle@windriver.com> wrote:
> For anyone with an old or
> broken system, they will need to download (or build) the buildtools..
> install it and have it in their path prior to running oe-core.


Does a download location already exist with downloads available for
people who receive this error when building? If so, could it be made
part of the error message? Or maybe instead of asking people to use a
working version of "make" the error message could instruct them to use
one of these buildtools-tarball SDKs (and provide instructions for
doing so)?

Re: [PATCH 1/3 v2] sanity.bbclass: Check for the known broken version of make

[Paul Eggleton]

On Tuesday 09 July 2013 16:14:55 Trevor Woerner wrote:
> On 8 July 2013 17:55, Mark Hatle <mark.hatle@windriver.com> wrote:
> > For anyone with an old or broken system, they will need to download (or
> > build) the buildtools..
> > install it and have it in their path prior to running oe-core.
> 
> Does a download location already exist with downloads available for
> people who receive this error when building? If so, could it be made
> part of the error message? Or maybe instead of asking people to use a
> working version of "make" the error message could instruct them to use
> one of these buildtools-tarball SDKs (and provide instructions for
> doing so)?

According to Beth, the current (nightly build) can be found here:

 http://autobuilder.yoctoproject.org/pub/nightly/CURRENT/buildtools/

Location for the upcoming 1.5 release when that is available:

 http://downloads.yoctoproject.org/releases/yocto/yocto-1.5/buildtools/

The tricky part is we don't have a solid location yet, only one being built 
from master. Perhaps we could link to an intermediary page with explanatory 
text that can link to the appropriate location.

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre

Re: [PATCH 1/3 v2] sanity.bbclass: Check for the known broken version of make

[Trevor Woerner]

On 19 July 2013 05:41, Paul Eggleton <paul.eggleton@linux.intel.com> wrote:
> On Tuesday 09 July 2013 16:14:55 Trevor Woerner wrote:
>> On 8 July 2013 17:55, Mark Hatle <mark.hatle@windriver.com> wrote:
>> > For anyone with an old or broken system, they will need to download (or
>> > build) the buildtools..
>> > install it and have it in their path prior to running oe-core.
>>
>> Does a download location already exist with downloads available for
>> people who receive this error when building? If so, could it be made
>> part of the error message? Or maybe instead of asking people to use a
>> working version of "make" the error message could instruct them to use
>> one of these buildtools-tarball SDKs (and provide instructions for
>> doing so)?
>
> According to Beth, the current (nightly build) can be found here:
>
>  http://autobuilder.yoctoproject.org/pub/nightly/CURRENT/buildtools/
>
> Location for the upcoming 1.5 release when that is available:
>
>  http://downloads.yoctoproject.org/releases/yocto/yocto-1.5/buildtools/
>
> The tricky part is we don't have a solid location yet, only one being built
> from master. Perhaps we could link to an intermediary page with explanatory
> text that can link to the appropriate location.


That would be quite helpful, and maybe even help people to help
themselves should they come across this problem when trying to build.