[Buildroot] [PATCH 1/2] download/cvs: add a 10 minute timeout

[Buildroot] [PATCH 1/2] download/cvs: add a 10 minute timeout

[Arnout Vandecappelle]

Apparently, CVS servers can be deadlocked and in that case clients will
retry connecting to them indefinitely. Cfr.
http://autobuild.buildroot.net/results/23d/23d1034b33d0354de15de2ec4a8ccd0603e8db78/build-end.log
Apparently, the sf.net CVS server got in such a deadlock on 2018-05-18,
and almost 2 weeks later it is still not fixed.

Instead of just hanging, we should fall back on BR2_SECONDARY_SITE. To
achieve this, it's sufficient to add a timeout to the CVS command.

The timeout value is of course arbitrary. However, we can assume that
nobody will be putting large projects under CVS any more. So if the
download takes more than 5 minutes, it's probably broken. Let's put the
timeout at 10 minutes then.

Fixes:
http://autobuild.buildroot.net/results/db3/db33d4fa507fb3b4132423cd0a7e25a1fe6e4105
http://autobuild.buildroot.net/results/b6d/b6d927dcc73ac8d754422577dacefff4ff918a5c
http://autobuild.buildroot.net/results/23d/23d1034b33d0354de15de2ec4a8ccd0603e8db78
http://autobuild.buildroot.net/results/127/1272a3aa3077e434c9805ec3034f35e6fcc330d4

Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
---
 support/download/cvs | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/support/download/cvs b/support/download/cvs
index 3f77b849e4..9d0dc3cb3a 100755
--- a/support/download/cvs
+++ b/support/download/cvs
@@ -33,9 +33,13 @@ done
 shift $((OPTIND-1)) # Get rid of our options
 
 # Caller needs to single-quote its arguments to prevent them from
-# being expanded a second time (in case there are spaces in them)
+# being expanded a second time (in case there are spaces in them).
+# If the CVS server is deadlocked, the client will never return (cfr.
+# http://autobuild.buildroot.net/results/23d/23d1034b33d0354de15de2ec4a8ccd0603e8db78/build-end.log
+# ). Since nobody sane will put large code bases in CVS, a timeout of
+# 10 minutes should do the trick.
 _cvs() {
-    eval ${CVS} "${@}"
+    eval timeout 10m ${CVS} "${@}"
 }
 
 if [[ ${rev} =~ ^[0-9] ]]; then
-- 
2.17.1

[Buildroot] [PATCH 2/2] package/expect: download tarball instead of cvs

[Arnout Vandecappelle]

More than 3 years after the 5.45.3 release, a tarball for that release
was uploaded to sourceforge.net. The differences between this tarball
and the CVS checkout are minimal:

- There are no CVS directories, of course, but we don't need them.
- File timestamps are different.
- expect.tests is missing, but we don't execute tests anyway.
- configure script is different, but we AUTORECONF anyway.
- 'fixcat' script is missing, but it is not called anywhere.

Since sourceforge.net has broken CVS downloads, now is a good time to
switch away from it.

While we're at it, add a hash file including license.

Fixes:
http://autobuild.buildroot.net/results/db3/db33d4fa507fb3b4132423cd0a7e25a1fe6e4105
http://autobuild.buildroot.net/results/b6d/b6d927dcc73ac8d754422577dacefff4ff918a5c
http://autobuild.buildroot.net/results/23d/23d1034b33d0354de15de2ec4a8ccd0603e8db78
http://autobuild.buildroot.net/results/127/1272a3aa3077e434c9805ec3034f35e6fcc330d4

Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Cc: Fabio Porcedda <fabio.porcedda@gmail.com>
---
Either this one or the previous one is sufficient to fix the issue, of
course. Still, I think both are relevant for master.

Note that there is also a 5.45.4 release from a few months ago. But I
don't know if anyone still uses expect :-).

Note that this was the last internal package to use CVS. So we no
longer have autobuild coverage of the cvs download method.
---
 package/expect/expect.hash | 4 ++++
 package/expect/expect.mk   | 7 ++++---
 2 files changed, 8 insertions(+), 3 deletions(-)
 create mode 100644 package/expect/expect.hash

diff --git a/package/expect/expect.hash b/package/expect/expect.hash
new file mode 100644
index 0000000000..c89f323906
--- /dev/null
+++ b/package/expect/expect.hash
@@ -0,0 +1,4 @@
+# From https://sourceforge.net/projects/expect/files/Expect/5.45.3/expect5.45.3.tar.gz.SHA256
+sha256  c520717b7195944a69ce1492ec82ca0ac3f3baf060804e6c5ee6d505ea512be9  expect5.45.3.tar.gz
+# Locally calculated
+sha256  b2415b17dc8d9a287f4509047ef5ac3436baef7ba7c50faef5222dcdf61a2bab  README
diff --git a/package/expect/expect.mk b/package/expect/expect.mk
index d78ff6c208..ccde33a71b 100644
--- a/package/expect/expect.mk
+++ b/package/expect/expect.mk
@@ -4,12 +4,13 @@
 #
 ################################################################################
 
-# Version 5.45.3
-EXPECT_VERSION = 2014-05-02
-EXPECT_SITE = cvs://expect.cvs.sourceforge.net:/cvsroot/expect
+EXPECT_VERSION = 5.45.3
+EXPECT_SITE = https://sourceforge.net/projects/expect/files/Expect/$(EXPECT_VERSION)
+EXPECT_SOURCE = expect$(EXPECT_VERSION).tar.gz
 EXPECT_LICENSE = Public domain
 EXPECT_LICENSE_FILES = README
 
+# 0001-enable-cross-compilation.patch
 EXPECT_AUTORECONF = YES
 EXPECT_DEPENDENCIES = tcl
 EXPECT_CONF_OPTS = --with-tcl=$(BUILD_DIR)/tcl-$(TCL_VERSION)/unix
-- 
2.17.1

[Buildroot] [PATCH 1/2] download/cvs: add a 10 minute timeout

[Peter Korsgaard]

>>>>> "Arnout" == Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> writes:

 > Apparently, CVS servers can be deadlocked and in that case clients will
 > retry connecting to them indefinitely. Cfr.
 > http://autobuild.buildroot.net/results/23d/23d1034b33d0354de15de2ec4a8ccd0603e8db78/build-end.log
 > Apparently, the sf.net CVS server got in such a deadlock on 2018-05-18,
 > and almost 2 weeks later it is still not fixed.

 > Instead of just hanging, we should fall back on BR2_SECONDARY_SITE. To
 > achieve this, it's sufficient to add a timeout to the CVS command.

 > The timeout value is of course arbitrary. However, we can assume that
 > nobody will be putting large projects under CVS any more.

Tell that to the BSD people ;)

 > So if the download takes more than 5 minutes, it's probably
 > broken. Let's put the timeout at 10 minutes then.

 > Fixes:
 > http://autobuild.buildroot.net/results/db3/db33d4fa507fb3b4132423cd0a7e25a1fe6e4105
 > http://autobuild.buildroot.net/results/b6d/b6d927dcc73ac8d754422577dacefff4ff918a5c
 > http://autobuild.buildroot.net/results/23d/23d1034b33d0354de15de2ec4a8ccd0603e8db78
 > http://autobuild.buildroot.net/results/127/1272a3aa3077e434c9805ec3034f35e6fcc330d4

This is somewhat ugly - But lets indeed keep pragmatic, almost no
projects use CVS anymore.

Committed, thanks.

-- 
Bye, Peter Korsgaard

[Buildroot] [PATCH 2/2] package/expect: download tarball instead of cvs

[Peter Korsgaard]

>>>>> "Arnout" == Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> writes:

 > More than 3 years after the 5.45.3 release, a tarball for that release
 > was uploaded to sourceforge.net. The differences between this tarball
 > and the CVS checkout are minimal:

 > - There are no CVS directories, of course, but we don't need them.
 > - File timestamps are different.
 > - expect.tests is missing, but we don't execute tests anyway.
 > - configure script is different, but we AUTORECONF anyway.
 > - 'fixcat' script is missing, but it is not called anywhere.

 > Since sourceforge.net has broken CVS downloads, now is a good time to
 > switch away from it.

 > While we're at it, add a hash file including license.

 > Fixes:
 > http://autobuild.buildroot.net/results/db3/db33d4fa507fb3b4132423cd0a7e25a1fe6e4105
 > http://autobuild.buildroot.net/results/b6d/b6d927dcc73ac8d754422577dacefff4ff918a5c
 > http://autobuild.buildroot.net/results/23d/23d1034b33d0354de15de2ec4a8ccd0603e8db78
 > http://autobuild.buildroot.net/results/127/1272a3aa3077e434c9805ec3034f35e6fcc330d4

 > Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
 > Cc: Fabio Porcedda <fabio.porcedda@gmail.com>
 > ---
 > Either this one or the previous one is sufficient to fix the issue, of
 > course. Still, I think both are relevant for master.

 > Note that there is also a 5.45.4 release from a few months ago. But I
 > don't know if anyone still uses expect :-).

 > Note that this was the last internal package to use CVS. So we no
 > longer have autobuild coverage of the cvs download method.

True, but with so few upstream projects using CVS that was bound to
happen sooner or later.

Committed, thanks.

-- 
Bye, Peter Korsgaard

[Buildroot] [PATCH 2/2] package/expect: download tarball instead of cvs

[Thomas Petazzoni]

Hello,

On Thu, 31 May 2018 09:05:20 +0200, Arnout Vandecappelle
(Essensium/Mind) wrote:

> diff --git a/package/expect/expect.hash b/package/expect/expect.hash
> new file mode 100644
> index 0000000000..c89f323906
> --- /dev/null
> +++ b/package/expect/expect.hash
> @@ -0,0 +1,4 @@
> +# From https://sourceforge.net/projects/expect/files/Expect/5.45.3/expect5.45.3.tar.gz.SHA256
> +sha256  c520717b7195944a69ce1492ec82ca0ac3f3baf060804e6c5ee6d505ea512be9  expect5.45.3.tar.gz
> +# Locally calculated
> +sha256  b2415b17dc8d9a287f4509047ef5ac3436baef7ba7c50faef5222dcdf61a2bab  README
> diff --git a/package/expect/expect.mk b/package/expect/expect.mk
> index d78ff6c208..ccde33a71b 100644
> --- a/package/expect/expect.mk
> +++ b/package/expect/expect.mk
> @@ -4,12 +4,13 @@
>  #
>  ################################################################################
>  
> -# Version 5.45.3
> -EXPECT_VERSION = 2014-05-02
> -EXPECT_SITE = cvs://expect.cvs.sourceforge.net:/cvsroot/expect
> +EXPECT_VERSION = 5.45.3
> +EXPECT_SITE = https://sourceforge.net/projects/expect/files/Expect/$(EXPECT_VERSION)
> +EXPECT_SOURCE = expect$(EXPECT_VERSION).tar.gz
>  EXPECT_LICENSE = Public domain
>  EXPECT_LICENSE_FILES = README

Romain's autobuiler is not happy with this:

  http://autobuild.buildroot.net/results/dd2/dd29953f28577674070717cbb5feba8e12aad8e3/build-end.log

it downloads some html file. But the same URL downloaded here provides
the correct tarball.

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin (formerly Free Electrons)
Embedded Linux and Kernel engineering
https://bootlin.com

[Buildroot] [PATCH 2/2] package/expect: download tarball instead of cvs

[Romain Naour]

Hi Thomas,

Le 01/06/2018 ? 11:09, Thomas Petazzoni a ?crit?:
> Hello,
> 
> On Thu, 31 May 2018 09:05:20 +0200, Arnout Vandecappelle
> (Essensium/Mind) wrote:
> 
>> diff --git a/package/expect/expect.hash b/package/expect/expect.hash
>> new file mode 100644
>> index 0000000000..c89f323906
>> --- /dev/null
>> +++ b/package/expect/expect.hash
>> @@ -0,0 +1,4 @@
>> +# From https://sourceforge.net/projects/expect/files/Expect/5.45.3/expect5.45.3.tar.gz.SHA256
>> +sha256  c520717b7195944a69ce1492ec82ca0ac3f3baf060804e6c5ee6d505ea512be9  expect5.45.3.tar.gz
>> +# Locally calculated
>> +sha256  b2415b17dc8d9a287f4509047ef5ac3436baef7ba7c50faef5222dcdf61a2bab  README
>> diff --git a/package/expect/expect.mk b/package/expect/expect.mk
>> index d78ff6c208..ccde33a71b 100644
>> --- a/package/expect/expect.mk
>> +++ b/package/expect/expect.mk
>> @@ -4,12 +4,13 @@
>>  #
>>  ################################################################################
>>  
>> -# Version 5.45.3
>> -EXPECT_VERSION = 2014-05-02
>> -EXPECT_SITE = cvs://expect.cvs.sourceforge.net:/cvsroot/expect
>> +EXPECT_VERSION = 5.45.3
>> +EXPECT_SITE = https://sourceforge.net/projects/expect/files/Expect/$(EXPECT_VERSION)
>> +EXPECT_SOURCE = expect$(EXPECT_VERSION).tar.gz
>>  EXPECT_LICENSE = Public domain
>>  EXPECT_LICENSE_FILES = README
> 
> Romain's autobuiler is not happy with this:
> 
>   http://autobuild.buildroot.net/results/dd2/dd29953f28577674070717cbb5feba8e12aad8e3/build-end.log
> 
> it downloads some html file. But the same URL downloaded here provides
> the correct tarball.

$ sha256sum instance-?/dl/expect-*
2b1833202a2371812e9faf67eb0127871b9bd53720e52460552e165534fbde88
instance-1/dl/expect-2014-05-02.tar.gz
48c7a5d2a2081384fb42aaac5eedbec6b5511aa02398611b4d4f6754053990e9
instance-2/dl/expect-2014-05-02.tar.gz

$ ls -lt instance-?/dl/expect-2014-05-02.tar.gz
-rw-r--r--. 2 naourr naourr 635868 25 mai   01:12
instance-1/dl/expect-2014-05-02.tar.gz
-rw-r--r--. 1 naourr naourr 636242 25 avril 11:44
instance-2/dl/expect-2014-05-02.tar.gz

So, it seems Buildroot generated an archive named expect-2014-05-02.tar.gz after
downloading from cvs. Unfortunately, it's the same as the new upstream archive...

Note: tar --version
tar (GNU tar) 1.30

I'll remove the wrong (old) archive.

Best regards,
Romain

> 
> Thomas
> 

[Buildroot] [PATCH 2/2] package/expect: download tarball instead of cvs

[Romain Naour]

Le 01/06/2018 ? 12:14, Romain Naour a ?crit?:
> Hi Thomas,
> 
> Le 01/06/2018 ? 11:09, Thomas Petazzoni a ?crit?:
>> Hello,
>>
>> On Thu, 31 May 2018 09:05:20 +0200, Arnout Vandecappelle
>> (Essensium/Mind) wrote:
>>
>>> diff --git a/package/expect/expect.hash b/package/expect/expect.hash
>>> new file mode 100644
>>> index 0000000000..c89f323906
>>> --- /dev/null
>>> +++ b/package/expect/expect.hash
>>> @@ -0,0 +1,4 @@
>>> +# From https://sourceforge.net/projects/expect/files/Expect/5.45.3/expect5.45.3.tar.gz.SHA256
>>> +sha256  c520717b7195944a69ce1492ec82ca0ac3f3baf060804e6c5ee6d505ea512be9  expect5.45.3.tar.gz
>>> +# Locally calculated
>>> +sha256  b2415b17dc8d9a287f4509047ef5ac3436baef7ba7c50faef5222dcdf61a2bab  README
>>> diff --git a/package/expect/expect.mk b/package/expect/expect.mk
>>> index d78ff6c208..ccde33a71b 100644
>>> --- a/package/expect/expect.mk
>>> +++ b/package/expect/expect.mk
>>> @@ -4,12 +4,13 @@
>>>  #
>>>  ################################################################################
>>>  
>>> -# Version 5.45.3
>>> -EXPECT_VERSION = 2014-05-02
>>> -EXPECT_SITE = cvs://expect.cvs.sourceforge.net:/cvsroot/expect
>>> +EXPECT_VERSION = 5.45.3
>>> +EXPECT_SITE = https://sourceforge.net/projects/expect/files/Expect/$(EXPECT_VERSION)
>>> +EXPECT_SOURCE = expect$(EXPECT_VERSION).tar.gz
>>>  EXPECT_LICENSE = Public domain
>>>  EXPECT_LICENSE_FILES = README
>>
>> Romain's autobuiler is not happy with this:
>>
>>   http://autobuild.buildroot.net/results/dd2/dd29953f28577674070717cbb5feba8e12aad8e3/build-end.log
>>
>> it downloads some html file. But the same URL downloaded here provides
>> the correct tarball.
> 
> $ sha256sum instance-?/dl/expect-*
> 2b1833202a2371812e9faf67eb0127871b9bd53720e52460552e165534fbde88
> instance-1/dl/expect-2014-05-02.tar.gz
> 48c7a5d2a2081384fb42aaac5eedbec6b5511aa02398611b4d4f6754053990e9
> instance-2/dl/expect-2014-05-02.tar.gz
> 
> $ ls -lt instance-?/dl/expect-2014-05-02.tar.gz
> -rw-r--r--. 2 naourr naourr 635868 25 mai   01:12
> instance-1/dl/expect-2014-05-02.tar.gz
> -rw-r--r--. 1 naourr naourr 636242 25 avril 11:44
> instance-2/dl/expect-2014-05-02.tar.gz
> 
> So, it seems Buildroot generated an archive named expect-2014-05-02.tar.gz after
> downloading from cvs. Unfortunately, it's the same as the new upstream archive...

Sorry, the new archive is expect5.45.3.tar.gz

I downloaded this archive using a make source on this machine without any issue.

sha256sum dl/expect/expect5.45.3.tar.gz
c520717b7195944a69ce1492ec82ca0ac3f3baf060804e6c5ee6d505ea512be9
dl/expect/expect5.45.3.tar.gz

It's maybe a sourceforge issue (again).

Best regards,
Romain

> 
> Note: tar --version
> tar (GNU tar) 1.30
> 
> I'll remove the wrong (old) archive.
> 
> Best regards,
> Romain
> 
>>
>> Thomas
>>
> 

[Buildroot] [PATCH 2/2] package/expect: download tarball instead of cvs

[Arnout Vandecappelle]

On 01-06-18 11:09, Thomas Petazzoni wrote:
> Hello,
> 
> On Thu, 31 May 2018 09:05:20 +0200, Arnout Vandecappelle
> (Essensium/Mind) wrote:
> 
>> diff --git a/package/expect/expect.hash b/package/expect/expect.hash
>> new file mode 100644
>> index 0000000000..c89f323906
>> --- /dev/null
>> +++ b/package/expect/expect.hash
>> @@ -0,0 +1,4 @@
>> +# From https://sourceforge.net/projects/expect/files/Expect/5.45.3/expect5.45.3.tar.gz.SHA256
>> +sha256  c520717b7195944a69ce1492ec82ca0ac3f3baf060804e6c5ee6d505ea512be9  expect5.45.3.tar.gz
>> +# Locally calculated
>> +sha256  b2415b17dc8d9a287f4509047ef5ac3436baef7ba7c50faef5222dcdf61a2bab  README
>> diff --git a/package/expect/expect.mk b/package/expect/expect.mk
>> index d78ff6c208..ccde33a71b 100644
>> --- a/package/expect/expect.mk
>> +++ b/package/expect/expect.mk
>> @@ -4,12 +4,13 @@
>>  #
>>  ################################################################################
>>  
>> -# Version 5.45.3
>> -EXPECT_VERSION = 2014-05-02
>> -EXPECT_SITE = cvs://expect.cvs.sourceforge.net:/cvsroot/expect
>> +EXPECT_VERSION = 5.45.3
>> +EXPECT_SITE = https://sourceforge.net/projects/expect/files/Expect/$(EXPECT_VERSION)
>> +EXPECT_SOURCE = expect$(EXPECT_VERSION).tar.gz
>>  EXPECT_LICENSE = Public domain
>>  EXPECT_LICENSE_FILES = README
> 
> Romain's autobuiler is not happy with this:
> 
>   http://autobuild.buildroot.net/results/dd2/dd29953f28577674070717cbb5feba8e12aad8e3/build-end.log
> 
> it downloads some html file. But the same URL downloaded here provides
> the correct tarball.

 Yeah, in Romain's autobuilder it is redirected to

https://sourceforge.net/#!/projects/expect/files/Expect/5.45.3/expect5.45.3.tar.gz

while locally I get

https://sourceforge.net/projects/expect/files/Expect/5.45.3/expect5.45.3.tar.gz/download

which is the right one...


 Regards,
 Arnout


> 
> Thomas
> 

-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF

[Buildroot] [PATCH 2/2] package/expect: download tarball instead of cvs

[Peter Korsgaard]

>>>>> "Arnout" == Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> writes:

 > More than 3 years after the 5.45.3 release, a tarball for that release
 > was uploaded to sourceforge.net. The differences between this tarball
 > and the CVS checkout are minimal:

 > - There are no CVS directories, of course, but we don't need them.
 > - File timestamps are different.
 > - expect.tests is missing, but we don't execute tests anyway.
 > - configure script is different, but we AUTORECONF anyway.
 > - 'fixcat' script is missing, but it is not called anywhere.

 > Since sourceforge.net has broken CVS downloads, now is a good time to
 > switch away from it.

 > While we're at it, add a hash file including license.

 > Fixes:
 > http://autobuild.buildroot.net/results/db3/db33d4fa507fb3b4132423cd0a7e25a1fe6e4105
 > http://autobuild.buildroot.net/results/b6d/b6d927dcc73ac8d754422577dacefff4ff918a5c
 > http://autobuild.buildroot.net/results/23d/23d1034b33d0354de15de2ec4a8ccd0603e8db78
 > http://autobuild.buildroot.net/results/127/1272a3aa3077e434c9805ec3034f35e6fcc330d4

 > Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
 > Cc: Fabio Porcedda <fabio.porcedda@gmail.com>
 > ---
 > Either this one or the previous one is sufficient to fix the issue, of
 > course. Still, I think both are relevant for master.

 > Note that there is also a 5.45.4 release from a few months ago. But I
 > don't know if anyone still uses expect :-).

 > Note that this was the last internal package to use CVS. So we no
 > longer have autobuild coverage of the cvs download method.

Committed to 2018.02.x, thanks.

-- 
Bye, Peter Korsgaard