* [Buildroot] [Bug 10511] Packages get downloaded uncompressed with wget
2017-11-21 14:37 [Buildroot] [Bug 10511] New: Packages get downloaded uncompressed with wget bugzilla at busybox.net
@ 2017-11-21 20:12 ` bugzilla at busybox.net
2017-12-18 14:03 ` bugzilla at busybox.net
` (4 subsequent siblings)
5 siblings, 0 replies; 7+ messages in thread
From: bugzilla at busybox.net @ 2017-11-21 20:12 UTC (permalink / raw)
To: buildroot
https://bugs.busybox.net/show_bug.cgi?id=10511
--- Comment #1 from Thomas Petazzoni <thomas.petazzoni@free-electrons.com> ---
Could you indicate how to reproduce the problem you're seeing with the official
Buildroot, and provide a minimal Buildroot .config / scenario that exhibits the
issue ? Thanks!
--
You are receiving this mail because:
You are on the CC list for the bug.
^ permalink raw reply [flat|nested] 7+ messages in thread
* [Buildroot] [Bug 10511] Packages get downloaded uncompressed with wget
2017-11-21 14:37 [Buildroot] [Bug 10511] New: Packages get downloaded uncompressed with wget bugzilla at busybox.net
2017-11-21 20:12 ` [Buildroot] [Bug 10511] " bugzilla at busybox.net
@ 2017-12-18 14:03 ` bugzilla at busybox.net
2018-01-10 20:57 ` bugzilla at busybox.net
` (3 subsequent siblings)
5 siblings, 0 replies; 7+ messages in thread
From: bugzilla at busybox.net @ 2017-12-18 14:03 UTC (permalink / raw)
To: buildroot
https://bugs.busybox.net/show_bug.cgi?id=10511
--- Comment #2 from Frank Hunleth <fhunleth@troodon-software.com> ---
(In reply to Thomas Petazzoni from comment #1)
This is due to a change in behavior with wget 1.19.2. This discussion describes
the issue: http://lists.gnu.org/archive/html/bug-wget/2017-11/msg00000.html. To
reproduce, I installed wget 1.19.2 on my system and added the following line to
qemu_arm_versatile_defconfig:
BR2_PACKAGE_ERLANG=y
Here's the result of running "make source". You can see the MD5 mismatch from
the erlang.org copy. That's because it was automatically decompressed by wget.
umask 0022 && make -C /home/fhunleth/buildroot.upstream
O=/home/fhunleth/buildroot.upstream/o/wgettest/. source
>>> erlang 20.0 Downloading
--2017-12-18 08:46:07-- http://www.erlang.org/download/otp_src_20.0.tar.gz
Resolving www.erlang.org... 31.172.186.60
Connecting to www.erlang.org|31.172.186.60|:80... connected.
HTTP request sent, awaiting response... 301 Moved Permanently
Location: http://erlang.org/download/otp_src_20.0.tar.gz [following]
--2017-12-18 08:46:08-- http://erlang.org/download/otp_src_20.0.tar.gz
Resolving erlang.org... 192.121.151.106
Connecting to erlang.org|192.121.151.106|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 87346046 (83M) [application/x-tar]
Saving to:
?/home/fhunleth/buildroot.upstream/o/wgettest/build/.otp_src_20.0.tar.gz.5GQ6qy/output?
/home/fhunleth/buildroot.upstream/o/
100%[======================================================================>]
83.30M 582KB/s in 2m 27s
2017-12-18 08:48:35 (582 KB/s) -
?/home/fhunleth/buildroot.upstream/o/wgettest/build/.otp_src_20.0.tar.gz.5GQ6qy/output?
saved [237383680]
ERROR: otp_src_20.0.tar.gz has wrong md5 hash:
ERROR: expected: 2faed2c3519353e6bc2501ed4d8e6ae7
ERROR: got : 6d1210c2a3a9b178a751809016ef51d2
ERROR: Incomplete download, or man-in-the-middle (MITM) attack
--2017-12-18 08:48:35-- http://sources.buildroot.net/otp_src_20.0.tar.gz
Resolving sources.buildroot.net... 176.9.16.109
Connecting to sources.buildroot.net|176.9.16.109|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 87346046 (83M) [application/x-gzip]
Saving to:
?/home/fhunleth/buildroot.upstream/o/wgettest/build/.otp_src_20.0.tar.gz.hY7Oae/output?
/home/fhunleth/buildroot.upstream/o/
100%[======================================================================>]
83.30M 3.56MB/s in 23s
2017-12-18 08:48:59 (3.59 MB/s) -
?/home/fhunleth/buildroot.upstream/o/wgettest/build/.otp_src_20.0.tar.gz.hY7Oae/output?
saved [87346046/87346046]
otp_src_20.0.tar.gz: OK (md5: 2faed2c3519353e6bc2501ed4d8e6ae7)
otp_src_20.0.tar.gz: OK (sha256:
fe80e1e14a2772901be717694bb30ac4e9a07eee0cc7a28988724cbd21476811)
otp_src_20.0.tar.gz: OK (md5: 2faed2c3519353e6bc2501ed4d8e6ae7)
otp_src_20.0.tar.gz: OK (sha256:
fe80e1e14a2772901be717694bb30ac4e9a07eee0cc7a28988724cbd21476811)
Verify that the invalidate MD5 was done on the uncompressed file:
$ gunzip otp_src_20.0.tar.gz
$ md5sum otp_src_20.0.tar
6d1210c2a3a9b178a751809016ef51d2 otp_src_20.0.tar
It seems like a configuration change on erlang.org could fix the problem as
well, but I'm not sure that's the right answer for Buildroot.
--
You are receiving this mail because:
You are on the CC list for the bug.
^ permalink raw reply [flat|nested] 7+ messages in thread
* [Buildroot] [Bug 10511] Packages get downloaded uncompressed with wget
2017-11-21 14:37 [Buildroot] [Bug 10511] New: Packages get downloaded uncompressed with wget bugzilla at busybox.net
2017-11-21 20:12 ` [Buildroot] [Bug 10511] " bugzilla at busybox.net
2017-12-18 14:03 ` bugzilla at busybox.net
@ 2018-01-10 20:57 ` bugzilla at busybox.net
2018-01-10 21:17 ` bugzilla at busybox.net
` (2 subsequent siblings)
5 siblings, 0 replies; 7+ messages in thread
From: bugzilla at busybox.net @ 2018-01-10 20:57 UTC (permalink / raw)
To: buildroot
https://bugs.busybox.net/show_bug.cgi?id=10511
--- Comment #3 from Thomas Petazzoni <thomas.petazzoni@free-electrons.com> ---
Thanks for the investigation. This problem has been fixed/worked-around
upstream by commit
http://git.savannah.gnu.org/cgit/wget.git/commit/?id=047746eb765d212cbf05afac16f4bd73257d8186
which ensures that .gz and .tgz files don't get uncompressed.
However wget 1.19.2 remains affected.
Options:
- Ignore the problem, and leave users of 1.19.2 affected by the problem.
- Bail out if wget 1.19.2 is used, and leave it to the user to use a suitable
wget version
- Detect if wget >= 1.19.2, and pass --compression=none
Thoughts?
--
You are receiving this mail because:
You are on the CC list for the bug.
^ permalink raw reply [flat|nested] 7+ messages in thread
* [Buildroot] [Bug 10511] Packages get downloaded uncompressed with wget
2017-11-21 14:37 [Buildroot] [Bug 10511] New: Packages get downloaded uncompressed with wget bugzilla at busybox.net
` (2 preceding siblings ...)
2018-01-10 20:57 ` bugzilla at busybox.net
@ 2018-01-10 21:17 ` bugzilla at busybox.net
2018-04-02 15:39 ` bugzilla at busybox.net
2018-04-02 16:17 ` bugzilla at busybox.net
5 siblings, 0 replies; 7+ messages in thread
From: bugzilla at busybox.net @ 2018-01-10 21:17 UTC (permalink / raw)
To: buildroot
https://bugs.busybox.net/show_bug.cgi?id=10511
Yann E. MORIN <yann.morin.1998@free.fr> changed:
What |Removed |Added
----------------------------------------------------------------------------
CC| |yann.morin.1998 at free.fr
--- Comment #4 from Yann E. MORIN <yann.morin.1998@free.fr> ---
Thomas, Jean, All,
> - Detect if wget >= 1.19.2, and pass --compression=none
That looks like the best solution IMHO. Unfortunately, wget versions < 1.19.2
do not accept the --compression option, so we really must make it conditional.
My 2ct.
Regards,
Yann E. MORIN.
--
You are receiving this mail because:
You are on the CC list for the bug.
^ permalink raw reply [flat|nested] 7+ messages in thread
* [Buildroot] [Bug 10511] Packages get downloaded uncompressed with wget
2017-11-21 14:37 [Buildroot] [Bug 10511] New: Packages get downloaded uncompressed with wget bugzilla at busybox.net
` (3 preceding siblings ...)
2018-01-10 21:17 ` bugzilla at busybox.net
@ 2018-04-02 15:39 ` bugzilla at busybox.net
2018-04-02 16:17 ` bugzilla at busybox.net
5 siblings, 0 replies; 7+ messages in thread
From: bugzilla at busybox.net @ 2018-04-02 15:39 UTC (permalink / raw)
To: buildroot
https://bugs.busybox.net/show_bug.cgi?id=10511
Thomas Petazzoni <thomas.petazzoni@free-electrons.com> changed:
What |Removed |Added
----------------------------------------------------------------------------
Resolution|--- |WONTFIX
Status|NEW |RESOLVED
--- Comment #5 from Thomas Petazzoni <thomas.petazzoni@free-electrons.com> ---
This problem only affects wget 1.19.2. In 1.19.3, a patch was merged that
disables auto-decompression for files ending in .gz and .tgz, and then in
1.19.4, the auto-decompression was disabled entirely, as it was causing too
many problems.
Since the problem has been fixed in dot-releases of wget, most reasonable
distributions that were using 1.19.2 have probably upgraded to 1.19.3 or later.
In addition, for those stuck with wget 1.19.2, the fallback to
sources.buildroot.net works (even if admittedly the bogus hashes can be scary).
But considering all of this, we don't want to introduce extra complexity in
Buildroot just to workaround a problem affecting only wget 1.19.2 when
downloading files only from HTTP servers having a specific configuration.
--
You are receiving this mail because:
You are on the CC list for the bug.
^ permalink raw reply [flat|nested] 7+ messages in thread
* [Buildroot] [Bug 10511] Packages get downloaded uncompressed with wget
2017-11-21 14:37 [Buildroot] [Bug 10511] New: Packages get downloaded uncompressed with wget bugzilla at busybox.net
` (4 preceding siblings ...)
2018-04-02 15:39 ` bugzilla at busybox.net
@ 2018-04-02 16:17 ` bugzilla at busybox.net
5 siblings, 0 replies; 7+ messages in thread
From: bugzilla at busybox.net @ 2018-04-02 16:17 UTC (permalink / raw)
To: buildroot
https://bugs.busybox.net/show_bug.cgi?id=10511
--- Comment #6 from Jean Parpaillon <jean.parpaillon@free.fr> ---
Thanks for your time !
--
You are receiving this mail because:
You are on the CC list for the bug.
^ permalink raw reply [flat|nested] 7+ messages in thread