From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0B552C3527D for ; Thu, 14 Apr 2022 16:03:53 +0000 (UTC) Received: from mail-lf1-f45.google.com (mail-lf1-f45.google.com [209.85.167.45]) by mx.groups.io with SMTP id smtpd.web09.1547.1649871708645490313 for ; Wed, 13 Apr 2022 10:41:49 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@gmail.com header.s=20210112 header.b=DAxBSRrO; spf=pass (domain: gmail.com, ip: 209.85.167.45, mailfrom: quaresma.jose@gmail.com) Received: by mail-lf1-f45.google.com with SMTP id u7so4835639lfs.8 for ; Wed, 13 Apr 2022 10:41:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=wdci/4NElYzwAdzZmDrZqkBCJNp/3d3nan+a3Mlq0O4=; b=DAxBSRrOJHgLqu/FtCxSBYD/+WkoSR/84fbVIHkuNxiIdNP267pUTo+SPjnjgwzYdq RGS3OYGOIdIxBZ0XMiR/ll9UbawpB6OE0G8j2eTCnRofNWa5zRWLGHlDRKg0uF6PLdgm 6KMdrzUMPgWSS+F9DMkJriwjjwAYnIOS9tsBBx2tqpmEd2hMXJzqsOs7II0ZAys0lLPo nC4/5VE38OEKCdzyUNwANGlhGHWIJuX84Vkngc/IbgoD6rSimEI0akfSCuO2hGgsSpKR y4TKeAMvfpEqy+LzFgk2FhA2Ctx7k5nc6FwQR/n7NfHyvgPi1rSGm2qqVJvH9adsK7gz cuhQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=wdci/4NElYzwAdzZmDrZqkBCJNp/3d3nan+a3Mlq0O4=; b=FndlzZ4dDXFLtnLYjXG48+1Tj5EXHgrQiBOwRqQsnr1+ZB8Wr14bAzwobp5I66XSRw UcI+vG5b1HGYmtL7JRS+lvZ8I7y63jJ50kgyi2PS4qJ4IhfSNl4h6UqomBykD03cl+Z1 wIo/n/HImCAukrKU5IqnCiyYkr1+7mv7eQObNMnAtuHhZtJlwHyPsE5TnXFdNNyFRD+I dMW60CSIidPAdWYpTjc2klOcJ12bmvQM5u9NJJgsalveS2xjnILtRhcNNUIn1R4+xKrf B/+Rgpu29arkpf456CjzHJpObjtWNTaVKE41mrswEAtpVNkvUU7xym3gSgVVhd40O1bI 495g== X-Gm-Message-State: AOAM533IT+z/WiIH8y12VSjesIfVBuCjetOj2aEgC997oDnX3EtepR4p n+8mIaJNAwOtQXUnfEqylJObdis/I2es6xsmGew= X-Google-Smtp-Source: ABdhPJxK3TO30FfcoZnqKTAZHYYKmWSPGaoIteffT/6RTV5z+HxJYG4Fsl5TJTIYD290ruvVDbK05MuE+bC6544/zAs= X-Received: by 2002:a05:6512:753:b0:46b:b5e6:1621 with SMTP id c19-20020a056512075300b0046bb5e61621mr7449565lfs.94.1649871706546; Wed, 13 Apr 2022 10:41:46 -0700 (PDT) MIME-Version: 1.0 References: <20220329130741.2430737-1-ross.burton@arm.com> <16E57E79FD292EFA.13992@lists.openembedded.org> <16E583EB139C493B.16998@lists.openembedded.org> In-Reply-To: <16E583EB139C493B.16998@lists.openembedded.org> From: Jose Quaresma Date: Wed, 13 Apr 2022 18:41:34 +0100 Message-ID: Subject: Re: [OE-core] [PATCH][dunfell] zlib: backport the fix for CVE-2018-25032 To: Jose Quaresma Cc: Steve Sakoman , Ralph Siemsen , Ross Burton , "Mittal, Anuj" , Patches and discussions about the oe-core layer Content-Type: multipart/alternative; boundary="000000000000fe670d05dc8cb046" List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Thu, 14 Apr 2022 16:03:53 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/164350 --000000000000fe670d05dc8cb046 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Jose Quaresma via lists.openembedded.org escreveu no dia quarta, 13/04/2022 =C3=A0= (s) 18:11: > Hi, > > Steve Sakoman escreveu no dia quarta, 13/04/2022 =C3= =A0(s) > 17:02: > >> On Wed, Apr 13, 2022 at 5:31 AM Steve Sakoman via >> lists.openembedded.org >> wrote: >> > >> > On Tue, Apr 12, 2022 at 3:21 PM Ralph Siemsen >> wrote: >> > > >> > > On Tue, Apr 12, 2022 at 5:49 PM Steve Sakoman >> wrote: >> > > >> > > > I added a debug option to the failing command and did another >> autobuilder run. >> > > > >> > > > You can see the output here: >> > > > >> > > > https://errors.yoctoproject.org/Errors/Details/654608/ >> > > >> > > Okay, same error, "Hash Sum mismatch". And if I squint between all t= he >> > > URL-encoding, I can see the md5/sha1/sha256/sha512sum values. >> > > >> > > The "apt update" command is doing the following: >> > > - fetch the file called "Release" >> > > - fetch the file called "Packages.gz" --> error occurs here >> > > >> > > Looking inside the Release file, it is plain text, and contains the >> > > md5/sha1/sha256/sha512 sums of both Packages and Packages.gz (and al= so >> > > the first two lines of Release). >> > > >> > > Manually checking each of those sums reveals an inconsistency: all t= he >> > > sha256 values inside Release are incorrect, while all the other >> > > md1/sha1/sha512 values are correct. >> > > >> > > And when we look at the URL-encoded debug info... the sha256 value i= s >> > > the correct one for Packages.gz (as computed manually). However it >> > > does not match the (incorrect) value within the Release file. Thus i= t >> > > seems apt-get is justified when it complains about "Hash Sum >> > > mismatch". >> > > >> > > Going back to my Ubuntu system, and looking at the generated Release >> > > file... all the checksums are correct, including the sha256sum. >> > > >> > > So I am now looking into how Release file gets generated... as the >> > > problem appears to be there... and it happens on Fedora but not >> > > Ubuntu. >> > >> > As far as I can tell it is done here: >> > >> > >> https://git.yoctoproject.org/poky/tree/meta/lib/oe/package_manager.py?h= =3Ddunfell#n301 >> > >> > > One additional point to add: on the same Fedora 35 system, I did a >> > > full rebuild *without* with xz/gzip CVE fixes, and the apt failure >> > > still occurs. To be certain, I nuked cache, sstate-cache and tmp (so >> > > basically the entire build directory) and the rebuild took several >> > > hours. >> > >> > Now that is really strange! In my experience it has only appeared >> > after adding the zlib or xz CVE fix patches. >> > >> > I just started two runs on the autobuilder, with the zlib patch as the >> > only difference. Both on Fedora 35. >> >> Both runs completed and I'm still seeing success without the zlib patch: >> >> https://autobuilder.yoctoproject.org/typhoon/#/builders/50/builds/5069 >> >> and failure with the patch: >> >> https://autobuilder.yoctoproject.org/typhoon/#/builders/50/builds/5070 > > > It seems the test that failed is something related with the apt. > Is this repo hosted on 192.168.7.5 shared between master and dunfell > branches? > I ask this because there are some issues with apt [1] on master and it ca= n > be related to this. > The server is started in the test. Sorry for the noise and please discard my comment. Started HTTPService on 0.0.0.0:42261 Jose > > [1] apt: add apt selftest to test signed package feeds) > > Started HTTPService on 0.0.0.0:35637 > Traceback (most recent call last): > File > "/home/pokybuild/yocto-worker/pkgman-deb-non-deb/build/meta/lib/oeqa/core= /decorator/__init__.py", > line 36, in wrapped_f > return func(*args, **kwargs) > File > "/home/pokybuild/yocto-worker/pkgman-deb-non-deb/build/meta/lib/oeqa/core= /decorator/__init__.py", > line 36, in wrapped_f > return func(*args, **kwargs) > File > "/home/pokybuild/yocto-worker/pkgman-deb-non-deb/build/meta/lib/oeqa/core= /decorator/__init__.py", > line 36, in wrapped_f > return func(*args, **kwargs) > File > "/home/pokybuild/yocto-worker/pkgman-deb-non-deb/build/meta/lib/oeqa/runt= ime/cases/apt.py", > line 50, in test_apt_install_from_repo > self.pkg('update') > File > "/home/pokybuild/yocto-worker/pkgman-deb-non-deb/build/meta/lib/oeqa/runt= ime/cases/apt.py", > line 17, in pkg > self.assertEqual(status, expected, message) > AssertionError: 100 !=3D 0 : apt-get update > Ign:1 http://192.168.7.5:42261 ./ InRelease > Get:2 http://192.168.7.5:42261 ./ Release [1213 B] > Ign:3 http://192.168.7.5:42261 ./ Release.gpg > Get:4 http://192.168.7.5:42261 ./ Packages [59.3 kB] > Err:4 http://192.168.7.5:42261 ./ Packages > Hash Sum mismatch > Fetched 60.5 kB in 20s (3020 B/s) > Reading package lists... > W: The repository 'http://192.168.7.5:42261 ./ Release' is not signed. > E: Failed to fetch http://192.168.7.5:42261/./Packages.gz Hash Sum > mismatch > E: Some index files failed to download. They have been ignored, or old > ones used instead. > > Jose > > >> >> Steve >> >> >> >> > > -- > Best regards, > > Jos=C3=A9 Quaresma > > -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D- > Links: You receive all messages sent to this group. > View/Reply Online (#164347): > https://lists.openembedded.org/g/openembedded-core/message/164347 > Mute This Topic: https://lists.openembedded.org/mt/90107518/5052612 > Group Owner: openembedded-core+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [ > quaresma.jose@gmail.com] > -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D- > > --=20 Best regards, Jos=C3=A9 Quaresma --000000000000fe670d05dc8cb046 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


=
Jose Quaresma via lists.openembedded.org <quaresma.jose=3Dgmail.com@lists.openembedded= .org> escreveu no dia quarta, 13/04/2022 =C3=A0(s) 18:11:
<= blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-l= eft:1px solid rgb(204,204,204);padding-left:1ex">
Hi,

Steve Sakoman <steve@sakoman.com> escreveu no dia quarta, 13/04/2022 = =C3=A0(s) 17:02:
On Wed, Apr 13, 2022 at 5:31 AM Steve Sakoman via
lists.openembedded.org <steve=3Dsakoman.com@lists.openembedded.org<= /a>>
wrote:
>
> On Tue, Apr 12, 2022 at 3:21 PM Ralph Siemsen <ralph.siemsen@linaro.org> = wrote:
> >
> > On Tue, Apr 12, 2022 at 5:49 PM Steve Sakoman <steve@sakoman.com> wrote: > >
> > > I added a debug option to the failing command and did anothe= r autobuilder run.
> > >
> > > You can see the output here:
> > >
> > > https://errors.yoctoproject.org= /Errors/Details/654608/
> >
> > Okay, same error, "Hash Sum mismatch". And if I squint = between all the
> > URL-encoding, I can see the md5/sha1/sha256/sha512sum values.
> >
> > The "apt update" command is doing the following:
> > - fetch the file called "Release"
> > - fetch the file called "Packages.gz" --> error occu= rs here
> >
> > Looking inside the Release file, it is plain text, and contains t= he
> > md5/sha1/sha256/sha512 sums of both Packages and Packages.gz (and= also
> > the first two lines of Release).
> >
> > Manually checking each of those sums reveals an inconsistency: al= l the
> > sha256 values inside Release are incorrect, while all the other > > md1/sha1/sha512 values are correct.
> >
> > And when we look at the URL-encoded debug info... the sha256 valu= e is
> > the correct one for Packages.gz (as computed manually). However i= t
> > does not match the (incorrect) value within the Release file. Thu= s it
> > seems apt-get is justified when it complains about "Hash Sum=
> > mismatch".
> >
> > Going back to my Ubuntu system, and looking at the generated Rele= ase
> > file... all the checksums are correct, including the sha256sum. > >
> > So I am now looking into how Release file gets generated... as th= e
> > problem appears to be there... and it happens on Fedora but not > > Ubuntu.
>
> As far as I can tell it is done here:
>
> https://g= it.yoctoproject.org/poky/tree/meta/lib/oe/package_manager.py?h=3Ddunfell#n3= 01
>
> > One additional point to add: on the same Fedora 35 system, I did = a
> > full rebuild *without* with xz/gzip CVE fixes, and the apt failur= e
> > still occurs. To be certain, I nuked cache, sstate-cache and tmp = (so
> > basically the entire build directory) and the rebuild took severa= l
> > hours.
>
> Now that is really strange!=C2=A0 In my experience it has only appeare= d
> after adding the zlib or xz CVE fix patches.
>
> I just started two runs on the autobuilder, with the zlib patch as the=
> only difference.=C2=A0 Both on Fedora 35.

Both runs completed and I'm still seeing success without the zlib patch= :

https://autobuilder.yoctoproje= ct.org/typhoon/#/builders/50/builds/5069

and failure with the patch:

https://autobuilder.yoctoproje= ct.org/typhoon/#/builders/50/builds/5070

It seems the test that failed is something related with the apt.=C2=A0
Is this repo hosted on 192.168.7.5 shared between=C2=A0master and d= unfell branches?
I ask this because there are some issues with ap= t [1] on master and it can be related to this.

The server is started in the test.
Sorry fo= r the noise and please discard my comment.

Started HTTP= Service on 0.0.0.0:42261

Jose
=C2=A0=

[1] apt: add=C2=A0apt=C2=A0sel= ftest to test signed package feeds)

Started HTTPService on 0.0.0.0:35637
Traceback (most recent = call last):
File "/home/pokybuild/yocto-worker/pkgman-deb-non-deb/bui= ld/meta/lib/oeqa/core/decorator/__init__.py", line 36, in wrapped_f
= return func(*args, **kwargs)
File "/home/pokybuild/yocto-worker/pkgm= an-deb-non-deb/build/meta/lib/oeqa/core/decorator/__init__.py", line 3= 6, in wrapped_f
return func(*args, **kwargs)
File "/home/pokybuild= /yocto-worker/pkgman-deb-non-deb/build/meta/lib/oeqa/core/decorator/__init_= _.py", line 36, in wrapped_f
return func(*args, **kwargs)
File &qu= ot;/home/pokybuild/yocto-worker/pkgman-deb-non-deb/build/meta/lib/oeqa/runt= ime/cases/apt.py", line 50, in test_apt_install_from_repo
self.pkg(= 'update')
File "/home/pokybuild/yocto-worker/pkgman-deb-non-d= eb/build/meta/lib/oeqa/runtime/cases/apt.py", line 17, in pkg
self.= assertEqual(status, expected, message)
= AssertionError: 100 !=3D 0 : apt-get = update
Ign:1 http://1= 92.168.7.5:42261 ./ InRelease
Get:2 http://192.168.7.5:42261 ./ Release [1213 B]<= /span>
Ign:3 http://192.168.7.5:422= 61 ./ Release.gpg
Get:4 http://192.168.7.5:42261 ./ Packages [59.3 kB]
Err:4 http://192.168.7.5:42261 ./ = Packages
Hash Sum mismatch
Fetched 60.5 kB in 20s (3020 B/s)<= /div>
Reading package= lists...
W: The repository 'http://192.168.7.5:42261 ./ Release' is not signed.
E: F= ailed to fetch http://192.168.7.5:42261/./Packages.gz Hash Sum mismatch
E: Some index files failed to down= load. They have been ignored, or old ones used instead.
= =C2=A0
Jose



Steve





--
Best regards,

Jos=C3=A9 Quaresma

-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-
Links: You receive all messages sent to this group.
View/Reply Online (#164347): https:= //lists.openembedded.org/g/openembedded-core/message/164347
Mute This Topic: https://lists.openembedded.org/mt= /90107518/5052612
Group Owner: openembedded-core+owner@lists.openembedded.org<= br> Unsubscribe: https://lists.openembedded.org/= g/openembedded-core/unsub [quaresma.jose@gmail.com]
-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-



--
Best regards,

Jos=C3=A9= Quaresma
--000000000000fe670d05dc8cb046--