From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <prvs=85754256d=Mikko.Rapeli@bmw.de>
Received: from esa6.bmw.c3s2.iphmx.com (esa6.bmw.c3s2.iphmx.com
	[68.232.139.124])
	by mail.openembedded.org (Postfix) with ESMTP id 8312B608B7
	for <openembedded-core@lists.openembedded.org>;
	Fri, 26 Feb 2016 14:56:26 +0000 (UTC)
Received: from esagw5.bmwgroup.com (HELO esagw5.muc) ([160.46.252.46]) by
	esa6.bmw.c3s2.iphmx.com with ESMTP/TLS; 26 Feb 2016 15:56:26 +0100
Received: from unknown (HELO esabb5.muc) ([160.50.100.47]) by esagw5.muc with
	ESMTP/TLS; 26 Feb 2016 15:56:26 +0100
Received: from smuch53a.muc (HELO SMUCH53A.europe.bmw.corp) ([160.46.137.113])
	by esabb5.muc with ESMTP/TLS; 26 Feb 2016 15:56:27 +0100
Received: from SMUCM65A.europe.bmw.corp ([160.46.134.155]) by
	SMUCH53A.europe.bmw.corp ([160.46.137.113]) with mapi id 14.03.0248.002;
	Fri, 26 Feb 2016 15:56:25 +0100
From: <Mikko.Rapeli@bmw.de>
To: <mariano.lopez@linux.intel.com>
Thread-Topic: [OE-core] [PATCH 0/3] Add initial capability to check CVEs for
	recipes
Thread-Index: AQHRb8YPTPKRB7zS1kqsvPG9SjQg2J88n/SAgAAQYICAAAu0AIABLzSAgABuDICAAAIhAA==
Date: Fri, 26 Feb 2016 14:56:24 +0000
Message-ID: <20160226145624.GJ6210@loska>
References: <cover.1456327117.git.mariano.lopez@linux.intel.com>
	<20160225121421.GB6210@loska> <20160225122912.GC6210@loska>
	<20160225132748.GD6210@loska> <20160225140942.GE6210@loska>
	<20160226081455.GH6210@loska> <56D065CF.9060303@linux.intel.com>
In-Reply-To: <56D065CF.9060303@linux.intel.com>
Accept-Language: en-US, de-DE
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
x-originating-ip: [192.168.44.99]
MIME-Version: 1.0
Cc: openembedded-core@lists.openembedded.org
Subject: Re: [PATCH 0/3] Add initial capability to check CVEs for recipes
X-BeenThere: openembedded-core@lists.openembedded.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Patches and discussions about the oe-core layer
	<openembedded-core.lists.openembedded.org>
List-Unsubscribe: <http://lists.openembedded.org/mailman/options/openembedded-core>,
	<mailto:openembedded-core-request@lists.openembedded.org?subject=unsubscribe>
List-Archive: <http://lists.openembedded.org/pipermail/openembedded-core/>
List-Post: <mailto:openembedded-core@lists.openembedded.org>
List-Help: <mailto:openembedded-core-request@lists.openembedded.org?subject=help>
List-Subscribe: <http://lists.openembedded.org/mailman/listinfo/openembedded-core>,
	<mailto:openembedded-core-request@lists.openembedded.org?subject=subscribe>
X-List-Received-Date: Fri, 26 Feb 2016 14:56:28 -0000
Content-Language: en-US
Content-Type: text/plain; charset="us-ascii"
Content-ID: <F500388BDF9C954E953175B902E27930@bmwmail.corp>
Content-Transfer-Encoding: quoted-printable

On Fri, Feb 26, 2016 at 08:48:47AM -0600, Mariano Lopez wrote:
> On 02/26/2016 02:14 AM, Mikko.Rapeli@bmw.de wrote:
> >Hi,
> >
> >On my developer machine the cve-check ran ok for dizzy but on build serv=
er
> >with sstate-cache and rmwork enabled it failed with what looks like a ra=
ce
> >condition when scanning the patch files:
> >
> >17:45:36 ERROR: Error executing a python function in /home/builder/src/b=
ase/poky/meta/recipes-extended/mailx/mailx_12.5.bb:
> >17:45:36
> >17:45:36 The stack trace of python calls that resulted in this exception=
/failure was:
> >17:45:36 File: 'do_cve_check', lineno: 17, function: <module>
> >17:45:36      0013:    else:
> >17:45:36      0014:        bb.note("Failed to update CVE database, skipp=
ing CVE check")
> >17:45:36      0015:
> >17:45:36      0016:
> >17:45:36  *** 0017:do_cve_check(d)
> >17:45:36      0018:
> >17:45:37 File: 'do_cve_check', lineno: 8, function: do_cve_check
> >17:45:37      0004:    Check recipe for patched and unpatched CVEs
> >17:45:37      0005:    """
> >17:45:37      0006:
> >17:45:37      0007:    if os.path.exists(d.getVar("CVE_CHECK_TMP_FILE", =
True)):
> >17:45:37  *** 0008:        patched_cves =3D get_patches_cves(d)
> >17:45:37      0009:        patched, unpatched =3D check_cves(d, patched_=
cves)
> >17:45:37      0010:        if patched or unpatched:
> >17:45:37      0011:            cve_data =3D get_cve_info(d, patched + un=
patched)
> >17:45:37      0012:            cve_write_data(d, patched, unpatched, cve=
_data)
> >17:45:37 File: 'cve-check.bbclass', lineno: 13, function: get_patches_cv=
es
> >17:45:37      0009:    cve_match =3D re.compile("CVE:( CVE\-\d+\-\d+)+")
> >17:45:37      0010:    patched_cves =3D set()
> >17:45:37      0011:    for url in src_patches(d):
> >17:45:37      0012:        patch_file =3D bb.fetch.decodeurl(url)[2]
> >17:45:37  *** 0013:        with open(patch_file, "r") as f:
> >17:45:37      0014:            patch_text =3D f.read()
> >17:45:37      0015:
> >17:45:37      0016:        # Search for the "CVE: " line
> >17:45:37      0017:        match =3D cve_match.search(patch_text)
> >17:45:37 Exception: IOError: [Errno 2] No such file or directory: '/home=
/builder/src/base/build/tmp/work/corei7-64-linux/mailx/12.5-r2/heirloom-mai=
lx_12.5-1.diff'
> >17:45:37
> >17:45:37 ERROR: Function failed: do_cve_check
> >
> >So could this be caused by cve-check changes or is this just a side effe=
ct
> >of some other recipe problems?
> >
> >I could not see that kind of fixes in master.
> >
> >-Mikko
>=20
> The changes in patch series were minimal and actually this part of the co=
de
> wasn't touched at all. That part of the code will look for all the files =
in
> the SRC_URI variable and will look for the "CVE:" tag in order to find
> patches that solve CVEs.

Yep, the code seems straight forward.

> It seems the problem is with the bitbake fetcher, or the recipe;
> unfortunately the fetcher is one of the components that most change betwe=
en
> releases. Another thing to check is that if actually there is a
> heirloom-mailx_12.5-1.diff file in the paths that the fetcher look for. Y=
ou
> can check this in the cve_check or patch log in the work directory of the
> recipe.

Unfortunately the file is there if I check with devshell but I have now
four different CI runs with this failure. Only difference to my developer
machine is sstate cache. Build machines maintain their own sstate cache.

-Mikko=