From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wr1-f65.google.com (mail-wr1-f65.google.com [209.85.221.65]) by mail.openembedded.org (Postfix) with ESMTP id 15DFE7DC62 for ; Tue, 30 Apr 2019 12:13:01 +0000 (UTC) Received: by mail-wr1-f65.google.com with SMTP id l2so5076186wrb.9 for ; Tue, 30 Apr 2019 05:13:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=qokJoPeWRQ/Ph8byJIft4MPNoqJATAMuCmI7af0eKIo=; b=nwxGbBos5fYEKBuMaR69dY+Zi3zTBLb/Zj2iCHlnSeYHUJbVeWnfC6W6wX8kVuXx6/ IODTW7hUIwYJOtaLQOLzZzboB1QPCWquJhTdhfaJ76GR/015+a39LhZi+6DGJlDUbPk7 7HXW9/Suwh1SiosRw/XRLN8THlSTXK0wTD+ktZjAgqR3Ij9xnLm1783G5RSCPaRGRN8Y UBhxwX9Zd0h9rYb22cckmdaZxK+P7HxMjrx5pRr64LTpant2GrewjHniGctkM28zGgA9 ZhVUA95qewvwPwlD4vTOcXCVcxpAaKr/rrvSDVofCE3J/fo3vSzIkLIE1BI0LI2nF7Nc cTIw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=qokJoPeWRQ/Ph8byJIft4MPNoqJATAMuCmI7af0eKIo=; b=rtacyq2pZcho9UIYkIw6WhzFo397Pb+3fjxzsuYebad5xRD2LznpHBp6Q6RxmheBdm SWyMhHAjRso8NHuDZGYdt9fB1aG301FyJED5S5cJaYQGIR0KTF7H0Fj+PNvpRN1yeMwL num1wItV/f/iQ7smLQlI2Vhc+IgxA64lPAivXv52LU5uAjkKNJuvC5FUqk1k6Z5GfrqO mcFfP+J5MsWbYygzgdwduoOR0H8Wy433rYDSw+HRSu9Sh3kq9BugVpuovai2w5o0yeuC rCBzV+kIsZgl4bIDGeR83EsDvepjnKqRKJujG/G4nkeKFnEpx82t6fUrJC8bXPlzOOID 1FKg== X-Gm-Message-State: APjAAAVtgkPtACUmnkBNIrLete0OwNkYlHXylW6Ve7Lrr1zTIGVnQH7V qRjUyS4+rHrbuSUO5oKFz9w2xw9ZuASzhIKwa2AfUXTG X-Google-Smtp-Source: APXvYqx15uLM7n2Cg7AJcysKJgkJnw/ufmXre6Bi2zFDE4m3CGLpKb43PO53pSNO5zRkoqXmMOgP6GxbAN08TihtUVE= X-Received: by 2002:adf:eb84:: with SMTP id t4mr9795616wrn.43.1556626382645; Tue, 30 Apr 2019 05:13:02 -0700 (PDT) MIME-Version: 1.0 References: <9f8a038abf7b7eff75038ccef5f066bfd37e58f5.camel@linuxfoundation.org> <0c45f88f-6056-2fa6-1db1-f7ce413ba0f3@windriver.com> In-Reply-To: <0c45f88f-6056-2fa6-1db1-f7ce413ba0f3@windriver.com> From: Martin Jansa Date: Tue, 30 Apr 2019 14:13:00 +0200 Message-ID: To: Robert Yang Cc: bitbake-devel Subject: Re: [PATCH 2/2] bitbake: build.py: check dependendent task for addtask X-BeenThere: bitbake-devel@lists.openembedded.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Patches and discussion that advance bitbake development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Apr 2019 12:13:02 -0000 Content-Type: multipart/alternative; boundary="00000000000095ed5a0587be5275" --00000000000095ed5a0587be5275 Content-Type: text/plain; charset="UTF-8" I just got few thousands of warnings thanks to this, they might be legitimate as write_bom_data and write_abi_xml_data are our own tasks added in https://github.com/webosose/meta-webosose/blob/master/meta-webos/classes/webos_base.bbclass very long time ago (based on fetchall task which was added in 2006 :)) - there might be better way to add them now, I'll check how to do it better or else I might need to disable this warning locally. Based on "utility-tasks: Drop fetchall and checkuriall tasks" I guess the right way to do the same now will be to convert our scripts to call it with --runall. It's pity that the warning is shown repeatedly. grep -c WARNING:.*dependent.task BUILD/log/cooker/m16p3/console-latest.log 30409 and that was only very small build not even finished yet: Currently 7 running tasks (1142 of 2514) .. WARNING: cross-localedef-native: dependent task do_deploy does not exist WARNING: cross-localedef-native: dependent task do_package_write_ipk does not exist WARNING: cross-localedef-native: dependent task do_deploy does not exist WARNING: cross-localedef-native: dependent task do_package_write_ipk does not exist WARNING: cross-localedef-native: dependent task write_bom_data does not exist WARNING: cross-localedef-native: dependent task write_abi_xml_data does not exist WARNING: autoconf-archive-native: dependent task do_deploy does not exist WARNING: autoconf-archive-native: dependent task do_package_write_ipk does not exist WARNING: autoconf-archive-native: dependent task do_deploy does not exist WARNING: autoconf-archive-native: dependent task do_package_write_ipk does not exist WARNING: autoconf-archive-native: dependent task write_bom_data does not exist WARNING: autoconf-archive-native: dependent task write_abi_xml_data does not exist WARNING: cross-localedef-native: dependent task do_deploy does not exist WARNING: cross-localedef-native: dependent task do_package_write_ipk does not exist WARNING: cross-localedef-native: dependent task do_deploy does not exist WARNING: cross-localedef-native: dependent task do_package_write_ipk does not exist WARNING: cross-localedef-native: dependent task write_bom_data does not exist WARNING: cross-localedef-native: dependent task write_abi_xml_data does not exist WARNING: cross-localedef-native: dependent task do_deploy does not exist WARNING: cross-localedef-native: dependent task do_package_write_ipk does not exist WARNING: cross-localedef-native: dependent task do_deploy does not exist WARNING: cross-localedef-native: dependent task do_package_write_ipk does not exist WARNING: cross-localedef-native: dependent task write_bom_data does not exist WARNING: cross-localedef-native: dependent task write_abi_xml_data does not exist WARNING: cross-localedef-native: dependent task do_deploy does not exist WARNING: cross-localedef-native: dependent task do_package_write_ipk does not exist WARNING: cross-localedef-native: dependent task do_deploy does not exist WARNING: cross-localedef-native: dependent task do_package_write_ipk does not exist WARNING: cross-localedef-native: dependent task write_bom_data does not exist WARNING: cross-localedef-native: dependent task write_abi_xml_data does not exist .. On Fri, Apr 26, 2019 at 4:49 AM Robert Yang wrote: > > > On 4/25/19 6:50 PM, Richard Purdie wrote: > > On Thu, 2019-04-25 at 18:01 +0800, Robert Yang wrote: > >> The following command is incorrect, but was ignored silently, that may > suprise > >> users: > >> > >> addtask task after task_not_existed > >> > >> This patch can check and warn for it. It would be better to also check > "before" > >> tasks, but there is no easier way to do it. > >> > >> [YOCTO #13282] > >> > >> Signed-off-by: Robert Yang > >> --- > >> bitbake/lib/bb/build.py | 3 +++ > >> 1 file changed, 3 insertions(+) > >> > >> diff --git a/bitbake/lib/bb/build.py b/bitbake/lib/bb/build.py > >> index 7571421..861e9a9 100644 > >> --- a/bitbake/lib/bb/build.py > >> +++ b/bitbake/lib/bb/build.py > >> @@ -815,6 +815,9 @@ def add_tasks(tasklist, d): > >> task_deps['parents'][task] = [] > >> if 'deps' in flags: > >> for dep in flags['deps']: > >> + # Check and warn for "addtask task after foo" while > foo does not exist > >> + if not dep in tasklist: > >> + bb.warn('%s: dependent task %s does not exist!' % > (d.getVar('PN'), dep)) > >> dep = d.expand(dep) > >> task_deps['parents'][task].append(dep) > > > > I can't help wonder if this change has races, depending on the order of > > the addtask and deltask expressions. Do most layers parse cleanly with > > this? > > The bb.build.add_tasks() is only called by ast.py's finalize(), I think > that > everything should be ready in finalize stage, and there should be no races. > I tested this with meta-oe, python perl, selinux and others, didn't see > any problems (unless I depended on an invalid task). > > > > > At one point bitbake did support "floating" invalid tasks, I can't > > remember if we changed that or not though. > > This warning is good for checking typos during development, I think that > user can call "bb.build.addtask()" dynamically in a function to fix the > warning > for floating tasks? > > // Robert > > > > > Cheers, > > > > Richard > > > > > -- > _______________________________________________ > bitbake-devel mailing list > bitbake-devel@lists.openembedded.org > http://lists.openembedded.org/mailman/listinfo/bitbake-devel > --00000000000095ed5a0587be5275 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


grep -c = WARNING:.*dependent.task BUILD/log/cooker/m16p3/console-latest.log=C2=A0
30409

and that was only very s= mall build not even finished yet:
Currently=C2=A0 7 running tasks= (1142 of 2514)

..
WARNING: cross-l= ocaledef-native: dependent task do_deploy does not exist
WARNING:= cross-localedef-native: dependent task do_package_write_ipk does not exist=
WARNING: cross-localedef-native: dependent task do_deploy does n= ot exist
WARNING: cross-localedef-native: dependent task do_packa= ge_write_ipk does not exist
WARNING: cross-localedef-native: depe= ndent task write_bom_data does not exist
WARNING: cross-localedef= -native: dependent task write_abi_xml_data does not exist
WARNING= : autoconf-archive-native: dependent task do_deploy does not exist
WARNING: autoconf-archive-native: dependent task do_package_write_ipk doe= s not exist
WARNING: autoconf-archive-native: dependent task do_d= eploy does not exist
WARNING: autoconf-archive-native: dependent = task do_package_write_ipk does not exist
WARNING: autoconf-archiv= e-native: dependent task write_bom_data does not exist
WARNING: a= utoconf-archive-native: dependent task write_abi_xml_data does not exist
WARNING: cross-localedef-native: dependent task do_deploy does not = exist
WARNING: cross-localedef-native: dependent task do_package_= write_ipk does not exist
WARNING: cross-localedef-native: depende= nt task do_deploy does not exist
WARNING: cross-localedef-native:= dependent task do_package_write_ipk does not exist
WARNING: cros= s-localedef-native: dependent task write_bom_data does not exist
= WARNING: cross-localedef-native: dependent task write_abi_xml_data does not= exist
WARNING: cross-localedef-native: dependent task do_deploy = does not exist
WARNING: cross-localedef-native: dependent task do= _package_write_ipk does not exist
WARNING: cross-localedef-native= : dependent task do_deploy does not exist
WARNING: cross-localede= f-native: dependent task do_package_write_ipk does not exist
WARN= ING: cross-localedef-native: dependent task write_bom_data does not exist
WARNING: cross-localedef-native: dependent task write_abi_xml_data= does not exist
WARNING: cross-localedef-native: dependent task d= o_deploy does not exist
WARNING: cross-localedef-native: dependen= t task do_package_write_ipk does not exist
WARNING: cross-localed= ef-native: dependent task do_deploy does not exist
WARNING: cross= -localedef-native: dependent task do_package_write_ipk does not exist
=
WARNING: cross-localedef-native: dependent task write_bom_data does no= t exist
WARNING: cross-localedef-native: dependent task write_abi= _xml_data does not exist
..

On Fri, Apr 26, 2019 at 4:49 AM Robert Yang <liezhi.yang@windriver.com> wrote:


On 4/25/19 6:50 PM, Richard Purdie wrote:
> On Thu, 2019-04-25 at 18:01 +0800, Robert Yang wrote:
>> The following command is incorrect, but was ignored silently, that= may suprise
>> users:
>>
>> addtask task after task_not_existed
>>
>> This patch can check and warn for it. It would be better to also c= heck "before"
>> tasks, but there is no easier way to do it.
>>
>> [YOCTO #13282]
>>
>> Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
>> ---
>>=C2=A0 =C2=A0bitbake/lib/bb/build.py | 3 +++
>>=C2=A0 =C2=A01 file changed, 3 insertions(+)
>>
>> diff --git a/bitbake/lib/bb/build.py b/bitbake/lib/bb/build.py
>> index 7571421..861e9a9 100644
>> --- a/bitbake/lib/bb/build.py
>> +++ b/bitbake/lib/bb/build.py
>> @@ -815,6 +815,9 @@ def add_tasks(tasklist, d):
>>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0task_deps['parents'= ;][task] =3D []
>>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0if 'deps' in flags= :
>>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0for dep in f= lags['deps']:
>> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 # Check a= nd warn for "addtask task after foo" while foo does not exist
>> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 if not de= p in tasklist:
>> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 bb.warn('%s: dependent task %s does not exist!' % (d.getVar(= 'PN'), dep))
>>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0dep =3D d.expand(dep)
>>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0task_deps['parents'][task].append(dep)
>
> I can't help wonder if this change has races, depending on the ord= er of
> the addtask and deltask expressions. Do most layers parse cleanly with=
> this?

The bb.build.add_tasks() is only called by ast.py's finalize(), I think= that
everything should be ready in finalize stage, and there should be no races.=
I tested this with meta-oe, python perl, selinux and others, didn't see=
any problems (unless I depended on an invalid task).

>
> At one point bitbake did support "floating" invalid tasks, I= can't
> remember if we changed that or not though.

This warning is good for checking typos during development, I think that user can call "bb.build.addtask()" dynamically in a function to f= ix the warning
for floating tasks?

// Robert

>
> Cheers,
>
> Richard
>
>
--
_______________________________________________
bitbake-devel mailing list
b= itbake-devel@lists.openembedded.org
http://lists.openembedded.org/mailman/li= stinfo/bitbake-devel
--00000000000095ed5a0587be5275--