Re: [OE-core] [PATCH V5 6/6] cargo-update-recipe-crates: don't walk on the whole dir

["Frédéric Martinsons" <frederic.martinsons@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 6847 bytes --]

By the way, about this "chickens and eggs" problem, isn't it the same for a
regular recipe you just upgraded?

Personnaly, when I update a recipe, I let bitbake tell me what is the new
checksum expected and put it the recipe (as the error message says).

But I'm aware this is not exactly the same, since a cargo based recipe
could contain a ton of crate:// uri and if you apply this method, you have
to copy the new checksum one by one, bitbake error after another...

Don't know how to make this better and I plead guilty for not mentioning
that in a dedicated commit message.

Of course, if someone come up with a smoothier way of doing this, I'll make
a new patch.

Le jeu. 30 mars 2023, 21:50, Frédéric Martinsons <
frederic.martinsons@gmail.com> a écrit :

> Well I see what you mean, I'll take a look at your example to try to find
> out if multiple Cargo.lock could be expected.
>
> And for your second remark, yes, there is a chicken and eggs issue for
> updating crates checksum from scratch.
> You didn't miss anything, I came across this when updating crates checksum
> that was "pristine"
>  and I only manage to execute update_crates by locally patching
> bitbake/lib/bb/fetch2/crate.py for
> recommends_checksum method to return False .
> I'm aware that is not ideal but I don't know how to make this better
> (maybe make do_update_crates
> after do_fetch instead of after do_patch ? )
>
>
> Le jeu. 30 mars 2023, 21:23, Martin Jansa <martin.jansa@gmail.com> a
> écrit :
>
>> I don't remember the exact details now, but when I was working on
>> updating solana recipes to use this
>>
>> https://github.com/webosose/meta-webosose/commit/9bdfae7988077d0211eeee79cc339d0770cd86b4
>>
>> the S was pointing to just some subdirectory and multiple Cargo.locks
>> files were parsed in other directories as well, that's why I was adding
>>
>> https://git.openembedded.org/openembedded-core/commit/meta/classes-recipe/cargo-update-recipe-crates.bbclass?id=7636a2b8080521ed2ad54b0edce47a8742a12d58
>>
>> in the end I've changed to use the root directory in S and just
>> set CARGO_SRC_DIR to right subdirectory to build.
>>
>> I don't have much experience with crates other than these solana recipes,
>> but isn't there some valid use-case for multiple Cargo.locks?
>> I assume Alex in original implementation didn't use the os.walk just to
>> make it more complicated :).
>>
>> And FWIW when trying to regenerate these .inc files with current master
>> and with:
>> $ bitbake -c update_crates solana-keygen
>> it fails with:
>> ERROR: solana-keygen-1.14.5-r0 do_fetch: No checksum specified for
>> '/OE/lge/build/webos/mickledore/downloads/Inflector-0.11.4.crate', please
>> add at least one to the recipe:
>> SRC_URI[Inflector.sha256sum] =
>> "fe438c63458706e03479442743baae6c88256498e6431708f6dfc520a26515d3"
>> ERROR: solana-keygen-1.14.5-r0 do_fetch: Bitbake Fetcher Error:
>> NoChecksumError('Missing SRC_URI checksum', '
>> https://crates.io/api/v1/crates/Inflector/0.11.4/download')
>>
>> Isn't it an chicken&egg issue now when do_fetch enforces checksums to be
>> used? Or am I just missing some of the pending changes or just using it
>> incorrectly?
>>
>>
>> On Thu, Mar 30, 2023 at 6:34 PM Alex Kiernan <alex.kiernan@gmail.com>
>> wrote:
>>
>>> On Thu, Mar 30, 2023 at 4:45 PM <frederic.martinsons@gmail.com> wrote:
>>> >
>>> > From: Frederic Martinsons <frederic.martinsons@gmail.com>
>>> >
>>> > There is no need to do such things, Cargo.lock file
>>> > has to be at the root of CARGO_LOCK_SRC_DIR.
>>> > This avoid finding other possible Cargo.lock that
>>> > would be in subdir (for example if a patch is applied
>>> > on the recipe, we can have .pc subdir in S and a Cargo.lock
>>> > can be there)
>>> >
>>> > Signed-off-by: Frederic Martinsons <frederic.martinsons@gmail.com>
>>> > ---
>>> >  .../cargo-update-recipe-crates.bbclass               | 12 ++++++++----
>>> >  1 file changed, 8 insertions(+), 4 deletions(-)
>>> >
>>> > diff --git a/meta/classes-recipe/cargo-update-recipe-crates.bbclass
>>> b/meta/classes-recipe/cargo-update-recipe-crates.bbclass
>>> > index daa363b0dd..549cfe627e 100644
>>> > --- a/meta/classes-recipe/cargo-update-recipe-crates.bbclass
>>> > +++ b/meta/classes-recipe/cargo-update-recipe-crates.bbclass
>>> > @@ -68,10 +68,14 @@ def get_crates(f):
>>> >  import os
>>> >  crates = "# Autogenerated with 'bitbake -c update_crates ${PN}'\n\n"
>>> >  found = False
>>> > -for root, dirs, files in os.walk('${CARGO_LOCK_SRC_DIR}'):
>>> > -    for file in files:
>>> > -        if file == 'Cargo.lock':
>>> > -            crates += get_crates(os.path.join(root, file))
>>> > +for file in os.listdir('${CARGO_LOCK_SRC_DIR}'):
>>> > +    if file == 'Cargo.lock':
>>> > +        try:
>>> > +            cargo_lock_path = os.path.join('${CARGO_LOCK_SRC_DIR}',
>>> file)
>>> > +            crates += get_crates(cargo_lock_path)
>>> > +        except Exception as e:
>>> > +            raise ValueError("Cannot parse '%s'" % cargo_lock_path)
>>> from e
>>> > +        else:
>>> >              found = True
>>> >  if not found:
>>> >      raise ValueError("Unable to find Cargo.lock in
>>> ${CARGO_LOCK_SRC_DIR}")
>>>
>>> Isn't this just a long-winded version of something like this
>>> (completely untested):
>>>
>>> diff --git a/meta/classes-recipe/cargo-update-recipe-crates.bbclass
>>> b/meta/classes-recipe/cargo-update-recipe-crates.bbclass
>>> index daa363b0dd65..22ddcfa5c1e3 100644
>>> --- a/meta/classes-recipe/cargo-update-recipe-crates.bbclass
>>> +++ b/meta/classes-recipe/cargo-update-recipe-crates.bbclass
>>> @@ -67,14 +67,7 @@ def get_crates(f):
>>>
>>>  import os
>>>  crates = "# Autogenerated with 'bitbake -c update_crates ${PN}'\n\n"
>>> -found = False
>>> -for root, dirs, files in os.walk('${CARGO_LOCK_SRC_DIR}'):
>>> -    for file in files:
>>> -        if file == 'Cargo.lock':
>>> -            crates += get_crates(os.path.join(root, file))
>>> -            found = True
>>> -if not found:
>>> -    raise ValueError("Unable to find Cargo.lock in
>>> ${CARGO_LOCK_SRC_DIR}")
>>> +crates += get_crates(os.path.join("${CARGO_LOCK_SRC_DIR}",
>>> "Cargo.lock"))
>>>  open("${TARGET_FILE}", 'w').write(crates)
>>>  EOF
>>>
>>> --
>>> Alex Kiernan
>>>
>>> -=-=-=-=-=-=-=-=-=-=-=-
>>> Links: You receive all messages sent to this group.
>>> View/Reply Online (#179344):
>>> https://lists.openembedded.org/g/openembedded-core/message/179344
>>> Mute This Topic: https://lists.openembedded.org/mt/97953740/3617156
>>> Group Owner: openembedded-core+owner@lists.openembedded.org
>>> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [
>>> Martin.Jansa@gmail.com]
>>> -=-=-=-=-=-=-=-=-=-=-=-
>>>
>>>

[-- Attachment #2: Type: text/html, Size: 9920 bytes --]