one way hash analogy of yocto

one way hash analogy of yocto

[Rob Prowel]

What I really need is information about introspection or the ability to 
query an existing yocto distro to return data about features included 
and more to the point, to understand EXACTLY where recipes or classes 
are overridden or appended.  You can only do so much with recursive grep 
and find scripts.  Based on what I've seen so far, it seems that 
capability is lacking in yocto.

In essence I have what is like a one way hash ABCDEFG-->X and I have X 
but cannot fully figure out the whole input that generated it.

An example would be an overridden task method like

do_something

I need a quick way to determine the highest priority level that task was 
overridden or appended in for a given recipe.

a command like "bitbake -describe -c do_something my-recipe"

and it should also be possible to query this kind of information for 
variables and not just methods or tasks.

Do these tools exist or am I looking at parsing megabytes of verbose 
build data and tracing output to find what I'm looking for?

and no, I didn't create the distro...I'm an end user customer who was 
given an undocumented distro with quazi-proprietary BSP layers in it 
that not only describe our specific product purchased, but the vendor's 
others products as well, which complicates matters.

Thoughts?

Re: [yocto] one way hash analogy of yocto

[Quentin Schulz]

Hi Robm

On Fri, Jul 31, 2020 at 08:45:17AM -0400, Rob Prowel wrote:
> What I really need is information about introspection or the ability to
> query an existing yocto distro to return data about features included and
> more to the point, to understand EXACTLY where recipes or classes are
> overridden or appended.  You can only do so much with recursive grep and
> find scripts.  Based on what I've seen so far, it seems that capability is
> lacking in yocto.
> 
> In essence I have what is like a one way hash ABCDEFG-->X and I have X but
> cannot fully figure out the whole input that generated it.
> 
> An example would be an overridden task method like
> 
> do_something
> 
> I need a quick way to determine the highest priority level that task was
> overridden or appended in for a given recipe.
> 
> a command like "bitbake -describe -c do_something my-recipe"
> 
> and it should also be possible to query this kind of information for
> variables and not just methods or tasks.
> 
> Do these tools exist or am I looking at parsing megabytes of verbose build
> data and tracing output to find what I'm looking for?
> 

For location of bbappends and recipes, etc.:
bitbake-layers show-appends
bitbake-layers show-recipes

For variables:
bitbake <recipe> -e | more
and find the line starting with the variable you're looking for. From
there, look the lines just before and you'll have the history of how the
variables was constructed.

bitbake <recipe> -e | awk '/^# \$FOOBAR \[/,/^FOOBAR/'
might do it for you. Not tested by me so can't attest. (source:
https://theyoctojester.info/session_14/main.html)

I do not know if there is such a mechanism for tasks, but the
bitbake-layers should help you get started in your quest of overridden
tasks.

Also, I'm not sure it matters much where the tasks are overridden? Just
override them again with your own bbappend (especially if it's a layer
you don't own (read upstream or provided by vendor)).

Quentin

Re: [yocto] one way hash analogy of yocto

[Mikko Rapeli]

On Fri, Jul 31, 2020 at 08:45:17AM -0400, Rob Prowel wrote:
> What I really need is information about introspection or the ability to
> query an existing yocto distro to return data about features included and
> more to the point, to understand EXACTLY where recipes or classes are
> overridden or appended.  You can only do so much with recursive grep and
> find scripts.  Based on what I've seen so far, it seems that capability is
> lacking in yocto.
> 
> In essence I have what is like a one way hash ABCDEFG-->X and I have X but
> cannot fully figure out the whole input that generated it.
> 
> An example would be an overridden task method like
> 
> do_something
> 
> I need a quick way to determine the highest priority level that task was
> overridden or appended in for a given recipe.
> 
> a command like "bitbake -describe -c do_something my-recipe"

bitbake -e my-recipe ?

Maybe grep for the variable of interest?

> and it should also be possible to query this kind of information for
> variables and not just methods or tasks.
> 
> Do these tools exist or am I looking at parsing megabytes of verbose build
> data and tracing output to find what I'm looking for?
> 
> and no, I didn't create the distro...I'm an end user customer who was given
> an undocumented distro with quazi-proprietary BSP layers in it that not only
> describe our specific product purchased, but the vendor's others products as
> well, which complicates matters.
> 
> Thoughts?

When adding layers, only way to be sure is to actually review the layers
and their bbappends. I tend to BBMASK everything out which I haven't checked
at least on BSP layer side. Open source layers from yocto community are better.

Cheers,

-Mikko

> 

> 

Re: [yocto] one way hash analogy of yocto

[Rob Prowel]

On 2020-07-31 08:53, Quentin Schulz wrote:

> 
> I do not know if there is such a mechanism for tasks, but the
> bitbake-layers should help you get started in your quest of overridden
> tasks.
> 
> Also, I'm not sure it matters much where the tasks are overridden? Just
> override them again with your own bbappend (especially if it's a layer
> you don't own (read upstream or provided by vendor)).

Thanks for the prompt followup.  Yeah, the two comments above are 
related in my nightmare so I'll address both of them together.

vendor has layers that I am NOT modifying, and unfortunately I'm not 
100% sure that they didn't modify poky or meta-oe in what they shipped 
to me.  Just haven't had time to download and compare those layers to 
yocto official sources.

I really do need to be able to inspect what they've done in their BSP 
layer to be able to effectively override or modify it though.  Things 
very easily break when I attempt to make changes.

Their as-delivered image builder creates an executable and the only way 
to update their board is to boot into linux on the board and run that 
executable (on the board) to update the secondary flash, then boot into 
the secondary flash and update the primary...which takes forever on QSPI 
NOR.  Yeah, they disabled the jtag interface so you HAVE to be in a 
boot-able linux image on the board to update it.

My development model is to reconfigure to use nfsroot while developing 
and configuring and only update the flash for relatively solid release 
to be tested.

but thanks...will investigate the commands you mentioned.  I really do 
need a quick way of seeing the authoritative version of a task though.