[PATCH]Variables - WKS_FILE tip finding one used

[PATCH]Variables - WKS_FILE tip finding one used

[Janne Kiiskila]

[-- Attachment #1.1: Type: text/plain, Size: 712 bytes --]


Adding tip on how to find the WKS_FILE being used in the build.
This information found from a StackOverflow article:

https://stackoverflow.com/questions/49912475/adding-a-partition-in-yocto-generated-image

provided by "Nayfe".

Attaching the patch as an attachment, since it seems Microsoft Outlook does something naughty with the whitespaces and thus corrupts the patch.


IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.

[-- Attachment #1.2: Type: text/html, Size: 2917 bytes --]

[-- Attachment #2: 0001-Variables-WKS_FILE-tip-finding-WKS_FILE-used.patch --]
[-- Type: application/octet-stream, Size: 1362 bytes --]

From 599888a3dc429f3c4487b62b6064994ad26ab192 Mon Sep 17 00:00:00 2001
From: Janne Kiiskila <janne.kiiskila@pelion.com>
Date: Mon, 12 Apr 2021 15:38:31 +0300
Subject: [PATCH] Variables - WKS_FILE tip finding WKS_FILE used

Adding tip on how to find the WKS_FILE being used in the build.
This information found from a StackOverflow article:

https://stackoverflow.com/questions/49912475/adding-a-partition-in-yocto-generated-image

provided by "Nayfe".

Signed-off-by: Janne Kiiskila <janne.kiiskila@pelion.com>
---
 documentation/ref-manual/variables.rst | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/documentation/ref-manual/variables.rst b/documentation/ref-manual/variables.rst
index 19f5e34c58..f083f77f95 100644
--- a/documentation/ref-manual/variables.rst
+++ b/documentation/ref-manual/variables.rst
@@ -8794,6 +8794,9 @@ system and gives an overview of their function and contents.
       section in the Yocto Project Development Tasks Manual. For details on
       the kickstart file format, see the ":doc:`/ref-manual/kickstart`" Chapter.
 
+      To find out which WKS_FILE your build is using, you can run
+      ``bitbake -e |grep "^WKS_FILE="``
+
    :term:`WKS_FILE_DEPENDS`
       When placed in the recipe that builds your image, this variable lists
       build-time dependencies. The ``WKS_FILE_DEPENDS`` variable is only
-- 
2.17.1

Re: [docs] [PATCH]Variables - WKS_FILE tip finding one used

[Michael Opdenacker]

Hi Janne,

On 4/12/21 2:48 PM, Janne Kiiskila wrote:
>
>  
>
> Adding tip on how to find the WKS_FILE being used in the build.
>
> This information found from a StackOverflow article:
>
>  
>
> https://stackoverflow.com/questions/49912475/adding-a-partition-in-yocto-generated-image
> <https://stackoverflow.com/questions/49912475/adding-a-partition-in-yocto-generated-image>
>
>  
>
> provided by "Nayfe".
>
>  
>
> Attaching the patch as an attachment, since it seems Microsoft Outlook
> does something naughty with the whitespaces and thus corrupts the patch.
>

Thanks for the patch anyway! Next time you may actually include your
patch both as an inline attachment, so that people can comment on the
patch contents, and as a regular attachment so that I can apply the patch.

I have a small improvement to suggest in your patch. Here it is:

diff --git a/documentation/ref-manual/variables.rst
b/documentation/ref-manual/variables.rst
index 19f5e34c58..f083f77f95 100644
--- a/documentation/ref-manual/variables.rst
+++ b/documentation/ref-manual/variables.rst
@@ -8794,6 +8794,9 @@ system and gives an overview of their function and
contents.
       section in the Yocto Project Development Tasks Manual. For details on
       the kickstart file format, see the ":doc:`/ref-manual/kickstart`"
Chapter.
 
+      To find out which WKS_FILE your build is using, you can run
+      ``bitbake -e |grep "^WKS_FILE="``
+
    :term:`WKS_FILE_DEPENDS`
       When placed in the recipe that builds your image, this variable lists
       build-time dependencies. The ``WKS_FILE_DEPENDS`` variable is only
-- 
2.17.1


Instead, I would just use (note the added space after the pipe, and the
removed double quotes):

bitbake -e | grep ^WKS_FILE=

At least, this would be consistent with a similar line in another manual:

documentation/dev-manual/common-tasks.rst:   $ bitbake -e basename |
grep ^WORKDIR=

Actually, still for consistency with this file (and with the majority of
other manuals), you could even write:

::

   $ bitbake -e | grep ^WKS_FILE=

If you agree, would you mind re-sending an update, please?

Thanks again,

Michael.

-- 
Michael Opdenacker, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

Re: [docs] [PATCH]Variables - WKS_FILE tip finding one used

[Janne Kiiskila]

[-- Attachment #1: Type: text/plain, Size: 5239 bytes --]

Hei,

It is (unfortunately) amazing how wrong emails can still get things, so what I have here in MS Outlook the response is actually nearly unreadable.
Outlook manages to get in some strange extra characters there -  �� $ bitbake -e | grep ^WKS_FILE=

I am now assuming, based on what I read via the web-interface at https://lists.yoctoproject.org/g/docs/topic/patch_variables_wks_file/82036822?p=,,,20,0,0,0::recentpostdate%2Fsticky,,,20,2,0,82036822
That it actually means:

    ::

    $ bitbake -e | grep ^WKS_FILE=

I have updated the patch accordingly, attaching the new version of that here and also including it in text format for (hopefully) quicker/readable reviews.


From 5d6b91efb1b024146fe57d02b7cf289afc57509c Mon Sep 17 00:00:00 2001
From: Janne Kiiskila <janne.kiiskila@pelion.com>
Date: Mon, 12 Apr 2021 15:38:31 +0300
Subject: [PATCH] Variables - WKS_FILE tip finding WKS_FILE used

Adding tip on how to find the WKS_FILE being used in the build.
This information found from a StackOverflow article:

https://stackoverflow.com/questions/49912475/adding-a-partition-in-yocto-generated-image

provided by "Nayfe".

Updated version based on feedback from Michael Opdenacker.

Signed-off-by: Janne Kiiskila <janne.kiiskila@pelion.com>
---
 documentation/ref-manual/variables.rst | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/documentation/ref-manual/variables.rst b/documentation/ref-manual/variables.rst
index 19f5e34c58..392f9de391 100644
--- a/documentation/ref-manual/variables.rst
+++ b/documentation/ref-manual/variables.rst
@@ -8794,6 +8794,11 @@ system and gives an overview of their function and contents.
       section in the Yocto Project Development Tasks Manual. For details on
       the kickstart file format, see the ":doc:`/ref-manual/kickstart`" Chapter.

+      To find out which WKS_FILE your build is using, you can run
+      ::
+
+      $ bitbake -e | grep ^WKS_FILE=
+
    :term:`WKS_FILE_DEPENDS`
       When placed in the recipe that builds your image, this variable lists
       build-time dependencies. The ``WKS_FILE_DEPENDS`` variable is only
--
2.17.1


-----Original Message-----
From: docs@lists.yoctoproject.org <docs@lists.yoctoproject.org> On Behalf Of Michael Opdenacker via lists.yoctoproject.org
Sent: maanantai 12. huhtikuuta 2021 21.00
To: Janne Kiiskila <Janne.Kiiskila@pelion.com>; docs@lists.yoctoproject.org
Subject: Re: [docs] [PATCH]Variables - WKS_FILE tip finding one used

Hi Janne,

On 4/12/21 2:48 PM, Janne Kiiskila wrote:
>
> �
>
> Adding tip on how to find the WKS_FILE being used in the build.
>
> This information found from a StackOverflow article:
>
> �
>
> https://stackoverflow.com/questions/49912475/adding-a-partition-in-yoc
> to-generated-image
> <https://stackoverflow.com/questions/49912475/adding-a-partition-in-yo
> cto-generated-image>
>
> �
>
> provided by "Nayfe".
>
> �
>
> Attaching the patch as an attachment, since it seems Microsoft Outlook
> does something naughty with the whitespaces and thus corrupts the patch.
>

Thanks for the patch anyway! Next time you may actually include your patch both as an inline attachment, so that people can comment on the patch contents, and as a regular attachment so that I can apply the patch.

I have a small improvement to suggest in your patch. Here it is:

diff --git a/documentation/ref-manual/variables.rst
b/documentation/ref-manual/variables.rst
index 19f5e34c58..f083f77f95 100644
--- a/documentation/ref-manual/variables.rst
+++ b/documentation/ref-manual/variables.rst
@@ -8794,6 +8794,9 @@ system and gives an overview of their function and contents.
������ section in the Yocto Project Development Tasks Manual. For details on ������ the kickstart file format, see the ":doc:`/ref-manual/kickstart`"
Chapter.
�
+����� To find out which WKS_FILE your build is using, you can
+run ����� ``bitbake -e |grep "^WKS_FILE="``
+
��� :term:`WKS_FILE_DEPENDS`
������ When placed in the recipe that builds your image, this variable lists ������ build-time dependencies. The ``WKS_FILE_DEPENDS`` variable is only
--
2.17.1


Instead, I would just use (note the added space after the pipe, and the
removed double quotes):

bitbake -e | grep ^WKS_FILE=

At least, this would be consistent with a similar line in another manual:

documentation/dev-manual/common-tasks.rst:�� $ bitbake -e basename |
grep ^WORKDIR=

Actually, still for consistency with this file (and with the majority of
other manuals), you could even write:

::

�� $ bitbake -e | grep ^WKS_FILE=

If you agree, would you mind re-sending an update, please?

Thanks again,

Michael.

--
Michael Opdenacker, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.

[-- Attachment #2: 0001-Variables-WKS_FILE-tip-finding-WKS_FILE-used.patch --]
[-- Type: application/octet-stream, Size: 1434 bytes --]

From 5d6b91efb1b024146fe57d02b7cf289afc57509c Mon Sep 17 00:00:00 2001
From: Janne Kiiskila <janne.kiiskila@pelion.com>
Date: Mon, 12 Apr 2021 15:38:31 +0300
Subject: [PATCH] Variables - WKS_FILE tip finding WKS_FILE used

Adding tip on how to find the WKS_FILE being used in the build.
This information found from a StackOverflow article:

https://stackoverflow.com/questions/49912475/adding-a-partition-in-yocto-generated-image

provided by "Nayfe".

Updated version based on feedback from Michael Opdenacker.

Signed-off-by: Janne Kiiskila <janne.kiiskila@pelion.com>
---
 documentation/ref-manual/variables.rst | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/documentation/ref-manual/variables.rst b/documentation/ref-manual/variables.rst
index 19f5e34c58..392f9de391 100644
--- a/documentation/ref-manual/variables.rst
+++ b/documentation/ref-manual/variables.rst
@@ -8794,6 +8794,11 @@ system and gives an overview of their function and contents.
       section in the Yocto Project Development Tasks Manual. For details on
       the kickstart file format, see the ":doc:`/ref-manual/kickstart`" Chapter.
 
+      To find out which WKS_FILE your build is using, you can run
+      ::
+
+      $ bitbake -e | grep ^WKS_FILE=
+
    :term:`WKS_FILE_DEPENDS`
       When placed in the recipe that builds your image, this variable lists
       build-time dependencies. The ``WKS_FILE_DEPENDS`` variable is only
-- 
2.17.1

Re: [docs] [PATCH]Variables - WKS_FILE tip finding one used

[Robert P. J. Day]

[-- Attachment #1: Type: text/plain, Size: 1083 bytes --]

On Mon, 12 Apr 2021, Michael Opdenacker wrote:

> Hi Janne,
>
> On 4/12/21 2:48 PM, Janne Kiiskila wrote:
> >
> >  
> >
> > Adding tip on how to find the WKS_FILE being used in the build.

... big snip ...

>  
> +      To find out which WKS_FILE your build is using, you can run
> +      ``bitbake -e |grep "^WKS_FILE="``
> +
>     :term:`WKS_FILE_DEPENDS`
>        When placed in the recipe that builds your image, this variable lists
>        build-time dependencies. The ``WKS_FILE_DEPENDS`` variable is only

  while this is undeniably useful information, the same tip could be
added to pretty much every variable in the variable glossary, so i'm
not sure we want to start doing that; i think it would be far more
useful to make this a general hint somewhere.

  i also recall, from years gone by, that chris larson had a cool
utility named "bb" that would search in a *proper* way for the values
of variables by actually digging into the data dictionary, rather than
the clearly hackier way of grep'ing through the output of "bitbake
-e".

rday

Re: [docs] [PATCH]Variables - WKS_FILE tip finding one used

[Michael Opdenacker]

Janne, Robert,

On 4/13/21 11:32 AM, Robert P. J. Day wrote:
>
>   while this is undeniably useful information, the same tip could be
> added to pretty much every variable in the variable glossary, so i'm
> not sure we want to start doing that; i think it would be far more
> useful to make this a general hint somewhere.
>
>   i also recall, from years gone by, that chris larson had a cool
> utility named "bb" that would search in a *proper* way for the values
> of variables by actually digging into the data dictionary, rather than
> the clearly hackier way of grep'ing through the output of "bitbake
> -e".


Thanks for your contribution and review (respectively).

Do others have opinions about such a change? It would help to make a
decision...

Thanks in advance for jumping in...

Cheers,

Michael.

-- 
Michael Opdenacker, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

Re: [docs] [PATCH]Variables - WKS_FILE tip finding one used

[Robert P. J. Day]

Quoting Michael Opdenacker <michael.opdenacker@bootlin.com>:

> Janne, Robert,
>
> On 4/13/21 11:32 AM, Robert P. J. Day wrote:
>>
>>   while this is undeniably useful information, the same tip could be
>> added to pretty much every variable in the variable glossary, so i'm
>> not sure we want to start doing that; i think it would be far more
>> useful to make this a general hint somewhere.
>>
>>   i also recall, from years gone by, that chris larson had a cool
>> utility named "bb" that would search in a *proper* way for the values
>> of variables by actually digging into the data dictionary, rather than
>> the clearly hackier way of grep'ing through the output of "bitbake
>> -e".
>
>
> Thanks for your contribution and review (respectively).
>
> Do others have opinions about such a change? It would help to make a
> decision...
>
> Thanks in advance for jumping in...

   the other problem (i could have mentioned) is that that particular
grep incantation has the obvious flaw that it will not find variables
that are not at the beginning of the line, and that represents all of
those variables that are prefixed by "export":

export AR="aarch64-wrs-linux-gcc-ar"
export AS="aarch64-wrs-linux-as "
export BUILD_AR="ar"
export BUILD_AS="as "
export BUILD_CC="gcc "
export BUILD_CCLD="gcc "
export  
BUILD_CFLAGS="-isystem/home/rday51/repos/wrlinux/wrlinux/build/tmp-glibc/work/cortexa53-wrs-linux/defaultpkgname/1.0-r0/recipe-sysroot-native/usr/include -O2  
-pipe"
export BUILD_CPP="gcc  -E"
export  
BUILD_CPPFLAGS="-isystem/home/rday51/repos/wrlinux/wrlinux/build/tmp-glibc/work/cortexa53-wrs-linux/defaultpkgname/1.0-r0/recipe-sysroot-native/usr/include"
export BUILD_CXX="g++ "
export  
BUILD_CXXFLAGS="-isystem/home/rday51/repos/wrlinux/wrlinux/build/tmp-glibc/work/cortexa53-wrs-linux/defaultpkgname/1.0-r0/recipe-sysroot-native/usr/include -O2  
-pipe"
export BUILD_FC="gfortran "
export BUILD_LD="ld "
... and on and on ...

WKS_FILE is fine, of course, but it would be dangerous to
generalize this trick without the appropriate warning.

rday

Re: [docs] [PATCH]Variables - WKS_FILE tip finding one used

[Richard Purdie]

On Tue, 2021-04-13 at 20:39 +0200, Michael Opdenacker wrote:
> Janne, Robert,
> 
> On 4/13/21 11:32 AM, Robert P. J. Day wrote:
> > 
> >   while this is undeniably useful information, the same tip could be
> > added to pretty much every variable in the variable glossary, so i'm
> > not sure we want to start doing that; i think it would be far more
> > useful to make this a general hint somewhere.
> > 
> >   i also recall, from years gone by, that chris larson had a cool
> > utility named "bb" that would search in a *proper* way for the values
> > of variables by actually digging into the data dictionary, rather than
> > the clearly hackier way of grep'ing through the output of "bitbake
> > -e".
> 
> 
> Thanks for your contribution and review (respectively).
> 
> Do others have opinions about such a change? It would help to make a
> decision...
> 
> Thanks in advance for jumping in...

I think information on looking at the value of any variable should be 
documented centrally/generically rather than on each entry.

As Robert mentions, there was a nice bb util that used to make it easier,
there is also an open bug for writing such a util with the tinfoil API
which would be the modern way to do it. That shouldn't be a huge task 
but nobody has taken it up yet...

Cheers,

Richard

Re: [docs] [PATCH]Variables - WKS_FILE tip finding one used

[Janne Kiiskila]

Hei,

From my point of view - I spent several weeks last week debugging this issue and tried to look it up from this spot in the docs. I think you all have good intentions here and that's all fine and admirable.

However, if you do not have a clear intent to quickly do any of the below mentioned documents (and it seems we are missing a tool as well) - then you're just stalling.

We can put this in and help a few miserable persons in their painful road with Yocto NOW, rather than ... sometime in the uncertain future.

If we find the persons to do the wonderful desired state, then it's easy enough to remove this commit later on and add that "General debugging tips" section that uses that new tool later.

But, that's just my 25 cents.

Best Regards,


Janne


-----Original Message-----
From: Richard Purdie <richard.purdie@linuxfoundation.org>
Sent: tiistai 13. huhtikuuta 2021 22.42
To: Michael Opdenacker <michael.opdenacker@bootlin.com>; Robert P. J. Day <rpjday@crashcourse.ca>
Cc: Janne Kiiskila <Janne.Kiiskila@pelion.com>; docs@lists.yoctoproject.org
Subject: Re: [docs] [PATCH]Variables - WKS_FILE tip finding one used

On Tue, 2021-04-13 at 20:39 +0200, Michael Opdenacker wrote:
> Janne, Robert,
>
> On 4/13/21 11:32 AM, Robert P. J. Day wrote:
> >
> >   while this is undeniably useful information, the same tip could be
> > added to pretty much every variable in the variable glossary, so i'm
> > not sure we want to start doing that; i think it would be far more
> > useful to make this a general hint somewhere.
> >
> >   i also recall, from years gone by, that chris larson had a cool
> > utility named "bb" that would search in a *proper* way for the
> > values of variables by actually digging into the data dictionary,
> > rather than the clearly hackier way of grep'ing through the output
> > of "bitbake -e".
>
>
> Thanks for your contribution and review (respectively).
>
> Do others have opinions about such a change? It would help to make a
> decision...
>
> Thanks in advance for jumping in...

I think information on looking at the value of any variable should be documented centrally/generically rather than on each entry.

As Robert mentions, there was a nice bb util that used to make it easier, there is also an open bug for writing such a util with the tinfoil API which would be the modern way to do it. That shouldn't be a huge task but nobody has taken it up yet...

Cheers,

Richard

IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.

Re: [docs] [PATCH]Variables - WKS_FILE tip finding one used

[Richard Purdie]

On Wed, 2021-04-14 at 08:24 +0000, Janne Kiiskila wrote:
> Hei,
> 
> From my point of view - I spent several weeks last week debugging this issue 
> and tried to look it up from this spot in the docs. I think you all have good 
> intentions here and that's all fine and admirable.
> 
> However, if you do not have a clear intent to quickly do any of the below 
> mentioned documents (and it seems we are missing a tool as well) - then you're 
> just stalling.
> 
> We can put this in and help a few miserable persons in their painful road with 
> Yocto NOW, rather than ... sometime in the uncertain future.
> 
> If we find the persons to do the wonderful desired state, then it's easy enough 
> to remove this commit later on and add that "General debugging tips" section 
> that uses that new tool later.
> 
> But, that's just my 25 cents.

Adding that information to a single variable would mean that anyone else with a 
similar problem but with a different variable wouldn't be helped. I therefore 
think weeither document this somewhere generically or we don't do it. Taken
to an extreme, we wouldn't add that command to every variable in the glossary
as that wouldn't make sense.

With regard to the tool, I found the open bug for it:

https://bugzilla.yoctoproject.org/show_bug.cgi?id=10748

and since I was messing with tinfoil anyway on something unrelated, I've just 
added a bit of tinfoil script that shows how we'd implement it to the bug.

It needs a fix to the tinfoil data API:

http://git.yoctoproject.org/cgit.cgi/poky-contrib/commit/?h=rpurdie/t222&id=42c6c93f934b37345411a579592d2c305f9f4e3b

which is pretty horrific as a patch. Hopefully this is at least progress...

Cheers,

Richard