Recipe Grep'ing

Recipe Grep'ing

[Chuck Wolber]

[-- Attachment #1: Type: text/plain, Size: 1010 bytes --]

I was pondering putting some work in to a fairly large patch set aimed at
making recipes easier to grep through, and wanted to get some feedback
before I put time and effort into it.

I have often found that the following pattern cleanly describes the "what"
and the "why" when grep'ing through recipes to search for things:

FOO += "item1"
FOO += "item2"

Whereas this pattern gives us the "what", but not the "why":

FOO = "item1 \
             item2 \
            "

After discussing this with Richard Purdie on IRC, I also understand that
the latter pattern benefits some forms of build output. In addition, for
SRC_URI, the "why" is normally fairly obvious from context clues.

So, is there any interest in accepting a patch set of that nature for Yocto
and OE repositories? If so, what variables and situations should be
considered "off limits" to a change like that?

..Ch:W..


-- 
*"Perfection must be reached by degrees; she requires the slow hand of
time." - Voltaire*

[-- Attachment #2: Type: text/html, Size: 1516 bytes --]

Re: [yocto] Recipe Grep'ing

[Khem Raj]

On 5/5/21 5:41 PM, Chuck Wolber wrote:
> I was pondering putting some work in to a fairly large patch set aimed 
> at making recipes easier to grep through, and wanted to get some 
> feedback before I put time and effort into it.
> 
> I have often found that the following pattern cleanly describes the 
> "what" and the "why" when grep'ing through recipes to search for things:
> 
> FOO += "item1"
> FOO += "item2"
> 
> Whereas this pattern gives us the "what", but not the "why":
> 
> FOO = "item1 \
>               item2 \
>              "
> 
> After discussing this with Richard Purdie on IRC, I also understand that 
> the latter pattern benefits some forms of build output. In addition, for 
> SRC_URI, the "why" is normally fairly obvious from context clues.
> 
> So, is there any interest in accepting a patch set of that nature for 
> Yocto and OE repositories? If so, what variables and situations should 
> be considered "off limits" to a change like that?

I think it is a good idea, however, to make it persistent it would be 
nice to have a linter, which can do such checks and perhaps we can 
enable this in autobuilders so we can keep such cleansups maintained

> 
> ..Ch:W..
> 
> 
> -- 
> *"Perfection must be reached by degrees; she requires the slow hand of 
> time." - Voltaire*
> 
> 
> 
> 

Re: [yocto] Recipe Grep'ing

[Robert Joslyn]

On Wed, 2021-05-05 at 17:50 -0700, Khem Raj wrote:
> 
> 
> On 5/5/21 5:41 PM, Chuck Wolber wrote:
> > I was pondering putting some work in to a fairly large patch set
> > aimed 
> > at making recipes easier to grep through, and wanted to get some 
> > feedback before I put time and effort into it.
> > 
> > "what" and the "why" when grep'ing through recipes to search for
> > things:
> > 
> > FOO += "item1"
> > FOO += "item2"
> > 
> > Whereas this pattern gives us the "what", but not the "why":
> > 
> > FOO = "item1 \
> >               item2 \
> >              "
> > 
> > After discussing this with Richard Purdie on IRC, I also understand
> > that 
> > the latter pattern benefits some forms of build output. In addition,
> > for 
> > SRC_URI, the "why" is normally fairly obvious from context clues.
> > 
> > So, is there any interest in accepting a patch set of that nature for
> > Yocto and OE repositories? If so, what variables and situations
> > should 
> > be considered "off limits" to a change like that?
> 
> nice to have a linter, which can do such checks and perhaps we can 
> enable this in autobuilders so we can keep such cleansups maintained

There is the oe-stylize.py script that attempts to format recipes
according to the style guide:
https://git.openembedded.org/meta-openembedded/tree/contrib/oe-stylize.py

Last time I played with it, I was a bit disappointed with some of the
changes it makes, some of which are different than what devtool does.
When I need to introduce new developers to bitbake, I'd love to be able
to hand them oe-stylize or something similar and just tell them to run
it before committing to make sure everything is formatted consistently.

I've had updating oe-stylize.py on my TODO list for a while, but more
important things always come up.

Robert 
> 

Re: [yocto] Recipe Grep'ing

[Chuck Wolber]

On Sat, May 8, 2021 at 6:25 PM Robert Joslyn
<robert.joslyn@redrectangle.org> wrote:

> There is the oe-stylize.py script that attempts to format recipes
> according to the style guide:
> https://git.openembedded.org/meta-openembedded/tree/contrib/oe-stylize.py
>
> Last time I played with it, I was a bit disappointed with some of the
> changes it makes, some of which are different than what devtool does.
> When I need to introduce new developers to bitbake, I'd love to be able
> to hand them oe-stylize or something similar and just tell them to run
> it before committing to make sure everything is formatted consistently.
>
> I've had updating oe-stylize.py on my TODO list for a while, but more
> important things always come up.

Given what I have seen so far, I am wondering if that is the right
place to start.

To summarize:

There appears to be general agreement that the idea is a good one, but a large
patch wall is considered rather objectionable by at least Bruce Ashfield.

Khem Raj brought up a good point that a big change like the one I am proposing
needs some sort of tooling to make sure we do not regress.

And the above reply from Robert Joslyn has me wondering if there is already
something in place to do the linting Khem Raj is referring to, or if
oe-stylize.py
would form the basis for such a tool if we added it to an automated system.

So taking a step back, does it make sense to update the guidance on the
styleguide (https://www.openembedded.org/wiki/Styleguide) page first?

If so, I would be happy to make the updates. I requested an account,
but I got an
error - "Error sending mail: Unknown error in PHP's mail() function."

..Ch:W..

-- 
"Perfection must be reached by degrees; she requires the slow hand of
time." - Voltaire

Re: [yocto] Recipe Grep'ing

[Bruce Ashfield]

On Wed, May 5, 2021 at 8:42 PM Chuck Wolber <chuckwolber@gmail.com> wrote:
>
> I was pondering putting some work in to a fairly large patch set aimed at making recipes easier to grep through, and wanted to get some feedback before I put time and effort into it.
>
> I have often found that the following pattern cleanly describes the "what" and the "why" when grep'ing through recipes to search for things:
>
> FOO += "item1"
> FOO += "item2"
>
> Whereas this pattern gives us the "what", but not the "why":
>
> FOO = "item1 \
>              item2 \
>             "
>
> After discussing this with Richard Purdie on IRC, I also understand that the latter pattern benefits some forms of build output. In addition, for SRC_URI, the "why" is normally fairly obvious from context clues.

The other issue with large formatting patches, is that they make a
history "wall".  So when we are debugging and trying to figure why a
change was made, you always have to jump over the wall (or dig under
it) to get the real information.

As such, I'm not a fan of changes like this, unless they are coupled
to some sort of functional change. So I'd be hesitant to take them
into recipes that I end up holding the bag when things break.

Cheers,

Bruce

>
> So, is there any interest in accepting a patch set of that nature for Yocto and OE repositories? If so, what variables and situations should be considered "off limits" to a change like that?
>
> ..Ch:W..
>
>
> --
> "Perfection must be reached by degrees; she requires the slow hand of time." - Voltaire
>
> 
>


-- 
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

Re: [yocto] Recipe Grep'ing

[Khem Raj]

On Wed, May 5, 2021 at 7:13 PM Bruce Ashfield <bruce.ashfield@gmail.com> wrote:
>
> On Wed, May 5, 2021 at 8:42 PM Chuck Wolber <chuckwolber@gmail.com> wrote:
> >
> > I was pondering putting some work in to a fairly large patch set aimed at making recipes easier to grep through, and wanted to get some feedback before I put time and effort into it.
> >
> > I have often found that the following pattern cleanly describes the "what" and the "why" when grep'ing through recipes to search for things:
> >
> > FOO += "item1"
> > FOO += "item2"
> >
> > Whereas this pattern gives us the "what", but not the "why":
> >
> > FOO = "item1 \
> >              item2 \
> >             "
> >
> > After discussing this with Richard Purdie on IRC, I also understand that the latter pattern benefits some forms of build output. In addition, for SRC_URI, the "why" is normally fairly obvious from context clues.
>
> The other issue with large formatting patches, is that they make a
> history "wall".  So when we are debugging and trying to figure why a
> change was made, you always have to jump over the wall (or dig under
> it) to get the real information.
>

right, that's what if we want to do this then it has to be persisted
with otherwise it will end up being one time exercise with all these
losses you described.

> As such, I'm not a fan of changes like this, unless they are coupled
> to some sort of functional change. So I'd be hesitant to take them
> into recipes that I end up holding the bag when things break.
>
> Cheers,
>
> Bruce
>
> >
> > So, is there any interest in accepting a patch set of that nature for Yocto and OE repositories? If so, what variables and situations should be considered "off limits" to a change like that?
> >
> > ..Ch:W..
> >
> >
> > --
> > "Perfection must be reached by degrees; she requires the slow hand of time." - Voltaire
> >
> >
> >
>
>
> --
> - Thou shalt not follow the NULL pointer, for chaos and madness await
> thee at its end
> - "Use the force Harry" - Gandalf, Star Trek II
>
> 
>