still confused about valid wildcarding for bbappend files

still confused about valid wildcarding for bbappend files

[Robert P. J. Day]

[-- Attachment #1: Type: text/plain, Size: 686 bytes --]


from current bitbake manual:

https://docs.yoctoproject.org/bitbake/bitbake-user-manual/bitbake-user-manual-intro.html?highlight=wildcard#append-files

"The use of the ” % ” character is limited in that it only works
directly in front of the .bbappend portion of the append file’s name.
You cannot use the wildcard character in any other location of the
name."

  that *appears* to suggest that you can have one (and exactly one)
"%" before the ".bbappend" suffix. so how does that explanation cover
examples in some of the OE layers like:

  meta-security/recipes-kernel/linux/linux-%_5.%.bbappend
  meta-anaconda/recipes-gnome/gtk+/gtk+3_%.%.%.bbappend

rday

Re: [bitbake-devel] still confused about valid wildcarding for bbappend files

[Mark Hatle]

On 4/4/21 2:51 PM, Robert P. J. Day wrote:
> 
> from current bitbake manual:
> 
> https://docs.yoctoproject.org/bitbake/bitbake-user-manual/bitbake-user-manual-intro.html?highlight=wildcard#append-files
> 
> "The use of the ” % ” character is limited in that it only works
> directly in front of the .bbappend portion of the append file’s name.
> You cannot use the wildcard character in any other location of the
> name."

The filename is broken up into: name_version.bbappend  I believe the intention
is that only "version" can contain wild cards.

>   that *appears* to suggest that you can have one (and exactly one)
> "%" before the ".bbappend" suffix. so how does that explanation cover
> examples in some of the OE layers like:
> 
>   meta-security/recipes-kernel/linux/linux-%_5.%.bbappend
>   meta-anaconda/recipes-gnome/gtk+/gtk+3_%.%.%.bbappend

"%" is a wildcard for the _version_ only.

It is my understanding it can be used in any part of the version string, but can
NOT be used in the name.  The above example shows it being used in the linux-
recipe name.  So perhaps that has changed.

The later example (gtk+3) it's being used in the versioning, but looks
unnecessary as it's just patching all of the version elements anyway.

The general case is:

recipe_1.0.3.bb

recipe_%.bbappend - match all versions

recipe_1.%.bbappend - match all 1.x versions

recipe_1.0.%.bbappend - match all 1.0.x versions

Any other usage might work, but I doubt has been seriously tested.  (And the key
isn't that the '%' be right before the bbappend!  It just happens to be there.)

--Mark

> rday
> 
> 
> 
> 
> 

Re: [bitbake-devel] still confused about valid wildcarding for bbappend files

[Robert P. J. Day]

[-- Attachment #1: Type: text/plain, Size: 2187 bytes --]

On Tue, 6 Apr 2021, Mark Hatle wrote:

> On 4/4/21 2:51 PM, Robert P. J. Day wrote:
> >
> > from current bitbake manual:
> >
> > https://docs.yoctoproject.org/bitbake/bitbake-user-manual/bitbake-user-manual-intro.html?highlight=wildcard#append-files
> >
> > "The use of the ” % ” character is limited in that it only works
> > directly in front of the .bbappend portion of the append file’s name.
> > You cannot use the wildcard character in any other location of the
> > name."
>
> The filename is broken up into: name_version.bbappend I believe the
> intention is that only "version" can contain wild cards.

  that was my understanding all this time.

> >   that *appears* to suggest that you can have one (and exactly
> > one) "%" before the ".bbappend" suffix. so how does that
> > explanation cover examples in some of the OE layers like:
> >
> >   meta-security/recipes-kernel/linux/linux-%_5.%.bbappend
> >   meta-anaconda/recipes-gnome/gtk+/gtk+3_%.%.%.bbappend
>
> "%" is a wildcard for the _version_ only.
>
> It is my understanding it can be used in any part of the version
> string, but can NOT be used in the name.  The above example shows it
> being used in the linux- recipe name.  So perhaps that has changed.

  agreed, that example is a bit of a surprise.

> The later example (gtk+3) it's being used in the versioning, but looks
> unnecessary as it's just patching all of the version elements anyway.

  perhaps, unless by adding the periods, it's saying that the version
number must contain *at least* three components (since the final "%"
will always suck up whatever remains). but that's just a guess, and
probably wildly wrong.

> The general case is:
>
> recipe_1.0.3.bb
>
> recipe_%.bbappend - match all versions
>
> recipe_1.%.bbappend - match all 1.x versions
>
> recipe_1.0.%.bbappend - match all 1.0.x versions
>
> Any other usage might work, but I doubt has been seriously tested.
> (And the key isn't that the '%' be right before the bbappend!  It
> just happens to be there.)

  that's pretty much the way i'd always thought of it, but perhaps
we've been mistaken all this time. :-)

rday

Re: [bitbake-devel] still confused about valid wildcarding for bbappend files

[Richard Purdie]

On Tue, 2021-04-06 at 12:34 -0400, Robert P. J. Day wrote:
> On Tue, 6 Apr 2021, Mark Hatle wrote:
> 
> > On 4/4/21 2:51 PM, Robert P. J. Day wrote:
> > > 
> > > from current bitbake manual:
> > > 
> > > https://docs.yoctoproject.org/bitbake/bitbake-user-manual/bitbake-user-manual-intro.html?highlight=wildcard#append-files
> > > 
> > > "The use of the ” % ” character is limited in that it only works
> > > directly in front of the .bbappend portion of the append file’s name.
> > > You cannot use the wildcard character in any other location of the
> > > name."
> > 
> > The filename is broken up into: name_version.bbappend I believe the
> > intention is that only "version" can contain wild cards.
> 
>   that was my understanding all this time.
> 
> > >   that *appears* to suggest that you can have one (and exactly
> > > one) "%" before the ".bbappend" suffix. so how does that
> > > explanation cover examples in some of the OE layers like:
> > > 
> > >   meta-security/recipes-kernel/linux/linux-%_5.%.bbappend
> > >   meta-anaconda/recipes-gnome/gtk+/gtk+3_%.%.%.bbappend
> > 
> > "%" is a wildcard for the _version_ only.
> > 
> > It is my understanding it can be used in any part of the version
> > string, but can NOT be used in the name.  The above example shows it
> > being used in the linux- recipe name.  So perhaps that has changed.
> 
>   agreed, that example is a bit of a surprise.
> 
> > The later example (gtk+3) it's being used in the versioning, but looks
> > unnecessary as it's just patching all of the version elements anyway.
> 
>   perhaps, unless by adding the periods, it's saying that the version
> number must contain *at least* three components (since the final "%"
> will always suck up whatever remains). but that's just a guess, and
> probably wildly wrong.
> 
> > The general case is:
> > 
> > recipe_1.0.3.bb
> > 
> > recipe_%.bbappend - match all versions
> > 
> > recipe_1.%.bbappend - match all 1.x versions
> > 
> > recipe_1.0.%.bbappend - match all 1.0.x versions
> > 
> > Any other usage might work, but I doubt has been seriously tested.
> > (And the key isn't that the '%' be right before the bbappend!  It
> > just happens to be there.)
> 
>   that's pretty much the way i'd always thought of it, but perhaps
> we've been mistaken all this time. :-)

As Mark says, the above is what is documented and meant to work. 
Easiest way to figure this out is to check what the code actually 
implements to handle it. The code (from cooker.py) says:

    def get_file_appends(self, fn):
        """
        Returns a list of .bbappend files to apply to fn
        """
        filelist = []
        f = os.path.basename(fn)
        for b in self.bbappends:
            (bbappend, filename) = b
            if (bbappend == f) or ('%' in bbappend and bbappend.startswith(f[:bbappend.index('%')])):
                filelist.append(filename)
        return tuple(filelist)

so it actually truncates the expression at the first % and ignores 
anything after the first one. That is there to chop the bbappend off
but can obviously chop off more.

meta-security/recipes-kernel/linux/linux-%_5.%.bbappend
may as well be:
meta-security/recipes-kernel/linux/linux-%.bbappend

so matches a bit more than expected and

meta-anaconda/recipes-gnome/gtk+/gtk+3_%.%.%.bbappend
is effectively
meta-anaconda/recipes-gnome/gtk+/gtk+3_%.bbappend

Cheers,

Richard

Re: [bitbake-devel] still confused about valid wildcarding for bbappend files

[Robert P. J. Day]

[-- Attachment #1: Type: text/plain, Size: 3871 bytes --]

On Tue, 6 Apr 2021, Richard Purdie wrote:

> On Tue, 2021-04-06 at 12:34 -0400, Robert P. J. Day wrote:
> > On Tue, 6 Apr 2021, Mark Hatle wrote:
> >
> > > On 4/4/21 2:51 PM, Robert P. J. Day wrote:
> > > >
> > > > from current bitbake manual:
> > > >
> > > > https://docs.yoctoproject.org/bitbake/bitbake-user-manual/bitbake-user-manual-intro.html?highlight=wildcard#append-files
> > > >
> > > > "The use of the ” % ” character is limited in that it only works
> > > > directly in front of the .bbappend portion of the append file’s name.
> > > > You cannot use the wildcard character in any other location of the
> > > > name."
> > >
> > > The filename is broken up into: name_version.bbappend I believe the
> > > intention is that only "version" can contain wild cards.
> >
> >   that was my understanding all this time.
> >
> > > >   that *appears* to suggest that you can have one (and exactly
> > > > one) "%" before the ".bbappend" suffix. so how does that
> > > > explanation cover examples in some of the OE layers like:
> > > >
> > > >   meta-security/recipes-kernel/linux/linux-%_5.%.bbappend
> > > >   meta-anaconda/recipes-gnome/gtk+/gtk+3_%.%.%.bbappend
> > >
> > > "%" is a wildcard for the _version_ only.
> > >
> > > It is my understanding it can be used in any part of the version
> > > string, but can NOT be used in the name.  The above example shows it
> > > being used in the linux- recipe name.  So perhaps that has changed.
> >
> >   agreed, that example is a bit of a surprise.
> >
> > > The later example (gtk+3) it's being used in the versioning, but looks
> > > unnecessary as it's just patching all of the version elements anyway.
> >
> >   perhaps, unless by adding the periods, it's saying that the version
> > number must contain *at least* three components (since the final "%"
> > will always suck up whatever remains). but that's just a guess, and
> > probably wildly wrong.
> >
> > > The general case is:
> > >
> > > recipe_1.0.3.bb
> > >
> > > recipe_%.bbappend - match all versions
> > >
> > > recipe_1.%.bbappend - match all 1.x versions
> > >
> > > recipe_1.0.%.bbappend - match all 1.0.x versions
> > >
> > > Any other usage might work, but I doubt has been seriously tested.
> > > (And the key isn't that the '%' be right before the bbappend!  It
> > > just happens to be there.)
> >
> >   that's pretty much the way i'd always thought of it, but perhaps
> > we've been mistaken all this time. :-)
>
> As Mark says, the above is what is documented and meant to work. 
> Easiest way to figure this out is to check what the code actually
> implements to handle it. The code (from cooker.py) says:
>
>     def get_file_appends(self, fn):
>         """
>         Returns a list of .bbappend files to apply to fn
>         """
>         filelist = []
>         f = os.path.basename(fn)
>         for b in self.bbappends:
>             (bbappend, filename) = b
>             if (bbappend == f) or ('%' in bbappend and bbappend.startswith(f[:bbappend.index('%')])):
>                 filelist.append(filename)
>         return tuple(filelist)
>
> so it actually truncates the expression at the first % and ignores 
> anything after the first one. That is there to chop the bbappend off
> but can obviously chop off more.
>
> meta-security/recipes-kernel/linux/linux-%_5.%.bbappend
> may as well be:
> meta-security/recipes-kernel/linux/linux-%.bbappend
>
> so matches a bit more than expected and
>
> meta-anaconda/recipes-gnome/gtk+/gtk+3_%.%.%.bbappend
> is effectively
> meta-anaconda/recipes-gnome/gtk+/gtk+3_%.bbappend

  ok, i can submit a patch to clarify that in the bitbake user manual,
with the understanding that it's *strongly* encouraged to use this
feature for wildcarding the version number only.

rday

Re: [bitbake-devel] still confused about valid wildcarding for bbappend files

[Richard Purdie]

On Wed, 2021-04-07 at 05:02 -0400, Robert P. J. Day wrote:
> On Tue, 6 Apr 2021, Richard Purdie wrote:
> 
> > On Tue, 2021-04-06 at 12:34 -0400, Robert P. J. Day wrote:
> > > On Tue, 6 Apr 2021, Mark Hatle wrote:
> > 
> > As Mark says, the above is what is documented and meant to work. 
> > Easiest way to figure this out is to check what the code actually
> > implements to handle it. The code (from cooker.py) says:
> > 
> >     def get_file_appends(self, fn):
> >         """
> >         Returns a list of .bbappend files to apply to fn
> >         """
> >         filelist = []
> >         f = os.path.basename(fn)
> >         for b in self.bbappends:
> >             (bbappend, filename) = b
> >             if (bbappend == f) or ('%' in bbappend and bbappend.startswith(f[:bbappend.index('%')])):
> >                 filelist.append(filename)
> >         return tuple(filelist)
> > 
> > so it actually truncates the expression at the first % and ignores 
> > anything after the first one. That is there to chop the bbappend off
> > but can obviously chop off more.
> > 
> > meta-security/recipes-kernel/linux/linux-%_5.%.bbappend
> > may as well be:
> > meta-security/recipes-kernel/linux/linux-%.bbappend
> > 
> > so matches a bit more than expected and
> > 
> > meta-anaconda/recipes-gnome/gtk+/gtk+3_%.%.%.bbappend
> > is effectively
> > meta-anaconda/recipes-gnome/gtk+/gtk+3_%.bbappend
> 
>   ok, i can submit a patch to clarify that in the bitbake user manual,
> with the understanding that it's *strongly* encouraged to use this
> feature for wildcarding the version number only.

The only guarantee we're making is the one we've documented, the rest is an 
implementation detail. It answers your question about what those bbappends
actually do but I really don't want to document the implementation details
as that then becomes the required behaviour.

Cheers,

Richard