[v2 PATCH] base.bbclass: Add COMPATIBLE_OS, useful for non-Linux target recipes

[v2 PATCH] base.bbclass: Add COMPATIBLE_OS, useful for non-Linux target recipes

[Mark Hatle]

Add the ability to generate non-Linux based components, such as
baremetal, FreeRTOS, Zypher, etc, exist.  There is a need for a way
to take components as specific to a given OS.

This is especially important in a multiconfig build where you may be
targeting different OSes which different sets of recipes in a common
set of layers.

Setting a default of (matching bitbake.conf's default TARGET_OS):

   COMPATIBLE_OS ?= "linux${LIBCEXTENSION}${ABIEXTENSION}"

will allow all existing recipes to be tagged as Linux specific, so
only non-Linux recipes would need to be tagged with specific
compatibility.

For a baremetal recipes, the following was used to test this code:

   COMPATIBLE_OS = "elf"
   COMPATIBLE_OS_arm = "eabi"

Signed-off-by: Mark Hatle <mark.hatle@kernel.crashing.org>
---
V2:
Only difference to V1 is typograhic fixes to the commit message.

 meta/classes/base.bbclass    | 7 +++++++
 meta/conf/documentation.conf | 1 +
 2 files changed, 8 insertions(+)

diff --git a/meta/classes/base.bbclass b/meta/classes/base.bbclass
index 45f9435fd8..c123c5dc50 100644
--- a/meta/classes/base.bbclass
+++ b/meta/classes/base.bbclass
@@ -509,6 +509,13 @@ python () {
         d.setVarFlag('do_devshell', 'fakeroot', '1')
         d.appendVarFlag('do_devshell', 'depends', ' virtual/fakeroot-native:do_populate_sysroot')
 
+    need_os = d.getVar('COMPATIBLE_OS')
+    if need_os and not d.getVar('PARSE_ALL_RECIPES', False):
+        import re
+        this_os = d.getVar('TARGET_OS')
+        if not re.match(need_os, this_os):
+            raise bb.parse.SkipRecipe("incompatible with os %s (not in COMPATIBLE_OS)" % this_os)
+
     need_machine = d.getVar('COMPATIBLE_MACHINE')
     if need_machine and not d.getVar('PARSE_ALL_RECIPES', False):
         import re
diff --git a/meta/conf/documentation.conf b/meta/conf/documentation.conf
index 0b21d1f63e..7a87540f43 100644
--- a/meta/conf/documentation.conf
+++ b/meta/conf/documentation.conf
@@ -111,6 +111,7 @@ COMBINED_FEATURES[doc] = "A set of features common between MACHINE_FEATURES and
 COMMON_LICENSE_DIR[doc] = "Points to meta/files/common-licenses in the Source Directory, which is where generic license files reside."
 COMPATIBLE_HOST[doc] = "A regular expression that resolves to one or more hosts (when the recipe is native) or one or more targets (when the recipe is non-native) with which a recipe is compatible."
 COMPATIBLE_MACHINE[doc] = "A regular expression that resolves to one or more target machines with which a recipe is compatible."
+COMPATIBLE_OS[doc] = "A regular expression that resolves to one or more target oses which a recipe is compatible."
 COMPLEMENTARY_GLOB[doc] = "Defines wildcards to match when installing a list of complementary packages for all the packages installed in an image."
 CONFFILES[doc] = "Identifies editable or configurable files that are part of a package."
 CONFIG_SITE[doc] = "A list of files that contains autoconf test results relevant to the current build. This variable is used by the Autotools utilities when running configure."
-- 
2.17.1

Re: [OE-core] [v2 PATCH] base.bbclass: Add COMPATIBLE_OS, useful for non-Linux target recipes

[Richard Purdie]

On Thu, 2020-03-26 at 10:25 -0500, Mark Hatle wrote:
> Add the ability to generate non-Linux based components, such as
> baremetal, FreeRTOS, Zypher, etc, exist.  There is a need for a way
> to take components as specific to a given OS.
> 
> This is especially important in a multiconfig build where you may be
> targeting different OSes which different sets of recipes in a common
> set of layers.
> 
> Setting a default of (matching bitbake.conf's default TARGET_OS):
> 
>    COMPATIBLE_OS ?= "linux${LIBCEXTENSION}${ABIEXTENSION}"
> 
> will allow all existing recipes to be tagged as Linux specific, so
> only non-Linux recipes would need to be tagged with specific
> compatibility.
> 
> For a baremetal recipes, the following was used to test this code:
> 
>    COMPATIBLE_OS = "elf"
>    COMPATIBLE_OS_arm = "eabi"
> 
> Signed-off-by: Mark Hatle <mark.hatle@kernel.crashing.org>
> ---
> V2:
> Only difference to V1 is typograhic fixes to the commit message.
> 
>  meta/classes/base.bbclass    | 7 +++++++
>  meta/conf/documentation.conf | 1 +
>  2 files changed, 8 insertions(+)
> 
> diff --git a/meta/classes/base.bbclass b/meta/classes/base.bbclass
> index 45f9435fd8..c123c5dc50 100644
> --- a/meta/classes/base.bbclass
> +++ b/meta/classes/base.bbclass
> @@ -509,6 +509,13 @@ python () {
>          d.setVarFlag('do_devshell', 'fakeroot', '1')
>          d.appendVarFlag('do_devshell', 'depends', '
> virtual/fakeroot-native:do_populate_sysroot')
>  
> +    need_os = d.getVar('COMPATIBLE_OS')
> +    if need_os and not d.getVar('PARSE_ALL_RECIPES', False):
> +        import re
> +        this_os = d.getVar('TARGET_OS')
> +        if not re.match(need_os, this_os):
> +            raise bb.parse.SkipRecipe("incompatible with os %s (not
> in COMPATIBLE_OS)" % this_os)
> +
>      need_machine = d.getVar('COMPATIBLE_MACHINE')
>      if need_machine and not d.getVar('PARSE_ALL_RECIPES', False):
>          import re
> diff --git a/meta/conf/documentation.conf
> b/meta/conf/documentation.conf
> index 0b21d1f63e..7a87540f43 100644
> --- a/meta/conf/documentation.conf
> +++ b/meta/conf/documentation.conf
> @@ -111,6 +111,7 @@ COMBINED_FEATURES[doc] = "A set of features
> common between MACHINE_FEATURES and
>  COMMON_LICENSE_DIR[doc] = "Points to meta/files/common-licenses in
> the Source Directory, which is where generic license files reside."
>  COMPATIBLE_HOST[doc] = "A regular expression that resolves to one or
> more hosts (when the recipe is native) or one or more targets (when
> the recipe is non-native) with which a recipe is compatible."

How does COMPATIBLE_OS compare to COMPATIBLE_HOST (documented above)?

Cheers

Richard

Re: [OE-core] [v2 PATCH] base.bbclass: Add COMPATIBLE_OS, useful for non-Linux target recipes

[Mark Hatle]

> On Thu, 2020-03-26 at 10:25 -0500, Mark Hatle wrote:
>> Add the ability to generate non-Linux based components, such as
>> baremetal, FreeRTOS, Zypher, etc, exist.  There is a need for a way
>> to take components as specific to a given OS.
>>
>> This is especially important in a multiconfig build where you may be
>> targeting different OSes which different sets of recipes in a common
>> set of layers.
>>
>> Setting a default of (matching bitbake.conf's default TARGET_OS):
>>
>>    COMPATIBLE_OS ?= "linux${LIBCEXTENSION}${ABIEXTENSION}"
>>
>> will allow all existing recipes to be tagged as Linux specific, so
>> only non-Linux recipes would need to be tagged with specific
>> compatibility.
>>
>> For a baremetal recipes, the following was used to test this code:
>>
>>    COMPATIBLE_OS = "elf"
>>    COMPATIBLE_OS_arm = "eabi"
>>
>> Signed-off-by: Mark Hatle <mark.hatle@kernel.crashing.org>
>> ---
>> V2:
>> Only difference to V1 is typograhic fixes to the commit message.
>>
>>  meta/classes/base.bbclass    | 7 +++++++
>>  meta/conf/documentation.conf | 1 +
>>  2 files changed, 8 insertions(+)
>>
>> diff --git a/meta/classes/base.bbclass b/meta/classes/base.bbclass
>> index 45f9435fd8..c123c5dc50 100644
>> --- a/meta/classes/base.bbclass
>> +++ b/meta/classes/base.bbclass
>> @@ -509,6 +509,13 @@ python () {
>>          d.setVarFlag('do_devshell', 'fakeroot', '1')
>>          d.appendVarFlag('do_devshell', 'depends', '
>> virtual/fakeroot-native:do_populate_sysroot')
>>
>> +    need_os = d.getVar('COMPATIBLE_OS')
>> +    if need_os and not d.getVar('PARSE_ALL_RECIPES', False):
>> +        import re
>> +        this_os = d.getVar('TARGET_OS')
>> +        if not re.match(need_os, this_os):
>> +            raise bb.parse.SkipRecipe("incompatible with os %s (not
>> in COMPATIBLE_OS)" % this_os)
>> +
>>      need_machine = d.getVar('COMPATIBLE_MACHINE')
>>      if need_machine and not d.getVar('PARSE_ALL_RECIPES', False):
>>          import re
>> diff --git a/meta/conf/documentation.conf
>> b/meta/conf/documentation.conf
>> index 0b21d1f63e..7a87540f43 100644
>> --- a/meta/conf/documentation.conf
>> +++ b/meta/conf/documentation.conf
>> @@ -111,6 +111,7 @@ COMBINED_FEATURES[doc] = "A set of features
>> common between MACHINE_FEATURES and
>>  COMMON_LICENSE_DIR[doc] = "Points to meta/files/common-licenses in
>> the Source Directory, which is where generic license files reside."
>>  COMPATIBLE_HOST[doc] = "A regular expression that resolves to one or
>> more hosts (when the recipe is native) or one or more targets (when
>> the recipe is non-native) with which a recipe is compatible."
>
> How does COMPATIBLE_OS compare to COMPATIBLE_HOST (documented above)?

HOST_SYS="aarch64-oe-linux"

So the equivalent to the COMPATIBLE_OS would be:

COMPATIBLE_HOST ?= ".*-linux.*"

So it COULD be used in most cases.

When I tried this before though, I could get it to work in a multiconfig
setting.

Specifically what I tried was setting, in my local.conf:

COMPATIBLE_HOST ?= ".*-linux${LIBCEXTENSION}${ABIEXTENSION}"

And then in the recipes:

COMPATIBLE_HOST = ".*-elf"
COMPATIBLE_HOST_arm = ".*-eabi"

But then the second one matches the linux ABIEXTENSION and broke..

So really the issue is that I -only- ever want to match on the OS part of
the "SYS" string.  And done to the multiple natures of the '-', it's
difficult to get right.

I didn't try something complex like: [^-]-[^-]-eabi

--Mark

> Cheers
>
> Richard
>

Re: [OE-core] [v2 PATCH] base.bbclass: Add COMPATIBLE_OS, useful for non-Linux target recipes

[Richard Purdie]

On Thu, 2020-03-26 at 12:05 -0500, Mark Hatle wrote:
> > On Thu, 2020-03-26 at 10:25 -0500, Mark Hatle wrote:
> > > Add the ability to generate non-Linux based components, such as
> > > baremetal, FreeRTOS, Zypher, etc, exist.  There is a need for a
> > > way
> > > to take components as specific to a given OS.
> > > 
> > > This is especially important in a multiconfig build where you may
> > > be
> > > targeting different OSes which different sets of recipes in a
> > > common
> > > set of layers.
> > > 
> > > Setting a default of (matching bitbake.conf's default TARGET_OS):
> > > 
> > >    COMPATIBLE_OS ?= "linux${LIBCEXTENSION}${ABIEXTENSION}"
> > > 
> > > will allow all existing recipes to be tagged as Linux specific,
> > > so
> > > only non-Linux recipes would need to be tagged with specific
> > > compatibility.
> > > 
> > > For a baremetal recipes, the following was used to test this
> > > code:
> > > 
> > >    COMPATIBLE_OS = "elf"
> > >    COMPATIBLE_OS_arm = "eabi"
> > > 
> > > Signed-off-by: Mark Hatle <mark.hatle@kernel.crashing.org>
> > > ---
> > > V2:
> > > Only difference to V1 is typograhic fixes to the commit message.
> > > 
> > >  meta/classes/base.bbclass    | 7 +++++++
> > >  meta/conf/documentation.conf | 1 +
> > >  2 files changed, 8 insertions(+)
> > > 
> > > diff --git a/meta/classes/base.bbclass
> > > b/meta/classes/base.bbclass
> > > index 45f9435fd8..c123c5dc50 100644
> > > --- a/meta/classes/base.bbclass
> > > +++ b/meta/classes/base.bbclass
> > > @@ -509,6 +509,13 @@ python () {
> > >          d.setVarFlag('do_devshell', 'fakeroot', '1')
> > >          d.appendVarFlag('do_devshell', 'depends', '
> > > virtual/fakeroot-native:do_populate_sysroot')
> > > 
> > > +    need_os = d.getVar('COMPATIBLE_OS')
> > > +    if need_os and not d.getVar('PARSE_ALL_RECIPES', False):
> > > +        import re
> > > +        this_os = d.getVar('TARGET_OS')
> > > +        if not re.match(need_os, this_os):
> > > +            raise bb.parse.SkipRecipe("incompatible with os %s
> > > (not
> > > in COMPATIBLE_OS)" % this_os)
> > > +
> > >      need_machine = d.getVar('COMPATIBLE_MACHINE')
> > >      if need_machine and not d.getVar('PARSE_ALL_RECIPES',
> > > False):
> > >          import re
> > > diff --git a/meta/conf/documentation.conf
> > > b/meta/conf/documentation.conf
> > > index 0b21d1f63e..7a87540f43 100644
> > > --- a/meta/conf/documentation.conf
> > > +++ b/meta/conf/documentation.conf
> > > @@ -111,6 +111,7 @@ COMBINED_FEATURES[doc] = "A set of features
> > > common between MACHINE_FEATURES and
> > >  COMMON_LICENSE_DIR[doc] = "Points to meta/files/common-licenses
> > > in
> > > the Source Directory, which is where generic license files
> > > reside."
> > >  COMPATIBLE_HOST[doc] = "A regular expression that resolves to
> > > one or
> > > more hosts (when the recipe is native) or one or more targets
> > > (when
> > > the recipe is non-native) with which a recipe is compatible."
> > 
> > How does COMPATIBLE_OS compare to COMPATIBLE_HOST (documented
> > above)?
> 
> HOST_SYS="aarch64-oe-linux"
> 
> So the equivalent to the COMPATIBLE_OS would be:
> 
> COMPATIBLE_HOST ?= ".*-linux.*"
> 
> So it COULD be used in most cases.
> 
> When I tried this before though, I could get it to work in a
> multiconfig
> setting.
> 
> Specifically what I tried was setting, in my local.conf:
> 
> COMPATIBLE_HOST ?= ".*-linux${LIBCEXTENSION}${ABIEXTENSION}"
> 
> And then in the recipes:
> 
> COMPATIBLE_HOST = ".*-elf"
> COMPATIBLE_HOST_arm = ".*-eabi"
> 
> But then the second one matches the linux ABIEXTENSION and broke..
> 
> So really the issue is that I -only- ever want to match on the OS
> part of
> the "SYS" string.  And done to the multiple natures of the '-', it's
> difficult to get right.
> 
> I didn't try something complex like: [^-]-[^-]-eabi

From what I remember, COMPATIBLE_HOST has full re support so something
should be possible...

Cheers,

Richard

Re: [OE-core] [v2 PATCH] base.bbclass: Add COMPATIBLE_OS, useful for non-Linux target recipes

[Mark Hatle]

I talked with RP off the list and got this working.

Below is what I ended up doing to prevent the default packages from working with
baremetal, unless they were configured to.

In my baremetal distro.conf:

COMPATOS = ""
COMPATOS_class-target = ".*-linux${LIBCEXTENSION}${ABIEXTENSION}"
COMPATIBLE_HOST ?= "${COMPATOS}"

Then in each of the baremetal -only- recipes:

COMPATIBLE_HOST = ".*-elf"
COMPATIBLE_HOST_arm = "[^-]*-[^-]*-eabi"

In the recipes for BOTH baremetal and Linux:

COMPATIBLE_HOST = "${HOST_SYS}"

(Setting to "" should work as well..)

--Mark

On 3/26/20 12:05 PM, Mark Hatle wrote:
>> On Thu, 2020-03-26 at 10:25 -0500, Mark Hatle wrote:
>>> Add the ability to generate non-Linux based components, such as
>>> baremetal, FreeRTOS, Zypher, etc, exist.  There is a need for a way
>>> to take components as specific to a given OS.
>>>
>>> This is especially important in a multiconfig build where you may be
>>> targeting different OSes which different sets of recipes in a common
>>> set of layers.
>>>
>>> Setting a default of (matching bitbake.conf's default TARGET_OS):
>>>
>>>    COMPATIBLE_OS ?= "linux${LIBCEXTENSION}${ABIEXTENSION}"
>>>
>>> will allow all existing recipes to be tagged as Linux specific, so
>>> only non-Linux recipes would need to be tagged with specific
>>> compatibility.
>>>
>>> For a baremetal recipes, the following was used to test this code:
>>>
>>>    COMPATIBLE_OS = "elf"
>>>    COMPATIBLE_OS_arm = "eabi"
>>>
>>> Signed-off-by: Mark Hatle <mark.hatle@kernel.crashing.org>
>>> ---
>>> V2:
>>> Only difference to V1 is typograhic fixes to the commit message.
>>>
>>>  meta/classes/base.bbclass    | 7 +++++++
>>>  meta/conf/documentation.conf | 1 +
>>>  2 files changed, 8 insertions(+)
>>>
>>> diff --git a/meta/classes/base.bbclass b/meta/classes/base.bbclass
>>> index 45f9435fd8..c123c5dc50 100644
>>> --- a/meta/classes/base.bbclass
>>> +++ b/meta/classes/base.bbclass
>>> @@ -509,6 +509,13 @@ python () {
>>>          d.setVarFlag('do_devshell', 'fakeroot', '1')
>>>          d.appendVarFlag('do_devshell', 'depends', '
>>> virtual/fakeroot-native:do_populate_sysroot')
>>>
>>> +    need_os = d.getVar('COMPATIBLE_OS')
>>> +    if need_os and not d.getVar('PARSE_ALL_RECIPES', False):
>>> +        import re
>>> +        this_os = d.getVar('TARGET_OS')
>>> +        if not re.match(need_os, this_os):
>>> +            raise bb.parse.SkipRecipe("incompatible with os %s (not
>>> in COMPATIBLE_OS)" % this_os)
>>> +
>>>      need_machine = d.getVar('COMPATIBLE_MACHINE')
>>>      if need_machine and not d.getVar('PARSE_ALL_RECIPES', False):
>>>          import re
>>> diff --git a/meta/conf/documentation.conf
>>> b/meta/conf/documentation.conf
>>> index 0b21d1f63e..7a87540f43 100644
>>> --- a/meta/conf/documentation.conf
>>> +++ b/meta/conf/documentation.conf
>>> @@ -111,6 +111,7 @@ COMBINED_FEATURES[doc] = "A set of features
>>> common between MACHINE_FEATURES and
>>>  COMMON_LICENSE_DIR[doc] = "Points to meta/files/common-licenses in
>>> the Source Directory, which is where generic license files reside."
>>>  COMPATIBLE_HOST[doc] = "A regular expression that resolves to one or
>>> more hosts (when the recipe is native) or one or more targets (when
>>> the recipe is non-native) with which a recipe is compatible."
>>
>> How does COMPATIBLE_OS compare to COMPATIBLE_HOST (documented above)?
> 
> HOST_SYS="aarch64-oe-linux"
> 
> So the equivalent to the COMPATIBLE_OS would be:
> 
> COMPATIBLE_HOST ?= ".*-linux.*"
> 
> So it COULD be used in most cases.
> 
> When I tried this before though, I could get it to work in a multiconfig
> setting.
> 
> Specifically what I tried was setting, in my local.conf:
> 
> COMPATIBLE_HOST ?= ".*-linux${LIBCEXTENSION}${ABIEXTENSION}"
> 
> And then in the recipes:
> 
> COMPATIBLE_HOST = ".*-elf"
> COMPATIBLE_HOST_arm = ".*-eabi"
> 
> But then the second one matches the linux ABIEXTENSION and broke..
> 
> So really the issue is that I -only- ever want to match on the OS part of
> the "SYS" string.  And done to the multiple natures of the '-', it's
> difficult to get right.
> 
> I didn't try something complex like: [^-]-[^-]-eabi
> 
> --Mark
> 
>> Cheers
>>
>> Richard
>>
> 
> 
> 
>