Ownership issue in package contents

Ownership issue in package contents

[Mario Domenech Goulart]

Hi,

I'm observing an issue with directory ownership in package
contents.  Not sure if I'm overlooking something really basic or
if I hit a bug.

Here's a minimal recipe to illustrate the problem:

$ cat foo.bb
LICENSE = "CLOSED"

inherit useradd

USERADD_PACKAGES =  "${PN}"
USERADD_PARAM_${PN} = "foo"

do_install() {
    install -d -m 755 ${D}${libdir}/foo
        touch ${D}${libdir}/foo/bar
            chown -R foo:foo ${D}${libdir}/foo
            }

Here's the package content:

$ dpkg -c foo_1.0-r0.0_cortexa9hf-vfp-neon.ipk
drwxrwxrwx root/root         0 2015-03-27 13:37 ./
drwxr-xr-x root/root         0 2015-03-27 13:37 ./usr/
drwxr-xr-x root/root         0 2015-03-27 13:37 ./usr/lib/
drwxr-xr-x root/root         0 2015-03-27 13:37 ./usr/lib/foo/
-rw-r--r-- foo/foo           0 2015-03-27 13:37 ./usr/lib/foo/bar


Note that, although I run "chown -R foo:foo ${D}${libdir}/foo" in
the recipe, ./usr/lib/foo/ in the package is owned by root.
However, its content has the right ownership.

I also generated and checked the rpm package and observed the
same issue.

Best wishes.
Mario
-- 
http://www.ossystems.com.br

Re: Ownership issue in package contents

[Otavio Salvador]

Hello Mario,

On Fri, Mar 27, 2015 at 2:31 PM, Mario Domenech Goulart
<mario@ossystems.com.br> wrote:
...
> I also generated and checked the rpm package and observed the
> same issue.
...

I tested the foo recipe on master (OE-Core:2d923d6) and I see the same behavior.

-- 
Otavio Salvador                             O.S. Systems
http://www.ossystems.com.br        http://code.ossystems.com.br
Mobile: +55 (53) 9981-7854            Mobile: +1 (347) 903-9750

Re: Ownership issue in package contents

[Burton, Ross]

[-- Attachment #1: Type: text/plain, Size: 344 bytes --]

On 27 March 2015 at 17:31, Mario Domenech Goulart <mario@ossystems.com.br>
wrote:

> Note that, although I run "chown -R foo:foo ${D}${libdir}/foo" in
> the recipe, ./usr/lib/foo/ in the package is owned by root.
> However, its content has the right ownership.
>

Looks like a bug in pseudo to me, can you file a bug for that?

Ross

[-- Attachment #2: Type: text/html, Size: 762 bytes --]

Re: Ownership issue in package contents

[Mario Domenech Goulart]

Hi Ross,

On Tue, 31 Mar 2015 14:50:06 +0100 "Burton, Ross" <ross.burton@intel.com> wrote:

> On 27 March 2015 at 17:31, Mario Domenech Goulart <mario@ossystems.com.br> wrote:
>
>     Note that, although I run "chown -R foo:foo ${D}${libdir}/foo" in
>     the recipe, ./usr/lib/foo/ in the package is owned by root.
>     However, its content has the right ownership.
>
> Looks like a bug in pseudo to me, can you file a bug for that?

Sure.  Filed #7554.

Best wishes.
Mario
-- 
http://www.ossystems.com.br

Re: Ownership issue in package contents

[Mark Hatle]

On 3/31/15 12:20 PM, Mario Domenech Goulart wrote:
> Hi Ross,
> 
> On Tue, 31 Mar 2015 14:50:06 +0100 "Burton, Ross" <ross.burton@intel.com> wrote:
> 
>> On 27 March 2015 at 17:31, Mario Domenech Goulart <mario@ossystems.com.br> wrote:
>>
>>     Note that, although I run "chown -R foo:foo ${D}${libdir}/foo" in
>>     the recipe, ./usr/lib/foo/ in the package is owned by root.
>>     However, its content has the right ownership.
>>
>> Looks like a bug in pseudo to me, can you file a bug for that?
> 
> Sure.  Filed #7554.

I'd suggest you look at meta/classes/package.bbclass "fixup_perms" function.

The ${D}${libdir} (and above) are "corrected" to be 'root:root' by this
function.  I don't know why 'foo' would be, but if it's a standard defined
variable -- or if 'directory walking' is enabled it could end up doing this as well.

The control file for this is in meta/files/fs-perms.txt (unless otherwise
defined by a distribution or other configuration file.)

Format of the file is:

# The format of this file
#
#<path> <mode>  <uid>   <gid>   <walk>  <fmode> <fuid>  <fgid>
#
# or
#
#<path> link <target>
#
# <path>: directory path
# <mode>: mode for directory
# <uid>:  uid for directory
# <gid>:  gid for directory
# <walk>: recursively walk the directory?  true or false
# <fmode>: if walking, new mode for files
# <fuid>:  if walking, new uid for files
# <fgid>:  if walking, new gid for files
# <target>: turn the directory into a symlink point to target

The default is:

base_prefix	0755 root root false - - -
prefix		0755 root root false - - -
exec_prefix	0755 root root false - - -
base_bindir	0755 root root false - - -
base_sbindir	0755 root root false - - -
base_libdir	0755 root root false - - -
datadir		0755 root root false - - -
sysconfdir	0755 root root false - - -
servicedir	0755 root root false - - -
sharedstatedir	0755 root root false - - -
localstatedir	0755 root root false - - -
infodir		0755 root root false - - -
mandir		0755 root root false - - -
docdir		0755 root root false - - -
bindir		0755 root root false - - -
sbindir		0755 root root false - - -
libexecdir	0755 root root false - - -
libdir		0755 root root false - - -
includedir	0755 root root false - - -
oldincludedir	0755 root root false - - -

The easiest way to debug all of this is to edit meta/classes/package.bbclass,
find the following lines:

    # Debug -- list out in-memory table
    #for dir in fs_perms_table:
    #    bb.note("Fixup Perms: %s: %s" % (dir, str(fs_perms_table[dir])))

remove the '#' from the 'for' and following lines, then inspect the table from
the build logs.

Similarly, you can trace the actual fixups, by searching for def fix_perms(path,
mode, uid, gid, dir):, and removing the comment from the two bb.note entries.

This should give you an idea if this code is replacing your value with it's own.

(I give this a VERY low probability that it's a bug in pseudo.)

--Mark

> Best wishes.
> Mario
> 

Re: Ownership issue in package contents

[Burton, Ross]

[-- Attachment #1: Type: text/plain, Size: 322 bytes --]

On 31 March 2015 at 19:23, Mark Hatle <mark.hatle@windriver.com> wrote:

> The easiest way to debug all of this is to edit
> meta/classes/package.bbclass,
> find the following lines:
>

Or change the files that are chowned to exist somewhere else that doesn't
get touched by fixup, like /testing/foo/bar.

Ross

[-- Attachment #2: Type: text/html, Size: 735 bytes --]

Re: Ownership issue in package contents

[Otavio Salvador]

On Tue, Mar 31, 2015 at 5:12 PM, Burton, Ross <ross.burton@intel.com> wrote:
>
> On 31 March 2015 at 19:23, Mark Hatle <mark.hatle@windriver.com> wrote:
>>
>> The easiest way to debug all of this is to edit
>> meta/classes/package.bbclass,
>> find the following lines:
>
> Or change the files that are chowned to exist somewhere else that doesn't
> get touched by fixup, like /testing/foo/bar.

This seems to be a design flaw. I understand we ought to have
somewhere to specify a default however the useradd class should
somehow instrument the code for those recipes to be able to override
this.

-- 
Otavio Salvador                             O.S. Systems
http://www.ossystems.com.br        http://code.ossystems.com.br
Mobile: +55 (53) 9981-7854            Mobile: +1 (347) 903-9750

Re: Ownership issue in package contents

[Mario Domenech Goulart]

Hi Mark,

On Tue, 31 Mar 2015 13:23:00 -0500 Mark Hatle <mark.hatle@windriver.com> wrote:

> On 3/31/15 12:20 PM, Mario Domenech Goulart wrote:
>> 
>> On Tue, 31 Mar 2015 14:50:06 +0100 "Burton, Ross" <ross.burton@intel.com> wrote:
>> 
>>> On 27 March 2015 at 17:31, Mario Domenech Goulart <mario@ossystems.com.br> wrote:
>>>
>>>     Note that, although I run "chown -R foo:foo ${D}${libdir}/foo" in
>>>     the recipe, ./usr/lib/foo/ in the package is owned by root.
>>>     However, its content has the right ownership.
>>>
>>> Looks like a bug in pseudo to me, can you file a bug for that?
>> 
>> Sure.  Filed #7554.
>
> I'd suggest you look at meta/classes/package.bbclass "fixup_perms" function.
>
> The ${D}${libdir} (and above) are "corrected" to be 'root:root' by this
> function.  I don't know why 'foo' would be, but if it's a standard defined
> variable -- or if 'directory walking' is enabled it could end up doing this as well.
>
> The control file for this is in meta/files/fs-perms.txt (unless otherwise
> defined by a distribution or other configuration file.)

Thanks a lot.  You seem to have guided me exactly to what causes the
issue.

>
> Format of the file is:
>
> # The format of this file
> #
> #<path> <mode>  <uid>   <gid>   <walk>  <fmode> <fuid>  <fgid>
...
>
> The default is:
...
> libexecdir	0755 root root false - - -
...

This variable seems to be the cause of problems:

$ bitbake -e foo | grep libexecdir=
export libexecdir="/usr/lib/foo"

As far as I understand, package.bbclass may use a user-configured
permissions table (via FILESYSTEM_PERMS_TABLES), but I'm not sure if
this is the right "fix" for the case in question.  I'd have to hardcode
the owner of /usr/lib/foo to be "foo", but foo may not be available when
packaging other recipes.

Should libexecdir actually be in the `target_path_vars' variable
(package.bbclass)?

Best wishes.
Mario
-- 
http://www.ossystems.com.br

Re: Ownership issue in package contents

[Mark Hatle]

On 3/31/15 3:33 PM, Mario Domenech Goulart wrote:
> Hi Mark,
> 
> On Tue, 31 Mar 2015 13:23:00 -0500 Mark Hatle <mark.hatle@windriver.com> wrote:
> 
>> On 3/31/15 12:20 PM, Mario Domenech Goulart wrote:
>>>
>>> On Tue, 31 Mar 2015 14:50:06 +0100 "Burton, Ross" <ross.burton@intel.com> wrote:
>>>
>>>> On 27 March 2015 at 17:31, Mario Domenech Goulart <mario@ossystems.com.br> wrote:
>>>>
>>>>     Note that, although I run "chown -R foo:foo ${D}${libdir}/foo" in
>>>>     the recipe, ./usr/lib/foo/ in the package is owned by root.
>>>>     However, its content has the right ownership.
>>>>
>>>> Looks like a bug in pseudo to me, can you file a bug for that?
>>>
>>> Sure.  Filed #7554.
>>
>> I'd suggest you look at meta/classes/package.bbclass "fixup_perms" function.
>>
>> The ${D}${libdir} (and above) are "corrected" to be 'root:root' by this
>> function.  I don't know why 'foo' would be, but if it's a standard defined
>> variable -- or if 'directory walking' is enabled it could end up doing this as well.
>>
>> The control file for this is in meta/files/fs-perms.txt (unless otherwise
>> defined by a distribution or other configuration file.)
> 
> Thanks a lot.  You seem to have guided me exactly to what causes the
> issue.
> 
>>
>> Format of the file is:
>>
>> # The format of this file
>> #
>> #<path> <mode>  <uid>   <gid>   <walk>  <fmode> <fuid>  <fgid>
> ...
>>
>> The default is:
> ...
>> libexecdir	0755 root root false - - -
> ...
> 
> This variable seems to be the cause of problems:
> 
> $ bitbake -e foo | grep libexecdir=
> export libexecdir="/usr/lib/foo"
> 
> As far as I understand, package.bbclass may use a user-configured
> permissions table (via FILESYSTEM_PERMS_TABLES), but I'm not sure if
> this is the right "fix" for the case in question.  I'd have to hardcode
> the owner of /usr/lib/foo to be "foo", but foo may not be available when
> packaging other recipes.

Ok, good this answers the question as to "why" it's happening.  You can easily
fix this by adding a configuration specific fs-perms.txt file (can name it
anything) that overrides that setting and add it to the FILESYSTEM_PERMS_TABLES.
 You can do this globally in a layer, a distribution or even just a recipe.

In your recipe you can likely do something like:

FILESYSTEM_PERMS_TABLES ?= "files/fs-perms.txt"
FILESYSTEM_PERMS_TABLES += "${THISDIR}/files/recipe-perms.txt"

(Do the ?= first in case it's already set by someone else, then add your recipe
specific perms later)

Contents of the "${THISDIR}/files/recipe-perms.txt":

${libexecdir} 0755 myuid mygid true - myuid mygid

Then you can even skip the chown -R, as the system will do it for you.

> Should libexecdir actually be in the `target_path_vars' variable
> (package.bbclass)?

That is a good question, and something likely worthy of investigating.  Perhaps
removing it from the default set is the correct general purpose change.

I wouldn't do it on an existing release, but its worth considering at the
beginning of a new release.

--Mark

> Best wishes.
> Mario
> 

Re: Ownership issue in package contents

[Mario Domenech Goulart]

On Tue, 31 Mar 2015 15:51:50 -0500 Mark Hatle <mark.hatle@windriver.com> wrote:

> On 3/31/15 3:33 PM, Mario Domenech Goulart wrote:
>> 
>> On Tue, 31 Mar 2015 13:23:00 -0500 Mark Hatle <mark.hatle@windriver.com> wrote:
>> 
>>> On 3/31/15 12:20 PM, Mario Domenech Goulart wrote:
>>>>
>>>> On Tue, 31 Mar 2015 14:50:06 +0100 "Burton, Ross" <ross.burton@intel.com> wrote:
>>>>
>>>>> On 27 March 2015 at 17:31, Mario Domenech Goulart <mario@ossystems.com.br> wrote:
>>>>>
>>>>>     Note that, although I run "chown -R foo:foo ${D}${libdir}/foo" in
>>>>>     the recipe, ./usr/lib/foo/ in the package is owned by root.
>>>>>     However, its content has the right ownership.
>>>>>
>>>>> Looks like a bug in pseudo to me, can you file a bug for that?
>>>>
>>>> Sure.  Filed #7554.
>>>
>>> I'd suggest you look at meta/classes/package.bbclass "fixup_perms" function.
>>>
>>> The ${D}${libdir} (and above) are "corrected" to be 'root:root' by this
>>> function.  I don't know why 'foo' would be, but if it's a standard defined
>>> variable -- or if 'directory walking' is enabled it could end up doing this as well.
>>>
>>> The control file for this is in meta/files/fs-perms.txt (unless otherwise
>>> defined by a distribution or other configuration file.)
>> 
>> Thanks a lot.  You seem to have guided me exactly to what causes the
>> issue.
>> 
>>>
>>> Format of the file is:
>>>
>>> # The format of this file
>>> #
>>> #<path> <mode>  <uid>   <gid>   <walk>  <fmode> <fuid>  <fgid>
>> ...
>>>
>>> The default is:
>> ...
>>> libexecdir	0755 root root false - - -
>> ...
>> 
>> This variable seems to be the cause of problems:
>> 
>> $ bitbake -e foo | grep libexecdir=
>> export libexecdir="/usr/lib/foo"
>> 
>> As far as I understand, package.bbclass may use a user-configured
>> permissions table (via FILESYSTEM_PERMS_TABLES), but I'm not sure if
>> this is the right "fix" for the case in question.  I'd have to hardcode
>> the owner of /usr/lib/foo to be "foo", but foo may not be available when
>> packaging other recipes.
>
> Ok, good this answers the question as to "why" it's happening.  You can easily
> fix this by adding a configuration specific fs-perms.txt file (can name it
> anything) that overrides that setting and add it to the FILESYSTEM_PERMS_TABLES.
>  You can do this globally in a layer, a distribution or even just a recipe.
>
> In your recipe you can likely do something like:
>
> FILESYSTEM_PERMS_TABLES ?= "files/fs-perms.txt"
> FILESYSTEM_PERMS_TABLES += "${THISDIR}/files/recipe-perms.txt"
>
> (Do the ?= first in case it's already set by someone else, then add your recipe
> specific perms later)
>
> Contents of the "${THISDIR}/files/recipe-perms.txt":
>
> ${libexecdir} 0755 myuid mygid true - myuid mygid
>
> Then you can even skip the chown -R, as the system will do it for you.

I actually had tried that, but I got errors -- probably because the
ownership will be set for each package that installs ${libexecdir}, and
which is processed before the recipe that actually creates the
user/group specified for ${libexecdir} in the file pointed by
FILESYSTEM_PERMS_TABLES.

>> Should libexecdir actually be in the `target_path_vars' variable
>> (package.bbclass)?
>
> That is a good question, and something likely worthy of investigating.  Perhaps
> removing it from the default set is the correct general purpose change.
>
> I wouldn't do it on an existing release, but its worth considering at the
> beginning of a new release.

Best wishes.
Mario
-- 
http://www.ossystems.com.br

Re: Ownership issue in package contents

[Mark Hatle]

On 3/31/15 4:01 PM, Mario Domenech Goulart wrote:
> On Tue, 31 Mar 2015 15:51:50 -0500 Mark Hatle <mark.hatle@windriver.com> wrote:
> 
>> On 3/31/15 3:33 PM, Mario Domenech Goulart wrote:
>>>
>>> On Tue, 31 Mar 2015 13:23:00 -0500 Mark Hatle <mark.hatle@windriver.com> wrote:
>>>
>>>> On 3/31/15 12:20 PM, Mario Domenech Goulart wrote:
>>>>>
>>>>> On Tue, 31 Mar 2015 14:50:06 +0100 "Burton, Ross" <ross.burton@intel.com> wrote:
>>>>>
>>>>>> On 27 March 2015 at 17:31, Mario Domenech Goulart <mario@ossystems.com.br> wrote:
>>>>>>
>>>>>>     Note that, although I run "chown -R foo:foo ${D}${libdir}/foo" in
>>>>>>     the recipe, ./usr/lib/foo/ in the package is owned by root.
>>>>>>     However, its content has the right ownership.
>>>>>>
>>>>>> Looks like a bug in pseudo to me, can you file a bug for that?
>>>>>
>>>>> Sure.  Filed #7554.
>>>>
>>>> I'd suggest you look at meta/classes/package.bbclass "fixup_perms" function.
>>>>
>>>> The ${D}${libdir} (and above) are "corrected" to be 'root:root' by this
>>>> function.  I don't know why 'foo' would be, but if it's a standard defined
>>>> variable -- or if 'directory walking' is enabled it could end up doing this as well.
>>>>
>>>> The control file for this is in meta/files/fs-perms.txt (unless otherwise
>>>> defined by a distribution or other configuration file.)
>>>
>>> Thanks a lot.  You seem to have guided me exactly to what causes the
>>> issue.
>>>
>>>>
>>>> Format of the file is:
>>>>
>>>> # The format of this file
>>>> #
>>>> #<path> <mode>  <uid>   <gid>   <walk>  <fmode> <fuid>  <fgid>
>>> ...
>>>>
>>>> The default is:
>>> ...
>>>> libexecdir	0755 root root false - - -
>>> ...
>>>
>>> This variable seems to be the cause of problems:
>>>
>>> $ bitbake -e foo | grep libexecdir=
>>> export libexecdir="/usr/lib/foo"
>>>
>>> As far as I understand, package.bbclass may use a user-configured
>>> permissions table (via FILESYSTEM_PERMS_TABLES), but I'm not sure if
>>> this is the right "fix" for the case in question.  I'd have to hardcode
>>> the owner of /usr/lib/foo to be "foo", but foo may not be available when
>>> packaging other recipes.
>>
>> Ok, good this answers the question as to "why" it's happening.  You can easily
>> fix this by adding a configuration specific fs-perms.txt file (can name it
>> anything) that overrides that setting and add it to the FILESYSTEM_PERMS_TABLES.
>>  You can do this globally in a layer, a distribution or even just a recipe.
>>
>> In your recipe you can likely do something like:
>>
>> FILESYSTEM_PERMS_TABLES ?= "files/fs-perms.txt"
>> FILESYSTEM_PERMS_TABLES += "${THISDIR}/files/recipe-perms.txt"
>>
>> (Do the ?= first in case it's already set by someone else, then add your recipe
>> specific perms later)
>>
>> Contents of the "${THISDIR}/files/recipe-perms.txt":
>>
>> ${libexecdir} 0755 myuid mygid true - myuid mygid
>>
>> Then you can even skip the chown -R, as the system will do it for you.
> 
> I actually had tried that, but I got errors -- probably because the
> ownership will be set for each package that installs ${libexecdir}, and
> which is processed before the recipe that actually creates the
> user/group specified for ${libexecdir} in the file pointed by
> FILESYSTEM_PERMS_TABLES.

This is a bug then.  The owner/group correction is supposed to be made AFTER the
user/groups have been added to the system (sysroot) via the adduser.  THAT is a
bug that IMHO should be fixed sooner, rather then later.

It might be as simple as the install sysroot (${D}) configuration with pseudo
isn't pointing to the right /etc/passwd and /etc/group.  I believe it should be
pointing to the one in the regular sysroot repository.

--Mark

>>> Should libexecdir actually be in the `target_path_vars' variable
>>> (package.bbclass)?
>>
>> That is a good question, and something likely worthy of investigating.  Perhaps
>> removing it from the default set is the correct general purpose change.
>>
>> I wouldn't do it on an existing release, but its worth considering at the
>> beginning of a new release.
> 
> Best wishes.
> Mario
> 

Re: Ownership issue in package contents

[Mario Domenech Goulart]

On Tue, 31 Mar 2015 16:09:42 -0500 Mark Hatle <mark.hatle@windriver.com> wrote:

> On 3/31/15 4:01 PM, Mario Domenech Goulart wrote:
>> On Tue, 31 Mar 2015 15:51:50 -0500 Mark Hatle <mark.hatle@windriver.com> wrote:
>> 
>>> On 3/31/15 3:33 PM, Mario Domenech Goulart wrote:
>>>>
>>>> On Tue, 31 Mar 2015 13:23:00 -0500 Mark Hatle <mark.hatle@windriver.com> wrote:
>>>>
>>>>> On 3/31/15 12:20 PM, Mario Domenech Goulart wrote:
>>>>>>
>>>>>> On Tue, 31 Mar 2015 14:50:06 +0100 "Burton, Ross" <ross.burton@intel.com> wrote:
>>>>>>
>>>>>>> On 27 March 2015 at 17:31, Mario Domenech Goulart <mario@ossystems.com.br> wrote:
>>>>>>>
>>>>>>>     Note that, although I run "chown -R foo:foo ${D}${libdir}/foo" in
>>>>>>>     the recipe, ./usr/lib/foo/ in the package is owned by root.
>>>>>>>     However, its content has the right ownership.
>>>>>>>
>>>>>>> Looks like a bug in pseudo to me, can you file a bug for that?
>>>>>>
>>>>>> Sure.  Filed #7554.
>>>>>
>>>>> I'd suggest you look at meta/classes/package.bbclass "fixup_perms" function.
>>>>>
>>>>> The ${D}${libdir} (and above) are "corrected" to be 'root:root' by this
>>>>> function.  I don't know why 'foo' would be, but if it's a standard defined
>>>>> variable -- or if 'directory walking' is enabled it could end up doing this as well.
>>>>>
>>>>> The control file for this is in meta/files/fs-perms.txt (unless otherwise
>>>>> defined by a distribution or other configuration file.)
>>>>
>>>> Thanks a lot.  You seem to have guided me exactly to what causes the
>>>> issue.
>>>>
>>>>>
>>>>> Format of the file is:
>>>>>
>>>>> # The format of this file
>>>>> #
>>>>> #<path> <mode>  <uid>   <gid>   <walk>  <fmode> <fuid>  <fgid>
>>>> ...
>>>>>
>>>>> The default is:
>>>> ...
>>>>> libexecdir	0755 root root false - - -
>>>> ...
>>>>
>>>> This variable seems to be the cause of problems:
>>>>
>>>> $ bitbake -e foo | grep libexecdir=
>>>> export libexecdir="/usr/lib/foo"
>>>>
>>>> As far as I understand, package.bbclass may use a user-configured
>>>> permissions table (via FILESYSTEM_PERMS_TABLES), but I'm not sure if
>>>> this is the right "fix" for the case in question.  I'd have to hardcode
>>>> the owner of /usr/lib/foo to be "foo", but foo may not be available when
>>>> packaging other recipes.
>>>
>>> Ok, good this answers the question as to "why" it's happening.  You can easily
>>> fix this by adding a configuration specific fs-perms.txt file (can name it
>>> anything) that overrides that setting and add it to the FILESYSTEM_PERMS_TABLES.
>>>  You can do this globally in a layer, a distribution or even just a recipe.
>>>
>>> In your recipe you can likely do something like:
>>>
>>> FILESYSTEM_PERMS_TABLES ?= "files/fs-perms.txt"
>>> FILESYSTEM_PERMS_TABLES += "${THISDIR}/files/recipe-perms.txt"
>>>
>>> (Do the ?= first in case it's already set by someone else, then add your recipe
>>> specific perms later)
>>>
>>> Contents of the "${THISDIR}/files/recipe-perms.txt":
>>>
>>> ${libexecdir} 0755 myuid mygid true - myuid mygid
>>>
>>> Then you can even skip the chown -R, as the system will do it for you.
>> 
>> I actually had tried that, but I got errors -- probably because the
>> ownership will be set for each package that installs ${libexecdir}, and
>> which is processed before the recipe that actually creates the
>> user/group specified for ${libexecdir} in the file pointed by
>> FILESYSTEM_PERMS_TABLES.
>
> This is a bug then.  The owner/group correction is supposed to be made AFTER the
> user/groups have been added to the system (sysroot) via the adduser.  THAT is a
> bug that IMHO should be fixed sooner, rather then later.
>
> It might be as simple as the install sysroot (${D}) configuration with pseudo
> isn't pointing to the right /etc/passwd and /etc/group.  I believe it should be
> pointing to the one in the regular sysroot repository.

I should also have mentioned that initially I set
FILESYSTEM_PERMS_TABLES globally.

Now I set it in the foo recipe, but still get errors:

----------------------------8<---------------------------------
ERROR: Error executing a python function in .../foo.bb:

The stack trace of python calls that resulted in this exception/failure was:
File: 'fixup_perms', lineno: 231, function: <module>
     0227:                    each_file = os.path.join(root, f)
     0228:                    fix_perms(each_file, fs_perms_table[dir].fmode, fs_perms_table[dir].fuid, fs_perms_table[dir].fgid, dir)
     0229:
     0230:
 *** 0231:fixup_perms(d)
     0232:
File: 'fixup_perms', lineno: 173, function: fixup_perms
     0169:                if len(lsplit) != 8 and not (len(lsplit) == 3 and lsplit[1].lower() == "link"):
     0170:                    msg = "Fixup perms: %s invalid line: %s" % (conf, line)
     0171:                    package_qa_handle_error("perm-line", msg, d)
     0172:                    continue
 *** 0173:                entry = fs_perms_entry(d.expand(line))
     0174:                if entry and entry.path:
     0175:                    fs_perms_table[entry.path] = entry
     0176:            f.close()
     0177:
File: 'fixup_perms', lineno: 32, function: __init__
  File "fixup_perms", line 32, in __init__

File: 'fixup_perms', lineno: 43, function: _setdir
  File "fixup_perms", line 43, in _setdir

File: 'fixup_perms', lineno: 67, function: _procuid
  File "fixup_perms", line 67, in _procuid

Exception: KeyError: 'getpwnam(): name not found: foo'
---------------------------->8---------------------------------

I still have the chown line in do_install, and that seems to "work"
(i.e., it doesn't cause errors).  The error above seems be caused by
something in do_package (fix_perms, specifically) that is not using the
proper authentication database.

Best wishes.
Mario
-- 
http://www.ossystems.com.br

Re: Ownership issue in package contents

[Mark Hatle]

On 3/31/15 4:21 PM, Mario Domenech Goulart wrote:
> On Tue, 31 Mar 2015 16:09:42 -0500 Mark Hatle <mark.hatle@windriver.com> wrote:
> 
>> On 3/31/15 4:01 PM, Mario Domenech Goulart wrote:
>>> On Tue, 31 Mar 2015 15:51:50 -0500 Mark Hatle <mark.hatle@windriver.com> wrote:
>>>
>>>> On 3/31/15 3:33 PM, Mario Domenech Goulart wrote:
>>>>>
>>>>> On Tue, 31 Mar 2015 13:23:00 -0500 Mark Hatle <mark.hatle@windriver.com> wrote:
>>>>>
>>>>>> On 3/31/15 12:20 PM, Mario Domenech Goulart wrote:
>>>>>>>
>>>>>>> On Tue, 31 Mar 2015 14:50:06 +0100 "Burton, Ross" <ross.burton@intel.com> wrote:
>>>>>>>
>>>>>>>> On 27 March 2015 at 17:31, Mario Domenech Goulart <mario@ossystems.com.br> wrote:
>>>>>>>>
>>>>>>>>     Note that, although I run "chown -R foo:foo ${D}${libdir}/foo" in
>>>>>>>>     the recipe, ./usr/lib/foo/ in the package is owned by root.
>>>>>>>>     However, its content has the right ownership.
>>>>>>>>
>>>>>>>> Looks like a bug in pseudo to me, can you file a bug for that?
>>>>>>>
>>>>>>> Sure.  Filed #7554.
>>>>>>
>>>>>> I'd suggest you look at meta/classes/package.bbclass "fixup_perms" function.
>>>>>>
>>>>>> The ${D}${libdir} (and above) are "corrected" to be 'root:root' by this
>>>>>> function.  I don't know why 'foo' would be, but if it's a standard defined
>>>>>> variable -- or if 'directory walking' is enabled it could end up doing this as well.
>>>>>>
>>>>>> The control file for this is in meta/files/fs-perms.txt (unless otherwise
>>>>>> defined by a distribution or other configuration file.)
>>>>>
>>>>> Thanks a lot.  You seem to have guided me exactly to what causes the
>>>>> issue.
>>>>>
>>>>>>
>>>>>> Format of the file is:
>>>>>>
>>>>>> # The format of this file
>>>>>> #
>>>>>> #<path> <mode>  <uid>   <gid>   <walk>  <fmode> <fuid>  <fgid>
>>>>> ...
>>>>>>
>>>>>> The default is:
>>>>> ...
>>>>>> libexecdir	0755 root root false - - -
>>>>> ...
>>>>>
>>>>> This variable seems to be the cause of problems:
>>>>>
>>>>> $ bitbake -e foo | grep libexecdir=
>>>>> export libexecdir="/usr/lib/foo"
>>>>>
>>>>> As far as I understand, package.bbclass may use a user-configured
>>>>> permissions table (via FILESYSTEM_PERMS_TABLES), but I'm not sure if
>>>>> this is the right "fix" for the case in question.  I'd have to hardcode
>>>>> the owner of /usr/lib/foo to be "foo", but foo may not be available when
>>>>> packaging other recipes.
>>>>
>>>> Ok, good this answers the question as to "why" it's happening.  You can easily
>>>> fix this by adding a configuration specific fs-perms.txt file (can name it
>>>> anything) that overrides that setting and add it to the FILESYSTEM_PERMS_TABLES.
>>>>  You can do this globally in a layer, a distribution or even just a recipe.
>>>>
>>>> In your recipe you can likely do something like:
>>>>
>>>> FILESYSTEM_PERMS_TABLES ?= "files/fs-perms.txt"
>>>> FILESYSTEM_PERMS_TABLES += "${THISDIR}/files/recipe-perms.txt"
>>>>
>>>> (Do the ?= first in case it's already set by someone else, then add your recipe
>>>> specific perms later)
>>>>
>>>> Contents of the "${THISDIR}/files/recipe-perms.txt":
>>>>
>>>> ${libexecdir} 0755 myuid mygid true - myuid mygid
>>>>
>>>> Then you can even skip the chown -R, as the system will do it for you.
>>>
>>> I actually had tried that, but I got errors -- probably because the
>>> ownership will be set for each package that installs ${libexecdir}, and
>>> which is processed before the recipe that actually creates the
>>> user/group specified for ${libexecdir} in the file pointed by
>>> FILESYSTEM_PERMS_TABLES.
>>
>> This is a bug then.  The owner/group correction is supposed to be made AFTER the
>> user/groups have been added to the system (sysroot) via the adduser.  THAT is a
>> bug that IMHO should be fixed sooner, rather then later.
>>
>> It might be as simple as the install sysroot (${D}) configuration with pseudo
>> isn't pointing to the right /etc/passwd and /etc/group.  I believe it should be
>> pointing to the one in the regular sysroot repository.
> 
> I should also have mentioned that initially I set
> FILESYSTEM_PERMS_TABLES globally.
> 
> Now I set it in the foo recipe, but still get errors:
> 
> ----------------------------8<---------------------------------
> ERROR: Error executing a python function in .../foo.bb:
> 
> The stack trace of python calls that resulted in this exception/failure was:
> File: 'fixup_perms', lineno: 231, function: <module>
>      0227:                    each_file = os.path.join(root, f)
>      0228:                    fix_perms(each_file, fs_perms_table[dir].fmode, fs_perms_table[dir].fuid, fs_perms_table[dir].fgid, dir)
>      0229:
>      0230:
>  *** 0231:fixup_perms(d)
>      0232:
> File: 'fixup_perms', lineno: 173, function: fixup_perms
>      0169:                if len(lsplit) != 8 and not (len(lsplit) == 3 and lsplit[1].lower() == "link"):
>      0170:                    msg = "Fixup perms: %s invalid line: %s" % (conf, line)
>      0171:                    package_qa_handle_error("perm-line", msg, d)
>      0172:                    continue
>  *** 0173:                entry = fs_perms_entry(d.expand(line))
>      0174:                if entry and entry.path:
>      0175:                    fs_perms_table[entry.path] = entry
>      0176:            f.close()
>      0177:
> File: 'fixup_perms', lineno: 32, function: __init__
>   File "fixup_perms", line 32, in __init__
> 
> File: 'fixup_perms', lineno: 43, function: _setdir
>   File "fixup_perms", line 43, in _setdir
> 
> File: 'fixup_perms', lineno: 67, function: _procuid
>   File "fixup_perms", line 67, in _procuid
> 
> Exception: KeyError: 'getpwnam(): name not found: foo'
> ---------------------------->8---------------------------------
> 
> I still have the chown line in do_install, and that seems to "work"
> (i.e., it doesn't cause errors).  The error above seems be caused by
> something in do_package (fix_perms, specifically) that is not using the
> proper authentication database.

What is odd, if it's working in do_install, then it should be working in the
package set.

You should probably look at dumping the value of PSEUDO_PASSWD in the do_install
step, and then later in the package.bbclass when it fails.  If it fails we can
compare the values and see what the mismatch might be.

It would be good if you could dump the environment at this point and see where
the password/group file are being sourced from.

Try something like:

                    package_qa_handle_error("perm-line", msg, d)
                    continue
+               try:
+                   entry = fs_perms_entry(d.expand(line))
+               except:
+                   bb.error("debug: PSEUDO_PASSWD = '%s'" %
os.getenv("PSEUDO_PASSWD"))
+                   raise
               if entry and entry.path:
                    fs_perms_table[entry.path] = entry

The code will still "crash", but should print out the value of PSEUDO_PASSWD
first.  I would expect that PSEUDO_PASSWD would be pointing into the
tmp/sysroot/<machine> directory.

--Mark

> Best wishes.
> Mario
> 

Re: Ownership issue in package contents

[Mario Domenech Goulart]

Hi Mark and all,

On Tue, 31 Mar 2015 16:47:36 -0500 Mark Hatle <mark.hatle@windriver.com> wrote:

> On 3/31/15 4:21 PM, Mario Domenech Goulart wrote:
>> On Tue, 31 Mar 2015 16:09:42 -0500 Mark Hatle <mark.hatle@windriver.com> wrote:
>> 
>>> On 3/31/15 4:01 PM, Mario Domenech Goulart wrote:
>>>> On Tue, 31 Mar 2015 15:51:50 -0500 Mark Hatle <mark.hatle@windriver.com> wrote:
>>>>
>>>>> On 3/31/15 3:33 PM, Mario Domenech Goulart wrote:
>>>>>>
>>>>>> On Tue, 31 Mar 2015 13:23:00 -0500 Mark Hatle <mark.hatle@windriver.com> wrote:
>>>>>>
>>>>>>> On 3/31/15 12:20 PM, Mario Domenech Goulart wrote:
>>>>>>>>
>>>>>>>> On Tue, 31 Mar 2015 14:50:06 +0100 "Burton, Ross" <ross.burton@intel.com> wrote:
>>>>>>>>
>>>>>>>>> On 27 March 2015 at 17:31, Mario Domenech Goulart <mario@ossystems.com.br> wrote:
>>>>>>>>>
>>>>>>>>>     Note that, although I run "chown -R foo:foo ${D}${libdir}/foo" in
>>>>>>>>>     the recipe, ./usr/lib/foo/ in the package is owned by root.
>>>>>>>>>     However, its content has the right ownership.
>>>>>>>>>
>>>>>>>>> Looks like a bug in pseudo to me, can you file a bug for that?
>>>>>>>>
>>>>>>>> Sure.  Filed #7554.
>>>>>>>
>>>>>>> I'd suggest you look at meta/classes/package.bbclass "fixup_perms" function.
>>>>>>>
>>>>>>> The ${D}${libdir} (and above) are "corrected" to be 'root:root' by this
>>>>>>> function.  I don't know why 'foo' would be, but if it's a standard defined
>>>>>>> variable -- or if 'directory walking' is enabled it could end up doing this as well.
>>>>>>>
>>>>>>> The control file for this is in meta/files/fs-perms.txt (unless otherwise
>>>>>>> defined by a distribution or other configuration file.)
>>>>>>
>>>>>> Thanks a lot.  You seem to have guided me exactly to what causes the
>>>>>> issue.
>>>>>>
>>>>>>>
>>>>>>> Format of the file is:
>>>>>>>
>>>>>>> # The format of this file
>>>>>>> #
>>>>>>> #<path> <mode>  <uid>   <gid>   <walk>  <fmode> <fuid>  <fgid>
>>>>>> ...
>>>>>>>
>>>>>>> The default is:
>>>>>> ...
>>>>>>> libexecdir	0755 root root false - - -
>>>>>> ...
>>>>>>
>>>>>> This variable seems to be the cause of problems:
>>>>>>
>>>>>> $ bitbake -e foo | grep libexecdir=
>>>>>> export libexecdir="/usr/lib/foo"
>>>>>>
>>>>>> As far as I understand, package.bbclass may use a user-configured
>>>>>> permissions table (via FILESYSTEM_PERMS_TABLES), but I'm not sure if
>>>>>> this is the right "fix" for the case in question.  I'd have to hardcode
>>>>>> the owner of /usr/lib/foo to be "foo", but foo may not be available when
>>>>>> packaging other recipes.
>>>>>
>>>>> Ok, good this answers the question as to "why" it's happening.  You can easily
>>>>> fix this by adding a configuration specific fs-perms.txt file (can name it
>>>>> anything) that overrides that setting and add it to the FILESYSTEM_PERMS_TABLES.
>>>>>  You can do this globally in a layer, a distribution or even just a recipe.
>>>>>
>>>>> In your recipe you can likely do something like:
>>>>>
>>>>> FILESYSTEM_PERMS_TABLES ?= "files/fs-perms.txt"
>>>>> FILESYSTEM_PERMS_TABLES += "${THISDIR}/files/recipe-perms.txt"
>>>>>
>>>>> (Do the ?= first in case it's already set by someone else, then add your recipe
>>>>> specific perms later)
>>>>>
>>>>> Contents of the "${THISDIR}/files/recipe-perms.txt":
>>>>>
>>>>> ${libexecdir} 0755 myuid mygid true - myuid mygid
>>>>>
>>>>> Then you can even skip the chown -R, as the system will do it for you.
>>>>
>>>> I actually had tried that, but I got errors -- probably because the
>>>> ownership will be set for each package that installs ${libexecdir}, and
>>>> which is processed before the recipe that actually creates the
>>>> user/group specified for ${libexecdir} in the file pointed by
>>>> FILESYSTEM_PERMS_TABLES.
>>>
>>> This is a bug then.  The owner/group correction is supposed to be made AFTER the
>>> user/groups have been added to the system (sysroot) via the adduser.  THAT is a
>>> bug that IMHO should be fixed sooner, rather then later.
>>>
>>> It might be as simple as the install sysroot (${D}) configuration with pseudo
>>> isn't pointing to the right /etc/passwd and /etc/group.  I believe it should be
>>> pointing to the one in the regular sysroot repository.
>> 
>> I should also have mentioned that initially I set
>> FILESYSTEM_PERMS_TABLES globally.
>> 
>> Now I set it in the foo recipe, but still get errors:
>> 
>> ----------------------------8<---------------------------------
>> ERROR: Error executing a python function in .../foo.bb:
>> 
>> The stack trace of python calls that resulted in this exception/failure was:
>> File: 'fixup_perms', lineno: 231, function: <module>
>>      0227:                    each_file = os.path.join(root, f)
>>      0228:                    fix_perms(each_file, fs_perms_table[dir].fmode, fs_perms_table[dir].fuid, fs_perms_table[dir].fgid, dir)
>>      0229:
>>      0230:
>>  *** 0231:fixup_perms(d)
>>      0232:
>> File: 'fixup_perms', lineno: 173, function: fixup_perms
>>      0169:                if len(lsplit) != 8 and not (len(lsplit) == 3 and lsplit[1].lower() == "link"):
>>      0170:                    msg = "Fixup perms: %s invalid line: %s" % (conf, line)
>>      0171:                    package_qa_handle_error("perm-line", msg, d)
>>      0172:                    continue
>>  *** 0173:                entry = fs_perms_entry(d.expand(line))
>>      0174:                if entry and entry.path:
>>      0175:                    fs_perms_table[entry.path] = entry
>>      0176:            f.close()
>>      0177:
>> File: 'fixup_perms', lineno: 32, function: __init__
>>   File "fixup_perms", line 32, in __init__
>> 
>> File: 'fixup_perms', lineno: 43, function: _setdir
>>   File "fixup_perms", line 43, in _setdir
>> 
>> File: 'fixup_perms', lineno: 67, function: _procuid
>>   File "fixup_perms", line 67, in _procuid
>> 
>> Exception: KeyError: 'getpwnam(): name not found: foo'
>> ---------------------------->8---------------------------------
>> 
>> I still have the chown line in do_install, and that seems to "work"
>> (i.e., it doesn't cause errors).  The error above seems be caused by
>> something in do_package (fix_perms, specifically) that is not using the
>> proper authentication database.
>
> What is odd, if it's working in do_install, then it should be working in the
> package set.
>
> You should probably look at dumping the value of PSEUDO_PASSWD in the do_install
> step, and then later in the package.bbclass when it fails.  If it fails we can
> compare the values and see what the mismatch might be.
>
> It would be good if you could dump the environment at this point and see where
> the password/group file are being sourced from.
>
> Try something like:
>
>                     package_qa_handle_error("perm-line", msg, d)
>                     continue
> +               try:
> +                   entry = fs_perms_entry(d.expand(line))
> +               except:
> +                   bb.error("debug: PSEUDO_PASSWD = '%s'" %
> os.getenv("PSEUDO_PASSWD"))
> +                   raise
>                if entry and entry.path:
>                     fs_perms_table[entry.path] = entry
>
> The code will still "crash", but should print out the value of PSEUDO_PASSWD
> first.  I would expect that PSEUDO_PASSWD would be pointing into the
> tmp/sysroot/<machine> directory.

Thanks for all the hints and for your patience.

It looks like I was caught by a messed up TMPDIR.  After I removed
TMPDIR and baked the foo recipe, everything works.  I could make it work
by adding

  FILESYSTEM_PERMS_TABLES = "files/fs-perms.txt ${THISDIR}/${PN}/fs-perms-foo.txt"

to the foo.bb recipe.  In this case files/fs-perms.txt is the global
default one and ${THISDIR}/${PN}/fs-perms-foo.txt is the foo-specific
one that has one line only, which sets the permissions for
${libexecdir}.

I had to explicitly prepend files/fs-perms.txt because packages.bbclass
would not pick files/fs-perms.txt if FILESYSTEM_PERMS_TABLES is set.

Best wishes.
Mario
-- 
http://www.ossystems.com.br

Re: Ownership issue in package contents

[Mark Hatle]

On 4/6/15 7:59 AM, Mario Domenech Goulart wrote:
> Hi Mark and all,

...

> Thanks for all the hints and for your patience.
> 
> It looks like I was caught by a messed up TMPDIR.  After I removed
> TMPDIR and baked the foo recipe, everything works.  I could make it work
> by adding
> 
>   FILESYSTEM_PERMS_TABLES = "files/fs-perms.txt ${THISDIR}/${PN}/fs-perms-foo.txt"
> 
> to the foo.bb recipe.  In this case files/fs-perms.txt is the global
> default one and ${THISDIR}/${PN}/fs-perms-foo.txt is the foo-specific
> one that has one line only, which sets the permissions for
> ${libexecdir}.
> 
> I had to explicitly prepend files/fs-perms.txt because packages.bbclass
> would not pick files/fs-perms.txt if FILESYSTEM_PERMS_TABLES is set.

I've often thought that we probably should make FILESYSTEM_PERMS_TABLES into a
globally defined configuration (bitbake.conf) and remove the default from the
package.bbclass.  The design here was old, and based on even older code that was
specific to device node generation.

A patch to do this would likely be accepted.  (I'd make it so that an empty
value would simply run through just the default internal values.)

--Mark

> Best wishes.
> Mario
> 

Re: Ownership issue in package contents

[Otavio Salvador]

On Mon, Apr 6, 2015 at 11:53 AM, Mark Hatle <mark.hatle@windriver.com> wrote:
> On 4/6/15 7:59 AM, Mario Domenech Goulart wrote:
>> Hi Mark and all,
>
> ...
>
>> Thanks for all the hints and for your patience.
>>
>> It looks like I was caught by a messed up TMPDIR.  After I removed
>> TMPDIR and baked the foo recipe, everything works.  I could make it work
>> by adding
>>
>>   FILESYSTEM_PERMS_TABLES = "files/fs-perms.txt ${THISDIR}/${PN}/fs-perms-foo.txt"
>>
>> to the foo.bb recipe.  In this case files/fs-perms.txt is the global
>> default one and ${THISDIR}/${PN}/fs-perms-foo.txt is the foo-specific
>> one that has one line only, which sets the permissions for
>> ${libexecdir}.
>>
>> I had to explicitly prepend files/fs-perms.txt because packages.bbclass
>> would not pick files/fs-perms.txt if FILESYSTEM_PERMS_TABLES is set.
>
> I've often thought that we probably should make FILESYSTEM_PERMS_TABLES into a
> globally defined configuration (bitbake.conf) and remove the default from the
> package.bbclass.  The design here was old, and based on even older code that was
> specific to device node generation.
>
> A patch to do this would likely be accepted.  (I'd make it so that an empty
> value would simply run through just the default internal values.)

I would think that OE-Core/files/fs-perms.txt should always be run as
first item and recipe or layer ones could be appended to it. So we can
avoid a lot of duplication and provide a more predictable behavior.

-- 
Otavio Salvador                             O.S. Systems
http://www.ossystems.com.br        http://code.ossystems.com.br
Mobile: +55 (53) 9981-7854            Mobile: +1 (347) 903-9750

Re: Ownership issue in package contents

[Mark Hatle]

On 4/6/15 9:57 AM, Otavio Salvador wrote:
> On Mon, Apr 6, 2015 at 11:53 AM, Mark Hatle <mark.hatle@windriver.com> wrote:
>> On 4/6/15 7:59 AM, Mario Domenech Goulart wrote:
>>> Hi Mark and all,
>>
>> ...
>>
>>> Thanks for all the hints and for your patience.
>>>
>>> It looks like I was caught by a messed up TMPDIR.  After I removed
>>> TMPDIR and baked the foo recipe, everything works.  I could make it work
>>> by adding
>>>
>>>   FILESYSTEM_PERMS_TABLES = "files/fs-perms.txt ${THISDIR}/${PN}/fs-perms-foo.txt"
>>>
>>> to the foo.bb recipe.  In this case files/fs-perms.txt is the global
>>> default one and ${THISDIR}/${PN}/fs-perms-foo.txt is the foo-specific
>>> one that has one line only, which sets the permissions for
>>> ${libexecdir}.
>>>
>>> I had to explicitly prepend files/fs-perms.txt because packages.bbclass
>>> would not pick files/fs-perms.txt if FILESYSTEM_PERMS_TABLES is set.
>>
>> I've often thought that we probably should make FILESYSTEM_PERMS_TABLES into a
>> globally defined configuration (bitbake.conf) and remove the default from the
>> package.bbclass.  The design here was old, and based on even older code that was
>> specific to device node generation.
>>
>> A patch to do this would likely be accepted.  (I'd make it so that an empty
>> value would simply run through just the default internal values.)
> 
> I would think that OE-Core/files/fs-perms.txt should always be run as
> first item and recipe or layer ones could be appended to it. So we can
> avoid a lot of duplication and provide a more predictable behavior.
> 

It wasn't done this way originally, because the assumption was that a
distribution layer may want to provide it's own file and avoid using any
defaults from OE.

--Mark