Installation order question with RPM backend

Installation order question with RPM backend

[Xu, Dongxiao]

Hi Mark,

I met a strange issue while using RPM to generate the rootfs.

In the installation list, if we have 2 RPM packages, say A.rpm and
B.rpm. package A RDEPENDS on package B. While installing the two
packages? Does RPM ensures to install B first and then install A?

The real issue is: we have certain packages that need to run
useradd/groupadd at rootfs time, for example, the dbus. However the
useradd/groupadd bbclass RDEPENDS on base-files, which provides
the /etc/group file. While installing the final image, sometimes we saw
it installs dbus firstly and then base-files, causing the
useradd/groupadd script error since it could not find /etc/group file.

I tried ipk and it doesn't have problem since it ensures to install
base-files firstly.

Any comment is welcome.

Thanks,
Dongxiao

Re: Installation order question with RPM backend

[Mark Hatle]

On 4/11/12 10:14 AM, Xu, Dongxiao wrote:
> Hi Mark,
>
> I met a strange issue while using RPM to generate the rootfs.
>
> In the installation list, if we have 2 RPM packages, say A.rpm and
> B.rpm. package A RDEPENDS on package B. While installing the two
> packages? Does RPM ensures to install B first and then install A?
>
> The real issue is: we have certain packages that need to run
> useradd/groupadd at rootfs time, for example, the dbus. However the
> useradd/groupadd bbclass RDEPENDS on base-files, which provides
> the /etc/group file. While installing the final image, sometimes we saw
> it installs dbus firstly and then base-files, causing the
> useradd/groupadd script error since it could not find /etc/group file.

it does enforce install order, however the /etc/group, /etc/passwd files (last 
time I checked) were being put into place by the post install scripts.  The 
scripting order is handled somewhat independently of the package install order. 
  (post install scripts get delayed intentionally for performance reasons. 
There is a way to hint a dependency for them as well...)

The passwd/group files are fairly unique files, and generally are installed 
-first- (individually) before any other packages on most RPM installations. 
After that the methods and install ordering works...

--Mark

> I tried ipk and it doesn't have problem since it ensures to install
> base-files firstly.
>
> Any comment is welcome.
>
> Thanks,
> Dongxiao
>

Re: Installation order question with RPM backend

[Xu, Dongxiao]

On Wed, 2012-04-11 at 10:25 -0500, Mark Hatle wrote:
> On 4/11/12 10:14 AM, Xu, Dongxiao wrote:
> > Hi Mark,
> >
> > I met a strange issue while using RPM to generate the rootfs.
> >
> > In the installation list, if we have 2 RPM packages, say A.rpm and
> > B.rpm. package A RDEPENDS on package B. While installing the two
> > packages? Does RPM ensures to install B first and then install A?
> >
> > The real issue is: we have certain packages that need to run
> > useradd/groupadd at rootfs time, for example, the dbus. However the
> > useradd/groupadd bbclass RDEPENDS on base-files, which provides
> > the /etc/group file. While installing the final image, sometimes we saw
> > it installs dbus firstly and then base-files, causing the
> > useradd/groupadd script error since it could not find /etc/group file.
> 
> it does enforce install order, however the /etc/group, /etc/passwd files (last 
> time I checked) were being put into place by the post install scripts.  The 
> scripting order is handled somewhat independently of the package install order. 
>   (post install scripts get delayed intentionally for performance reasons. 
> There is a way to hint a dependency for them as well...)
> 
> The passwd/group files are fairly unique files, and generally are installed 
> -first- (individually) before any other packages on most RPM installations. 
> After that the methods and install ordering works...
> 
But does the following log indicates the dbus-1 is installed before
base-passwd?

dbus-1                ##################################################
 Adding system startup
for /distro/sdb/build-basic/tmp/work/qemux86-poky-linux/hob-image-hob-basic-1.0-r0/rootfs/etc/init.d/dbus-1.
kernel-module-uvesafb ##################################################
libusb-compat         ##################################################
base-passwd           ##################################################

Thanks,
Dongxiao

> --Mark
> 
> > I tried ipk and it doesn't have problem since it ensures to install
> > base-files firstly.
> >
> > Any comment is welcome.
> >
> > Thanks,
> > Dongxiao
> >
> 

Re: Installation order question with RPM backend

[Mark Hatle]

On 4/11/12 10:37 AM, Xu, Dongxiao wrote:
> On Wed, 2012-04-11 at 10:25 -0500, Mark Hatle wrote:
>> On 4/11/12 10:14 AM, Xu, Dongxiao wrote:
>>> Hi Mark,
>>>
>>> I met a strange issue while using RPM to generate the rootfs.
>>>
>>> In the installation list, if we have 2 RPM packages, say A.rpm and
>>> B.rpm. package A RDEPENDS on package B. While installing the two
>>> packages? Does RPM ensures to install B first and then install A?
>>>
>>> The real issue is: we have certain packages that need to run
>>> useradd/groupadd at rootfs time, for example, the dbus. However the
>>> useradd/groupadd bbclass RDEPENDS on base-files, which provides
>>> the /etc/group file. While installing the final image, sometimes we saw
>>> it installs dbus firstly and then base-files, causing the
>>> useradd/groupadd script error since it could not find /etc/group file.
>>
>> it does enforce install order, however the /etc/group, /etc/passwd files (last
>> time I checked) were being put into place by the post install scripts.  The
>> scripting order is handled somewhat independently of the package install order.
>>    (post install scripts get delayed intentionally for performance reasons.
>> There is a way to hint a dependency for them as well...)
>>
>> The passwd/group files are fairly unique files, and generally are installed
>> -first- (individually) before any other packages on most RPM installations.
>> After that the methods and install ordering works...
>>
> But does the following log indicates the dbus-1 is installed before
> base-passwd?
>
> dbus-1                ##################################################
>   Adding system startup
> for /distro/sdb/build-basic/tmp/work/qemux86-poky-linux/hob-image-hob-basic-1.0-r0/rootfs/etc/init.d/dbus-1.
> kernel-module-uvesafb ##################################################
> libusb-compat         ##################################################
> base-passwd           ##################################################

Certainly appears that way.. but we'd need to look into the packages and 
understand the requirements as they are defined and trace them to see if there 
is a problem w/ the ordering or if the packages have a problem.

You will often see mysterious reordering when there is a circular dependency. 
RPM has to break this dependency in some way, and it does it by simply choosing 
one or the other.  (There is a hint mechanism for circular dependencies, but 
we've never used it.)

My suggestion is lets look at the package runtime dependenices and manually 
trace them..  Focus on dbus-1 and base-passwd... and see if the order is right 
or wrong or if there is a circular dependency.

(Also our version of RPM 5 is a bit old, and there are some known bugs in it.. 
as far as I know, none of them with the dependency resolution or code paths we 
follow.)

--Mark

> Thanks,
> Dongxiao
>
>> --Mark
>>
>>> I tried ipk and it doesn't have problem since it ensures to install
>>> base-files firstly.
>>>
>>> Any comment is welcome.
>>>
>>> Thanks,
>>> Dongxiao
>>>
>>
>
>

Re: Installation order question with RPM backend

[Xu, Dongxiao]

On Wed, 2012-04-11 at 10:45 -0500, Mark Hatle wrote:
> On 4/11/12 10:37 AM, Xu, Dongxiao wrote:
> > On Wed, 2012-04-11 at 10:25 -0500, Mark Hatle wrote:
> >> On 4/11/12 10:14 AM, Xu, Dongxiao wrote:
> >>> Hi Mark,
> >>>
> >>> I met a strange issue while using RPM to generate the rootfs.
> >>>
> >>> In the installation list, if we have 2 RPM packages, say A.rpm and
> >>> B.rpm. package A RDEPENDS on package B. While installing the two
> >>> packages? Does RPM ensures to install B first and then install A?
> >>>
> >>> The real issue is: we have certain packages that need to run
> >>> useradd/groupadd at rootfs time, for example, the dbus. However the
> >>> useradd/groupadd bbclass RDEPENDS on base-files, which provides
> >>> the /etc/group file. While installing the final image, sometimes we saw
> >>> it installs dbus firstly and then base-files, causing the
> >>> useradd/groupadd script error since it could not find /etc/group file.
> >>
> >> it does enforce install order, however the /etc/group, /etc/passwd files (last
> >> time I checked) were being put into place by the post install scripts.  The
> >> scripting order is handled somewhat independently of the package install order.
> >>    (post install scripts get delayed intentionally for performance reasons.
> >> There is a way to hint a dependency for them as well...)
> >>
> >> The passwd/group files are fairly unique files, and generally are installed
> >> -first- (individually) before any other packages on most RPM installations.
> >> After that the methods and install ordering works...
> >>
> > But does the following log indicates the dbus-1 is installed before
> > base-passwd?
> >
> > dbus-1                ##################################################
> >   Adding system startup
> > for /distro/sdb/build-basic/tmp/work/qemux86-poky-linux/hob-image-hob-basic-1.0-r0/rootfs/etc/init.d/dbus-1.
> > kernel-module-uvesafb ##################################################
> > libusb-compat         ##################################################
> > base-passwd           ##################################################
> 
> Certainly appears that way.. but we'd need to look into the packages and 
> understand the requirements as they are defined and trace them to see if there 
> is a problem w/ the ordering or if the packages have a problem.
> 
> You will often see mysterious reordering when there is a circular dependency. 
> RPM has to break this dependency in some way, and it does it by simply choosing 
> one or the other.  (There is a hint mechanism for circular dependencies, but 
> we've never used it.)
> 
> My suggestion is lets look at the package runtime dependenices and manually 
> trace them..  Focus on dbus-1 and base-passwd... and see if the order is right 
> or wrong or if there is a circular dependency.

I checked the dbus.spec and base-passwd.spec.
For dbus.spec, there is a line:

%package -n dbus-1
Requires: base-passwd

And for base-passwd, there is no dbus exists in base-passwd.spec.

Thanks,
Dongxiao

> 
> (Also our version of RPM 5 is a bit old, and there are some known bugs in it.. 
> as far as I know, none of them with the dependency resolution or code paths we 
> follow.)
> 
> --Mark
> 
> > Thanks,
> > Dongxiao
> >
> >> --Mark
> >>
> >>> I tried ipk and it doesn't have problem since it ensures to install
> >>> base-files firstly.
> >>>
> >>> Any comment is welcome.
> >>>
> >>> Thanks,
> >>> Dongxiao
> >>>
> >>
> >
> >
> 

Re: Installation order question with RPM backend

[Mark Hatle]

On 4/11/12 10:51 AM, Xu, Dongxiao wrote:
> On Wed, 2012-04-11 at 10:45 -0500, Mark Hatle wrote:
>> On 4/11/12 10:37 AM, Xu, Dongxiao wrote:
>>> On Wed, 2012-04-11 at 10:25 -0500, Mark Hatle wrote:
>>>> On 4/11/12 10:14 AM, Xu, Dongxiao wrote:
>>>>> Hi Mark,
>>>>>
>>>>> I met a strange issue while using RPM to generate the rootfs.
>>>>>
>>>>> In the installation list, if we have 2 RPM packages, say A.rpm and
>>>>> B.rpm. package A RDEPENDS on package B. While installing the two
>>>>> packages? Does RPM ensures to install B first and then install A?
>>>>>
>>>>> The real issue is: we have certain packages that need to run
>>>>> useradd/groupadd at rootfs time, for example, the dbus. However the
>>>>> useradd/groupadd bbclass RDEPENDS on base-files, which provides
>>>>> the /etc/group file. While installing the final image, sometimes we saw
>>>>> it installs dbus firstly and then base-files, causing the
>>>>> useradd/groupadd script error since it could not find /etc/group file.
>>>>
>>>> it does enforce install order, however the /etc/group, /etc/passwd files (last
>>>> time I checked) were being put into place by the post install scripts.  The
>>>> scripting order is handled somewhat independently of the package install order.
>>>>     (post install scripts get delayed intentionally for performance reasons.
>>>> There is a way to hint a dependency for them as well...)
>>>>
>>>> The passwd/group files are fairly unique files, and generally are installed
>>>> -first- (individually) before any other packages on most RPM installations.
>>>> After that the methods and install ordering works...
>>>>
>>> But does the following log indicates the dbus-1 is installed before
>>> base-passwd?
>>>
>>> dbus-1                ##################################################
>>>    Adding system startup
>>> for /distro/sdb/build-basic/tmp/work/qemux86-poky-linux/hob-image-hob-basic-1.0-r0/rootfs/etc/init.d/dbus-1.
>>> kernel-module-uvesafb ##################################################
>>> libusb-compat         ##################################################
>>> base-passwd           ##################################################
>>
>> Certainly appears that way.. but we'd need to look into the packages and
>> understand the requirements as they are defined and trace them to see if there
>> is a problem w/ the ordering or if the packages have a problem.
>>
>> You will often see mysterious reordering when there is a circular dependency.
>> RPM has to break this dependency in some way, and it does it by simply choosing
>> one or the other.  (There is a hint mechanism for circular dependencies, but
>> we've never used it.)
>>
>> My suggestion is lets look at the package runtime dependenices and manually
>> trace them..  Focus on dbus-1 and base-passwd... and see if the order is right
>> or wrong or if there is a circular dependency.
>
> I checked the dbus.spec and base-passwd.spec.
> For dbus.spec, there is a line:
>
> %package -n dbus-1
> Requires: base-passwd
>
> And for base-passwd, there is no dbus exists in base-passwd.spec.

You need to query the binary packages for the real values..

rpm -qp <package> --requires

(and --provides will show what it provides..)  then match those two up.. If you 
still don't come across some type of circular dependency, then let me know.. 
I'll start investigating further. (there is additional dependency resolution 
debugging I can enable, but it's difficult to explain how to interpret the tons 
of lines..)

--Mark

> Thanks,
> Dongxiao
>
>>
>> (Also our version of RPM 5 is a bit old, and there are some known bugs in it..
>> as far as I know, none of them with the dependency resolution or code paths we
>> follow.)
>>
>> --Mark
>>
>>> Thanks,
>>> Dongxiao
>>>
>>>> --Mark
>>>>
>>>>> I tried ipk and it doesn't have problem since it ensures to install
>>>>> base-files firstly.
>>>>>
>>>>> Any comment is welcome.
>>>>>
>>>>> Thanks,
>>>>> Dongxiao
>>>>>
>>>>
>>>
>>>
>>
>
>

Re: Installation order question with RPM backend

[Xu, Dongxiao]

On Wed, 2012-04-11 at 10:56 -0500, Mark Hatle wrote:
> On 4/11/12 10:51 AM, Xu, Dongxiao wrote:
> > On Wed, 2012-04-11 at 10:45 -0500, Mark Hatle wrote:
> >> On 4/11/12 10:37 AM, Xu, Dongxiao wrote:
> >>> On Wed, 2012-04-11 at 10:25 -0500, Mark Hatle wrote:
> >>>> On 4/11/12 10:14 AM, Xu, Dongxiao wrote:
> >>>>> Hi Mark,
> >>>>>
> >>>>> I met a strange issue while using RPM to generate the rootfs.
> >>>>>
> >>>>> In the installation list, if we have 2 RPM packages, say A.rpm and
> >>>>> B.rpm. package A RDEPENDS on package B. While installing the two
> >>>>> packages? Does RPM ensures to install B first and then install A?
> >>>>>
> >>>>> The real issue is: we have certain packages that need to run
> >>>>> useradd/groupadd at rootfs time, for example, the dbus. However the
> >>>>> useradd/groupadd bbclass RDEPENDS on base-files, which provides
> >>>>> the /etc/group file. While installing the final image, sometimes we saw
> >>>>> it installs dbus firstly and then base-files, causing the
> >>>>> useradd/groupadd script error since it could not find /etc/group file.
> >>>>
> >>>> it does enforce install order, however the /etc/group, /etc/passwd files (last
> >>>> time I checked) were being put into place by the post install scripts.  The
> >>>> scripting order is handled somewhat independently of the package install order.
> >>>>     (post install scripts get delayed intentionally for performance reasons.
> >>>> There is a way to hint a dependency for them as well...)
> >>>>
> >>>> The passwd/group files are fairly unique files, and generally are installed
> >>>> -first- (individually) before any other packages on most RPM installations.
> >>>> After that the methods and install ordering works...
> >>>>
> >>> But does the following log indicates the dbus-1 is installed before
> >>> base-passwd?
> >>>
> >>> dbus-1                ##################################################
> >>>    Adding system startup
> >>> for /distro/sdb/build-basic/tmp/work/qemux86-poky-linux/hob-image-hob-basic-1.0-r0/rootfs/etc/init.d/dbus-1.
> >>> kernel-module-uvesafb ##################################################
> >>> libusb-compat         ##################################################
> >>> base-passwd           ##################################################
> >>
> >> Certainly appears that way.. but we'd need to look into the packages and
> >> understand the requirements as they are defined and trace them to see if there
> >> is a problem w/ the ordering or if the packages have a problem.
> >>
> >> You will often see mysterious reordering when there is a circular dependency.
> >> RPM has to break this dependency in some way, and it does it by simply choosing
> >> one or the other.  (There is a hint mechanism for circular dependencies, but
> >> we've never used it.)
> >>
> >> My suggestion is lets look at the package runtime dependenices and manually
> >> trace them..  Focus on dbus-1 and base-passwd... and see if the order is right
> >> or wrong or if there is a circular dependency.
> >
> > I checked the dbus.spec and base-passwd.spec.
> > For dbus.spec, there is a line:
> >
> > %package -n dbus-1
> > Requires: base-passwd
> >
> > And for base-passwd, there is no dbus exists in base-passwd.spec.
> 
> You need to query the binary packages for the real values..
> 
> rpm -qp <package> --requires

Here I pasted the command output. I think there is no circular
dependency here.

It is relatively easy to reproduce this rootfs error:
For a certain image bb file, say task-core-basic. Change the definition
of the installation package with the following line:
PACKAGE_INSTALL = "task-core-basic task-core-ssh-openssh
task-core-apps-console task-core-boot"
# bitbake core-image-basic

Then you will see the useradd/groupadd error and the dependency order
issue.

Thanks,
Dongxiao

dongxiao@dongxiao-osel:/distro/sdb/build-basic/tmp/deploy/rpm/i586$ rpm
-qp base-passwd-3.5.24-r0.i586.rpm --requires
warning: base-passwd-3.5.24-r0.i586.rpm: Header V4 DSA/SHA1 Signature,
key ID a8d359bb: NOKEY
/bin/sh
dongxiao@dongxiao-osel:/distro/sdb/build-basic/tmp/deploy/rpm/i586$ rpm
-qp dbus-1-1.4.16-r3.0.i586.rpm --requires
warning: dbus-1-1.4.16-r3.0.i586.rpm: Header V4 DSA/SHA1 Signature, key
ID 2c60056d: NOKEY
update-rc.d
libdbus-1-3 >= 1.4.16
libc6 >= 2.13
shadow
libexpat1 >= 2.0.1
base-passwd
/bin/sh
/bin/sh
/bin/sh
/bin/sh
/bin/sh
rtld(GNU_HASH)
libc.so.6(GLIBC_2.0)
libc.so.6
libc.so.6(GLIBC_2.1)
libc.so.6(GLIBC_2.10)
libpthread.so.0(GLIBC_2.0)
libc.so.6(GLIBC_2.9)
libc.so.6(GLIBC_2.2)
librt.so.1
librt.so.1(GLIBC_2.2)
libpthread.so.0(GLIBC_2.3.2)
libpthread.so.0
libc.so.6(GLIBC_2.2.4)
libexpat.so.1
libc.so.6(GLIBC_2.1.2)
libpthread.so.0(GLIBC_2.3.3)
libc.so.6(GLIBC_2.4)
libdbus-1.so.3


> 
> (and --provides will show what it provides..)  then match those two up.. If you 
> still don't come across some type of circular dependency, then let me know.. 
> I'll start investigating further. (there is additional dependency resolution 
> debugging I can enable, but it's difficult to explain how to interpret the tons 
> of lines..)
> 
> --Mark
> 
> > Thanks,
> > Dongxiao
> >
> >>
> >> (Also our version of RPM 5 is a bit old, and there are some known bugs in it..
> >> as far as I know, none of them with the dependency resolution or code paths we
> >> follow.)
> >>
> >> --Mark
> >>
> >>> Thanks,
> >>> Dongxiao
> >>>
> >>>> --Mark
> >>>>
> >>>>> I tried ipk and it doesn't have problem since it ensures to install
> >>>>> base-files firstly.
> >>>>>
> >>>>> Any comment is welcome.
> >>>>>
> >>>>> Thanks,
> >>>>> Dongxiao
> >>>>>
> >>>>
> >>>
> >>>
> >>
> >
> >
> 

Re: Installation order question with RPM backend

[Richard Purdie]

On Wed, 2012-04-11 at 10:56 -0500, Mark Hatle wrote:
> On 4/11/12 10:51 AM, Xu, Dongxiao wrote:
> > On Wed, 2012-04-11 at 10:45 -0500, Mark Hatle wrote:
> >> On 4/11/12 10:37 AM, Xu, Dongxiao wrote:
> >>> On Wed, 2012-04-11 at 10:25 -0500, Mark Hatle wrote:
> >>>> On 4/11/12 10:14 AM, Xu, Dongxiao wrote:
> >>>>> Hi Mark,
> >>>>>
> >>>>> I met a strange issue while using RPM to generate the rootfs.
> >>>>>
> >>>>> In the installation list, if we have 2 RPM packages, say A.rpm and
> >>>>> B.rpm. package A RDEPENDS on package B. While installing the two
> >>>>> packages? Does RPM ensures to install B first and then install A?
> >>>>>
> >>>>> The real issue is: we have certain packages that need to run
> >>>>> useradd/groupadd at rootfs time, for example, the dbus. However the
> >>>>> useradd/groupadd bbclass RDEPENDS on base-files, which provides
> >>>>> the /etc/group file. While installing the final image, sometimes we saw
> >>>>> it installs dbus firstly and then base-files, causing the
> >>>>> useradd/groupadd script error since it could not find /etc/group file.
> >>>>
> >>>> it does enforce install order, however the /etc/group, /etc/passwd files (last
> >>>> time I checked) were being put into place by the post install scripts.  The
> >>>> scripting order is handled somewhat independently of the package install order.
> >>>>     (post install scripts get delayed intentionally for performance reasons.
> >>>> There is a way to hint a dependency for them as well...)
> >>>>
> >>>> The passwd/group files are fairly unique files, and generally are installed
> >>>> -first- (individually) before any other packages on most RPM installations.
> >>>> After that the methods and install ordering works...
> >>>>
> >>> But does the following log indicates the dbus-1 is installed before
> >>> base-passwd?
> >>>
> >>> dbus-1                ##################################################
> >>>    Adding system startup
> >>> for /distro/sdb/build-basic/tmp/work/qemux86-poky-linux/hob-image-hob-basic-1.0-r0/rootfs/etc/init.d/dbus-1.
> >>> kernel-module-uvesafb ##################################################
> >>> libusb-compat         ##################################################
> >>> base-passwd           ##################################################
> >>
> >> Certainly appears that way.. but we'd need to look into the packages and
> >> understand the requirements as they are defined and trace them to see if there
> >> is a problem w/ the ordering or if the packages have a problem.
> >>
> >> You will often see mysterious reordering when there is a circular dependency.
> >> RPM has to break this dependency in some way, and it does it by simply choosing
> >> one or the other.  (There is a hint mechanism for circular dependencies, but
> >> we've never used it.)
> >>
> >> My suggestion is lets look at the package runtime dependenices and manually
> >> trace them..  Focus on dbus-1 and base-passwd... and see if the order is right
> >> or wrong or if there is a circular dependency.
> >
> > I checked the dbus.spec and base-passwd.spec.
> > For dbus.spec, there is a line:
> >
> > %package -n dbus-1
> > Requires: base-passwd
> >
> > And for base-passwd, there is no dbus exists in base-passwd.spec.
> 
> You need to query the binary packages for the real values..
> 
> rpm -qp <package> --requires
> 
> (and --provides will show what it provides..)  then match those two up.. If you 
> still don't come across some type of circular dependency, then let me know.. 
> I'll start investigating further. (there is additional dependency resolution 
> debugging I can enable, but it's difficult to explain how to interpret the tons 
> of lines..)

Let me jump in here since I've looked at the rpm code paths in question
before.

RPM basically works its way through the install manifest so if you have
for example:

base-passwd
shadow
dbus-1
libc

then base-passwd and shadow will "stall" their installation until their
dependency on libc is satisfied. Previously, we thought it was good
enough to put base-passwd and shadow at the top but its now becoming
clear we need all their dependencies at the top too.

There are other ongoing discussions about the best way to fix this.
There are failures on the autobuilder at the moment related to it.

Cheers,

Richard