RPM multilib package installation issue

RPM multilib package installation issue

[Xu, Dongxiao]

Hi Mark and Richard,

I am trying to setup a RPM multilib system that, it is a qemux86-64 base image with MULTILIB_IMAGE_INSTALL = "lib32-connman-gnome". With several fixes, the build can pass.

However in run time testing I met a problem that, for those libraries whose base/multilib versions packages will be both built out (like libgtk, it has "libgtk-2.0-2.22.1-r2.x86_64.rpm" and "libgtk-2.0-2.22.1-r2.x86.rpm"), the rpm will only installs the lib32 version of it.

Therefore one question is, if there are two rpm packages with the same PN, PV, PR, but different architecture (like our multilib case), then we run command "rpm -ivh libgtk", which version of libgtk will be installed? Or does rpm have any parameter to force installing them both? Actually multilib requires to install them both with certain order. 

Thanks,
Dongxiao 

Re: RPM multilib package installation issue

[Mark Hatle]

On 9/2/11 2:33 AM, Xu, Dongxiao wrote:
> Hi Mark and Richard,
> 
> I am trying to setup a RPM multilib system that, it is a qemux86-64 base
> image with MULTILIB_IMAGE_INSTALL = "lib32-connman-gnome". With several
> fixes, the build can pass.
> 
> However in run time testing I met a problem that, for those libraries whose
> base/multilib versions packages will be both built out (like libgtk, it has
> "libgtk-2.0-2.22.1-r2.x86_64.rpm" and "libgtk-2.0-2.22.1-r2.x86.rpm"), the
> rpm will only installs the lib32 version of it.

During filesystem construction the system uses dependencies to decide what to
install.  If you build a 32-bit connman-gnome and it requires other 32-bit
libraries the dependency scanner will either pick them up and install them, or
error due to missing dependencies.

In the manual case you would use "rpm -Uhv <path>" manually specifying which one
you want.  RPM will detect a multilib package and will allow installation of
both versions.  (Note always use rpm -U and not rpm -i..  rpm -i just blindly
installs the software with no checking if an existing version exists.)

> Therefore one question is, if there are two rpm packages with the same PN,
> PV, PR, but different architecture (like our multilib case), then we run
> command "rpm -ivh libgtk", which version of libgtk will be installed? Or does
> rpm have any parameter to force installing them both? Actually multilib
> requires to install them both with certain order.

No specific order should be necessary on a multilib system.  As long as the end
dependencies are satisfied the resulting filesystem will work.  (Exceptions to
this are when there are pre and post install scripts that have their own unique
dependencies.. but those are not the normal case for OE-Core/Yocto.)

--Mark

> Thanks, Dongxiao
> 

Re: RPM multilib package installation issue

[McClintock Matthew-B29882]

On Fri, Sep 2, 2011 at 10:03 AM, Mark Hatle <mark.hatle@windriver.com> wrote:
>> However in run time testing I met a problem that, for those libraries whose
>> base/multilib versions packages will be both built out (like libgtk, it has
>> "libgtk-2.0-2.22.1-r2.x86_64.rpm" and "libgtk-2.0-2.22.1-r2.x86.rpm"), the
>> rpm will only installs the lib32 version of it.
>
> During filesystem construction the system uses dependencies to decide what to
> install.  If you build a 32-bit connman-gnome and it requires other 32-bit
> libraries the dependency scanner will either pick them up and install them, or
> error due to missing dependencies.

Some you are saying since we do not build an actual binary that
depends on the 64bit library we just drop the libraries we requested
to be installed?

-M

Re: RPM multilib package installation issue

[Mark Hatle]

On 9/2/11 10:40 AM, McClintock Matthew-B29882 wrote:
> On Fri, Sep 2, 2011 at 10:03 AM, Mark Hatle <mark.hatle@windriver.com> wrote:
>>> However in run time testing I met a problem that, for those libraries whose
>>> base/multilib versions packages will be both built out (like libgtk, it has
>>> "libgtk-2.0-2.22.1-r2.x86_64.rpm" and "libgtk-2.0-2.22.1-r2.x86.rpm"), the
>>> rpm will only installs the lib32 version of it.
>>
>> During filesystem construction the system uses dependencies to decide what to
>> install.  If you build a 32-bit connman-gnome and it requires other 32-bit
>> libraries the dependency scanner will either pick them up and install them, or
>> error due to missing dependencies.
> 
> Some you are saying since we do not build an actual binary that
> depends on the 64bit library we just drop the libraries we requested
> to be installed?
> 
> -M

The normal OE approach is to resolve all items by run-time dependencies.  That
is why a lot more is built then installed.

So if you want a system capable of running bash, you would create a task (and
related image) that simply has a RDEPENDS on "bash".  Everything else is
determined automatically.  So even if you build ncurses for 32-bit and 64-bit,
only the one needed by bash will be installed.

If you change this and specify that your task RDEPENDS is "bash lib64-ncurses".
 Then the system will determine the dependencies based on both bash and the
64-bit ncurses.  This will result in a system (likely 32-bit in this example)
that includes bash and lib64-ncurses and all of it's dependencies...

The point is, if you explicitly tell the system to install something it is
expected to be installed.  If you don't the automatic resolution is used to
determine install components and not everything you want may be installed.

--Mark

Re: RPM multilib package installation issue

[McClintock Matthew-B29882]

On Fri, Sep 2, 2011 at 10:48 AM, Mark Hatle <mark.hatle@windriver.com> wrote:
> The normal OE approach is to resolve all items by run-time dependencies.  That
> is why a lot more is built then installed.
>
> So if you want a system capable of running bash, you would create a task (and
> related image) that simply has a RDEPENDS on "bash".  Everything else is
> determined automatically.  So even if you build ncurses for 32-bit and 64-bit,
> only the one needed by bash will be installed.
>
> If you change this and specify that your task RDEPENDS is "bash lib64-ncurses".
>  Then the system will determine the dependencies based on both bash and the
> 64-bit ncurses.  This will result in a system (likely 32-bit in this example)
> that includes bash and lib64-ncurses and all of it's dependencies...
>
> The point is, if you explicitly tell the system to install something it is
> expected to be installed.  If you don't the automatic resolution is used to
> determine install components and not everything you want may be installed.

One would expect then if I did a:

MULTILIB_IMAGE_INSTALL += "lib64-perl"

I would be a 64bit perl and a slew of 64 bit libraries installed. I'm
still trying to figure out why this is not happening on my particular
build.

-M

Re: RPM multilib package installation issue

[Mark Hatle]

On 9/2/11 11:24 AM, McClintock Matthew-B29882 wrote:
> On Fri, Sep 2, 2011 at 10:48 AM, Mark Hatle <mark.hatle@windriver.com> wrote:
>> The normal OE approach is to resolve all items by run-time dependencies.  That
>> is why a lot more is built then installed.
>>
>> So if you want a system capable of running bash, you would create a task (and
>> related image) that simply has a RDEPENDS on "bash".  Everything else is
>> determined automatically.  So even if you build ncurses for 32-bit and 64-bit,
>> only the one needed by bash will be installed.
>>
>> If you change this and specify that your task RDEPENDS is "bash lib64-ncurses".
>>  Then the system will determine the dependencies based on both bash and the
>> 64-bit ncurses.  This will result in a system (likely 32-bit in this example)
>> that includes bash and lib64-ncurses and all of it's dependencies...
>>
>> The point is, if you explicitly tell the system to install something it is
>> expected to be installed.  If you don't the automatic resolution is used to
>> determine install components and not everything you want may be installed.
> 
> One would expect then if I did a:
> 
> MULTILIB_IMAGE_INSTALL += "lib64-perl"
> 
> I would be a 64bit perl and a slew of 64 bit libraries installed. I'm
> still trying to figure out why this is not happening on my particular
> build.

For everything I see, that should work.

In classes/image.bbclass, RDEPENDS is augmented by the contents of
IMAGE_INSTALL, LINGUAS_INSTALL, MULTILIB_IMAGE_INSTALL, and NORMAL_FEATURE_INSTALL.

The place to start looking for this is within the log.do_rootfs in the rootfs
directory you are working on.  You should see something that indicates it's
trying to install the "lib64-perl".  (in the case of RPM, it should eventually
rename it to "perl" -- but choose the 64-bit version.)

--Mark

> -M

Re: RPM multilib package installation issue

[McClintock Matthew-B29882]

On Fri, Sep 2, 2011 at 12:09 PM, Mark Hatle <mark.hatle@windriver.com> wrote:
> For everything I see, that should work.
>
> In classes/image.bbclass, RDEPENDS is augmented by the contents of
> IMAGE_INSTALL, LINGUAS_INSTALL, MULTILIB_IMAGE_INSTALL, and NORMAL_FEATURE_INSTALL.

The bitbake side is fine, It's getting put into RDEPENDS and all the
packages are beinh built properly. RPMs are built as well.

> The place to start looking for this is within the log.do_rootfs in the rootfs
> directory you are working on.  You should see something that indicates it's
> trying to install the "lib64-perl".  (in the case of RPM, it should eventually
> rename it to "perl" -- but choose the 64-bit version.)

However, I don't see code that invokes package_rpm.bbclass properly.
It looks like it could work, but no where is MLPREFIX set for image.
There is some override or something missing on my end or in the code
to properly export these vars for the ML rpm install scenario.

Further more, from what I see no where in rootfs_rpm.bbclass do we
pass MULTILIB_PACKAGE_INSTALL into the list of packages to be
installed... thus they fall off. ipk looks like it might have this
stuff done.

-M

-M

Re: RPM multilib package installation issue

[Mark Hatle]

On 9/2/11 1:36 PM, McClintock Matthew-B29882 wrote:
> On Fri, Sep 2, 2011 at 12:09 PM, Mark Hatle <mark.hatle@windriver.com> wrote:
>> For everything I see, that should work.
>>
>> In classes/image.bbclass, RDEPENDS is augmented by the contents of
>> IMAGE_INSTALL, LINGUAS_INSTALL, MULTILIB_IMAGE_INSTALL, and NORMAL_FEATURE_INSTALL.
> 
> The bitbake side is fine, It's getting put into RDEPENDS and all the
> packages are beinh built properly. RPMs are built as well.
> 
>> The place to start looking for this is within the log.do_rootfs in the rootfs
>> directory you are working on.  You should see something that indicates it's
>> trying to install the "lib64-perl".  (in the case of RPM, it should eventually
>> rename it to "perl" -- but choose the 64-bit version.)
> 
> However, I don't see code that invokes package_rpm.bbclass properly.
> It looks like it could work, but no where is MLPREFIX set for image.
> There is some override or something missing on my end or in the code
> to properly export these vars for the ML rpm install scenario.
> 
> Further more, from what I see no where in rootfs_rpm.bbclass do we
> pass MULTILIB_PACKAGE_INSTALL into the list of packages to be
> installed... thus they fall off. ipk looks like it might have this
> stuff done.

This looks like a typo of some sorts:

image.bbclass:RDEPENDS += "${IMAGE_INSTALL} ${LINGUAS_INSTALL}
${MULTILIB_IMAGE_INSTALL} ${NORMAL_FEATURE_INSTALL}"

image.bbclass:export MULTILIB_PACKAGE_INSTALL ?= "${MULTILIB_IMAGE_INSTALL}"

So you should be setting MULTILIB_IMAGE_INSTALL, this should feed into the
RDEPENDS and that should in turn seed the roofs stuff..  So there looks to be a
disconnect of some sort between the implementation of the image.bbclass and the
individual package_*.bbclasses.

(The only place I see MULTILIB_PACKAGE_INSTALL is in rootfs_ipk.bbclass.)

The way the rootfs_rpm (which uses package_rpm) gets the list of things to
install is via:

package_linguas=INSTALL_PACKAGES_LINGUAS_RPM=LINGUAS_INSTALL

package_to_install=INSTALL_PACKAGES_NORMAL_RPM=PACKAGE_INSTALL="${IMAGE_INSTALL}
 ${FEATURE_INSTALL}"

package_attemptonly=INSTALL_PACKAGES_ATTEMPTONLY_RPM=PACKAGE_INSTALL_ATTEMPTONLY=FEATURE_INSTALL_OPTIONAL

Where you see RPM above, replace with ipk and deb and the same ordering should
be used.  So based on that, RDEPENDS is being populated from the
"image.bbclass", but nothing is populating the MULTILIB_IMAGE_INSTALL like it
should be.  Either it needs to be added to the 'PACKAGE_INSTALL' or something
similar needs to be invented and added to each of the rootfs_* types.  (Note,
there needs to be a translation step between the definition of
MULTILIB_IMAGE_INSTALL and where it's used.. since things like "zlib" can become
"libz1" via name translation.  (translation happens in runtime_mapping_rename if
I remember correctly.. in image.bbclass)

I'd suggest you file a bug about this in the yocto system.  We'll get someone to
work on it...  (I agree it's broken BTW..)  The ipk usage may turn out to be an
example for the deb and rpm usage... but the three should continue to be in sync
for most of it.

--Mark

> -M
> 
> -M

Re: RPM multilib package installation issue

[McClintock Matthew-B29882]

FYI: ipkg appears to work fine if you have
MULTILIB_IMAGE_INSTALL and MULTILIB_PACKAGE_ARCH set.

Will file a bug shortly.

-M

On Fri, Sep 2, 2011 at 1:56 PM, Mark Hatle <mark.hatle@windriver.com> wrote:
> On 9/2/11 1:36 PM, McClintock Matthew-B29882 wrote:
>> On Fri, Sep 2, 2011 at 12:09 PM, Mark Hatle <mark.hatle@windriver.com> wrote:
>>> For everything I see, that should work.
>>>
>>> In classes/image.bbclass, RDEPENDS is augmented by the contents of
>>> IMAGE_INSTALL, LINGUAS_INSTALL, MULTILIB_IMAGE_INSTALL, and NORMAL_FEATURE_INSTALL.
>>
>> The bitbake side is fine, It's getting put into RDEPENDS and all the
>> packages are beinh built properly. RPMs are built as well.
>>
>>> The place to start looking for this is within the log.do_rootfs in the rootfs
>>> directory you are working on.  You should see something that indicates it's
>>> trying to install the "lib64-perl".  (in the case of RPM, it should eventually
>>> rename it to "perl" -- but choose the 64-bit version.)
>>
>> However, I don't see code that invokes package_rpm.bbclass properly.
>> It looks like it could work, but no where is MLPREFIX set for image.
>> There is some override or something missing on my end or in the code
>> to properly export these vars for the ML rpm install scenario.
>>
>> Further more, from what I see no where in rootfs_rpm.bbclass do we
>> pass MULTILIB_PACKAGE_INSTALL into the list of packages to be
>> installed... thus they fall off. ipk looks like it might have this
>> stuff done.
>
> This looks like a typo of some sorts:
>
> image.bbclass:RDEPENDS += "${IMAGE_INSTALL} ${LINGUAS_INSTALL}
> ${MULTILIB_IMAGE_INSTALL} ${NORMAL_FEATURE_INSTALL}"
>
> image.bbclass:export MULTILIB_PACKAGE_INSTALL ?= "${MULTILIB_IMAGE_INSTALL}"
>
> So you should be setting MULTILIB_IMAGE_INSTALL, this should feed into the
> RDEPENDS and that should in turn seed the roofs stuff..  So there looks to be a
> disconnect of some sort between the implementation of the image.bbclass and the
> individual package_*.bbclasses.
>
> (The only place I see MULTILIB_PACKAGE_INSTALL is in rootfs_ipk.bbclass.)
>
> The way the rootfs_rpm (which uses package_rpm) gets the list of things to
> install is via:
>
> package_linguas=INSTALL_PACKAGES_LINGUAS_RPM=LINGUAS_INSTALL
>
> package_to_install=INSTALL_PACKAGES_NORMAL_RPM=PACKAGE_INSTALL="${IMAGE_INSTALL}
>  ${FEATURE_INSTALL}"
>
> package_attemptonly=INSTALL_PACKAGES_ATTEMPTONLY_RPM=PACKAGE_INSTALL_ATTEMPTONLY=FEATURE_INSTALL_OPTIONAL
>
> Where you see RPM above, replace with ipk and deb and the same ordering should
> be used.  So based on that, RDEPENDS is being populated from the
> "image.bbclass", but nothing is populating the MULTILIB_IMAGE_INSTALL like it
> should be.  Either it needs to be added to the 'PACKAGE_INSTALL' or something
> similar needs to be invented and added to each of the rootfs_* types.  (Note,
> there needs to be a translation step between the definition of
> MULTILIB_IMAGE_INSTALL and where it's used.. since things like "zlib" can become
> "libz1" via name translation.  (translation happens in runtime_mapping_rename if
> I remember correctly.. in image.bbclass)
>
> I'd suggest you file a bug about this in the yocto system.  We'll get someone to
> work on it...  (I agree it's broken BTW..)  The ipk usage may turn out to be an
> example for the deb and rpm usage... but the three should continue to be in sync
> for most of it.
>
> --Mark
>
>> -M
>>
>> -M
>
> _______________________________________________
> yocto mailing list
> yocto@yoctoproject.org
> https://lists.yoctoproject.org/listinfo/yocto
>

Re: RPM multilib package installation issue

[McClintock Matthew-B29882]

On Fri, Sep 2, 2011 at 2:29 PM, McClintock Matthew-B29882
<B29882@freescale.com> wrote:
> FYI: ipkg appears to work fine if you have
> MULTILIB_IMAGE_INSTALL and MULTILIB_PACKAGE_ARCH set.
>
> Will file a bug shortly.

Bug submitted.

http://bugzilla.yoctoproject.org/show_bug.cgi?id=1448

-M

Re: RPM multilib package installation issue

[Xu, Dongxiao]

[-- Attachment #1: Type: text/plain, Size: 2607 bytes --]

Hi Mark,

> -----Original Message-----
> From: Mark Hatle [mailto:mark.hatle@windriver.com]
> Sent: Friday, September 02, 2011 11:03 PM
> To: Xu, Dongxiao
> Cc: Richard Purdie (richard.purdie@linuxfoundation.org);
> yocto@yoctoproject.org
> Subject: Re: RPM multilib package installation issue
> 
> On 9/2/11 2:33 AM, Xu, Dongxiao wrote:
> > Hi Mark and Richard,
> >
> > I am trying to setup a RPM multilib system that, it is a qemux86-64
> > base image with MULTILIB_IMAGE_INSTALL = "lib32-connman-gnome". With
> > several fixes, the build can pass.
> >
> > However in run time testing I met a problem that, for those libraries
> > whose base/multilib versions packages will be both built out (like
> > libgtk, it has "libgtk-2.0-2.22.1-r2.x86_64.rpm" and
> > "libgtk-2.0-2.22.1-r2.x86.rpm"), the rpm will only installs the lib32 version of
> it.
> 
> During filesystem construction the system uses dependencies to decide what
> to install.  If you build a 32-bit connman-gnome and it requires other 32-bit
> libraries the dependency scanner will either pick them up and install them, or
> error due to missing dependencies.
> 
> In the manual case you would use "rpm -Uhv <path>" manually specifying which
> one you want.  RPM will detect a multilib package and will allow installation of
> both versions.  (Note always use rpm -U and not rpm -i..  rpm -i just blindly
> installs the software with no checking if an existing version exists.)

In our poky system, I saw we use some command to generate install_solution.manifest, and then use "rpm -Uvh" to install them.

I attached my install_solution.manifest here. From the list we saw that some libraries are only installed once. For example the db, both 32bit and 64 bit are needed, however only 32bit is installed (libgtk is another example). 

Thanks,
Dongxiao

> 
> > Therefore one question is, if there are two rpm packages with the same
> > PN, PV, PR, but different architecture (like our multilib case), then
> > we run command "rpm -ivh libgtk", which version of libgtk will be
> > installed? Or does rpm have any parameter to force installing them
> > both? Actually multilib requires to install them both with certain order.
> 
> No specific order should be necessary on a multilib system.  As long as the end
> dependencies are satisfied the resulting filesystem will work.  (Exceptions to
> this are when there are pre and post install scripts that have their own unique
> dependencies.. but those are not the normal case for OE-Core/Yocto.)
> 
> --Mark
> 
> > Thanks, Dongxiao
> >


[-- Attachment #2: install_solution.manifest --]
[-- Type: application/x-ms-manifest, Size: 33773 bytes --]