gio-querymodules: error while loading shared libraries: libffi.so.6: cannot open shared object file

gio-querymodules: error while loading shared libraries: libffi.so.6: cannot open shared object file

[Bryan Evenson]

All,

I'm having some upgrade issues related to upgrading libglib-2.0-0.  I have a device that is based on the morty release that I am upgrading to a build based on the dunfell release.  I am using opkg for a package manager.  During upgrade I see the error message in the subject.  I think I have an idea of what is happening and I'm looking for help on how to fix it.

I seeh that libglib-2.0-0 depends on libffi.  Between the morty release and the dunfell release, libffi upgraded from libffi6 to libffi7.  When I upgrade my system, libffi upgrades from libffi6 to libffi7 prior to libglib-2.0-0 upgrading.  When libglib-2.0-0 upgrades, it calls '/usr/libexec/gio-querymodules /usr/lib/gio/modules/' in its postrm script.  This postrm script fails and gives the error listed the subject line.  I'm assuming that this fails because gio-querymodules depends on libffi6 which is no longer installed.

Has anyone run into a similar problem and found a way to fix it?  I've tried a few different ways to force a different upgrade order and so far nothing works.

Thanks,
Bryan

Re: [OE-core] gio-querymodules: error while loading shared libraries: libffi.so.6: cannot open shared object file

[Alexander Kanavin]

Upgrading from one yocto release to a different yocto release with a
package manager is not supported or tested. You need to replace the
image completely.

Alex

On Mon, 7 Feb 2022 at 19:49, Bryan Evenson <bevenson@melinkcorp.com> wrote:
>
> All,
>
> I'm having some upgrade issues related to upgrading libglib-2.0-0.  I have a device that is based on the morty release that I am upgrading to a build based on the dunfell release.  I am using opkg for a package manager.  During upgrade I see the error message in the subject.  I think I have an idea of what is happening and I'm looking for help on how to fix it.
>
> I seeh that libglib-2.0-0 depends on libffi.  Between the morty release and the dunfell release, libffi upgraded from libffi6 to libffi7.  When I upgrade my system, libffi upgrades from libffi6 to libffi7 prior to libglib-2.0-0 upgrading.  When libglib-2.0-0 upgrades, it calls '/usr/libexec/gio-querymodules /usr/lib/gio/modules/' in its postrm script.  This postrm script fails and gives the error listed the subject line.  I'm assuming that this fails because gio-querymodules depends on libffi6 which is no longer installed.
>
> Has anyone run into a similar problem and found a way to fix it?  I've tried a few different ways to force a different upgrade order and so far nothing works.
>
> Thanks,
> Bryan
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#161463): https://lists.openembedded.org/g/openembedded-core/message/161463
> Mute This Topic: https://lists.openembedded.org/mt/88979156/1686489
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alex.kanavin@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>

RE: [OE-core] gio-querymodules: error while loading shared libraries: libffi.so.6: cannot open shared object file

[Bryan Evenson]

Alex,

> -----Original Message-----
> From: Alexander Kanavin <alex.kanavin@gmail.com>
> Sent: Monday, February 7, 2022 1:58 PM
> To: Bryan Evenson <bevenson@melinkcorp.com>
> Cc: Patches and discussions about the oe-core layer <openembedded-
> core@lists.openembedded.org>
> Subject: Re: [OE-core] gio-querymodules: error while loading shared
> libraries: libffi.so.6: cannot open shared object file
> 
> Upgrading from one yocto release to a different yocto release with a package
> manager is not supported or tested. You need to replace the image
> completely.
> 

I traced back and I see the gio-querymodules call is inserted in the postrm by gio-module-class.bbclass.  Based on the commit notes, the postrm and postinst steps are an optimization and are not required to succeed.  Therefore I'd argue this should fail gracefully; just because we haven't yet seen this fail during minor upgrade attempts doesn't mean it can't.  I'd propose changing the line:

    ${libexecdir}/${MLPREFIX}gio-querymodules ${libdir}/gio/modules/

in gio-module-cache.bbclass to:

    ${libexecdir}/${MLPREFIX}gio-querymodules ${libdir}/gio/modules/ || true

I tested this change and it fixed my upgrade issue.  I still saw the error message but libglib-2.0 completed its upgrade successfully.  If you don't see any issues with this change, I'll send a patch out later today.

Thanks,
Bryan

> Alex
> 
> On Mon, 7 Feb 2022 at 19:49, Bryan Evenson <bevenson@melinkcorp.com>
> wrote:
> >
> > All,
> >
> > I'm having some upgrade issues related to upgrading libglib-2.0-0.  I have a
> device that is based on the morty release that I am upgrading to a build
> based on the dunfell release.  I am using opkg for a package manager.  During
> upgrade I see the error message in the subject.  I think I have an idea of what
> is happening and I'm looking for help on how to fix it.
> >
> > I seeh that libglib-2.0-0 depends on libffi.  Between the morty release and
> the dunfell release, libffi upgraded from libffi6 to libffi7.  When I upgrade my
> system, libffi upgrades from libffi6 to libffi7 prior to libglib-2.0-0 upgrading.
> When libglib-2.0-0 upgrades, it calls '/usr/libexec/gio-querymodules
> /usr/lib/gio/modules/' in its postrm script.  This postrm script fails and gives
> the error listed the subject line.  I'm assuming that this fails because gio-
> querymodules depends on libffi6 which is no longer installed.
> >
> > Has anyone run into a similar problem and found a way to fix it?  I've tried a
> few different ways to force a different upgrade order and so far nothing
> works.
> >
> > Thanks,
> > Bryan
> >
> > -=-=-=-=-=-=-=-=-=-=-=-
> > Links: You receive all messages sent to this group.
> > View/Reply Online (#161463):
> > https://lists.openembedded.org/g/openembedded-core/message/161463
> > Mute This Topic: https://lists.openembedded.org/mt/88979156/1686489
> > Group Owner: openembedded-core+owner@lists.openembedded.org
> > Unsubscribe: https://lists.openembedded.org/g/openembedded-
> core/unsub
> > [alex.kanavin@gmail.com]
> > -=-=-=-=-=-=-=-=-=-=-=-
> >

Re: [OE-core] gio-querymodules: error while loading shared libraries: libffi.so.6: cannot open shared object file

[Alexander Kanavin]

[-- Attachment #1: Type: text/plain, Size: 1171 bytes --]

On Tue, 8 Feb 2022 at 16:25, Bryan Evenson <bevenson@melinkcorp.com> wrote:

> I traced back and I see the gio-querymodules call is inserted in the
> postrm by gio-module-class.bbclass.  Based on the commit notes, the postrm
> and postinst steps are an optimization and are not required to succeed.
> Therefore I'd argue this should fail gracefully; just because we haven't
> yet seen this fail during minor upgrade attempts doesn't mean it can't.
> I'd propose changing the line:
>
>     ${libexecdir}/${MLPREFIX}gio-querymodules ${libdir}/gio/modules/
>
> in gio-module-cache.bbclass to:
>
>     ${libexecdir}/${MLPREFIX}gio-querymodules ${libdir}/gio/modules/ ||
> true
>
> I tested this change and it fixed my upgrade issue.  I still saw the error
> message but libglib-2.0 completed its upgrade successfully.  If you don't
> see any issues with this change, I'll send a patch out later today.
>

This change will mask all failures that happen for any reason, not just the
one you saw, so I am not going to accept it. The correct fix is to figure
out what would be the correct order of updating and why opkg isn't able to
perform the mass-update in that order.

Alex

[-- Attachment #2: Type: text/html, Size: 1657 bytes --]

RE: [OE-core] gio-querymodules: error while loading shared libraries: libffi.so.6: cannot open shared object file

[Bryan Evenson]

[-- Attachment #1: Type: text/plain, Size: 2628 bytes --]

Alex,

From: Alexander Kanavin <alex.kanavin@gmail.com>
Sent: Tuesday, February 8, 2022 11:59 AM
To: Bryan Evenson <bevenson@melinkcorp.com>
Cc: Patches and discussions about the oe-core layer <openembedded-core@lists.openembedded.org>
Subject: Re: [OE-core] gio-querymodules: error while loading shared libraries: libffi.so.6: cannot open shared object file

On Tue, 8 Feb 2022 at 16:25, Bryan Evenson <bevenson@melinkcorp.com<mailto:bevenson@melinkcorp.com>> wrote:
I traced back and I see the gio-querymodules call is inserted in the postrm by gio-module-class.bbclass.  Based on the commit notes, the postrm and postinst steps are an optimization and are not required to succeed.  Therefore I'd argue this should fail gracefully; just because we haven't yet seen this fail during minor upgrade attempts doesn't mean it can't.  I'd propose changing the line:

    ${libexecdir}/${MLPREFIX}gio-querymodules ${libdir}/gio/modules/

in gio-module-cache.bbclass to:

    ${libexecdir}/${MLPREFIX}gio-querymodules ${libdir}/gio/modules/ || true

I tested this change and it fixed my upgrade issue.  I still saw the error message but libglib-2.0 completed its upgrade successfully.  If you don't see any issues with this change, I'll send a patch out later today.

This change will mask all failures that happen for any reason, not just the one you saw, so I am not going to accept it. The correct fix is to figure out what would be the correct order of updating and why opkg isn't able to perform the mass-update in that order.

Alex

As best I can tell, this is breaking because libffi upgraded from libffi6 to libffi7.  The old version of glib-2.0 depends on libffi6 and the new version of glib-2.0 depends on libffi7.  Opkg starts to upgrade glib-2.0, but it first upgrades the dependent packages.  This removes libffi6 and installs libffi7.  Then, the postrm script runs for glib-2.0 using the old binaries.  The gio-querymodules call then fails because it tries to load libffi6 which no longer exists.  Even if I was able to defer upgrading libffi from libffi6 to libffi7 until after glib-2.0 upgrades, I’d then run into a similar problem with the postinst script.  In that case, gio-querymodules would fail because it would try to load libffi7 which would not be installed yet.

I understand that you don’t want errors to go through undetected.  Would an appropriate error message, such as:

    ${libexecdir}/${MLPREFIX}gio-querymodules ${libdir}/gio/modules/ || echo “GIO module caching failed.  You may need to manually update the GIO module cache.”

Work for you?

Thanks,
Bryan

[-- Attachment #2: Type: text/html, Size: 5608 bytes --]

Re: [OE-core] gio-querymodules: error while loading shared libraries: libffi.so.6: cannot open shared object file

[Alexander Kanavin]

On Tue, 8 Feb 2022 at 21:36, Bryan Evenson <bevenson@melinkcorp.com> wrote:
> As best I can tell, this is breaking because libffi upgraded from libffi6 to libffi7.  The old version of glib-2.0 depends on libffi6 and the new version of glib-2.0 depends on libffi7.  Opkg starts to upgrade glib-2.0, but it first upgrades the dependent packages.  This removes libffi6 and installs libffi7.  Then, the postrm script runs for glib-2.0 using the old binaries.  The gio-querymodules call then fails because it tries to load libffi6 which no longer exists.  Even if I was able to defer upgrading libffi from libffi6 to libffi7 until after glib-2.0 upgrades, I’d then run into a similar problem with the postinst script.  In that case, gio-querymodules would fail because it would try to load libffi7 which would not be installed yet.
> I understand that you don’t want errors to go through undetected.  Would an appropriate error message, such as:
>     ${libexecdir}/${MLPREFIX}gio-querymodules ${libdir}/gio/modules/ || echo “GIO module caching failed.  You may need to manually update the GIO module cache.”
> Work for you?

That is not an error message, that is a warning message, and no, that
is not appropriate either, because it's likely to go unseen, or
discarded.

I think you should study how desktop distributions handle this, both
rpm-based and deb-based. How do their postinst/postrms look like, and
in what sequence do they run?

As mentioned before, Yocto does not test or support mass-upgrades via
package manager: if we want to fix this, we better do it properly from
the start.

Alex

RE: [OE-core] gio-querymodules: error while loading shared libraries: libffi.so.6: cannot open shared object file

[Bryan Evenson]

Alex,

> -----Original Message-----
> From: Alexander Kanavin <alex.kanavin@gmail.com>
> Sent: Tuesday, February 8, 2022 4:38 PM
> To: Bryan Evenson <bevenson@melinkcorp.com>
> Cc: Patches and discussions about the oe-core layer <openembedded-
> core@lists.openembedded.org>
> Subject: Re: [OE-core] gio-querymodules: error while loading shared
> libraries: libffi.so.6: cannot open shared object file
> 
> On Tue, 8 Feb 2022 at 21:36, Bryan Evenson <bevenson@melinkcorp.com>
> wrote:
> > As best I can tell, this is breaking because libffi upgraded from libffi6 to
> libffi7.  The old version of glib-2.0 depends on libffi6 and the new version of
> glib-2.0 depends on libffi7.  Opkg starts to upgrade glib-2.0, but it first
> upgrades the dependent packages.  This removes libffi6 and installs libffi7.
> Then, the postrm script runs for glib-2.0 using the old binaries.  The gio-
> querymodules call then fails because it tries to load libffi6 which no longer
> exists.  Even if I was able to defer upgrading libffi from libffi6 to libffi7 until
> after glib-2.0 upgrades, I’d then run into a similar problem with the postinst
> script.  In that case, gio-querymodules would fail because it would try to load
> libffi7 which would not be installed yet.
> > I understand that you don’t want errors to go through undetected.  Would
> an appropriate error message, such as:
> >     ${libexecdir}/${MLPREFIX}gio-querymodules ${libdir}/gio/modules/ ||
> echo “GIO module caching failed.  You may need to manually update the GIO
> module cache.”
> > Work for you?
> 
> That is not an error message, that is a warning message, and no, that is not
> appropriate either, because it's likely to go unseen, or discarded.
> 
> I think you should study how desktop distributions handle this, both rpm-
> based and deb-based. How do their postinst/postrms look like, and in what
> sequence do they run?

For the Debian upgrade policy (flowchart here: https://www.debian.org/doc/debian-policy/ap-flowcharts.html), successful events occur in the following order:
1. Call prerm script from the old package
2. Call preinst script from the new package
3. Unpack new files and save backup of old files
4. Call postrm script from the old package
5. Delete old files
6. Call postinst script from the new package
7. Exit package upgrade

Opkg does something slightly different so it can operate with a smaller memory footprint.  For opkg the event order is:
1. Call prerm script from the old package
2. Call preinst script from the new package
3. Call postrm script from the old package
4. Unpack new files and delete old files
5. Call postinst script from the new package
6. Exit package upgrade

This saves space since there won't be a copy of every file in the package during upgrade.  It also reduces code complexity in opkg.  But, it also means the new files aren't available until the postinst stage.  This is why I'm seeing the failure that I am with opkg.  So the postrm script is failing because opkg isn't meeting the Debian policy.

However, I still think that a package upgrade should have a way to exit gracefully for something that is an optimization and not required for operation.  In my case, my device isn't even using the GIO library and the gio-querymodules serves no purpose.  I don't know enough about the entire Glib package to know If there is a way to make this functionality optional or not.  If there isn't, then I will continue with my own out-of-tree changes to fix the operational problems I am seeing on my device.

Thanks,
Bryan

> 
> As mentioned before, Yocto does not test or support mass-upgrades via
> package manager: if we want to fix this, we better do it properly from the
> start.
> 
> Alex

Re: [OE-core] gio-querymodules: error while loading shared libraries: libffi.so.6: cannot open shared object file

[Alexander Kanavin]

On Fri, 11 Feb 2022 at 14:46, Bryan Evenson <bevenson@melinkcorp.com> wrote:

> However, I still think that a package upgrade should have a way to exit gracefully for something that is an optimization and not required for operation.  In my case, my device isn't even using the GIO library and the gio-querymodules serves no purpose.  I don't know enough about the entire Glib package to know If there is a way to make this functionality optional or not.  If there isn't, then I will continue with my own out-of-tree changes to fix the operational problems I am seeing on my device.

Thanks, does rpm do something similar to debian? This does look as a
shortcoming in opkg design; what would help you is perhaps a comman
line option for opkg, --ignore-scriptlet-failures or similar.

Alex

RE: [OE-core] gio-querymodules: error while loading shared libraries: libffi.so.6: cannot open shared object file

[Bryan Evenson]

Alex,

> -----Original Message-----
> From: Alexander Kanavin <alex.kanavin@gmail.com>
> Sent: Friday, February 11, 2022 10:06 AM
> To: Bryan Evenson <bevenson@melinkcorp.com>
> Cc: Patches and discussions about the oe-core layer <openembedded-
> core@lists.openembedded.org>
> Subject: Re: [OE-core] gio-querymodules: error while loading shared
> libraries: libffi.so.6: cannot open shared object file
> 
> On Fri, 11 Feb 2022 at 14:46, Bryan Evenson <bevenson@melinkcorp.com>
> wrote:
> 
> > However, I still think that a package upgrade should have a way to exit
> gracefully for something that is an optimization and not required for
> operation.  In my case, my device isn't even using the GIO library and the gio-
> querymodules serves no purpose.  I don't know enough about the entire
> Glib package to know If there is a way to make this functionality optional or
> not.  If there isn't, then I will continue with my own out-of-tree changes to fix
> the operational problems I am seeing on my device.
> 
> Thanks, does rpm do something similar to debian? This does look as a
> shortcoming in opkg design; what would help you is perhaps a comman line
> option for opkg, --ignore-scriptlet-failures or similar.

From what I have read, it appears RPM functions very similar to Debian.  The new package files are installed prior to postrm, with the old files temporarily backed up and removed later.  I'll look at some of the other opkg options to see if there are some ignore options that I haven't tried yet.

Thanks,
Bryan

> 
> Alex

Re: [OE-core] gio-querymodules: error while loading shared libraries: libffi.so.6: cannot open shared object file

[Richard Purdie]

On Fri, 2022-02-11 at 15:41 +0000, Bryan Evenson wrote:
> Alex,
> 
> > -----Original Message-----
> > From: Alexander Kanavin <alex.kanavin@gmail.com>
> > Sent: Friday, February 11, 2022 10:06 AM
> > To: Bryan Evenson <bevenson@melinkcorp.com>
> > Cc: Patches and discussions about the oe-core layer <openembedded-
> > core@lists.openembedded.org>
> > Subject: Re: [OE-core] gio-querymodules: error while loading shared
> > libraries: libffi.so.6: cannot open shared object file
> > 
> > On Fri, 11 Feb 2022 at 14:46, Bryan Evenson <bevenson@melinkcorp.com>
> > wrote:
> > 
> > > However, I still think that a package upgrade should have a way to exit
> > gracefully for something that is an optimization and not required for
> > operation.  In my case, my device isn't even using the GIO library and the gio-
> > querymodules serves no purpose.  I don't know enough about the entire
> > Glib package to know If there is a way to make this functionality optional or
> > not.  If there isn't, then I will continue with my own out-of-tree changes to fix
> > the operational problems I am seeing on my device.
> > 
> > Thanks, does rpm do something similar to debian? This does look as a
> > shortcoming in opkg design; what would help you is perhaps a comman line
> > option for opkg, --ignore-scriptlet-failures or similar.
> 
> From what I have read, it appears RPM functions very similar to Debian.  The
> new package files are installed prior to postrm, with the old files
> temporarily backed up and removed later.  I'll look at some of the other opkg
> options to see if there are some ignore options that I haven't tried yet.

I think this would be worth raising with the opkg maintainer as this does sound
like an issue with opkg :/.

Cheers,

Richard