Implicit RPROVIDES somewhere ?

Implicit RPROVIDES somewhere ?

[Joel GUITTET]

[-- Attachment #1: Type: text/plain, Size: 840 bytes --]

Hello,

I have a target with multilib aarch64+arm.
I'm building a library "foo" that DEPENDS of library "bar". I need the library "foo" to be installed in the rootfs, but not the library "bar" (for specific reason in my project).
The recipe of library "foo" DEPENDS of "bar" but no RDEPENDS is used.

arm arch: I observe that the library "foo" is installed in "/usr/lib", and library "bar" is not installed, as expected.
aarch64 arch: I observe that the library "foo" is installed in "/usr/lib64", and library "bar" is installed in "/usr/lib64" as well, which is not what I want !!

It seems there is like an implicit RDEPENDS there, but only for the aarch64 architecture.

I'm looking to the right configuration to do here and will appreciate any inputs/topics to be checked in my project to progress on this.

Thanks
Joel

[-- Attachment #2: Type: text/html, Size: 900 bytes --]

Re: [yocto] Implicit RPROVIDES somewhere ?

[Quentin Schulz]

Hi Joel,

On 4/16/24 16:50, Joel GUITTET via lists.yoctoproject.org wrote:
> You don't often get email from jguittet.opensource=witekio.com@lists.yoctoproject.org. Learn why this is important<https://aka.ms/LearnAboutSenderIdentification>
> Hello,
> 
> I have a target with multilib aarch64+arm.
> I'm building a library "foo" that DEPENDS of library "bar". I need the library "foo" to be installed in the rootfs, but not the library "bar" (for specific reason in my project).

It'd be nice to know what those specific reasons are, maybe they are not 
reasonable in the context of building with Yocto :) (and we may be able 
to suggest other options, like rather fixing the software build system 
instead of hacking deep into Yocto if there's no other option).

> The recipe of library "foo" DEPENDS of "bar" but no RDEPENDS is used.
> 
> arm arch: I observe that the library "foo" is installed in "/usr/lib", and library "bar" is not installed, as expected.
> aarch64 arch: I observe that the library "foo" is installed in "/usr/lib64", and library "bar" is installed in "/usr/lib64" as well, which is not what I want !!
> 
> It seems there is like an implicit RDEPENDS there, but only for the aarch64 architecture.
> 

Packages providing SONAME entries in a shared library/binary from 
another package are always part of RDEPENDS so I would start there.

https://docs.yoctoproject.org/overview-manual/concepts.html#automatically-added-runtime-dependencies

You can check by using readelf or objdump on your libraries/binaries in 
foo packages. I **think** bitbake also tells you about this implicit 
dependency and the reason for it if you increase the debug level (-D -DD 
or even -DDD?).

It could very well be that your SW is built differently for aarch64 than 
for arm and it actually links against libbar. THen you would need to fix 
your piece of SW to not link against it.

Cheers,
Quentin

Re: [yocto] Implicit RPROVIDES somewhere ?

[Joel GUITTET]

[-- Attachment #1: Type: text/plain, Size: 2554 bytes --]

Hello Quentin,

Thanks for the quick reply on this question!

The reason of this organisation is that my customer is responsible of providing library "bar" (he cannot share the source code with me), and we are responsible of providing library "foo". It's not confortable for sure, but it's the actual state. So my customer provides me the "bar" libraries (arm and aarch64) compiled (so files), that I copy/paste in my layer. And then I build "foo" libraries that are linked to "bar" and additionally I have to install only the "foo" libraries in the rootfs.

The library that is building "foo" have nothing specific to the architecture. The IMAGE_INSTALL has "lib32-foo" and "foo".
But something to mention that may have an impact here : because of this organization above my recipe that is building "foo" have the following (else I don't succeed to build):

> 
> FILES_SOLIBSDEV = ""
> INSANE_SKIP:${PN} += "dev-so"
> INSANE_SKIP:${PN} += "file-rdeps"
> 

I checked the SONAME / NEEDED of the libraries with readelf for arm and aarch64 "foo" libraries:

> 
> readelf -d path/to/image/1.0-r0/rootfs/usr/lib/libfoo.so
> 
> Dynamic section at offset 0xad3c contains 31 entries:
> Tag        Type                         Name/Value
> 0x00000001 (NEEDED)                     Shared library: [libcrypto.so.3]
> 0x00000001 (NEEDED)                     Shared library: [libbar.so]
> 0x00000001 (NEEDED)                     Shared library: [libc.so.6]
> 0x00000001 (NEEDED)                     Shared library: [ld-linux.so.3]
> 0x0000000e (SONAME)                     Library soname: [libfoo.so]
> 
> 
> 

etc (details removed for brevity)

> 
> readelf -d path/to/image/1.0-r0/rootfs/usr/lib64/libfoo.so
> 
> Dynamic section at offset 0xfa58 contains 32 entries:
> Tag        Type                         Name/Value
> 0x0000000000000001 (NEEDED)             Shared library: [libcrypto.so.3]
> 0x0000000000000001 (NEEDED)             Shared library: [libbar.so]
> 0x0000000000000001 (NEEDED)             Shared library: [libc.so.6]
> 0x0000000000000001 (NEEDED)             Shared library:
> [ld-linux-aarch64.so.1]
> 0x000000000000000e (SONAME)             Library soname: [libfoo.so]
> 

etc (details removed for brevity)

So actually this doesn't helped me to understand the implicit RDEPENDS on aarch64 that is not present on arm arch because both indicate the need libbar.so.

Joel

[-- Attachment #2: Type: text/html, Size: 3492 bytes --]

Re: [yocto] Implicit RPROVIDES somewhere ?

[Quentin Schulz]

Hi Joel,

On 4/16/24 19:09, Joel GUITTET via Lists.Yoctoproject.Org wrote:
> You don't often get email from jguittet.opensource=witekio.com@lists.yoctoproject.org. Learn why this is important<https://aka.ms/LearnAboutSenderIdentification>
> Hello Quentin,
> 
> Thanks for the quick reply on this question!
> 
> The reason of this organisation is that my customer is responsible of providing library "bar" (he cannot share the source code with me), and we are responsible of providing library "foo". It's not confortable for sure, but it's the actual state. So my customer provides me the "bar" libraries (arm and aarch64) compiled (so files), that I copy/paste in my layer. And then I build "foo" libraries that are linked to "bar" and additionally I have to install only the "foo" libraries in the rootfs.
> 

So far so good. Not an uncommon setup, bar recipe would be following the 
prebuilt binary section in the documentation. 
https://docs.yoctoproject.org/dev-manual/prebuilt-libraries.html

The thing I do not understand is why you do NOT want bar on the rootfs? 
Your foo shared library explicitly states that it requires libbar.so to 
be able to run, why do you not want libbar.so in your rootfs?

> The library that is building "foo" have nothing specific to the architecture. The IMAGE_INSTALL has "lib32-foo" and "foo".
> But something to mention that may have an impact here : because of this organization above my recipe that is building "foo" have the following (else I don't succeed to build):

This smells like bad build system/instructions for the foo recipe. All 
libraries built by Yocto should be versioned, I highly recommend to fix 
your foo SW to have some version in there to not have to deal with this. 
It's best practice and will help you in the long run anyway (actually 
trying to figure out which version the customer is using by simply 
looking at the filename for example). But I don't think that should 
impact libfoo being added or not.

> 
> FILES_SOLIBSDEV = ""
> INSANE_SKIP:${PN} += "dev-so"
> INSANE_SKIP:${PN} += "file-rdeps"
> 

This INSANE_SKIP isn't that common and I don't think it's that good of 
an idea?

There's one thing I missed is that you're building with multilib. Do you 
actually have a lib32-bar package and a bar package? Is your libbar.so 
available in both aarch32 and aarch64 architectures?

I assume that the automagic RDEPENDS got broken for lib32-foo and you 
are not seeing it because of
INSANE_SKIP:${PN} += "file-rdeps"

It's probably trying to locate a multilib lib32 package containing 
libbar.so but cannot find any and complains about it. The aarch64 
package does find a package for libbar.so so it adds it automatically.

Cheers,
Quentin

Re: [yocto] Implicit RPROVIDES somewhere ?

[Joel GUITTET]

[-- Attachment #1: Type: text/plain, Size: 1465 bytes --]

Hello Quentin,

Yes not uncommon setup except my prebuilt libraries don't match the initial expectations:
1. The filename must have the version appended, for example: libbar.so.1.2.3. (can be easily renamed by the way)
2. The library must have the ELF tag SONAME set to the major version of the library, for example: libbar.so.1. (it has no SONAME at all and it's not possible to have it, and not wanted to patch the library before using it to build libfoo).

My customer don't want me to add the bar libraries in the rootfs. he ask me to add only the foo libraries, for both architectures aarch64 and arm. Yes I have bar libraries pre-compiled for both aarch64 and arm too. So this is lib32-bar and bar.

> 
> I assume that the automagic RDEPENDS got broken for lib32-foo and you are
> not seeing it because of INSANE_SKIP:${PN} += "file-rdeps"
> It's probably trying to locate a multilib lib32 package containing
> libbar.so but cannot find any and complains about it. The aarch64 package
> does find a package for libbar.so so it adds it automatically.
> 

Interesting remark. Maybe I have to play with the INSANE_SKIP(s).
So the "file-rdeps" skip breaks the RDEPENDS on lib32-foo, which permit me to not integrate lib32-bar in the rootfs (what I want), but it's maybe not working with aarch64 ? Is there a "aarch64-file-rdeps" or similar (I don't think so) that could help to break the RDEPENDS of my library foo on the library bar ?

Joel

[-- Attachment #2: Type: text/html, Size: 1685 bytes --]

Re: [yocto] Implicit RPROVIDES somewhere ?

[Quentin Schulz]

Hi Joel,

On 4/18/24 10:50, Joel GUITTET via Lists.Yoctoproject.Org wrote:
> Hello Quentin,
> 
> Yes not uncommon setup except my prebuilt libraries don't match the initial expectations:
> 1. The filename must have the version appended, for example: libbar.so.1.2.3. (can be easily renamed by the way)
> 2. The library must have the ELF tag SONAME set to the major version of the library, for example: libbar.so.1. (it has no SONAME at all and it's not possible to have it, and not wanted to patch the library before using it to build libfoo).
> 

libbar not being versioned is common too, so that should be covered by 
the docs? libfoo though, as being built by Yocto should really be 
versioned itself. It doesn't need to use versioned libs as dependencies 
(though it would really be a good idea ;) ).

> My customer don't want me to add the bar libraries in the rootfs. he ask me to add only the foo libraries, for both architectures aarch64 and arm. Yes I have bar libraries pre-compiled for both aarch64 and arm too. So this is lib32-bar and bar.
> 

Your customer is requesting an image that cannot work? I'm not sure to 
understand his logic? What are they trying to prevent to happen? If they 
want to go this route, maybe using dlopen instead of linking against 
would be better maybe? This way the auto RDEPENDS mechanism wouldn't 
find it?

I assume he wants to share the bar library himself and not through your 
image... An option is just to ask him to build the image himself instead 
by providing all the layers and setup to him, and he'll have the ability 
to build an image with whatever he wants in it, but an image that 
actually works.

>>
>> I assume that the automagic RDEPENDS got broken for lib32-foo and you are
>> not seeing it because of INSANE_SKIP:${PN} += "file-rdeps"
>> It's probably trying to locate a multilib lib32 package containing
>> libbar.so but cannot find any and complains about it. The aarch64 package
>> does find a package for libbar.so so it adds it automatically.
>>
> 
> Interesting remark. Maybe I have to play with the INSANE_SKIP(s). > So the "file-rdeps" skip breaks the RDEPENDS on lib32-foo, which 
permit me to not integrate lib32-bar in the rootfs (what I want), but 
it's maybe not working with aarch64 ? Is there a "aarch64-file-rdeps" or 
similar (I don't think so) that could help to break the RDEPENDS of my 
library foo on the library bar ?
> 

You won't be able to further break the build system by using 
INSANE_SKIP. INSANE_SKIP is for silencing warnings/errors. It's usually 
when one messes things up but says "trust me I know what I'm doing" to 
Yocto.

So here, your build is broken for lib32-foo but you silenced it. To 
achieve what you want, you need to break it for foo, and then silence it 
as well.

Digging a bit, maybe EXCLUDE_FROM_SHLIBS could be used in foo recipe to 
exclude bar package. I guess you could also look into making the bar 
recipe use PRIVATE_LIBS for libbar.so.

But honestly, this is just going against what Yocto should be used for, 
so try to find some compromise with your customer to do things properly :)

Cheers,
Quentin

Re: [yocto] Implicit RPROVIDES somewhere ?

[Joel GUITTET]

[-- Attachment #1: Type: text/plain, Size: 132 bytes --]

Quentin,

> 
> this is just going against what Yocto should be used for,

Thanks. I totally agree with you!

Regards,
Joel

[-- Attachment #2: Type: text/html, Size: 230 bytes --]