Re: [OE-core] Current native SDK glibc compat

["Anatol Belski" <anbelski@linux.microsoft.com>]

[-- Attachment #1: Type: text/plain, Size: 7107 bytes --]

Hi,

On 2/25/2021 1:23 AM, Randy MacLeod wrote:
> On 2021-02-24 6:57 p.m., Richard Purdie wrote:
>> On Wed, 2021-02-24 at 21:16 +0100, Anatol Belski wrote:
>>> On 2/24/2021 5:49 PM, Richard Purdie wrote:
>>>> No, I mean the dynamic loader pointer.
>>>>
>>>> $ tmp/sysroots-uninative/x86_64-linux/usr/bin/patchelf-uninative 
>>>> python3-native/python3.9 --print-interpreter
>>>> [...]tmp/sysroots-uninative/x86_64-linux/lib/ld-linux-x86-64.so.2
>>>>
>>>> Above I'm showing that a native binary in the build (python3-native)
>>>> has the interpreter (dynamic loader) set to our uninative ld.so.
>>>> The SDK is similar.
>>>>
>>>>> As the binary where the issue was sighted has this
>>>>>
>>>>> $ chrpath $(which cargo)
>>>>> /tmp/rust-sdk-deploy-18/sysroots/x86_64-pokysdk-linux/usr/bin/cargo:
>>>>> RUNPATH=$ORIGIN/../lib
>>>>>
>>>>>
>>>>> but then, the DSOs have no rpath set, eg.
>>>>>
>>>>> $ chrpath
>>>>> /tmp/rust-sdk-deploy-18/sysroots/x86_64-pokysdk-linux/usr/bin/../lib/libcrypto.so.1.1 
>>>>>
>>>>> /tmp/rust-sdk-deploy-18/sysroots/x86_64-pokysdk-linux/usr/bin/../lib/libcrypto.so.1.1: 
>>>>>
>>>>> no rpath or runpath tag found.
>>>>>
>>>>>
>>>>> so it might lead to the interferrence with the host. Does it perhaps
>>>>> need both $ORIGIN/../../lib and $ORIGIN/../lib if binaries are in 
>>>>> /usr ?
>>>> Our dynamic loader knows how to use the specific sysroot and then
>>>> fall back to /usr and /lib.
>>>
>>> Thanks a lot for explaining this.
>>>
>>> Getting back to the initial case led to the question, i'm now able to
>>> check what is the correct dynamic loader:
>>>
>>> $ tmp/sysroots-uninative/x86_64-linux/usr/bin/patchelf-uninative
>>> /tmp/rust-sdk-deploy-18/sysroots/x86_64-pokysdk-linux/usr/bin/cargo
>>> --print-interpreter
>>> /tmp/rust-sdk-deploy-18/sysroots/x86_64-pokysdk-linux/lib/ld-linux-x86-64.so.2 
>>>
>>>
>>> and also could use --print-needed which would be similar to "readelf -d
>>> ", but it'd be still not possible to check things at the runtime? OFC
>>> knowing what i know now, it's possible to locate the DSO and check
>>> symbols like this:
>>>
>>> $ x86_64-poky-linux-objdump -T
>>> ./sysroots/x86_64-pokysdk-linux/lib/libc.so.6 | grep GLIBC_2.33
>>>
>>> just seems the catch points are quite subtle :) Perhaps there could 
>>> be a
>>> recommendation on a good way validating the binaries in the SDK HOST 
>>> part?
>>
>> If you what to know what a partiular command is doing for symbols at 
>> runtime,
>> you can run it as:
>>
>> LD_DEBUG=all <command>
>>
>> which will have the loader spew out a lot of information about how it is
>> resolving the symbols.

Thanks for this hint. Applied to the current dev artifacts i can now see 
what is loaded:

$ LD_DEBUG=all cargo 2>&1 | grep GLIBC_2.33
      29463:     checking for version `GLIBC_2.33' in file 
/tmp/rust-sdk-deploy-18/sysroots/x86_64-pokysdk-linux/lib/libc.so.6 [0] 
required by file 
/tmp/rust-sdk-deploy-18/sysroots/x86_64-pokysdk-linux/usr/bin/../lib/libcrypto.so.1.1 
[0]
      29463:     checking for version `GLIBC_2.33' in file 
/tmp/rust-sdk-deploy-18/sysroots/x86_64-pokysdk-linux/lib/libc.so.6 [0] 
required by file 
/tmp/rust-sdk-deploy-18/sysroots/x86_64-pokysdk-linux/usr/bin/../lib/libcurl.so.4 
[0]


>
> I had other things to do today but for 'fun', in the background, I took:
>   poky/master + meta-oe/master + meta-rust with Anatol's SDK commits
> and bisected poky back to:
>
>   7b8df042d0 (HEAD, refs/bisect/bad) glibc: Upgrade to 2.33
>        (From OE-Core rev: aa87638cf4f2bef66df92f961c7814f6b482fd3d)
>
> This is when I expected problems with the SDK to start but
> I thought it was worth confirming mechanically.
>
> Note that the two last bisected commits after the uprev of glibc fail
> to even complete the populate_sdk due to the error:
>
>    ERROR: core-image-minimal-1.0-r0 do_populate_sdk:
>    The postinstall intercept hook 'update_gio_module_cache' failed,

I recall failures in post install scripts failing due to missing files, 
where scripts under scripts/postinst-intercepts/ sometimes make wrong 
assumptions. Unfortunately we never caught the causes. Anyway, this one 
doesn't look related to glibc whatsoever.

> Once Richard is happy with the basic meta-rust merge, I can spend more
> time and actually look at the linking and debug what's going wrong
> if Anatol has not figure it out before that.
>
Yeah, i'll continue working on Rust SDK integration and in particular 
checking through teh current patch. On the particular glbic issue - 
while I built using master to pursue a reproduce, I also used ldd for 
the checks. However, as we've just learned ldd would be the wrong way - 
other than that, the binaries wasn't showing the warnings about missing 
symbols at runtime. I'm also more focused on the Rust itself, but anyway 
will keep eyes open to this.


Regards

Anatol


> ../Randy
>
>
> $ git bisect log
>
> git bisect start
> # bad: [1fe4a25f2244fdf67d96109dcb4f49ed75c18b32]
>     diffoscope: Ensure rpm is configured correctly
>
> git bisect bad 1fe4a25f2244fdf67d96109dcb4f49ed75c18b32
> # bad: [1fe4a25f2244fdf67d96109dcb4f49ed75c18b32]
>     diffoscope: Ensure rpm is configured correctly
>
> git bisect bad 1fe4a25f2244fdf67d96109dcb4f49ed75c18b32
> # good: [77d9b5f02aec991540926fe144076e122c17f64d]
>     pseudo: Update to work with glibc 2.33
>
> git bisect good 77d9b5f02aec991540926fe144076e122c17f64d
> # bad: [b6018a1625c22393c1f94e9d23d84f7842b95f24]
>     acpica: Fix reproducibility issues
>
> git bisect bad b6018a1625c22393c1f94e9d23d84f7842b95f24
> # bad: [7283a0b3b6ca49d0d2e13593333a580ef10439a8]
>     bitbake: bblayers/action: When adding layers,
>        catch BBHandledException
>
> git bisect bad 7283a0b3b6ca49d0d2e13593333a580ef10439a8
> # bad: [c5d80f154de30a6f59f28d4d9d05edcd78469765]
>     selftest/reproducible: remove spirv-tools-dev from exclusion list
>
> git bisect bad c5d80f154de30a6f59f28d4d9d05edcd78469765
> # bad: [adcd9608a725d99e2e6ca74d4a31e6edb353af0c]
>     bitbake: hashserv: client: Fix handling of null responses
>
> git bisect bad adcd9608a725d99e2e6ca74d4a31e6edb353af0c
> # bad: [071f23ad79ac37743d97928f92ded0da61ba9e63]
>     bash: Disable bracketed input by default
>
> git bisect bad 071f23ad79ac37743d97928f92ded0da61ba9e63
> # bad: [8e3b42f44217c9f868d6b8bd52808d357b516a2b]
>     glibc: Require full ISA support for x86-64 level marker
>
> git bisect bad 8e3b42f44217c9f868d6b8bd52808d357b516a2b
> # bad: [51cf44884971d6fc919f2a799fb08ee61acf7518]
>     glibc: Enable cet
>
> git bisect bad 51cf44884971d6fc919f2a799fb08ee61acf7518
> # bad: [7b8df042d0c175388d6230f008b1c83d5c5cd5da]
>     glibc: Upgrade to 2.33
>
> git bisect bad 7b8df042d0c175388d6230f008b1c83d5c5cd5da
> # first bad commit: [7b8df042d0c175388d6230f008b1c83d5c5cd5da] glibc: 
> Upgrade to 2.33
>
>
> ---
>>
>> Cheers,
>>
>> Richard
>>
>
>
>
> 
>

[-- Attachment #2: Type: text/html, Size: 10872 bytes --]