[RFC] python3-cryptogaphy upgrade to 40.0.2 issues

[RFC] python3-cryptogaphy upgrade to 40.0.2 issues

[Tim Orling]

[-- Attachment #1: Type: text/plain, Size: 5598 bytes --]

I am working on an upgrade from 39.0.2 to 40.0.2 for python3-cryptography:
https://git.yoctoproject.org/poky-contrib/log/?h=timo/python3-cryptography_40.0.2

The patch at the tip of HEAD is perhaps not needed, as the compiling of
`openssl-sys` was not the problem. We might need tweaks to not vendor
OpenSSL, but I do not think this is the root of the problem right now.

The issue I am struggling with is "Python.h" is not being found:
|   cargo:rerun-if-env-changed=CRATE_CC_NO_DEFAULTS
|   CRATE_CC_NO_DEFAULTS = Some("1")
|   Info: default compiler flags are disabled
|   cargo:rerun-if-env-changed=CC_x86_64-poky-linux-gnu
|   CC_x86_64-poky-linux-gnu = None
|   cargo:rerun-if-env-changed=CC_x86_64_poky_linux_gnu
|   CC_x86_64_poky_linux_gnu = None
|   cargo:rerun-if-env-changed=TARGET_CC
|   TARGET_CC = None
|   cargo:rerun-if-env-changed=CC
|   CC = Some("x86_64-poky-linux-gcc  -m64 -march=core2 -mtune=core2 -msse3
-mfpmath=sse -fstack-protector-strong  -O2 -D_FORTIFY_SOURCE=2 -Wformat
-Wformat-security -Werror=format-security
--sysroot=/srv/build/ttorling/build-upgrades/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/recipe-sysroot")
|   cargo:rerun-if-env-changed=CFLAGS_x86_64-poky-linux-gnu
|   CFLAGS_x86_64-poky-linux-gnu = None
|   cargo:rerun-if-env-changed=CFLAGS_x86_64_poky_linux_gnu
|   CFLAGS_x86_64_poky_linux_gnu = None
|   cargo:rerun-if-env-changed=TARGET_CFLAGS
|   TARGET_CFLAGS = None
|   cargo:rerun-if-env-changed=CFLAGS
|   CFLAGS = Some(" -O2 -pipe -g -feliminate-unused-debug-types
-fmacro-prefix-map=/srv/build/ttorling/build-upgrades/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/cryptography-40.0.2=/usr/src/debug/python3-cryptography/40.0.2-r0
 -fdebug-prefix-map=/srv/build/ttorling/build-upgrades/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/cryptography-40.0.2=/usr/src/debug/python3-cryptography/40.0.2-r0
 -fmacro-prefix-map=/srv/build/ttorling/build-upgrades/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/build=/usr/src/debug/python3-cryptography/40.0.2-r0
 -fdebug-prefix-map=/srv/build/ttorling/build-upgrades/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/build=/usr/src/debug/python3-cryptography/40.0.2-r0
 -fdebug-prefix-map=/srv/build/ttorling/build-upgrades/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/recipe-sysroot=
 -fmacro-prefix-map=/srv/build/ttorling/build-upgrades/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/recipe-sysroot=
 -fdebug-prefix-map=/srv/build/ttorling/build-upgrades/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/recipe-sysroot-native=
")
|   cargo:rerun-if-env-changed=CRATE_CC_NO_DEFAULTS
|   CRATE_CC_NO_DEFAULTS = Some("1")
|   Info: default compiler flags are disabled
|   running: "x86_64-poky-linux-gcc" "-m64" "-march=core2" "-mtune=core2"
"-msse3" "-mfpmath=sse" "-fstack-protector-strong" "-O2"
"-D_FORTIFY_SOURCE=2" "-Wformat" "-Wformat-security"
"-Werror=format-security"
"--sysroot=/srv/build/ttorling/build-upgrades/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/recipe-sysroot"
"-O2" "-pipe" "-g" "-feliminate-unused-debug-types"
"-fmacro-prefix-map=/srv/build/ttorling/build-upgrades/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/cryptography-40.0.2=/usr/src/debug/python3-cryptography/40.0.2-r0"
"-fdebug-prefix-map=/srv/build/ttorling/build-upgrades/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/cryptography-40.0.2=/usr/src/debug/python3-cryptography/40.0.2-r0"
"-fmacro-prefix-map=/srv/build/ttorling/build-upgrades/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/build=/usr/src/debug/python3-cryptography/40.0.2-r0"
"-fdebug-prefix-map=/srv/build/ttorling/build-upgrades/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/build=/usr/src/debug/python3-cryptography/40.0.2-r0"
"-fdebug-prefix-map=/srv/build/ttorling/build-upgrades/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/recipe-sysroot="
"-fmacro-prefix-map=/srv/build/ttorling/build-upgrades/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/recipe-sysroot="
"-fdebug-prefix-map=/srv/build/ttorling/build-upgrades/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/recipe-sysroot-native="
"-I" "/usr/include/python3.11" "-I"
"/srv/build/ttorling/build-upgrades/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/recipe-sysroot/usr/include"
"-Wconversion" "-Wno-error=sign-conversion" "-Wno-unused-parameter"
"-DPy_LIMITED_API=0x030600f0" "-o"
"/srv/build/ttorling/build-upgrades/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/build/target/x86_64-poky-linux-gnu/release/build/cryptography-rust-aa00e39a952c07ee/out/9f543b84f9f4dceb-_openssl.o"
"-c"
"/srv/build/ttorling/build-upgrades/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/build/target/x86_64-poky-linux-gnu/release/build/cryptography-rust-aa00e39a952c07ee/out/_openssl.c"
|
cargo:warning=/srv/build/ttorling/build-upgrades/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/build/target/x86_64-poky-linux-gnu/release/build/cryptography-rust-aa00e39a952c07ee/out/_openssl.c:57:10:
fatal error: Python.h: No such file or directory
|   cargo:warning=   57 | #include <Python.h>
|   cargo:warning=      |          ^~~~~~~~~~
|   cargo:warning=compilation terminated.
|   exit status: 1
|
|   --- stderr

which I believe is happening during:
https://github.com/pyca/cryptography/blob/40.0.x/src/rust/build.rs#L74

I would appreciate input from any crustaceans out there with a better
understanding of what might be happening.

--Tim

[-- Attachment #2: Type: text/html, Size: 6528 bytes --]

Re: [OE-core] [RFC] python3-cryptogaphy upgrade to 40.0.2 issues

[Frédéric Martinsons]

[-- Attachment #1: Type: text/plain, Size: 6586 bytes --]

Hello Tim,

I'll take a look today since I already put my nose in this package for rust
ptest patch series, it was unsuccessful but I want to know more on this
package to kill two birds with one stone (hope so)

Le mer. 3 mai 2023, 19:31, Tim Orling <ticotimo@gmail.com> a écrit :

> I am working on an upgrade from 39.0.2 to 40.0.2 for python3-cryptography:
>
> https://git.yoctoproject.org/poky-contrib/log/?h=timo/python3-cryptography_40.0.2
>
> The patch at the tip of HEAD is perhaps not needed, as the compiling of
> `openssl-sys` was not the problem. We might need tweaks to not vendor
> OpenSSL, but I do not think this is the root of the problem right now.
>
> The issue I am struggling with is "Python.h" is not being found:
> |   cargo:rerun-if-env-changed=CRATE_CC_NO_DEFAULTS
> |   CRATE_CC_NO_DEFAULTS = Some("1")
> |   Info: default compiler flags are disabled
> |   cargo:rerun-if-env-changed=CC_x86_64-poky-linux-gnu
> |   CC_x86_64-poky-linux-gnu = None
> |   cargo:rerun-if-env-changed=CC_x86_64_poky_linux_gnu
> |   CC_x86_64_poky_linux_gnu = None
> |   cargo:rerun-if-env-changed=TARGET_CC
> |   TARGET_CC = None
> |   cargo:rerun-if-env-changed=CC
> |   CC = Some("x86_64-poky-linux-gcc  -m64 -march=core2 -mtune=core2
> -msse3 -mfpmath=sse -fstack-protector-strong  -O2 -D_FORTIFY_SOURCE=2
> -Wformat -Wformat-security -Werror=format-security
> --sysroot=/srv/build/ttorling/build-upgrades/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/recipe-sysroot")
> |   cargo:rerun-if-env-changed=CFLAGS_x86_64-poky-linux-gnu
> |   CFLAGS_x86_64-poky-linux-gnu = None
> |   cargo:rerun-if-env-changed=CFLAGS_x86_64_poky_linux_gnu
> |   CFLAGS_x86_64_poky_linux_gnu = None
> |   cargo:rerun-if-env-changed=TARGET_CFLAGS
> |   TARGET_CFLAGS = None
> |   cargo:rerun-if-env-changed=CFLAGS
> |   CFLAGS = Some(" -O2 -pipe -g -feliminate-unused-debug-types
> -fmacro-prefix-map=/srv/build/ttorling/build-upgrades/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/cryptography-40.0.2=/usr/src/debug/python3-cryptography/40.0.2-r0
>  -fdebug-prefix-map=/srv/build/ttorling/build-upgrades/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/cryptography-40.0.2=/usr/src/debug/python3-cryptography/40.0.2-r0
>  -fmacro-prefix-map=/srv/build/ttorling/build-upgrades/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/build=/usr/src/debug/python3-cryptography/40.0.2-r0
>  -fdebug-prefix-map=/srv/build/ttorling/build-upgrades/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/build=/usr/src/debug/python3-cryptography/40.0.2-r0
>  -fdebug-prefix-map=/srv/build/ttorling/build-upgrades/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/recipe-sysroot=
>  -fmacro-prefix-map=/srv/build/ttorling/build-upgrades/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/recipe-sysroot=
>  -fdebug-prefix-map=/srv/build/ttorling/build-upgrades/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/recipe-sysroot-native=
> ")
> |   cargo:rerun-if-env-changed=CRATE_CC_NO_DEFAULTS
> |   CRATE_CC_NO_DEFAULTS = Some("1")
> |   Info: default compiler flags are disabled
> |   running: "x86_64-poky-linux-gcc" "-m64" "-march=core2" "-mtune=core2"
> "-msse3" "-mfpmath=sse" "-fstack-protector-strong" "-O2"
> "-D_FORTIFY_SOURCE=2" "-Wformat" "-Wformat-security"
> "-Werror=format-security"
> "--sysroot=/srv/build/ttorling/build-upgrades/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/recipe-sysroot"
> "-O2" "-pipe" "-g" "-feliminate-unused-debug-types"
> "-fmacro-prefix-map=/srv/build/ttorling/build-upgrades/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/cryptography-40.0.2=/usr/src/debug/python3-cryptography/40.0.2-r0"
> "-fdebug-prefix-map=/srv/build/ttorling/build-upgrades/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/cryptography-40.0.2=/usr/src/debug/python3-cryptography/40.0.2-r0"
> "-fmacro-prefix-map=/srv/build/ttorling/build-upgrades/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/build=/usr/src/debug/python3-cryptography/40.0.2-r0"
> "-fdebug-prefix-map=/srv/build/ttorling/build-upgrades/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/build=/usr/src/debug/python3-cryptography/40.0.2-r0"
> "-fdebug-prefix-map=/srv/build/ttorling/build-upgrades/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/recipe-sysroot="
> "-fmacro-prefix-map=/srv/build/ttorling/build-upgrades/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/recipe-sysroot="
> "-fdebug-prefix-map=/srv/build/ttorling/build-upgrades/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/recipe-sysroot-native="
> "-I" "/usr/include/python3.11" "-I"
> "/srv/build/ttorling/build-upgrades/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/recipe-sysroot/usr/include"
> "-Wconversion" "-Wno-error=sign-conversion" "-Wno-unused-parameter"
> "-DPy_LIMITED_API=0x030600f0" "-o"
> "/srv/build/ttorling/build-upgrades/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/build/target/x86_64-poky-linux-gnu/release/build/cryptography-rust-aa00e39a952c07ee/out/9f543b84f9f4dceb-_openssl.o"
> "-c"
> "/srv/build/ttorling/build-upgrades/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/build/target/x86_64-poky-linux-gnu/release/build/cryptography-rust-aa00e39a952c07ee/out/_openssl.c"
> |
> cargo:warning=/srv/build/ttorling/build-upgrades/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/build/target/x86_64-poky-linux-gnu/release/build/cryptography-rust-aa00e39a952c07ee/out/_openssl.c:57:10:
> fatal error: Python.h: No such file or directory
> |   cargo:warning=   57 | #include <Python.h>
> |   cargo:warning=      |          ^~~~~~~~~~
> |   cargo:warning=compilation terminated.
> |   exit status: 1
> |
> |   --- stderr
>
> which I believe is happening during:
> https://github.com/pyca/cryptography/blob/40.0.x/src/rust/build.rs#L74
>
> I would appreciate input from any crustaceans out there with a better
> understanding of what might be happening.
>
> --Tim
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#180834):
> https://lists.openembedded.org/g/openembedded-core/message/180834
> Mute This Topic: https://lists.openembedded.org/mt/98667212/6213388
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [
> frederic.martinsons@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
>

[-- Attachment #2: Type: text/html, Size: 8197 bytes --]

Re: [OE-core] [RFC] python3-cryptogaphy upgrade to 40.0.2 issues

[Peter Bergin]

On 2023-05-03 19:30, Tim Orling wrote:
> I am working on an upgrade from 39.0.2 to 40.0.2 for 
> python3-cryptography:
> https://git.yoctoproject.org/poky-contrib/log/?h=timo/python3-cryptography_40.0.2
>
> The patch at the tip of HEAD is perhaps not needed, as the compiling 
> of `openssl-sys` was not the problem. We might need tweaks to not 
> vendor OpenSSL, but I do not think this is the root of the problem 
> right now.
>
> The issue I am struggling with is "Python.h" is not being found:
> |   cargo:rerun-if-env-changed=CRATE_CC_NO_DEFAULTS
> |   CRATE_CC_NO_DEFAULTS = Some("1")
> |   Info: default compiler flags are disabled
> |   cargo:rerun-if-env-changed=CC_x86_64-poky-linux-gnu
> |   CC_x86_64-poky-linux-gnu = None
> |   cargo:rerun-if-env-changed=CC_x86_64_poky_linux_gnu
> |   CC_x86_64_poky_linux_gnu = None
> |   cargo:rerun-if-env-changed=TARGET_CC
> |   TARGET_CC = None
> |   cargo:rerun-if-env-changed=CC
> |   CC = Some("x86_64-poky-linux-gcc  -m64 -march=core2 -mtune=core2 
> -msse3 -mfpmath=sse -fstack-protector-strong  -O2 -D_FORTIFY_SOURCE=2 
> -Wformat -Wformat-security -Werror=format-security 
> --sysroot=/srv/build/ttorling/build-upgrades/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/recipe-sysroot")
> |   cargo:rerun-if-env-changed=CFLAGS_x86_64-poky-linux-gnu
> |   CFLAGS_x86_64-poky-linux-gnu = None
> |   cargo:rerun-if-env-changed=CFLAGS_x86_64_poky_linux_gnu
> |   CFLAGS_x86_64_poky_linux_gnu = None
> |   cargo:rerun-if-env-changed=TARGET_CFLAGS
> |   TARGET_CFLAGS = None
> |   cargo:rerun-if-env-changed=CFLAGS
> |   CFLAGS = Some(" -O2 -pipe -g -feliminate-unused-debug-types 
> -fmacro-prefix-map=/srv/build/ttorling/build-upgrades/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/cryptography-40.0.2=/usr/src/debug/python3-cryptography/40.0.2-r0 
>  -fdebug-prefix-map=/srv/build/ttorling/build-upgrades/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/cryptography-40.0.2=/usr/src/debug/python3-cryptography/40.0.2-r0 
>  -fmacro-prefix-map=/srv/build/ttorling/build-upgrades/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/build=/usr/src/debug/python3-cryptography/40.0.2-r0 
>  -fdebug-prefix-map=/srv/build/ttorling/build-upgrades/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/build=/usr/src/debug/python3-cryptography/40.0.2-r0 
>  -fdebug-prefix-map=/srv/build/ttorling/build-upgrades/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/recipe-sysroot= 
>  -fmacro-prefix-map=/srv/build/ttorling/build-upgrades/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/recipe-sysroot= 
>  -fdebug-prefix-map=/srv/build/ttorling/build-upgrades/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/recipe-sysroot-native= 
> ")
> |   cargo:rerun-if-env-changed=CRATE_CC_NO_DEFAULTS
> |   CRATE_CC_NO_DEFAULTS = Some("1")
> |   Info: default compiler flags are disabled
> |   running: "x86_64-poky-linux-gcc" "-m64" "-march=core2" 
> "-mtune=core2" "-msse3" "-mfpmath=sse" "-fstack-protector-strong" 
> "-O2" "-D_FORTIFY_SOURCE=2" "-Wformat" "-Wformat-security" 
> "-Werror=format-security" 
> "--sysroot=/srv/build/ttorling/build-upgrades/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/recipe-sysroot" 
> "-O2" "-pipe" "-g" "-feliminate-unused-debug-types" 
> "-fmacro-prefix-map=/srv/build/ttorling/build-upgrades/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/cryptography-40.0.2=/usr/src/debug/python3-cryptography/40.0.2-r0" 
> "-fdebug-prefix-map=/srv/build/ttorling/build-upgrades/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/cryptography-40.0.2=/usr/src/debug/python3-cryptography/40.0.2-r0" 
> "-fmacro-prefix-map=/srv/build/ttorling/build-upgrades/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/build=/usr/src/debug/python3-cryptography/40.0.2-r0" 
> "-fdebug-prefix-map=/srv/build/ttorling/build-upgrades/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/build=/usr/src/debug/python3-cryptography/40.0.2-r0" 
> "-fdebug-prefix-map=/srv/build/ttorling/build-upgrades/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/recipe-sysroot=" 
> "-fmacro-prefix-map=/srv/build/ttorling/build-upgrades/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/recipe-sysroot=" 
> "-fdebug-prefix-map=/srv/build/ttorling/build-upgrades/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/recipe-sysroot-native=" 
> "-I" "/usr/include/python3.11" "-I" 
> "/srv/build/ttorling/build-upgrades/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/recipe-sysroot/usr/include" 
> "-Wconversion" "-Wno-error=sign-conversion" "-Wno-unused-parameter" 
> "-DPy_LIMITED_API=0x030600f0" "-o" 
> "/srv/build/ttorling/build-upgrades/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/build/target/x86_64-poky-linux-gnu/release/build/cryptography-rust-aa00e39a952c07ee/out/9f543b84f9f4dceb-_openssl.o" 
> "-c" 
> "/srv/build/ttorling/build-upgrades/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/build/target/x86_64-poky-linux-gnu/release/build/cryptography-rust-aa00e39a952c07ee/out/_openssl.c"

So the issue is most probably '-I /usr/include/python3.11' among the 
arguments to the compiler.

> | 
> cargo:warning=/srv/build/ttorling/build-upgrades/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/build/target/x86_64-poky-linux-gnu/release/build/cryptography-rust-aa00e39a952c07ee/out/_openssl.c:57:10: 
> fatal error: Python.h: No such file or directory
> |   cargo:warning=   57 | #include <Python.h>
> |   cargo:warning=      |          ^~~~~~~~~~
> |   cargo:warning=compilation terminated.
> |   exit status: 1
> |
> |   --- stderr
>
> which I believe is happening during:
> https://github.com/pyca/cryptography/blob/40.0.x/src/rust/build.rs#L74
>
At 
https://github.com/pyca/cryptography/blob/f816b457494e010b655cd7fdcd30e3446f86a703/src/rust/build.rs#L46 
the path to python includes is defined by calling 'python3 -c "import 
sysconfig; print(sysconfig.get_path('include'))"' (but from inside 
rust). With print-debugging this seems to return the bad string 
'/usr/include/python3.11' which is then passed to cc::Build

I executed this in recipe context:

do_compile:prepend() {
     which python3
     python3 -c "import sysconfig; print(sysconfig.get_path('include'))"
}

and then I got the correct return value:

| 
/work/yocto/poky/build/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/recipe-sysroot-native/usr/bin/python3-native/python3
| 
/work/yocto/poky/build/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/recipe-sysroot-native/usr/include/python3.11

On my machine I have python3.10 so the include path is fetched from 
correct python inside the build. The question is how, why and where the 
path is stripped?

/Peter


> I would appreciate input from any crustaceans out there with a better 
> understanding of what might be happening.
>
> --Tim
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#180834): https://lists.openembedded.org/g/openembedded-core/message/180834
> Mute This Topic: https://lists.openembedded.org/mt/98667212/3617552
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [peter@berginkonsult.se]
> -=-=-=-=-=-=-=-=-=-=-=-
>

Re: [OE-core] [RFC] python3-cryptogaphy upgrade to 40.0.2 issues

[Alexander Kanavin]

On Thu, 4 May 2023 at 10:41, Peter Bergin <peter@berginkonsult.se> wrote:

> At
> https://github.com/pyca/cryptography/blob/f816b457494e010b655cd7fdcd30e3446f86a703/src/rust/build.rs#L46
> the path to python includes is defined by calling 'python3 -c "import
> sysconfig; print(sysconfig.get_path('include'))"' (but from inside
> rust). With print-debugging this seems to return the bad string
> '/usr/include/python3.11' which is then passed to cc::Build
>
> I executed this in recipe context:
>
> do_compile:prepend() {
>      which python3
>      python3 -c "import sysconfig; print(sysconfig.get_path('include'))"
> }
>
> and then I got the correct return value:
>
> |
> /work/yocto/poky/build/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/recipe-sysroot-native/usr/bin/python3-native/python3
> |
> /work/yocto/poky/build/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/recipe-sysroot-native/usr/include/python3.11

This is pointing to the native sysroot in a target build, so it is
most likely not actually correct. I believe 'inherit
python3targetconfig' may help, as it substitutes most places where
native python gets queried by components with target-specific values.

Alex

Re: [OE-core] [RFC] python3-cryptogaphy upgrade to 40.0.2 issues

[Frédéric Martinsons]

[-- Attachment #1: Type: text/plain, Size: 1844 bytes --]

On Thu, 4 May 2023 at 11:00, Alexander Kanavin <alex.kanavin@gmail.com>
wrote:

> On Thu, 4 May 2023 at 10:41, Peter Bergin <peter@berginkonsult.se> wrote:
>
> > At
> >
> https://github.com/pyca/cryptography/blob/f816b457494e010b655cd7fdcd30e3446f86a703/src/rust/build.rs#L46
> > the path to python includes is defined by calling 'python3 -c "import
> > sysconfig; print(sysconfig.get_path('include'))"' (but from inside
> > rust). With print-debugging this seems to return the bad string
> > '/usr/include/python3.11' which is then passed to cc::Build
> >
> > I executed this in recipe context:
> >
> > do_compile:prepend() {
> >      which python3
> >      python3 -c "import sysconfig; print(sysconfig.get_path('include'))"
> > }
> >
> > and then I got the correct return value:
> >
> > |
> >
> /work/yocto/poky/build/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/recipe-sysroot-native/usr/bin/python3-native/python3
> > |
> >
> /work/yocto/poky/build/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/recipe-sysroot-native/usr/include/python3.11
>
> This is pointing to the native sysroot in a target build, so it is
> most likely not actually correct. I believe 'inherit
> python3targetconfig' may help, as it substitutes most places where
> native python gets queried by components with target-specific values.
>
> Alex
>

Unless I'm mistaken , in the error message we can see that the compiler
specified the sysroot correctly (recipe-sysroot and recipe-sysroot-native)
and so the path /usr/include/python3.11 is correct under this sysroot
(Python.h lies there).

I would like to have a verbose output which show the compiler command args
for python3-cryptography 39.0.2 to compare
the compiler arguments but I didn't manage to do it for now (tried to put a
config.toml in src/rust/.cargo with term.verbose=true).

[-- Attachment #2: Type: text/html, Size: 2653 bytes --]

Re: [OE-core] [RFC] python3-cryptogaphy upgrade to 40.0.2 issues

[Frédéric Martinsons]

[-- Attachment #1: Type: text/plain, Size: 2103 bytes --]

Le jeu. 4 mai 2023, 11:31, Frédéric Martinsons <
frederic.martinsons@gmail.com> a écrit :

>
>
> On Thu, 4 May 2023 at 11:00, Alexander Kanavin <alex.kanavin@gmail.com>
> wrote:
>
>> On Thu, 4 May 2023 at 10:41, Peter Bergin <peter@berginkonsult.se> wrote:
>>
>> > At
>> >
>> https://github.com/pyca/cryptography/blob/f816b457494e010b655cd7fdcd30e3446f86a703/src/rust/build.rs#L46
>> > the path to python includes is defined by calling 'python3 -c "import
>> > sysconfig; print(sysconfig.get_path('include'))"' (but from inside
>> > rust). With print-debugging this seems to return the bad string
>> > '/usr/include/python3.11' which is then passed to cc::Build
>> >
>> > I executed this in recipe context:
>> >
>> > do_compile:prepend() {
>> >      which python3
>> >      python3 -c "import sysconfig; print(sysconfig.get_path('include'))"
>> > }
>> >
>> > and then I got the correct return value:
>> >
>> > |
>> >
>> /work/yocto/poky/build/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/recipe-sysroot-native/usr/bin/python3-native/python3
>> > |
>> >
>> /work/yocto/poky/build/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/recipe-sysroot-native/usr/include/python3.11
>>
>> This is pointing to the native sysroot in a target build, so it is
>> most likely not actually correct. I believe 'inherit
>> python3targetconfig' may help, as it substitutes most places where
>> native python gets queried by components with target-specific values.
>>
>> Alex
>>
>
> Unless I'm mistaken , in the error message we can see that the compiler
> specified the sysroot correctly (recipe-sysroot and recipe-sysroot-native)
>

recipe-sysroot and NOT recipe-sysroot-native

and so the path /usr/include/python3.11 is correct under this sysroot
> (Python.h lies there).
>
> I would like to have a verbose output which show the compiler command args
> for python3-cryptography 39.0.2 to compare
> the compiler arguments but I didn't manage to do it for now (tried to put
> a config.toml in src/rust/.cargo with term.verbose=true).
>

[-- Attachment #2: Type: text/html, Size: 3436 bytes --]

Re: [OE-core] [RFC] python3-cryptogaphy upgrade to 40.0.2 issues

[Peter Bergin]

[-- Attachment #1: Type: text/plain, Size: 2517 bytes --]


On 2023-05-04 12:03, Frederic Martinsons wrote:
>
>
> Le jeu. 4 mai 2023, 11:31, Frédéric Martinsons 
> <frederic.martinsons@gmail.com> a écrit :
>
>
>
>     On Thu, 4 May 2023 at 11:00, Alexander Kanavin
>     <alex.kanavin@gmail.com> wrote:
>
>         On Thu, 4 May 2023 at 10:41, Peter Bergin
>         <peter@berginkonsult.se> wrote:
>
>         > At
>         >
>         https://github.com/pyca/cryptography/blob/f816b457494e010b655cd7fdcd30e3446f86a703/src/rust/build.rs#L46
>         > the path to python includes is defined by calling 'python3
>         -c "import
>         > sysconfig; print(sysconfig.get_path('include'))"' (but from
>         inside
>         > rust). With print-debugging this seems to return the bad string
>         > '/usr/include/python3.11' which is then passed to cc::Build
>         >
>         > I executed this in recipe context:
>         >
>         > do_compile:prepend() {
>         >      which python3
>         >      python3 -c "import sysconfig;
>         print(sysconfig.get_path('include'))"
>         > }
>         >
>         > and then I got the correct return value:
>         >
>         > |
>         >
>         /work/yocto/poky/build/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/recipe-sysroot-native/usr/bin/python3-native/python3
>         > |
>         >
>         /work/yocto/poky/build/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/recipe-sysroot-native/usr/include/python3.11
>
>         This is pointing to the native sysroot in a target build, so it is
>         most likely not actually correct. I believe 'inherit
>         python3targetconfig' may help, as it substitutes most places where
>         native python gets queried by components with target-specific
>         values.
>
>         Alex
>
>
>     Unless I'm mistaken , in the error message we can see that the
>     compiler specified the sysroot correctly (recipe-sysroot and
>     recipe-sysroot-native)
>
>
> recipe-sysroot and NOT recipe-sysroot-native

'--sysroot' is specified correct but that is not automatically add 
search paths to '-I' directive, if I'm not misreading this: 
https://gcc.gnu.org/onlinedocs/gcc/Directory-Options.html . When you use 
--sysroot with gcc you have to add = or $SYSROOT to your -I directive in 
order to add it to search path for headers. With that information I 
guess '-I =/usr/lib/python3.11' or '-I \$SYSROOT/usr/lib/python3.11' can 
work but '-I /usr/lib/python3.11' will point to your host.

/Peter


[-- Attachment #2: Type: text/html, Size: 5645 bytes --]

Re: [OE-core] [RFC] python3-cryptogaphy upgrade to 40.0.2 issues

[Peter Bergin]

On 2023-05-04 11:00, Alexander Kanavin wrote:
> On Thu, 4 May 2023 at 10:41, Peter Bergin <peter@berginkonsult.se> wrote:
>
>> At
>> https://github.com/pyca/cryptography/blob/f816b457494e010b655cd7fdcd30e3446f86a703/src/rust/build.rs#L46
>> the path to python includes is defined by calling 'python3 -c "import
>> sysconfig; print(sysconfig.get_path('include'))"' (but from inside
>> rust). With print-debugging this seems to return the bad string
>> '/usr/include/python3.11' which is then passed to cc::Build
>>
>> I executed this in recipe context:
>>
>> do_compile:prepend() {
>>       which python3
>>       python3 -c "import sysconfig; print(sysconfig.get_path('include'))"
>> }
>>
>> and then I got the correct return value:
>>
>> |
>> /work/yocto/poky/build/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/recipe-sysroot-native/usr/bin/python3-native/python3
>> |
>> /work/yocto/poky/build/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/recipe-sysroot-native/usr/include/python3.11
> This is pointing to the native sysroot in a target build, so it is
> most likely not actually correct. I believe 'inherit
> python3targetconfig' may help, as it substitutes most places where
> native python gets queried by components with target-specific values.
>
> Alex
>
The bad thing is that the build.rs script is running python script to 
figure out the path to the headers. I'm not sure we can use target 
python exe in the build steps without running qemu instance or something 
which seems overkill in this case. We have the information about python 
include path for the target in the variable PYTHON_INCLUDE_DIR.

With this patch the compilation will find Python.h:

$ git diff src/rust/build.rs
diff --git a/src/rust/build.rs b/src/rust/build.rs
index 01177ac..7dc13fe 100644
--- a/src/rust/build.rs
+++ b/src/rust/build.rs
@@ -43,11 +43,8 @@ fn main() {
      )
      .unwrap();
      println!("cargo:rustc-cfg=python_implementation=\"{}\"", python_impl);
-    let python_include = run_python_script(
-        &python,
-        "import sysconfig; print(sysconfig.get_path('include'), end='')",
-    )
-    .unwrap();
+    let python_include = env::var("PYTHON_INCLUDE_DIR").unwrap();
+
      let openssl_include =
          std::env::var_os("DEP_OPENSSL_INCLUDE").expect("unable to find 
openssl include path");
      let openssl_c = Path::new(&out_dir).join("_openssl.c");


But it fails later in the linking step.

.....linux/python3-cryptography/40.0.2-r0/recipe-sysroot/usr/lib/rustlib/x86_64-poky-linux-gnu/lib" 
"-L" 
"/work/yocto/poky/build/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/python3-cryptography-40.0.2/target/x86_64-poky-linux-gnu/release/build/cryptography-rust-aa00e39a952c07ee/out" 
"-L" 
"/work/yocto/poky/build/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/recipe-sysroot/usr/lib" 
"-L" 
"/work/yocto/poky/build/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/recipe-sysroot-native/usr/lib/rustlib/x86_64-poky-linux-gnu/lib" 
"-Wl,-Bstatic" "-l_openssl.a" 
"/work/yocto/poky/build/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/recipe-sysroot/usr/lib/rustlib/x86_64-poky-linux-gnu/lib/libcompiler_builtins-45df5d471f0d20d0.rlib" 
"-Wl,-Bdynamic" "-lssl" "-lcrypto" "-lgcc_s" "-lutil" "-lrt" "-lpthread" 
"-lm" "-ldl" "-lc" "-Wl,--eh-frame-hdr" "-Wl,-znoexecstack" "-L" 
"/work/yocto/poky/build/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/recipe-sysroot-native/usr/lib/rustlib/x86_64-poky-linux-gnu/lib" 
"-o" 
"/work/yocto/poky/build/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/python3-cryptography-40.0.2/target/x86_64-poky-linux-gnu/release/deps/libcryptography_rust.so" 
"-Wl,--gc-sections" "-shared" "-Wl,-O1" "-nodefaultlibs"
   = note: 
/work/yocto/poky/build/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/recipe-sysroot-native/usr/bin/x86_64-poky-linux/../../libexec/x86_64-poky-linux/gcc/x86_64-poky-linux/12.2.0/ld: 
/work/yocto/poky/build/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/python3-cryptography-40.0.2/target/x86_64-poky-linux-gnu/release/build/cryptography-rust-aa00e39a952c07ee/out/lib_openssl.a.a(9bdf47a443b7a07f-_openssl.o): 
warning: relocation against `_Py_NoneStruct' in read-only section `.text'
/work/yocto/poky/build/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/recipe-sysroot-native/usr/bin/x86_64-poky-linux/../../libexec/x86_64-poky-linux/gcc/x86_64-poky-linux/12.2.0/ld: 
/work/yocto/poky/build/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/python3-cryptography-40.0.2/target/x86_64-poky-linux-gnu/release/build/cryptography-rust-aa00e39a952c07ee/out/lib_openssl.a.a(9bdf47a443b7a07f-_openssl.o): 
relocation R_X86_64_PC32 against undefined symbol `_Py_NoneStruct' can 
not be used when making a shared object; recompile with -fPIC
/work/yocto/poky/build/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/recipe-sysroot-native/usr/bin/x86_64-poky-linux/../../libexec/x86_64-poky-linux/gcc/x86_64-poky-linux/12.2.0/ld: 
final link failed: bad value
           collect2: error: ld returned 1 exit status


There are some '-L .../recipe-sysroot-native/usr/lib/rustlib/...' that 
can probably causing this. Don't know if it helps us forward?

/Peter

Re: [OE-core] [RFC] python3-cryptogaphy upgrade to 40.0.2 issues

[Alexander Kanavin]

I think you should look at prior art - other recipes where this works
correctly. For example rpm, a fairly simple component, is using
python.h and linking with libpython. How is it working there?

Alex

On Thu, 4 May 2023 at 14:00, Peter Bergin <peter@berginkonsult.se> wrote:
>
>
> On 2023-05-04 11:00, Alexander Kanavin wrote:
> > On Thu, 4 May 2023 at 10:41, Peter Bergin <peter@berginkonsult.se> wrote:
> >
> >> At
> >> https://github.com/pyca/cryptography/blob/f816b457494e010b655cd7fdcd30e3446f86a703/src/rust/build.rs#L46
> >> the path to python includes is defined by calling 'python3 -c "import
> >> sysconfig; print(sysconfig.get_path('include'))"' (but from inside
> >> rust). With print-debugging this seems to return the bad string
> >> '/usr/include/python3.11' which is then passed to cc::Build
> >>
> >> I executed this in recipe context:
> >>
> >> do_compile:prepend() {
> >>       which python3
> >>       python3 -c "import sysconfig; print(sysconfig.get_path('include'))"
> >> }
> >>
> >> and then I got the correct return value:
> >>
> >> |
> >> /work/yocto/poky/build/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/recipe-sysroot-native/usr/bin/python3-native/python3
> >> |
> >> /work/yocto/poky/build/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/recipe-sysroot-native/usr/include/python3.11
> > This is pointing to the native sysroot in a target build, so it is
> > most likely not actually correct. I believe 'inherit
> > python3targetconfig' may help, as it substitutes most places where
> > native python gets queried by components with target-specific values.
> >
> > Alex
> >
> The bad thing is that the build.rs script is running python script to
> figure out the path to the headers. I'm not sure we can use target
> python exe in the build steps without running qemu instance or something
> which seems overkill in this case. We have the information about python
> include path for the target in the variable PYTHON_INCLUDE_DIR.
>
> With this patch the compilation will find Python.h:
>
> $ git diff src/rust/build.rs
> diff --git a/src/rust/build.rs b/src/rust/build.rs
> index 01177ac..7dc13fe 100644
> --- a/src/rust/build.rs
> +++ b/src/rust/build.rs
> @@ -43,11 +43,8 @@ fn main() {
>       )
>       .unwrap();
>       println!("cargo:rustc-cfg=python_implementation=\"{}\"", python_impl);
> -    let python_include = run_python_script(
> -        &python,
> -        "import sysconfig; print(sysconfig.get_path('include'), end='')",
> -    )
> -    .unwrap();
> +    let python_include = env::var("PYTHON_INCLUDE_DIR").unwrap();
> +
>       let openssl_include =
>           std::env::var_os("DEP_OPENSSL_INCLUDE").expect("unable to find
> openssl include path");
>       let openssl_c = Path::new(&out_dir).join("_openssl.c");
>
>
> But it fails later in the linking step.
>
> .....linux/python3-cryptography/40.0.2-r0/recipe-sysroot/usr/lib/rustlib/x86_64-poky-linux-gnu/lib"
> "-L"
> "/work/yocto/poky/build/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/python3-cryptography-40.0.2/target/x86_64-poky-linux-gnu/release/build/cryptography-rust-aa00e39a952c07ee/out"
> "-L"
> "/work/yocto/poky/build/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/recipe-sysroot/usr/lib"
> "-L"
> "/work/yocto/poky/build/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/recipe-sysroot-native/usr/lib/rustlib/x86_64-poky-linux-gnu/lib"
> "-Wl,-Bstatic" "-l_openssl.a"
> "/work/yocto/poky/build/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/recipe-sysroot/usr/lib/rustlib/x86_64-poky-linux-gnu/lib/libcompiler_builtins-45df5d471f0d20d0.rlib"
> "-Wl,-Bdynamic" "-lssl" "-lcrypto" "-lgcc_s" "-lutil" "-lrt" "-lpthread"
> "-lm" "-ldl" "-lc" "-Wl,--eh-frame-hdr" "-Wl,-znoexecstack" "-L"
> "/work/yocto/poky/build/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/recipe-sysroot-native/usr/lib/rustlib/x86_64-poky-linux-gnu/lib"
> "-o"
> "/work/yocto/poky/build/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/python3-cryptography-40.0.2/target/x86_64-poky-linux-gnu/release/deps/libcryptography_rust.so"
> "-Wl,--gc-sections" "-shared" "-Wl,-O1" "-nodefaultlibs"
>    = note:
> /work/yocto/poky/build/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/recipe-sysroot-native/usr/bin/x86_64-poky-linux/../../libexec/x86_64-poky-linux/gcc/x86_64-poky-linux/12.2.0/ld:
> /work/yocto/poky/build/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/python3-cryptography-40.0.2/target/x86_64-poky-linux-gnu/release/build/cryptography-rust-aa00e39a952c07ee/out/lib_openssl.a.a(9bdf47a443b7a07f-_openssl.o):
> warning: relocation against `_Py_NoneStruct' in read-only section `.text'
> /work/yocto/poky/build/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/recipe-sysroot-native/usr/bin/x86_64-poky-linux/../../libexec/x86_64-poky-linux/gcc/x86_64-poky-linux/12.2.0/ld:
> /work/yocto/poky/build/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/python3-cryptography-40.0.2/target/x86_64-poky-linux-gnu/release/build/cryptography-rust-aa00e39a952c07ee/out/lib_openssl.a.a(9bdf47a443b7a07f-_openssl.o):
> relocation R_X86_64_PC32 against undefined symbol `_Py_NoneStruct' can
> not be used when making a shared object; recompile with -fPIC
> /work/yocto/poky/build/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/recipe-sysroot-native/usr/bin/x86_64-poky-linux/../../libexec/x86_64-poky-linux/gcc/x86_64-poky-linux/12.2.0/ld:
> final link failed: bad value
>            collect2: error: ld returned 1 exit status
>
>
> There are some '-L .../recipe-sysroot-native/usr/lib/rustlib/...' that
> can probably causing this. Don't know if it helps us forward?
>
> /Peter
>

Re: [OE-core] [RFC] python3-cryptogaphy upgrade to 40.0.2 issues

[Frédéric Martinsons]

[-- Attachment #1: Type: text/plain, Size: 2892 bytes --]

Le jeu. 4 mai 2023, 13:42, Peter Bergin <peter@berginkonsult.se> a écrit :

>
> On 2023-05-04 12:03, Frederic Martinsons wrote:
>
>
>
> Le jeu. 4 mai 2023, 11:31, Frédéric Martinsons <
> frederic.martinsons@gmail.com> a écrit :
>
>>
>>
>> On Thu, 4 May 2023 at 11:00, Alexander Kanavin <alex.kanavin@gmail.com>
>> wrote:
>>
>>> On Thu, 4 May 2023 at 10:41, Peter Bergin <peter@berginkonsult.se>
>>> wrote:
>>>
>>> > At
>>> >
>>> https://github.com/pyca/cryptography/blob/f816b457494e010b655cd7fdcd30e3446f86a703/src/rust/build.rs#L46
>>> > the path to python includes is defined by calling 'python3 -c "import
>>> > sysconfig; print(sysconfig.get_path('include'))"' (but from inside
>>> > rust). With print-debugging this seems to return the bad string
>>> > '/usr/include/python3.11' which is then passed to cc::Build
>>> >
>>> > I executed this in recipe context:
>>> >
>>> > do_compile:prepend() {
>>> >      which python3
>>> >      python3 -c "import sysconfig;
>>> print(sysconfig.get_path('include'))"
>>> > }
>>> >
>>> > and then I got the correct return value:
>>> >
>>> > |
>>> >
>>> /work/yocto/poky/build/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/recipe-sysroot-native/usr/bin/python3-native/python3
>>> > |
>>> >
>>> /work/yocto/poky/build/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/recipe-sysroot-native/usr/include/python3.11
>>>
>>> This is pointing to the native sysroot in a target build, so it is
>>> most likely not actually correct. I believe 'inherit
>>> python3targetconfig' may help, as it substitutes most places where
>>> native python gets queried by components with target-specific values.
>>>
>>> Alex
>>>
>>
>> Unless I'm mistaken , in the error message we can see that the compiler
>> specified the sysroot correctly (recipe-sysroot and recipe-sysroot-native)
>>
>
> recipe-sysroot and NOT recipe-sysroot-native
>
> '--sysroot' is specified correct but that is not automatically add search
> paths to '-I' directive, if I'm not misreading this:
> https://gcc.gnu.org/onlinedocs/gcc/Directory-Options.html . When you use
> --sysroot with gcc you have to add = or $SYSROOT to your -I directive in
> order to add it to search path for headers. With that information I guess
> '-I =/usr/lib/python3.11' or '-I \$SYSROOT/usr/lib/python3.11' can work but
> '-I /usr/lib/python3.11' will point to your host.
>
> /Peter
>
I'm confused by the gcc docs, from the same links you gave:

--sysroot=dir
       Use dir as the logical root directory for headers and libraries. For
example, if the compiler normally searches for headers in /usr/include and
libraries in /usr/lib, it instead searches dir/usr/include and dir/usr/lib.

So I don't know what to think (I never see -I used with '=' or $SYSROOT).
I'll try to know more about that for my curiosity

>

[-- Attachment #2: Type: text/html, Size: 6237 bytes --]

Re: [OE-core] [RFC] python3-cryptogaphy upgrade to 40.0.2 issues

[Peter Bergin]

[-- Attachment #1: Type: text/plain, Size: 4225 bytes --]


On 2023-05-04 19:13, Frederic Martinsons wrote:
>
>
> Le jeu. 4 mai 2023, 13:42, Peter Bergin <peter@berginkonsult.se> a écrit :
>
>
>     On 2023-05-04 12:03, Frederic Martinsons wrote:
>>
>>
>>     Le jeu. 4 mai 2023, 11:31, Frédéric Martinsons
>>     <frederic.martinsons@gmail.com> a écrit :
>>
>>
>>
>>         On Thu, 4 May 2023 at 11:00, Alexander Kanavin
>>         <alex.kanavin@gmail.com> wrote:
>>
>>             On Thu, 4 May 2023 at 10:41, Peter Bergin
>>             <peter@berginkonsult.se> wrote:
>>
>>             > At
>>             >
>>             https://github.com/pyca/cryptography/blob/f816b457494e010b655cd7fdcd30e3446f86a703/src/rust/build.rs#L46
>>             > the path to python includes is defined by calling
>>             'python3 -c "import
>>             > sysconfig; print(sysconfig.get_path('include'))"' (but
>>             from inside
>>             > rust). With print-debugging this seems to return the
>>             bad string
>>             > '/usr/include/python3.11' which is then passed to cc::Build
>>             >
>>             > I executed this in recipe context:
>>             >
>>             > do_compile:prepend() {
>>             >      which python3
>>             >      python3 -c "import sysconfig;
>>             print(sysconfig.get_path('include'))"
>>             > }
>>             >
>>             > and then I got the correct return value:
>>             >
>>             > |
>>             >
>>             /work/yocto/poky/build/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/recipe-sysroot-native/usr/bin/python3-native/python3
>>             > |
>>             >
>>             /work/yocto/poky/build/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/recipe-sysroot-native/usr/include/python3.11
>>
>>             This is pointing to the native sysroot in a target build,
>>             so it is
>>             most likely not actually correct. I believe 'inherit
>>             python3targetconfig' may help, as it substitutes most
>>             places where
>>             native python gets queried by components with
>>             target-specific values.
>>
>>             Alex
>>
>>
>>         Unless I'm mistaken , in the error message we can see that
>>         the compiler specified the sysroot correctly (recipe-sysroot
>>         and recipe-sysroot-native)
>>
>>
>>     recipe-sysroot and NOT recipe-sysroot-native
>
>     '--sysroot' is specified correct but that is not automatically add
>     search paths to '-I' directive, if I'm not misreading this:
>     https://gcc.gnu.org/onlinedocs/gcc/Directory-Options.html . When
>     you use --sysroot with gcc you have to add = or $SYSROOT to your
>     -I directive in order to add it to search path for headers. With
>     that information I guess '-I =/usr/lib/python3.11' or '-I
>     \$SYSROOT/usr/lib/python3.11' can work but '-I
>     /usr/lib/python3.11' will point to your host.
>
>     /Peter
>
> I'm confused by the gcc docs, from the same links you gave:
>
> --sysroot=dir
>        Use dir as the logical root directory for headers and 
> libraries. For example, if the compiler normally searches for headers 
> in /usr/include and libraries in /usr/lib, it instead searches 
> dir/usr/include and dir/usr/lib.
>
> So I don't know what to think (I never see -I used with '=' or 
> $SYSROOT). I'll try to know more about that for my curiosity

An example to illustrate:

   $ cat main.c
   #include "my-include.h"

   int main(void) {
       return 0;
   }
   $ tree dir1
   dir1
   └── dir2
       └── my-include.h

   1 directory, 1 file
   $ gcc main.c
   main.c:1:10: fatal error: my-include.h: No such file or directory
       1 | #include "my-include.h"
         |          ^~~~~~~~~~~~~~
   compilation terminated.
   $ gcc -I dir1/dir2 main.c
   $ gcc --sysroot=$PWD/dir1 -I dir2 main.c
   main.c:1:10: fatal error: my-include.h: No such file or directory
       1 | #include "my-include.h"
         |          ^~~~~~~~~~~~~~
   compilation terminated.
   $ gcc --sysroot=$PWD/dir1 -I =/dir2 main.c
   $ gcc --sysroot=$PWD/dir1 -I \$SYSROOT/dir2 main.c

/Peter


[-- Attachment #2: Type: text/html, Size: 10502 bytes --]

Re: [OE-core] [RFC] python3-cryptogaphy upgrade to 40.0.2 issues

[Frédéric Martinsons]

[-- Attachment #1: Type: text/plain, Size: 3973 bytes --]

Le jeu. 4 mai 2023, 21:44, Peter Bergin <peter@berginkonsult.se> a écrit :

>
> On 2023-05-04 19:13, Frederic Martinsons wrote:
>
>
>
> Le jeu. 4 mai 2023, 13:42, Peter Bergin <peter@berginkonsult.se> a écrit :
>
>>
>> On 2023-05-04 12:03, Frederic Martinsons wrote:
>>
>>
>>
>> Le jeu. 4 mai 2023, 11:31, Frédéric Martinsons <
>> frederic.martinsons@gmail.com> a écrit :
>>
>>>
>>>
>>> On Thu, 4 May 2023 at 11:00, Alexander Kanavin <alex.kanavin@gmail.com>
>>> wrote:
>>>
>>>> On Thu, 4 May 2023 at 10:41, Peter Bergin <peter@berginkonsult.se>
>>>> wrote:
>>>>
>>>> > At
>>>> >
>>>> https://github.com/pyca/cryptography/blob/f816b457494e010b655cd7fdcd30e3446f86a703/src/rust/build.rs#L46
>>>> > the path to python includes is defined by calling 'python3 -c "import
>>>> > sysconfig; print(sysconfig.get_path('include'))"' (but from inside
>>>> > rust). With print-debugging this seems to return the bad string
>>>> > '/usr/include/python3.11' which is then passed to cc::Build
>>>> >
>>>> > I executed this in recipe context:
>>>> >
>>>> > do_compile:prepend() {
>>>> >      which python3
>>>> >      python3 -c "import sysconfig;
>>>> print(sysconfig.get_path('include'))"
>>>> > }
>>>> >
>>>> > and then I got the correct return value:
>>>> >
>>>> > |
>>>> >
>>>> /work/yocto/poky/build/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/recipe-sysroot-native/usr/bin/python3-native/python3
>>>> > |
>>>> >
>>>> /work/yocto/poky/build/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/recipe-sysroot-native/usr/include/python3.11
>>>>
>>>> This is pointing to the native sysroot in a target build, so it is
>>>> most likely not actually correct. I believe 'inherit
>>>> python3targetconfig' may help, as it substitutes most places where
>>>> native python gets queried by components with target-specific values.
>>>>
>>>> Alex
>>>>
>>>
>>> Unless I'm mistaken , in the error message we can see that the compiler
>>> specified the sysroot correctly (recipe-sysroot and recipe-sysroot-native)
>>>
>>
>> recipe-sysroot and NOT recipe-sysroot-native
>>
>> '--sysroot' is specified correct but that is not automatically add search
>> paths to '-I' directive, if I'm not misreading this:
>> https://gcc.gnu.org/onlinedocs/gcc/Directory-Options.html . When you use
>> --sysroot with gcc you have to add = or $SYSROOT to your -I directive in
>> order to add it to search path for headers. With that information I guess
>> '-I =/usr/lib/python3.11' or '-I \$SYSROOT/usr/lib/python3.11' can work but
>> '-I /usr/lib/python3.11' will point to your host.
>>
>> /Peter
>>
> I'm confused by the gcc docs, from the same links you gave:
>
> --sysroot=dir
>        Use dir as the logical root directory for headers and libraries.
> For example, if the compiler normally searches for headers in /usr/include
> and libraries in /usr/lib, it instead searches dir/usr/include and
> dir/usr/lib.
>
> So I don't know what to think (I never see -I used with '=' or $SYSROOT).
> I'll try to know more about that for my curiosity
>
> An example to illustrate:
>
>   $ cat main.c
>   #include "my-include.h"
>
>   int main(void) {
>       return 0;
>   }
>   $ tree dir1
>   dir1
>   └── dir2
>       └── my-include.h
>
>   1 directory, 1 file
>   $ gcc main.c
>   main.c:1:10: fatal error: my-include.h: No such file or directory
>       1 | #include "my-include.h"
>         |          ^~~~~~~~~~~~~~
>   compilation terminated.
>   $ gcc -I dir1/dir2 main.c
>   $ gcc --sysroot=$PWD/dir1 -I dir2 main.c
>   main.c:1:10: fatal error: my-include.h: No such file or directory
>       1 | #include "my-include.h"
>         |          ^~~~~~~~~~~~~~
>   compilation terminated.
>   $ gcc --sysroot=$PWD/dir1 -I =/dir2 main.c
>   $ gcc --sysroot=$PWD/dir1 -I \$SYSROOT/dir2 main.c
>
> /Peter
>
Thanks Peter, that was exactly what I wanted to try.

[-- Attachment #2: Type: text/html, Size: 9934 bytes --]

Re: [OE-core] [RFC] python3-cryptogaphy upgrade to 40.0.2 issues

[Tim Orling]

[-- Attachment #1: Type: text/plain, Size: 5768 bytes --]

On Thu, May 4, 2023 at 5:00 AM Peter Bergin <peter@berginkonsult.se> wrote:

>
> On 2023-05-04 11:00, Alexander Kanavin wrote:
> > On Thu, 4 May 2023 at 10:41, Peter Bergin <peter@berginkonsult.se>
> wrote:
> >
> >> At
> >>
> https://github.com/pyca/cryptography/blob/f816b457494e010b655cd7fdcd30e3446f86a703/src/rust/build.rs#L46
> >> the path to python includes is defined by calling 'python3 -c "import
> >> sysconfig; print(sysconfig.get_path('include'))"' (but from inside
> >> rust). With print-debugging this seems to return the bad string
> >> '/usr/include/python3.11' which is then passed to cc::Build
> >>
> >> I executed this in recipe context:
> >>
> >> do_compile:prepend() {
> >>       which python3
> >>       python3 -c "import sysconfig;
> print(sysconfig.get_path('include'))"
> >> }
> >>
> >> and then I got the correct return value:
> >>
> >> |
> >>
> /work/yocto/poky/build/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/recipe-sysroot-native/usr/bin/python3-native/python3
> >> |
> >>
> /work/yocto/poky/build/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/recipe-sysroot-native/usr/include/python3.11
> > This is pointing to the native sysroot in a target build, so it is
> > most likely not actually correct. I believe 'inherit
> > python3targetconfig' may help, as it substitutes most places where
> > native python gets queried by components with target-specific values.
> >
> > Alex
> >
> The bad thing is that the build.rs script is running python script to
> figure out the path to the headers. I'm not sure we can use target
> python exe in the build steps without running qemu instance or something
> which seems overkill in this case. We have the information about python
> include path for the target in the variable PYTHON_INCLUDE_DIR.
>
> With this patch the compilation will find Python.h:
>
> $ git diff src/rust/build.rs
> diff --git a/src/rust/build.rs b/src/rust/build.rs
> index 01177ac..7dc13fe 100644
> --- a/src/rust/build.rs
> +++ b/src/rust/build.rs
> @@ -43,11 +43,8 @@ fn main() {
>       )
>       .unwrap();
>       println!("cargo:rustc-cfg=python_implementation=\"{}\"",
> python_impl);
> -    let python_include = run_python_script(
> -        &python,
> -        "import sysconfig; print(sysconfig.get_path('include'), end='')",
> -    )
> -    .unwrap();
> +    let python_include = env::var("PYTHON_INCLUDE_DIR").unwrap();
> +
>       let openssl_include =
>           std::env::var_os("DEP_OPENSSL_INCLUDE").expect("unable to find
> openssl include path");
>       let openssl_c = Path::new(&out_dir).join("_openssl.c");
>
>
This patch at least got `python3-cryptography-native` to build to
completion.


>
> But it fails later in the linking step.
>
> .....linux/python3-cryptography/40.0.2-r0/recipe-sysroot/usr/lib/rustlib/x86_64-poky-linux-gnu/lib"
>
> "-L"
> "/work/yocto/poky/build/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/python3-cryptography-40.0.2/target/x86_64-poky-linux-gnu/release/build/cryptography-rust-aa00e39a952c07ee/out"
>
> "-L"
> "/work/yocto/poky/build/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/recipe-sysroot/usr/lib"
>
> "-L"
> "/work/yocto/poky/build/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/recipe-sysroot-native/usr/lib/rustlib/x86_64-poky-linux-gnu/lib"
>
> "-Wl,-Bstatic" "-l_openssl.a"
> "/work/yocto/poky/build/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/recipe-sysroot/usr/lib/rustlib/x86_64-poky-linux-gnu/lib/libcompiler_builtins-45df5d471f0d20d0.rlib"
>
> "-Wl,-Bdynamic" "-lssl" "-lcrypto" "-lgcc_s" "-lutil" "-lrt" "-lpthread"
> "-lm" "-ldl" "-lc" "-Wl,--eh-frame-hdr" "-Wl,-znoexecstack" "-L"
> "/work/yocto/poky/build/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/recipe-sysroot-native/usr/lib/rustlib/x86_64-poky-linux-gnu/lib"
>
> "-o"
> "/work/yocto/poky/build/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/python3-cryptography-40.0.2/target/x86_64-poky-linux-gnu/release/deps/libcryptography_rust.so"
>
> "-Wl,--gc-sections" "-shared" "-Wl,-O1" "-nodefaultlibs"
>    = note:
> /work/yocto/poky/build/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/recipe-sysroot-native/usr/bin/x86_64-poky-linux/../../libexec/x86_64-poky-linux/gcc/x86_64-poky-linux/12.2.0/ld:
>
> /work/yocto/poky/build/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/python3-cryptography-40.0.2/target/x86_64-poky-linux-gnu/release/build/cryptography-rust-aa00e39a952c07ee/out/lib_openssl.a.a(9bdf47a443b7a07f-_openssl.o):
>
> warning: relocation against `_Py_NoneStruct' in read-only section `.text'
> /work/yocto/poky/build/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/recipe-sysroot-native/usr/bin/x86_64-poky-linux/../../libexec/x86_64-poky-linux/gcc/x86_64-poky-linux/12.2.0/ld:
>
> /work/yocto/poky/build/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/python3-cryptography-40.0.2/target/x86_64-poky-linux-gnu/release/build/cryptography-rust-aa00e39a952c07ee/out/lib_openssl.a.a(9bdf47a443b7a07f-_openssl.o):
>
> relocation R_X86_64_PC32 against undefined symbol `_Py_NoneStruct' can
> not be used when making a shared object; recompile with -fPIC
> /work/yocto/poky/build/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/recipe-sysroot-native/usr/bin/x86_64-poky-linux/../../libexec/x86_64-poky-linux/gcc/x86_64-poky-linux/12.2.0/ld:
>
> final link failed: bad value
>            collect2: error: ld returned 1 exit status
>
>
> There are some '-L .../recipe-sysroot-native/usr/lib/rustlib/...' that
> can probably causing this. Don't know if it helps us forward?
>
> /Peter
>
>

[-- Attachment #2: Type: text/html, Size: 7928 bytes --]

Re: [OE-core] [RFC] python3-cryptogaphy upgrade to 40.0.2 issues

[Peter Bergin]

[-- Attachment #1: Type: text/plain, Size: 7069 bytes --]

Hi,

a short status update:

On 2023-05-05 06:01, Tim Orling wrote:
>
>
> On Thu, May 4, 2023 at 5:00 AM Peter Bergin <peter@berginkonsult.se> 
> wrote:
>
>
>     On 2023-05-04 11:00, Alexander Kanavin wrote:
>     > On Thu, 4 May 2023 at 10:41, Peter Bergin
>     <peter@berginkonsult.se> wrote:
>     >
>     >> At
>     >>
>     https://github.com/pyca/cryptography/blob/f816b457494e010b655cd7fdcd30e3446f86a703/src/rust/build.rs#L46
>     >> the path to python includes is defined by calling 'python3 -c
>     "import
>     >> sysconfig; print(sysconfig.get_path('include'))"' (but from inside
>     >> rust). With print-debugging this seems to return the bad string
>     >> '/usr/include/python3.11' which is then passed to cc::Build
>     >>
>     >> I executed this in recipe context:
>     >>
>     >> do_compile:prepend() {
>     >>       which python3
>     >>       python3 -c "import sysconfig;
>     print(sysconfig.get_path('include'))"
>     >> }
>     >>
>     >> and then I got the correct return value:
>     >>
>     >> |
>     >>
>     /work/yocto/poky/build/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/recipe-sysroot-native/usr/bin/python3-native/python3
>     >> |
>     >>
>     /work/yocto/poky/build/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/recipe-sysroot-native/usr/include/python3.11
>     > This is pointing to the native sysroot in a target build, so it is
>     > most likely not actually correct. I believe 'inherit
>     > python3targetconfig' may help, as it substitutes most places where
>     > native python gets queried by components with target-specific
>     values.
>     >
>     > Alex
>     >
>     The bad thing is that the build.rs <http://build.rs> script is
>     running python script to
>     figure out the path to the headers. I'm not sure we can use target
>     python exe in the build steps without running qemu instance or
>     something
>     which seems overkill in this case. We have the information about
>     python
>     include path for the target in the variable PYTHON_INCLUDE_DIR.
>
>     With this patch the compilation will find Python.h:
>
>     $ git diff src/rust/build.rs <http://build.rs>
>     diff --git a/src/rust/build.rs <http://build.rs>
>     b/src/rust/build.rs <http://build.rs>
>     index 01177ac..7dc13fe 100644
>     --- a/src/rust/build.rs <http://build.rs>
>     +++ b/src/rust/build.rs <http://build.rs>
>     @@ -43,11 +43,8 @@ fn main() {
>           )
>           .unwrap();
>     println!("cargo:rustc-cfg=python_implementation=\"{}\"", python_impl);
>     -    let python_include = run_python_script(
>     -        &python,
>     -        "import sysconfig; print(sysconfig.get_path('include'),
>     end='')",
>     -    )
>     -    .unwrap();
>     +    let python_include = env::var("PYTHON_INCLUDE_DIR").unwrap();
>     +
>           let openssl_include =
>     std::env::var_os("DEP_OPENSSL_INCLUDE").expect("unable to find
>     openssl include path");
>           let openssl_c = Path::new(&out_dir).join("_openssl.c");
>
>
> This patch at least got `python3-cryptography-native` to build to 
> completion.

And I was also able to build python3-cryptography for 'qemuarm64'. 
Opened an issue at https://github.com/pyca/cryptography/issues/8867 to 
see if it is something to be fixed upstream.

>
>     But it fails later in the linking step.
>
>     .....linux/python3-cryptography/40.0.2-r0/recipe-sysroot/usr/lib/rustlib/x86_64-poky-linux-gnu/lib"
>
>     "-L"
>     "/work/yocto/poky/build/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/python3-cryptography-40.0.2/target/x86_64-poky-linux-gnu/release/build/cryptography-rust-aa00e39a952c07ee/out"
>
>     "-L"
>     "/work/yocto/poky/build/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/recipe-sysroot/usr/lib"
>
>     "-L"
>     "/work/yocto/poky/build/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/recipe-sysroot-native/usr/lib/rustlib/x86_64-poky-linux-gnu/lib"
>
>     "-Wl,-Bstatic" "-l_openssl.a"
>     "/work/yocto/poky/build/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/recipe-sysroot/usr/lib/rustlib/x86_64-poky-linux-gnu/lib/libcompiler_builtins-45df5d471f0d20d0.rlib"
>
>     "-Wl,-Bdynamic" "-lssl" "-lcrypto" "-lgcc_s" "-lutil" "-lrt"
>     "-lpthread"
>     "-lm" "-ldl" "-lc" "-Wl,--eh-frame-hdr" "-Wl,-znoexecstack" "-L"
>     "/work/yocto/poky/build/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/recipe-sysroot-native/usr/lib/rustlib/x86_64-poky-linux-gnu/lib"
>
>     "-o"
>     "/work/yocto/poky/build/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/python3-cryptography-40.0.2/target/x86_64-poky-linux-gnu/release/deps/libcryptography_rust.so"
>
>     "-Wl,--gc-sections" "-shared" "-Wl,-O1" "-nodefaultlibs"
>        = note:
>     /work/yocto/poky/build/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/recipe-sysroot-native/usr/bin/x86_64-poky-linux/../../libexec/x86_64-poky-linux/gcc/x86_64-poky-linux/12.2.0/ld:
>
>     /work/yocto/poky/build/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/python3-cryptography-40.0.2/target/x86_64-poky-linux-gnu/release/build/cryptography-rust-aa00e39a952c07ee/out/lib_openssl.a.a(9bdf47a443b7a07f-_openssl.o):
>
>     warning: relocation against `_Py_NoneStruct' in read-only section
>     `.text'
>     /work/yocto/poky/build/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/recipe-sysroot-native/usr/bin/x86_64-poky-linux/../../libexec/x86_64-poky-linux/gcc/x86_64-poky-linux/12.2.0/ld:
>
>     /work/yocto/poky/build/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/python3-cryptography-40.0.2/target/x86_64-poky-linux-gnu/release/build/cryptography-rust-aa00e39a952c07ee/out/lib_openssl.a.a(9bdf47a443b7a07f-_openssl.o):
>
>     relocation R_X86_64_PC32 against undefined symbol `_Py_NoneStruct'
>     can
>     not be used when making a shared object; recompile with -fPIC
>     /work/yocto/poky/build/tmp/work/core2-64-poky-linux/python3-cryptography/40.0.2-r0/recipe-sysroot-native/usr/bin/x86_64-poky-linux/../../libexec/x86_64-poky-linux/gcc/x86_64-poky-linux/12.2.0/ld:
>
>     final link failed: bad value
>                collect2: error: ld returned 1 exit status
>
>
As I was able to complete a build for 'qemuarm64' but it fails on 
'qemux86_64' I am now trying to find differences....  Help appreciated! 
I have compared on the level of rust and how cargo is called and I can 
not find any difference apart from '-target [x86|aarch]64-poky-linux-gnu'.

My thoughts is that it can have to do with the binding to openssl and 
how it is built. Found an old issue 
(https://www.openembedded.org/pipermail/openembedded-core/2016-December/130334.html) 
with similar error message. But I'm not sure how it is related and 
neither how openssl integrates in to the build, I guess it is through 
openssl-sys. Any ideas around this are welcome. Is there a difference in 
'-fPIC' between x86_64 and arm64?

/Peter


[-- Attachment #2: Type: text/html, Size: 11631 bytes --]