* [PATCH V3] glibc: fix create thread failed in old unprivileged docker [not found] <169FF34664D4ACC1.22970@lists.openembedded.org> @ 2021-08-30 2:20 ` hongxu 2021-08-31 20:11 ` Richard Purdie [not found] ` <169FF42BF9440142.22970@lists.openembedded.org> 1 sibling, 1 reply; 7+ messages in thread From: hongxu @ 2021-08-30 2:20 UTC (permalink / raw) To: openembedded-core, richard.purdie, raj.khem # Changed in V3: add missing Upstream-Status tag Since upstream commit [d8ea0d0168 Add an internal wrapper for clone, clone2 and clone3] applied, start a unprivileged container (docker run without --privileged), it creates a thread failed in container. In commit d8ea0d0168, it calls __clone3 if HAVE_CLONE3_WAPPER is defined. If __clone3 returns -1 with ENOSYS, fall back to clone or clone2. The newest docker has fixed the issue in commit [1], but it was applied only on master, to backward compatibility with old docker, discussed with glibc maintainer [2], explicitly disable clone3 wrapper work by removing macro definition of HAVE_CLONE3_WRAPPER [1] https://github.com/moby/moby/commit/9f6b562dd12ef7b1f9e2f8e6f2ab6477790a6594 [2] https://sourceware.org/pipermail/libc-alpha/2021-August/130591.html Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com> --- ...hread-failed-in-unprivileged-process.patch | 94 +++++++++++++++++++ meta/recipes-core/glibc/glibc_2.34.bb | 1 + 2 files changed, 95 insertions(+) create mode 100644 meta/recipes-core/glibc/glibc/0001-fix-create-thread-failed-in-unprivileged-process.patch diff --git a/meta/recipes-core/glibc/glibc/0001-fix-create-thread-failed-in-unprivileged-process.patch b/meta/recipes-core/glibc/glibc/0001-fix-create-thread-failed-in-unprivileged-process.patch new file mode 100644 index 0000000000..d3316c8c42 --- /dev/null +++ b/meta/recipes-core/glibc/glibc/0001-fix-create-thread-failed-in-unprivileged-process.patch @@ -0,0 +1,94 @@ +From 116fcbcbf5edbd7692f48280e996884c3df0e993 Mon Sep 17 00:00:00 2001 +From: Hongxu Jia <hongxu.jia@windriver.com> +Date: Sun, 29 Aug 2021 20:49:16 +0800 +Subject: [PATCH] fix create thread failed in unprivileged process + +Since commit [d8ea0d0168 Add an internal wrapper for clone, clone2 and clone3] +applied, start a unprivileged container (docker run without --privileged), +it creates a thread failed in container. + +In commit d8ea0d0168, it calls __clone3 if HAVE_CLONE3_WAPPER is defined. If +__clone3 returns -1 with ENOSYS, fall back to clone or clone2. + +The newest docker has fixed the issue in commit [1], but it was applied +only on master, to backward compatibility with old docker, we explicitly +disable clone3 wrapper work by removing macro definition of HAVE_CLONE3_WRAPPER + +[1] https://github.com/moby/moby/commit/9f6b562dd12ef7b1f9e2f8e6f2ab6477790a6594 + +Here are the test steps: + +1) Prepare test code +cat > conftest.c <<ENDOF + #include <pthread.h> + #include <stdio.h> + +int check_me = 0; +void* func(void* data) {check_me = 42; printf("start thread: check_me %d\n", check_me); return &check_me;} +int main() +{ + pthread_t t; + void *ret; + pthread_create (&t, 0, func, 0); + pthread_join (t, &ret); + printf("check_me %d, p %p\n", check_me, &ret); + return (check_me != 42 || ret != &check_me); +} + +ENDOF + +2) Compile +gcc -o conftest -pthread conftest.c + +3) Start a container with glibc 2.34 installed +[skip details] +docker run -it <container-image-name> bash + +4) Run conftest without this patch +$ ./conftest +check_me 0, p 0x7ffd91ccd400 + +5) Run conftest with this patch +$ ./conftest +start thread: check_me 42 +check_me 42, p 0x7ffe253c6f20 + +Upstream-Status: Inappropriate [oe specific] + +Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com> +--- + sysdeps/unix/sysv/linux/i386/sysdep.h | 3 ++- + sysdeps/unix/sysv/linux/x86_64/sysdep.h | 3 ++- + 2 files changed, 4 insertions(+), 2 deletions(-) + +diff --git a/sysdeps/unix/sysv/linux/i386/sysdep.h b/sysdeps/unix/sysv/linux/i386/sysdep.h +index 8a9911b7ac..60d5cb2d9a 100644 +--- a/sysdeps/unix/sysv/linux/i386/sysdep.h ++++ b/sysdeps/unix/sysv/linux/i386/sysdep.h +@@ -291,7 +291,8 @@ struct libc_do_syscall_args + # define HAVE_TIME_VSYSCALL "__vdso_time" + # define HAVE_CLOCK_GETRES_VSYSCALL "__vdso_clock_getres" + +-# define HAVE_CLONE3_WRAPPER 1 ++/* Disable it to backward compatibility with old docker */ ++//# define HAVE_CLONE3_WRAPPER 1 + + # undef HAVE_INTERNAL_BRK_ADDR_SYMBOL + # define HAVE_INTERNAL_BRK_ADDR_SYMBOL 1 +diff --git a/sysdeps/unix/sysv/linux/x86_64/sysdep.h b/sysdeps/unix/sysv/linux/x86_64/sysdep.h +index 327e59388b..a7bc2cc686 100644 +--- a/sysdeps/unix/sysv/linux/x86_64/sysdep.h ++++ b/sysdeps/unix/sysv/linux/x86_64/sysdep.h +@@ -377,7 +377,8 @@ + # define HAVE_GETCPU_VSYSCALL "__vdso_getcpu" + # define HAVE_CLOCK_GETRES64_VSYSCALL "__vdso_clock_getres" + +-# define HAVE_CLONE3_WRAPPER 1 ++/* Disable it to backward compatibility with old docker */ ++//# define HAVE_CLONE3_WRAPPER 1 + + # define SINGLE_THREAD_BY_GLOBAL 1 + +-- +2.27.0 + diff --git a/meta/recipes-core/glibc/glibc_2.34.bb b/meta/recipes-core/glibc/glibc_2.34.bb index eafc0216ff..46de1dae43 100644 --- a/meta/recipes-core/glibc/glibc_2.34.bb +++ b/meta/recipes-core/glibc/glibc_2.34.bb @@ -57,6 +57,7 @@ SRC_URI = "${GLIBC_GIT_URI};branch=${SRCBRANCH};name=glibc \ file://0030-powerpc-Do-not-ask-compiler-for-finding-arch.patch \ file://0001-CVE-2021-38604.patch \ file://0002-CVE-2021-38604.patch \ + file://0001-fix-create-thread-failed-in-unprivileged-process.patch \ " S = "${WORKDIR}/git" B = "${WORKDIR}/build-${TARGET_SYS}" -- 2.30.2 ^ permalink raw reply related [flat|nested] 7+ messages in thread
* Re: [PATCH V3] glibc: fix create thread failed in old unprivileged docker 2021-08-30 2:20 ` [PATCH V3] glibc: fix create thread failed in old unprivileged docker hongxu @ 2021-08-31 20:11 ` Richard Purdie 2021-09-01 0:22 ` hongxu 0 siblings, 1 reply; 7+ messages in thread From: Richard Purdie @ 2021-08-31 20:11 UTC (permalink / raw) To: Hongxu Jia, openembedded-core, raj.khem On Sun, 2021-08-29 at 19:20 -0700, Hongxu Jia wrote: > # Changed in V3: add missing Upstream-Status tag > > Since upstream commit [d8ea0d0168 Add an internal wrapper for clone, clone2 > and clone3] applied, start a unprivileged container (docker run without > --privileged), it creates a thread failed in container. > > In commit d8ea0d0168, it calls __clone3 if HAVE_CLONE3_WAPPER is defined. If > __clone3 returns -1 with ENOSYS, fall back to clone or clone2. > > The newest docker has fixed the issue in commit [1], but it was applied > only on master, to backward compatibility with old docker, discussed > with glibc maintainer [2], explicitly disable clone3 wrapper work by removing > macro definition of HAVE_CLONE3_WRAPPER > > [1] https://github.com/moby/moby/commit/9f6b562dd12ef7b1f9e2f8e6f2ab6477790a6594 > [2] https://sourceware.org/pipermail/libc-alpha/2021-August/130591.html > > Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com> Just for reference, I'm going with v1 of this fix. The reason is that I do want clone3 to work on target and I do want the wrapper to be available. Falling back in the EPERM case is therefore easiest. I did wonder about making the fix nativesdk specific however I decided that was going to complicate upgrades and so on a bit too much. The v1 fix is simple enough it can be used everywhere without too many side effects. Cheers, Richard ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH V3] glibc: fix create thread failed in old unprivileged docker 2021-08-31 20:11 ` Richard Purdie @ 2021-09-01 0:22 ` hongxu 0 siblings, 0 replies; 7+ messages in thread From: hongxu @ 2021-09-01 0:22 UTC (permalink / raw) To: Richard Purdie, openembedded-core, raj.khem On 9/1/21 4:11 AM, Richard Purdie wrote: > [Please note: This e-mail is from an EXTERNAL e-mail address] > > On Sun, 2021-08-29 at 19:20 -0700, Hongxu Jia wrote: >> # Changed in V3: add missing Upstream-Status tag >> >> Since upstream commit [d8ea0d0168 Add an internal wrapper for clone, clone2 >> and clone3] applied, start a unprivileged container (docker run without >> --privileged), it creates a thread failed in container. >> >> In commit d8ea0d0168, it calls __clone3 if HAVE_CLONE3_WAPPER is defined. If >> __clone3 returns -1 with ENOSYS, fall back to clone or clone2. >> >> The newest docker has fixed the issue in commit [1], but it was applied >> only on master, to backward compatibility with old docker, discussed >> with glibc maintainer [2], explicitly disable clone3 wrapper work by removing >> macro definition of HAVE_CLONE3_WRAPPER >> >> [1] https://github.com/moby/moby/commit/9f6b562dd12ef7b1f9e2f8e6f2ab6477790a6594 >> [2] https://sourceware.org/pipermail/libc-alpha/2021-August/130591.html >> >> Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com> > Just for reference, I'm going with v1 of this fix. > > The reason is that I do want clone3 to work on target and I do want the wrapper > to be available. Falling back in the EPERM case is therefore easiest. > > I did wonder about making the fix nativesdk specific however I decided that was > going to complicate upgrades and so on a bit too much. The v1 fix is simple > enough it can be used everywhere without too many side effects. Absolutely agree //Hongxu > Cheers, > > Richard > ^ permalink raw reply [flat|nested] 7+ messages in thread
[parent not found: <169FF42BF9440142.22970@lists.openembedded.org>]
* Upgrade uninative to fix multiple native build failures in old unprivileged docker [not found] ` <169FF42BF9440142.22970@lists.openembedded.org> @ 2021-08-31 1:36 ` hongxu 2021-08-31 17:58 ` [OE-core] " Robert Berger 0 siblings, 1 reply; 7+ messages in thread From: hongxu @ 2021-08-31 1:36 UTC (permalink / raw) To: openembedded-core, richard.purdie, raj.khem, Michael Halstead Hi Richard & Michael, The reason why I submitted patch [glibc: fix create thread failed in old unprivileged docker] is there are build failures with uninative 3.3 under old unprivileged docker. If the glibc fix is OK, would you please upgrade a new uninative that contains the fix. Here are the steps to reproduce the failures 1) Run a container without privileged $ docker pull resin/yocto-build-env $ docker run -it -e HOST_UID="$(id -u)" -e DISTRO="poky" -v $PWD:/mnt -w /mnt resin/yocto-build-env bash 2) Poky build preparation $ apt update && apt install -y liblz4-tool zstd $ useradd -u $HOST_UID -m -d /mnt/test -s /bin/bash test $ su test $ cd ~ 3) Setup a Poky project with uninative enabled $ git clone --branch master --single-branch git://git.yoctoproject.org/poky $ cd poky && . ./oe-init-build-env $ echo 'INHERIT += "uninative"' >> conf/local.conf 4) Build failures ... $ bitbake ninja-native |ninja: fatal: posix_spawn: Operation not permitted $ bitbake go-cross-core2-64 |runtime/cgo: pthread_create failed: Operation not permitted $ bitbake pkgconfig-native | configure: error: I can't find the libraries for the thread implementation ... After applying the glibc fix, I built a uninative tarball on my local project to replace uninative 3.3. Then repeat above steps, no above failures any more //Hongxu ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [OE-core] Upgrade uninative to fix multiple native build failures in old unprivileged docker 2021-08-31 1:36 ` Upgrade uninative to fix multiple native build failures " hongxu @ 2021-08-31 17:58 ` Robert Berger 2021-08-31 18:18 ` Martin Jansa 2021-08-31 19:50 ` Richard Purdie 0 siblings, 2 replies; 7+ messages in thread From: Robert Berger @ 2021-08-31 17:58 UTC (permalink / raw) To: hongxu, openembedded-core, richard.purdie, raj.khem, Michael Halstead Hi, I just experienced this problem in Docker version 19.03.13, build 4484c46d9d > $ bitbake pkgconfig-native > | configure: error: I can't find the libraries for the thread > implementation > ... Can you please tell me what patches I need to apply to get this to work? How will this be handled upstream? I guess people would want to run a build in an unprivileged container. Regards, Robert ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [OE-core] Upgrade uninative to fix multiple native build failures in old unprivileged docker 2021-08-31 17:58 ` [OE-core] " Robert Berger @ 2021-08-31 18:18 ` Martin Jansa 2021-08-31 19:50 ` Richard Purdie 1 sibling, 0 replies; 7+ messages in thread From: Martin Jansa @ 2021-08-31 18:18 UTC (permalink / raw) To: Robert Berger Cc: hongxu, Patches and discussions about the oe-core layer, Richard Purdie, Khem Raj, Michael Halstead [-- Attachment #1: Type: text/plain, Size: 703 bytes --] On Tue, Aug 31, 2021 at 7:58 PM Robert Berger <oecore.mailinglist@gmail.com> wrote: > Hi, > > I just experienced this problem in Docker version 19.03.13, build > 4484c46d9d > > > $ bitbake pkgconfig-native > > | configure: error: I can't find the libraries for the thread > > implementation > > ... > > Can you please tell me what patches I need to apply to get this to work? > https://lists.openembedded.org/g/openembedded-core/message/155465 and rebuild new uninative tarball with this applied. > How will this be handled upstream? > Hopefully v20.10.9 docker will be released soon with: https://github.com/moby/moby/commit/9f6b562dd12ef7b1f9e2f8e6f2ab6477790a6594 [-- Attachment #2: Type: text/html, Size: 1390 bytes --] ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [OE-core] Upgrade uninative to fix multiple native build failures in old unprivileged docker 2021-08-31 17:58 ` [OE-core] " Robert Berger 2021-08-31 18:18 ` Martin Jansa @ 2021-08-31 19:50 ` Richard Purdie 1 sibling, 0 replies; 7+ messages in thread From: Richard Purdie @ 2021-08-31 19:50 UTC (permalink / raw) To: Robert Berger, hongxu, openembedded-core, raj.khem, Michael Halstead On Tue, 2021-08-31 at 20:58 +0300, Robert Berger wrote: > Hi, > > I just experienced this problem in Docker version 19.03.13, build 4484c46d9d > > > $ bitbake pkgconfig-native > > | configure: error: I can't find the libraries for the thread > > implementation > > ... > > Can you please tell me what patches I need to apply to get this to work? > > How will this be handled upstream? > > I guess people would want to run a build in an unprivileged container. You could also revert the last uninative upgrade and use an older glibc uninative for now. We'll get a fix merged soon and a new uninative built which can work around the issue. Cheers, Richard ^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2021-09-01 0:22 UTC | newest] Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- [not found] <169FF34664D4ACC1.22970@lists.openembedded.org> 2021-08-30 2:20 ` [PATCH V3] glibc: fix create thread failed in old unprivileged docker hongxu 2021-08-31 20:11 ` Richard Purdie 2021-09-01 0:22 ` hongxu [not found] ` <169FF42BF9440142.22970@lists.openembedded.org> 2021-08-31 1:36 ` Upgrade uninative to fix multiple native build failures " hongxu 2021-08-31 17:58 ` [OE-core] " Robert Berger 2021-08-31 18:18 ` Martin Jansa 2021-08-31 19:50 ` Richard Purdie
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.