* [PATCH V3] glibc: fix create thread failed in old unprivileged docker
[not found] <169FF34664D4ACC1.22970@lists.openembedded.org>
@ 2021-08-30 2:20 ` hongxu
2021-08-31 20:11 ` Richard Purdie
[not found] ` <169FF42BF9440142.22970@lists.openembedded.org>
1 sibling, 1 reply; 7+ messages in thread
From: hongxu @ 2021-08-30 2:20 UTC (permalink / raw)
To: openembedded-core, richard.purdie, raj.khem
# Changed in V3: add missing Upstream-Status tag
Since upstream commit [d8ea0d0168 Add an internal wrapper for clone, clone2
and clone3] applied, start a unprivileged container (docker run without
--privileged), it creates a thread failed in container.
In commit d8ea0d0168, it calls __clone3 if HAVE_CLONE3_WAPPER is defined. If
__clone3 returns -1 with ENOSYS, fall back to clone or clone2.
The newest docker has fixed the issue in commit [1], but it was applied
only on master, to backward compatibility with old docker, discussed
with glibc maintainer [2], explicitly disable clone3 wrapper work by removing
macro definition of HAVE_CLONE3_WRAPPER
[1] https://github.com/moby/moby/commit/9f6b562dd12ef7b1f9e2f8e6f2ab6477790a6594
[2] https://sourceware.org/pipermail/libc-alpha/2021-August/130591.html
Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
---
...hread-failed-in-unprivileged-process.patch | 94 +++++++++++++++++++
meta/recipes-core/glibc/glibc_2.34.bb | 1 +
2 files changed, 95 insertions(+)
create mode 100644 meta/recipes-core/glibc/glibc/0001-fix-create-thread-failed-in-unprivileged-process.patch
diff --git a/meta/recipes-core/glibc/glibc/0001-fix-create-thread-failed-in-unprivileged-process.patch b/meta/recipes-core/glibc/glibc/0001-fix-create-thread-failed-in-unprivileged-process.patch
new file mode 100644
index 0000000000..d3316c8c42
--- /dev/null
+++ b/meta/recipes-core/glibc/glibc/0001-fix-create-thread-failed-in-unprivileged-process.patch
@@ -0,0 +1,94 @@
+From 116fcbcbf5edbd7692f48280e996884c3df0e993 Mon Sep 17 00:00:00 2001
+From: Hongxu Jia <hongxu.jia@windriver.com>
+Date: Sun, 29 Aug 2021 20:49:16 +0800
+Subject: [PATCH] fix create thread failed in unprivileged process
+
+Since commit [d8ea0d0168 Add an internal wrapper for clone, clone2 and clone3]
+applied, start a unprivileged container (docker run without --privileged),
+it creates a thread failed in container.
+
+In commit d8ea0d0168, it calls __clone3 if HAVE_CLONE3_WAPPER is defined. If
+__clone3 returns -1 with ENOSYS, fall back to clone or clone2.
+
+The newest docker has fixed the issue in commit [1], but it was applied
+only on master, to backward compatibility with old docker, we explicitly
+disable clone3 wrapper work by removing macro definition of HAVE_CLONE3_WRAPPER
+
+[1] https://github.com/moby/moby/commit/9f6b562dd12ef7b1f9e2f8e6f2ab6477790a6594
+
+Here are the test steps:
+
+1) Prepare test code
+cat > conftest.c <<ENDOF
+ #include <pthread.h>
+ #include <stdio.h>
+
+int check_me = 0;
+void* func(void* data) {check_me = 42; printf("start thread: check_me %d\n", check_me); return &check_me;}
+int main()
+{
+ pthread_t t;
+ void *ret;
+ pthread_create (&t, 0, func, 0);
+ pthread_join (t, &ret);
+ printf("check_me %d, p %p\n", check_me, &ret);
+ return (check_me != 42 || ret != &check_me);
+}
+
+ENDOF
+
+2) Compile
+gcc -o conftest -pthread conftest.c
+
+3) Start a container with glibc 2.34 installed
+[skip details]
+docker run -it <container-image-name> bash
+
+4) Run conftest without this patch
+$ ./conftest
+check_me 0, p 0x7ffd91ccd400
+
+5) Run conftest with this patch
+$ ./conftest
+start thread: check_me 42
+check_me 42, p 0x7ffe253c6f20
+
+Upstream-Status: Inappropriate [oe specific]
+
+Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
+---
+ sysdeps/unix/sysv/linux/i386/sysdep.h | 3 ++-
+ sysdeps/unix/sysv/linux/x86_64/sysdep.h | 3 ++-
+ 2 files changed, 4 insertions(+), 2 deletions(-)
+
+diff --git a/sysdeps/unix/sysv/linux/i386/sysdep.h b/sysdeps/unix/sysv/linux/i386/sysdep.h
+index 8a9911b7ac..60d5cb2d9a 100644
+--- a/sysdeps/unix/sysv/linux/i386/sysdep.h
++++ b/sysdeps/unix/sysv/linux/i386/sysdep.h
+@@ -291,7 +291,8 @@ struct libc_do_syscall_args
+ # define HAVE_TIME_VSYSCALL "__vdso_time"
+ # define HAVE_CLOCK_GETRES_VSYSCALL "__vdso_clock_getres"
+
+-# define HAVE_CLONE3_WRAPPER 1
++/* Disable it to backward compatibility with old docker */
++//# define HAVE_CLONE3_WRAPPER 1
+
+ # undef HAVE_INTERNAL_BRK_ADDR_SYMBOL
+ # define HAVE_INTERNAL_BRK_ADDR_SYMBOL 1
+diff --git a/sysdeps/unix/sysv/linux/x86_64/sysdep.h b/sysdeps/unix/sysv/linux/x86_64/sysdep.h
+index 327e59388b..a7bc2cc686 100644
+--- a/sysdeps/unix/sysv/linux/x86_64/sysdep.h
++++ b/sysdeps/unix/sysv/linux/x86_64/sysdep.h
+@@ -377,7 +377,8 @@
+ # define HAVE_GETCPU_VSYSCALL "__vdso_getcpu"
+ # define HAVE_CLOCK_GETRES64_VSYSCALL "__vdso_clock_getres"
+
+-# define HAVE_CLONE3_WRAPPER 1
++/* Disable it to backward compatibility with old docker */
++//# define HAVE_CLONE3_WRAPPER 1
+
+ # define SINGLE_THREAD_BY_GLOBAL 1
+
+--
+2.27.0
+
diff --git a/meta/recipes-core/glibc/glibc_2.34.bb b/meta/recipes-core/glibc/glibc_2.34.bb
index eafc0216ff..46de1dae43 100644
--- a/meta/recipes-core/glibc/glibc_2.34.bb
+++ b/meta/recipes-core/glibc/glibc_2.34.bb
@@ -57,6 +57,7 @@ SRC_URI = "${GLIBC_GIT_URI};branch=${SRCBRANCH};name=glibc \
file://0030-powerpc-Do-not-ask-compiler-for-finding-arch.patch \
file://0001-CVE-2021-38604.patch \
file://0002-CVE-2021-38604.patch \
+ file://0001-fix-create-thread-failed-in-unprivileged-process.patch \
"
S = "${WORKDIR}/git"
B = "${WORKDIR}/build-${TARGET_SYS}"
--
2.30.2
^ permalink raw reply related [flat|nested] 7+ messages in thread
* Upgrade uninative to fix multiple native build failures in old unprivileged docker
[not found] ` <169FF42BF9440142.22970@lists.openembedded.org>
@ 2021-08-31 1:36 ` hongxu
2021-08-31 17:58 ` [OE-core] " Robert Berger
0 siblings, 1 reply; 7+ messages in thread
From: hongxu @ 2021-08-31 1:36 UTC (permalink / raw)
To: openembedded-core, richard.purdie, raj.khem, Michael Halstead
Hi Richard & Michael,
The reason why I submitted patch [glibc: fix create thread failed in old
unprivileged docker] is there are build failures with uninative 3.3 under
old unprivileged docker. If the glibc fix is OK, would you please upgrade
a new uninative that contains the fix.
Here are the steps to reproduce the failures
1) Run a container without privileged
$ docker pull resin/yocto-build-env
$ docker run -it -e HOST_UID="$(id -u)" -e DISTRO="poky" -v $PWD:/mnt -w
/mnt resin/yocto-build-env bash
2) Poky build preparation
$ apt update && apt install -y liblz4-tool zstd
$ useradd -u $HOST_UID -m -d /mnt/test -s /bin/bash test
$ su test
$ cd ~
3) Setup a Poky project with uninative enabled
$ git clone --branch master --single-branch git://git.yoctoproject.org/poky
$ cd poky && . ./oe-init-build-env
$ echo 'INHERIT += "uninative"' >> conf/local.conf
4) Build failures
...
$ bitbake ninja-native
|ninja: fatal: posix_spawn: Operation not permitted
$ bitbake go-cross-core2-64
|runtime/cgo: pthread_create failed: Operation not permitted
$ bitbake pkgconfig-native
| configure: error: I can't find the libraries for the thread implementation
...
After applying the glibc fix, I built a uninative tarball on my local
project to replace uninative 3.3. Then repeat above steps, no above
failures any more
//Hongxu
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [OE-core] Upgrade uninative to fix multiple native build failures in old unprivileged docker
2021-08-31 1:36 ` Upgrade uninative to fix multiple native build failures " hongxu
@ 2021-08-31 17:58 ` Robert Berger
2021-08-31 18:18 ` Martin Jansa
2021-08-31 19:50 ` Richard Purdie
0 siblings, 2 replies; 7+ messages in thread
From: Robert Berger @ 2021-08-31 17:58 UTC (permalink / raw)
To: hongxu, openembedded-core, richard.purdie, raj.khem, Michael Halstead
Hi,
I just experienced this problem in Docker version 19.03.13, build 4484c46d9d
> $ bitbake pkgconfig-native
> | configure: error: I can't find the libraries for the thread
> implementation
> ...
Can you please tell me what patches I need to apply to get this to work?
How will this be handled upstream?
I guess people would want to run a build in an unprivileged container.
Regards,
Robert
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [OE-core] Upgrade uninative to fix multiple native build failures in old unprivileged docker
2021-08-31 17:58 ` [OE-core] " Robert Berger
@ 2021-08-31 18:18 ` Martin Jansa
2021-08-31 19:50 ` Richard Purdie
1 sibling, 0 replies; 7+ messages in thread
From: Martin Jansa @ 2021-08-31 18:18 UTC (permalink / raw)
To: Robert Berger
Cc: hongxu, Patches and discussions about the oe-core layer,
Richard Purdie, Khem Raj, Michael Halstead
[-- Attachment #1: Type: text/plain, Size: 703 bytes --]
On Tue, Aug 31, 2021 at 7:58 PM Robert Berger <oecore.mailinglist@gmail.com>
wrote:
> Hi,
>
> I just experienced this problem in Docker version 19.03.13, build
> 4484c46d9d
>
> > $ bitbake pkgconfig-native
> > | configure: error: I can't find the libraries for the thread
> > implementation
> > ...
>
> Can you please tell me what patches I need to apply to get this to work?
>
https://lists.openembedded.org/g/openembedded-core/message/155465
and rebuild new uninative tarball with this applied.
> How will this be handled upstream?
>
Hopefully v20.10.9 docker will be released soon with:
https://github.com/moby/moby/commit/9f6b562dd12ef7b1f9e2f8e6f2ab6477790a6594
[-- Attachment #2: Type: text/html, Size: 1390 bytes --]
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [OE-core] Upgrade uninative to fix multiple native build failures in old unprivileged docker
2021-08-31 17:58 ` [OE-core] " Robert Berger
2021-08-31 18:18 ` Martin Jansa
@ 2021-08-31 19:50 ` Richard Purdie
1 sibling, 0 replies; 7+ messages in thread
From: Richard Purdie @ 2021-08-31 19:50 UTC (permalink / raw)
To: Robert Berger, hongxu, openembedded-core, raj.khem, Michael Halstead
On Tue, 2021-08-31 at 20:58 +0300, Robert Berger wrote:
> Hi,
>
> I just experienced this problem in Docker version 19.03.13, build 4484c46d9d
>
> > $ bitbake pkgconfig-native
> > | configure: error: I can't find the libraries for the thread
> > implementation
> > ...
>
> Can you please tell me what patches I need to apply to get this to work?
>
> How will this be handled upstream?
>
> I guess people would want to run a build in an unprivileged container.
You could also revert the last uninative upgrade and use an older glibc
uninative for now.
We'll get a fix merged soon and a new uninative built which can work around the
issue.
Cheers,
Richard
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH V3] glibc: fix create thread failed in old unprivileged docker
2021-08-30 2:20 ` [PATCH V3] glibc: fix create thread failed in old unprivileged docker hongxu
@ 2021-08-31 20:11 ` Richard Purdie
2021-09-01 0:22 ` hongxu
0 siblings, 1 reply; 7+ messages in thread
From: Richard Purdie @ 2021-08-31 20:11 UTC (permalink / raw)
To: Hongxu Jia, openembedded-core, raj.khem
On Sun, 2021-08-29 at 19:20 -0700, Hongxu Jia wrote:
> # Changed in V3: add missing Upstream-Status tag
>
> Since upstream commit [d8ea0d0168 Add an internal wrapper for clone, clone2
> and clone3] applied, start a unprivileged container (docker run without
> --privileged), it creates a thread failed in container.
>
> In commit d8ea0d0168, it calls __clone3 if HAVE_CLONE3_WAPPER is defined. If
> __clone3 returns -1 with ENOSYS, fall back to clone or clone2.
>
> The newest docker has fixed the issue in commit [1], but it was applied
> only on master, to backward compatibility with old docker, discussed
> with glibc maintainer [2], explicitly disable clone3 wrapper work by removing
> macro definition of HAVE_CLONE3_WRAPPER
>
> [1] https://github.com/moby/moby/commit/9f6b562dd12ef7b1f9e2f8e6f2ab6477790a6594
> [2] https://sourceware.org/pipermail/libc-alpha/2021-August/130591.html
>
> Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
Just for reference, I'm going with v1 of this fix.
The reason is that I do want clone3 to work on target and I do want the wrapper
to be available. Falling back in the EPERM case is therefore easiest.
I did wonder about making the fix nativesdk specific however I decided that was
going to complicate upgrades and so on a bit too much. The v1 fix is simple
enough it can be used everywhere without too many side effects.
Cheers,
Richard
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH V3] glibc: fix create thread failed in old unprivileged docker
2021-08-31 20:11 ` Richard Purdie
@ 2021-09-01 0:22 ` hongxu
0 siblings, 0 replies; 7+ messages in thread
From: hongxu @ 2021-09-01 0:22 UTC (permalink / raw)
To: Richard Purdie, openembedded-core, raj.khem
On 9/1/21 4:11 AM, Richard Purdie wrote:
> [Please note: This e-mail is from an EXTERNAL e-mail address]
>
> On Sun, 2021-08-29 at 19:20 -0700, Hongxu Jia wrote:
>> # Changed in V3: add missing Upstream-Status tag
>>
>> Since upstream commit [d8ea0d0168 Add an internal wrapper for clone, clone2
>> and clone3] applied, start a unprivileged container (docker run without
>> --privileged), it creates a thread failed in container.
>>
>> In commit d8ea0d0168, it calls __clone3 if HAVE_CLONE3_WAPPER is defined. If
>> __clone3 returns -1 with ENOSYS, fall back to clone or clone2.
>>
>> The newest docker has fixed the issue in commit [1], but it was applied
>> only on master, to backward compatibility with old docker, discussed
>> with glibc maintainer [2], explicitly disable clone3 wrapper work by removing
>> macro definition of HAVE_CLONE3_WRAPPER
>>
>> [1] https://github.com/moby/moby/commit/9f6b562dd12ef7b1f9e2f8e6f2ab6477790a6594
>> [2] https://sourceware.org/pipermail/libc-alpha/2021-August/130591.html
>>
>> Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
> Just for reference, I'm going with v1 of this fix.
>
> The reason is that I do want clone3 to work on target and I do want the wrapper
> to be available. Falling back in the EPERM case is therefore easiest.
>
> I did wonder about making the fix nativesdk specific however I decided that was
> going to complicate upgrades and so on a bit too much. The v1 fix is simple
> enough it can be used everywhere without too many side effects.
Absolutely agree
//Hongxu
> Cheers,
>
> Richard
>
^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2021-09-01 0:22 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
[not found] <169FF34664D4ACC1.22970@lists.openembedded.org>
2021-08-30 2:20 ` [PATCH V3] glibc: fix create thread failed in old unprivileged docker hongxu
2021-08-31 20:11 ` Richard Purdie
2021-09-01 0:22 ` hongxu
[not found] ` <169FF42BF9440142.22970@lists.openembedded.org>
2021-08-31 1:36 ` Upgrade uninative to fix multiple native build failures " hongxu
2021-08-31 17:58 ` [OE-core] " Robert Berger
2021-08-31 18:18 ` Martin Jansa
2021-08-31 19:50 ` Richard Purdie
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.