[PATCH V3] glibc: fix create thread failed in old unprivileged docker

[PATCH V3] glibc: fix create thread failed in old unprivileged docker

[hongxu]

# Changed in V3: add missing Upstream-Status tag

Since upstream commit [d8ea0d0168 Add an internal wrapper for clone, clone2
and clone3] applied, start a unprivileged container (docker run without
--privileged), it creates a thread failed in container.

In commit d8ea0d0168, it calls __clone3 if HAVE_CLONE3_WAPPER is defined.  If
__clone3 returns -1 with ENOSYS, fall back to clone or clone2.

The newest docker has fixed the issue in commit [1], but it was applied
only on master, to backward compatibility with old docker, discussed
with glibc maintainer [2], explicitly disable clone3 wrapper work by removing
macro definition of HAVE_CLONE3_WRAPPER

[1] https://github.com/moby/moby/commit/9f6b562dd12ef7b1f9e2f8e6f2ab6477790a6594
[2] https://sourceware.org/pipermail/libc-alpha/2021-August/130591.html

Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
---
 ...hread-failed-in-unprivileged-process.patch | 94 +++++++++++++++++++
 meta/recipes-core/glibc/glibc_2.34.bb         |  1 +
 2 files changed, 95 insertions(+)
 create mode 100644 meta/recipes-core/glibc/glibc/0001-fix-create-thread-failed-in-unprivileged-process.patch

diff --git a/meta/recipes-core/glibc/glibc/0001-fix-create-thread-failed-in-unprivileged-process.patch b/meta/recipes-core/glibc/glibc/0001-fix-create-thread-failed-in-unprivileged-process.patch
new file mode 100644
index 0000000000..d3316c8c42
--- /dev/null
+++ b/meta/recipes-core/glibc/glibc/0001-fix-create-thread-failed-in-unprivileged-process.patch
@@ -0,0 +1,94 @@
+From 116fcbcbf5edbd7692f48280e996884c3df0e993 Mon Sep 17 00:00:00 2001
+From: Hongxu Jia <hongxu.jia@windriver.com>
+Date: Sun, 29 Aug 2021 20:49:16 +0800
+Subject: [PATCH] fix create thread failed in unprivileged process
+
+Since commit [d8ea0d0168 Add an internal wrapper for clone, clone2 and clone3]
+applied, start a unprivileged container (docker run without --privileged),
+it creates a thread failed in container.
+
+In commit d8ea0d0168, it calls __clone3 if HAVE_CLONE3_WAPPER is defined.  If
+__clone3 returns -1 with ENOSYS, fall back to clone or clone2.
+
+The newest docker has fixed the issue in commit [1], but it was applied
+only on master, to backward compatibility with old docker, we explicitly
+disable clone3 wrapper work by removing macro definition of HAVE_CLONE3_WRAPPER
+
+[1] https://github.com/moby/moby/commit/9f6b562dd12ef7b1f9e2f8e6f2ab6477790a6594
+
+Here are the test steps:
+
+1) Prepare test code
+cat > conftest.c <<ENDOF
+ #include <pthread.h>
+ #include <stdio.h>
+
+int check_me = 0;
+void* func(void* data) {check_me = 42; printf("start thread: check_me %d\n", check_me); return &check_me;}
+int main()
+{
+  pthread_t t;
+  void *ret;
+  pthread_create (&t, 0, func, 0);
+  pthread_join (t, &ret);
+  printf("check_me %d, p %p\n", check_me, &ret);
+  return (check_me != 42 || ret != &check_me);
+}
+
+ENDOF
+
+2) Compile
+gcc -o conftest -pthread conftest.c
+
+3) Start a container with glibc 2.34 installed
+[skip details]
+docker run -it <container-image-name> bash
+
+4) Run conftest without this patch
+$ ./conftest
+check_me 0, p 0x7ffd91ccd400
+
+5) Run conftest with this patch
+$ ./conftest
+start thread: check_me 42
+check_me 42, p 0x7ffe253c6f20
+
+Upstream-Status: Inappropriate [oe specific]
+
+Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
+---
+ sysdeps/unix/sysv/linux/i386/sysdep.h   | 3 ++-
+ sysdeps/unix/sysv/linux/x86_64/sysdep.h | 3 ++-
+ 2 files changed, 4 insertions(+), 2 deletions(-)
+
+diff --git a/sysdeps/unix/sysv/linux/i386/sysdep.h b/sysdeps/unix/sysv/linux/i386/sysdep.h
+index 8a9911b7ac..60d5cb2d9a 100644
+--- a/sysdeps/unix/sysv/linux/i386/sysdep.h
++++ b/sysdeps/unix/sysv/linux/i386/sysdep.h
+@@ -291,7 +291,8 @@ struct libc_do_syscall_args
+ # define HAVE_TIME_VSYSCALL             "__vdso_time"
+ # define HAVE_CLOCK_GETRES_VSYSCALL     "__vdso_clock_getres"
+ 
+-# define HAVE_CLONE3_WRAPPER		1
++/* Disable it to backward compatibility with old docker */
++//# define HAVE_CLONE3_WRAPPER		1
+ 
+ # undef HAVE_INTERNAL_BRK_ADDR_SYMBOL
+ # define HAVE_INTERNAL_BRK_ADDR_SYMBOL 1
+diff --git a/sysdeps/unix/sysv/linux/x86_64/sysdep.h b/sysdeps/unix/sysv/linux/x86_64/sysdep.h
+index 327e59388b..a7bc2cc686 100644
+--- a/sysdeps/unix/sysv/linux/x86_64/sysdep.h
++++ b/sysdeps/unix/sysv/linux/x86_64/sysdep.h
+@@ -377,7 +377,8 @@
+ # define HAVE_GETCPU_VSYSCALL		"__vdso_getcpu"
+ # define HAVE_CLOCK_GETRES64_VSYSCALL   "__vdso_clock_getres"
+ 
+-# define HAVE_CLONE3_WRAPPER			1
++/* Disable it to backward compatibility with old docker */
++//# define HAVE_CLONE3_WRAPPER			1
+ 
+ # define SINGLE_THREAD_BY_GLOBAL		1
+ 
+-- 
+2.27.0
+
diff --git a/meta/recipes-core/glibc/glibc_2.34.bb b/meta/recipes-core/glibc/glibc_2.34.bb
index eafc0216ff..46de1dae43 100644
--- a/meta/recipes-core/glibc/glibc_2.34.bb
+++ b/meta/recipes-core/glibc/glibc_2.34.bb
@@ -57,6 +57,7 @@ SRC_URI =  "${GLIBC_GIT_URI};branch=${SRCBRANCH};name=glibc \
            file://0030-powerpc-Do-not-ask-compiler-for-finding-arch.patch \
            file://0001-CVE-2021-38604.patch \
            file://0002-CVE-2021-38604.patch \
+           file://0001-fix-create-thread-failed-in-unprivileged-process.patch \
            "
 S = "${WORKDIR}/git"
 B = "${WORKDIR}/build-${TARGET_SYS}"
-- 
2.30.2

Upgrade uninative to fix multiple native build failures in old unprivileged docker

[hongxu]

Hi Richard & Michael,

The reason why I submitted patch [glibc: fix create thread failed in old
unprivileged docker] is there are build failures with uninative 3.3 under
old unprivileged docker. If the glibc fix is OK, would you please upgrade
a new uninative that contains the fix.

Here are the steps to reproduce the failures
1) Run a container without privileged
$ docker pull resin/yocto-build-env
$ docker run -it -e HOST_UID="$(id -u)" -e DISTRO="poky" -v $PWD:/mnt -w 
/mnt resin/yocto-build-env bash

2) Poky build preparation
$ apt update && apt install -y liblz4-tool zstd
$ useradd -u $HOST_UID -m -d /mnt/test -s /bin/bash test
$ su test
$ cd ~

3) Setup a Poky project with uninative enabled
$ git clone --branch master --single-branch git://git.yoctoproject.org/poky
$ cd poky && . ./oe-init-build-env
$ echo 'INHERIT += "uninative"' >> conf/local.conf

4) Build failures
...
$ bitbake ninja-native
|ninja: fatal: posix_spawn: Operation not permitted

$ bitbake go-cross-core2-64
|runtime/cgo: pthread_create failed: Operation not permitted

$ bitbake pkgconfig-native
| configure: error: I can't find the libraries for the thread implementation
...

After applying the glibc fix, I built a uninative tarball on my local
project to replace uninative 3.3. Then repeat above steps, no above
failures any more

//Hongxu

Re: [OE-core] Upgrade uninative to fix multiple native build failures in old unprivileged docker

[Robert Berger]

Hi,

I just experienced this problem in Docker version 19.03.13, build 4484c46d9d

 > $ bitbake pkgconfig-native
 > | configure: error: I can't find the libraries for the thread
 > implementation
 > ...

Can you please tell me what patches I need to apply to get this to work?

How will this be handled upstream?

I guess people would want to run a build in an unprivileged container.

Regards,

Robert

Re: [OE-core] Upgrade uninative to fix multiple native build failures in old unprivileged docker

[Martin Jansa]

[-- Attachment #1: Type: text/plain, Size: 703 bytes --]

On Tue, Aug 31, 2021 at 7:58 PM Robert Berger <oecore.mailinglist@gmail.com>
wrote:

> Hi,
>
> I just experienced this problem in Docker version 19.03.13, build
> 4484c46d9d
>
>  > $ bitbake pkgconfig-native
>  > | configure: error: I can't find the libraries for the thread
>  > implementation
>  > ...
>
> Can you please tell me what patches I need to apply to get this to work?
>

https://lists.openembedded.org/g/openembedded-core/message/155465

and rebuild new uninative tarball with this applied.


> How will this be handled upstream?
>

Hopefully v20.10.9 docker will be released soon with:
https://github.com/moby/moby/commit/9f6b562dd12ef7b1f9e2f8e6f2ab6477790a6594

[-- Attachment #2: Type: text/html, Size: 1390 bytes --]

Re: [OE-core] Upgrade uninative to fix multiple native build failures in old unprivileged docker

[Richard Purdie]

On Tue, 2021-08-31 at 20:58 +0300, Robert Berger wrote:
> Hi,
> 
> I just experienced this problem in Docker version 19.03.13, build 4484c46d9d
> 
>  > $ bitbake pkgconfig-native
>  > | configure: error: I can't find the libraries for the thread
>  > implementation
>  > ...
> 
> Can you please tell me what patches I need to apply to get this to work?
> 
> How will this be handled upstream?
> 
> I guess people would want to run a build in an unprivileged container.

You could also revert the last uninative upgrade and use an older glibc
uninative for now.

We'll get a fix merged soon and a new uninative built which can work around the
issue.

Cheers,

Richard

Re: [PATCH V3] glibc: fix create thread failed in old unprivileged docker

[Richard Purdie]

On Sun, 2021-08-29 at 19:20 -0700, Hongxu Jia wrote:
> # Changed in V3: add missing Upstream-Status tag
> 
> Since upstream commit [d8ea0d0168 Add an internal wrapper for clone, clone2
> and clone3] applied, start a unprivileged container (docker run without
> --privileged), it creates a thread failed in container.
> 
> In commit d8ea0d0168, it calls __clone3 if HAVE_CLONE3_WAPPER is defined.  If
> __clone3 returns -1 with ENOSYS, fall back to clone or clone2.
> 
> The newest docker has fixed the issue in commit [1], but it was applied
> only on master, to backward compatibility with old docker, discussed
> with glibc maintainer [2], explicitly disable clone3 wrapper work by removing
> macro definition of HAVE_CLONE3_WRAPPER
> 
> [1] https://github.com/moby/moby/commit/9f6b562dd12ef7b1f9e2f8e6f2ab6477790a6594
> [2] https://sourceware.org/pipermail/libc-alpha/2021-August/130591.html
> 
> Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>

Just for reference, I'm going with v1 of this fix.

The reason is that I do want clone3 to work on target and I do want the wrapper
to be available. Falling back in the EPERM case is therefore easiest.

I did wonder about making the fix nativesdk specific however I decided that was
going to complicate upgrades and so on a bit too much. The v1 fix is simple
enough it can be used everywhere without too many side effects.

Cheers,

Richard

Re: [PATCH V3] glibc: fix create thread failed in old unprivileged docker

[hongxu]

On 9/1/21 4:11 AM, Richard Purdie wrote:
> [Please note: This e-mail is from an EXTERNAL e-mail address]
>
> On Sun, 2021-08-29 at 19:20 -0700, Hongxu Jia wrote:
>> # Changed in V3: add missing Upstream-Status tag
>>
>> Since upstream commit [d8ea0d0168 Add an internal wrapper for clone, clone2
>> and clone3] applied, start a unprivileged container (docker run without
>> --privileged), it creates a thread failed in container.
>>
>> In commit d8ea0d0168, it calls __clone3 if HAVE_CLONE3_WAPPER is defined.  If
>> __clone3 returns -1 with ENOSYS, fall back to clone or clone2.
>>
>> The newest docker has fixed the issue in commit [1], but it was applied
>> only on master, to backward compatibility with old docker, discussed
>> with glibc maintainer [2], explicitly disable clone3 wrapper work by removing
>> macro definition of HAVE_CLONE3_WRAPPER
>>
>> [1] https://github.com/moby/moby/commit/9f6b562dd12ef7b1f9e2f8e6f2ab6477790a6594
>> [2] https://sourceware.org/pipermail/libc-alpha/2021-August/130591.html
>>
>> Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
> Just for reference, I'm going with v1 of this fix.
>
> The reason is that I do want clone3 to work on target and I do want the wrapper
> to be available. Falling back in the EPERM case is therefore easiest.
>
> I did wonder about making the fix nativesdk specific however I decided that was
> going to complicate upgrades and so on a bit too much. The v1 fix is simple
> enough it can be used everywhere without too many side effects.

Absolutely agree

//Hongxu

> Cheers,
>
> Richard
>