* [Buildroot] issue with timer_create and openblas after update buildroot from 2019_02 to 2021_02 @ 2021-04-07 9:01 Arthur Lambert 2021-04-08 22:59 ` Peter Seiderer 0 siblings, 1 reply; 6+ messages in thread From: Arthur Lambert @ 2021-04-07 9:01 UTC (permalink / raw) To: buildroot I am trying to update buildroot from 2019_02 tag to 2021_02. After the update I have some issue with timer_create function from uclibc . I get -1 return error with EAGAIN errno. The error only occurred with SIGEV_THREAD type. I was able to track the code responsible from the error without understanding the reason output/build/uclibc-1.0.37/libpthread/nptl/sysdeps/unix/sysv/linux/timer_create.c line-150 >> { >> /* Create the helper thread. */ >> pthread_once (&__helper_once, __start_helper_thread); >> if (__helper_tid == 0) >> { >> /* No resources to start the helper thread. */ >> __set_errno (EAGAIN); >> return -1; >> } I am was also able to reproduce the issue with a minimalist piece of code. The issue seems to be related to openblas. I dont see the link between BLAS and thread/timer issue ?? >> #include <stdlib.h> >> #include <unistd.h> >> #include <stdio.h> >> #include <signal.h> >> #include <time.h> >> >> >> int main(void) >> { >> >> struct sigevent alarm_sigevt; >> timer_t alarm_timer; >> int ret = 0; >> >> alarm_sigevt.sigev_notify = SIGEV_THREAD; >> alarm_sigevt.sigev_notify_function = NULL; >> alarm_sigevt.sigev_notify_attributes = NULL; >> alarm_sigevt.sigev_value.sival_ptr = NULL; >> ret = timer_create(CLOCK_REALTIME, &alarm_sigevt, &alarm_timer); >> printf ("ret : %d\n", ret); >> >> exit(EXIT_SUCCESS); >> } I compile the code like this : /home/arthur/work/buildroot/atto_dreem3_dev/output/host/bin/arm-buildroot-linux-uclibcgnueabihf-gcc test_timer.c -o test_timer I run the binary on my arm target : # ./test_timer ret : 0 # Everything is working fine here. I just add openblas library now : /home/arthur/work/buildroot/atto_dreem3_dev/output/host/bin/arm-buildroot-linux-uclibcgnueabihf-gcc test_timer.c -o test_timer -lopenblas I now run the new binary : # ./test_timer ret : -1 An issue with openblas which init too many timer ? -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20210407/9f6f31fe/attachment.html> ^ permalink raw reply [flat|nested] 6+ messages in thread
* [Buildroot] issue with timer_create and openblas after update buildroot from 2019_02 to 2021_02 2021-04-07 9:01 [Buildroot] issue with timer_create and openblas after update buildroot from 2019_02 to 2021_02 Arthur Lambert @ 2021-04-08 22:59 ` Peter Seiderer 2021-04-09 7:53 ` Arthur Lambert 0 siblings, 1 reply; 6+ messages in thread From: Peter Seiderer @ 2021-04-08 22:59 UTC (permalink / raw) To: buildroot Hello Arthur, On Wed, 7 Apr 2021 11:01:59 +0200, Arthur Lambert <lambertarthur22@gmail.com> wrote: > I am trying to update buildroot from 2019_02 tag to 2021_02. After the > update I have > some issue with timer_create function from uclibc . I get -1 return error > with EAGAIN errno. > The error only occurred with SIGEV_THREAD type. > > I was able to track the code responsible from the error without > understanding the reason > > output/build/uclibc-1.0.37/libpthread/nptl/sysdeps/unix/sysv/linux/timer_create.c > line-150 > > >> { > >> /* Create the helper thread. */ > >> pthread_once (&__helper_once, __start_helper_thread); > >> if (__helper_tid == 0) > >> { > >> /* No resources to start the helper thread. */ > >> __set_errno (EAGAIN); > >> return -1; > >> } > > I am was also able to reproduce the issue with a minimalist piece of code. > The issue seems > to be related to openblas. I dont see the link between BLAS and > thread/timer issue ?? > > >> #include <stdlib.h> > >> #include <unistd.h> > >> #include <stdio.h> > >> #include <signal.h> > >> #include <time.h> > >> > >> > >> int main(void) > >> { > >> > >> struct sigevent alarm_sigevt; > >> timer_t alarm_timer; > >> int ret = 0; > >> > >> alarm_sigevt.sigev_notify = SIGEV_THREAD; > >> alarm_sigevt.sigev_notify_function = NULL; > >> alarm_sigevt.sigev_notify_attributes = NULL; > >> alarm_sigevt.sigev_value.sival_ptr = NULL; > >> ret = timer_create(CLOCK_REALTIME, &alarm_sigevt, &alarm_timer); > >> printf ("ret : %d\n", ret); > >> > >> exit(EXIT_SUCCESS); > >> } > > I compile the code like this : > > /home/arthur/work/buildroot/atto_dreem3_dev/output/host/bin/arm-buildroot-linux-uclibcgnueabihf-gcc > test_timer.c -o test_timer > > I run the binary on my arm target : > > # ./test_timer > ret : 0 > # > > Everything is working fine here. The strace log looks like the following: [...] 235 stat("/lib/ld-uClibc.so.0", {st_mode=S_IFREG|0755, st_size=28408, ...}) = 0 235 mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb6f6b000 235 set_tls(0xb6f6b490) = 0 235 mprotect(0x20000, 4096, PROT_READ) = 0 235 mprotect(0xb6f40000, 4096, PROT_READ) = 0 235 mprotect(0xb6f6d000, 4096, PROT_READ) = 0 235 set_tid_address(0xb6f6b068) = 235 235 set_robust_list(0xb6f6b06c, 12) = 0 235 rt_sigaction(SIGRTMIN, {sa_handler=0xb6f239e8, sa_mask=[], sa_flags=SA_RESTORER|SA_SIGINFO, sa_restorer=0xb6ed4908}, NULL, 8) = 0 235 rt_sigaction(SIGRT_1, {sa_handler=0xb6f23aac, sa_mask=[], sa_flags=SA_RESTORER|SA_RESTART|SA_SIGINFO, sa_restorer=0xb6ed4908}, NULL, 8) = 0 235 rt_sigprocmask(SIG_UNBLOCK, [RTMIN RT_1], NULL, 8) = 0 235 ugetrlimit(RLIMIT_STACK, {rlim_cur=8192*1024, rlim_max=RLIM_INFINITY}) = 0 235 ioctl(0, TCGETS, {B115200 opost isig icanon echo ...}) = 0 235 ioctl(1, TCGETS, {B115200 opost isig icanon echo ...}) = 0 235 rt_sigprocmask(SIG_SETMASK, ~[RT_1], [], 8) = 0 235 mmap2(NULL, 16384, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_STACK, -1, 0) = 0xb6f67000 235 brk(NULL) = 0x1934000 235 brk(0x1935000) = 0x1935000 235 mprotect(0xb6f67000, 4096, PROT_NONE) = 0 235 clone(child_stack=0xb6f6a038, flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID, parent_tid=[236], tls=0xb6f6a960, child_tidptr=0xb6f6a538) = 236 235 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 235 futex(0xb6f57824, FUTEX_WAKE_PRIVATE, 2147483647) = 0 235 timer_create(CLOCK_REALTIME, {sigev_value={sival_int=26427544, sival_ptr=0x1934098}, sigev_signo=SIGRTMIN, sigev_notify=SIGEV_THREAD_ID, sigev_notify_thread_id=236}, [0]) = 0 235 write(1, "ret : 0\n", 8) = 8 235 exit_group(0) = ? 236 +++ exited with 0 +++ 235 +++ exited with 0 +++ > I just add openblas library now : > > /home/arthur/work/buildroot/atto_dreem3_dev/output/host/bin/arm-buildroot-linux-uclibcgnueabihf-gcc > test_timer.c -o test_timer -lopenblas > > I now run the new binary : > > # ./test_timer > ret : -1 > > An issue with openblas which init too many timer ? The strace log looks like the following: [...] 240 stat("/lib/ld-uClibc.so.0", {st_mode=S_IFREG|0755, st_size=28408, ...}) = 0 240 mmap2(NULL, 64304, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb6d0d000 240 mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb6f93000 240 set_tls(0xb6d0d490) = 0 240 mprotect(0x20000, 4096, PROT_READ) = 0 240 mprotect(0xb6f7b000, 8192, PROT_READ) = 0 240 mprotect(0xb6d9b000, 4096, PROT_READ) = 0 240 mprotect(0xb6f95000, 4096, PROT_READ) = 0 240 set_tid_address(0xb6d0d068) = 240 240 set_robust_list(0xb6d0d06c, 12) = 0 240 rt_sigaction(SIGRTMIN, {sa_handler=0xb6d7e9e8, sa_mask=[], sa_flags=SA_RESTORER|SA_SIGINFO, sa_restorer=0xb6d2f908}, NULL, 8) = 0 240 rt_sigaction(SIGRT_1, {sa_handler=0xb6d7eaac, sa_mask=[], sa_flags=SA_RESTORER|SA_RESTART|SA_SIGINFO, sa_restorer=0xb6d2f908}, NULL, 8) = 0 240 rt_sigprocmask(SIG_UNBLOCK, [RTMIN RT_1], NULL, 8) = 0 240 ugetrlimit(RLIMIT_STACK, {rlim_cur=8192*1024, rlim_max=RLIM_INFINITY}) = 0 240 ioctl(0, TCGETS, {B115200 opost isig icanon echo ...}) = 0 240 ioctl(1, TCGETS, {B115200 opost isig icanon echo ...}) = 0 240 open("/sys/devices/system/cpu", O_RDONLY|O_NONBLOCK|O_CLOEXEC|O_DIRECTORY) = 3 240 fstat64(3, {st_mode=S_IFDIR|0755, st_size=0, ...}) = 0 240 fcntl(3, F_SETFD, FD_CLOEXEC) = 0 240 brk(NULL) = 0x1485000 240 brk(0x1486000) = 0x1486000 240 brk(0x1487000) = 0x1487000 240 getdents64(3, 0xbed7fa28 /* 13 entries */, 4096) = 392 240 getdents64(3, 0xbed7fa28 /* 0 entries */, 4096) = 0 240 close(3) = 0 240 rt_sigprocmask(SIG_SETMASK, ~[RT_1], [], 8) = 0 240 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 240 futex(0xb6db2824, FUTEX_WAKE_PRIVATE, 2147483647) = 0 240 write(1, "ret : -1\n", 9) = 9 240 exit_group(0) = ? 240 +++ exited with 0 +++ No clone() and no timer_create() syscall...., but the syscall sequence from open("/sys/devices/system/cpu",...) to getdents64()/close() looks like the nprocessors_conf() implementation from uclibc-1.0.38/libc/unistd/sysconf.c..., strange.... Regards, Peter ^ permalink raw reply [flat|nested] 6+ messages in thread
* [Buildroot] issue with timer_create and openblas after update buildroot from 2019_02 to 2021_02 2021-04-08 22:59 ` Peter Seiderer @ 2021-04-09 7:53 ` Arthur Lambert 2021-04-09 19:18 ` Peter Seiderer 0 siblings, 1 reply; 6+ messages in thread From: Arthur Lambert @ 2021-04-09 7:53 UTC (permalink / raw) To: buildroot I was able to fix the issue by checking the Openblas history commits. But I don't really understand the issue... The issue is coming from here: https://github.com/buildroot/buildroot/commit/6f29cdeee40e556d26b791fabaff84a09e1a2d5d#diff-0995e357753caee10bbc553c2e7aa033d1df05c0c9799916729d4881d7735d26 By default, the multithreading setting is enabled on Openblas. This setting is really confusing me. For me, I have to enable the multithreading setting to use Openblas in a multithreading context (this is my case !). But it is the opposite! There is this information in the Openblas USAGE file : >> #### How can I use OpenBLAS in multi-threaded applications? >> >> If your application is already multi-threaded, it will conflict with OpenBLAS >> multi-threading. Thus, you must set OpenBLAS to use single thread in any of the >> following ways: >> >> * `export OPENBLAS_NUM_THREADS=1` in the environment variables. >> * Call `openblas_set_num_threads(1)` in the application on runtime. >> * Build OpenBLAS single thread version, e.g. `make USE_THREAD=0` So I have to add this in my defconfig to build Openblas with singlethread support to fix the problem: iff --git a/configs/xxxx_defconfig b/configs/dxxxxxx_defconfig index 4714080d7e..a760627e2c 100644 --- a/configs/xxxxx_defconfig +++ b/configs/xxxx_defconfig @@ -45,6 +45,7 @@ BR2_PACKAGE_PAHO_MQTT_C=y BR2_PACKAGE_ELFUTILS=y BR2_PACKAGE_LIBUNWIND=y BR2_PACKAGE_OPENBLAS=y +# BR2_PACKAGE_OPENBLAS_USE_THREAD is not set BR2_PACKAGE_PROTOBUF=y BR2_PACKAGE_PROTOBUF_C=y BR2_PACKAGE_HAVEGED=y How this "conflict" can cause an error on the timer_create feature from uclibc? Is my change correct? -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20210409/ed9ba3df/attachment.html> ^ permalink raw reply related [flat|nested] 6+ messages in thread
* [Buildroot] issue with timer_create and openblas after update buildroot from 2019_02 to 2021_02 2021-04-09 7:53 ` Arthur Lambert @ 2021-04-09 19:18 ` Peter Seiderer 2021-04-13 19:52 ` Peter Seiderer 0 siblings, 1 reply; 6+ messages in thread From: Peter Seiderer @ 2021-04-09 19:18 UTC (permalink / raw) To: buildroot Hello Arthur, On Fri, 9 Apr 2021 09:53:23 +0200, Arthur Lambert <lambertarthur22@gmail.com> wrote: > I was able to fix the issue by checking the Openblas history commits. But I > don't really understand the issue... > > The issue is coming from here: > https://github.com/buildroot/buildroot/commit/6f29cdeee40e556d26b791fabaff84a09e1a2d5d#diff-0995e357753caee10bbc553c2e7aa033d1df05c0c9799916729d4881d7735d26 > > By default, the multithreading setting is enabled on Openblas. This setting > is really confusing me. For me, I have to enable the multithreading setting > to use Openblas > in a multithreading context (this is my case !). But it is the opposite! > There is this information in the Openblas USAGE file : > > >> #### How can I use OpenBLAS in multi-threaded applications? > >> > >> If your application is already multi-threaded, it will conflict with > OpenBLAS > >> multi-threading. Thus, you must set OpenBLAS to use single thread in any > of the > >> following ways: > >> > >> * `export OPENBLAS_NUM_THREADS=1` in the environment variables. > >> * Call `openblas_set_num_threads(1)` in the application on runtime. > >> * Build OpenBLAS single thread version, e.g. `make USE_THREAD=0` > > So I have to add this in my defconfig to build Openblas with > singlethread support to fix the problem: > > iff --git a/configs/xxxx_defconfig b/configs/dxxxxxx_defconfig > index 4714080d7e..a760627e2c 100644 > --- a/configs/xxxxx_defconfig > +++ b/configs/xxxx_defconfig > @@ -45,6 +45,7 @@ BR2_PACKAGE_PAHO_MQTT_C=y > BR2_PACKAGE_ELFUTILS=y > BR2_PACKAGE_LIBUNWIND=y > BR2_PACKAGE_OPENBLAS=y > +# BR2_PACKAGE_OPENBLAS_USE_THREAD is not set > BR2_PACKAGE_PROTOBUF=y > BR2_PACKAGE_PROTOBUF_C=y > BR2_PACKAGE_HAVEGED=y > > How this "conflict" can cause an error on the timer_create feature from > uclibc? This commit gives you only the chance to disable openblas threading support (and circumvent the problem for you), but is not the decisive change between buildroot 2019.02 and 2021.02 regarding uclibc/openblas, because already the old/2019.02 version should have threading enabled (only without an buildroot option to disable it)... The version change from 2019.02 to 2021.02 are: - openblas from 0.2.20 to 0.3.9 - uclibc from 1.0.31 to 1.0.38 - gcc/binutils/etc (did not yet check yet)? But I did some debugging, with openblas linking the timer_create() call fails because of a failing pthread_create() because of a stack-size check in allocate_stack()... Without openblas (o.k - one call via timer_create()/pthread_create()): Breakpoint 1, allocate_stack (attr=0x7fed1079c8, pdp=0x7fed107920, stack=0x7fed107928) at libpthread/nptl/allocatestack.c:472 472 pd = get_cached_stack (&size, &mem); (gdb) p *attr $1 = {schedparam = {__sched_priority = 0}, schedpolicy = 0, flags = 0, guardsize = 4096, stackaddr = 0x0, stacksize = 16384, cpuset = 0x0, cpusetsize = 0} (gdb) p size $2 = 16384 (gdb) p __static_tls_size $3 = 1712 With openblas (failure - three time o.k call from openblas, one failing call from timer_create()): Breakpoint 1, allocate_stack (attr=0x7fa286de48 <default_attr>, pdp=0x7febf5fd80, stack=0x7febf5fd88) at libpthread/nptl/allocatestack.c:472 472 pd = get_cached_stack (&size, &mem); (gdb) p *attr $1 = {schedparam = {__sched_priority = 0}, schedpolicy = 0, flags = 0, guardsize = 1, stackaddr = 0x0, stacksize = 0, cpuset = 0x0, cpusetsize = 0} (gdb) p size $2 = 2097152 (gdb) p __static_tls_size $3 = 63152 [...] Thread 1 "test_timer_crea" hit Breakpoint 2, allocate_stack (attr=0x7febf61068, pdp=0x7febf60fc0, stack=0x7febf60fc8) at libpthread/nptl/allocatestack.c:468 468 return EINVAL; (gdb) p *attr $4 = {schedparam = {__sched_priority = 0}, schedpolicy = 0, flags = 0, guardsize = 4096, stackaddr = 0x0, stacksize = 16384, cpuset = 0x0, cpusetsize = 0} (gdb) p size $5 = 16384 (gdb) p __static_tls_size $6 = 63152 - with openblas linking the thread-local-storage (__static_tls_size) is increased from 1712 to 63152 - the openblas threads are created with default pthread_attr_t/nullptr, the stack size size is calculated to 2097152 the check succeeds - the timer_create pthread_create is called with pthread_attr_t created via 66 (void) pthread_attr_init (&attr); 167 (void) pthread_attr_setstacksize (&attr, PTHREAD_STACK_MIN); so stacksize is given as 16384, the check against the increased thread-local- storage (63152 + guadsize + MINIMAL_REST_STACK + padding) fails... Maybe a uclibc bug/defect...., or an expert question: is the pthread_create/ pthread_attr_t stacksize inclusive or exclusive the reserved size for thread-local-storage usage (my judgment would be exclusive as the thread-local storage size is dependents on external parameters/library linking etc.)? > Is my change correct? If it works for you yes ;-), but does not fix the underlying/generally problem... Regards, Peter ^ permalink raw reply [flat|nested] 6+ messages in thread
* [Buildroot] issue with timer_create and openblas after update buildroot from 2019_02 to 2021_02 2021-04-09 19:18 ` Peter Seiderer @ 2021-04-13 19:52 ` Peter Seiderer 2021-04-16 21:26 ` Peter Seiderer 0 siblings, 1 reply; 6+ messages in thread From: Peter Seiderer @ 2021-04-13 19:52 UTC (permalink / raw) To: buildroot Hello Arthur, Waldemar, On Fri, 9 Apr 2021 21:18:10 +0200, Peter Seiderer <ps.report@gmx.net> wrote: > Hello Arthur, > > On Fri, 9 Apr 2021 09:53:23 +0200, Arthur Lambert <lambertarthur22@gmail.com> wrote: > > > I was able to fix the issue by checking the Openblas history commits. But I > > don't really understand the issue... > > > > The issue is coming from here: > > https://github.com/buildroot/buildroot/commit/6f29cdeee40e556d26b791fabaff84a09e1a2d5d#diff-0995e357753caee10bbc553c2e7aa033d1df05c0c9799916729d4881d7735d26 > > > > By default, the multithreading setting is enabled on Openblas. This setting > > is really confusing me. For me, I have to enable the multithreading setting > > to use Openblas > > in a multithreading context (this is my case !). But it is the opposite! > > There is this information in the Openblas USAGE file : > > > > >> #### How can I use OpenBLAS in multi-threaded applications? > > >> > > >> If your application is already multi-threaded, it will conflict with > > OpenBLAS > > >> multi-threading. Thus, you must set OpenBLAS to use single thread in any > > of the > > >> following ways: > > >> > > >> * `export OPENBLAS_NUM_THREADS=1` in the environment variables. > > >> * Call `openblas_set_num_threads(1)` in the application on runtime. > > >> * Build OpenBLAS single thread version, e.g. `make USE_THREAD=0` > > > > So I have to add this in my defconfig to build Openblas with > > singlethread support to fix the problem: > > > > iff --git a/configs/xxxx_defconfig b/configs/dxxxxxx_defconfig > > index 4714080d7e..a760627e2c 100644 > > --- a/configs/xxxxx_defconfig > > +++ b/configs/xxxx_defconfig > > @@ -45,6 +45,7 @@ BR2_PACKAGE_PAHO_MQTT_C=y > > BR2_PACKAGE_ELFUTILS=y > > BR2_PACKAGE_LIBUNWIND=y > > BR2_PACKAGE_OPENBLAS=y > > +# BR2_PACKAGE_OPENBLAS_USE_THREAD is not set > > BR2_PACKAGE_PROTOBUF=y > > BR2_PACKAGE_PROTOBUF_C=y > > BR2_PACKAGE_HAVEGED=y > > > > How this "conflict" can cause an error on the timer_create feature from > > uclibc? > > This commit gives you only the chance to disable openblas threading support > (and circumvent the problem for you), but is not the decisive change > between buildroot 2019.02 and 2021.02 regarding uclibc/openblas, because > already the old/2019.02 version should have threading enabled (only without > an buildroot option to disable it)... > > The version change from 2019.02 to 2021.02 are: > > - openblas from 0.2.20 to 0.3.9 > - uclibc from 1.0.31 to 1.0.38 > - gcc/binutils/etc (did not yet check yet)? > > But I did some debugging, with openblas linking the timer_create() call > fails because of a failing pthread_create() because of a stack-size > check in allocate_stack()... > > > Without openblas (o.k - one call via timer_create()/pthread_create()): > > Breakpoint 1, allocate_stack (attr=0x7fed1079c8, pdp=0x7fed107920, stack=0x7fed107928) > at libpthread/nptl/allocatestack.c:472 > 472 pd = get_cached_stack (&size, &mem); > (gdb) p *attr > $1 = {schedparam = {__sched_priority = 0}, schedpolicy = 0, flags = 0, guardsize = 4096, > stackaddr = 0x0, stacksize = 16384, cpuset = 0x0, cpusetsize = 0} > (gdb) p size > $2 = 16384 > (gdb) p __static_tls_size > $3 = 1712 > > > With openblas (failure - three time o.k call from openblas, one failing call from timer_create()): > > Breakpoint 1, allocate_stack (attr=0x7fa286de48 <default_attr>, pdp=0x7febf5fd80, stack=0x7febf5fd88) > at libpthread/nptl/allocatestack.c:472 > 472 pd = get_cached_stack (&size, &mem); > (gdb) p *attr > $1 = {schedparam = {__sched_priority = 0}, schedpolicy = 0, flags = 0, guardsize = 1, stackaddr = 0x0, > stacksize = 0, cpuset = 0x0, cpusetsize = 0} > (gdb) p size > $2 = 2097152 > (gdb) p __static_tls_size > $3 = 63152 > > [...] > > Thread 1 "test_timer_crea" hit Breakpoint 2, allocate_stack (attr=0x7febf61068, pdp=0x7febf60fc0, > stack=0x7febf60fc8) at libpthread/nptl/allocatestack.c:468 > 468 return EINVAL; > (gdb) p *attr > $4 = {schedparam = {__sched_priority = 0}, schedpolicy = 0, flags = 0, guardsize = 4096, > stackaddr = 0x0, stacksize = 16384, cpuset = 0x0, cpusetsize = 0} > (gdb) p size > $5 = 16384 > (gdb) p __static_tls_size > $6 = 63152 > > > - with openblas linking the thread-local-storage (__static_tls_size) is > increased from 1712 to 63152 > > - the openblas threads are created with default pthread_attr_t/nullptr, > the stack size size is calculated to 2097152 the check succeeds > > - the timer_create pthread_create is called with pthread_attr_t created > via > > 66 (void) pthread_attr_init (&attr); > 167 (void) pthread_attr_setstacksize (&attr, PTHREAD_STACK_MIN); > > so stacksize is given as 16384, the check against the increased thread-local- > storage (63152 + guadsize + MINIMAL_REST_STACK + padding) fails... > > Maybe a uclibc bug/defect...., or an expert question: is the pthread_create/ > pthread_attr_t stacksize inclusive or exclusive the reserved size for > thread-local-storage usage (my judgment would be exclusive as the thread-local > storage size is dependents on external parameters/library linking etc.)? I think uclibc-ng needs something as implemented for glibc (see [1]), using __static_tls_size + PTHREAD_STACK_MIN as stack size for timer_create thread (take the thread local storage stack part into account)..., draft patch like the following works for the openblas/ timer_create example: diff --git a/libpthread/nptl/init.c b/libpthread/nptl/init.c index 4959d5ed8..fefdda1a5 100644 --- a/libpthread/nptl/init.c +++ b/libpthread/nptl/init.c @@ -337,3 +337,9 @@ __pthread_initialize_minimal_internal (void) } strong_alias (__pthread_initialize_minimal_internal, __pthread_initialize_minimal) + +size_t +__pthread_get_minstack (const pthread_attr_t *attr) +{ + return __static_tls_size + PTHREAD_STACK_MIN; +} diff --git a/libpthread/nptl/pthreadP.h b/libpthread/nptl/pthreadP.h index 13205512a..c686a4ca5 100644 --- a/libpthread/nptl/pthreadP.h +++ b/libpthread/nptl/pthreadP.h @@ -377,6 +377,7 @@ weak_function; extern void __pthread_init_static_tls (struct link_map *) attribute_hidden; +extern size_t __pthread_get_minstack (const pthread_attr_t *attr); /* Namespace save aliases. */ extern int __pthread_getschedparam (pthread_t thread_id, int *policy, diff --git a/libpthread/nptl/sysdeps/unix/sysv/linux/timer_routines.c b/libpthread/nptl/sysdeps/unix/sysv/linux/timer_routines.c index 514913317..60f2a724c 100644 --- a/libpthread/nptl/sysdeps/unix/sysv/linux/timer_routines.c +++ b/libpthread/nptl/sysdeps/unix/sysv/linux/timer_routines.c @@ -164,7 +164,7 @@ __start_helper_thread (void) and should go away automatically when canceled. */ pthread_attr_t attr; (void) pthread_attr_init (&attr); - (void) pthread_attr_setstacksize (&attr, PTHREAD_STACK_MIN); + (void) pthread_attr_setstacksize (&attr, __pthread_get_minstack (&attr)); /* Block all signals in the helper thread but SIGSETXID. To do this thoroughly we temporarily have to block all signals here. The Regards, Peter [1] https://sourceware.org/git/?p=glibc.git;a=commitdiff;h=2c1094bd700e63a8d7f547b3f5495bedb55c0a08;hp=3b8dfc621bfd320c924a3cd597086d3473da1cf4 > > > Is my change correct? > > If it works for you yes ;-), but does not fix the underlying/generally problem... > > Regards, > Peter > _______________________________________________ > buildroot mailing list > buildroot at busybox.net > http://lists.busybox.net/mailman/listinfo/buildroot ^ permalink raw reply related [flat|nested] 6+ messages in thread
* [Buildroot] issue with timer_create and openblas after update buildroot from 2019_02 to 2021_02 2021-04-13 19:52 ` Peter Seiderer @ 2021-04-16 21:26 ` Peter Seiderer 0 siblings, 0 replies; 6+ messages in thread From: Peter Seiderer @ 2021-04-16 21:26 UTC (permalink / raw) To: buildroot On Tue, 13 Apr 2021 21:52:39 +0200, Peter Seiderer <ps.report@gmx.net> wrote: > Hello Arthur, Waldemar, > > On Fri, 9 Apr 2021 21:18:10 +0200, Peter Seiderer <ps.report@gmx.net> wrote: > > > Hello Arthur, > > > > On Fri, 9 Apr 2021 09:53:23 +0200, Arthur Lambert <lambertarthur22@gmail.com> wrote: > > > > > I was able to fix the issue by checking the Openblas history commits. But I > > > don't really understand the issue... > > > > > > The issue is coming from here: > > > https://github.com/buildroot/buildroot/commit/6f29cdeee40e556d26b791fabaff84a09e1a2d5d#diff-0995e357753caee10bbc553c2e7aa033d1df05c0c9799916729d4881d7735d26 > > > > > > By default, the multithreading setting is enabled on Openblas. This setting > > > is really confusing me. For me, I have to enable the multithreading setting > > > to use Openblas > > > in a multithreading context (this is my case !). But it is the opposite! > > > There is this information in the Openblas USAGE file : > > > > > > >> #### How can I use OpenBLAS in multi-threaded applications? > > > >> > > > >> If your application is already multi-threaded, it will conflict with > > > OpenBLAS > > > >> multi-threading. Thus, you must set OpenBLAS to use single thread in any > > > of the > > > >> following ways: > > > >> > > > >> * `export OPENBLAS_NUM_THREADS=1` in the environment variables. > > > >> * Call `openblas_set_num_threads(1)` in the application on runtime. > > > >> * Build OpenBLAS single thread version, e.g. `make USE_THREAD=0` > > > > > > So I have to add this in my defconfig to build Openblas with > > > singlethread support to fix the problem: > > > > > > iff --git a/configs/xxxx_defconfig b/configs/dxxxxxx_defconfig > > > index 4714080d7e..a760627e2c 100644 > > > --- a/configs/xxxxx_defconfig > > > +++ b/configs/xxxx_defconfig > > > @@ -45,6 +45,7 @@ BR2_PACKAGE_PAHO_MQTT_C=y > > > BR2_PACKAGE_ELFUTILS=y > > > BR2_PACKAGE_LIBUNWIND=y > > > BR2_PACKAGE_OPENBLAS=y > > > +# BR2_PACKAGE_OPENBLAS_USE_THREAD is not set > > > BR2_PACKAGE_PROTOBUF=y > > > BR2_PACKAGE_PROTOBUF_C=y > > > BR2_PACKAGE_HAVEGED=y > > > > > > How this "conflict" can cause an error on the timer_create feature from > > > uclibc? > > > > This commit gives you only the chance to disable openblas threading support > > (and circumvent the problem for you), but is not the decisive change > > between buildroot 2019.02 and 2021.02 regarding uclibc/openblas, because > > already the old/2019.02 version should have threading enabled (only without > > an buildroot option to disable it)... > > > > The version change from 2019.02 to 2021.02 are: > > > > - openblas from 0.2.20 to 0.3.9 > > - uclibc from 1.0.31 to 1.0.38 > > - gcc/binutils/etc (did not yet check yet)? > > > > But I did some debugging, with openblas linking the timer_create() call > > fails because of a failing pthread_create() because of a stack-size > > check in allocate_stack()... > > > > > > Without openblas (o.k - one call via timer_create()/pthread_create()): > > > > Breakpoint 1, allocate_stack (attr=0x7fed1079c8, pdp=0x7fed107920, stack=0x7fed107928) > > at libpthread/nptl/allocatestack.c:472 > > 472 pd = get_cached_stack (&size, &mem); > > (gdb) p *attr > > $1 = {schedparam = {__sched_priority = 0}, schedpolicy = 0, flags = 0, guardsize = 4096, > > stackaddr = 0x0, stacksize = 16384, cpuset = 0x0, cpusetsize = 0} > > (gdb) p size > > $2 = 16384 > > (gdb) p __static_tls_size > > $3 = 1712 > > > > > > With openblas (failure - three time o.k call from openblas, one failing call from timer_create()): > > > > Breakpoint 1, allocate_stack (attr=0x7fa286de48 <default_attr>, pdp=0x7febf5fd80, stack=0x7febf5fd88) > > at libpthread/nptl/allocatestack.c:472 > > 472 pd = get_cached_stack (&size, &mem); > > (gdb) p *attr > > $1 = {schedparam = {__sched_priority = 0}, schedpolicy = 0, flags = 0, guardsize = 1, stackaddr = 0x0, > > stacksize = 0, cpuset = 0x0, cpusetsize = 0} > > (gdb) p size > > $2 = 2097152 > > (gdb) p __static_tls_size > > $3 = 63152 > > > > [...] > > > > Thread 1 "test_timer_crea" hit Breakpoint 2, allocate_stack (attr=0x7febf61068, pdp=0x7febf60fc0, > > stack=0x7febf60fc8) at libpthread/nptl/allocatestack.c:468 > > 468 return EINVAL; > > (gdb) p *attr > > $4 = {schedparam = {__sched_priority = 0}, schedpolicy = 0, flags = 0, guardsize = 4096, > > stackaddr = 0x0, stacksize = 16384, cpuset = 0x0, cpusetsize = 0} > > (gdb) p size > > $5 = 16384 > > (gdb) p __static_tls_size > > $6 = 63152 > > > > > > - with openblas linking the thread-local-storage (__static_tls_size) is > > increased from 1712 to 63152 > > > > - the openblas threads are created with default pthread_attr_t/nullptr, > > the stack size size is calculated to 2097152 the check succeeds > > > > - the timer_create pthread_create is called with pthread_attr_t created > > via > > > > 66 (void) pthread_attr_init (&attr); > > 167 (void) pthread_attr_setstacksize (&attr, PTHREAD_STACK_MIN); > > > > so stacksize is given as 16384, the check against the increased thread-local- > > storage (63152 + guadsize + MINIMAL_REST_STACK + padding) fails... > > > > Maybe a uclibc bug/defect...., or an expert question: is the pthread_create/ > > pthread_attr_t stacksize inclusive or exclusive the reserved size for > > thread-local-storage usage (my judgment would be exclusive as the thread-local > > storage size is dependents on external parameters/library linking etc.)? > > I think uclibc-ng needs something as implemented for glibc (see [1]), using > > __static_tls_size + PTHREAD_STACK_MIN > > as stack size for timer_create thread (take the thread local storage stack > part into account)..., draft patch like the following works for the openblas/ > timer_create example: > > diff --git a/libpthread/nptl/init.c b/libpthread/nptl/init.c > index 4959d5ed8..fefdda1a5 100644 > --- a/libpthread/nptl/init.c > +++ b/libpthread/nptl/init.c > @@ -337,3 +337,9 @@ __pthread_initialize_minimal_internal (void) > } > strong_alias (__pthread_initialize_minimal_internal, > __pthread_initialize_minimal) > + > +size_t > +__pthread_get_minstack (const pthread_attr_t *attr) > +{ > + return __static_tls_size + PTHREAD_STACK_MIN; > +} > diff --git a/libpthread/nptl/pthreadP.h b/libpthread/nptl/pthreadP.h > index 13205512a..c686a4ca5 100644 > --- a/libpthread/nptl/pthreadP.h > +++ b/libpthread/nptl/pthreadP.h > @@ -377,6 +377,7 @@ weak_function; > > extern void __pthread_init_static_tls (struct link_map *) attribute_hidden; > > +extern size_t __pthread_get_minstack (const pthread_attr_t *attr); > > /* Namespace save aliases. */ > extern int __pthread_getschedparam (pthread_t thread_id, int *policy, > diff --git a/libpthread/nptl/sysdeps/unix/sysv/linux/timer_routines.c b/libpthread/nptl/sysdeps/unix/sysv/linux/timer_routines.c > index 514913317..60f2a724c 100644 > --- a/libpthread/nptl/sysdeps/unix/sysv/linux/timer_routines.c > +++ b/libpthread/nptl/sysdeps/unix/sysv/linux/timer_routines.c > @@ -164,7 +164,7 @@ __start_helper_thread (void) > and should go away automatically when canceled. */ > pthread_attr_t attr; > (void) pthread_attr_init (&attr); > - (void) pthread_attr_setstacksize (&attr, PTHREAD_STACK_MIN); > + (void) pthread_attr_setstacksize (&attr, __pthread_get_minstack (&attr)); > > /* Block all signals in the helper thread but SIGSETXID. To do this > thoroughly we temporarily have to block all signals here. The > Patch suggested upstream, see: https://mailman.uclibc-ng.org/pipermail/devel/2021-April/002064.html Regards, Peter > > Regards, > Peter > > [1] https://sourceware.org/git/?p=glibc.git;a=commitdiff;h=2c1094bd700e63a8d7f547b3f5495bedb55c0a08;hp=3b8dfc621bfd320c924a3cd597086d3473da1cf4 > > > > > > Is my change correct? > > > > If it works for you yes ;-), but does not fix the underlying/generally problem... > > > > Regards, > > Peter > > _______________________________________________ > > buildroot mailing list > > buildroot at busybox.net > > http://lists.busybox.net/mailman/listinfo/buildroot > > _______________________________________________ > buildroot mailing list > buildroot at busybox.net > http://lists.busybox.net/mailman/listinfo/buildroot ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2021-04-16 21:26 UTC | newest] Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2021-04-07 9:01 [Buildroot] issue with timer_create and openblas after update buildroot from 2019_02 to 2021_02 Arthur Lambert 2021-04-08 22:59 ` Peter Seiderer 2021-04-09 7:53 ` Arthur Lambert 2021-04-09 19:18 ` Peter Seiderer 2021-04-13 19:52 ` Peter Seiderer 2021-04-16 21:26 ` Peter Seiderer
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.