pthreads should rdepend on libgcc

pthreads should rdepend on libgcc

[Bartosz Golaszewski]

This isn't a patch as I'm not sure how to fix the right way it but it
seems to my that every program that calls pthread_exit() indirectly
rdepends on libgcc_s.so.1 being installed in the system (at least when
using glibc).

This is because the implementation of __pthread_exit() in glibc calls
pthread_cancel() which leads to pthread_cancel_init() being called in
which said shared object is being loaded with dlopen(). When this
fails, the program aborts with the following error message:

    libgcc_s.so.1 must be installed for pthread_cancel to work
    Aborted

A quick way of reproducing it is building a core-image-minimal (which
by default doesn't contain libgcc) with the core python3 package and
running the simple python HTTP server:

    python3 -m http.server 8000

Fetching any file will result in the above crash of the python process
when the thread serving the file exits.

I'm not sure if the same issue is present in musl - the implementation
of __pthread_exit() there is different.

Please advise on how to best approach this.

Thanks in advance,
Bartosz Golaszewski

Re: pthreads should rdepend on libgcc

[Mark Hatle]

On 11/13/19 2:34 AM, Bartosz Golaszewski wrote:
> This isn't a patch as I'm not sure how to fix the right way it but it
> seems to my that every program that calls pthread_exit() indirectly
> rdepends on libgcc_s.so.1 being installed in the system (at least when
> using glibc).
> 
> This is because the implementation of __pthread_exit() in glibc calls
> pthread_cancel() which leads to pthread_cancel_init() being called in
> which said shared object is being loaded with dlopen(). When this
> fails, the program aborts with the following error message:
> 
>     libgcc_s.so.1 must be installed for pthread_cancel to work
>     Aborted

Fix for this would be to declare a runtime-dependency on libgcc_s.so.1 in the
glibc libpthread package.

--Mark

> A quick way of reproducing it is building a core-image-minimal (which
> by default doesn't contain libgcc) with the core python3 package and
> running the simple python HTTP server:
> 
>     python3 -m http.server 8000
> 
> Fetching any file will result in the above crash of the python process
> when the thread serving the file exits.
> 
> I'm not sure if the same issue is present in musl - the implementation
> of __pthread_exit() there is different.
> 
> Please advise on how to best approach this.
> 
> Thanks in advance,
> Bartosz Golaszewski
> 

Re: pthreads should rdepend on libgcc

[Ross Burton]

On 13/11/2019 15:04, Mark Hatle wrote:
>> This is because the implementation of __pthread_exit() in glibc calls
>> pthread_cancel() which leads to pthread_cancel_init() being called in
>> which said shared object is being loaded with dlopen(). When this
>> fails, the program aborts with the following error message:
>>
>>      libgcc_s.so.1 must be installed for pthread_cancel to work
>>      Aborted
> 
> Fix for this would be to declare a runtime-dependency on libgcc_s.so.1 in the
> glibc libpthread package.

On my todo list for a long time has been:

1) a QA test to find binaries that link to pthread_cancel but don't 
depend on libgcc_s
2) Automatically finding use of pthread_cancel and adding the RDEPENDS 
automatically.

(2) is definitely the best long term solution, but as Mark says: the now 
fix is to add libgcc to RDEPENDS.  There's plenty of prior art in oe-core:

$ git grep -h RDEPENDS.*libgcc
RDEPENDS_${PN}-ptest += "make ${@bb.utils.contains('PACKAGECONFIG', 
'python', 'libgcc python3-core python3-logging python3-shell 
python3-stringold python3-threading python3-unittest ${PN}-python', '', d)}"
RDEPENDS_${PN} = "libgcc"
RDEPENDS_${PN}-ptest = "${PN}-modules ${PN}-tests unzip bzip2 libgcc 
tzdata-europe coreutils sed"
RDEPENDS_${PN}-ptest += "libgcc"
RDEPENDS_${PN} = "libgcc"
RDEPENDS_${PN}-ptest += "libgcc"
RDEPENDS_${PN}-ptest += "libgcc"

Ross

Re: pthreads should rdepend on libgcc

[Khem Raj]

On Wed, Nov 13, 2019 at 7:50 AM Ross Burton <ross.burton@intel.com> wrote:
>
> On 13/11/2019 15:04, Mark Hatle wrote:
> >> This is because the implementation of __pthread_exit() in glibc calls
> >> pthread_cancel() which leads to pthread_cancel_init() being called in
> >> which said shared object is being loaded with dlopen(). When this
> >> fails, the program aborts with the following error message:
> >>
> >>      libgcc_s.so.1 must be installed for pthread_cancel to work
> >>      Aborted
> >
> > Fix for this would be to declare a runtime-dependency on libgcc_s.so.1 in the
> > glibc libpthread package.
>
> On my todo list for a long time has been:
>
> 1) a QA test to find binaries that link to pthread_cancel but don't
> depend on libgcc_s
> 2) Automatically finding use of pthread_cancel and adding the RDEPENDS
> automatically.
>
> (2) is definitely the best long term solution, but as Mark says: the now
> fix is to add libgcc to RDEPENDS.  There's plenty of prior art in oe-core:
>
> $ git grep -h RDEPENDS.*libgcc
> RDEPENDS_${PN}-ptest += "make ${@bb.utils.contains('PACKAGECONFIG',
> 'python', 'libgcc python3-core python3-logging python3-shell
> python3-stringold python3-threading python3-unittest ${PN}-python', '', d)}"
> RDEPENDS_${PN} = "libgcc"
> RDEPENDS_${PN}-ptest = "${PN}-modules ${PN}-tests unzip bzip2 libgcc
> tzdata-europe coreutils sed"
> RDEPENDS_${PN}-ptest += "libgcc"
> RDEPENDS_${PN} = "libgcc"
> RDEPENDS_${PN}-ptest += "libgcc"
> RDEPENDS_${PN}-ptest += "libgcc"
>

There are two issues here, one is where dependency is missing which
this solution will address
however, there could be another problem, where libgcc_s is lazily
loaded during pthread_exit and
application is not using root user which many daemons do where they
use less privileged user to run
and it would fail in the exact same manner. I also see few other uses
of libgcc_s.so being dlopened in
nptl unwind code and libbacktrace in glibc. so even if you were to do
one of above you won't solve the
problem. in fact the part this will solve can simply be solved by
adding libgcc_s as rdep for libc6
and then if we still get failures of such kind then we know its the
second problem since we would have
taken care of the first issue with rdep. Its fine if we want to be
surgical, but it might be over-optimization

Re: pthreads should rdepend on libgcc

[Bartosz Golaszewski]

śr., 13 lis 2019 o 23:37 Khem Raj <raj.khem@gmail.com> napisał(a):
>
> On Wed, Nov 13, 2019 at 7:50 AM Ross Burton <ross.burton@intel.com> wrote:
> >
> > On 13/11/2019 15:04, Mark Hatle wrote:
> > >> This is because the implementation of __pthread_exit() in glibc calls
> > >> pthread_cancel() which leads to pthread_cancel_init() being called in
> > >> which said shared object is being loaded with dlopen(). When this
> > >> fails, the program aborts with the following error message:
> > >>
> > >>      libgcc_s.so.1 must be installed for pthread_cancel to work
> > >>      Aborted
> > >
> > > Fix for this would be to declare a runtime-dependency on libgcc_s.so.1 in the
> > > glibc libpthread package.
> >
> > On my todo list for a long time has been:
> >
> > 1) a QA test to find binaries that link to pthread_cancel but don't
> > depend on libgcc_s
> > 2) Automatically finding use of pthread_cancel and adding the RDEPENDS
> > automatically.
> >
> > (2) is definitely the best long term solution, but as Mark says: the now
> > fix is to add libgcc to RDEPENDS.  There's plenty of prior art in oe-core:
> >
> > $ git grep -h RDEPENDS.*libgcc
> > RDEPENDS_${PN}-ptest += "make ${@bb.utils.contains('PACKAGECONFIG',
> > 'python', 'libgcc python3-core python3-logging python3-shell
> > python3-stringold python3-threading python3-unittest ${PN}-python', '', d)}"
> > RDEPENDS_${PN} = "libgcc"
> > RDEPENDS_${PN}-ptest = "${PN}-modules ${PN}-tests unzip bzip2 libgcc
> > tzdata-europe coreutils sed"
> > RDEPENDS_${PN}-ptest += "libgcc"
> > RDEPENDS_${PN} = "libgcc"
> > RDEPENDS_${PN}-ptest += "libgcc"
> > RDEPENDS_${PN}-ptest += "libgcc"
> >
>
> There are two issues here, one is where dependency is missing which
> this solution will address
> however, there could be another problem, where libgcc_s is lazily
> loaded during pthread_exit and
> application is not using root user which many daemons do where they
> use less privileged user to run
> and it would fail in the exact same manner. I also see few other uses
> of libgcc_s.so being dlopened in
> nptl unwind code and libbacktrace in glibc. so even if you were to do
> one of above you won't solve the
> problem. in fact the part this will solve can simply be solved by
> adding libgcc_s as rdep for libc6
> and then if we still get failures of such kind then we know its the
> second problem since we would have
> taken care of the first issue with rdep. Its fine if we want to be
> surgical, but it might be over-optimization

So can I submit such a patch then?

Bart

> --
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: pthreads should rdepend on libgcc

[Adrian Bunk]

On Wed, Nov 13, 2019 at 02:36:56PM -0800, Khem Raj wrote:
>...
> however, there could be another problem, where libgcc_s is lazily
> loaded during pthread_exit and
> application is not using root user which many daemons do where they
> use less privileged user to run
> and it would fail in the exact same manner. I also see few other uses
> of libgcc_s.so being dlopened in
> nptl unwind code and libbacktrace in glibc.
>...

Why would this be a problem?

dlopen() does not require root privileges.

cu
Adrian

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Re: pthreads should rdepend on libgcc

[Bartosz Golaszewski]

śr., 13 lis 2019 o 16:04 Mark Hatle <mark.hatle@kernel.crashing.org> napisał(a):
>
>
>
> On 11/13/19 2:34 AM, Bartosz Golaszewski wrote:
> > This isn't a patch as I'm not sure how to fix the right way it but it
> > seems to my that every program that calls pthread_exit() indirectly
> > rdepends on libgcc_s.so.1 being installed in the system (at least when
> > using glibc).
> >
> > This is because the implementation of __pthread_exit() in glibc calls
> > pthread_cancel() which leads to pthread_cancel_init() being called in
> > which said shared object is being loaded with dlopen(). When this
> > fails, the program aborts with the following error message:
> >
> >     libgcc_s.so.1 must be installed for pthread_cancel to work
> >     Aborted
>
> Fix for this would be to declare a runtime-dependency on libgcc_s.so.1 in the
> glibc libpthread package.
>

There is no real libpthread package. There's libpthread-stubs but it
only contains pkgconfig files. The actual pthread shared library is in
libc6 package and its inclusion doesn't seem to be conditional. In
that case - isn't it better to just make libgcc_s.so.1 part of libc6?

Bart

> --Mark
>
> > A quick way of reproducing it is building a core-image-minimal (which
> > by default doesn't contain libgcc) with the core python3 package and
> > running the simple python HTTP server:
> >
> >     python3 -m http.server 8000
> >
> > Fetching any file will result in the above crash of the python process
> > when the thread serving the file exits.
> >
> > I'm not sure if the same issue is present in musl - the implementation
> > of __pthread_exit() there is different.
> >
> > Please advise on how to best approach this.
> >
> > Thanks in advance,
> > Bartosz Golaszewski
> >

Re: pthreads should rdepend on libgcc

[Richard Purdie]

On Wed, 2019-11-13 at 22:03 +0100, Bartosz Golaszewski wrote:
> śr., 13 lis 2019 o 16:04 Mark Hatle <mark.hatle@kernel.crashing.org>
> napisał(a):
> > On 11/13/19 2:34 AM, Bartosz Golaszewski wrote:
> > > This isn't a patch as I'm not sure how to fix the right way it
> > > but it
> > > seems to my that every program that calls pthread_exit()
> > > indirectly
> > > rdepends on libgcc_s.so.1 being installed in the system (at least
> > > when
> > > using glibc).
> > > 
> > > This is because the implementation of __pthread_exit() in glibc
> > > calls
> > > pthread_cancel() which leads to pthread_cancel_init() being
> > > called in
> > > which said shared object is being loaded with dlopen(). When this
> > > fails, the program aborts with the following error message:
> > > 
> > >     libgcc_s.so.1 must be installed for pthread_cancel to work
> > >     Aborted
> > 
> > Fix for this would be to declare a runtime-dependency on
> > libgcc_s.so.1 in the
> > glibc libpthread package.
> > 
> 
> There is no real libpthread package. There's libpthread-stubs but it
> only contains pkgconfig files. The actual pthread shared library is
> in
> libc6 package and its inclusion doesn't seem to be conditional. In
> that case - isn't it better to just make libgcc_s.so.1 part of libc6?

There have been strong objections to that in the past which brings us
back to Ross' proposal...

Cheers,

Richard

Re: pthreads should rdepend on libgcc

[Khem Raj]

On Wed, Nov 13, 2019 at 12:34 AM Bartosz Golaszewski <brgl@bgdev.pl> wrote:
>
> This isn't a patch as I'm not sure how to fix the right way it but it
> seems to my that every program that calls pthread_exit() indirectly
> rdepends on libgcc_s.so.1 being installed in the system (at least when
> using glibc).
>
> This is because the implementation of __pthread_exit() in glibc calls
> pthread_cancel() which leads to pthread_cancel_init() being called in
> which said shared object is being loaded with dlopen(). When this
> fails, the program aborts with the following error message:
>
>     libgcc_s.so.1 must be installed for pthread_cancel to work
>     Aborted
>
> A quick way of reproducing it is building a core-image-minimal (which
> by default doesn't contain libgcc) with the core python3 package and
> running the simple python HTTP server:
>
>     python3 -m http.server 8000
>
> Fetching any file will result in the above crash of the python process
> when the thread serving the file exits.
>
> I'm not sure if the same issue is present in musl - the implementation
> of __pthread_exit() there is different.
>

musl will not have this issue since it does not depend on libgcc
unwinder implicitly.

> Please advise on how to best approach this.
>

use musl ;)

> Thanks in advance,
> Bartosz Golaszewski
> --
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: pthreads should rdepend on libgcc

[Bartosz Golaszewski]

śr., 13 lis 2019 o 23:39 Khem Raj <raj.khem@gmail.com> napisał(a):
>
> On Wed, Nov 13, 2019 at 12:34 AM Bartosz Golaszewski <brgl@bgdev.pl> wrote:
> >
> > This isn't a patch as I'm not sure how to fix the right way it but it
> > seems to my that every program that calls pthread_exit() indirectly
> > rdepends on libgcc_s.so.1 being installed in the system (at least when
> > using glibc).
> >
> > This is because the implementation of __pthread_exit() in glibc calls
> > pthread_cancel() which leads to pthread_cancel_init() being called in
> > which said shared object is being loaded with dlopen(). When this
> > fails, the program aborts with the following error message:
> >
> >     libgcc_s.so.1 must be installed for pthread_cancel to work
> >     Aborted
> >
> > A quick way of reproducing it is building a core-image-minimal (which
> > by default doesn't contain libgcc) with the core python3 package and
> > running the simple python HTTP server:
> >
> >     python3 -m http.server 8000
> >
> > Fetching any file will result in the above crash of the python process
> > when the thread serving the file exits.
> >
> > I'm not sure if the same issue is present in musl - the implementation
> > of __pthread_exit() there is different.
> >
>
> musl will not have this issue since it does not depend on libgcc
> unwinder implicitly.
>
> > Please advise on how to best approach this.
> >
>
> use musl ;)

I'd love to, but I'm stuck with systemd. :)

Bart

>
> > Thanks in advance,
> > Bartosz Golaszewski
> > --
> > _______________________________________________
> > Openembedded-core mailing list
> > Openembedded-core@lists.openembedded.org
> > http://lists.openembedded.org/mailman/listinfo/openembedded-core