Re: Rseq registration: Google tcmalloc vs glibc

From: Joel Fernandes <joel@joelfernandes.org>
To: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Cc: Chris Kennelly <ckennelly@google.com>,
	Paul Turner <pjt@google.com>, Florian Weimer <fweimer@redhat.com>,
	"Carlos O'Donell" <codonell@redhat.com>,
	libc-alpha <libc-alpha@sourceware.org>,
	linux-kernel <linux-kernel@vger.kernel.org>,
	Peter Zijlstra <peterz@infradead.org>,
	paulmck <paulmck@kernel.org>, Boqun Feng <boqun.feng@gmail.com>,
	Brian Geffon <bgeffon@google.com>
Subject: Re: Rseq registration: Google tcmalloc vs glibc
Date: Tue, 25 Feb 2020 22:24:48 -0500	[thread overview]
Message-ID: <CAEXW_YRT7AjaJs7mPyNd=J6fhBicYwGbQMK2Senwm3cBhFvWPw@mail.gmail.com> (raw)
In-Reply-To: <1683022606.3452.1582301632640.JavaMail.zimbra@efficios.com>

On Fri, Feb 21, 2020 at 11:13 AM Mathieu Desnoyers
<mathieu.desnoyers@efficios.com> wrote:
>
> ----- On Feb 21, 2020, at 10:49 AM, Joel Fernandes, Google joel@joelfernandes.org wrote:
>
> [...]
> >>
> >> 3) Use the  __rseq_abi TLS cpu_id field to know whether Rseq has been
> >> registered.
> >>
> >> - Current protocol in the most recent glibc integration patch set.
> >> - Not supported yet by Linux kernel rseq selftests,
> >> - Not supported yet by tcmalloc,
> >>
> >> Use the per-thread state to figure out whether each thread need to register
> >> Rseq individually.
> >>
> >> Works for integration between a library which exists for the entire lifetime
> >> of the executable (e.g. glibc) and other libraries. However, it does not
> >> allow a set of libraries which are dlopen'd/dlclose'd to co-exist without
> >> having a library like glibc handling the registration present.
> >
> > Mathieu, could you share more details about why during dlopen/close
> > libraries we cannot use the same __rseq_abi TLS to detect that rseq was
> > registered?
>
> Sure,
>
> A library which is only loaded and never closed during the execution of the
> program can let the kernel implicitly unregister rseq at thread exit. For
> the dlopen/dlclose use-case, we need to be able to explicitly unregister
> each thread's __rseq_abi which sit in a library which is going to be
> dlclose'd.

Mathieu, Thanks a lot for the explanation, it makes complete sense. It
sounds from Chris's reply that tcmalloc already checks
__rseq_abi.cpu_id and is not dlopened/closed. Considering these, it
seems to already handle things properly - CMIIW.

Chris, Brian, is there any other concern to upgrading of tcmalloc
version in ChromeOS? I believe there was some concern about detection
of rseq kernel support. A quick look at tcmalloc shows it does not do
such detection, but I can stand corrected. One more thing, currently
tcmalloc does not use rseq on ARM. If I recall, ARM does have rseq
support as well. So we ought to enable it for that arch as well if
possible. Why not enable it on all arches and then dynamically detect
at runtime if needed support is available?

Thanks,
Joel