From: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
To: "H. Peter Anvin" <hpa@zytor.com>
Cc: Thomas Gleixner <tglx@linutronix.de>,
Peter Zijlstra <peterz@infradead.org>,
Andrew Morton <akpm@linux-foundation.org>,
Russell King <linux@arm.linux.org.uk>,
Ingo Molnar <mingo@redhat.com>,
linux-kernel@vger.kernel.org,
linux-api <linux-api@vger.kernel.org>,
Paul Turner <pjt@google.com>, Andrew Hunter <ahh@google.com>,
Andy Lutomirski <luto@amacapital.net>,
Andi Kleen <andi@firstfloor.org>,
Dave Watson <davejwatson@fb.com>, Chris Lameter <cl@linux.com>,
Ben Maurer <bmaurer@fb.com>, rostedt <rostedt@goodmis.org>,
"Paul E. McKenney" <paulmck@linux.vnet.ibm.com>,
Josh Triplett <josh@joshtriplett.org>,
Catalin Marinas <catalin.marinas@arm.com>,
Will Deacon <will.deacon@arm.com>,
Michael Kerrisk <mtk.manpages@gmail.com>,
Linus Torvalds <torvalds@linux-foundation.org>
Subject: Re: [PATCH v4 1/5] getcpu_cache system call: cache CPU number of running thread
Date: Sat, 27 Feb 2016 00:40:30 +0000 (UTC) [thread overview]
Message-ID: <1150363257.9781.1456533630895.JavaMail.zimbra@efficios.com> (raw)
In-Reply-To: <7096DA23-3908-40DC-A46B-C4CF2252CEE8@zytor.com>
----- On Feb 26, 2016, at 6:04 PM, H. Peter Anvin hpa@zytor.com wrote:
> On February 26, 2016 12:24:15 PM PST, Mathieu Desnoyers
> <mathieu.desnoyers@efficios.com> wrote:
>>----- On Feb 26, 2016, at 1:01 PM, Thomas Gleixner tglx@linutronix.de
>>wrote:
>>
>>> On Fri, 26 Feb 2016, Mathieu Desnoyers wrote:
>>>> ----- On Feb 26, 2016, at 11:29 AM, Thomas Gleixner
>>tglx@linutronix.de wrote:
>>>> > Right. There is no point in having two calls and two update
>>mechanisms for a
>>>> > very similar purpose.
>>>> >
>>>> > So let userspace have one struct where cpu/seq and whatever is
>>required for
>>>> > rseq is located and flag at register time which parts of the
>>struct need to be
>>>> > updated.
>>>>
>>>> If we put both cpu/seq/other in that structure, why not plan ahead
>>and make
>>>> it extensible then ?
>>>>
>>>> That looks very much like the "Thread-local ABI" series I posted
>>last year.
>>>> See https://lkml.org/lkml/2015/12/22/464
>>>>
>>>> Here is why I ended up introducing the specialized "getcpu_cache"
>>system call
>>>> rather than the "generic" system call (quote from the getcpu_cache
>>changelog):
>>>>
>>>> Rationale for the getcpu_cache system call rather than the
>>thread-local
>>>> ABI system call proposed earlier:
>>>>
>>>> Rather than doing a "generic" thread-local ABI, specialize this
>>system
>>>> call for a cpu number cache only. Anyway, the thread-local ABI
>>approach
>>>> would have required that we introduce "feature" flags, which
>>would have
>>>> ended up reimplementing multiplexing of features on top of a
>>system
>>>> call. It seems better to introduce one system call per feature
>>instead.
>>>>
>>>> If everyone end up preferring that we introduce a system call that
>>implements
>>>> many features at once, that's indeed something we can do, but I
>>remember
>>>> being told in the past that this is generally a bad idea.
>>>
>>> It's a bad idea if you mix stuff which does not belong together, but
>>if you
>>> have stuff which shares a substantial amount of things then it makes
>>a lot of
>>> sense. Especially if it adds similar stuff into hotpathes.
>>>
>>>> For one thing, it would make the interface more cumbersome to deal
>>with
>>>> from user-space in terms of feature detection: if we want to make
>>this
>>>> interface extensible, in addition to check -1, errno=ENOSYS,
>>userspace
>>>> would have to deal with a field containing the length of the
>>structure
>>>> as expected by user-space and kernel, and feature flags to see the
>>common
>>>> set of features supported by kernel and user-space.
>>>>
>>>> Having one system call per feature seems simpler to handle in terms
>>of
>>>> feature availability detection from a userspace point of view.
>>>
>>> That might well be, but that does not justify two fastpath updates,
>>two
>>> seperate pointers to handle, etc ....
>>
>>Keeping two separate pointers in the task_struct rather than a single
>>one
>>might indeed be unwelcome, but I'm not sure I fully grasp the fast path
>>argument in this case: getcpu_cache only sets a notifier thread flag
>>on thread migration, whereas AFAIU rseq adds code to context switch and
>>signal
>>delivery, which are prone to have a higher impact.
>>
>>Indeed both will have their own code in the resume notifier, but is it
>>really
>>a fast path ?
>>
>>From my point of view, making it easy for userspace to just enable
>>getcpu_cache
>>without having the scheduler and signal delivery fast-path overhead of
>>rseq seems
>>like a good thing. I'm not all that sure that saving an extra pointer
>>in
>>task_struct justifies the added system call interface complexity.
>>
>>Thanks,
>>
>>Mathieu
>
> I think it would be a good idea to make this a general pointer for the kernel to
> be able to write per thread state to user space, which obviously can't be done
> with the vDSO.
>
> This means the libc per thread startup should query the kernel for the size of
> this structure and allocate thread local data accordingly. We can then grow
> this structure if needed without making the ABI even more complex.
>
> This is more than a system call: this is an entirely new way for userspace to
> interact with the kernel. Therefore we should make it a general facility.
I'm really glad to see I'm not the only one seeing potential for
genericity here. :-) This is exactly what I had in mind
last year when proposing the thread_local_abi() system call:
a generic way to register an extensible per-thread data structure
so the kernel can communicate with user-space and vice-versa.
Rather than having the libc query the kernel for size of the structure,
I would recommend that libc tells the kernel the size of the thread-local
ABI structure it supports. The idea here is that both the kernel and libc
need to know about the fields in that structure to allow a two-way
interaction. Fields known only by either the kernel or userspace
are useless for a given thread anyway. This way, libc could statically
define the structure.
I would be tempted to also add "features" flags, so both user-space
and the kernel could tell each other what they support: user-space
would announce the set of features it supports, and it could also
query the kernel for the set of supported features. One simple approach
would be to use a uint64_t as type for those feature flags, and
reserve the last bit for extending to future flags if we ever have
more than 64.
Thoughts ?
Thanks,
Mathieu
--
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com
next prev parent reply other threads:[~2016-02-27 0:40 UTC|newest]
Thread overview: 55+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-02-23 23:28 [PATCH v4 0/5] getcpu_cache system call for 4.6 Mathieu Desnoyers
2016-02-23 23:28 ` [PATCH v4 1/5] getcpu_cache system call: cache CPU number of running thread Mathieu Desnoyers
2016-02-24 11:11 ` Thomas Gleixner
2016-02-24 17:17 ` Mathieu Desnoyers
2016-02-25 23:32 ` Rasmus Villemoes
2016-02-26 17:47 ` Mathieu Desnoyers
2016-02-25 9:56 ` Peter Zijlstra
2016-02-25 16:55 ` Mathieu Desnoyers
2016-02-25 17:04 ` Peter Zijlstra
2016-02-25 17:17 ` Mathieu Desnoyers
2016-02-26 11:33 ` Peter Zijlstra
2016-02-26 16:29 ` Thomas Gleixner
2016-02-26 17:20 ` Mathieu Desnoyers
2016-02-26 18:01 ` Thomas Gleixner
2016-02-26 20:24 ` Mathieu Desnoyers
2016-02-26 23:04 ` H. Peter Anvin
2016-02-27 0:40 ` Mathieu Desnoyers [this message]
2016-02-27 6:24 ` H. Peter Anvin
2016-02-27 14:15 ` Mathieu Desnoyers
2016-02-27 14:58 ` Peter Zijlstra
2016-02-27 18:35 ` Linus Torvalds
2016-02-27 19:01 ` H. Peter Anvin
2016-02-27 23:53 ` Mathieu Desnoyers
[not found] ` <CA+55aFwcgwRxvVBz5kk_3O8dESXAGJ4KHBkf=pSXjiS7Xh4NwA@mail.gmail.com>
[not found] ` <1082926946.10326.1456619994590.JavaMail.zimbra@efficios.com>
2016-02-28 0:57 ` Linus Torvalds
2016-02-28 14:32 ` Mathieu Desnoyers
2016-02-29 10:35 ` Peter Zijlstra
2016-03-01 20:23 ` Mathieu Desnoyers
2016-03-01 21:32 ` Peter Zijlstra
2016-03-01 21:36 ` Peter Zijlstra
2016-03-01 21:47 ` H. Peter Anvin
2016-03-02 10:34 ` Peter Zijlstra
2016-02-29 10:32 ` Peter Zijlstra
2016-02-29 10:39 ` Arnd Bergmann
2016-02-29 12:41 ` Mathieu Desnoyers
2016-02-29 13:08 ` Arnd Bergmann
2016-02-29 18:19 ` H. Peter Anvin
2016-03-02 10:44 ` Geert Uytterhoeven
2016-03-01 18:25 ` H. Peter Anvin
2016-03-01 18:40 ` Mathieu Desnoyers
2016-02-28 13:07 ` Geert Uytterhoeven
2016-02-28 16:21 ` Linus Torvalds
2016-02-29 10:01 ` Peter Zijlstra
2016-02-27 15:04 ` H. Peter Anvin
2016-02-23 23:28 ` [PATCH v4 2/5] getcpu_cache: ARM resume notifier Mathieu Desnoyers
2016-02-23 23:28 ` [PATCH v4 3/5] getcpu_cache: wire up ARM system call Mathieu Desnoyers
2016-02-24 0:54 ` kbuild test robot
2016-02-24 1:05 ` [PATCH v4 (updated)] " Mathieu Desnoyers
2016-02-24 5:28 ` kbuild test robot
2016-02-24 6:54 ` kbuild test robot
2016-02-23 23:28 ` [PATCH v4 4/5] getcpu_cache: x86 32/64 resume notifier Mathieu Desnoyers
2016-02-23 23:28 ` [PATCH v4 5/5] getcpu_cache: wire up x86 32/64 system call Mathieu Desnoyers
2016-02-24 1:36 ` [PATCH v4 0/5] getcpu_cache system call for 4.6 H. Peter Anvin
2016-02-24 4:09 ` Mathieu Desnoyers
2016-02-24 20:07 ` H. Peter Anvin
2016-02-24 22:38 ` Mathieu Desnoyers
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1150363257.9781.1456533630895.JavaMail.zimbra@efficios.com \
--to=mathieu.desnoyers@efficios.com \
--cc=ahh@google.com \
--cc=akpm@linux-foundation.org \
--cc=andi@firstfloor.org \
--cc=bmaurer@fb.com \
--cc=catalin.marinas@arm.com \
--cc=cl@linux.com \
--cc=davejwatson@fb.com \
--cc=hpa@zytor.com \
--cc=josh@joshtriplett.org \
--cc=linux-api@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux@arm.linux.org.uk \
--cc=luto@amacapital.net \
--cc=mingo@redhat.com \
--cc=mtk.manpages@gmail.com \
--cc=paulmck@linux.vnet.ibm.com \
--cc=peterz@infradead.org \
--cc=pjt@google.com \
--cc=rostedt@goodmis.org \
--cc=tglx@linutronix.de \
--cc=torvalds@linux-foundation.org \
--cc=will.deacon@arm.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).