* ucontext, kernel vs. userspace (glibc)
@ 2021-08-30 10:40 Benjamin Herrenschmidt
2021-08-31 17:44 ` Catalin Marinas
2021-09-02 12:42 ` Mark Brown
0 siblings, 2 replies; 6+ messages in thread
From: Benjamin Herrenschmidt @ 2021-08-30 10:40 UTC (permalink / raw)
To: linux-arm-kernel; +Cc: Catalin Marinas, Will Deacon, Dave Martin, benh
Hi Folks !
So I'm discovering arm64 intricacies and today, as I was looking at SVE
support (in the context of distro glibc backports.. don't ask), I
noticed that glibc has no provision for dealing with kernel generated
ucontext's in its {get,set,swap}_context functions...
(It says so explicitly in the code unless I misunderstood).
So one thing we did to "solve" this on ppc64 a while ago was to create
a swapcontext syscall which can operate as all 3 operations (you can
have NULL arguments), which also handles the sigprocmask (bonus:
atomically with the context get/set from a userspace perspective).
Would it make sense to do something similar on aarch64 ? (And have
glibc then exploit it).
The hard-to-solve thing is the case where the SVE context spills
outside of the ucontext itself, in the extra room on the stack, since
programs that "now" about ucontext will not have allocated space for
that, so that's more/less a lost cause already.
Cheers,
Ben.
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 6+ messages in thread* Re: ucontext, kernel vs. userspace (glibc) 2021-08-30 10:40 ucontext, kernel vs. userspace (glibc) Benjamin Herrenschmidt @ 2021-08-31 17:44 ` Catalin Marinas 2021-09-02 12:42 ` Mark Brown 1 sibling, 0 replies; 6+ messages in thread From: Catalin Marinas @ 2021-08-31 17:44 UTC (permalink / raw) To: Benjamin Herrenschmidt Cc: linux-arm-kernel, Will Deacon, benh, Szabolcs Nagy, Mark Brown Hi Ben, On Mon, Aug 30, 2021 at 08:40:03PM +1000, Benjamin Herrenschmidt wrote: > So I'm discovering arm64 intricacies and today, as I was looking at SVE > support (in the context of distro glibc backports.. don't ask), I > noticed that glibc has no provision for dealing with kernel generated > ucontext's in its {get,set,swap}_context functions... > > (It says so explicitly in the code unless I misunderstood). > > So one thing we did to "solve" this on ppc64 a while ago was to create > a swapcontext syscall which can operate as all 3 operations (you can > have NULL arguments), which also handles the sigprocmask (bonus: > atomically with the context get/set from a userspace perspective). > > Would it make sense to do something similar on aarch64 ? (And have > glibc then exploit it). > > The hard-to-solve thing is the case where the SVE context spills > outside of the ucontext itself, in the extra room on the stack, since > programs that "now" about ucontext will not have allocated space for > that, so that's more/less a lost cause already. I haven't fully parsed your email yet but adding a Mark B for SVE and Szabolcs for glibc (and removing Dave who left Arm recently). -- Catalin _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: ucontext, kernel vs. userspace (glibc) 2021-08-30 10:40 ucontext, kernel vs. userspace (glibc) Benjamin Herrenschmidt 2021-08-31 17:44 ` Catalin Marinas @ 2021-09-02 12:42 ` Mark Brown 2021-09-03 7:14 ` Benjamin Herrenschmidt 1 sibling, 1 reply; 6+ messages in thread From: Mark Brown @ 2021-09-02 12:42 UTC (permalink / raw) To: Benjamin Herrenschmidt Cc: linux-arm-kernel, Catalin Marinas, Will Deacon, benh, Szabolcs Nagy [-- Attachment #1.1: Type: text/plain, Size: 1448 bytes --] On Mon, Aug 30, 2021 at 08:40:03PM +1000, Benjamin Herrenschmidt wrote: > So I'm discovering arm64 intricacies and today, as I was looking at SVE > support (in the context of distro glibc backports.. don't ask), I > noticed that glibc has no provision for dealing with kernel generated > ucontext's in its {get,set,swap}_context functions... > (It says so explicitly in the code unless I misunderstood). > So one thing we did to "solve" this on ppc64 a while ago was to create > a swapcontext syscall which can operate as all 3 operations (you can > have NULL arguments), which also handles the sigprocmask (bonus: > atomically with the context get/set from a userspace perspective). > Would it make sense to do something similar on aarch64 ? (And have > glibc then exploit it). I think the usefulness of such an interface is mainly a question for userspace - I don't immediately see any issue with implementing it if it's useful to people. > The hard-to-solve thing is the case where the SVE context spills > outside of the ucontext itself, in the extra room on the stack, since > programs that "now" about ucontext will not have allocated space for > that, so that's more/less a lost cause already. You can figure out the maximum possible size for a context so it would be possible to define a mechanism for pointing to extra data I guess but yeah, it's going to be a problem when we start seeing systems with large enough register state. [-- Attachment #1.2: signature.asc --] [-- Type: application/pgp-signature, Size: 488 bytes --] [-- Attachment #2: Type: text/plain, Size: 176 bytes --] _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: ucontext, kernel vs. userspace (glibc) 2021-09-02 12:42 ` Mark Brown @ 2021-09-03 7:14 ` Benjamin Herrenschmidt 2021-09-03 11:02 ` Szabolcs Nagy 2021-09-03 12:25 ` Mark Brown 0 siblings, 2 replies; 6+ messages in thread From: Benjamin Herrenschmidt @ 2021-09-03 7:14 UTC (permalink / raw) To: Mark Brown Cc: linux-arm-kernel, Catalin Marinas, Will Deacon, Szabolcs Nagy, benh On Thu, 2021-09-02 at 13:42 +0100, Mark Brown wrote: > On Mon, Aug 30, 2021 at 08:40:03PM +1000, Benjamin Herrenschmidt wrote: > > > So I'm discovering arm64 intricacies and today, as I was looking at SVE > > support (in the context of distro glibc backports.. don't ask), I > > noticed that glibc has no provision for dealing with kernel generated > > ucontext's in its {get,set,swap}_context functions... > > (It says so explicitly in the code unless I misunderstood). > > So one thing we did to "solve" this on ppc64 a while ago was to create > > a swapcontext syscall which can operate as all 3 operations (you can > > have NULL arguments), which also handles the sigprocmask (bonus: > > atomically with the context get/set from a userspace perspective). > > Would it make sense to do something similar on aarch64 ? (And have > > glibc then exploit it). > > I think the usefulness of such an interface is mainly a question for > userspace - I don't immediately see any issue with implementing it if > it's useful to people. Well, the problem as far as I can tell is that the glibc implementation of these today. They support "FPSIMD" but that's about it (so no SVE or anything else) along with a comment: /* Check for FP SIMD context. We don't support restoring contexts created by the kernel, so this context must have been created by getcontext. Hence we can rely on the first extension block being the FP SIMD context. */ That said, a bit of reading around seems to indicate that the expecation of being able to setcontext() back to a signal handler generated context has been deprecated by the standard and broken on x86 for a while in Linux, so I suppose that is less of an issue. That said, there is still some advantage in letting the kernel implement these as it would allow the kernel to support various "extensions" such as SVE (as long as there is room) transparently without having to change glibc. In fact, isn't it possible for glibc to define its own ucontext structure for applications to use that can potentially have a larger reserved area ? By passing that size to the syscall, you can essentially get userspace ready for future extensions... within limits. > > The hard-to-solve thing is the case where the SVE context spills > > outside of the ucontext itself, in the extra room on the stack, since > > programs that "now" about ucontext will not have allocated space for > > that, so that's more/less a lost cause already. > > You can figure out the maximum possible size for a context so it would > be possible to define a mechanism for pointing to extra data I guess but > yeah, it's going to be a problem when we start seeing systems with large > enough register state. Extra data for userspace generated ucontext's isn't going to fly much, there's really no "place" to put it (those things can be part of structures etc...) and no "hook" to allocate/free sub structures. So you need whatever struct ucontext is used in userspace to be big enough. That said, I think the current one might be enough for sve512 (I need to check) and we could have glibc define something much bigger (16KB ?) without much damage I suspect. Nagyu ? What do you think ? Cheers, Ben. _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: ucontext, kernel vs. userspace (glibc) 2021-09-03 7:14 ` Benjamin Herrenschmidt @ 2021-09-03 11:02 ` Szabolcs Nagy 2021-09-03 12:25 ` Mark Brown 1 sibling, 0 replies; 6+ messages in thread From: Szabolcs Nagy @ 2021-09-03 11:02 UTC (permalink / raw) To: Benjamin Herrenschmidt Cc: Mark Brown, linux-arm-kernel, Catalin Marinas, Will Deacon, benh The 09/03/2021 17:14, Benjamin Herrenschmidt wrote: > On Thu, 2021-09-02 at 13:42 +0100, Mark Brown wrote: > > On Mon, Aug 30, 2021 at 08:40:03PM +1000, Benjamin Herrenschmidt wrote: > > > > > So I'm discovering arm64 intricacies and today, as I was looking at SVE > > > support (in the context of distro glibc backports.. don't ask), I > > > noticed that glibc has no provision for dealing with kernel generated > > > ucontext's in its {get,set,swap}_context functions... > > > (It says so explicitly in the code unless I misunderstood). > > > So one thing we did to "solve" this on ppc64 a while ago was to create > > > a swapcontext syscall which can operate as all 3 operations (you can > > > have NULL arguments), which also handles the sigprocmask (bonus: > > > atomically with the context get/set from a userspace perspective). > > > Would it make sense to do something similar on aarch64 ? (And have > > > glibc then exploit it). > > > > I think the usefulness of such an interface is mainly a question for > > userspace - I don't immediately see any issue with implementing it if > > it's useful to people. > > Well, the problem as far as I can tell is that the glibc implementation > of these today. They support "FPSIMD" but that's about it (so no SVE or > anything else) along with a comment: > > /* Check for FP SIMD context. We don't support restoring > contexts created by the kernel, so this context must have > been created by getcontext. Hence we can rely on the > first extension block being the FP SIMD context. */ > > That said, a bit of reading around seems to indicate that the > expecation of being able to setcontext() back to a signal handler > generated context has been deprecated by the standard and broken on x86 > for a while in Linux, so I suppose that is less of an issue. yes, setcontext is not expected to work with kernel context. > > That said, there is still some advantage in letting the kernel > implement these as it would allow the kernel to support various > "extensions" such as SVE (as long as there is room) transparently > without having to change glibc. > > In fact, isn't it possible for glibc to define its own ucontext > structure for applications to use that can potentially have a larger > reserved area ? By passing that size to the syscall, you can > essentially get userspace ready for future extensions... within limits. i think this can be a discussion for libc-alpha, but i don't think there is interest in using the libc context functions with kernel signal contexts, that turned out to be problematic historically. but if there is a use-case that can be discussed. > > > > The hard-to-solve thing is the case where the SVE context spills > > > outside of the ucontext itself, in the extra room on the stack, since > > > programs that "now" about ucontext will not have allocated space for > > > that, so that's more/less a lost cause already. > > > > You can figure out the maximum possible size for a context so it would > > be possible to define a mechanism for pointing to extra data I guess but > > yeah, it's going to be a problem when we start seeing systems with large > > enough register state. > > Extra data for userspace generated ucontext's isn't going to fly much, > there's really no "place" to put it (those things can be part of > structures etc...) and no "hook" to allocate/free sub structures. > > So you need whatever struct ucontext is used in userspace to be big > enough. > > That said, I think the current one might be enough for sve512 (I need > to check) and we could have glibc define something much bigger (16KB ?) > without much damage I suspect. > > Nagyu ? What do you think ? i think we only want to change set/get/swapcontext if there is a use-case for this. currently only a small bit of fp state has to be saved/restored. and there can be security concerns since we have features like bti that limits where one can jump (arbitrary pc in setcontext does not work). > > Cheers, > Ben. > _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: ucontext, kernel vs. userspace (glibc) 2021-09-03 7:14 ` Benjamin Herrenschmidt 2021-09-03 11:02 ` Szabolcs Nagy @ 2021-09-03 12:25 ` Mark Brown 1 sibling, 0 replies; 6+ messages in thread From: Mark Brown @ 2021-09-03 12:25 UTC (permalink / raw) To: Benjamin Herrenschmidt Cc: linux-arm-kernel, Catalin Marinas, Will Deacon, Szabolcs Nagy, benh [-- Attachment #1.1: Type: text/plain, Size: 2347 bytes --] On Fri, Sep 03, 2021 at 05:14:28PM +1000, Benjamin Herrenschmidt wrote: > Well, the problem as far as I can tell is that the glibc implementation > of these today. They support "FPSIMD" but that's about it (so no SVE or > anything else) along with a comment: > /* Check for FP SIMD context. We don't support restoring > contexts created by the kernel, so this context must have > been created by getcontext. Hence we can rely on the > first extension block being the FP SIMD context. */ The kernel does generate a FPSIMD context in addition to any SVE context for compatibility, though that doesn't mean you can actually fully restore them successfully with glibc. For contexts generated by glibc note that unless a function signature involves a SVE type the SVE register contents are caller saved: https://github.com/ARM-software/abi-aa/blob/main/aapcs64/aapcs64.rst#id48 so glibc will be fine just ignoring SVE when it generates contexts given that it'll be doing that inside a non-SVE function call. Like Szabolcs says BTI is going to cause issues restoring kernel generated contexts even without extensions like SVE adding extra register state. > > You can figure out the maximum possible size for a context so it would > > be possible to define a mechanism for pointing to extra data I guess but > > yeah, it's going to be a problem when we start seeing systems with large > > enough register state. > Extra data for userspace generated ucontext's isn't going to fly much, > there's really no "place" to put it (those things can be part of > structures etc...) and no "hook" to allocate/free sub structures. > So you need whatever struct ucontext is used in userspace to be big > enough. Indeed, it'd have to be joined up with an increase of the userspace ucontext. > That said, I think the current one might be enough for sve512 (I need > to check) and we could have glibc define something much bigger (16KB ?) > without much damage I suspect. Yes, the current context should be big enough for 512 bit SVE - that's why the kernel clamps the default SVE vector length to 512 bits, so we sidestep these issues by default even if the user happens to have a system that can do larger vector lengths. Not an immediate issue with actual hardware in any case, though that code currently kicks in on qemu's cpu=max. [-- Attachment #1.2: signature.asc --] [-- Type: application/pgp-signature, Size: 488 bytes --] [-- Attachment #2: Type: text/plain, Size: 176 bytes --] _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2021-09-03 12:27 UTC | newest] Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2021-08-30 10:40 ucontext, kernel vs. userspace (glibc) Benjamin Herrenschmidt 2021-08-31 17:44 ` Catalin Marinas 2021-09-02 12:42 ` Mark Brown 2021-09-03 7:14 ` Benjamin Herrenschmidt 2021-09-03 11:02 ` Szabolcs Nagy 2021-09-03 12:25 ` Mark Brown
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.