* [kernel-hardening] It looks like there will be no more public versions of PaX and Grsec. @ 2017-04-26 21:05 Daniel Cegiełka 2017-04-26 22:04 ` Kees Cook 0 siblings, 1 reply; 35+ messages in thread From: Daniel Cegiełka @ 2017-04-26 21:05 UTC (permalink / raw) To: kernel-hardening https://grsecurity.net/passing_the_baton_faq.php Best regards, Daniel ^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [kernel-hardening] It looks like there will be no more public versions of PaX and Grsec. 2017-04-26 21:05 [kernel-hardening] It looks like there will be no more public versions of PaX and Grsec Daniel Cegiełka @ 2017-04-26 22:04 ` Kees Cook 2017-05-01 22:01 ` Mathias Krause 0 siblings, 1 reply; 35+ messages in thread From: Kees Cook @ 2017-04-26 22:04 UTC (permalink / raw) To: Daniel Cegiełka; +Cc: kernel-hardening On Wed, Apr 26, 2017 at 2:05 PM, Daniel Cegiełka <daniel.cegielka@gmail.com> wrote: > https://grsecurity.net/passing_the_baton_faq.php Yeah, I'm sad to see them go. PaX Team was just last night helping with some details of PAX_REFCOUNT as it could appear in the upstream refcount_t API. I hope they'll still help out from time to time. It does underscore the critical need to upstream stuff, though. Forks of projects might disappear at any time. :( -Kees -- Kees Cook Pixel Security ^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [kernel-hardening] It looks like there will be no more public versions of PaX and Grsec. 2017-04-26 22:04 ` Kees Cook @ 2017-05-01 22:01 ` Mathias Krause 2017-05-02 0:09 ` Rik van Riel ` (4 more replies) 0 siblings, 5 replies; 35+ messages in thread From: Mathias Krause @ 2017-05-01 22:01 UTC (permalink / raw) To: Kees Cook; +Cc: Daniel Cegiełka, kernel-hardening On 27 April 2017 at 00:04, Kees Cook <keescook@chromium.org> wrote: > On Wed, Apr 26, 2017 at 2:05 PM, Daniel Cegiełka > <daniel.cegielka@gmail.com> wrote: >> https://grsecurity.net/passing_the_baton_faq.php > > Yeah, I'm sad to see them go. PaX Team was just last night helping > with some details of PAX_REFCOUNT as it could appear in the upstream > refcount_t API. I hope they'll still help out from time to time. > > It does underscore the critical need to upstream stuff, though. Forks > of projects might disappear at any time. :( Now, since grsecurity and PaX went private, quite a few users (including me) are left in the dark and are, to say the least, slightly pissed. But not everybody seems to be barking at the right tree. So I've a few comments and questions for the KSPP. I think the main reason for Brad and PaX Team to make their work private is the increased amount of work KSSP has put on them without providing any valuable work in return. They just don't want to be forced to maintain and fix-up the variants of grsecurity/PaX features KSPP lands upstream. And no, the work of KSPP does not make their life easier, in fact, it makes it harder. Harder for two reasons: First, the code does not end up verbatim, it's always changed, taken out of context and "enhanced". Second is the loose of control over the code. But let me elaborate those two a little further. The first point directly forces them to think through the upstream incarnation of a feature, how it differs from their original one and, for those parts, how to fix them up to work properly to fit their needs, to fit their security requirements. Those changes generate a lot of conflicts with their version of the code which takes time to fiddle out. As happened for __ro_after_init, MEMORY_SANITIZE, USERCOPY,... which only went in in reduced and modified form, generating needless work on their side -- from their point of view. The second point, the loose of control over the code, is even worse. Not getting any more conflicts when porting the grsecurity/PaX patch to a new kernel release makes changes to code that used to live in grsecurity/PaX probably go unnoticed. And, as it seem to be the case, upstream developers are not always familiar with all the gory details and might introduce weaknesses and bugs. This not only makes upstream Linux less secure, it makes grsecurity and PaX less secure, too. So I can understand why they've done this. Still, with the loose of the public availability of the patch, Linux security has suffered a lot. Not only is upstream far far away to reach a level that is available today in grsecurity and PaX, no, it also won't benefit from new developments any more. What a great achievement! :( *sigh* I think the intention of the KSPP is good -- making vanilla Linux more secure. But the way it does its work harms overall Linux security. It does hurt mine, that's for sure! I know the value of grsecurity and PaX in particular and know how easy it still is to exploit a vanilla Linux. Features like KERNEXEC and RAP make it almost impossible for an attacker to abuse a memory corruption bug in the kernel. Many unnamed features -- unnamed because not under an #ifdef -- make exploiting use-after-free bugs much less interesting or reduce the race window for TOCTTOU bugs involving user copy operations. None of this can be found in vanilla Linux. Probably never will... So, here's my list of questions for the KSPP: 1/ When will I be able to switch to a vanilla Linux kernel that is equivalently hardened as a grsecurity/PaX kernel used to be? 2/ Who will maintain this code and how? 3/ Who ensures the coverage and quality won't suffer for each new kernel release? Judging from the planed EOL for the v4.9 LTS kernel -- the last one a grsecurity patch was publicly available for --, there are less than two years to finish the work for item 1 to ensure a secure and smooth transition from grsecurity to upstream Linux. Will the KSPP be able to achieve this?... I have my doubts. Even if so, item 3 will be a hard problem to solve as the Linux kernel development model does not support such a tree-wide review point in the release cycle. Assuming there are people able and willing to judge and review the code base for required changes (e.g. atomic_t -> refcount_t changes or function pointer cleanups for RAP), how would those be able to simply get in those changes at, say, time of rc5? -- without having to argue with a horde of maintainers of the individual subsystems involved? Quite a lot of questions. The external patch solved them all by not having to deal with upstream Linux development and not having the code available until it's ready. But now it's gone and no adequate replacement is on the horizon. What will KSPP do about it? I had a reasonably secure kernel, now it's gone. :( Regards, Mathias ^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [kernel-hardening] It looks like there will be no more public versions of PaX and Grsec. 2017-05-01 22:01 ` Mathias Krause @ 2017-05-02 0:09 ` Rik van Riel 2017-05-02 14:46 ` Shawn 2017-05-02 21:16 ` Mathias Krause 2017-05-02 0:39 ` Olof Johansson ` (3 subsequent siblings) 4 siblings, 2 replies; 35+ messages in thread From: Rik van Riel @ 2017-05-02 0:09 UTC (permalink / raw) To: Mathias Krause, Kees Cook; +Cc: Daniel Cegiełka, kernel-hardening On Tue, 2017-05-02 at 00:01 +0200, Mathias Krause wrote: > I think the intention of the KSPP is good -- making vanilla Linux > more > secure. But the way it does its work harms overall Linux security. It > does hurt mine, that's for sure! Yeah, no. The grsecurity people produced patches that were used on maybe a few tens of thousands of systems, while the KSPP code will end up enhancing the security of over a billion Android devices. Those Android devices are more likely to require hardening, too, since they do not receive security updates as quickly as the systems maintained by grsecurity users. Integrating hardening into the upstream kernel is a good thing for security, not a bad thing. ^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [kernel-hardening] It looks like there will be no more public versions of PaX and Grsec. 2017-05-02 0:09 ` Rik van Riel @ 2017-05-02 14:46 ` Shawn 2017-05-02 18:55 ` Kees Cook 2017-05-02 21:16 ` Mathias Krause 1 sibling, 1 reply; 35+ messages in thread From: Shawn @ 2017-05-02 14:46 UTC (permalink / raw) To: Rik van Riel Cc: Mathias Krause, Kees Cook, Daniel Cegiełka, kernel-hardening On Tue, May 2, 2017 at 8:09 AM, Rik van Riel <riel@redhat.com> wrote: > On Tue, 2017-05-02 at 00:01 +0200, Mathias Krause wrote: > >> I think the intention of the KSPP is good -- making vanilla Linux >> more >> secure. But the way it does its work harms overall Linux security. It >> does hurt mine, that's for sure! > > Yeah, no. The grsecurity people produced patches > that were used on maybe a few tens of thousands > of systems, while the KSPP code will end up > enhancing the security of over a billion Android > devices. > Are you serious think KSPP can solve Android security issue without changing the current Android eco-system? It's about one billion Android device( most are old armv7 based) can be exploited and being used by criminal and BIGBROs( NSA/CIA/PLA/whatever-A*/etc) with low cost. I made hardened PoC( based on PaX + PXN + some code hardening) for Nexus 7 2013 back in 2015. It proved that it can defeat "massive" exp without much perf impact. But I don't see any sign those cellphone vendors would port any KSPP features into the old devices. How about new phone? I ported a couple of KSPP features( all originally done or inspired by PaX/Grsecurity) to Pixel XL: https://github.com/hardenedlinux/armv7-nexus7-grsec Google's Pixel/Pixel2 may be the one of few cellphones can getting benefit from KSPP. Otherwise, my own phone is running with much more hardening features than Android O( released in Oct?) but it's still not secure enough to defeat customized exploit. > Those Android devices are more likely to require > hardening, too, since they do not receive security > updates as quickly as the systems maintained by > grsecurity users. > Don't worry about it. PaX/Grsecurity can defeat multiple public exploits without any fix. Maybe some ppl will go to maintain 4.9 LTS. > Integrating hardening into the upstream kernel is > a good thing for security, not a bad thing. > It depends how much security you need. -- GNU powered it... GPL protect it... God blessing it... regards Shawn ^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [kernel-hardening] It looks like there will be no more public versions of PaX and Grsec. 2017-05-02 14:46 ` Shawn @ 2017-05-02 18:55 ` Kees Cook 2017-05-03 4:50 ` Shawn 0 siblings, 1 reply; 35+ messages in thread From: Kees Cook @ 2017-05-02 18:55 UTC (permalink / raw) To: Shawn Cc: Rik van Riel, Mathias Krause, Daniel Cegiełka, kernel-hardening On Tue, May 2, 2017 at 7:46 AM, Shawn <citypw@gmail.com> wrote: > On Tue, May 2, 2017 at 8:09 AM, Rik van Riel <riel@redhat.com> wrote: >> On Tue, 2017-05-02 at 00:01 +0200, Mathias Krause wrote: >> >>> I think the intention of the KSPP is good -- making vanilla Linux >>> more >>> secure. But the way it does its work harms overall Linux security. It >>> does hurt mine, that's for sure! >> >> Yeah, no. The grsecurity people produced patches >> that were used on maybe a few tens of thousands >> of systems, while the KSPP code will end up >> enhancing the security of over a billion Android >> devices. >> > Are you serious think KSPP can solve Android security issue without > changing the current Android eco-system? It's about one billion > Android device( most are old armv7 based) can be exploited and being > used by criminal and BIGBROs( NSA/CIA/PLA/whatever-A*/etc) with low > cost. I made hardened PoC( based on PaX + PXN + some code hardening) > for Nexus 7 2013 back in 2015. It proved that it can defeat "massive" > exp without much perf impact. But I don't see any sign those cellphone > vendors would port any KSPP features into the old devices. How about > new phone? I ported a couple of KSPP features( all originally done or > inspired by PaX/Grsecurity) to Pixel XL: > > https://github.com/hardenedlinux/armv7-nexus7-grsec I would love it if things would get rolled out to older phones. This isn't something upstream has much influence over, though. But we have to look to the future. Eventually all the old phones will get replaced, and the new ones will be running newer kernels, etc. The problem with the phone (and IoT) ecosystem is an entirely separate problem. There are some folks chipping away at it, but it's tangential to upstream gaining more defenses. I'd also note that this problem isn't solved by grsecurity either --- how many phones are running the grsecurity patch? > Google's Pixel/Pixel2 may be the one of few cellphones can getting > benefit from KSPP. Otherwise, my own phone is running with much more > hardening features than Android O( released in Oct?) but it's still > not secure enough to defeat customized exploit. > >> Those Android devices are more likely to require >> hardening, too, since they do not receive security >> updates as quickly as the systems maintained by >> grsecurity users. >> > Don't worry about it. PaX/Grsecurity can defeat multiple public > exploits without any fix. Maybe some ppl will go to maintain 4.9 LTS. This is kind of true. RAP was x86_64 only, so it didn't help armv7 nor arm64. Hardened usercopy wasn't ported by grsecurity to arm64 (upstream did that, though it was trivial). UDEREF wasn't ported by grsecurity to arm64, ARM filled that gap by inventing the mechanisms to provide PAN emulation. There are all kinds of other protections aren't arch-specific (e.g. heap hardening). Those would make good things to focus on upstreaming if you were interested. CopperheadOS is using a bunch of those things already: https://github.com/CopperheadOS/kernel_google_marlin/commits/nougat-mr2-release So really, upstreaming defenses has really helped the future phone ecosystem, even if the "don't update old devices" attitude continues to not change with vendors. -Kees -- Kees Cook Pixel Security ^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [kernel-hardening] It looks like there will be no more public versions of PaX and Grsec. 2017-05-02 18:55 ` Kees Cook @ 2017-05-03 4:50 ` Shawn 2017-05-03 18:56 ` Rik van Riel 2017-05-04 5:45 ` Kees Cook 0 siblings, 2 replies; 35+ messages in thread From: Shawn @ 2017-05-03 4:50 UTC (permalink / raw) To: Kees Cook Cc: Rik van Riel, Mathias Krause, Daniel Cegiełka, kernel-hardening On Wed, May 3, 2017 at 2:55 AM, Kees Cook <keescook@chromium.org> wrote: > On Tue, May 2, 2017 at 7:46 AM, Shawn <citypw@gmail.com> wrote: >> On Tue, May 2, 2017 at 8:09 AM, Rik van Riel <riel@redhat.com> wrote: >>> On Tue, 2017-05-02 at 00:01 +0200, Mathias Krause wrote: >>> >>>> I think the intention of the KSPP is good -- making vanilla Linux >>>> more >>>> secure. But the way it does its work harms overall Linux security. It >>>> does hurt mine, that's for sure! >>> >>> Yeah, no. The grsecurity people produced patches >>> that were used on maybe a few tens of thousands >>> of systems, while the KSPP code will end up >>> enhancing the security of over a billion Android >>> devices. >>> >> Are you serious think KSPP can solve Android security issue without >> changing the current Android eco-system? It's about one billion >> Android device( most are old armv7 based) can be exploited and being >> used by criminal and BIGBROs( NSA/CIA/PLA/whatever-A*/etc) with low >> cost. I made hardened PoC( based on PaX + PXN + some code hardening) >> for Nexus 7 2013 back in 2015. It proved that it can defeat "massive" >> exp without much perf impact. But I don't see any sign those cellphone >> vendors would port any KSPP features into the old devices. How about >> new phone? I ported a couple of KSPP features( all originally done or >> inspired by PaX/Grsecurity) to Pixel XL: >> >> https://github.com/hardenedlinux/armv7-nexus7-grsec > > I would love it if things would get rolled out to older phones. This > isn't something upstream has much influence over, though. But we have > to look to the future. Eventually all the old phones will get > replaced, and the new ones will be running newer kernels, etc. > > The problem with the phone (and IoT) ecosystem is an entirely separate > problem. There are some folks chipping away at it, but it's tangential > to upstream gaining more defenses. I'd also note that this problem > isn't solved by grsecurity either --- how many phones are running the > grsecurity patch? > The fragmentation of Android eco-system may be inevitable. The whole chains is too long from ASOP/BSP/Vendors and it affect the security fix being delivered to the end user. According to my own statistic from my customers, there will be more than 7 millions of Android phone will be using some features of PaX/Grsec this year. I've been pushing those "simplified" mitigation from PaX/Grsecurity to some vendors. That's why I have to find out what exploitable bugs is possible to turned to be "massive" exp and which one is being massive used already. To me, it's obviously my problem solved by PaX/Grsecurity. I can't wait for Google's hepl to solve the problem. I shared what I found about "massive" exp with you and other Google folksl at last linux security summit. I really appreciate you guys helped some patches goes into AOSP. But it's just a starting point. Maybe in Google's point of view, everybody should buy the new phone( btw, I really love Pixel XL). But I have responsible to taking care of old devices. >> Google's Pixel/Pixel2 may be the one of few cellphones can getting >> benefit from KSPP. Otherwise, my own phone is running with much more >> hardening features than Android O( released in Oct?) but it's still >> not secure enough to defeat customized exploit. >> >>> Those Android devices are more likely to require >>> hardening, too, since they do not receive security >>> updates as quickly as the systems maintained by >>> grsecurity users. >>> >> Don't worry about it. PaX/Grsecurity can defeat multiple public >> exploits without any fix. Maybe some ppl will go to maintain 4.9 LTS. > > This is kind of true. RAP was x86_64 only, so it didn't help armv7 nor > arm64. Hardened usercopy wasn't ported by grsecurity to arm64 > (upstream did that, though it was trivial). UDEREF wasn't ported by > grsecurity to arm64, ARM filled that gap by inventing the mechanisms > to provide PAN emulation. There are all kinds of other protections > aren't arch-specific (e.g. heap hardening). Those would make good > things to focus on upstreaming if you were interested. CopperheadOS is > using a bunch of those things already: > > https://github.com/CopperheadOS/kernel_google_marlin/commits/nougat-mr2-release > Yeah, I know the history. Intel/ARM never credited PaX/Grsecurity about the contribution of KERNEXEC/UDEREF, which is the origins of hardware implementation, e.g: SMEP/SMAP in x86, PXN/PAN in armv7/arm64. I don't blame Spender/PaX team;-) I thank to ARM maintainers who did those mitigation for arm64. But I thank more to PaX/Grsecurity who created it from nothing. btw: I share the same view with Mathias Krause and other ppl who really concern the real sense of security. I like KSPP in the 1st place. But now I lost PaX/Grsecurity test patch. Who should I blame? I think we made our point very clear here: https://hardenedlinux.github.io/announcement/2017/04/29/hardenedlinux-statement2.html > So really, upstreaming defenses has really helped the future phone > ecosystem, even if the "don't update old devices" attitude continues > to not change with vendors. > Agree! I'm afraid I can't help much on upstream because that's not my customer want in short-term. S0rry, my idealist side never fit in KSPP. -- GNU powered it... GPL protect it... God blessing it... regards Shawn ^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [kernel-hardening] It looks like there will be no more public versions of PaX and Grsec. 2017-05-03 4:50 ` Shawn @ 2017-05-03 18:56 ` Rik van Riel 2017-05-03 19:36 ` Daniel Micay 2017-05-04 5:45 ` Kees Cook 1 sibling, 1 reply; 35+ messages in thread From: Rik van Riel @ 2017-05-03 18:56 UTC (permalink / raw) To: Shawn, Kees Cook; +Cc: Mathias Krause, Daniel Cegiełka, kernel-hardening [-- Attachment #1: Type: text/plain, Size: 1464 bytes --] On Wed, 2017-05-03 at 12:50 +0800, Shawn wrote: > The fragmentation of Android eco-system may be inevitable. The whole > chains is too long from ASOP/BSP/Vendors and it affect the security > fix being delivered to the end user. According to my own statistic > from my customers, there will be more than 7 millions of Android > phone > will be using some features of PaX/Grsec this year. That is great news. I am glad to hear the hardening features are being used on that many phones. Of course, given the fragmentation of the eco-system, the only thing that can get the hardening on all of the (new) phones in the future will be getting the hardening features into the upstream kernel. > btw: I share the same view with Mathias Krause and other ppl who > really concern the real sense of security. I like KSPP in the 1st > place. But now I lost PaX/Grsecurity test patch. Who should I blame? I am not sure anyone deserves blame for this situation. Spender has been doing what is best for his business, and his work is an important asset for security-minded people. Kees and the other KSPP contributors have been doing what is best for the community, and wide-spread adoption of hardening functionality. The important question to ask is "what do we do now?" I suspect the answer is upstreaming more and more of the grsecurity functionality, so nobody needs to carry around that patch any more. -- All rights reversed [-- Attachment #2: This is a digitally signed message part --] [-- Type: application/pgp-signature, Size: 473 bytes --] ^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [kernel-hardening] It looks like there will be no more public versions of PaX and Grsec. 2017-05-03 18:56 ` Rik van Riel @ 2017-05-03 19:36 ` Daniel Micay 0 siblings, 0 replies; 35+ messages in thread From: Daniel Micay @ 2017-05-03 19:36 UTC (permalink / raw) To: Rik van Riel, Shawn, Kees Cook Cc: Mathias Krause, Daniel Cegiełka, kernel-hardening [-- Attachment #1: Type: text/plain, Size: 2029 bytes --] On Wed, 2017-05-03 at 14:56 -0400, Rik van Riel wrote: > On Wed, 2017-05-03 at 12:50 +0800, Shawn wrote: > > > The fragmentation of Android eco-system may be inevitable. The whole > > chains is too long from ASOP/BSP/Vendors and it affect the security > > fix being delivered to the end user. According to my own statistic > > from my customers, there will be more than 7 millions of Android > > phone > > will be using some features of PaX/Grsec this year. > > That is great news. I am glad to hear the hardening features > are being used on that many phones. > > Of course, given the fragmentation of the eco-system, the > only thing that can get the hardening on all of the (new) > phones in the future will be getting the hardening features > into the upstream kernel. Just worth noting that the upstream in this case almost always includes the Android common kernel. There's still some baseline out-of-tree Android code, although there's much less than there used to be so the vast majority of the code these days is SoC vendor code needed by a non- Android Linux distribution on those devices too. That's forked into the SoC vendor kernels and then the device kernels if applicable (some are device-specific, but some vendors like Sony have moved to having a shared kernel and the Pixel / Pixel XL share a kernel since they're basically the same thing). Google can also verify that hardening is present via the Compatibility Test Suite if it can be detected from an unprivileged userspace app. Hopefully they'll turn passing their new privileged vts test suite into a requirement too so they can test for kernel self protection features. For example, Android devices are required to have perf_event_paranoid=3 even though it was rejected upstream for the time being. If a clearly useful change is rejected, that doesn't mean Google won't add it to their common kernel which will then propagate at least to new devices from other vendors and their own first-party released devices. [-- Attachment #2: This is a digitally signed message part --] [-- Type: application/pgp-signature, Size: 866 bytes --] ^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [kernel-hardening] It looks like there will be no more public versions of PaX and Grsec. 2017-05-03 4:50 ` Shawn 2017-05-03 18:56 ` Rik van Riel @ 2017-05-04 5:45 ` Kees Cook 2017-05-04 6:47 ` Lionel Debroux 2017-05-04 14:11 ` Shawn 1 sibling, 2 replies; 35+ messages in thread From: Kees Cook @ 2017-05-04 5:45 UTC (permalink / raw) To: Shawn Cc: Rik van Riel, Mathias Krause, Daniel Cegiełka, kernel-hardening On Tue, May 2, 2017 at 9:50 PM, Shawn <citypw@gmail.com> wrote: > The fragmentation of Android eco-system may be inevitable. The whole > chains is too long from ASOP/BSP/Vendors and it affect the security > fix being delivered to the end user. According to my own statistic > from my customers, there will be more than 7 millions of Android phone > will be using some features of PaX/Grsec this year. I've been pushing > those "simplified" mitigation from PaX/Grsecurity to some vendors. > That's why I have to find out what exploitable bugs is possible to > turned to be "massive" exp and which one is being massive used > already. To me, it's obviously my problem solved by PaX/Grsecurity. I > can't wait for Google's hepl to solve the problem. I shared what I > found about "massive" exp with you and other Google folksl at last > linux security summit. I really appreciate you guys helped some > patches goes into AOSP. But it's just a starting point. Maybe in > Google's point of view, everybody should buy the new phone( btw, I > really love Pixel XL). But I have responsible to taking care of old > devices. My concerns aren't limited to just phones, since the pattern of "stick with one kernel version" is endemic in the industry. Even distros are mostly guilty of this, though their production cycle time is much shorter. I'm glad to have people like you working on this, especially in places where it's traditionally not gotten a lot of attention! > Yeah, I know the history. Intel/ARM never credited PaX/Grsecurity > about the contribution of KERNEXEC/UDEREF, which is the origins of > hardware implementation, e.g: SMEP/SMAP in x86, PXN/PAN in > armv7/arm64. I don't blame Spender/PaX team;-) I thank to ARM > maintainers who did those mitigation for arm64. But I thank more to > PaX/Grsecurity who created it from nothing. Yeah, no doubt. The stuff PaX and grsecurity invented was amazing! And I credit them as helping me see that killing individual bugs isn't sufficient for security defense work. > btw: I share the same view with Mathias Krause and other ppl who > really concern the real sense of security. I like KSPP in the 1st > place. But now I lost PaX/Grsecurity test patch. Who should I blame? I > think we made our point very clear here: > > https://hardenedlinux.github.io/announcement/2017/04/29/hardenedlinux-statement2.html Well, there's exactly one group that decided to pull the patches. If you want to assign blame, I think it's clear it falls squarely on their shoulders. To the first point at your URL, there's a lot of inaccuracies. KSPP is an upstream group; CII funds individuals (not the whole of KSPP) that make independent proposals. In fact, the very first CII funded person was ephox from grsecurity. So ... if CII funding contributors to upstream was the problem, then grsecurity created their own problem? That makes no sense. As for "hardly accomplished anything compared to grsecurity" that's a pretty unfair comparison to make. Upstream has been really focusing on this for maybe 2 years at best now? And grsecurity has been doing it for over a decade? And besides, "upstream doesn't have all of grsecurity upstreamed" isn't a reason to blame upstream for grsecurity going private. How can you say "introduced more bugs"? I'd be happy to fix them, if you see them. And as I talked about in my reply to Mathias, of course things look incomplete in upstream: they're being developed like everything else in upstream: incrementally. It is not possible to just fork-lift everything from grsecurity into upstream. It's just not how upstream development works. To this notion that there is some kind of conspiracy by LF to snub grsecurity is just flat out silly. LF isn't upstream (though, sure, they support a lot of the work). Upstream developers can't control what or how LF says things (and few of us have time to deal with that). If you look at things in upstream that came from grsecurity, you'll see plenty of credit given. That sure seems like upstream "revealing the truth to the public". Look at the copyright notices in the gcc plugins, in mm/usercopy.c, and the notes in the Kconfigs, and the commit messages. Look at my own blog posts covering the features, look at the KSPP wiki page. The vast majority of things attributable to grsecurity, do, in fact, give credit. So upstream certainly isn't "blatantly stealing credit from grsecurity", so why would LF? And if this single PR statement from LF is still seen as a sudden marketing war against grsecurity, then I would ask "How is making the patches private the solution?" Grsecurity has spent over a decade mocking and berating upstream (just look at LWN), so how does a single article from LF suddenly undo all their work to differentiate themselves? This whole line of reasoning is convoluted, and frankly, has absolutely nothing to do with upstream's efforts to protect Linux users. It'd be nice if I could convince you otherwise (see my reply to Mathias), but upstream hasn't "created more work" for grsecurity. And even if you still believe that, making their patches private doesn't reduce the amount of work they have to do. It just forces people to pay them. You later speak of philosophy, so let me ask you: if they've forced their users to pay for access because their features were being upstreamed, then they do not want these protections available to everyone, they want them available only to those who would pay for them. To me, though, it still can't be this simplistic. They've mostly rejected spending free time to upstream things, they've mostly rejected being paid to upstream things, and their efforts to be paid to just work on grsecurity in the open (see the CII thread from a while back) haven't worked, or are insufficient in some way. What is it that grsecurity wants? It looks very much like they want to stay a fork and be paid for it. I certainly can't blame them for this; upstreaming is hard and as Mathias says, they lose a certain level of control over the results. Fundamentally, it's their time; they can do whatever they want with it. And looking to be paid for your time is of course perfectly sensible. But if this desire to stay a fork is true, then their primary goal is NOT protecting as many people as possible, it's protecting a subset of people as perfectly as they can. Again, this is fine. Upstream and grsecurity can coexist in this state (since again, upstreaming doesn't increase their work), and this was the state for over a decade. But suddenly grsecurity went private. So it would appear that grsecurity couldn't make enough money with their patches staying public, so they've made them private. If that's true, then it means that everyone who was using grsecurity but wasn't a sponsor of grsecurity's work holds some level of responsibility for grsecurity's decision. It could be argued that upstream is "using grsecurity but wasn't a sponsor", but this would be missing an important point: upstream's contribution to grsecurity is Linux itself, which has giant value. (And for my part personally, I did everything I could to pay grsecurity, and succeeded a couple times over the years.) With this we have come back around to looking at the GPL, which you mentioned. This is specifically what the GPL was created to protect against. Why hasn't grsecurity paid money to the tens of thousands of contributors to the Linux kernel? Because they don't have to, and that is part of the understanding about using the GPL for a project: everyone contributes what they can, and everyone benefits. And if there are forks, people can use each other's work so no one is risking being in a situation where someone is unfairly benefiting from someone else's work. You had invoked RMS, so, here we are. It looks like grsecurity has gone private because upstream started exercising the princples of the GPL to expand how many upstream users would benefit from grsecurity's work in the same way grsecurity's users benefited from upstream's work. In answer to "who should I blame?" I'd say this is squarely on grsecurity, not anyone else. They've chosen to benefit from upstream's future work without making their future work available to upstream. But maybe this is all wrong. Neither of us can speak for grsecurity, nor know their motivations. I've always tried to avoid these kinds of discussions for that very reason, and to stay focused on the technical work. I can only truly speak for myself. I was faced with a seemingly simple problem: Linux users need to be protected, the vast majority of Linux users run upstream-derived kernels, but many of the best defenses lived in grsecurity. What I'd learned from working at an industry group (OSDL, the pre-LF), working at a distro (Canonical on Ubuntu), and now working on devices and services (Google), is that engineering teams in most places won't risk basing their work on a fork of upstream, whether it be support concerns, cost concerns, complexity concerns, or whatever. At the end of the day, it was clear that the only path forward was to focus upstream on the same defense principles grsecurity espoused, and to then start porting or creating these kinds of defenses in upstream. (Which is the very thing grsecurity told upstream to do at the first Linux Security Summit in 2010!) Doing this work was very time consuming, so I tried to collect interested people together to work under a single umbrella to help change the upstream culture, examine flaws, port or invent defenses, test, document, etc. So, in the end, I can't speak to grsecurity's true motivations, but I can speak to mine, and what I'd like upstream to be doing: protecting as many Linux users as completely as possible. What's possible in upstream differs from what was possible in grsecurity, so no one should expect the same things between the two. Comparing the two projects is fraught with error, and blaming anyone other than grsecurity for going private is ridiculous. > Agree! I'm afraid I can't help much on upstream because that's not my > customer want in short-term. S0rry, my idealist side never fit in > KSPP. Sure, understood. I'm delighted you've spent time getting kernel hardening stuff landed in AOSP, though; for vendors that can be convinced to care about backporting these things, it's great to have a place to point to for them to get your work! And it's fine not to contribute to upstream, since luckily there will be some people who can. Upstream will continue to work on security defenses, but at the very least, it'd be nice if people would be constructive in their criticisms. We've gone over a decade with grsecurity's frequently toxic rhetoric only alienating people instead of productively working together to reach what should be the common goal of security work: defending users. -Kees -- Kees Cook Pixel Security ^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [kernel-hardening] It looks like there will be no more public versions of PaX and Grsec. 2017-05-04 5:45 ` Kees Cook @ 2017-05-04 6:47 ` Lionel Debroux 2017-05-05 19:54 ` Kees Cook 2017-05-04 14:11 ` Shawn 1 sibling, 1 reply; 35+ messages in thread From: Lionel Debroux @ 2017-05-04 6:47 UTC (permalink / raw) To: Kees Cook, Shawn Cc: Rik van Riel, Mathias Krause, Daniel Cegiełka, kernel-hardening Hi, (I hope that silly webmail is not going to butcher my e-mail's layout, this time...) > It'd be nice if I could convince you otherwise (see my reply to > Mathias), but upstream hasn't "created more work" for grsecurity. > [...] > (since again, upstreaming doesn't increase their work) Uh, do you really mean that ? Because on the contrary, it's a (real) _fact_ that upstreaming something directly taken from, or inspired by, the PaX/grsecurity work _does_ increase their work, through at least three well-identified mechanisms: * adjusting their work to match upstream's changes. That's relatively simple if the changes are pure renames, but as we know, upstream changes are not always (not often, even, they'd probably say) that simple - and do not always come without drawbacks on complexity or protective abilities, either. spender has been posting (complaining) multiple times about that, giving specific examples - which you may have seen by yourself. * helping upstream perform the integration in the first place. They had to spend much time helping the mainlining process of some bits. See it indirectly in spender's posts questioning mainline's ability to perform _sustained_ maintenance of the integrated features (even if they received some minor fixes on some GCC plugins in return for the integration, as you mentioned). Don't let spender's usual writing style (not necessarily more abrasive than Linus', BTW) shadow the technical reasons behind his complaints, some are arguably valid. * unapplying, one by one, the hunks mentioned earlier for improvements related to (mostly) CONSTIFY, RANDSTRUCT, RAP. They've been carrying these patches for years, with very little maintenance: the vast majority of these hunks didn't change from one upstream version to the next one. We've already discussed at length the non-technical reasons why nobody picked them up. To tell the truth, the fact that upstreaming said changes does, as a matter of fact, increase their work load was the second, auxiliary reason why, after several patches in 2010 and 2012, I didn't spend more of my free time mainlining some of these small, scattered changes, rather than on other endeavours. I find myself caring quite a bit less about that second reason now that the patches have become private, but the main reason of the dishearteningly expensive mainlining process largely remains valid... I do understand the need for reviews in general, but CONSTIFY or RANDSTRUCT-related hunks are arguably special. > And even if you still believe that, making their patches private > doesn't reduce the amount of work they have to do. It just forces > people to pay them. Yes, that is true. Arguably, in some ways, since making their patches private increases the incentive to mainline hunks thereof, it _increases_ their work over some areas in the short and mid-term. Bye, Lionel. ^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [kernel-hardening] It looks like there will be no more public versions of PaX and Grsec. 2017-05-04 6:47 ` Lionel Debroux @ 2017-05-05 19:54 ` Kees Cook 0 siblings, 0 replies; 35+ messages in thread From: Kees Cook @ 2017-05-05 19:54 UTC (permalink / raw) To: Lionel Debroux Cc: Shawn, Rik van Riel, Mathias Krause, Daniel Cegiełka, kernel-hardening On Wed, May 3, 2017 at 11:47 PM, Lionel Debroux <lionel_debroux@yahoo.fr> wrote: >> It'd be nice if I could convince you otherwise (see my reply to >> Mathias), but upstream hasn't "created more work" for grsecurity. >> [...] >> (since again, upstreaming doesn't increase their work) > Uh, do you really mean that ? > Because on the contrary, it's a (real) _fact_ that upstreaming > something directly taken from, or inspired by, the PaX/grsecurity I think it would be noise compared to all the regular work needed when doing forward porting, yes. As I said in my reply to Mathias, I think some things make it harder, some things make it easier, and others don't change anything. Other stuff happening in upstream can be WAY more disruptive, like rewriting x86_64 entry code. I bet that was terrible to do a forward port after. > work _does_ increase their work, through at least three > well-identified mechanisms: > * adjusting their work to match upstream's changes. That's relatively > simple if the changes are pure renames, but as we know, upstream > changes are not always (not often, even, they'd probably say) that > simple - and do not always come without drawbacks on complexity or > protective abilities, either. spender has been posting (complaining) > multiple times about that, giving specific examples - which you may > have seen by yourself. This is what I think is covered by regular upstreaming, mentioned above. On the level, it looks like it's not a meaningfully different amount of work. Maybe I'm wrong, but you can go look at the evidence I linked to about how forward porting was performed in the past. > * helping upstream perform the integration in the first place. They > had to spend much time helping the mainlining process of some bits. > See it indirectly in spender's posts questioning mainline's ability > to perform _sustained_ maintenance of the integrated features (even > if they received some minor fixes on some GCC plugins in return for > the integration, as you mentioned). Well, again, they get all of upstream. The hardened usercopy upstreaming triggered a massive uaccess rewrite that means all of the arch-specific code has gone away so there's one place for usercopy stuff. That implies both reduction in forward porting work and evidence of "sustained maintenance". My little typo fix patch was something I sent because I noticed it affected things that weren't yet upstream, so it felt silly not to send it since I was right there looking at the code. > Don't let spender's usual writing style (not necessarily more abrasive > than Linus', BTW) shadow the technical reasons behind his complaints, > some are arguably valid. Neither of their styles excuse the other, but yes, I absolutely understand that if I can dig through the mockery and insults, I can almost always find very compelling technical analysis from grsecurity. > * unapplying, one by one, the hunks mentioned earlier for improvements > related to (mostly) CONSTIFY, RANDSTRUCT, RAP. They've been carrying > these patches for years, with very little maintenance: the vast > majority of these hunks didn't change from one upstream version to the > next one. We've already discussed at length the non-technical reasons > why nobody picked them up. Well, again, the evidence in the recent kernel patches would indicate that's not how things have been dealt with. > To tell the truth, the fact that upstreaming said changes does, as a > matter of fact, increase their work load was the second, auxiliary > reason why, after several patches in 2010 and 2012, I didn't spend > more of my free time mainlining some of these small, scattered changes, > rather than on other endeavours. I find myself caring quite a bit less > about that second reason now that the patches have become private, but > the main reason of the dishearteningly expensive mainlining process > largely remains valid... I do understand the need for reviews in > general, but CONSTIFY or RANDSTRUCT-related hunks are arguably special. Just out of curiosity, what changes did you upstream and what was communicated to you about them being disruptive? Constify is quite special-purpose, yes. Randstruct less so (and linux-next now only needs a few more fix-ups, I've upstreamed the rest I've identified). >> And even if you still believe that, making their patches private >> doesn't reduce the amount of work they have to do. It just forces >> people to pay them. > Yes, that is true. Arguably, in some ways, since making their patches > private increases the incentive to mainline hunks thereof, it > _increases_ their work over some areas in the short and mid-term. I wish grsecurity had responded to my requests for guidance about how to make their forward-porting easier. Having that kind of work myself in the past, I'll just continue to try to organize things in ways that seems the least disruptive to me, and make suggestions to other people about doing the same. -Kees -- Kees Cook Pixel Security ^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [kernel-hardening] It looks like there will be no more public versions of PaX and Grsec. 2017-05-04 5:45 ` Kees Cook 2017-05-04 6:47 ` Lionel Debroux @ 2017-05-04 14:11 ` Shawn 2017-05-04 16:03 ` Greg KH 1 sibling, 1 reply; 35+ messages in thread From: Shawn @ 2017-05-04 14:11 UTC (permalink / raw) To: Kees Cook Cc: Rik van Riel, Mathias Krause, Daniel Cegiełka, kernel-hardening On Thu, May 4, 2017 at 1:45 PM, Kees Cook <keescook@chromium.org> wrote: >> btw: I share the same view with Mathias Krause and other ppl who >> really concern the real sense of security. I like KSPP in the 1st >> place. But now I lost PaX/Grsecurity test patch. Who should I blame? I >> think we made our point very clear here: >> >> https://hardenedlinux.github.io/announcement/2017/04/29/hardenedlinux-statement2.html > > Well, there's exactly one group that decided to pull the patches. If > you want to assign blame, I think it's clear it falls squarely on > their shoulders. > > To the first point at your URL, there's a lot of inaccuracies. KSPP is > an upstream group; CII funds individuals (not the whole of KSPP) that > make independent proposals. In fact, the very first CII funded person > was ephox from grsecurity. So ... if CII funding contributors to > upstream was the problem, then grsecurity created their own problem? > That makes no sense. > > As for "hardly accomplished anything compared to grsecurity" that's a > pretty unfair comparison to make. Upstream has been really focusing on > this for maybe 2 years at best now? And grsecurity has been doing it > for over a decade? And besides, "upstream doesn't have all of > grsecurity upstreamed" isn't a reason to blame upstream for grsecurity > going private. > > How can you say "introduced more bugs"? I'd be happy to fix them, if > you see them. And as I talked about in my reply to Mathias, of course > things look incomplete in upstream: they're being developed like > everything else in upstream: incrementally. It is not possible to just > fork-lift everything from grsecurity into upstream. It's just not how > upstream development works. > > To this notion that there is some kind of conspiracy by LF to snub > grsecurity is just flat out silly. LF isn't upstream (though, sure, > they support a lot of the work). Upstream developers can't control > what or how LF says things (and few of us have time to deal with > that). If you look at things in upstream that came from grsecurity, > you'll see plenty of credit given. That sure seems like upstream > "revealing the truth to the public". Look at the copyright notices in > the gcc plugins, in mm/usercopy.c, and the notes in the Kconfigs, and > the commit messages. Look at my own blog posts covering the features, > look at the KSPP wiki page. The vast majority of things attributable > to grsecurity, do, in fact, give credit. So upstream certainly isn't > "blatantly stealing credit from grsecurity", so why would LF? And if > this single PR statement from LF is still seen as a sudden marketing > war against grsecurity, then I would ask "How is making the patches > private the solution?" Grsecurity has spent over a decade mocking and > berating upstream (just look at LWN), so how does a single article > from LF suddenly undo all their work to differentiate themselves? This > whole line of reasoning is convoluted, and frankly, has absolutely > nothing to do with upstream's efforts to protect Linux users. > I don't have the problem with ephox funded by CII for upstreaming GCC plugins. I blame LF/CII's( I can't speak for other ppl from HardenedLinux) PR because they've been doing PR since the day 1 of KSPP. A lot of ppl being misled by their improper PR including BLUG( Beijing GNU/Linux User Group) and my customers. I don't know how LF sells their "service" or "membership" and what stories they tell to enterprise management. Eventually some of my customers indeed being affected by some false sense of security advertisement about linux security. As a security consultant, how am I suppose to answer the question like "there are a bunch of exploits in the wild and the linux( KSPP?) can't mitigate, which isn't the stories we've had heard" from customers? I understand it's unfair to compare KSPP with PaX/Grsecurity. But the fact is there are multiple exploitable bugs in the wild that's the reason I can not rely on KSPP now: https://github.com/hardenedlinux/grsecurity-101-tutorials/blob/master/kernel_vuln_exp.md You may still remember those days that "one null-ptr deref exploit can rule them all". It was my nightmare that endless breaches from both outside and inside. PaX's KERNEXEC/UDEREF saved my time back then. As the evidences show that PaX/Grsec is still the most effective defense in RING0 so far. That announcement only represented the POV from a group of ppl. From my( and other ppl from HardenedLinux) perspective, Linux foundation is a commercial company and very good at PR but zero integrity to us. They don't respect individuals and the community. But I never said KSPP don't credited PaX/Grsecurity in the commits or Kconfig. I've been reviewing you guys work, which porting some pieces of PaX/Grsecurity code/features to the mainline. I'm glad to see linux kernel is raising the bar. I translated KSPP documentation into Chinese and let more ppl( both community and commercial customers need education) know there are some ppl care security and trying to achieve the security level what PaX/Grsecurity can provided. But what LF PR does is really harmful to my efforts. I'll have to spend my extra time on explanation and writing reports for those victims( customers, other vendors, even one of my investors). As I see, LF was trying to sabotage the small open source consulting business like mine. Otherwise, I don't care about Spender's personality stuff. All I concern is the practical defense solution to protect our asset from those threats not only from Ring 0. Both fortunate and unfortunate that PaX/Grsec is still an option. We build our defense solution based on PaX/Grsecurity and some of our work is open source at HardenedLinux's github. > It'd be nice if I could convince you otherwise (see my reply to > Mathias), but upstream hasn't "created more work" for grsecurity. And > even if you still believe that, making their patches private doesn't > reduce the amount of work they have to do. It just forces people to > pay them. > > You later speak of philosophy, so let me ask you: if they've forced > their users to pay for access because their features were being > upstreamed, then they do not want these protections available to > everyone, they want them available only to those who would pay for > them. To me, though, it still can't be this simplistic. They've mostly > rejected spending free time to upstream things, they've mostly > rejected being paid to upstream things, and their efforts to be paid > to just work on grsecurity in the open (see the CII thread from a > while back) haven't worked, or are insufficient in some way. What is > it that grsecurity wants? > > It looks very much like they want to stay a fork and be paid for it. I > certainly can't blame them for this; upstreaming is hard and as > Mathias says, they lose a certain level of control over the results. > Fundamentally, it's their time; they can do whatever they want with > it. And looking to be paid for your time is of course perfectly > sensible. But if this desire to stay a fork is true, then their > primary goal is NOT protecting as many people as possible, it's > protecting a subset of people as perfectly as they can. Again, this is > fine. Upstream and grsecurity can coexist in this state (since again, > upstreaming doesn't increase their work), and this was the state for > over a decade. But suddenly grsecurity went private. > > So it would appear that grsecurity couldn't make enough money with > their patches staying public, so they've made them private. If that's > true, then it means that everyone who was using grsecurity but wasn't > a sponsor of grsecurity's work holds some level of responsibility for > grsecurity's decision. > > It could be argued that upstream is "using grsecurity but wasn't a > sponsor", but this would be missing an important point: upstream's > contribution to grsecurity is Linux itself, which has giant value. > (And for my part personally, I did everything I could to pay > grsecurity, and succeeded a couple times over the years.) With this we > have come back around to looking at the GPL, which you mentioned. This > is specifically what the GPL was created to protect against. Why > hasn't grsecurity paid money to the tens of thousands of contributors > to the Linux kernel? Because they don't have to, and that is part of > the understanding about using the GPL for a project: everyone > contributes what they can, and everyone benefits. And if there are > forks, people can use each other's work so no one is risking being in > a situation where someone is unfairly benefiting from someone else's > work. > > You had invoked RMS, so, here we are. It looks like grsecurity has > gone private because upstream started exercising the princples of the > GPL to expand how many upstream users would benefit from grsecurity's > work in the same way grsecurity's users benefited from upstream's > work. In answer to "who should I blame?" I'd say this is squarely on > grsecurity, not anyone else. They've chosen to benefit from upstream's > future work without making their future work available to upstream. > > But maybe this is all wrong. Neither of us can speak for grsecurity, > nor know their motivations. I've always tried to avoid these kinds of > discussions for that very reason, and to stay focused on the technical > work. I can only truly speak for myself. I was faced with a seemingly > simple problem: Linux users need to be protected, the vast majority of > Linux users run upstream-derived kernels, but many of the best > defenses lived in grsecurity. What I'd learned from working at an > industry group (OSDL, the pre-LF), working at a distro (Canonical on > Ubuntu), and now working on devices and services (Google), is that > engineering teams in most places won't risk basing their work on a > fork of upstream, whether it be support concerns, cost concerns, > complexity concerns, or whatever. At the end of the day, it was clear > that the only path forward was to focus upstream on the same defense > principles grsecurity espoused, and to then start porting or creating > these kinds of defenses in upstream. (Which is the very thing > grsecurity told upstream to do at the first Linux Security Summit in > 2010!) Doing this work was very time consuming, so I tried to collect > interested people together to work under a single umbrella to help > change the upstream culture, examine flaws, port or invent defenses, > test, document, etc. > Maybe you are right from your perspective. I appreciate what you've contributed to the FLOSS community since you were working for Canonical. Upstream is good but it's always raising the bar but never being good enough for the defensive side. It never bothers me when I worked for two towers of GNU/Linux vendor( SUSE/RH). It bothers me now and the desert of the real is happening every single day, while spend another years of waiting is not practical to those who concerns their security. Speaking of GPL, closing the public access isn't violate GPL technically. On the other hand, the authors of PaX/Grsecurity can do anything they want. It's their rights. To be honest, ppl who concerns security are minority. It's sad to see this group of ppl lost test patch. They shared their work with FLOSS world for 16 years. I don't think I have the rights to blame them. Anyway, I'd still blame LF. > So, in the end, I can't speak to grsecurity's true motivations, but I > can speak to mine, and what I'd like upstream to be doing: protecting > as many Linux users as completely as possible. What's possible in > upstream differs from what was possible in grsecurity, so no one > should expect the same things between the two. Comparing the two > projects is fraught with error, and blaming anyone other than > grsecurity for going private is ridiculous. > >> Agree! I'm afraid I can't help much on upstream because that's not my >> customer want in short-term. S0rry, my idealist side never fit in >> KSPP. > > Sure, understood. I'm delighted you've spent time getting kernel > hardening stuff landed in AOSP, though; for vendors that can be > convinced to care about backporting these things, it's great to have a > place to point to for them to get your work! And it's fine not to > contribute to upstream, since luckily there will be some people who > can. Upstream will continue to work on security defenses, but at the > very least, it'd be nice if people would be constructive in their > criticisms. We've gone over a decade with grsecurity's frequently > toxic rhetoric only alienating people instead of productively working > together to reach what should be the common goal of security work: > defending users. > Yeah, we have different battle to fight with. I truely hope KSPP can be an alternative mitigation solution in the future because the users could get benefit from open competitions. I think PaX team and Spender never fear to compete with any other defense solution. -- GNU powered it... GPL protect it... God blessing it... regards Shawn ^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [kernel-hardening] It looks like there will be no more public versions of PaX and Grsec. 2017-05-04 14:11 ` Shawn @ 2017-05-04 16:03 ` Greg KH 2017-05-04 17:12 ` Shawn 0 siblings, 1 reply; 35+ messages in thread From: Greg KH @ 2017-05-04 16:03 UTC (permalink / raw) To: Shawn Cc: Kees Cook, Rik van Riel, Mathias Krause, Daniel Cegiełka, kernel-hardening On Thu, May 04, 2017 at 10:11:04PM +0800, Shawn wrote: > That announcement only represented the POV from a group of ppl. From > my( and other ppl from HardenedLinux) perspective, Linux foundation is > a commercial company and very good at PR but zero integrity to us. A slight correction here please. The LF is a non-profit organization[1] set up to promote Linux and allow companies who want to see Linux succeed, get together and do this. The LF happens to sponsor a few kernel developers (me and Linus), but they can not tell us what to do at all. They also are a place that companies have come together to help with the state of security in the Linux and Open Source ecosystem, starting CII which offers grants to anyone who wants to get paid to do security work (new features, support, audits, etc.) CII doesn't make any money, it gives money away! Of course it does press releases saying what projects it funds in order to get other projects and people to submit project proposals to continue this work. I know of at least 2 new kernel security projects that recently got funding because of this. So there is no "integrity" that the LF can, or can not, have when it comes to anyone here as the LF doesn't actually _do_ anything when it comes to kernel development (again, other than funding 2 developers directly). > They don't respect individuals and the community. That's a load of crap, really. The LF has always had a kernel community developer as a full board member, and sponsors conferences, travel funding, hardware acquisition, intern programs, and lots of other stuff. I don't know of any kernel community request that the LF has _not_ funded, do you? The LF is all about making the whole community work well together, and that includes both individual developers and companies as this is a symbiotic relationship (companies use Linux, fund its development, create new hardware for Linux to run on, etc.) Without one part of the group, Linux would not succeed at all, and they know that quite well. If the LF didn't "individuals and the community", I know I wouldn't be working for them. So I don't know why anyone would be "upset" at the LF here, all they have done is actually fund people to do kernel security work, including members of the grsecurity team! How is doing that somehow "bad"? Do you want to go back to 2+ years ago when they were not doing this funding at all? And does no one remember how things were before there was a LF? Do you really want to go back to those days? Were they somehow better than things are now? As someone who remembers those times quite well, I can assure you that they were not. Sorry for the digression, greg k-h [1] Yes, it's structured as a trade organization, it has to be that way from a legal point of view in order for companies to be able to help Linux and work together. Without it, companies would be violating anti-trust laws and would not be able to help the community out at all. Think of the LF as the "Milk Advisory Board" for Linux. ^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [kernel-hardening] It looks like there will be no more public versions of PaX and Grsec. 2017-05-04 16:03 ` Greg KH @ 2017-05-04 17:12 ` Shawn 2017-05-04 17:23 ` Greg KH 0 siblings, 1 reply; 35+ messages in thread From: Shawn @ 2017-05-04 17:12 UTC (permalink / raw) To: Greg KH Cc: Kees Cook, Rik van Riel, Mathias Krause, Daniel Cegiełka, kernel-hardening Hi Greg On Fri, May 5, 2017 at 12:03 AM, Greg KH <gregkh@linuxfoundation.org> wrote: > On Thu, May 04, 2017 at 10:11:04PM +0800, Shawn wrote: >> That announcement only represented the POV from a group of ppl. From >> my( and other ppl from HardenedLinux) perspective, Linux foundation is >> a commercial company and very good at PR but zero integrity to us. > > A slight correction here please. The LF is a non-profit organization[1] > set up to promote Linux and allow companies who want to see Linux > succeed, get together and do this. The LF happens to sponsor a few > kernel developers (me and Linus), but they can not tell us what to do at > all. > Oh, that's new to me. LF is a non-profit organization. Maybe some "rumors" isn't true: I thought LF hired two "sales" people in HK last year? Can you confirm that? Otherwise, which NGO's runner has $344,220 salary per year? http://news.idg.no/cw/art.cfm?id=5A9F8343-BAA1-6432-72A26555784BF05E LF is growing so fast, look at that. Compensation is seems very promising even in 2013: https://projects.propublica.org/nonprofits/organizations/460503801 $499,705 is the total compensation in 2014? Wow..why this magic number? Because $500,000 would be excessive? http://pdfs.citizenaudit.org/2016_02_EO/46-0503801_990O_201412.pdf > They also are a place that companies have come together to help with the > state of security in the Linux and Open Source ecosystem, starting CII > which offers grants to anyone who wants to get paid to do security work > (new features, support, audits, etc.) CII doesn't make any money, it > gives money away! Of course it does press releases saying what projects > it funds in order to get other projects and people to submit project > proposals to continue this work. I know of at least 2 new kernel > security projects that recently got funding because of this. > > So there is no "integrity" that the LF can, or can not, have when it > comes to anyone here as the LF doesn't actually _do_ anything when it > comes to kernel development (again, other than funding 2 developers > directly). > I'm curious what's your offical relationship with Google? Cu'z you have this account( gregkh@google.com) from Google: https://android.googlesource.com/kernel/common.git/+/e88bb963b88d5579805b90e8d505739692095042 But you seems still working for LF, aren't you?: https://www.linuxfoundation.org/about/linux-foundation-fellows >> They don't respect individuals and the community. > > That's a load of crap, really. The LF has always had a kernel community > developer as a full board member, and sponsors conferences, travel > funding, hardware acquisition, intern programs, and lots of other stuff. > I don't know of any kernel community request that the LF has _not_ > funded, do you? > Really? That's also new to me. Let me give you some hints: The Linux Foundation: Not a Friend of Desktop Linux, the GPL, or Openness: http://fossforce.com/2017/04/lin-desktop-linux-gpl-openness/ OpenSSL after Heartbleed: https://lwn.net/Articles/703000/ Linux Foundation quietly drops community representation: https://mjg59.dreamwidth.org/39546.html > The LF is all about making the whole community work well together, and > that includes both individual developers and companies as this is a > symbiotic relationship (companies use Linux, fund its development, > create new hardware for Linux to run on, etc.) Without one part of the > group, Linux would not succeed at all, and they know that quite well. > > If the LF didn't "individuals and the community", I know I wouldn't be > working for them. > > So I don't know why anyone would be "upset" at the LF here, all they > have done is actually fund people to do kernel security work, including > members of the grsecurity team! How is doing that somehow "bad"? Do > you want to go back to 2+ years ago when they were not doing this > funding at all? > > And does no one remember how things were before there was a LF? Do you > really want to go back to those days? Were they somehow better than > things are now? As someone who remembers those times quite well, I can > assure you that they were not. > In my very "narrow" POV( only as FLOSS supporter and security consutlant), KSPP would be so much better without LF's involvement and PR -- GNU powered it... GPL protect it... God blessing it... regards Shawn ^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [kernel-hardening] It looks like there will be no more public versions of PaX and Grsec. 2017-05-04 17:12 ` Shawn @ 2017-05-04 17:23 ` Greg KH 0 siblings, 0 replies; 35+ messages in thread From: Greg KH @ 2017-05-04 17:23 UTC (permalink / raw) To: Shawn Cc: Kees Cook, Rik van Riel, Mathias Krause, Daniel Cegiełka, kernel-hardening On Fri, May 05, 2017 at 01:12:37AM +0800, Shawn wrote: > Hi Greg > > On Fri, May 5, 2017 at 12:03 AM, Greg KH <gregkh@linuxfoundation.org> wrote: > > On Thu, May 04, 2017 at 10:11:04PM +0800, Shawn wrote: > >> That announcement only represented the POV from a group of ppl. From > >> my( and other ppl from HardenedLinux) perspective, Linux foundation is > >> a commercial company and very good at PR but zero integrity to us. > > > > A slight correction here please. The LF is a non-profit organization[1] > > set up to promote Linux and allow companies who want to see Linux > > succeed, get together and do this. The LF happens to sponsor a few > > kernel developers (me and Linus), but they can not tell us what to do at > > all. > > > Oh, that's new to me. LF is a non-profit organization. Maybe some > "rumors" isn't true: I thought LF hired two "sales" people in HK last > year? Can you confirm that? And it's always been a non-profit, that should not be a surprise. And I don't have any idea what a "sales" person at the LF would do, perhaps the fact that the LF is hiring people to work with local companies to become members? Why not ask them what they do? > > So there is no "integrity" that the LF can, or can not, have when it > > comes to anyone here as the LF doesn't actually _do_ anything when it > > comes to kernel development (again, other than funding 2 developers > > directly). > > > I'm curious what's your offical relationship with Google? Cu'z you > have this account( gregkh@google.com) from Google: > https://android.googlesource.com/kernel/common.git/+/e88bb963b88d5579805b90e8d505739692095042 > > But you seems still working for LF, aren't you?: > https://www.linuxfoundation.org/about/linux-foundation-fellows What does my google account have to do with the LF? I've had that account for many many years now, to help Google out with kernel security work, including stable kernel releases. I have lots of other email addresses too, doesn't everyone? :) And yes, I still work for the LF. > In my very "narrow" POV( only as FLOSS supporter and security > consutlant), KSPP would be so much better without LF's involvement and > PR So the funding of grsecurity developers to get features merged into mainline was bad? That's a very strange viewpoint of "so much better". good luck! greg k-h ^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [kernel-hardening] It looks like there will be no more public versions of PaX and Grsec. 2017-05-02 0:09 ` Rik van Riel 2017-05-02 14:46 ` Shawn @ 2017-05-02 21:16 ` Mathias Krause 2017-05-02 21:50 ` Casey Schaufler ` (2 more replies) 1 sibling, 3 replies; 35+ messages in thread From: Mathias Krause @ 2017-05-02 21:16 UTC (permalink / raw) To: Rik van Riel; +Cc: Kees Cook, Daniel Cegiełka, kernel-hardening On 2 May 2017 at 02:09, Rik van Riel <riel@redhat.com> wrote: > On Tue, 2017-05-02 at 00:01 +0200, Mathias Krause wrote: > >> I think the intention of the KSPP is good -- making vanilla Linux >> more >> secure. But the way it does its work harms overall Linux security. It >> does hurt mine, that's for sure! > > Yeah, no. Well, yes, it does! Losing access to the grsecurity patch makes the systems I do care about much less secure. > The grsecurity people produced patches > that were used on maybe a few tens of thousands > of systems, Where did you pull that number from? Out of thin air, I guess. I know, for sure, there are many more installations. > while the KSPP code will end up > enhancing the security of over a billion Android > devices. Or making them more easily to DoS because features like VMAP_STACK and HARDENED_USERCOPY will likely fail hard when hitting a vendor's diver code base. Probably making them disable the problematic config options. Even upstream still has to fix related fallout. > Those Android devices are more likely to require > hardening, too, since they do not receive security > updates as quickly as the systems maintained by > grsecurity users. Why couldn't those devices benefit from grsecurity as well? Couldn't google or Samsung just integrate grsecurity into their Android kernels? They're far away from vanilla Linux anyway, so why not add just another patch to provide some matured security code base to protect those billion of Android devices? I'd guess, if a big player like google would sponsor / pay grsecurity to provide a patch for the relevant Android kernels, all sides would be happy: grsecurity for getting wider adoption, Android users for having secured systems. > Integrating hardening into the upstream kernel is > a good thing for security, not a bad thing. I never said it's a bad thing. Indeed I'm all for making vanilla Linux more secure. Just how KSPP tries to do it is IMHO wrong. Ripping hunks out of grsecurity and trying to integrate them into vanilla Linux without understanding all the interdependencies or even the features themselves, how would that provide security? By chance, maybe. But not intentional, as that requires having thought of every corner case and boundary condition. Regards, Mathias ^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [kernel-hardening] It looks like there will be no more public versions of PaX and Grsec. 2017-05-02 21:16 ` Mathias Krause @ 2017-05-02 21:50 ` Casey Schaufler 2017-05-02 22:57 ` Kees Cook 2017-05-03 19:02 ` Rik van Riel 2 siblings, 0 replies; 35+ messages in thread From: Casey Schaufler @ 2017-05-02 21:50 UTC (permalink / raw) To: Mathias Krause, Rik van Riel Cc: Kees Cook, Daniel Cegiełka, kernel-hardening On 5/2/2017 2:16 PM, Mathias Krause wrote: > On 2 May 2017 at 02:09, Rik van Riel <riel@redhat.com> wrote: >> On Tue, 2017-05-02 at 00:01 +0200, Mathias Krause wrote: >> >>> I think the intention of the KSPP is good -- making vanilla Linux >>> more >>> secure. But the way it does its work harms overall Linux security. It >>> does hurt mine, that's for sure! >> Yeah, no. > Well, yes, it does! Losing access to the grsecurity patch makes the > systems I do care about much less secure. Then pay for support. >> The grsecurity people produced patches >> that were used on maybe a few tens of thousands >> of systems, > Where did you pull that number from? Out of thin air, I guess. I know, > for sure, there are many more installations. What number are you suggesting, and where did you get it? >> while the KSPP code will end up >> enhancing the security of over a billion Android >> devices. > Or making them more easily to DoS because features like VMAP_STACK and > HARDENED_USERCOPY will likely fail hard when hitting a vendor's diver > code base. Probably making them disable the problematic config > options. Even upstream still has to fix related fallout. So you're saying that didn't happen when vendors tried to apply grsecurity patches? >> Those Android devices are more likely to require >> hardening, too, since they do not receive security >> updates as quickly as the systems maintained by >> grsecurity users. > Why couldn't those devices benefit from grsecurity as well? Couldn't > google or Samsung just integrate grsecurity into their Android > kernels? They're far away from vanilla Linux anyway, so why not add > just another patch to provide some matured security code base to > protect those billion of Android devices? I'd guess, if a big player > like google would sponsor / pay grsecurity to provide a patch for the > relevant Android kernels, all sides would be happy: grsecurity for > getting wider adoption, Android users for having secured systems. What makes you think that the "big players" didn't carefully consider using grsecurity? And calling grsecurity "just another patch" is like the Black Knight referring to his missing limbs as "flesh wounds". Grsecurity isn't a thing. It's a jumble of collections of tweaks and clever checks. Without some amount of breaking out the implementations into discrete features there's no way anyone doing long term upstream community maintenance was ever going to be happy. >> Integrating hardening into the upstream kernel is >> a good thing for security, not a bad thing. > I never said it's a bad thing. Indeed I'm all for making vanilla Linux > more secure. Just how KSPP tries to do it is IMHO wrong. Ripping hunks > out of grsecurity and trying to integrate them into vanilla Linux > without understanding all the interdependencies or even the features > themselves, how would that provide security? By chance, maybe. But not > intentional, as that requires having thought of every corner case and > boundary condition. I seriously doubt that grsecurity is as wonderful as you seem convinced it is. The fact that one can't identify independent features or their interdependencies unless you're one of the authors does not speak well for it. No, much better to examine the flying saucer at Roswell and use what you've learned to build understandable components than to "just clone" the starship. > Regards, > Mathias > ^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [kernel-hardening] It looks like there will be no more public versions of PaX and Grsec. 2017-05-02 21:16 ` Mathias Krause 2017-05-02 21:50 ` Casey Schaufler @ 2017-05-02 22:57 ` Kees Cook 2017-05-03 19:02 ` Rik van Riel 2 siblings, 0 replies; 35+ messages in thread From: Kees Cook @ 2017-05-02 22:57 UTC (permalink / raw) To: Mathias Krause; +Cc: Rik van Riel, Daniel Cegiełka, kernel-hardening On Tue, May 2, 2017 at 2:16 PM, Mathias Krause <minipli@googlemail.com> wrote: > On 2 May 2017 at 02:09, Rik van Riel <riel@redhat.com> wrote: >> while the KSPP code will end up >> enhancing the security of over a billion Android >> devices. > > Or making them more easily to DoS because features like VMAP_STACK and > HARDENED_USERCOPY will likely fail hard when hitting a vendor's diver > code base. Probably making them disable the problematic config > options. Even upstream still has to fix related fallout. A vendor who would disable HARDENED_USERCOPY would never have integrated grsecurity, so that's a totally false equivalency. I don't disagree that VMAP_STACK might have been implemented in a way to better catch driver mistakes, but the condition was deemed rare enough that it got implemented the way it did. You may object to the difference in engineering practices, but pragmatism comes in many flavors. And no one else developed vmap stacks for upstream, so I'd rather have it than not. >> Those Android devices are more likely to require >> hardening, too, since they do not receive security >> updates as quickly as the systems maintained by >> grsecurity users. > > Why couldn't those devices benefit from grsecurity as well? Couldn't > google or Samsung just integrate grsecurity into their Android > kernels? They're far away from vanilla Linux anyway, so why not add > just another patch to provide some matured security code base to > protect those billion of Android devices? I'd guess, if a big player The very fact that you're asking this question means you don't understand how vendors deal with the kernel. With a small engineering team, you can't afford to have changes from upstream you can't support, so you leave as much of the kernel at stock upstream as possible so you can get help from upstream when you need it. Now, ironically, these same vendors don't realize that the moment their kernel ages a few years, they're on the hook for supporting the ENTIRE kernel again, since upstream has left that kernel in the dust. Those organizations can't justify the resources needed to support an out-of-tree kernel as their starting point. Chrome OS was probably one of the most paranoid OS designs in a commercially available product and we still couldn't get agreement to use grsecurity. (Though I would point out support isn't the only issue: another is the risk of the fork disappearing. *cough*) > like google would sponsor / pay grsecurity to provide a patch for the > relevant Android kernels, all sides would be happy: grsecurity for > getting wider adoption, Android users for having secured systems. And yet this never happened, and so the only way to get the defenses to the general public was the upstream them. Which is something grsecurity refused to do, even when offered money to do it. >> Integrating hardening into the upstream kernel is >> a good thing for security, not a bad thing. > > I never said it's a bad thing. Indeed I'm all for making vanilla Linux > more secure. Just how KSPP tries to do it is IMHO wrong. Ripping hunks > out of grsecurity and trying to integrate them into vanilla Linux > without understanding all the interdependencies or even the features > themselves, how would that provide security? By chance, maybe. But not > intentional, as that requires having thought of every corner case and > boundary condition. So what's the solution? Give up? No. Want upstream to be some how better at porting or developing defenses? Okay, help us. "Do it better" isn't a particularly useful suggestion. Besides, the defenses aren't opaque, even if they're virtually undocumented. They can be extracted, studied, ported, and landed. For the defenses that we're interested in that came from grsecurity, this doesn't seem like a bad approach. If you have expertise in some of the areas that are being upstreamed, speak up and contribute. Want something to be upstreamed to your liking? Please do it. -Kees -- Kees Cook Pixel Security ^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [kernel-hardening] It looks like there will be no more public versions of PaX and Grsec. 2017-05-02 21:16 ` Mathias Krause 2017-05-02 21:50 ` Casey Schaufler 2017-05-02 22:57 ` Kees Cook @ 2017-05-03 19:02 ` Rik van Riel 2017-05-03 19:27 ` Daniel Micay 2 siblings, 1 reply; 35+ messages in thread From: Rik van Riel @ 2017-05-03 19:02 UTC (permalink / raw) To: Mathias Krause; +Cc: Kees Cook, Daniel Cegiełka, kernel-hardening [-- Attachment #1: Type: text/plain, Size: 3095 bytes --] On Tue, 2017-05-02 at 23:16 +0200, Mathias Krause wrote: > On 2 May 2017 at 02:09, Rik van Riel <riel@redhat.com> wrote: > > On Tue, 2017-05-02 at 00:01 +0200, Mathias Krause wrote: > > > > > I think the intention of the KSPP is good -- making vanilla Linux > > > more > > > secure. But the way it does its work harms overall Linux > > > security. It > > > does hurt mine, that's for sure! > > > > Yeah, no. > > Well, yes, it does! Losing access to the grsecurity patch makes the > systems I do care about much less secure. Your systems are not necessarily representative of the overall security of all Linux systems in the world, though. > > The grsecurity people produced patches > > that were used on maybe a few tens of thousands > > of systems, > > Where did you pull that number from? Out of thin air, I guess. I > know, > for sure, there are many more installations. I have seen a few emails suggesting there are millions of systems with grsecurity patches around, and I got the number wrong by several orders of magnitude. However, that does not invalidate my original point: the number of grsecurity protected systems is only a tiny fraction of the entire Linux installed base. It would be good to get hardening functionality upstream, in order to get a majority of Linux systems (that get deployed in the future) hardened out of the box. > > Those Android devices are more likely to require > > hardening, too, since they do not receive security > > updates as quickly as the systems maintained by > > grsecurity users. > > Why couldn't those devices benefit from grsecurity as well? Couldn't > google or Samsung just integrate grsecurity into their Android > kernels? They're far away from vanilla Linux anyway, so why not add > just another patch to provide some matured security code base to > protect those billion of Android devices? I'd guess, if a big player > like google would sponsor / pay grsecurity to provide a patch for the > relevant Android kernels, all sides would be happy: grsecurity for > getting wider adoption, Android users for having secured systems. They could, but the reality is many vendors will not bother enabling hardening features unless the code is already there and enabled by default in the upstream kernel. > > Integrating hardening into the upstream kernel is > > a good thing for security, not a bad thing. > > I never said it's a bad thing. Indeed I'm all for making vanilla > Linux > more secure. Just how KSPP tries to do it is IMHO wrong. Ripping > hunks > out of grsecurity and trying to integrate them into vanilla Linux > without understanding all the interdependencies or even the features > themselves, how would that provide security? By chance, maybe. But > not > intentional, as that requires having thought of every corner case and > boundary condition. How would you do it? Maintainers integrate code one patch series at a time. That is not a constraint you can work around, because code does need to be reviewed. -- All rights reversed [-- Attachment #2: This is a digitally signed message part --] [-- Type: application/pgp-signature, Size: 473 bytes --] ^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [kernel-hardening] It looks like there will be no more public versions of PaX and Grsec. 2017-05-03 19:02 ` Rik van Riel @ 2017-05-03 19:27 ` Daniel Micay 0 siblings, 0 replies; 35+ messages in thread From: Daniel Micay @ 2017-05-03 19:27 UTC (permalink / raw) To: Rik van Riel, Mathias Krause Cc: Kees Cook, Daniel Cegiełka, kernel-hardening [-- Attachment #1: Type: text/plain, Size: 1425 bytes --] > Maintainers integrate code one patch series at a > time. That is not a constraint you can work around, > because code does need to be reviewed. By not requiring fixes for issues like undefined function pointer usage or other undefined behavior to be split up in hundreds of patches to go through many different maintainers / many different trees. The same goes for things like basic constification (i.e. const and __ro_after_init) that are clear cut and don't involve a performance compromise (so an equivalent to pax_{open_close}_kernel would be different). It's extremely unrealistic to get type-based Control Flow Integrity like RAP for the mainline kernel if the fixes cannot be queued up in a single tree. It also deters people from working on any of these small, incremental improvements since they need to split it all up and try to deal with dozens of maintainers, with most of the patches being lost. See what happened to past attempts at this stuff. Of course it needs to be reviewed and hopefully tested, but for fixing clear cases of undefined behavior (i.e. those caught by UBSan or other sanitizers and not covered by flags like -fno-strict-overflow -fno- strict-aliasing) or uncontroversial things like missing const (without other code changes beyond making pointers const), it should be possible to land fixes via one tree. It isn't an issue that's specific to security patches. [-- Attachment #2: This is a digitally signed message part --] [-- Type: application/pgp-signature, Size: 866 bytes --] ^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [kernel-hardening] It looks like there will be no more public versions of PaX and Grsec. 2017-05-01 22:01 ` Mathias Krause 2017-05-02 0:09 ` Rik van Riel @ 2017-05-02 0:39 ` Olof Johansson 2017-05-02 0:44 ` Casey Schaufler ` (2 subsequent siblings) 4 siblings, 0 replies; 35+ messages in thread From: Olof Johansson @ 2017-05-02 0:39 UTC (permalink / raw) To: Mathias Krause; +Cc: Kees Cook, Daniel Cegiełka, kernel-hardening On Mon, May 1, 2017 at 3:01 PM, Mathias Krause <minipli@googlemail.com> wrote: > The first point directly forces them to think through the upstream > incarnation of a feature, how it differs from their original one and, > for those parts, how to fix them up to work properly to fit their > needs, to fit their security requirements. Those changes generate a > lot of conflicts with their version of the code which takes time to > fiddle out. As happened for __ro_after_init, MEMORY_SANITIZE, > USERCOPY,... which only went in in reduced and modified form, > generating needless work on their side -- from their point of view. This is the classic pain point of initial upstreaming when you have been out-of-tree for a long time. We see this all the time for hardware support as well. It's a hump to climb, and once you're over it, things settle down again. The quicker you can get past it, the better off you are. > The second point, the loose of control over the code, is even worse. > Not getting any more conflicts when porting the grsecurity/PaX patch > to a new kernel release makes changes to code that used to live in > grsecurity/PaX probably go unnoticed. And, as it seem to be the case, > upstream developers are not always familiar with all the gory details > and might introduce weaknesses and bugs. This not only makes upstream > Linux less secure, it makes grsecurity and PaX less secure, too. The solution to this is to get involved and review code and educate your peers when they're doing the wrong thing. Linux is developed and managed based on influence, not control. Influence requires participation and interaction. -Olof ^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [kernel-hardening] It looks like there will be no more public versions of PaX and Grsec. 2017-05-01 22:01 ` Mathias Krause 2017-05-02 0:09 ` Rik van Riel 2017-05-02 0:39 ` Olof Johansson @ 2017-05-02 0:44 ` Casey Schaufler 2017-05-02 0:54 ` Kees Cook 2017-05-02 11:11 ` David Gens 4 siblings, 0 replies; 35+ messages in thread From: Casey Schaufler @ 2017-05-02 0:44 UTC (permalink / raw) To: Mathias Krause, Kees Cook; +Cc: Daniel Cegiełka, kernel-hardening On 5/1/2017 3:01 PM, Mathias Krause wrote: > On 27 April 2017 at 00:04, Kees Cook <keescook@chromium.org> wrote: >> On Wed, Apr 26, 2017 at 2:05 PM, Daniel Cegiełka >> <daniel.cegielka@gmail.com> wrote: >>> https://grsecurity.net/passing_the_baton_faq.php >> Yeah, I'm sad to see them go. PaX Team was just last night helping >> with some details of PAX_REFCOUNT as it could appear in the upstream >> refcount_t API. I hope they'll still help out from time to time. >> >> It does underscore the critical need to upstream stuff, though. Forks >> of projects might disappear at any time. :( > Now, since grsecurity and PaX went private, quite a few users > (including me) are left in the dark and are, to say the least, > slightly pissed. But not everybody seems to be barking at the right > tree. So I've a few comments and questions for the KSPP. > > I think the main reason for Brad and PaX Team to make their work > private is the increased amount of work KSSP has put on them without > providing any valuable work in return. They just don't want to be > forced to maintain and fix-up the variants of grsecurity/PaX features > KSPP lands upstream. And no, the work of KSPP does not make their life > easier, in fact, it makes it harder. Harder for two reasons: First, > the code does not end up verbatim, it's always changed, taken out of > context and "enhanced". Second is the loose of control over the code. > But let me elaborate those two a little further. I can't speak for Brad and the PaX team, and I certainly can't claim to understand their motivation for abandoning the community they had built. I can say that maintaining a fork of the kernel for any purpose is a lot of work, and keeping a patch set as large and complex as grsecurity up to date is no mean feat. This would be true whether KSPP were active or not. As anyone who does kernel development outside of an isolated area can tell you, there's always someone messing up everything. There's good reason that code hasn't often been taken from grsecurity verbatim. Not the least of these is that teasing a particular "feature" out of the public patches is not easy. Another is that maintainers of other aspects of the system often dislike the affect the grsecurity code has on the aspect near and dear to their hearts, and won't accept the "value of security" argument. KSPP is committed to upstream. KSPP has to make the trade-offs grsecurity was never motivated to. > The first point directly forces them to think through the upstream > incarnation of a feature, how it differs from their original one and, > for those parts, how to fix them up to work properly to fit their > needs, to fit their security requirements. Those changes generate a > lot of conflicts with their version of the code which takes time to > fiddle out. As happened for __ro_after_init, MEMORY_SANITIZE, > USERCOPY,... which only went in in reduced and modified form, > generating needless work on their side -- from their point of view. > > The second point, the loose of control over the code, is even worse. > Not getting any more conflicts when porting the grsecurity/PaX patch > to a new kernel release makes changes to code that used to live in > grsecurity/PaX probably go unnoticed. And, as it seem to be the case, > upstream developers are not always familiar with all the gory details > and might introduce weaknesses and bugs. This not only makes upstream > Linux less secure, it makes grsecurity and PaX less secure, too. > > So I can understand why they've done this. Still, with the loose of > the public availability of the patch, Linux security has suffered a > lot. Not only is upstream far far away to reach a level that is > available today in grsecurity and PaX, no, it also won't benefit from > new developments any more. What a great achievement! :( You are confusing "Linux security" with "grsecurity (tm) security". grsecurity is a commercial product. Brad Spender has made it very clear that this is how he wants it viewed (Brad, feel free to correct me) and that is his call. Linux is a community project. As such, Linux has never before had the facilities KSPP is putting in now. Similar features have been available in a commercial fork, and thanks to the terms of the GPLv2, we have access to that code. > *sigh* > > I think the intention of the KSPP is good -- making vanilla Linux more > secure. But the way it does its work harms overall Linux security. No. Again, you are confusing a commercial product based on Linux with Linux. > It > does hurt mine, that's for sure! I know the value of grsecurity and > PaX in particular and know how easy it still is to exploit a vanilla > Linux. Features like KERNEXEC and RAP make it almost impossible for an > attacker to abuse a memory corruption bug in the kernel. Many unnamed > features -- unnamed because not under an #ifdef -- make exploiting > use-after-free bugs much less interesting or reduce the race window > for TOCTTOU bugs involving user copy operations. None of this can be > found in vanilla Linux. Probably never will... Patches welcome! > So, here's my list of questions for the KSPP: > 1/ When will I be able to switch to a vanilla Linux kernel that is > equivalently hardened as a grsecurity/PaX kernel used to be? Can't answer that. I spent months trying to unravel USERCOPY from the rest of the patch and can safely say that there's so much going on in grsecurity that I would never believe someone who told me they could identify all the features. > 2/ Who will maintain this code and how? Most often the authors of code will maintain it, but we've already seen cases where architecture specialists have begun treating KSPP code as their own. How? The usual community spirit of keeping things working. > 3/ Who ensures the coverage and quality won't suffer for each new > kernel release? Do you understand how community works? > > Judging from the planed EOL for the v4.9 LTS kernel -- the last one a > grsecurity patch was publicly available for --, there are less than > two years to finish the work for item 1 to ensure a secure and smooth > transition from grsecurity to upstream Linux. Will the KSPP be able to > achieve this?... I have my doubts. Grsecurity was a fork, and part of the reason for that is that parts of what is in it where not acceptable to other members of the community. Support of what is in a fork when the fork goes away is one reason not to rely on forks. > Even if so, item 3 will be a hard problem to solve as the Linux kernel > development model does not support such a tree-wide review point in > the release cycle. Assuming there are people able and willing to judge > and review the code base for required changes (e.g. atomic_t -> > refcount_t changes or function pointer cleanups for RAP), how would > those be able to simply get in those changes at, say, time of rc5? -- > without having to argue with a horde of maintainers of the individual > subsystems involved? You *always* have to argue with a horde of maintainers! That's what we do! Drop in the refcount_t change without getting buy-in from Al and David? I think not. > Quite a lot of questions. The external patch solved them all by not > having to deal with upstream Linux development and not having the code > available until it's ready. But now it's gone and no adequate > replacement is on the horizon. What will KSPP do about it? I had a > reasonably secure kernel, now it's gone. :( Pay Mr. Spender. Contribute patches to KSPP. > > > Regards, > Mathias > ^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [kernel-hardening] It looks like there will be no more public versions of PaX and Grsec. 2017-05-01 22:01 ` Mathias Krause ` (2 preceding siblings ...) 2017-05-02 0:44 ` Casey Schaufler @ 2017-05-02 0:54 ` Kees Cook 2017-05-11 1:24 ` PaX Team 2017-05-02 11:11 ` David Gens 4 siblings, 1 reply; 35+ messages in thread From: Kees Cook @ 2017-05-02 0:54 UTC (permalink / raw) To: Mathias Krause; +Cc: Daniel Cegiełka, kernel-hardening On Mon, May 1, 2017 at 3:01 PM, Mathias Krause <minipli@googlemail.com> wrote: > On 27 April 2017 at 00:04, Kees Cook <keescook@chromium.org> wrote: >> On Wed, Apr 26, 2017 at 2:05 PM, Daniel Cegiełka >> <daniel.cegielka@gmail.com> wrote: >>> https://grsecurity.net/passing_the_baton_faq.php >> >> Yeah, I'm sad to see them go. PaX Team was just last night helping >> with some details of PAX_REFCOUNT as it could appear in the upstream >> refcount_t API. I hope they'll still help out from time to time. >> >> It does underscore the critical need to upstream stuff, though. Forks >> of projects might disappear at any time. :( > > Now, since grsecurity and PaX went private, quite a few users > (including me) are left in the dark and are, to say the least, > slightly pissed. But not everybody seems to be barking at the right > tree. So I've a few comments and questions for the KSPP. First, let me adjust language here a bit to make things more clear. Talking about KSPP is like talking about net-dev: it is upstream, not a separate fork. It has a focus, like net-dev, but it is upstream. So there is no difference semantically between talking about KSPP and upstream. Additionally, while PaX Team, grsecurity, and ephox's work were technically separate development efforts (for example, just look at how PAX_USERCOPY differed between PaX and grsecurity), most people used the combination, which I'll just call grsecurity here. Both definitions will be helpful below... > I think the main reason for Brad and PaX Team to make their work > private is the increased amount of work KSSP has put on them without > providing any valuable work in return. They just don't want to be First, to think they didn't get valuable work in return from upstream is missing the forest for the trees. With every release of upstream, grsecurity would get tens of thousands of commits. If there wasn't benefit in these changes, grsecurity would never forward port to the latest upstream. The fact that it is a notable event that grsecurity has ceased updating their public patches is because people using grsecurity suddenly aren't getting the upstream changes, in addition to them not getting new grsecurity features. It is a totally false equivalency to say "upstream has [created work for|used code from] grsecurity without giving anything valuable in return". I'll come back to this later. Second, when HARDENED_USERCOPY was landing, I specifically asked grsecurity about the best way to make the upstreaming efforts easiest to deal with for them, since I knew there might be some work with their forward porting, and I didn't want to create undue work for them. They ignored me. I asked again a while later and was continued to be met with silence. Looking at the results in grsecurity, though, it's clear that they chose to integrate with upstream instead of maintaining a forked implementation. For example, the core PAX_USERCOPY logic lived in fs/exec.c. When I upstreamed it, I moved it to a place that made more sense in mm/usercopy.c. When grsecurity forward ported, they could have trivially left HARDENED_USERCOPY disabled, and kept PAX_USERCOPY using fs/exec.c, but they didn't. Since grsecurity has talked in the past about how their implementations are frequently arranged to make their forward porting easier, I think this is strong evidence that using upstream's implementation (with whatever further changes they wanted) was easier than keeping their own fork. This is further supported by grsecurity being paid by CII to upstream the gcc plugin infrastructure, where they used almost entirely the same paths, Kconfigs, and Makefile changes. So, given that grsecurity did not communicate at all about wanting forward porting made easier and that they've reused gcc plugins, hardened usercopy, etc, I think it's an entirely false claim that upstream is creating more work that normal forward porting. In some places, it even appears to be easier. Finally, even if you can somehow disregard the thousands of upstream changes benefiting grsecurity, and just want to look at the areas touched while upstreaming things from grsecurity, they have benefited from the upstreaming review by upstream finding bugs in various grsecurity features (some of which haven't even landed in upstream yet: Arnd Bergmann alone found tons of issues with the initify plugin and grsecurity fixed them). Hardened usercopy found bugs in grsecurity's slab implementation, gained strncpy_from_user() coverage, and triggered a massive consolidation of per-arch uaccess fixes which have found numerous upstream bugs that will now be fixed in grsecurity too. The current refcount_t work based on PAX_REFCOUNT uncovered a crash bug in grsecurity's implementation as well as a corner case in bounds checking. Upstreaming grsecurity features brings a huge amount of testing to bear on the code, which, like all the other things in upstream, grsecurity directly benefits from. And to top all of this off, while upstreaming the latent_entropy plugin, I noticed a English typo that was present in all the grsecurity gcc plugins, and when I sent them a trivial spelling fix patch for all their plugins (rather than just letting it stand and making work for them to catch it during forward porting and fix it everywhere else), they publicly mocked the patch (which they applied). So, no, unless you count fixing newly discovered bugs in their code, I again summarily reject the notion that upstreaming grsecurity features creates "more work for grsecurity without value in return". > forced to maintain and fix-up the variants of grsecurity/PaX features > KSPP lands upstream. And no, the work of KSPP does not make their life > easier, in fact, it makes it harder. Harder for two reasons: First, > the code does not end up verbatim, it's always changed, taken out of > context and "enhanced". Second is the loose of control over the code. > But let me elaborate those two a little further. See above. In many cases, they chose integration over maintaining a fork. In others, the two features coexist (for example, to various levels of needless redundancy, see arch/arm/mm/init.c's KERNEXEC overlap with RODATA). > The first point directly forces them to think through the upstream > incarnation of a feature, how it differs from their original one and, > for those parts, how to fix them up to work properly to fit their > needs, to fit their security requirements. Those changes generate a > lot of conflicts with their version of the code which takes time to > fiddle out. As happened for __ro_after_init, MEMORY_SANITIZE, > USERCOPY,... which only went in in reduced and modified form, > generating needless work on their side -- from their point of view. One of the costs of forking a project is dealing with forward porting. This problem is faced by all kinds of organizations (usually with small engineering teams), where some set of features is desired on top of Linux, and there are three choices for handling it, in ascending order of difficulty: - pick a kernel version, fork, develop features, use, abandon, repeat (see most IoT vendors) - pick a kernel version, fork, develop features, use, forward port to next kernel version, repeat (see most phone vendors) - develop features, upstream, use whatever future kernel desired (see most CPU vendors and distros) Upstreaming can be extremely time consuming. Grsecurity made it clear from long ago that they had no intention of upstreaming things because they wanted to use their time differently. This is entirely their choice, and one that, frankly, the vast majority of Linux forks make. But it means accepting the literally unending work of forward porting. That said, when I asked grsecurity if there I was any way they would accept payment to upstream things, they did briefly agree and upstreamed the gcc plugin infrastructure. So they were willing to upstream if paid, yet ultimately decided to stop, and to continue to forward port their patches. But now they made their work private, which, I should point out, doesn't change the amount of work for them, except maybe no longer getting bug reports from their users. To your specific examples, __ro_after_init is literally a one-line change: they just make in __read_only. This was some of my first attempts to make their forward porting work easy while upstream slowly incorporated features. The work around PAX_MEMORY_SANITIZE made the slab debug paths faster for everyone. I already talked about usercopy. So, neither of us can speak for grsecurity, but I reject your belief that upstream has somehow created needless work for them. Which brings me to a question I haven't seen anyone ask yet: why does grsecurity exist? If it was created to showcase strong security mechanisms to benefit Linux users, then why fight upstreaming? Why appear to abandon their users as soon as some of their work starts entering upstream? The number of users protected by these features has gone up by orders of magnitude, benefiting all Linux users, not just grsecurity's users. One could take the position that grsecurity desires to be differentiated from upstream so that they can point at how more protected grsecurity users are over upstream users, but as the gap shrinks, it will become harder to see the benefits. But that would seem to be still a long way down the road, given how time consuming upstreaming security features can be (regardless of whether they come from grsecurity or not). So if they just want to be better than upstream without caring about how many users are protected, then it's not hard to imagine grsecurity taking their patches private. I don't know that this is the actual rationale, but it really doesn't look to me like they want to protect as many Linux users as possible. And if protecting users isn't their goal, what is? > The second point, the loose of control over the code, is even worse. > Not getting any more conflicts when porting the grsecurity/PaX patch > to a new kernel release makes changes to code that used to live in > grsecurity/PaX probably go unnoticed. And, as it seem to be the case, > upstream developers are not always familiar with all the gory details > and might introduce weaknesses and bugs. This not only makes upstream > Linux less secure, it makes grsecurity and PaX less secure, too. You're speaking entirely in theoreticals, but I understand what you're trying to say. All forward porting runs this risk. Kernel internals change in ways that threaten even unchanged grsecurity features. To bring this down to earth, I would ask "how does grsecurity perform testing?" I can point to the many ways how upstream performs testing, including LKDTM for several of the security-sensitive features. As seen in the PAX_REFCOUNT porting work, the 4.9 grsecurity patch was clearly never actually tested since _any_ exercise of the PAX_REFCOUNT protection would Oops the kernel. I'm not saying they don't have tests, nor am I saying they didn't just make a one-time mistake, but I can point you directly to how upstream tests, how upstream developers find bugs in grsecurity features, and how things can improve once upstreamed (again, for example, the massive uaccess consolidation). As to familiarity with code, this is part of upstreaming. Upstreaming isn't just throwing a patch at the wall and running away: it needs to have an understandable and compelling changelog, it needs to clearly implemented and easy to maintain, etc. This differs greatly from grsecurity changes which were rarely commented, had nearly unusable changelogs, but only needed to be understood by a couple people. What would happen to grsecurity if one of their developers decided to give up on technology and go live in the woods? I would argue that features in upstream are significantly more resilient to bad things happening, but, we're now back where we started: a hard to prove opinion. > So I can understand why they've done this. Still, with the loose of > the public availability of the patch, Linux security has suffered a > lot. Not only is upstream far far away to reach a level that is > available today in grsecurity and PaX, no, it also won't benefit from > new developments any more. What a great achievement! :( Yeah, I'm quite disappointed too. When grsecurity talked at the first Linux Security Summit in 2010, they detailed a "ten year todo list" for upstream, itemizing all the features grsecurity had that upstream needed. Over the years I took them at their word that they wanted to see upstream improved but that they didn't have time to be bothered with upstreaming. And even finding money for upstreaming didn't really help things. And now that the needle has finally moved, and things are starting to actually land upstream, they go private. I thought they wanted to protect Linux users but not at the cost of upstreaming (which, again, I can certainly understand: it can be very hard). But now they've gone private. To me, that says I misunderstood their intentions from the very start. I'm glad to see that folks from the Hardened Gentoo project have started organizing to upstream grsecurity features too. I'm excited to find more people willing to risk their sanity and bring stuff upstream: https://wiki.gentoo.org/wiki/Hardened/Hardened_Kernel_Project > *sigh* > > I think the intention of the KSPP is good -- making vanilla Linux more > secure. But the way it does its work harms overall Linux security. It > does hurt mine, that's for sure! I know the value of grsecurity and I think I have thoroughly disproved this position. Something that I think is hard to see for people not involved in day-to-day upstream work is how very different the development workflows are between upstream and grsecurity. Upstream is normally evolutionary, doing things in easy to digest pieces, where as grsecurity could just land massive changes between releases. This makes it look like upstreaming is just landing tiny bits of features, instead of looking at the long view over time. Look at how __ro_after_init (while, yes, not being anywhere close to __read_only) has been slowly gaining ground in upstream. Look at how maybe one gcc plugin appears every couple of releases, or how usercopy continues to expand coverage. > PaX in particular and know how easy it still is to exploit a vanilla > Linux. Features like KERNEXEC and RAP make it almost impossible for an > attacker to abuse a memory corruption bug in the kernel. Many unnamed > features -- unnamed because not under an #ifdef -- make exploiting > use-after-free bugs much less interesting or reduce the race window > for TOCTTOU bugs involving user copy operations. None of this can be > found in vanilla Linux. Probably never will... These are all complaints to be made to grsecurity. They are the ones who provides those features originally, did not upstream them, and then took them away. As far as never finding them in upstream, I would remind you that grsecurity, while very impressive, is not the single source for all Linux kernel security improvements. For example, ARM created the PAN emulation code, HP is working on XPFO, etc. How much further along would upstream be with security features if you yourself had started sending patches like I've asked you repeatedly over the years? I wasn't being glib; I was being genuine. If you have time to criticize, you have time to write a patch. That said, I shouldn't give you too hard a time: you have 5 times the number of commits in the kernel than grsecurity has. > So, here's my list of questions for the KSPP: > 1/ When will I be able to switch to a vanilla Linux kernel that is > equivalently hardened as a grsecurity/PaX kernel used to be? How could anyone answer that question? I can't see the future. And besides, it's not like grsecurity was providing comprehensive protections. If you're using arm64, you might feel like you're already in a better position with upstream (got PAN emu, got hardened usercopy, still no RAP). If you're on x86_64, you might feel like you've very far (no SMAP emu, no RAP). Upstream's goal is protecting as many people as possible. > 2/ Who will maintain this code and how? > 3/ Who ensures the coverage and quality won't suffer for each new > kernel release? The upstream development community, just like everything else. I answered both of these already above. > Judging from the planed EOL for the v4.9 LTS kernel -- the last one a > grsecurity patch was publicly available for --, there are less than > two years to finish the work for item 1 to ensure a secure and smooth > transition from grsecurity to upstream Linux. Will the KSPP be able to > achieve this?... I have my doubts. Again, it depends on your perspective. And if you have doubts, then be part of the solution. What would you need to do to achieve your goal? > Even if so, item 3 will be a hard problem to solve as the Linux kernel > development model does not support such a tree-wide review point in > the release cycle. Assuming there are people able and willing to judge > and review the code base for required changes (e.g. atomic_t -> > refcount_t changes or function pointer cleanups for RAP), how would > those be able to simply get in those changes at, say, time of rc5? -- > without having to argue with a horde of maintainers of the individual > subsystems involved? This is an upstream development workflow problem that is especially frustrating for security work since it can regularly touch lots of areas, but it's one that I've been working to resolve. It's actually on my list of things to discuss at the next Kernel Summit, since making these wide changes is really irritating if it's going in via each maintainer. This doesn't have to be the case, but that's how I've been doing it lately since I can work on those pieces while waiting for reviews on other things. And in the case of refcount_t, the arduous work has provided some important discussions and reviews of existing refcounting in the kernel, so I can't say this workflow is entirely without merit. > Quite a lot of questions. The external patch solved them all by not > having to deal with upstream Linux development and not having the code > available until it's ready. But now it's gone and no adequate > replacement is on the horizon. What will KSPP do about it? I had a > reasonably secure kernel, now it's gone. :( Upstream was never in the business of providing the grsecurity patch, so you'll need to ask them why they abandoned their users. Upstream's work is unchanged. We'll still continue to work to bring security features into the kernel, and we'll continue to welcome new contributors. -Kees -- Kees Cook Pixel Security ^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [kernel-hardening] It looks like there will be no more public versions of PaX and Grsec. 2017-05-02 0:54 ` Kees Cook @ 2017-05-11 1:24 ` PaX Team 2017-05-11 16:30 ` Daniel Micay ` (2 more replies) 0 siblings, 3 replies; 35+ messages in thread From: PaX Team @ 2017-05-11 1:24 UTC (permalink / raw) To: Mathias Krause, Kees Cook; +Cc: Daniel Cegielka, kernel-hardening i think i'm not the only one to notice that the past two weeks have been an interesting experience to say the least. ever since our announcement about the fate of our test patch series we've been silent but not because we had nothing to say (as you've no doubt guessed by now, this mail is it). rather, we've used the opportunity the announcement created to see just what comes out of the woodwork. you see, we've always wondered about this kernel security game, who wants what, who's doing what, who's p(l)aying whom, etc. in other words, we wondered if the participants of the game would show their true colors in the situation. and show they did, it was well worth the wait. as far as i could piece together the narrative propagated by Kees Cook of Google, Greg Kroah-Hartman of the Linux Foundation, LWN's Jonathan Corbet, the Linux Foundation's CII, and other less known players, the story is that said entities have found (some of them after years of ignoring the problem) that their security needs could be best solved by upstreaming our code but all their attempts to work out a deal with us for said purpose have failed and thus we are to blame for everything under the sun and then some. and if that weren't enough, they then went on the offensive with lots of self-congratulating bravado and outright attacks on us and our projects. one can't help but notice that behind all the garbage spewed at us, someone was getting very worried about the fate of the KSPP fork of our work. what i can tell them now is that they will have a whole lot more to worry about once they finished reading our response. before we'll get to particular claims, we'll first look at some history to set the stage for some of our answers. my side started in 2000 as a hobby project that i worked on entirely in my free time. i used to jokingly refer to it as a 'weekend project' as that's literally when i had time for it due to my day jobs. later years didn't change much except for the periods when i was without a job (amounting to over a decade) and thus had more free time on my hand. Brad started his project a bit later than me and has always worked on it in his free time. as a true community project (KSPP's is very far from it, see below) we have received lots of generous help over the years in the form of hardware and financial donations that however didn't quite cover our costs so at some point we were forced to run the project in a more formal manner to be able to accept corporate sponsorship (this is what OSS was originally created for). at some point we also decided to begin supporting selected kernel versions for a longer period of time in addition to the latest upstream kernel. this proved to be a nice bonus and even incentive for corporate sponsors. all was well until a few years ago when we learned of several blatant copyright violations (not of just our code but by extension that of the upstream kernel too) and also trademark violations. pursuing these wasn't too successful and in order to prevent further abuse we decided to stop the public distribution of our stable patches and provide it as an entirely commercial product from then on (all still under the GPL, despite much speculation to the contrary). this proved to be a better decision and all seemed well for a few months until the Washington Post article on kernel security appeared. this turned out to be a game changer event in that linux security got a serious look at from high level corporate executives and resulted in Google tasking Kees Cook to start to work on the KSPP with the express purpose of upstreaming grsecurity - all without us. at this point we'd like to make it clear that any statements from Kees/Greg/etc to the contrary are simply *lies*, neither i nor Brad have ever been contacted by any of the corporations paying for the KSPP. since proving a negative is hard, instead hereby we give any corporate attorney, executive (or any other officer who thinks that they gave us an offer that we refused) permission to post any private emails including financial details, contract terms, etc that prove us wrong. as a sidenote, there's some irony in that Kees and Greg managed to tell contradicting lies as according to one we refused to get paid to upstream our code whereas the other said that the Linux Foundation had funded 'members of the grsecurity team'. in reality, neither happened as such offers that we could refuse or accept have never been made. it's also telling that when upstreaming our code came up in private conversations where Kees was present, his best suggestion at the time (around 2015 summer) was to try the freshly expanded CII instead of, say, his own employer Google (last month after he got an early notification about our plans regarding the test patches, he in fact admitted that Google was never going to pay us for the upstreaming work). the CII is also an interesting animal in that they can't seem to figure out what exactly it is they want. during the summer of 2015 i was CC'd into an email thread on cii-discuss where some of our users suggested that the CII take up funding our projects. at the time the CII stated that they would not provide funds that'd amount to effective employment and also that they'd be only interested in upstreaming our work. when i asked if they were then willing to fund thousands of hours of such work, i got no response and left it at that. now imagine my surprise when i saw their recently published annual report for 2016 which has this following beauty in there: One challenge we’ve encountered this year is finding skilled people to take on the work. While the desire to work on open source exists, without compensation it’s simply not feasible for many developers to do so. Emese Renfy, for example, had to step back from Kernel Self Protection Work because funding from CII took so long to approve. We’ve worked to resolve this with our new online grant system. Another solution is to find those who are already working on open source as a hobby and allow them to continue their work at a fully-funded, professional level. For example, Chris Lamb and Ximin Luo, have been able to give up their day job to focus exclusively on the Reproducible Builds project. so it now looks like that funds amounting to effective employment are possible, too bad they never told me about it (this is of course in direct contradiction to what they claimed in their blog and i again give them permission to post any emails/etc that prove the contrary). on a somewhat amusing sidenote, i can't help but notice that someone at the CII must have taken lessons from the KSPP in their copy-pasting skills as they managed to get Emese's last name wrong not once but twice then repeated the same in their blog for good measure. so what do we have so far about this KSPP business? Google and other companies made a decision to get our work upstream using their own resources only - so far so good, it's their business in every sense of the word. not involving us however turned out to be an unwise decision when they realized that their employees are completely unprepared for this work (we'll discuss some examples later). this then led to all kinds of social engineering attempts at trying to get us to help them out - all without getting paid for our time, as if we were just supposed to drop all other work and spend our free time on fixing their botched up attempts. as if this still wasn't disgusting and outragous enough, they went on the combative and had the guts to accuse us of not wanting to cooperate without also mentioning that they expected all this work from us for free taken away from our free time. at this point it should be clear that the KSPP is anything but a community project. rather, it's a joint effort of commercial companies such as Google/Intel/etc to spend their corporate resources on ripping our code and abusing our hard-earned reputation *and* have the audacity to expect us to assist in all this using our free time. this state of affairs made us realize that if this is what this 'community' wants, then they shall get it. now they're on their own and by looking at all the reactive mud slinging ever since, they're terrified about the fate of their fork. based on the level of incompetence they showed so far, it's not hard to see why. this brings us to second part of our response where we'll take a look at some of the claims Kees made and show them for what they really are. > It does underscore the critical need to upstream stuff, though. > Forks of projects might disappear at any time. :( this is somewhat rich coming from a Google employee considering the fate of various linux forks used in their devices. one may also wonder what will happen to the KSPP fork, especially now that Fuchsia is also set to obsolete all that work and remove the incentive to fund it much longer. upstreaming stuff also isn't at all enough to prevent code from disappearing, one needs to look no further than the recent removal of avr32 support due to bitrot. considering the force that sustains the KSPP (corporate money and interest) one can guess how long said code will survive bitrot vs. our track record that is the result of a fundamentally different kind of motivation. as another counterexample to why that 'critical need' can be overrated, one can also look to the unprivileged ping code (somewhat ironically, it's a security related feature, not unlike the topic here), introduced on this very same list years ago by Vasiliy Kulikov via a Google Summer of Code project. it was later maintained by upstream developers who then introduced several exploitable vulnerabilities in it. > Additionally, while PaX Team, grsecurity, and ephox's work > were technically separate development efforts (for example, just look > at how PAX_USERCOPY differed between PaX and grsecurity), FYI, USERCOPY wasn't separate efforts at all, we worked together on USERCOPY and decided to carry certain features in grsec only. this is a typical workflow for us, there've always been things that started out in PaX and were made into a complete feature in grsec or where i took back some code into PaX that i considered needed infrastructure/features in there. > Looking at the results in grsecurity, though, it's clear that they > chose to integrate with upstream instead of maintaining a forked > implementation. i think you got that backwards, our code is the original, the KSPP (and consequently upstream's) copy is just that, a fork (look at where you get the gcc plugin updates from among others). that said, why would we keep two similar copies of our own code around? as for 'integration', what i actually did was to clean up the mess you created, just look at the bits i 'awarded' with CONFIG_BROKEN_SECURITY and some more examples below. > This is further supported by grsecurity being paid by CII to upstream > the gcc plugin infrastructure, correction, grsecurity, which according to your own definition is a kernel patch, can't have been and thus never was paid by the CII, Emese was (and she isn't a grsecurity developer any more than i am an upstream linux developer). > I think it's an entirely false claim that upstream is creating > more work that normal forward porting. and i think you're wrong on that (see below). it's in fact yet another example to show your lack of understanding of what some of our features do. > Finally, even if you can somehow disregard the thousands of upstream > changes benefiting grsecurity[...] what exactly benefits *grsecurity* (you know, what you defined as a kernel patch) and not kernel users in general (which then means that said changes have nothing to do with us)? i can't think of much if anything in recent memory. > they have benefited from the upstreaming review by upstream > finding bugs in various grsecurity features oh yes, this myth that just can't die. we'll see in about a second what exactly those 'bugs' are. > Arnd Bergmann alone found tons of issues with the initify plugin > and grsecurity fixed them) actually that was something like 4 bugs in the plugin and we didn't fix them, Emese did. > Hardened usercopy found bugs in grsecurity's slab implementation, now we're getting somewhere, let's dive in. the first 'bug' was that *4.6* introduced red_left_pad to SLUB objects which needs to be taken into account when computing object boundaries and my code didn't. what you forgot to tell the world at the time (and ever since) is that it was you who took my code from *4.5* which didn't have red_left_pad and thus was perfectly fine as is. the actual bug was that you blindly copy-pasted my code (a recurring theme) without doing the one job you had: maintain this code and follow upstream evolution. the second 'bug' was the use of ksize vs. ->object_size which, if memory serves, a slab maintainer insisted on using. what this 'bugfix' achieved in one fell swoop is that all SLUB objects with padding can now be used to leak that padding which pretty much destroys the purpose of (your copy of) USERCOPY. just think about it, by definition that padding is uninitialized memory and thus a prime target for memory leak attempts that thanks to your 'bugfix' can now occur unimpeded. as an aspiring maintainer your proper response should have been to explain this to the slab maintainers and also why ksize is an abomination itself (its users by definition exercise UB) and that the problem it sets out to solve should be an API redesign instead (remove ksize and create a new allocation function that takes the size as reference and updates it on return so that the caller can learn how much memory it ended up getting). so that's two bugs so far, except none of them is ours but yours instead but nice try selling them otherwise. unfortunately the buck doesn's stop just there yet, there're more problems with this expert reviewed and maintained code (i wonder how on earth we managed to survive this far without such expertise). the third bug is that the comment describing __check_heap_object is meaningless garbage as already allocated objects by definition cannot be 'incorrectly sized' (at least not at this level, SIZE_OVERFLOW would be something closer to that purpose) and of course the purpose of USERCOPY has nothing to do with verifying object sizes anyway. what USERCOPY does want to verify is the size of the memory *copy* attempt (that is, the copy must fully fall inside the kernel object). this brings us to the fourth bug which is the verification of the kernel object pointer for being 'impossible'. that of course was never the purpose of USERCOPY as its design assumes a threat model where an attacker controls only the size of the copy but not the object pointer. if the latter is also assumed, all bets are off, there's no way such checks can catch an actual attack (checking pointer provenance is a very hard problem). there're further problems caused by your 'improvements' on our code. one is the needless change of the log message that breaks existing log analyzers looking for a pattern. the other problem is that the replacement of do_group_exit with a BUG is entirely unnecessary as a USERCOPY event is recoverable, there's no need to panic the whole system because of it (a bit like how a REFCOUNT event is recoverable to an extent). i also wonder what locks would be broken by BUG when copy*user functions can sleep and thus are not supposed to be called under locks anyway. (and if you didn't mean spinlocks then how would BUG handling break them?) last but not least, a quick look at your claim to fame due to: > [USERCOPY] gained strncpy_from_user() coverage [...] this is an entirely pointless exercise as an audit of the few callers of said function can show their correctness instead. so your 'coverage' adds nothing but pointless performance impact to it. FWIW, similar considerations made us not check copying with compile-time constant sizes at the time. i believe this brief overview at your achievements regarding USERCOPY shows that instead of fixing any bugs in our code, you in fact added a bunch of them. > The current refcount_t work based on PAX_REFCOUNT uncovered a > crash bug in grsecurity's implementation for the record, it was a bug introduced in the PaX 4.9 port only and you failed to spot it when copying it to a newer kernel (you also failed to notice what the underlying problem i wanted to fix was and thus didn't fix it yourself either at the time). note also that the reporting problem i tried to fix there (and fixed it differently in later patches) is still not properly addressed in your refcount port but at least you documented it now. > Upstreaming grsecurity features brings a huge amount of testing > to bear on the code, which, like all the other things in upstream, > grsecurity directly benefits from. not all things upstreamed benefit us (or other users for that matter). just look at VMAP_STACK (upstream NIH'd version of Brad's KSTACKOVERFLOW feature) and the entirely avoidable damage (and security bugs) it caused to end users ever since its release. > And to top all of this off, while upstreaming the latent_entropy > plugin, I noticed a English typo that was present in all the > grsecurity gcc plugins, and when I sent them a trivial spelling > fix patch for all their plugins[...] if only it had been all plugins... clearly you never tried a recursive grep and thus missed the same typo in RAP and SIZE_OVERFLOW. > [...] (rather than just letting it stand and making work for > them to catch it during forward porting and fix it everywhere > else), they publicly mocked the patch (which they applied). given the above, it'd have been better to just send an email about the typo and let us fix it ourselves properly which we ended up doing anyway instead of using your incomplete patch. > Upstreaming can be extremely time consuming. Grsecurity made it clear > from long ago that they had no intention of upstreaming things because > they wanted to use their time differently. not true at all. what we've always said was that upstreaming our code requires so much time that noone can reasonably expect us to do it in our free time. that the KSPP exists entirely by corporate decree is proof of that. what would actually limit any potential upstreaming effort on our side is the need to balance our time spent on upstreaming vs. time spent on new R&D. come to think of it, if the companies running the KSPP had played it smart, they'd have gone out of their way to engage us as that's the only way they'd have a chance to close the ever widening feature gap. > That said, when I asked grsecurity if there I was any way they would > accept payment to upstream things, they did briefly agree and > upstreamed the gcc plugin infrastructure. So they were willing to > upstream if paid, yet ultimately decided to stop, and to continue to > forward port their patches. i don't know who you asked as 'grsecurity' but the only thing i recall was some private conversations back in 2015 where you suggested to try to get the CII to fund an upstreaming project (where you yourself had no saying) and that turned out to be a downer. you also knew full well that Brad wouldn't have anything to do with the CII due to one of their member companies (that entirely coincidentally is also behind the KSPP) having violated his trademark. one can't also help but wonder why you didn't offer Google's money? because you have never been authorized by Google to act on their behalf (and thus can't have possibly made any such offers)? so i'm not sure what you think you did to get us 'accept payment to upstream things' but you've clearly never been in a position to make such an offer (and no, poorly disguised social engineering attempts don't count). by the way, it wasn't Brad who upstreamed the gcc plugins stuff but Emese (she's neither a PaX nor a grsecurity developer). > [...] I again summarily reject the notion that upstreaming > grsecurity features creates "more work for grsecurity without > value in return". > [...] > To your specific examples, __ro_after_init is literally a one-line > change: they just make in __read_only. This was some of my first > attempts to make their forward porting work easy while upstream slowly > incorporated features. this is false and only shows that you don't at all understand what __read_only really does. in particular, __read_only is enforced much earlier in modules (so much for it being 'after init') and thus all writes to such objects must be instrumented. this means that all new __ro_after_init uses must be carefully audited and any writes to them instrumented. this is anything but 'literally a one-line change' (though i can at least get the compiler to do the work for me to an extent). > The work around PAX_MEMORY_SANITIZE made the slab debug paths > faster for everyone. i wonder, how many systems actually enable those slab debug features at runtime due to their performance impact. > So, neither of us can speak for grsecurity, but I reject your belief > that upstream has somehow created needless work for them. fortunately it's got nothing to do with belief but simple facts that happen to contradict your belief. > Which brings me to a question I haven't seen anyone ask yet: why does > grsecurity exist? thanks for your concern for our existence but i think you should ask why the KSPP exists instead and how much longer it will be around. based on your statements here and in later mails it seems that you're very worried that your ability to copy-paste yet more of our code will diminish over time. unfortunately for you, putting up a fake happy face and pretending nothing happened won't actually make it any better for the KSPP. back to your question, you don't have to guess and can instead just look at what world class domain experts think of our work in their testimonials on the grsecurity website. it seems to me that you and the KSPP are stuck in the world of linux whereas our work has affected much more than that already. so in a sense you're right to say that we don't care about all linux users since we actually care about everyone else too and use our projects (the code, conference presentations, blogs, etc) to showcase defense technologies that everyone can benefit from, not just linux users. when will the KSPP reach this level, if it's on the corporate agenda all, that is? and if all this counts for nothing for you then consider our existence as the apparent enabler of your career. > You're speaking entirely in theoreticals, but I understand what you're > trying to say. All forward porting runs this risk. Kernel internals > change in ways that threaten even unchanged grsecurity features. To > bring this down to earth, I would ask "how does grsecurity perform > testing?" I can point to the many ways how upstream performs testing, > including LKDTM for several of the security-sensitive features. As > seen in the PAX_REFCOUNT porting work, the 4.9 grsecurity patch was > clearly never actually tested since _any_ exercise of the PAX_REFCOUNT > protection would Oops the kernel. you're wrong, i actually tested the code just not a last-minute one-liner change that was 'obviously correct' (well, it wasn't, it happens). on the other hand it's pretty funny that you pointed out this one bug as it was you who blindly copy-pasted it *and* never tested it yourself otherwise you would have found it, not Jann Horn. another funny thing is that you clearly never understood the REFCOUNT reporting code as LKDTM didn't even exercise those paths at the time. last but not least, the actual effect of the bug wasn't necessarily an oops, it all depended on how the offsets were encoded in the conditional jumps (as i explained all this in my response to Jann and you). > [...] but I can point you directly to how upstream tests, how upstream > developers find bugs in grsecurity features, and how things can improve once > upstreamed (again, for example, the massive uaccess consolidation). yep, things improve indeed, like the new vulnerabilities caused by ping sockets, VMAP_STACK, HARDENED_USERCOPY, SLAB_FREELIST_RANDOM, HARDENED_REFCOUNT, etc... > I think I have thoroughly disproved this position. Something that I > think is hard to see for people not involved in day-to-day upstream > work is how very different the development workflows are between > upstream and grsecurity. Upstream is normally evolutionary, doing > things in easy to digest pieces, where as grsecurity could just land > massive changes between releases. believe it or not but we also develop our features step-by-step, not unlike how any sane developer, upstream or not, does it. > These are all complaints to be made to grsecurity. They are the ones > who provides those features originally, did not upstream them, and > then took them away. i'm wondering, when are you going to stop the hypocrisy and this rhetoric of blaming us for not upstreaming our own code in our free time while the KSPP companies have been happily paying you and others for doing the same? > > So, here's my list of questions for the KSPP: > > 1/ When will I be able to switch to a vanilla Linux kernel that is > > equivalently hardened as a grsecurity/PaX kernel used to be? > > How could anyone answer that question? I can't see the future. well, it seems that not that long ago you did see the future as: grsec is going to stay a decade ahead of anyone else (not just Linux) > And besides, it's not like grsecurity was providing comprehensive > protections. If you're using arm64, you might feel like you're already > in a better position with upstream (got PAN emu, got hardened > usercopy, still no RAP). actually, by definition you can't be in a better position with upstream unless you're claiming that we actually reduce security with our code (please, do bring up my disabling of KASLR, happy to discuss it again). in fact due to the bugfixes we have for your USERCOPY fork (that is, we kept our own working code), our code *is* actually in a better position already. as for RAP, it works on any arch by design, i just don't enable it on archs where i haven't fixed all the function pointer abuse yet and marked up asm code (both arm and arm64 work already here). > Upstream's goal is protecting as many people as possible. the KSPP's goal is to further the agenda of the companies behind it (which is extracting profits for shareholders). that has nothing to do with "protecting as many people as possible" but everything to do with business goals du jour. if what you claim was true, they would have done it since the beginning and in a way that is not restricted to only linux users. > > 2/ Who will maintain this code and how? > > 3/ Who ensures the coverage and quality won't suffer for each new > > kernel release? > > The upstream development community, just like everything else. I > answered both of these already above. the correct answer is 'whoever the KSPP companies put on the job' though their performance so far doesn't leave one with great confidence as to the sustainability of the project. cheers till the next round, PaX Team ^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [kernel-hardening] It looks like there will be no more public versions of PaX and Grsec. 2017-05-11 1:24 ` PaX Team @ 2017-05-11 16:30 ` Daniel Micay 2017-05-11 18:02 ` Kees Cook 2017-07-31 13:38 ` Solar Designer 2 siblings, 0 replies; 35+ messages in thread From: Daniel Micay @ 2017-05-11 16:30 UTC (permalink / raw) To: PaX Team, Mathias Krause, Kees Cook; +Cc: Daniel Cegielka, kernel-hardening KSPP is going to change without public patches. There was a substantial community built around PaX and grsecurity. Hardened Gentoo, Alpine and Arch Linux all had/have official support integration and a significant number of users. Other people used it on their own. Few of those users are going to have access now so the best we have is upstream work (KSPP) and maintaining stuff downstream if it hasn't landed yet or is blocked by politics, etc. People aren't going to accept only having a 4.9 LTS with the last public patch. New hardware support is already chipping away at the usefulness of that. I agree with a lot of what you're saying, but that doesn't really matter anymore. I care about code, not politics and historical grievances. I think PaX / grsecurity has almost always been on the right side of these feuds but that changed when people trying to improve security upstream by landing changes from PaX / grsecurity or even just new code became the enemy. People get banned from the community (IRC) for even stating that they want to help improve upstream security. I was quite happy with the grsecurity patches so I lacked the motivation to make upstream contributions. I only tried to push things in the right direction on the mailing lists, which ended up getting me banned from a community that I put a lot of time/effort into. Other than the few with access to the commercial patches, the community either has the choice of staying loyal to you and having nothing or using and contributing to the remaining public efforts. In terms of funding contracting out work isn't how companies like Google do things. They hire people and do everything in-house, even if that means losing on lots of talented people not willing to do things their way. CII / OTF are the exception to the rule and then your funding would be subject to politics and could vanish at any time even if you succeed in getting it from them initially. They didn't single out grsecurity for this treatment, it's how they do things in general. We were encouraged to apply for OTF funding for CopperheadOS by people who supposedly had influence there and were rejected based on astoundingly stupid reasons after wasting time on it. AOSP is primarily permissively licensed, so we moved away from FOSS licensing -> non-commercial usage licensing for everything but the kernel due to getting nothing back from it while companies were taking and not giving back. Unfortunately, you don't have that option. I've always tried (and mostly failed) to get changes upstream into AOSP though. I'm not worried about running out of features to implement and it makes it easier to maintain since that stuff doesn't need to keep being ported forward, although there's a lot more time pressure to port than there is for Linux due to how it's released. The VMAP_STACK feature is upstream, so while reinventing KSTACKOVERFLOW was silly and rediscovering the same bugs even more so, that time ended up being wasted and the temporary stability hit was accepted. It's too late now and the upstream feature is going to be perfectly good. KASLR is a weak mitigation, but people decided to spend their time on that and it exists so I don't see the point in not getting the marginal utility it offers rather than disabling it or marginally useful checks in the USERCOPY code via CONFIG_BROKEN_SECURITY, where you're taking something away from your users essentially out of spite. I also don't understand the point of maintaining stuff like KSTACKOVERFLOW and a bunch of other features once the upstream solution works fine. Why keep a separate ASLR implementation with a legacy random number generation scheme vs. a few small changes to the upstream one, etc? https://github.com/thestinger/linux-hardened now exists because I am quite aware of the limitations of upstream work and I don't plan on waiting years to have features like RAP and page table protection. Still, it's only maintainable for us if we actually split everything up and document it which is half of the way to upstreaming everything. It makes sense to try to land as much as possible upstream for free code review / testing and a reduction of the maintenance burden. The goal isn't to upstream as much as possible, but rather than the means to an end which is getting back what we already had available via the work you published. If you can't tolerate people doing stuff like this, you're going to end up considering a lot of your old community as enemies... I don't see what else we are supposed to do. There's no way for the PaX / grsecurity commercial model to work for regular users. ^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [kernel-hardening] It looks like there will be no more public versions of PaX and Grsec. 2017-05-11 1:24 ` PaX Team 2017-05-11 16:30 ` Daniel Micay @ 2017-05-11 18:02 ` Kees Cook 2017-05-12 11:34 ` Hunger 2017-07-31 13:38 ` Solar Designer 2 siblings, 1 reply; 35+ messages in thread From: Kees Cook @ 2017-05-11 18:02 UTC (permalink / raw) To: PaX Team; +Cc: Mathias Krause, Daniel Cegielka, kernel-hardening PaX Team, I've always valued your thoughts and contributions. Your work has made the world a little more secure. I'm doing my best to make the world a lot more secure -- and it's nice when I get to follow your lead. You can think or say what you want about me, even if it's untrue, but I (and many others here) are still very willing and hopeful to collaborate with you. If a financial arrangement is desired, I believe my employer continues to be willing to explore it now just as they have in the past. It's unfortunate that the discourse has dropped to this level and continuing at that level isn't going to help. I don't really have much new to say beyond what I've already posted, but I'd ask that you take your harassment and antagonism somewhere else; we're trying to get work done. -Kees ^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [kernel-hardening] It looks like there will be no more public versions of PaX and Grsec. 2017-05-11 18:02 ` Kees Cook @ 2017-05-12 11:34 ` Hunger 0 siblings, 0 replies; 35+ messages in thread From: Hunger @ 2017-05-12 11:34 UTC (permalink / raw) To: Kees Cook; +Cc: PaX Team, Mathias Krause, Daniel Cegielka, kernel-hardening [-- Attachment #1: Type: text/plain, Size: 1001 bytes --] On Thu, May 11, 2017 at 8:02 PM, Kees Cook <keescook@chromium.org> wrote: > Your work has made > the world a little more secure. I'm doing my best to make the world a > lot more secure PaX and grsecurity had an influence on the whole industry. Every modern OS have NX support and ASLR now from Apple to Microsoft. PIE binaries and mprotect restrictions in Apple iOS. PaX UDEREF and KERNEXEC features implemented in hardware level, like ARM PXN, PAN and Intel SMEP, SMAP (still weaker protection than PaX btw). PaX RAP is leading in Control Flow Integrity. So how you make the world a lot more secure while just copy+pasting the 10+ years old grsecurity and PaX features and innovating nothing new? > If a financial arrangement is desired, I believe > my employer continues to be willing to explore it now just as they > have in the past. > If i read it correctly, pipacs and spender got no financial arrangement or any contact attempt about it in the past, so what will your employer continue? [-- Attachment #2: Type: text/html, Size: 1515 bytes --] ^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [kernel-hardening] It looks like there will be no more public versions of PaX and Grsec. 2017-05-11 1:24 ` PaX Team 2017-05-11 16:30 ` Daniel Micay 2017-05-11 18:02 ` Kees Cook @ 2017-07-31 13:38 ` Solar Designer 2 siblings, 0 replies; 35+ messages in thread From: Solar Designer @ 2017-07-31 13:38 UTC (permalink / raw) To: PaX Team; +Cc: kernel-hardening Hi all, I've just re-read PaX Team's lengthy message from May (back then, I only skimmed it). I felt I owed PaX/grsecurity at least as much to hopefully understand their position better, which I admit I still don't understand very well. But that's not why I post this belated reply. Rather, I found in the message a detail I feel I have to correct as the issue is partially my responsibility and I want to share the lessons learned: On Thu, May 11, 2017 at 03:24:35AM +0200, PaX Team wrote: > > It does underscore the critical need to upstream stuff, though. > > Forks of projects might disappear at any time. :( [...] > as another counterexample to why that 'critical need' can be > overrated, one can also look to the unprivileged ping code > (somewhat ironically, it's a security related feature, not > unlike the topic here), introduced on this very same list > years ago by Vasiliy Kulikov via a Google Summer of Code > project. it was later maintained by upstream developers who > then introduced several exploitable vulnerabilities in it. The exploitable vulnerabilities that I think PaX Team is referring to were introduced during forward-porting of the patch prior to it getting accepted upstream. Thus, in one way yes the bugs were part of the upstreaming effort, but in another way no they were not by upstream developers. Rather, in hindsight neither Vasiliy nor I (his GSoC mentor) were familiar enough with the target kernel version's APIs (the forward-porting was from 2.4.32 to 3.0), and we didn't perform enough testing (not of obscure cases not occurring in ping(1), such as disconnecting and reconnecting the same ping socket). Those bugs were found by different third-parties through fuzzing, and fixed by upstream kernel developers. Lessons (re-)learned include: 1. Tough to make non-trivial contributions to the kernel (especially by "outsiders" or/and after having been "on hiatus") without introducing bugs. Even if tests appear to pass at first (ping(1) worked fine via the new sockets, including if invoked repeatedly and concurrently). 2. Kernel APIs are very brittle, locking conventions are non-obvious. 3. Kernel hardening efforts should include making the APIs less brittle or/and to have things fail in less dangerous or at least less likely exploitable ways. In fact, in the aftermath of one of these bugs the list pointer poison values have been adjusted not to be valid addresses (which I think had been done in grsecurity before). The refcount overflow work is also relevant. 4. Forward-ports (and perhaps distro vendors' back-ports too?) tend to receive less scrutiny than brand new code, but bugs are just as easily introduced (or even easier if the porting isn't by the original author). Patch made to apply and the feature works, so we're OK? No, we are not. 5. One thing we did right was making the feature unavailable to non-root by default. We knew we'd be increasing the attack surface otherwise, and instead we wanted to introduce this layer of privsep - having ping SGID to a group (instead of SUID root), and having it use the new ping sockets, without exposing those sockets to potential exploits directly. This caution paid off. 6. As expected, some systems ended up exposing the sockets to users other than host root anyway. I expected this in container setups, where container root would have access (can't do much about this while also keeping root as powerful in their container as root would normally be on a Unix system). I didn't expect it, but Android also exposed the sockets to all users. 7. The advances in kernel syscall fuzzing are great. These days, if similar bugs get in I hope most would be found by someone running a fuzzer sooner (the ping socket bugs were found over quite a long period of time). Preferably, contributors themselves should run focused fuzzing efforts on the syscalls, socket types, etc. they add or modify. And non-focused too, so that interactions with other calls are exposed. Alexander ^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [kernel-hardening] It looks like there will be no more public versions of PaX and Grsec. 2017-05-01 22:01 ` Mathias Krause ` (3 preceding siblings ...) 2017-05-02 0:54 ` Kees Cook @ 2017-05-02 11:11 ` David Gens 2017-05-02 21:27 ` Mathias Krause 4 siblings, 1 reply; 35+ messages in thread From: David Gens @ 2017-05-02 11:11 UTC (permalink / raw) To: Mathias Krause; +Cc: Kees Cook, Daniel Cegiełka, kernel-hardening On 2017-05-02 00:01, Mathias Krause wrote: > On 27 April 2017 at 00:04, Kees Cook <keescook@chromium.org> wrote: >> On Wed, Apr 26, 2017 at 2:05 PM, Daniel Cegiełka >> <daniel.cegielka@gmail.com> wrote: >>> https://grsecurity.net/passing_the_baton_faq.php >> >> Yeah, I'm sad to see them go. PaX Team was just last night helping >> with some details of PAX_REFCOUNT as it could appear in the upstream >> refcount_t API. I hope they'll still help out from time to time. >> >> It does underscore the critical need to upstream stuff, though. Forks >> of projects might disappear at any time. :( > > Now, since grsecurity and PaX went private, quite a few users > (including me) are left in the dark and are, to say the least, > slightly pissed. But not everybody seems to be barking at the right > tree. So I've a few comments and questions for the KSPP. > > I think the main reason for Brad and PaX Team to make their work > private is the increased amount of work KSSP has put on them without > providing any valuable work in return. They just don't want to be > forced to maintain and fix-up the variants of grsecurity/PaX features > KSPP lands upstream. And no, the work of KSPP does not make their life > easier, in fact, it makes it harder. Harder for two reasons: First, > the code does not end up verbatim, it's always changed, taken out of > context and "enhanced". Second is the loose of control over the code. > But let me elaborate those two a little further. > > The first point directly forces them to think through the upstream > incarnation of a feature, how it differs from their original one and, > for those parts, how to fix them up to work properly to fit their > needs, to fit their security requirements. Those changes generate a > lot of conflicts with their version of the code which takes time to > fiddle out. As happened for __ro_after_init, MEMORY_SANITIZE, > USERCOPY,... which only went in in reduced and modified form, > generating needless work on their side -- from their point of view. > > The second point, the loose of control over the code, is even worse. > Not getting any more conflicts when porting the grsecurity/PaX patch > to a new kernel release makes changes to code that used to live in > grsecurity/PaX probably go unnoticed. And, as it seem to be the case, > upstream developers are not always familiar with all the gory details > and might introduce weaknesses and bugs. This not only makes upstream > Linux less secure, it makes grsecurity and PaX less secure, too. > > So I can understand why they've done this. Still, with the loose of > the public availability of the patch, Linux security has suffered a > lot. Not only is upstream far far away to reach a level that is > available today in grsecurity and PaX, no, it also won't benefit from > new developments any more. What a great achievement! :( > > *sigh* > > I think the intention of the KSPP is good -- making vanilla Linux more > secure. But the way it does its work harms overall Linux security. It > does hurt mine, that's for sure! I know the value of grsecurity and > PaX in particular and know how easy it still is to exploit a vanilla > Linux. Features like KERNEXEC and RAP make it almost impossible for an > attacker to abuse a memory corruption bug in the kernel. Many unnamed > features -- unnamed because not under an #ifdef -- make exploiting > use-after-free bugs much less interesting or reduce the race window > for TOCTTOU bugs involving user copy operations. None of this can be > found in vanilla Linux. Probably never will... So tell me how exactly a PaX-hardened kernel with a memory-corruption bug prevents an adversary from modifying critical data, such as the page tables in a data-only attack? IMHO the "PaX mitigates Memory-corruption" argument is wishful thinking, and now that PaX is gone users cannot fool themselves any longer into thinking "My System is Secure". That awareness is actually good no? > So, here's my list of questions for the KSPP: > 1/ When will I be able to switch to a vanilla Linux kernel that is > equivalently hardened as a grsecurity/PaX kernel used to be? > 2/ Who will maintain this code and how? > 3/ Who ensures the coverage and quality won't suffer for each new > kernel release? > > Judging from the planed EOL for the v4.9 LTS kernel -- the last one a > grsecurity patch was publicly available for --, there are less than > two years to finish the work for item 1 to ensure a secure and smooth > transition from grsecurity to upstream Linux. Will the KSPP be able to > achieve this?... I have my doubts. > > Even if so, item 3 will be a hard problem to solve as the Linux kernel > development model does not support such a tree-wide review point in > the release cycle. Assuming there are people able and willing to judge > and review the code base for required changes (e.g. atomic_t -> > refcount_t changes or function pointer cleanups for RAP), how would > those be able to simply get in those changes at, say, time of rc5? -- > without having to argue with a horde of maintainers of the individual > subsystems involved? > > Quite a lot of questions. The external patch solved them all by not > having to deal with upstream Linux development and not having the code > available until it's ready. But now it's gone and no adequate > replacement is on the horizon. What will KSPP do about it? I had a > reasonably secure kernel, now it's gone. :( > > > Regards, > Mathias Best, David ^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [kernel-hardening] It looks like there will be no more public versions of PaX and Grsec. 2017-05-02 11:11 ` David Gens @ 2017-05-02 21:27 ` Mathias Krause 2017-05-03 8:59 ` David Gens 0 siblings, 1 reply; 35+ messages in thread From: Mathias Krause @ 2017-05-02 21:27 UTC (permalink / raw) To: David Gens; +Cc: Kees Cook, Daniel Cegiełka, kernel-hardening On 2 May 2017 at 13:11, David Gens <david.gens@cs.tu-darmstadt.de> wrote: > So tell me how exactly a PaX-hardened kernel with a memory-corruption bug > prevents an adversary from modifying critical data, such as the page tables > in a data-only attack? It doesn't as it has not mitigation for data-only attacks yet. But neither does a vanilla kernel. It's even worse on vanilla Linux where the page tables simply can be written to with a write-what-where primitive. PaX can, however, protect the page tables from getting modified by having them write-protected. So yes, that makes it more secure. > IMHO the "PaX mitigates Memory-corruption" argument > is wishful thinking, and now that PaX is gone users cannot fool themselves > any longer into thinking "My System is Secure". That awareness is actually > good no? What about KERNEXEC and CONSTIFY taking care of protecting a lot of data structures? What about SIZE_OVERFLOW successfully preventing know overflow bugs, like the various keys related ones? What about RAP, ensuring control flow integrity? That's nothing, you say? Ridiculous. Thanks, Mathias ^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [kernel-hardening] It looks like there will be no more public versions of PaX and Grsec. 2017-05-02 21:27 ` Mathias Krause @ 2017-05-03 8:59 ` David Gens 2017-05-03 19:10 ` Rik van Riel 0 siblings, 1 reply; 35+ messages in thread From: David Gens @ 2017-05-03 8:59 UTC (permalink / raw) To: Mathias Krause; +Cc: Kees Cook, Daniel Cegiełka, kernel-hardening On 2017-05-02 23:27, Mathias Krause wrote: > On 2 May 2017 at 13:11, David Gens <david.gens@cs.tu-darmstadt.de> > wrote: >> So tell me how exactly a PaX-hardened kernel with a memory-corruption >> bug >> prevents an adversary from modifying critical data, such as the page >> tables >> in a data-only attack? > > It doesn't as it has not mitigation for data-only attacks yet. But > neither does a vanilla kernel. It's even worse on vanilla Linux where > the page tables simply can be written to with a write-what-where > primitive. PaX can, however, protect the page tables from getting > modified by having them write-protected. So yes, that makes it more > secure. > >> IMHO the "PaX mitigates Memory-corruption" argument >> is wishful thinking, and now that PaX is gone users cannot fool >> themselves >> any longer into thinking "My System is Secure". That awareness is >> actually >> good no? > > What about KERNEXEC and CONSTIFY taking care of protecting a lot of > data structures? What about SIZE_OVERFLOW successfully preventing know > overflow bugs, like the various keys related ones? What about RAP, > ensuring control flow integrity? That's nothing, you say? Ridiculous. Nope, I never said that. I said it is insufficient to mitigate memory corruption. Actually, KSPP even has the potential to improve things here. Granted, for the time being these PaX features are unavailable, but that was a one-sided political decision, and blaming others (based on non-factual claims) is not helping. I guess it would be nice to have a list of features that will be ported into KSPP, e.g., sorted by priority, and then users could easily check when to switch from the last PaX patches. But thats not how kernel development works (traditionally), because developers push features they (or their employers) are interested in. > Thanks, > Mathias Best, David ^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [kernel-hardening] It looks like there will be no more public versions of PaX and Grsec. 2017-05-03 8:59 ` David Gens @ 2017-05-03 19:10 ` Rik van Riel 0 siblings, 0 replies; 35+ messages in thread From: Rik van Riel @ 2017-05-03 19:10 UTC (permalink / raw) To: David Gens, Mathias Krause Cc: Kees Cook, Daniel Cegiełka, kernel-hardening [-- Attachment #1: Type: text/plain, Size: 681 bytes --] On Wed, 2017-05-03 at 10:59 +0200, David Gens wrote: > I guess it would be nice to have a list of features that will be > ported > into KSPP, > e.g., sorted by priority, and then users could easily check when to > switch from the > last PaX patches. But thats not how kernel development works > (traditionally), because > developers push features they (or their employers) are interested in. That is the difference between development and support. Developers will continue to do things, on their own schedule and/or the schedule of their employers. If you want grsecurity support, your best bet right now is to pay grsecurity. -- All rights reversed [-- Attachment #2: This is a digitally signed message part --] [-- Type: application/pgp-signature, Size: 473 bytes --] ^ permalink raw reply [flat|nested] 35+ messages in thread
[parent not found: <1788778362.1495506.1493751985632.ref@mail.yahoo.com>]
* Re: [kernel-hardening] It looks like there will be no more public versions of PaX and Grsec. [not found] <1788778362.1495506.1493751985632.ref@mail.yahoo.com> @ 2017-05-02 19:06 ` Lionel Debroux 2017-05-02 22:35 ` Kees Cook 0 siblings, 1 reply; 35+ messages in thread From: Lionel Debroux @ 2017-05-02 19:06 UTC (permalink / raw) To: Kees Cook, Mathias Krause; +Cc: Daniel Cegiełka, kernel-hardening Hi, Thanks Mathias for bringing up the matter, despite the way the attempt was bound to be received by upstream, let alone trolls... > > Assuming there are people able and willing to judge and review the > > code base for required changes (e.g. atomic_t -> refcount_t changes > > or function pointer cleanups for RAP), how would those be able to > > simply get in those changes at, say, time of rc5? -- without having > > to argue with a horde of maintainers of the individual subsystems > > involved? > This is an upstream development workflow problem that is especially > frustrating for security work since it can regularly touch lots of > areas, but it's one that I've been working to resolve. It's actually > on my list of things to discuss at the next Kernel Summit, since > making these wide changes is really irritating if it's going in via > each maintainer. Really irritating because really time-consuming, indeed. There's a genuine issue here. I too agree that mainline needs to grow a more efficient, more streamlined way to integrate a number of smaller cleanups and improvements, which _currently_ bring minor or even no functional changes on a mainline kernel. Creating such a process would be a major improvement. > If you have time to criticize, you have time to write a patch. True, and as you pointed, Mathias did. I can't speak for others, but right now, the prospect of spending some of my free time making the jump from browsing through the grsecurity patch and trying to determine which hunks relate to what and how they're useful to actually making + testing + writing a changelog explaining to maintainers not necessarily caring about security why the change makes sense and how it changes the binary's size + submitting patches to mainline (with wrong credit, it's not my work in the first place) and most of all watch a sizable subset of these patches be either dropped on the floor without a comment (this is not a theoretical concern, that's precisely what happened to most patches in Emese Revfy's large constification patchset years ago) or outright rejected because they _currently_ bring no functional change on a mainline kernel is... unappealing. Let's list several families of smaller cleanups and improvements made in PaX/grsec. You (Kees, Mathias) are fully aware about them but maybe some other readers aren't familiar with the contents of the monster patches: * using explicit struct members throughout the code, which is a prerequisite for making RANDSTRUCT (probably bound for mainline at some point, AFAIK) more broadly useful than if it's limited to a handful of whitelisted structs. grsecurity contains hundreds of such hunks. I think that 4 randomly chosen, simple and related samples are in drivers/net/wan/z85230.c; * building constant structures at compile-time (and making them const) rather than emitting both the room for that structure in a writable section (...) or on the stack, and the code to initialize that structure. That's related to CONSTIFY, it yields safer and usually smaller code+data. Over the years, overwriting needlessly writable ops structs proved to be used multiple times in exploit PoCs for mainline kernels, which often failed miserably on PaX/grsecurity (not just thanks to CONSTIFY, two or three among MEMORY_UDEREF, KERNEXEC and RANDSTRUCT foiled the PoC as well). See the hunks for (I think) drivers/cpufreq/sparc-us3-cpufreq.c drivers/gpu/drm/armada/armada_drv.c others in drivers/gpu/drm/ (constifying "*_max_ioctl(s?)" variables initialized with ARRAY_SIZE(a constant array of structs)) drivers/media/platform/omap/omap_vout.c drivers/memory/omap-gpmc.c drivers/mfd/rn5t618.c drivers/net/ethernet/hisilicon/hns/hns_dsaf_misc.* drivers/net/wireless/ath/wil6210/pcie_bus.c drivers/net/ethernet/amd/xgbe/* (most of the hunks in these files) drivers/video/fbdev/aty/aty128fb.c net/can/gw.c include/net/sctp/checksum.h ... and hundreds of others; * getting rid of some of the wrong casts between incompatible function pointer types. This often does not require adding wrappers, and sometimes even makes the code _more_ readable. These would be non- functional changes on a mainline kernel, and therefore especially likely to be ignored or rejected by maintainers. Again, hundreds of such hunks, which made up the bulk of the PaX & grsecurity patches' size increase when RAP was added. For instance, AFAICS: drivers/gpu/drm/*/psb_intel_sdvo.c and others: returning "enum drm_mode_status" instead of the less precise "int". drivers/media/dvb-frontends/cx*.c and others: likewise, returning "enum dvbfe_algo" instead of "int". drivers/net/wireless/atmel/atmel.c drivers/net/wireless/cisco/airo.c drivers/net/wireless/intersil/hostap/hostap_ioctl.c drivers/net/wireless/realtek/rtlwifi/pci.c (+ other WiFi drivers) arch/{arm,x86}/crypto/sha*glue.* fs/coda/dir.c * other samples of the untapped tidbits from PaX/grsec: * removing spurious "static" keywords on function-local variable declarations when the first thing the function does with the variable is to reinitialize it. See e.g. the drivers/mfd/max8925-i2c.c drivers/mfd/tps65910.c hunks, both of which were already in a grsecurity patch for 3.14 from May 2014 laying around on my HDD. * annotations in e.g. include/linux/seqlock.h include/linux/spinlock.h Integrating into mainline hundreds of these smaller improvements and cleanups from PaX/grsec, i.e. even a single-digit percentage of the grsecurity patch's hunks, would demonstrably improve the mainline code immediately or in the mid-term... but that integration work is harder than it could be :( Regards, Lionel. ^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [kernel-hardening] It looks like there will be no more public versions of PaX and Grsec. 2017-05-02 19:06 ` Lionel Debroux @ 2017-05-02 22:35 ` Kees Cook 0 siblings, 0 replies; 35+ messages in thread From: Kees Cook @ 2017-05-02 22:35 UTC (permalink / raw) To: Lionel Debroux Cc: Mathias Krause, Daniel Cegiełka, kernel-hardening, Julia Lawall On Tue, May 2, 2017 at 12:06 PM, Lionel Debroux <lionel_debroux@yahoo.fr> wrote: > Hi, > > > Thanks Mathias for bringing up the matter, despite the way the attempt > was bound to be received by upstream, let alone trolls... > >> > Assuming there are people able and willing to judge and review the >> > code base for required changes (e.g. atomic_t -> refcount_t changes >> > or function pointer cleanups for RAP), how would those be able to >> > simply get in those changes at, say, time of rc5? -- without having >> > to argue with a horde of maintainers of the individual subsystems >> > involved? >> This is an upstream development workflow problem that is especially > >> frustrating for security work since it can regularly touch lots of >> areas, but it's one that I've been working to resolve. It's actually >> on my list of things to discuss at the next Kernel Summit, since >> making these wide changes is really irritating if it's going in via >> each maintainer. > > Really irritating because really time-consuming, indeed. There's a > genuine issue here. I too agree that mainline needs to grow a more > efficient, more streamlined way to integrate a number of smaller > cleanups and improvements, which _currently_ bring minor or even no > functional changes on a mainline kernel. Creating such a process would > be a major improvement. Yup, and being able to show the cases for this is what I'm hoping to do at Kernel Summit. I currently think that for minor things, having a process to notify maintainers (seeking their Ack), but committing the changes if there is no NAK would be the best approach. This is how similar large changes happen now (see the uaccess conversion, for example), but it requires such a tree be maintained by someone that Linus will trust to pull from. I'm hoping to be that person, but I'll need to demonstrate the need with a few examples, and build that trust by doing it safely for a while. I've already started this a little bit with my KSPP tree that is pulled for -next. From this tree I've been carrying hardened usercopy changes (which touched lots of architectures initially), gcc plugin changes (which touch lots of files for annotations), and lately the CONFIG_DEBUG_RODATA renaming. The trick is finding the right balance and making sure maintainers have a chance to see things. The massive refcount_t conversion, for example, while really time-consuming has resulted in a lot of excellent discussion, identified many places where things were refcounting incorrectly, etc. In those cases, the "via maintainers" approach continues to make sense, even if painful. >> If you have time to criticize, you have time to write a patch. > > True, and as you pointed, Mathias did. > > I can't speak for others, but right now, the prospect of spending some > of my free time making the jump from > browsing through the grsecurity patch and trying to determine which > hunks relate to what and how they're useful to > actually making + testing + writing a changelog explaining to > maintainers not necessarily caring about security why the change If it helps, the other people on this mailing list care very much about security, so you're not alone in the effort of convincing maintainers of things they might not understand. With that said, that is fundamentally the cost of protecting the users of Linux. Linux is successful because of the specialization and diversity of its maintainers. In the same way not everyone can be a security expert, not everyone can be a USB expert, or a filesystem expert. > makes sense and how it changes the binary's size + submitting > patches to mainline (with wrong credit, it's not my work in the > first place) and most of all > watch a sizable subset of these patches be either dropped on the > floor without a comment (this is not a theoretical concern, that's And this flaw in the upstream culture is why I started KSPP: there IS a group of upstream developers that care about security, and we needed a single umbrella to stand under to help each other with developing and landing important defenses. Sometimes there is silence or incredulity for a patch, but that is part of what I hope KSPP will help shield contributors from. > precisely what happened to most patches in Emese Revfy's large > constification patchset years ago) or outright rejected because Yup, that was a shame. And it's that past cultural problem I want KSPP to help fix. And I think it's had some success there. Take the tty hardening patch being recently discussed. It hasn't landed yet, sure, but it wasn't ignored. Ways to improve it were found, etc. This is a big shift, IMO. Maybe I'm seeing it wrong, but I think the culture has moved. > they _currently_ bring no functional change on a mainline kernel > is... unappealing. Agreed. At the end of the day, no matter how the upstream culture improves, it is time consuming and frequently frustrating work to upstream things. That's why I don't think less of anyone who doesn't contribute. But when someone can spend the time criticizing things without trying to actually fix it... that's pretty disappointing. A lot of people see upstreaming just as developing code, but there's a lot more to helping fix the culture. Chime in on threads to help explain stuff, test code, etc. This is a community, and building it constructively takes time and effort. > Let's list several families of smaller cleanups and improvements made > in PaX/grsec. You (Kees, Mathias) are fully aware about them but maybe > some other readers aren't familiar with the contents of the monster > patches: > * using explicit struct members throughout the code, which is a > prerequisite for making RANDSTRUCT (probably bound for mainline at some > point, AFAIK) more broadly useful than if it's limited to a handful > of whitelisted structs. grsecurity contains hundreds of such hunks. I > think that 4 randomly chosen, simple and related samples are in > > drivers/net/wan/z85230.c; Yup. And getting those into upstream isn't hard at all. Most maintainers trivially take those patches, as you can see: $ git log --oneline | grep -i 'use designated initializers' | wc -l 30 As for z85230.c itself, that appears not to get covered by an x86, arm, or arm64 allyesconfig nor allmodconfig, since I didn't trip over it when porting randstruct: https://git.kernel.org/pub/scm/linux/kernel/git/kees/linux.git/log/?h=kspp/gcc-plugin/randstruct-next-20170418 And that's what I mean about incremental improvements. I hope to land the core of rand struct soon, and as others find things that need the designated initializers, they'll get fixed. And then we can move on to the discussions about how to deal with unnamed structs in task_struct (a gcc 4.6+ feature) for controlling what/where randomization is possible, etc, etc. Without forward progress, there is NO progress. And by the way, while porting randstruct, I landed -Werror=designated-init and adjusted the plugin to emit that annotation so mistakes could be found at build time: https://lkml.org/lkml/2017/3/21/604 And then offered the improvements to grsecurity, which was met with silence: http://www.openwall.com/lists/kernel-hardening/2017/02/01/9 > * building constant structures at compile-time (and making them const) > rather than emitting both the room for that structure in a writable > section (...) or on the stack, and the code to initialize that > structure. That's related to CONSTIFY, it yields safer and usually > smaller code+data. Over the years, overwriting needlessly writable ops > structs proved to be used multiple times in exploit PoCs for mainline > kernels, which often failed miserably on PaX/grsecurity (not just > thanks to CONSTIFY, two or three among MEMORY_UDEREF, KERNEXEC and > RANDSTRUCT foiled the PoC as well). > See the hunks for (I think) > > drivers/cpufreq/sparc-us3-cpufreq.c > drivers/gpu/drm/armada/armada_drv.c > others in drivers/gpu/drm/ (constifying "*_max_ioctl(s?)" variables > initialized with ARRAY_SIZE(a constant array of structs)) > drivers/media/platform/omap/omap_vout.c > drivers/memory/omap-gpmc.c > drivers/mfd/rn5t618.c > drivers/net/ethernet/hisilicon/hns/hns_dsaf_misc.* > drivers/net/wireless/ath/wil6210/pcie_bus.c > drivers/net/ethernet/amd/xgbe/* (most of the hunks in these files) > drivers/video/fbdev/aty/aty128fb.c > net/can/gw.c > include/net/sctp/checksum.h > ... and hundreds of others; Yup, this is a major area of kernel attack surface reduction. I kick-started upstream work on this with the introduction of __ro_after_init which could cover a subset of these changes, and it's being used more and more. Some coccinelle work went into further constification (and checkpatch work) of ops structures, and with the port of __read_only and the constify plugin: https://git.kernel.org/pub/scm/linux/kernel/git/kees/linux.git/log/?h=kspp/write-rarely upstream is getting closer to having even more of this coverage. In the meantime, these are all excellent individual patches to send upstream. Here's a less trivial one I ported from grsecurity, again, taking by the maintainer immediately: https://patchwork.kernel.org/patch/9570905/ > * getting rid of some of the wrong casts between incompatible function > pointer types. This often does not require adding wrappers, and > sometimes even makes the code _more_ readable. These would be non- > functional changes on a mainline kernel, and therefore especially > likely to be ignored or rejected by maintainers. Again, hundreds of > such hunks, which made up the bulk of the PaX & grsecurity patches' > size increase when RAP was added. For instance, AFAICS: > > drivers/gpu/drm/*/psb_intel_sdvo.c and others: returning "enum > drm_mode_status" instead of the less precise "int". > > drivers/media/dvb-frontends/cx*.c and others: likewise, returning > "enum dvbfe_algo" instead of "int". > > drivers/net/wireless/atmel/atmel.c > drivers/net/wireless/cisco/airo.c > drivers/net/wireless/intersil/hostap/hostap_ioctl.c > drivers/net/wireless/realtek/rtlwifi/pci.c > (+ other WiFi drivers) > arch/{arm,x86}/crypto/sha*glue.* > fs/coda/dir.c The function pointer type fixes are a huge clean-up and would have appeared "unjustified" to maintainers. (For a sense of scale, just look at the delta between the v4.4 grsecurity patch and the v4.5 patch. The bulk of the changes there were for RAP function pointer type fixes.) However that's a matter of describing why it's a useful change, and this is kind of thing the KSPP tree would be perfect for carrying. No one has spent the time on these patches yet, since RAP is pretty far down on the TODO list, especially given that it's x86_64-only. But that doesn't mean the fixes wouldn't be welcome. > * other samples of the untapped tidbits from PaX/grsec: > > * removing spurious "static" keywords on function-local variable > declarations when the first thing the function does with the > variable is to reinitialize it. See e.g. the > > drivers/mfd/max8925-i2c.c > drivers/mfd/tps65910.c > > hunks, both of which were already in a grsecurity patch for 3.14 > from May 2014 laying around on my HDD. This sounds like a great coccinelle rule to generate and fix these everywhere in the kernel. > * annotations in e.g. > > include/linux/seqlock.h > include/linux/spinlock.h It'd be great to find which testing tool noticed these being missing and see if there are others. Again, please upstream them. > Integrating into mainline hundreds of these smaller improvements and > cleanups from PaX/grsec, i.e. even a single-digit percentage of the > grsecurity patch's hunks, would demonstrably improve the mainline code > immediately or in the mid-term... but that integration work is harder > than it could be :( And that is what KSPP is here to help with. We'll help new contributors get this work done. Thanks! -Kees -- Kees Cook Pixel Security ^ permalink raw reply [flat|nested] 35+ messages in thread
end of thread, other threads:[~2017-07-31 13:38 UTC | newest] Thread overview: 35+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2017-04-26 21:05 [kernel-hardening] It looks like there will be no more public versions of PaX and Grsec Daniel Cegiełka 2017-04-26 22:04 ` Kees Cook 2017-05-01 22:01 ` Mathias Krause 2017-05-02 0:09 ` Rik van Riel 2017-05-02 14:46 ` Shawn 2017-05-02 18:55 ` Kees Cook 2017-05-03 4:50 ` Shawn 2017-05-03 18:56 ` Rik van Riel 2017-05-03 19:36 ` Daniel Micay 2017-05-04 5:45 ` Kees Cook 2017-05-04 6:47 ` Lionel Debroux 2017-05-05 19:54 ` Kees Cook 2017-05-04 14:11 ` Shawn 2017-05-04 16:03 ` Greg KH 2017-05-04 17:12 ` Shawn 2017-05-04 17:23 ` Greg KH 2017-05-02 21:16 ` Mathias Krause 2017-05-02 21:50 ` Casey Schaufler 2017-05-02 22:57 ` Kees Cook 2017-05-03 19:02 ` Rik van Riel 2017-05-03 19:27 ` Daniel Micay 2017-05-02 0:39 ` Olof Johansson 2017-05-02 0:44 ` Casey Schaufler 2017-05-02 0:54 ` Kees Cook 2017-05-11 1:24 ` PaX Team 2017-05-11 16:30 ` Daniel Micay 2017-05-11 18:02 ` Kees Cook 2017-05-12 11:34 ` Hunger 2017-07-31 13:38 ` Solar Designer 2017-05-02 11:11 ` David Gens 2017-05-02 21:27 ` Mathias Krause 2017-05-03 8:59 ` David Gens 2017-05-03 19:10 ` Rik van Riel [not found] <1788778362.1495506.1493751985632.ref@mail.yahoo.com> 2017-05-02 19:06 ` Lionel Debroux 2017-05-02 22:35 ` Kees Cook
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.