From: Andy Lutomirski <luto@kernel.org> To: PaX Team <pageexec@freemail.hu> Cc: Andy Lutomirski <luto@kernel.org>, Daniel Micay <danielmicay@gmail.com>, Mathias Krause <minipli@googlemail.com>, Thomas Gleixner <tglx@linutronix.de>, Kees Cook <keescook@chromium.org>, "kernel-hardening@lists.openwall.com" <kernel-hardening@lists.openwall.com>, Mark Rutland <mark.rutland@arm.com>, Hoeun Ryu <hoeun.ryu@gmail.com>, Emese Revfy <re.emese@gmail.com>, Russell King <linux@armlinux.org.uk>, X86 ML <x86@kernel.org>, "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>, "linux-arm-kernel@lists.infradead.org" <linux-arm-kernel@lists.infradead.org>, Peter Zijlstra <peterz@infradead.org> Subject: Re: [kernel-hardening] Re: [RFC v2][PATCH 04/11] x86: Implement __arch_rare_write_begin/unmap() Date: Mon, 10 Apr 2017 13:27:47 -0700 [thread overview] Message-ID: <CALCETrVrbCxQraJ7ZWxn_S0CV_Vmz=e4hWr2503+Ha9Fva5xNQ@mail.gmail.com> (raw) In-Reply-To: <58EBE153.31145.71853724@pageexec.freemail.hu> On Mon, Apr 10, 2017 at 12:47 PM, PaX Team <pageexec@freemail.hu> wrote: > On 9 Apr 2017 at 17:31, Andy Lutomirski wrote: > >> On Sun, Apr 9, 2017 at 1:24 PM, PaX Team <pageexec@freemail.hu> wrote: >> > >> I consider breaking buggy drivers (in a way that they either generally >> work okay > > do they work okay when the dma transfer goes to a buffer that crosses > physically non-contiguous page boundaries? Nope. Like I said, i considered making the debugging mandatory. I may still send patches to do that. >> By "leak" I mean that a bug or exploit causes unintended code to run >> with CR0.WP or a special CR3 or a special PTE or whatever loaded. > > how can a bug/exploit cause something like this? For example: a bug in entry logic, a bug in perf NMI handling, or even a bug in *nested* perf NMI handling (egads!). Or maybe some super nasty interaction with suspend/resume. These are all fairly unlikely (except the nested perf case), but still. As a concrete example, back before my big NMI improvement series, it was possible for an NMI return to invoke espfix and/or take an IRET fault. This *shouldn't* happen on return to a context with CR0.WP set, but it would be incredibly nasty if it did. The code is separated out now, so it should be okay... > >> PaX hooks the entry code to avoid leaks. > > PaX doesn't instrument enter/exit paths to prevent state leaks into interrupt > context (it's a useful sideeffect though), rather it's needed for correctness > if the kernel can be interrupted at all while it's open (address space switching > will need to handle this too but you have yet to address it). I don't think we disagree here. A leak would be a case of incorrectness. > >> >> At boot, choose a random address A. >> > >> > what is the threat that a random address defends against? >> >> Makes it harder to exploit a case where the CR3 setting leaks. > > if an attacker has the ability to cause this leak (details of which are subject > to the question i asked above) then why wouldn't he simply also make use of the > primitives to modify his target via the writable vma without ever having to know > the randomized address? i also wonder what exploit power you assume for this > attack and whether that is already enough to simply go after page tables, etc > instead of figuring out the alternative address space. I'm imagining the power to (a) cause some code path to execute while the kernel is "open" and (b) the ability to use the buggy code path in question to write a a fully- or partially-controlled address. With CR0.WP clear, this can write shellcode directly. With CR3 pointing to a page table that maps some parts of the kernel (but not text!) at a randomized offset, you need to figure out the offset and find some other target in the mapping that gets your exploit farther along. You can't write shellcode directly. > >> > the problem is that the amount of __read_only data extends beyond vmlinux, >> > i.e., this approach won't scale. another problem is that it can't be used >> > inside use_mm and switch_mm themselves (no read-only task structs or percpu >> > pgd for you ;) and probably several other contexts. >> >> Can you clarify these uses that extend beyond vmlinux? > > one obvious candidate is modules. how do you want to handle them? then there's > a whole bunch of dynamically allocated data that is a candidate for __read_only > treatment. Exactly the same way. Map those regions at the same offset, maybe even in the same VMA. There's no reason that an artificial VMA used for this purpose can't be many gigabytes long and have vm_ops that only allow access to certain things. But multiple VMAs would work, too. > >> > what is the threat model you're assuming for this feature? based on what i >> > have for PaX (arbitrary read/write access exploited for data-only attacks), >> > the above makes no sense to me... >> >> If I use the primitive to try to write a value to the wrong section >> (write to kernel text, for example), IMO it would be nice to OOPS >> instead of succeeding. > > this doesn't tell me what power you're assuming the attacker has. is it > my generic arbitrary read-write ability or something more restricted and > thus less realistic? i.e., how does the attacker get to 'use the primitive' > and (presumably) also control the ptr/data? > > as for your specific example, kernel text isn't 'non-rare-write data' that > you spoke of before, but that aside, what prevents an attacker from computing > his target ptr so that after your accessor rebases it, it'd point back to his > intended target instead? It's a restriction on what targets can be hit. With CR0.WP, you can hit anything that has a VA. With CR3, you can hit only that which is mapped. > will you range-check (find_vma eventually?) each time? > how will you make all this code safe from races from another task? the more > checks you make, the more likely that something sensitive will spill to memory > and be a target itself in order to hijack the sensitive write. There's no code here making the checks at write time. It's just page table / VMA setup. > >> Please keep in mind that, unlike PaX, uses of a pax_open_kernel()-like >> function will may be carefully audited by a friendly security expert >> such as yourself. It would be nice to harden the primitive to a >> reasonable extent against minor misuses such as putting it in a >> context where the compiler will emit mov-a-reg-with-WP-set-to-CR0; >> ret. > > i don't understand what's there to audit. if you want to treat a given piece > of data as __read_only then you have no choice but to allow writes to it via > the open/close mechanism and the compiler can tell you just where those > writes are (and even do the instrumentation when you get tired of doing it > by hand). > I mean auditing all uses of pax_open_kernel() or any other function that opens the kernel. That function is, as used in PaX, terrifying. PaX probably gets every user right, but I don't trust driver writers with a function like pax_open_kernel() that's as powerful as PaX's. Suppose you get driver code like this: void foo(int (*func)()) { pax_open_kernel(); *thingy = func(); pax_close_kernel(); } That would be a very, very juicy target for a ROP-like attack. Just get the kernel to call this function with func pointing to something that does a memcpy or similar into executable space. Boom, shellcode execution. If CR3 is used instead, exploiting this is considerably more complicated.
WARNING: multiple messages have this Message-ID (diff)
From: luto@kernel.org (Andy Lutomirski) To: linux-arm-kernel@lists.infradead.org Subject: [kernel-hardening] Re: [RFC v2][PATCH 04/11] x86: Implement __arch_rare_write_begin/unmap() Date: Mon, 10 Apr 2017 13:27:47 -0700 [thread overview] Message-ID: <CALCETrVrbCxQraJ7ZWxn_S0CV_Vmz=e4hWr2503+Ha9Fva5xNQ@mail.gmail.com> (raw) In-Reply-To: <58EBE153.31145.71853724@pageexec.freemail.hu> On Mon, Apr 10, 2017 at 12:47 PM, PaX Team <pageexec@freemail.hu> wrote: > On 9 Apr 2017 at 17:31, Andy Lutomirski wrote: > >> On Sun, Apr 9, 2017 at 1:24 PM, PaX Team <pageexec@freemail.hu> wrote: >> > >> I consider breaking buggy drivers (in a way that they either generally >> work okay > > do they work okay when the dma transfer goes to a buffer that crosses > physically non-contiguous page boundaries? Nope. Like I said, i considered making the debugging mandatory. I may still send patches to do that. >> By "leak" I mean that a bug or exploit causes unintended code to run >> with CR0.WP or a special CR3 or a special PTE or whatever loaded. > > how can a bug/exploit cause something like this? For example: a bug in entry logic, a bug in perf NMI handling, or even a bug in *nested* perf NMI handling (egads!). Or maybe some super nasty interaction with suspend/resume. These are all fairly unlikely (except the nested perf case), but still. As a concrete example, back before my big NMI improvement series, it was possible for an NMI return to invoke espfix and/or take an IRET fault. This *shouldn't* happen on return to a context with CR0.WP set, but it would be incredibly nasty if it did. The code is separated out now, so it should be okay... > >> PaX hooks the entry code to avoid leaks. > > PaX doesn't instrument enter/exit paths to prevent state leaks into interrupt > context (it's a useful sideeffect though), rather it's needed for correctness > if the kernel can be interrupted at all while it's open (address space switching > will need to handle this too but you have yet to address it). I don't think we disagree here. A leak would be a case of incorrectness. > >> >> At boot, choose a random address A. >> > >> > what is the threat that a random address defends against? >> >> Makes it harder to exploit a case where the CR3 setting leaks. > > if an attacker has the ability to cause this leak (details of which are subject > to the question i asked above) then why wouldn't he simply also make use of the > primitives to modify his target via the writable vma without ever having to know > the randomized address? i also wonder what exploit power you assume for this > attack and whether that is already enough to simply go after page tables, etc > instead of figuring out the alternative address space. I'm imagining the power to (a) cause some code path to execute while the kernel is "open" and (b) the ability to use the buggy code path in question to write a a fully- or partially-controlled address. With CR0.WP clear, this can write shellcode directly. With CR3 pointing to a page table that maps some parts of the kernel (but not text!) at a randomized offset, you need to figure out the offset and find some other target in the mapping that gets your exploit farther along. You can't write shellcode directly. > >> > the problem is that the amount of __read_only data extends beyond vmlinux, >> > i.e., this approach won't scale. another problem is that it can't be used >> > inside use_mm and switch_mm themselves (no read-only task structs or percpu >> > pgd for you ;) and probably several other contexts. >> >> Can you clarify these uses that extend beyond vmlinux? > > one obvious candidate is modules. how do you want to handle them? then there's > a whole bunch of dynamically allocated data that is a candidate for __read_only > treatment. Exactly the same way. Map those regions at the same offset, maybe even in the same VMA. There's no reason that an artificial VMA used for this purpose can't be many gigabytes long and have vm_ops that only allow access to certain things. But multiple VMAs would work, too. > >> > what is the threat model you're assuming for this feature? based on what i >> > have for PaX (arbitrary read/write access exploited for data-only attacks), >> > the above makes no sense to me... >> >> If I use the primitive to try to write a value to the wrong section >> (write to kernel text, for example), IMO it would be nice to OOPS >> instead of succeeding. > > this doesn't tell me what power you're assuming the attacker has. is it > my generic arbitrary read-write ability or something more restricted and > thus less realistic? i.e., how does the attacker get to 'use the primitive' > and (presumably) also control the ptr/data? > > as for your specific example, kernel text isn't 'non-rare-write data' that > you spoke of before, but that aside, what prevents an attacker from computing > his target ptr so that after your accessor rebases it, it'd point back to his > intended target instead? It's a restriction on what targets can be hit. With CR0.WP, you can hit anything that has a VA. With CR3, you can hit only that which is mapped. > will you range-check (find_vma eventually?) each time? > how will you make all this code safe from races from another task? the more > checks you make, the more likely that something sensitive will spill to memory > and be a target itself in order to hijack the sensitive write. There's no code here making the checks at write time. It's just page table / VMA setup. > >> Please keep in mind that, unlike PaX, uses of a pax_open_kernel()-like >> function will may be carefully audited by a friendly security expert >> such as yourself. It would be nice to harden the primitive to a >> reasonable extent against minor misuses such as putting it in a >> context where the compiler will emit mov-a-reg-with-WP-set-to-CR0; >> ret. > > i don't understand what's there to audit. if you want to treat a given piece > of data as __read_only then you have no choice but to allow writes to it via > the open/close mechanism and the compiler can tell you just where those > writes are (and even do the instrumentation when you get tired of doing it > by hand). > I mean auditing all uses of pax_open_kernel() or any other function that opens the kernel. That function is, as used in PaX, terrifying. PaX probably gets every user right, but I don't trust driver writers with a function like pax_open_kernel() that's as powerful as PaX's. Suppose you get driver code like this: void foo(int (*func)()) { pax_open_kernel(); *thingy = func(); pax_close_kernel(); } That would be a very, very juicy target for a ROP-like attack. Just get the kernel to call this function with func pointing to something that does a memcpy or similar into executable space. Boom, shellcode execution. If CR3 is used instead, exploiting this is considerably more complicated.
next prev parent reply other threads:[~2017-04-10 20:28 UTC|newest] Thread overview: 188+ messages / expand[flat|nested] mbox.gz Atom feed top 2017-03-29 18:15 [RFC v2] Introduce rare_write() infrastructure Kees Cook 2017-03-29 18:15 ` [kernel-hardening] " Kees Cook 2017-03-29 18:15 ` Kees Cook 2017-03-29 18:15 ` [RFC v2][PATCH 01/11] " Kees Cook 2017-03-29 18:15 ` [kernel-hardening] " Kees Cook 2017-03-29 18:15 ` Kees Cook 2017-03-29 18:23 ` Kees Cook 2017-03-29 18:23 ` [kernel-hardening] " Kees Cook 2017-03-29 18:23 ` Kees Cook 2017-03-30 7:44 ` Ho-Eun Ryu 2017-03-30 7:44 ` [kernel-hardening] " Ho-Eun Ryu 2017-03-30 7:44 ` Ho-Eun Ryu 2017-03-30 17:02 ` Kees Cook 2017-03-30 17:02 ` [kernel-hardening] " Kees Cook 2017-03-30 17:02 ` Kees Cook 2017-04-07 8:09 ` Ho-Eun Ryu 2017-04-07 8:09 ` [kernel-hardening] " Ho-Eun Ryu 2017-04-07 8:09 ` Ho-Eun Ryu 2017-04-07 20:38 ` Kees Cook 2017-04-07 20:38 ` [kernel-hardening] " Kees Cook 2017-04-07 20:38 ` Kees Cook 2017-03-29 18:15 ` [RFC v2][PATCH 02/11] lkdtm: add test for " Kees Cook 2017-03-29 18:15 ` [kernel-hardening] " Kees Cook 2017-03-29 18:15 ` Kees Cook 2017-03-30 9:34 ` [kernel-hardening] " Ian Campbell 2017-03-30 9:34 ` Ian Campbell 2017-03-30 16:16 ` Kees Cook 2017-03-30 16:16 ` Kees Cook 2017-03-30 16:16 ` Kees Cook 2017-03-29 18:15 ` [RFC v2][PATCH 03/11] net: switch sock_diag handlers to rare_write() Kees Cook 2017-03-29 18:15 ` [kernel-hardening] " Kees Cook 2017-03-29 18:15 ` Kees Cook 2017-03-29 18:15 ` [RFC v2][PATCH 04/11] x86: Implement __arch_rare_write_begin/unmap() Kees Cook 2017-03-29 18:15 ` [kernel-hardening] " Kees Cook 2017-03-29 18:15 ` Kees Cook 2017-03-29 22:38 ` Andy Lutomirski 2017-03-29 22:38 ` [kernel-hardening] " Andy Lutomirski 2017-03-29 22:38 ` Andy Lutomirski 2017-03-30 1:41 ` Kees Cook 2017-03-30 1:41 ` [kernel-hardening] " Kees Cook 2017-03-30 1:41 ` Kees Cook 2017-04-05 23:57 ` Andy Lutomirski 2017-04-05 23:57 ` [kernel-hardening] " Andy Lutomirski 2017-04-05 23:57 ` Andy Lutomirski 2017-04-06 0:14 ` Kees Cook 2017-04-06 0:14 ` [kernel-hardening] " Kees Cook 2017-04-06 0:14 ` Kees Cook 2017-04-06 15:59 ` Andy Lutomirski 2017-04-06 15:59 ` [kernel-hardening] " Andy Lutomirski 2017-04-06 15:59 ` Andy Lutomirski 2017-04-07 8:34 ` [kernel-hardening] " Mathias Krause 2017-04-07 8:34 ` Mathias Krause 2017-04-07 8:34 ` Mathias Krause 2017-04-07 9:46 ` Thomas Gleixner 2017-04-07 9:46 ` Thomas Gleixner 2017-04-07 9:46 ` Thomas Gleixner 2017-04-07 10:51 ` Mathias Krause 2017-04-07 10:51 ` Mathias Krause 2017-04-07 10:51 ` Mathias Krause 2017-04-07 13:14 ` Thomas Gleixner 2017-04-07 13:14 ` Thomas Gleixner 2017-04-07 13:14 ` Thomas Gleixner 2017-04-07 13:30 ` Mathias Krause 2017-04-07 13:30 ` Mathias Krause 2017-04-07 13:30 ` Mathias Krause 2017-04-07 16:14 ` Andy Lutomirski 2017-04-07 16:14 ` Andy Lutomirski 2017-04-07 16:14 ` Andy Lutomirski 2017-04-07 16:22 ` Mark Rutland 2017-04-07 16:22 ` Mark Rutland 2017-04-07 16:22 ` Mark Rutland 2017-04-07 19:58 ` PaX Team 2017-04-07 19:58 ` PaX Team 2017-04-07 19:58 ` PaX Team 2017-04-08 4:58 ` Andy Lutomirski 2017-04-08 4:58 ` Andy Lutomirski 2017-04-08 4:58 ` Andy Lutomirski 2017-04-09 12:47 ` PaX Team 2017-04-09 12:47 ` PaX Team 2017-04-09 12:47 ` PaX Team 2017-04-10 0:10 ` Andy Lutomirski 2017-04-10 0:10 ` Andy Lutomirski 2017-04-10 0:10 ` Andy Lutomirski 2017-04-10 10:42 ` PaX Team 2017-04-10 10:42 ` PaX Team 2017-04-10 10:42 ` PaX Team 2017-04-10 16:01 ` Andy Lutomirski 2017-04-10 16:01 ` Andy Lutomirski 2017-04-10 16:01 ` Andy Lutomirski 2017-04-07 20:44 ` Thomas Gleixner 2017-04-07 20:44 ` Thomas Gleixner 2017-04-07 20:44 ` Thomas Gleixner 2017-04-07 21:20 ` Kees Cook 2017-04-07 21:20 ` Kees Cook 2017-04-07 21:20 ` Kees Cook 2017-04-08 4:12 ` Daniel Micay 2017-04-08 4:12 ` Daniel Micay 2017-04-08 4:12 ` Daniel Micay 2017-04-08 4:13 ` Daniel Micay 2017-04-08 4:13 ` Daniel Micay 2017-04-08 4:13 ` Daniel Micay 2017-04-08 4:21 ` Daniel Micay 2017-04-08 4:21 ` Daniel Micay 2017-04-08 4:21 ` Daniel Micay 2017-04-08 5:07 ` Andy Lutomirski 2017-04-08 5:07 ` Andy Lutomirski 2017-04-08 5:07 ` Andy Lutomirski 2017-04-08 7:33 ` Daniel Micay 2017-04-08 7:33 ` Daniel Micay 2017-04-08 7:33 ` Daniel Micay 2017-04-08 15:20 ` Andy Lutomirski 2017-04-08 15:20 ` Andy Lutomirski 2017-04-08 15:20 ` Andy Lutomirski 2017-04-09 10:53 ` Ingo Molnar 2017-04-09 10:53 ` Ingo Molnar 2017-04-09 10:53 ` Ingo Molnar 2017-04-10 10:22 ` Mark Rutland 2017-04-10 10:22 ` Mark Rutland 2017-04-10 10:22 ` Mark Rutland 2017-04-09 20:24 ` PaX Team 2017-04-09 20:24 ` PaX Team 2017-04-09 20:24 ` PaX Team 2017-04-10 0:31 ` Andy Lutomirski 2017-04-10 0:31 ` Andy Lutomirski 2017-04-10 0:31 ` Andy Lutomirski 2017-04-10 19:47 ` PaX Team 2017-04-10 19:47 ` PaX Team 2017-04-10 19:47 ` PaX Team 2017-04-10 20:27 ` Andy Lutomirski [this message] 2017-04-10 20:27 ` Andy Lutomirski 2017-04-10 20:27 ` Andy Lutomirski 2017-04-10 20:13 ` Kees Cook 2017-04-10 20:13 ` Kees Cook 2017-04-10 20:13 ` Kees Cook 2017-04-10 20:17 ` Andy Lutomirski 2017-04-10 20:17 ` Andy Lutomirski 2017-04-10 20:17 ` Andy Lutomirski 2017-04-07 19:25 ` Thomas Gleixner 2017-04-07 19:25 ` Thomas Gleixner 2017-04-07 19:25 ` Thomas Gleixner 2017-04-07 14:45 ` Peter Zijlstra 2017-04-07 14:45 ` Peter Zijlstra 2017-04-07 14:45 ` Peter Zijlstra 2017-04-10 10:29 ` Mark Rutland 2017-04-10 10:29 ` Mark Rutland 2017-04-10 10:29 ` Mark Rutland 2017-04-07 19:52 ` PaX Team 2017-04-07 19:52 ` PaX Team 2017-04-07 19:52 ` PaX Team 2017-04-10 8:26 ` Thomas Gleixner 2017-04-10 8:26 ` Thomas Gleixner 2017-04-10 8:26 ` Thomas Gleixner 2017-04-10 19:55 ` PaX Team 2017-04-10 19:55 ` PaX Team 2017-04-10 19:55 ` PaX Team 2017-04-07 9:37 ` Peter Zijlstra 2017-04-07 9:37 ` [kernel-hardening] " Peter Zijlstra 2017-04-07 9:37 ` Peter Zijlstra 2017-03-29 18:15 ` [RFC v2][PATCH 05/11] ARM: mm: dump: Add domain to output Kees Cook 2017-03-29 18:15 ` [kernel-hardening] " Kees Cook 2017-03-29 18:15 ` Kees Cook 2017-03-29 18:15 ` [RFC v2][PATCH 06/11] ARM: domains: Extract common USER domain init Kees Cook 2017-03-29 18:15 ` [kernel-hardening] " Kees Cook 2017-03-29 18:15 ` Kees Cook 2017-03-29 18:15 ` [RFC v2][PATCH 07/11] ARM: mm: set DOMAIN_WR_RARE for rodata Kees Cook 2017-03-29 18:15 ` [kernel-hardening] " Kees Cook 2017-03-29 18:15 ` Kees Cook 2017-03-29 18:16 ` [RFC v2][PATCH 08/11] ARM: Implement __arch_rare_write_begin/end() Kees Cook 2017-03-29 18:16 ` [kernel-hardening] " Kees Cook 2017-03-29 18:16 ` Kees Cook 2017-04-07 9:36 ` Peter Zijlstra 2017-04-07 9:36 ` [kernel-hardening] " Peter Zijlstra 2017-04-07 9:36 ` Peter Zijlstra 2017-03-29 18:16 ` [RFC v2][PATCH 09/11] list: add rare_write() list helpers Kees Cook 2017-03-29 18:16 ` [kernel-hardening] " Kees Cook 2017-03-29 18:16 ` Kees Cook 2017-03-29 18:16 ` [RFC v2][PATCH 10/11] gcc-plugins: Add constify plugin Kees Cook 2017-03-29 18:16 ` [kernel-hardening] " Kees Cook 2017-03-29 18:16 ` Kees Cook 2017-03-29 18:16 ` [RFC v2][PATCH 11/11] cgroups: force all struct cftype const Kees Cook 2017-03-29 18:16 ` [kernel-hardening] " Kees Cook 2017-03-29 18:16 ` Kees Cook 2017-03-29 19:00 ` [RFC v2] Introduce rare_write() infrastructure Russell King - ARM Linux 2017-03-29 19:00 ` [kernel-hardening] " Russell King - ARM Linux 2017-03-29 19:00 ` Russell King - ARM Linux 2017-03-29 19:14 ` Kees Cook 2017-03-29 19:14 ` [kernel-hardening] " Kees Cook 2017-03-29 19:14 ` Kees Cook
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to='CALCETrVrbCxQraJ7ZWxn_S0CV_Vmz=e4hWr2503+Ha9Fva5xNQ@mail.gmail.com' \ --to=luto@kernel.org \ --cc=danielmicay@gmail.com \ --cc=hoeun.ryu@gmail.com \ --cc=keescook@chromium.org \ --cc=kernel-hardening@lists.openwall.com \ --cc=linux-arm-kernel@lists.infradead.org \ --cc=linux-kernel@vger.kernel.org \ --cc=linux@armlinux.org.uk \ --cc=mark.rutland@arm.com \ --cc=minipli@googlemail.com \ --cc=pageexec@freemail.hu \ --cc=peterz@infradead.org \ --cc=re.emese@gmail.com \ --cc=tglx@linutronix.de \ --cc=x86@kernel.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.