From: Khalid Aziz <khalid.aziz@oracle.com>
To: Kees Cook <keescook@chromium.org>
Cc: Andy Lutomirski <luto@kernel.org>,
Dave Hansen <dave.hansen@intel.com>,
Ingo Molnar <mingo@kernel.org>,
Juerg Haefliger <juergh@gmail.com>,
Tycho Andersen <tycho@tycho.ws>,
jsteckli@amazon.de, Andi Kleen <ak@linux.intel.com>,
Linus Torvalds <torvalds@linux-foundation.org>,
liran.alon@oracle.com,
Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>,
deepa.srinivasan@oracle.com, chris hyser <chris.hyser@oracle.com>,
Tyler Hicks <tyhicks@canonical.com>,
"Woodhouse, David" <dwmw@amazon.co.uk>,
Andrew Cooper <andrew.cooper3@citrix.com>,
Jon Masters <jcm@redhat.com>,
Boris Ostrovsky <boris.ostrovsky@oracle.com>,
kanth.ghatraju@oracle.com, joao.m.martins@oracle.com,
Jim Mattson <jmattson@google.com>,
pradeep.vincent@oracle.com, John Haxby <john.haxby@oracle.com>,
"Kirill A. Shutemov" <kirill.shutemov@linux.intel.com>,
Christoph Hellwig <hch@lst.de>,
steven.sistare@oracle.com,
Kernel Hardening <kernel-hardening@lists.openwall.com>,
Linux-MM <linux-mm@kvack.org>,
LKML <linux-kernel@vger.kernel.org>,
Thomas Gleixner <tglx@linutronix.de>
Subject: Re: [RFC PATCH v7 00/16] Add support for eXclusive Page Frame Ownership
Date: Thu, 10 Jan 2019 17:20:42 -0700 [thread overview]
Message-ID: <e8df9859-4629-52d7-9807-d87f1820a01d@oracle.com> (raw)
In-Reply-To: <CAGXu5jKS8XSw7nByaeXqgPbmRRw01E_zUYxLCk7zFepAVSw_aQ@mail.gmail.com>
[-- Attachment #1: Type: text/plain, Size: 4505 bytes --]
Thanks for looking this over.
On 1/10/19 4:07 PM, Kees Cook wrote:
> On Thu, Jan 10, 2019 at 1:10 PM Khalid Aziz <khalid.aziz@oracle.com> wrote:
>> I implemented a solution to reduce performance penalty and
>> that has had large impact. When XPFO code flushes stale TLB entries,
>> it does so for all CPUs on the system which may include CPUs that
>> may not have any matching TLB entries or may never be scheduled to
>> run the userspace task causing TLB flush. Problem is made worse by
>> the fact that if number of entries being flushed exceeds
>> tlb_single_page_flush_ceiling, it results in a full TLB flush on
>> every CPU. A rogue process can launch a ret2dir attack only from a
>> CPU that has dual mapping for its pages in physmap in its TLB. We
>> can hence defer TLB flush on a CPU until a process that would have
>> caused a TLB flush is scheduled on that CPU. I have added a cpumask
>> to task_struct which is then used to post pending TLB flush on CPUs
>> other than the one a process is running on. This cpumask is checked
>> when a process migrates to a new CPU and TLB is flushed at that
>> time. I measured system time for parallel make with unmodified 4.20
>> kernel, 4.20 with XPFO patches before this optimization and then
>> again after applying this optimization. Here are the results:
>>
>> Hardware: 96-core Intel Xeon Platinum 8160 CPU @ 2.10GHz, 768 GB RAM
>> make -j60 all
>>
>> 4.20 915.183s
>> 4.20+XPFO 24129.354s 26.366x
>> 4.20+XPFO+Deferred flush 1216.987s 1.330xx
>>
>>
>> Hardware: 4-core Intel Core i5-3550 CPU @ 3.30GHz, 8G RAM
>> make -j4 all
>>
>> 4.20 607.671s
>> 4.20+XPFO 1588.646s 2.614x
>> 4.20+XPFO+Deferred flush 794.473s 1.307xx
>
> Well that's an impressive improvement! Nice work. :)
>
> (Are the cpumask improvements possible to be extended to other TLB
> flushing needs? i.e. could there be other performance gains with that
> code even for a non-XPFO system?)
It may be usable for other situations as well but I have not given it
any thought yet. I will take a look.
>
>> 30+% overhead is still very high and there is room for improvement.
>> Dave Hansen had suggested batch updating TLB entries and Tycho had
>> created an initial implementation but I have not been able to get
>> that to work correctly. I am still working on it and I suspect we
>> will see a noticeable improvement in performance with that. In the
>> code I added, I post a pending full TLB flush to all other CPUs even
>> when number of TLB entries being flushed on current CPU does not
>> exceed tlb_single_page_flush_ceiling. There has to be a better way
>> to do this. I just haven't found an efficient way to implemented
>> delayed limited TLB flush on other CPUs.
>>
>> I am not entirely sure if switch_mm_irqs_off() is indeed the right
>> place to perform the pending TLB flush for a CPU. Any feedback on
>> that will be very helpful. Delaying full TLB flushes on other CPUs
>> seems to help tremendously, so if there is a better way to implement
>> the same thing than what I have done in patch 16, I am open to
>> ideas.
>
> Dave, Andy, Ingo, Thomas, does anyone have time to look this over?
>
>> Performance with this patch set is good enough to use these as
>> starting point for further refinement before we merge it into main
>> kernel, hence RFC.
>>
>> Since not flushing stale TLB entries creates a false sense of
>> security, I would recommend making TLB flush mandatory and eliminate
>> the "xpfotlbflush" kernel parameter (patch "mm, x86: omit TLB
>> flushing by default for XPFO page table modifications").
>
> At this point, yes, that does seem to make sense.
>
>> What remains to be done beyond this patch series:
>>
>> 1. Performance improvements
>> 2. Remove xpfotlbflush parameter
>> 3. Re-evaluate the patch "arm64/mm: Add support for XPFO to swiotlb"
>> from Juerg. I dropped it for now since swiotlb code for ARM has
>> changed a lot in 4.20.
>> 4. Extend the patch "xpfo, mm: Defer TLB flushes for non-current
>> CPUs" to other architectures besides x86.
>
> This seems like a good plan.
>
> I've put this series in one of my tree so that 0day will find it and
> grind tests...
> https://git.kernel.org/pub/scm/linux/kernel/git/kees/linux.git/log/?h=kspp/xpfo/v7
Thanks for doing that!
--
Khalid
[-- Attachment #2: pEpkey.asc --]
[-- Type: application/pgp-keys, Size: 2501 bytes --]
next prev parent reply other threads:[~2019-01-11 0:20 UTC|newest]
Thread overview: 45+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-01-10 21:09 [RFC PATCH v7 00/16] Add support for eXclusive Page Frame Ownership Khalid Aziz
2019-01-10 21:09 ` [RFC PATCH v7 01/16] mm: add MAP_HUGETLB support to vm_mmap Khalid Aziz
2019-01-10 21:09 ` [RFC PATCH v7 02/16] x86: always set IF before oopsing from page fault Khalid Aziz
2019-01-10 21:09 ` [RFC PATCH v7 03/16] mm, x86: Add support for eXclusive Page Frame Ownership (XPFO) Khalid Aziz
2019-01-10 21:09 ` [RFC PATCH v7 04/16] swiotlb: Map the buffer if it was unmapped by XPFO Khalid Aziz
2019-01-23 14:16 ` Konrad Rzeszutek Wilk
2019-01-10 21:09 ` [RFC PATCH v7 05/16] arm64/mm: Add support for XPFO Khalid Aziz
2019-01-23 14:20 ` Konrad Rzeszutek Wilk
2019-02-12 15:45 ` Khalid Aziz
2019-01-23 14:24 ` Konrad Rzeszutek Wilk
2019-02-12 15:52 ` Khalid Aziz
2019-02-12 20:01 ` Laura Abbott
2019-02-12 20:34 ` Khalid Aziz
2019-01-10 21:09 ` [RFC PATCH v7 06/16] xpfo: add primitives for mapping underlying memory Khalid Aziz
2019-01-10 21:09 ` [RFC PATCH v7 07/16] arm64/mm, xpfo: temporarily map dcache regions Khalid Aziz
2019-01-11 14:54 ` Tycho Andersen
2019-01-11 18:28 ` Khalid Aziz
2019-01-11 19:50 ` Tycho Andersen
2019-01-23 14:56 ` Konrad Rzeszutek Wilk
2019-01-10 21:09 ` [RFC PATCH v7 08/16] arm64/mm: disable section/contiguous mappings if XPFO is enabled Khalid Aziz
2019-01-10 21:09 ` [RFC PATCH v7 09/16] mm: add a user_virt_to_phys symbol Khalid Aziz
2019-01-23 15:03 ` Konrad Rzeszutek Wilk
2019-01-10 21:09 ` [RFC PATCH v7 10/16] lkdtm: Add test for XPFO Khalid Aziz
2019-01-10 21:09 ` [RFC PATCH v7 11/16] mm, x86: omit TLB flushing by default for XPFO page table modifications Khalid Aziz
2019-01-10 21:09 ` [RFC PATCH v7 12/16] xpfo, mm: remove dependency on CONFIG_PAGE_EXTENSION Khalid Aziz
2019-01-10 21:09 ` [RFC PATCH v7 13/16] xpfo, mm: optimize spinlock usage in xpfo_kunmap Khalid Aziz
2019-01-10 21:09 ` [RFC PATCH v7 14/16] EXPERIMENTAL: xpfo, mm: optimize spin lock usage in xpfo_kmap Khalid Aziz
2019-01-17 0:18 ` Laura Abbott
2019-01-17 15:14 ` Khalid Aziz
2019-01-10 21:09 ` [RFC PATCH v7 15/16] xpfo, mm: Fix hang when booting with "xpfotlbflush" Khalid Aziz
2019-01-10 21:09 ` [RFC PATCH v7 16/16] xpfo, mm: Defer TLB flushes for non-current CPUs (x86 only) Khalid Aziz
2019-01-10 23:07 ` [RFC PATCH v7 00/16] Add support for eXclusive Page Frame Ownership Kees Cook
2019-01-11 0:20 ` Khalid Aziz [this message]
2019-01-11 0:44 ` Andy Lutomirski
2019-01-11 21:45 ` Khalid Aziz
2019-01-10 23:40 ` Dave Hansen
2019-01-11 9:59 ` Peter Zijlstra
2019-01-11 18:21 ` Khalid Aziz
2019-01-11 20:42 ` Dave Hansen
2019-01-11 21:06 ` Andy Lutomirski
2019-01-11 23:25 ` Khalid Aziz
2019-01-11 23:23 ` Khalid Aziz
2019-01-16 1:28 ` Laura Abbott
[not found] ` <ciirm8o98gzm4z.fsf@u54ee758033e858cfa736.ant.amazon.com>
2019-01-16 15:16 ` Khalid Aziz
2019-01-17 23:38 ` Laura Abbott
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=e8df9859-4629-52d7-9807-d87f1820a01d@oracle.com \
--to=khalid.aziz@oracle.com \
--cc=ak@linux.intel.com \
--cc=andrew.cooper3@citrix.com \
--cc=boris.ostrovsky@oracle.com \
--cc=chris.hyser@oracle.com \
--cc=dave.hansen@intel.com \
--cc=deepa.srinivasan@oracle.com \
--cc=dwmw@amazon.co.uk \
--cc=hch@lst.de \
--cc=jcm@redhat.com \
--cc=jmattson@google.com \
--cc=joao.m.martins@oracle.com \
--cc=john.haxby@oracle.com \
--cc=jsteckli@amazon.de \
--cc=juergh@gmail.com \
--cc=kanth.ghatraju@oracle.com \
--cc=keescook@chromium.org \
--cc=kernel-hardening@lists.openwall.com \
--cc=kirill.shutemov@linux.intel.com \
--cc=konrad.wilk@oracle.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=liran.alon@oracle.com \
--cc=luto@kernel.org \
--cc=mingo@kernel.org \
--cc=pradeep.vincent@oracle.com \
--cc=steven.sistare@oracle.com \
--cc=tglx@linutronix.de \
--cc=torvalds@linux-foundation.org \
--cc=tycho@tycho.ws \
--cc=tyhicks@canonical.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).