From: Peter Zijlstra <peterz@infradead.org>
To: "Luck, Tony" <tony.luck@intel.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>,
Dan Williams <dan.j.williams@intel.com>,
Ingo Molnar <mingo@kernel.org>,
Linux List Kernel Mailing <linux-kernel@vger.kernel.org>,
Dave Hansen <dave.hansen@linux.intel.com>,
Andy Lutomirski <luto@kernel.org>, Borislav Petkov <bp@alien8.de>,
Thomas Gleixner <tglx@linutronix.de>,
Rik van Riel <riel@surriel.com>
Subject: [PATCH] x86/mm/cpa: Fix set_mce_nospec()
Date: Fri, 8 Feb 2019 13:08:59 +0100
Message-ID: <20190208120859.GH32511@hirez.programming.kicks-ass.net> (raw)
In-Reply-To: <20190207184021.GA17049@agluck-desk>
On Thu, Feb 07, 2019 at 10:40:21AM -0800, Luck, Tony wrote:
> Tried it out and it works!
Excellent, find below a proper patch!
---
Subject: x86/mm/cpa: Fix set_mce_nospec()
In commit:
fe0937b24ff5 ("x86/mm/cpa: Fold cpa_flush_range() and cpa_flush_array()
into a single cpa_flush() function")
I accidentally made the call to make_addr_canonical_again() go away,
this breaks set_mce_nospec().
Re-instate the call to convert the address back into canonical form
right before we do either CLFLUSH or INVLPG. Rename the function while
at it to be shorter (and less MAGA).
Reported-by: Tony Luck <tony.luck@intel.com>
Tested-by: Tony Luck <tony.luck@intel.com>
Fixes: fe0937b24ff5 ("x86/mm/cpa: Fold cpa_flush_range() and cpa_flush_array() into a single cpa_flush() function")
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
---
arch/x86/mm/pageattr.c | 50 +++++++++++++++++++++++++-------------------------
1 file changed, 25 insertions(+), 25 deletions(-)
diff --git a/arch/x86/mm/pageattr.c b/arch/x86/mm/pageattr.c
index 4f8972311a77..14e6119838a6 100644
--- a/arch/x86/mm/pageattr.c
+++ b/arch/x86/mm/pageattr.c
@@ -230,6 +230,29 @@ static bool __cpa_pfn_in_highmap(unsigned long pfn)
#endif
+/*
+ * See set_mce_nospec().
+ *
+ * Machine check recovery code needs to change cache mode of poisoned pages to
+ * UC to avoid speculative access logging another error. But passing the
+ * address of the 1:1 mapping to set_memory_uc() is a fine way to encourage a
+ * speculative access. So we cheat and flip the top bit of the address. This
+ * works fine for the code that updates the page tables. But at the end of the
+ * process we need to flush the TLB and cache and the non-canonical address
+ * causes a #GP fault when used by the INVLPG and CLFLUSH instructions.
+ *
+ * But in the common case we already have a canonical address. This code
+ * will fix the top bit if needed and is a no-op otherwise.
+ */
+static inline unsigned long fix_addr(unsigned long addr)
+{
+#ifdef CONFIG_X86_64
+ return (long)(addr << 1) >> 1;
+#else
+ return addr;
+#endif
+}
+
static unsigned long __cpa_addr(struct cpa_data *cpa, unsigned long idx)
{
if (cpa->flags & CPA_PAGES_ARRAY) {
@@ -313,7 +336,7 @@ void __cpa_flush_tlb(void *data)
unsigned int i;
for (i = 0; i < cpa->numpages; i++)
- __flush_tlb_one_kernel(__cpa_addr(cpa, i));
+ __flush_tlb_one_kernel(fix_addr(__cpa_addr(cpa, i)));
}
static void cpa_flush(struct cpa_data *data, int cache)
@@ -347,7 +370,7 @@ static void cpa_flush(struct cpa_data *data, int cache)
* Only flush present addresses:
*/
if (pte && (pte_val(*pte) & _PAGE_PRESENT))
- clflush_cache_range_opt((void *)addr, PAGE_SIZE);
+ clflush_cache_range_opt((void *)fix_addr(addr), PAGE_SIZE);
}
mb();
}
@@ -1627,29 +1650,6 @@ static int __change_page_attr_set_clr(struct cpa_data *cpa, int checkalias)
return ret;
}
-/*
- * Machine check recovery code needs to change cache mode of poisoned
- * pages to UC to avoid speculative access logging another error. But
- * passing the address of the 1:1 mapping to set_memory_uc() is a fine
- * way to encourage a speculative access. So we cheat and flip the top
- * bit of the address. This works fine for the code that updates the
- * page tables. But at the end of the process we need to flush the cache
- * and the non-canonical address causes a #GP fault when used by the
- * CLFLUSH instruction.
- *
- * But in the common case we already have a canonical address. This code
- * will fix the top bit if needed and is a no-op otherwise.
- */
-static inline unsigned long make_addr_canonical_again(unsigned long addr)
-{
-#ifdef CONFIG_X86_64
- return (long)(addr << 1) >> 1;
-#else
- return addr;
-#endif
-}
-
-
static int change_page_attr_set_clr(unsigned long *addr, int numpages,
pgprot_t mask_set, pgprot_t mask_clr,
int force_split, int in_flag,
next prev parent reply index
Thread overview: 18+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-12-24 23:11 [GIT PULL] x86/mm changes for v4.21 Ingo Molnar
2018-12-27 2:45 ` pr-tracker-bot
2019-02-07 0:17 ` Luck, Tony
2019-02-07 0:33 ` Dave Hansen
2019-02-07 9:50 ` Peter Zijlstra
2019-02-07 10:18 ` Peter Zijlstra
2019-02-07 11:50 ` Linus Torvalds
2019-02-07 14:01 ` Peter Zijlstra
2019-02-07 17:36 ` Luck, Tony
2019-02-07 17:57 ` Peter Zijlstra
2019-02-07 18:07 ` Andy Lutomirski
2019-02-07 18:46 ` Luck, Tony
2019-02-07 20:24 ` Andy Lutomirski
2019-02-07 22:53 ` Linus Torvalds
2019-02-07 23:05 ` Andy Lutomirski
2019-02-07 18:40 ` Luck, Tony
2019-02-08 12:08 ` Peter Zijlstra [this message]
2019-02-08 13:37 ` [tip:x86/urgent] x86/mm/cpa: Fix set_mce_nospec() tip-bot for Peter Zijlstra
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20190208120859.GH32511@hirez.programming.kicks-ass.net \
--to=peterz@infradead.org \
--cc=bp@alien8.de \
--cc=dan.j.williams@intel.com \
--cc=dave.hansen@linux.intel.com \
--cc=linux-kernel@vger.kernel.org \
--cc=luto@kernel.org \
--cc=mingo@kernel.org \
--cc=riel@surriel.com \
--cc=tglx@linutronix.de \
--cc=tony.luck@intel.com \
--cc=torvalds@linux-foundation.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
LKML Archive on lore.kernel.org
Archives are clonable:
git clone --mirror https://lore.kernel.org/lkml/0 lkml/git/0.git
git clone --mirror https://lore.kernel.org/lkml/1 lkml/git/1.git
git clone --mirror https://lore.kernel.org/lkml/2 lkml/git/2.git
git clone --mirror https://lore.kernel.org/lkml/3 lkml/git/3.git
git clone --mirror https://lore.kernel.org/lkml/4 lkml/git/4.git
git clone --mirror https://lore.kernel.org/lkml/5 lkml/git/5.git
git clone --mirror https://lore.kernel.org/lkml/6 lkml/git/6.git
git clone --mirror https://lore.kernel.org/lkml/7 lkml/git/7.git
git clone --mirror https://lore.kernel.org/lkml/8 lkml/git/8.git
git clone --mirror https://lore.kernel.org/lkml/9 lkml/git/9.git
# If you have public-inbox 1.1+ installed, you may
# initialize and index your mirror using the following commands:
public-inbox-init -V2 lkml lkml/ https://lore.kernel.org/lkml \
linux-kernel@vger.kernel.org
public-inbox-index lkml
Example config snippet for mirrors
Newsgroup available over NNTP:
nntp://nntp.lore.kernel.org/org.kernel.vger.linux-kernel
AGPL code for this site: git clone https://public-inbox.org/public-inbox.git