* [PATCH 1/3] x86/mm: Check for pfn instead of page in vmalloc_sync_one()
2019-07-19 18:46 [PATCH 0/3 v3] Sync unmappings in vmalloc/ioremap areas Joerg Roedel
@ 2019-07-19 18:46 ` Joerg Roedel
2019-07-19 18:46 ` [PATCH 2/3] x86/mm: Sync also unmappings in vmalloc_sync_all() Joerg Roedel
2019-07-19 18:46 ` [PATCH 3/3] mm/vmalloc: Sync unmappings in vunmap_page_range() Joerg Roedel
2 siblings, 0 replies; 7+ messages in thread
From: Joerg Roedel @ 2019-07-19 18:46 UTC (permalink / raw)
To: Dave Hansen, Andy Lutomirski, Peter Zijlstra, Thomas Gleixner,
Ingo Molnar, Borislav Petkov
Cc: Andrew Morton, linux-kernel, linux-mm, Joerg Roedel
From: Joerg Roedel <jroedel@suse.de>
Do not require a struct page for the mapped memory location
because it might not exist. This can happen when an
ioremapped region is mapped with 2MB pages.
Fixes: 5d72b4fba40ef ('x86, mm: support huge I/O mapping capability I/F')
Reviewed-by: Dave Hansen <dave.hansen@linux.intel.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
---
arch/x86/mm/fault.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/arch/x86/mm/fault.c b/arch/x86/mm/fault.c
index d1634c59ed56..d69f4e4d6918 100644
--- a/arch/x86/mm/fault.c
+++ b/arch/x86/mm/fault.c
@@ -183,7 +183,7 @@ static inline pmd_t *vmalloc_sync_one(pgd_t *pgd, unsigned long address)
if (!pmd_present(*pmd))
set_pmd(pmd, *pmd_k);
else
- BUG_ON(pmd_page(*pmd) != pmd_page(*pmd_k));
+ BUG_ON(pmd_pfn(*pmd) != pmd_pfn(*pmd_k));
return pmd_k;
}
--
2.17.1
^ permalink raw reply related [flat|nested] 7+ messages in thread
* [PATCH 2/3] x86/mm: Sync also unmappings in vmalloc_sync_all()
2019-07-19 18:46 [PATCH 0/3 v3] Sync unmappings in vmalloc/ioremap areas Joerg Roedel
2019-07-19 18:46 ` [PATCH 1/3] x86/mm: Check for pfn instead of page in vmalloc_sync_one() Joerg Roedel
@ 2019-07-19 18:46 ` Joerg Roedel
2019-07-19 18:46 ` [PATCH 3/3] mm/vmalloc: Sync unmappings in vunmap_page_range() Joerg Roedel
2 siblings, 0 replies; 7+ messages in thread
From: Joerg Roedel @ 2019-07-19 18:46 UTC (permalink / raw)
To: Dave Hansen, Andy Lutomirski, Peter Zijlstra, Thomas Gleixner,
Ingo Molnar, Borislav Petkov
Cc: Andrew Morton, linux-kernel, linux-mm, Joerg Roedel
From: Joerg Roedel <jroedel@suse.de>
With huge-page ioremap areas the unmappings also need to be
synced between all page-tables. Otherwise it can cause data
corruption when a region is unmapped and later re-used.
Make the vmalloc_sync_one() function ready to sync
unmappings and make sure vmalloc_sync_all() iterates over
all page-tables even when an unmapped PMD is found.
Fixes: 5d72b4fba40ef ('x86, mm: support huge I/O mapping capability I/F')
Reviewed-by: Dave Hansen <dave.hansen@linux.intel.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
---
arch/x86/mm/fault.c | 13 +++++--------
1 file changed, 5 insertions(+), 8 deletions(-)
diff --git a/arch/x86/mm/fault.c b/arch/x86/mm/fault.c
index d69f4e4d6918..8807916c712d 100644
--- a/arch/x86/mm/fault.c
+++ b/arch/x86/mm/fault.c
@@ -177,11 +177,12 @@ static inline pmd_t *vmalloc_sync_one(pgd_t *pgd, unsigned long address)
pmd = pmd_offset(pud, address);
pmd_k = pmd_offset(pud_k, address);
- if (!pmd_present(*pmd_k))
- return NULL;
- if (!pmd_present(*pmd))
+ if (pmd_present(*pmd) != pmd_present(*pmd_k))
set_pmd(pmd, *pmd_k);
+
+ if (!pmd_present(*pmd_k))
+ return NULL;
else
BUG_ON(pmd_pfn(*pmd) != pmd_pfn(*pmd_k));
@@ -203,17 +204,13 @@ void vmalloc_sync_all(void)
spin_lock(&pgd_lock);
list_for_each_entry(page, &pgd_list, lru) {
spinlock_t *pgt_lock;
- pmd_t *ret;
/* the pgt_lock only for Xen */
pgt_lock = &pgd_page_get_mm(page)->page_table_lock;
spin_lock(pgt_lock);
- ret = vmalloc_sync_one(page_address(page), address);
+ vmalloc_sync_one(page_address(page), address);
spin_unlock(pgt_lock);
-
- if (!ret)
- break;
}
spin_unlock(&pgd_lock);
}
--
2.17.1
^ permalink raw reply related [flat|nested] 7+ messages in thread
* [PATCH 3/3] mm/vmalloc: Sync unmappings in vunmap_page_range()
2019-07-19 18:46 [PATCH 0/3 v3] Sync unmappings in vmalloc/ioremap areas Joerg Roedel
2019-07-19 18:46 ` [PATCH 1/3] x86/mm: Check for pfn instead of page in vmalloc_sync_one() Joerg Roedel
2019-07-19 18:46 ` [PATCH 2/3] x86/mm: Sync also unmappings in vmalloc_sync_all() Joerg Roedel
@ 2019-07-19 18:46 ` Joerg Roedel
2019-07-22 8:11 ` Joerg Roedel
2 siblings, 1 reply; 7+ messages in thread
From: Joerg Roedel @ 2019-07-19 18:46 UTC (permalink / raw)
To: Dave Hansen, Andy Lutomirski, Peter Zijlstra, Thomas Gleixner,
Ingo Molnar, Borislav Petkov
Cc: Andrew Morton, linux-kernel, linux-mm, Joerg Roedel
From: Joerg Roedel <jroedel@suse.de>
On x86-32 with PTI enabled, parts of the kernel page-tables
are not shared between processes. This can cause mappings in
the vmalloc/ioremap area to persist in some page-tables
after the region is unmapped and released.
When the region is re-used the processes with the old
mappings do not fault in the new mappings but still access
the old ones.
This causes undefined behavior, in reality often data
corruption, kernel oopses and panics and even spontaneous
reboots.
Fix this problem by activly syncing unmaps in the
vmalloc/ioremap area to all page-tables in the system before
the regions can be re-used.
References: https://bugzilla.suse.com/show_bug.cgi?id=1118689
Reviewed-by: Dave Hansen <dave.hansen@linux.intel.com>
Fixes: 5d72b4fba40ef ('x86, mm: support huge I/O mapping capability I/F')
Signed-off-by: Joerg Roedel <jroedel@suse.de>
---
mm/vmalloc.c | 9 +++++++++
1 file changed, 9 insertions(+)
diff --git a/mm/vmalloc.c b/mm/vmalloc.c
index 4fa8d84599b0..e0fc963acc41 100644
--- a/mm/vmalloc.c
+++ b/mm/vmalloc.c
@@ -1258,6 +1258,12 @@ static bool __purge_vmap_area_lazy(unsigned long start, unsigned long end)
if (unlikely(valist == NULL))
return false;
+ /*
+ * First make sure the mappings are removed from all page-tables
+ * before they are freed.
+ */
+ vmalloc_sync_all();
+
/*
* TODO: to calculate a flush range without looping.
* The list can be up to lazy_max_pages() elements.
@@ -3038,6 +3044,9 @@ EXPORT_SYMBOL(remap_vmalloc_range);
/*
* Implement a stub for vmalloc_sync_all() if the architecture chose not to
* have one.
+ *
+ * The purpose of this function is to make sure the vmalloc area
+ * mappings are identical in all page-tables in the system.
*/
void __weak vmalloc_sync_all(void)
{
--
2.17.1
^ permalink raw reply related [flat|nested] 7+ messages in thread
* Re: [PATCH 3/3] mm/vmalloc: Sync unmappings in vunmap_page_range()
2019-07-19 18:46 ` [PATCH 3/3] mm/vmalloc: Sync unmappings in vunmap_page_range() Joerg Roedel
@ 2019-07-22 8:11 ` Joerg Roedel
2019-07-22 8:19 ` Thomas Gleixner
0 siblings, 1 reply; 7+ messages in thread
From: Joerg Roedel @ 2019-07-22 8:11 UTC (permalink / raw)
To: Joerg Roedel
Cc: Dave Hansen, Andy Lutomirski, Peter Zijlstra, Thomas Gleixner,
Ingo Molnar, Borislav Petkov, Andrew Morton, linux-kernel,
linux-mm
Srewed up the subject :(, it needs to be
"mm/vmalloc: Sync unmappings in __purge_vmap_area_lazy()"
of course.
On Fri, Jul 19, 2019 at 08:46:52PM +0200, Joerg Roedel wrote:
> From: Joerg Roedel <jroedel@suse.de>
>
> On x86-32 with PTI enabled, parts of the kernel page-tables
> are not shared between processes. This can cause mappings in
> the vmalloc/ioremap area to persist in some page-tables
> after the region is unmapped and released.
>
> When the region is re-used the processes with the old
> mappings do not fault in the new mappings but still access
> the old ones.
>
> This causes undefined behavior, in reality often data
> corruption, kernel oopses and panics and even spontaneous
> reboots.
>
> Fix this problem by activly syncing unmaps in the
> vmalloc/ioremap area to all page-tables in the system before
> the regions can be re-used.
>
> References: https://bugzilla.suse.com/show_bug.cgi?id=1118689
> Reviewed-by: Dave Hansen <dave.hansen@linux.intel.com>
> Fixes: 5d72b4fba40ef ('x86, mm: support huge I/O mapping capability I/F')
> Signed-off-by: Joerg Roedel <jroedel@suse.de>
> ---
> mm/vmalloc.c | 9 +++++++++
> 1 file changed, 9 insertions(+)
>
> diff --git a/mm/vmalloc.c b/mm/vmalloc.c
> index 4fa8d84599b0..e0fc963acc41 100644
> --- a/mm/vmalloc.c
> +++ b/mm/vmalloc.c
> @@ -1258,6 +1258,12 @@ static bool __purge_vmap_area_lazy(unsigned long start, unsigned long end)
> if (unlikely(valist == NULL))
> return false;
>
> + /*
> + * First make sure the mappings are removed from all page-tables
> + * before they are freed.
> + */
> + vmalloc_sync_all();
> +
> /*
> * TODO: to calculate a flush range without looping.
> * The list can be up to lazy_max_pages() elements.
> @@ -3038,6 +3044,9 @@ EXPORT_SYMBOL(remap_vmalloc_range);
> /*
> * Implement a stub for vmalloc_sync_all() if the architecture chose not to
> * have one.
> + *
> + * The purpose of this function is to make sure the vmalloc area
> + * mappings are identical in all page-tables in the system.
> */
> void __weak vmalloc_sync_all(void)
> {
> --
> 2.17.1
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH 3/3] mm/vmalloc: Sync unmappings in vunmap_page_range()
2019-07-22 8:11 ` Joerg Roedel
@ 2019-07-22 8:19 ` Thomas Gleixner
2019-07-22 8:29 ` Joerg Roedel
0 siblings, 1 reply; 7+ messages in thread
From: Thomas Gleixner @ 2019-07-22 8:19 UTC (permalink / raw)
To: Joerg Roedel
Cc: Joerg Roedel, Dave Hansen, Andy Lutomirski, Peter Zijlstra,
Ingo Molnar, Borislav Petkov, Andrew Morton, linux-kernel,
linux-mm
On Mon, 22 Jul 2019, Joerg Roedel wrote:
> Srewed up the subject :(, it needs to be
Un-Srewed it :)
^^^^^^
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH 3/3] mm/vmalloc: Sync unmappings in vunmap_page_range()
2019-07-22 8:19 ` Thomas Gleixner
@ 2019-07-22 8:29 ` Joerg Roedel
0 siblings, 0 replies; 7+ messages in thread
From: Joerg Roedel @ 2019-07-22 8:29 UTC (permalink / raw)
To: Thomas Gleixner
Cc: Joerg Roedel, Dave Hansen, Andy Lutomirski, Peter Zijlstra,
Ingo Molnar, Borislav Petkov, Andrew Morton, linux-kernel,
linux-mm
On Mon, Jul 22, 2019 at 10:19:32AM +0200, Thomas Gleixner wrote:
> On Mon, 22 Jul 2019, Joerg Roedel wrote:
>
> > Srewed up the subject :(, it needs to be
>
> Un-Srewed it :)
Thanks a lot :)
^ permalink raw reply [flat|nested] 7+ messages in thread