* [PATCH] xen/gntdev: Use mempolicy instead of VM_IO flag to avoid NUMA balancing
@ 2016-11-16 17:02 Boris Ostrovsky
2016-11-16 17:20 ` [Xen-devel] " David Vrabel
0 siblings, 1 reply; 2+ messages in thread
From: Boris Ostrovsky @ 2016-11-16 17:02 UTC (permalink / raw)
To: david.vrabel, jgross; +Cc: xen-devel, linux-kernel, olaf, Boris Ostrovsky
Commit 9c17d96500f7 ("xen/gntdev: Grant maps should not be subject to
NUMA balancing") set VM_IO flag to prevent grant maps from being
subjected to NUMA balancing.
It was discovered recently this this flag may cause page allocation
failures with the following stack:
check_vma_flags
__get_user_pages
__get_user_pages_locked
__get_user_pages_unlocked
get_user_pages_fast
iov_iter_get_pages
dio_refill_pages
do_direct_IO
do_blockdev_direct_IO
do_blockdev_direct_IO
ext4_direct_IO_read
generic_file_read_iter
aio_run_iocb
(which can happen if guest's vdisk has direct-io-safe option).
To avoid this, instead of setting VM_IO use mempolicy that prohibits page
migration (i.e. clear policy's MPOL_F_MOF|MPOL_F_MORON)
Reported-by: Olaf Hering <olaf@aepfle.de>
Signed-off-by: Boris Ostrovsky <boris.ostrovsky@oracle.com>
---
Unfortunately I haven't been able to trigger NUMA balancing
so while I tested this in general I am not sure I actually
exercised the code path.
drivers/xen/gntdev.c | 15 ++++++++++++++-
1 file changed, 14 insertions(+), 1 deletion(-)
diff --git a/drivers/xen/gntdev.c b/drivers/xen/gntdev.c
index bb95212..743e6f0 100644
--- a/drivers/xen/gntdev.c
+++ b/drivers/xen/gntdev.c
@@ -35,6 +35,7 @@
#include <linux/spinlock.h>
#include <linux/slab.h>
#include <linux/highmem.h>
+#include <linux/mempolicy.h>
#include <xen/xen.h>
#include <xen/grant_table.h>
@@ -1007,8 +1008,20 @@ static int gntdev_mmap(struct file *flip, struct vm_area_struct *vma)
vma->vm_ops = &gntdev_vmops;
- vma->vm_flags |= VM_DONTEXPAND | VM_DONTDUMP | VM_IO;
+ vma->vm_flags |= VM_DONTEXPAND | VM_DONTDUMP;
+#ifdef CONFIG_NUMA
+ /* Prevent NUMA balancing */
+ if (vma->vm_policy)
+ vma->vm_policy->flags &= ~(MPOL_F_MOF | MPOL_F_MORON);
+ else {
+ struct mempolicy *pol = get_task_policy(current);
+
+ vma->vm_policy = mpol_dup(pol);
+ if (vma->vm_policy)
+ vma->vm_policy->flags &= ~(MPOL_F_MOF | MPOL_F_MORON);
+ }
+#endif
if (use_ptemod)
vma->vm_flags |= VM_DONTCOPY;
--
2.5.5
^ permalink raw reply related [flat|nested] 2+ messages in thread
* Re: [Xen-devel] [PATCH] xen/gntdev: Use mempolicy instead of VM_IO flag to avoid NUMA balancing
2016-11-16 17:02 [PATCH] xen/gntdev: Use mempolicy instead of VM_IO flag to avoid NUMA balancing Boris Ostrovsky
@ 2016-11-16 17:20 ` David Vrabel
0 siblings, 0 replies; 2+ messages in thread
From: David Vrabel @ 2016-11-16 17:20 UTC (permalink / raw)
To: Boris Ostrovsky, david.vrabel, jgross; +Cc: xen-devel, olaf, linux-kernel
On 16/11/16 17:02, Boris Ostrovsky wrote:
> Commit 9c17d96500f7 ("xen/gntdev: Grant maps should not be subject to
> NUMA balancing") set VM_IO flag to prevent grant maps from being
> subjected to NUMA balancing.
>
> It was discovered recently this this flag may cause page allocation
> failures with the following stack:
It's not an allocation failure.
"It was discovered recently that this flag causes get_user_pages() to
always fail with -EFAULT."
> check_vma_flags
> __get_user_pages
> __get_user_pages_locked
> __get_user_pages_unlocked
> get_user_pages_fast
> iov_iter_get_pages
> dio_refill_pages
> do_direct_IO
> do_blockdev_direct_IO
> do_blockdev_direct_IO
> ext4_direct_IO_read
> generic_file_read_iter
> aio_run_iocb
>
> (which can happen if guest's vdisk has direct-io-safe option).
>
> To avoid this, instead of setting VM_IO use mempolicy that prohibits page
> migration (i.e. clear policy's MPOL_F_MOF|MPOL_F_MORON)
Reviewed-by: David Vrabel <david.vrabel@citrix.com>
With the corrected commit message.
David
^ permalink raw reply [flat|nested] 2+ messages in thread
end of thread, other threads:[~2016-11-16 17:20 UTC | newest]
Thread overview: 2+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-11-16 17:02 [PATCH] xen/gntdev: Use mempolicy instead of VM_IO flag to avoid NUMA balancing Boris Ostrovsky
2016-11-16 17:20 ` [Xen-devel] " David Vrabel
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).