[PATCH] KVM: IOMMU: hva align mapping page size

[PATCH] KVM: IOMMU: hva align mapping page size

[Greg Edwards]

When determining the page size we could use to map with the IOMMU, the
page size should be aligned with the hva, not the gfn.  The gfn may not
reflect the real alignment within the hugetlbfs file.

Most of the time, this works fine.  However, if the hugetlbfs file is
backed by non-contiguous huge pages, a multi-huge page memslot starts at
an unaligned offset within the hugetlbfs file, and the gfn is aligned
with respect to the huge page size, kvm_host_page_size() will return the
huge page size and we will use that to map with the IOMMU.

When we later unpin that same memslot, the IOMMU returns the unmap size
as the huge page size, and we happily unpin that many pfns in
monotonically increasing order, not realizing we are spanning
non-contiguous huge pages and partially unpin the wrong huge page.

Instead, ensure the IOMMU mapping page size is aligned with the hva
corresponding to the gfn, which does reflect the alignment within the
hugetlbfs file.

Signed-off-by: Greg Edwards <gedwards@ddn.com>
Cc: stable@vger.kernel.org
---
This resolves the bug previously reported (and misdiagnosed) here:

 http://www.spinics.net/lists/kvm/msg97599.html

 virt/kvm/iommu.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/virt/kvm/iommu.c b/virt/kvm/iommu.c
index 72a130b..0e2ff32 100644
--- a/virt/kvm/iommu.c
+++ b/virt/kvm/iommu.c
@@ -99,8 +99,8 @@ int kvm_iommu_map_pages(struct kvm *kvm, struct kvm_memory_slot *slot)
 		while ((gfn + (page_size >> PAGE_SHIFT)) > end_gfn)
 			page_size >>= 1;
 
-		/* Make sure gfn is aligned to the page size we want to map */
-		while ((gfn << PAGE_SHIFT) & (page_size - 1))
+		/* Make sure hva is aligned to the page size we want to map */
+		while (__gfn_to_hva_memslot(slot, gfn) & (page_size - 1))
 			page_size >>= 1;
 
 		/*
-- 
1.8.3.2

Re: [PATCH] KVM: IOMMU: hva align mapping page size

[Marcelo Tosatti]

On Fri, Nov 01, 2013 at 10:08:55AM -0600, Greg Edwards wrote:
> When determining the page size we could use to map with the IOMMU, the
> page size should be aligned with the hva, not the gfn.  The gfn may not
> reflect the real alignment within the hugetlbfs file.
> 
> Most of the time, this works fine.  However, if the hugetlbfs file is
> backed by non-contiguous huge pages, a multi-huge page memslot starts at
> an unaligned offset within the hugetlbfs file, and the gfn is aligned
> with respect to the huge page size, kvm_host_page_size() will return the
> huge page size and we will use that to map with the IOMMU.
> 
> When we later unpin that same memslot, the IOMMU returns the unmap size
> as the huge page size, and we happily unpin that many pfns in
> monotonically increasing order, not realizing we are spanning
> non-contiguous huge pages and partially unpin the wrong huge page.
> 
> Instead, ensure the IOMMU mapping page size is aligned with the hva
> corresponding to the gfn, which does reflect the alignment within the
> hugetlbfs file.
> 
> Signed-off-by: Greg Edwards <gedwards@ddn.com>
> Cc: stable@vger.kernel.org
> ---
> This resolves the bug previously reported (and misdiagnosed) here:
> 
>  http://www.spinics.net/lists/kvm/msg97599.html
> 
>  virt/kvm/iommu.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/virt/kvm/iommu.c b/virt/kvm/iommu.c
> index 72a130b..0e2ff32 100644
> --- a/virt/kvm/iommu.c
> +++ b/virt/kvm/iommu.c
> @@ -99,8 +99,8 @@ int kvm_iommu_map_pages(struct kvm *kvm, struct kvm_memory_slot *slot)
>  		while ((gfn + (page_size >> PAGE_SHIFT)) > end_gfn)
>  			page_size >>= 1;
>  
> -		/* Make sure gfn is aligned to the page size we want to map */
> -		while ((gfn << PAGE_SHIFT) & (page_size - 1))
> +		/* Make sure hva is aligned to the page size we want to map */
> +		while (__gfn_to_hva_memslot(slot, gfn) & (page_size - 1))
>  			page_size >>= 1;

gfn should be aligned to page size as well (IOMMU requirement), so don't
drop that check.

[PATCH v2] KVM: IOMMU: hva align mapping page size

[Greg Edwards]

When determining the page size we could use to map with the IOMMU, the
page size should also be aligned with the hva, not just the gfn.  The
gfn may not reflect the real alignment within the hugetlbfs file.

Signed-off-by: Greg Edwards <gedwards@ddn.com>
Cc: stable@vger.kernel.org
---
 virt/kvm/iommu.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/virt/kvm/iommu.c b/virt/kvm/iommu.c
index 72a130b..c329c8f 100644
--- a/virt/kvm/iommu.c
+++ b/virt/kvm/iommu.c
@@ -103,6 +103,10 @@ int kvm_iommu_map_pages(struct kvm *kvm, struct kvm_memory_slot *slot)
 		while ((gfn << PAGE_SHIFT) & (page_size - 1))
 			page_size >>= 1;
 
+		/* Make sure hva is aligned to the page size we want to map */
+		while (__gfn_to_hva_memslot(slot, gfn) & (page_size - 1))
+			page_size >>= 1;
+
 		/*
 		 * Pin all pages we are about to map in memory. This is
 		 * important because we unmap and unpin in 4kb steps later.
-- 
1.8.3.2

Re: [PATCH v2] KVM: IOMMU: hva align mapping page size

[Marcelo Tosatti]

On Mon, Nov 04, 2013 at 09:08:12AM -0700, Greg Edwards wrote:
> When determining the page size we could use to map with the IOMMU, the
> page size should also be aligned with the hva, not just the gfn.  The
> gfn may not reflect the real alignment within the hugetlbfs file.
> 
> Signed-off-by: Greg Edwards <gedwards@ddn.com>
> Cc: stable@vger.kernel.org
> ---
>  virt/kvm/iommu.c | 4 ++++
>  1 file changed, 4 insertions(+)
> 
> diff --git a/virt/kvm/iommu.c b/virt/kvm/iommu.c
> index 72a130b..c329c8f 100644
> --- a/virt/kvm/iommu.c
> +++ b/virt/kvm/iommu.c
> @@ -103,6 +103,10 @@ int kvm_iommu_map_pages(struct kvm *kvm, struct kvm_memory_slot *slot)
>  		while ((gfn << PAGE_SHIFT) & (page_size - 1))
>  			page_size >>= 1;
>  
> +		/* Make sure hva is aligned to the page size we want to map */
> +		while (__gfn_to_hva_memslot(slot, gfn) & (page_size - 1))
> +			page_size >>= 1;
> +
>  		/*
>  		 * Pin all pages we are about to map in memory. This is
>  		 * important because we unmap and unpin in 4kb steps later.
> -- 
> 1.8.3.2

Reviewed-by: Marcelo Tosatti <mtosatti@redhat.com>

Re: [PATCH v2] KVM: IOMMU: hva align mapping page size

[Gleb Natapov]

On Mon, Nov 04, 2013 at 09:08:12AM -0700, Greg Edwards wrote:
> When determining the page size we could use to map with the IOMMU, the
> page size should also be aligned with the hva, not just the gfn.  The
> gfn may not reflect the real alignment within the hugetlbfs file.
> 
For some reason you dropped very good commit message from v1. I applied
v2 with v1 commit message.

> Signed-off-by: Greg Edwards <gedwards@ddn.com>
> Cc: stable@vger.kernel.org
> ---
>  virt/kvm/iommu.c | 4 ++++
>  1 file changed, 4 insertions(+)
> 
> diff --git a/virt/kvm/iommu.c b/virt/kvm/iommu.c
> index 72a130b..c329c8f 100644
> --- a/virt/kvm/iommu.c
> +++ b/virt/kvm/iommu.c
> @@ -103,6 +103,10 @@ int kvm_iommu_map_pages(struct kvm *kvm, struct kvm_memory_slot *slot)
>  		while ((gfn << PAGE_SHIFT) & (page_size - 1))
>  			page_size >>= 1;
>  
> +		/* Make sure hva is aligned to the page size we want to map */
> +		while (__gfn_to_hva_memslot(slot, gfn) & (page_size - 1))
> +			page_size >>= 1;
> +
>  		/*
>  		 * Pin all pages we are about to map in memory. This is
>  		 * important because we unmap and unpin in 4kb steps later.
> -- 
> 1.8.3.2
> 
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

--
			Gleb.