[PATCH v4 0/3] KVM: MMU: fix kvm_is_mmio_pfn()

[PATCH v4 0/3] KVM: MMU: fix kvm_is_mmio_pfn()

[Haozhong Zhang]

Some reserved pages, such as those from NVDIMM DAX devices, are
not for MMIO, and can be mapped with cached memory type for better
performance. However, the above check misconceives those pages as
MMIO.  Because KVM maps MMIO pages with UC memory type, the
performance of guest accesses to those pages would be harmed.
Therefore, we check the host memory type by lookup_memtype() in
addition and only treat UC/UC- pages as MMIO.


Changes in v4:
 * Mask pfn_t and kvm_pfn_t specific flags in conversion.

Changes in v3:
 * Move cache mode check to pat.c as pat_pfn_is_uc()
 * Reintroduce converters between kvm_pfn_t and pfn_t.

Changes in v2:
 * Switch to lookup_memtype() to get host memory type.
 * Rewrite the comment in KVM MMU patch.
 * Remove v1 patch 2, which is not necessary in v2.


Haozhong Zhang (3):
  x86/mm: add function to check if a pfn is UC/UC-
  KVM: add converters between pfn_t and kvm_pfn_t
  KVM: MMU: consider host cache mode in MMIO page check

 arch/x86/include/asm/pat.h |  2 ++
 arch/x86/kvm/mmu.c         | 19 ++++++++++++++++++-
 arch/x86/mm/pat.c          | 14 ++++++++++++++
 include/linux/kvm_host.h   |  4 ++++
 4 files changed, 38 insertions(+), 1 deletion(-)

-- 
2.14.1

[PATCH v4 1/3] x86/mm: add function to check if a pfn is UC/UC-

[Haozhong Zhang]

pat_pfn_is_uc(pfn) is added and will be used by KVM to check whether
the memory type of a host pfn is UC/UC-.

Signed-off-by: Haozhong Zhang <haozhong.zhang@intel.com>
---
 arch/x86/include/asm/pat.h |  2 ++
 arch/x86/mm/pat.c          | 14 ++++++++++++++
 2 files changed, 16 insertions(+)

diff --git a/arch/x86/include/asm/pat.h b/arch/x86/include/asm/pat.h
index fffb2794dd89..4027d9fb10a8 100644
--- a/arch/x86/include/asm/pat.h
+++ b/arch/x86/include/asm/pat.h
@@ -21,4 +21,6 @@ int io_reserve_memtype(resource_size_t start, resource_size_t end,
 
 void io_free_memtype(resource_size_t start, resource_size_t end);
 
+bool pat_pfn_is_uc(pfn_t pfn);
+
 #endif /* _ASM_X86_PAT_H */
diff --git a/arch/x86/mm/pat.c b/arch/x86/mm/pat.c
index fe7d57a8fb60..71fe9c29727e 100644
--- a/arch/x86/mm/pat.c
+++ b/arch/x86/mm/pat.c
@@ -677,6 +677,20 @@ static enum page_cache_mode lookup_memtype(u64 paddr)
 	return rettype;
 }
 
+/**
+ * Check whether the memory type of a pfn is UC or UC-. The result is
+ * valid only when PAT is enabled.
+ *
+ * Returns true if it is; otherwise, returns false.
+ */
+bool pat_pfn_is_uc(pfn_t pfn)
+{
+	enum page_cache_mode cm = lookup_memtype(pfn_t_to_phys(pfn));
+
+	return cm == _PAGE_CACHE_MODE_UC || cm == _PAGE_CACHE_MODE_UC_MINUS;
+}
+EXPORT_SYMBOL_GPL(pat_pfn_is_uc);
+
 /**
  * io_reserve_memtype - Request a memory type mapping for a region of memory
  * @start: start (physical address) of the region
-- 
2.14.1

[PATCH v4 2/3] KVM: add converters between pfn_t and kvm_pfn_t

[Haozhong Zhang]

Signed-off-by: Haozhong Zhang <haozhong.zhang@intel.com>
---
 include/linux/kvm_host.h | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h
index 6882538eda32..caf6f7a6bdb2 100644
--- a/include/linux/kvm_host.h
+++ b/include/linux/kvm_host.h
@@ -27,6 +27,7 @@
 #include <linux/irqbypass.h>
 #include <linux/swait.h>
 #include <linux/refcount.h>
+#include <linux/pfn_t.h>
 #include <asm/signal.h>
 
 #include <linux/kvm.h>
@@ -67,6 +68,9 @@
 #define KVM_PFN_ERR_HWPOISON	(KVM_PFN_ERR_MASK + 1)
 #define KVM_PFN_ERR_RO_FAULT	(KVM_PFN_ERR_MASK + 2)
 
+#define pfn_t_to_kvm_pfn_t(x)	pfn_t_to_pfn(x)
+#define kvm_pfn_t_to_pfn_t(x)	pfn_to_pfn_t((x) & ~KVM_PFN_ERR_NOSLOT_MASK)
+
 /*
  * error pfns indicate that the gfn is in slot but faild to
  * translate it to pfn on host.
-- 
2.14.1

[PATCH v4 3/3] KVM: MMU: consider host cache mode in MMIO page check

[Haozhong Zhang]

Some reserved pages, such as those from NVDIMM DAX devices, are
not for MMIO, and can be mapped with cached memory type for better
performance. However, the above check misconceives those pages as
MMIO.  Because KVM maps MMIO pages with UC memory type, the
performance of guest accesses to those pages would be harmed.
Therefore, we check the host memory type by lookup_memtype() in
addition and only treat UC/UC- pages as MMIO.

Signed-off-by: Haozhong Zhang <haozhong.zhang@intel.com>
Reported-by: Cuevas Escareno, Ivan D <ivan.d.cuevas.escareno@intel.com>
Reported-by: Kumar, Karthik <karthik.kumar@intel.com>
---
 arch/x86/kvm/mmu.c | 19 ++++++++++++++++++-
 1 file changed, 18 insertions(+), 1 deletion(-)

diff --git a/arch/x86/kvm/mmu.c b/arch/x86/kvm/mmu.c
index 0b481cc9c725..e9ed0e666a83 100644
--- a/arch/x86/kvm/mmu.c
+++ b/arch/x86/kvm/mmu.c
@@ -2708,7 +2708,24 @@ static bool mmu_need_write_protect(struct kvm_vcpu *vcpu, gfn_t gfn,
 static bool kvm_is_mmio_pfn(kvm_pfn_t pfn)
 {
 	if (pfn_valid(pfn))
-		return !is_zero_pfn(pfn) && PageReserved(pfn_to_page(pfn));
+		return !is_zero_pfn(pfn) && PageReserved(pfn_to_page(pfn)) &&
+			/*
+			 * Some reserved pages, such as those from
+			 * NVDIMM DAX devices, are not for MMIO, and
+			 * can be mapped with cached memory type for
+			 * better performance. However, the above
+			 * check misconceives those pages as MMIO.
+			 * Because KVM maps MMIO pages with UC memory
+			 * type, the performance of guest accesses to
+			 * those pages would be harmed. Therefore, we
+			 * check the host memory type in addition and
+			 * only treat UC/UC- pages as MMIO.
+			 *
+			 * pat_pfn_is_uc() works only when PAT is enabled,
+			 * so check pat_enabled() as well.
+			 */
+			(!pat_enabled() ||
+			 pat_pfn_is_uc(kvm_pfn_t_to_pfn_t(pfn)));
 
 	return true;
 }
-- 
2.14.1

Re: [PATCH v4 3/3] KVM: MMU: consider host cache mode in MMIO page check

[Xiao Guangrong]

On 11/03/2017 01:53 PM, Haozhong Zhang wrote:
> Some reserved pages, such as those from NVDIMM DAX devices, are
> not for MMIO, and can be mapped with cached memory type for better
> performance. However, the above check misconceives those pages as
> MMIO.  Because KVM maps MMIO pages with UC memory type, the
> performance of guest accesses to those pages would be harmed.
> Therefore, we check the host memory type by lookup_memtype() in
> addition and only treat UC/UC- pages as MMIO.
> 
> Signed-off-by: Haozhong Zhang <haozhong.zhang@intel.com>
> Reported-by: Cuevas Escareno, Ivan D <ivan.d.cuevas.escareno@intel.com>
> Reported-by: Kumar, Karthik <karthik.kumar@intel.com>
> ---
>   arch/x86/kvm/mmu.c | 19 ++++++++++++++++++-
>   1 file changed, 18 insertions(+), 1 deletion(-)
> 
> diff --git a/arch/x86/kvm/mmu.c b/arch/x86/kvm/mmu.c
> index 0b481cc9c725..e9ed0e666a83 100644
> --- a/arch/x86/kvm/mmu.c
> +++ b/arch/x86/kvm/mmu.c
> @@ -2708,7 +2708,24 @@ static bool mmu_need_write_protect(struct kvm_vcpu *vcpu, gfn_t gfn,
>   static bool kvm_is_mmio_pfn(kvm_pfn_t pfn)
>   {
>   	if (pfn_valid(pfn))
> -		return !is_zero_pfn(pfn) && PageReserved(pfn_to_page(pfn));
> +		return !is_zero_pfn(pfn) && PageReserved(pfn_to_page(pfn)) &&
> +			/*
> +			 * Some reserved pages, such as those from
> +			 * NVDIMM DAX devices, are not for MMIO, and
> +			 * can be mapped with cached memory type for
> +			 * better performance. However, the above
> +			 * check misconceives those pages as MMIO.
> +			 * Because KVM maps MMIO pages with UC memory
> +			 * type, the performance of guest accesses to
> +			 * those pages would be harmed. Therefore, we
> +			 * check the host memory type in addition and
> +			 * only treat UC/UC- pages as MMIO.
> +			 *
> +			 * pat_pfn_is_uc() works only when PAT is enabled,
> +			 * so check pat_enabled() as well.
> +			 */
> +			(!pat_enabled() ||
> +			 pat_pfn_is_uc(kvm_pfn_t_to_pfn_t(pfn)));

Can it be compiled if !CONFIG_PAT?

It would be better if we move pat_enabled out of kvm as well, please refer
to pgprot_writecombine() which is implemented in pat.c and in
include\asm-generic\pgtable.h:

#ifndef pgprot_writecombine
#define pgprot_writecombine pgprot_noncached
#endif

Re: [PATCH v4 3/3] KVM: MMU: consider host cache mode in MMIO page check

[Haozhong Zhang]

On 11/03/17 14:54 +0800, Xiao Guangrong wrote:
> 
> 
> On 11/03/2017 01:53 PM, Haozhong Zhang wrote:
> > Some reserved pages, such as those from NVDIMM DAX devices, are
> > not for MMIO, and can be mapped with cached memory type for better
> > performance. However, the above check misconceives those pages as
> > MMIO.  Because KVM maps MMIO pages with UC memory type, the
> > performance of guest accesses to those pages would be harmed.
> > Therefore, we check the host memory type by lookup_memtype() in
> > addition and only treat UC/UC- pages as MMIO.
> > 
> > Signed-off-by: Haozhong Zhang <haozhong.zhang@intel.com>
> > Reported-by: Cuevas Escareno, Ivan D <ivan.d.cuevas.escareno@intel.com>
> > Reported-by: Kumar, Karthik <karthik.kumar@intel.com>
> > ---
> >   arch/x86/kvm/mmu.c | 19 ++++++++++++++++++-
> >   1 file changed, 18 insertions(+), 1 deletion(-)
> > 
> > diff --git a/arch/x86/kvm/mmu.c b/arch/x86/kvm/mmu.c
> > index 0b481cc9c725..e9ed0e666a83 100644
> > --- a/arch/x86/kvm/mmu.c
> > +++ b/arch/x86/kvm/mmu.c
> > @@ -2708,7 +2708,24 @@ static bool mmu_need_write_protect(struct kvm_vcpu *vcpu, gfn_t gfn,
> >   static bool kvm_is_mmio_pfn(kvm_pfn_t pfn)
> >   {
> >   	if (pfn_valid(pfn))
> > -		return !is_zero_pfn(pfn) && PageReserved(pfn_to_page(pfn));
> > +		return !is_zero_pfn(pfn) && PageReserved(pfn_to_page(pfn)) &&
> > +			/*
> > +			 * Some reserved pages, such as those from
> > +			 * NVDIMM DAX devices, are not for MMIO, and
> > +			 * can be mapped with cached memory type for
> > +			 * better performance. However, the above
> > +			 * check misconceives those pages as MMIO.
> > +			 * Because KVM maps MMIO pages with UC memory
> > +			 * type, the performance of guest accesses to
> > +			 * those pages would be harmed. Therefore, we
> > +			 * check the host memory type in addition and
> > +			 * only treat UC/UC- pages as MMIO.
> > +			 *
> > +			 * pat_pfn_is_uc() works only when PAT is enabled,
> > +			 * so check pat_enabled() as well.
> > +			 */
> > +			(!pat_enabled() ||
> > +			 pat_pfn_is_uc(kvm_pfn_t_to_pfn_t(pfn)));
> 
> Can it be compiled if !CONFIG_PAT?

Yes.

What I check via pat_enabled() is not only whether PAT support is
compiled, but also whether PAT is enabled at runtime.

> 
> It would be better if we move pat_enabled out of kvm as well,

Surely I can combine them in one function like

bool pat_pfn_is_uc(pfn_t pfn)
{
	enum page_cache_mode cm;

	if (!pat_enabled())
		return false;

	cm = lookup_memtype(pfn_t_to_phys(pfn));

	return cm == _PAGE_CACHE_MODE_UC || cm == _PAGE_CACHE_MODE_UC_MINUS;
}

but I need a good name to make its semantics clear, or is it enough to
just leave a comment like?

/*
 * Check via PAT whether the cache mode of a page if UC or UC-.
 *
 * Returns true, if PAT is enabled and the cache mode is UC or UC-.
 * Returns false otherwise.
 */


> please refer
> to pgprot_writecombine() which is implemented in pat.c and in
> include\asm-generic\pgtable.h:
> 
> #ifndef pgprot_writecombine
> #define pgprot_writecombine pgprot_noncached
> #endif
>

Re: [PATCH v4 3/3] KVM: MMU: consider host cache mode in MMIO page check

[Haozhong Zhang]

On 11/03/17 16:51 +0800, Haozhong Zhang wrote:
> On 11/03/17 14:54 +0800, Xiao Guangrong wrote:
> > 
> > 
> > On 11/03/2017 01:53 PM, Haozhong Zhang wrote:
> > > Some reserved pages, such as those from NVDIMM DAX devices, are
> > > not for MMIO, and can be mapped with cached memory type for better
> > > performance. However, the above check misconceives those pages as
> > > MMIO.  Because KVM maps MMIO pages with UC memory type, the
> > > performance of guest accesses to those pages would be harmed.
> > > Therefore, we check the host memory type by lookup_memtype() in
> > > addition and only treat UC/UC- pages as MMIO.
> > > 
> > > Signed-off-by: Haozhong Zhang <haozhong.zhang@intel.com>
> > > Reported-by: Cuevas Escareno, Ivan D <ivan.d.cuevas.escareno@intel.com>
> > > Reported-by: Kumar, Karthik <karthik.kumar@intel.com>
> > > ---
> > >   arch/x86/kvm/mmu.c | 19 ++++++++++++++++++-
> > >   1 file changed, 18 insertions(+), 1 deletion(-)
> > > 
> > > diff --git a/arch/x86/kvm/mmu.c b/arch/x86/kvm/mmu.c
> > > index 0b481cc9c725..e9ed0e666a83 100644
> > > --- a/arch/x86/kvm/mmu.c
> > > +++ b/arch/x86/kvm/mmu.c
> > > @@ -2708,7 +2708,24 @@ static bool mmu_need_write_protect(struct kvm_vcpu *vcpu, gfn_t gfn,
> > >   static bool kvm_is_mmio_pfn(kvm_pfn_t pfn)
> > >   {
> > >   	if (pfn_valid(pfn))
> > > -		return !is_zero_pfn(pfn) && PageReserved(pfn_to_page(pfn));
> > > +		return !is_zero_pfn(pfn) && PageReserved(pfn_to_page(pfn)) &&
> > > +			/*
> > > +			 * Some reserved pages, such as those from
> > > +			 * NVDIMM DAX devices, are not for MMIO, and
> > > +			 * can be mapped with cached memory type for
> > > +			 * better performance. However, the above
> > > +			 * check misconceives those pages as MMIO.
> > > +			 * Because KVM maps MMIO pages with UC memory
> > > +			 * type, the performance of guest accesses to
> > > +			 * those pages would be harmed. Therefore, we
> > > +			 * check the host memory type in addition and
> > > +			 * only treat UC/UC- pages as MMIO.
> > > +			 *
> > > +			 * pat_pfn_is_uc() works only when PAT is enabled,
> > > +			 * so check pat_enabled() as well.
> > > +			 */
> > > +			(!pat_enabled() ||
> > > +			 pat_pfn_is_uc(kvm_pfn_t_to_pfn_t(pfn)));
> > 
> > Can it be compiled if !CONFIG_PAT?
> 
> Yes.
> 
> What I check via pat_enabled() is not only whether PAT support is
> compiled, but also whether PAT is enabled at runtime.
> 
> > 
> > It would be better if we move pat_enabled out of kvm as well,
> 
> Surely I can combine them in one function like
> 
> bool pat_pfn_is_uc(pfn_t pfn)
> {
> 	enum page_cache_mode cm;
> 
> 	if (!pat_enabled())
> 		return false;
> 
> 	cm = lookup_memtype(pfn_t_to_phys(pfn));
> 
> 	return cm == _PAGE_CACHE_MODE_UC || cm == _PAGE_CACHE_MODE_UC_MINUS;
> }

In addition, I think it's better to split this function into
pat_pfn_is_uc() and pat_pfn_is_uc_minus() to avoid additional
confusion.

Haozhong

> 
> but I need a good name to make its semantics clear, or is it enough to
> just leave a comment like?
> 
> /*
>  * Check via PAT whether the cache mode of a page if UC or UC-.
>  *
>  * Returns true, if PAT is enabled and the cache mode is UC or UC-.
>  * Returns false otherwise.
>  */
> 
> 
> > please refer
> > to pgprot_writecombine() which is implemented in pat.c and in
> > include\asm-generic\pgtable.h:
> > 
> > #ifndef pgprot_writecombine
> > #define pgprot_writecombine pgprot_noncached
> > #endif
> >
> 
> 
> 

Re: [PATCH v4 3/3] KVM: MMU: consider host cache mode in MMIO page check

[Xiao Guangrong]

On 11/03/2017 04:51 PM, Haozhong Zhang wrote:
> On 11/03/17 14:54 +0800, Xiao Guangrong wrote:
>>
>>
>> On 11/03/2017 01:53 PM, Haozhong Zhang wrote:
>>> Some reserved pages, such as those from NVDIMM DAX devices, are
>>> not for MMIO, and can be mapped with cached memory type for better
>>> performance. However, the above check misconceives those pages as
>>> MMIO.  Because KVM maps MMIO pages with UC memory type, the
>>> performance of guest accesses to those pages would be harmed.
>>> Therefore, we check the host memory type by lookup_memtype() in
>>> addition and only treat UC/UC- pages as MMIO.
>>>
>>> Signed-off-by: Haozhong Zhang <haozhong.zhang@intel.com>
>>> Reported-by: Cuevas Escareno, Ivan D <ivan.d.cuevas.escareno@intel.com>
>>> Reported-by: Kumar, Karthik <karthik.kumar@intel.com>
>>> ---
>>>    arch/x86/kvm/mmu.c | 19 ++++++++++++++++++-
>>>    1 file changed, 18 insertions(+), 1 deletion(-)
>>>
>>> diff --git a/arch/x86/kvm/mmu.c b/arch/x86/kvm/mmu.c
>>> index 0b481cc9c725..e9ed0e666a83 100644
>>> --- a/arch/x86/kvm/mmu.c
>>> +++ b/arch/x86/kvm/mmu.c
>>> @@ -2708,7 +2708,24 @@ static bool mmu_need_write_protect(struct kvm_vcpu *vcpu, gfn_t gfn,
>>>    static bool kvm_is_mmio_pfn(kvm_pfn_t pfn)
>>>    {
>>>    	if (pfn_valid(pfn))
>>> -		return !is_zero_pfn(pfn) && PageReserved(pfn_to_page(pfn));
>>> +		return !is_zero_pfn(pfn) && PageReserved(pfn_to_page(pfn)) &&
>>> +			/*
>>> +			 * Some reserved pages, such as those from
>>> +			 * NVDIMM DAX devices, are not for MMIO, and
>>> +			 * can be mapped with cached memory type for
>>> +			 * better performance. However, the above
>>> +			 * check misconceives those pages as MMIO.
>>> +			 * Because KVM maps MMIO pages with UC memory
>>> +			 * type, the performance of guest accesses to
>>> +			 * those pages would be harmed. Therefore, we
>>> +			 * check the host memory type in addition and
>>> +			 * only treat UC/UC- pages as MMIO.
>>> +			 *
>>> +			 * pat_pfn_is_uc() works only when PAT is enabled,
>>> +			 * so check pat_enabled() as well.
>>> +			 */
>>> +			(!pat_enabled() ||
>>> +			 pat_pfn_is_uc(kvm_pfn_t_to_pfn_t(pfn)));
>>
>> Can it be compiled if !CONFIG_PAT?
> 
> Yes.
> 
> What I check via pat_enabled() is not only whether PAT support is
> compiled, but also whether PAT is enabled at runtime.

The issue is about pat_pfn_is_uc() which is implemented only if CONFIG_PAT is
enabled, but you used it here unconditionally.

I am not sure if gcc is smart enough to omit pat_pfn_is_uc() completely under
this case. If you really have done the test to compile kernel and KVM module
with CONFIG_PAT disabled, it is fine.

Re: [PATCH v4 3/3] KVM: MMU: consider host cache mode in MMIO page check

[Xiao Guangrong]

On 11/03/2017 05:02 PM, Haozhong Zhang wrote:
> On 11/03/17 16:51 +0800, Haozhong Zhang wrote:
>> On 11/03/17 14:54 +0800, Xiao Guangrong wrote:
>>>
>>>
>>> On 11/03/2017 01:53 PM, Haozhong Zhang wrote:
>>>> Some reserved pages, such as those from NVDIMM DAX devices, are
>>>> not for MMIO, and can be mapped with cached memory type for better
>>>> performance. However, the above check misconceives those pages as
>>>> MMIO.  Because KVM maps MMIO pages with UC memory type, the
>>>> performance of guest accesses to those pages would be harmed.
>>>> Therefore, we check the host memory type by lookup_memtype() in
>>>> addition and only treat UC/UC- pages as MMIO.
>>>>
>>>> Signed-off-by: Haozhong Zhang <haozhong.zhang@intel.com>
>>>> Reported-by: Cuevas Escareno, Ivan D <ivan.d.cuevas.escareno@intel.com>
>>>> Reported-by: Kumar, Karthik <karthik.kumar@intel.com>
>>>> ---
>>>>    arch/x86/kvm/mmu.c | 19 ++++++++++++++++++-
>>>>    1 file changed, 18 insertions(+), 1 deletion(-)
>>>>
>>>> diff --git a/arch/x86/kvm/mmu.c b/arch/x86/kvm/mmu.c
>>>> index 0b481cc9c725..e9ed0e666a83 100644
>>>> --- a/arch/x86/kvm/mmu.c
>>>> +++ b/arch/x86/kvm/mmu.c
>>>> @@ -2708,7 +2708,24 @@ static bool mmu_need_write_protect(struct kvm_vcpu *vcpu, gfn_t gfn,
>>>>    static bool kvm_is_mmio_pfn(kvm_pfn_t pfn)
>>>>    {
>>>>    	if (pfn_valid(pfn))
>>>> -		return !is_zero_pfn(pfn) && PageReserved(pfn_to_page(pfn));
>>>> +		return !is_zero_pfn(pfn) && PageReserved(pfn_to_page(pfn)) &&
>>>> +			/*
>>>> +			 * Some reserved pages, such as those from
>>>> +			 * NVDIMM DAX devices, are not for MMIO, and
>>>> +			 * can be mapped with cached memory type for
>>>> +			 * better performance. However, the above
>>>> +			 * check misconceives those pages as MMIO.
>>>> +			 * Because KVM maps MMIO pages with UC memory
>>>> +			 * type, the performance of guest accesses to
>>>> +			 * those pages would be harmed. Therefore, we
>>>> +			 * check the host memory type in addition and
>>>> +			 * only treat UC/UC- pages as MMIO.
>>>> +			 *
>>>> +			 * pat_pfn_is_uc() works only when PAT is enabled,
>>>> +			 * so check pat_enabled() as well.
>>>> +			 */
>>>> +			(!pat_enabled() ||
>>>> +			 pat_pfn_is_uc(kvm_pfn_t_to_pfn_t(pfn)));
>>>
>>> Can it be compiled if !CONFIG_PAT?
>>
>> Yes.
>>
>> What I check via pat_enabled() is not only whether PAT support is
>> compiled, but also whether PAT is enabled at runtime.
>>
>>>
>>> It would be better if we move pat_enabled out of kvm as well,
>>
>> Surely I can combine them in one function like
>>
>> bool pat_pfn_is_uc(pfn_t pfn)
>> {
>> 	enum page_cache_mode cm;
>>
>> 	if (!pat_enabled())
>> 		return false;
>>
>> 	cm = lookup_memtype(pfn_t_to_phys(pfn));
>>
>> 	return cm == _PAGE_CACHE_MODE_UC || cm == _PAGE_CACHE_MODE_UC_MINUS;
>> }
> 
> In addition, I think it's better to split this function into
> pat_pfn_is_uc() and pat_pfn_is_uc_minus() to avoid additional
> confusion.

Why not use pat_pfn_is_uc_or_uc_minus(). :)

Re: [PATCH v4 3/3] KVM: MMU: consider host cache mode in MMIO page check

[Haozhong Zhang]

On 11/03/17 17:10 +0800, Xiao Guangrong wrote:
> 
> 
> On 11/03/2017 04:51 PM, Haozhong Zhang wrote:
> > On 11/03/17 14:54 +0800, Xiao Guangrong wrote:
> > > 
> > > 
> > > On 11/03/2017 01:53 PM, Haozhong Zhang wrote:
> > > > Some reserved pages, such as those from NVDIMM DAX devices, are
> > > > not for MMIO, and can be mapped with cached memory type for better
> > > > performance. However, the above check misconceives those pages as
> > > > MMIO.  Because KVM maps MMIO pages with UC memory type, the
> > > > performance of guest accesses to those pages would be harmed.
> > > > Therefore, we check the host memory type by lookup_memtype() in
> > > > addition and only treat UC/UC- pages as MMIO.
> > > > 
> > > > Signed-off-by: Haozhong Zhang <haozhong.zhang@intel.com>
> > > > Reported-by: Cuevas Escareno, Ivan D <ivan.d.cuevas.escareno@intel.com>
> > > > Reported-by: Kumar, Karthik <karthik.kumar@intel.com>
> > > > ---
> > > >    arch/x86/kvm/mmu.c | 19 ++++++++++++++++++-
> > > >    1 file changed, 18 insertions(+), 1 deletion(-)
> > > > 
> > > > diff --git a/arch/x86/kvm/mmu.c b/arch/x86/kvm/mmu.c
> > > > index 0b481cc9c725..e9ed0e666a83 100644
> > > > --- a/arch/x86/kvm/mmu.c
> > > > +++ b/arch/x86/kvm/mmu.c
> > > > @@ -2708,7 +2708,24 @@ static bool mmu_need_write_protect(struct kvm_vcpu *vcpu, gfn_t gfn,
> > > >    static bool kvm_is_mmio_pfn(kvm_pfn_t pfn)
> > > >    {
> > > >    	if (pfn_valid(pfn))
> > > > -		return !is_zero_pfn(pfn) && PageReserved(pfn_to_page(pfn));
> > > > +		return !is_zero_pfn(pfn) && PageReserved(pfn_to_page(pfn)) &&
> > > > +			/*
> > > > +			 * Some reserved pages, such as those from
> > > > +			 * NVDIMM DAX devices, are not for MMIO, and
> > > > +			 * can be mapped with cached memory type for
> > > > +			 * better performance. However, the above
> > > > +			 * check misconceives those pages as MMIO.
> > > > +			 * Because KVM maps MMIO pages with UC memory
> > > > +			 * type, the performance of guest accesses to
> > > > +			 * those pages would be harmed. Therefore, we
> > > > +			 * check the host memory type in addition and
> > > > +			 * only treat UC/UC- pages as MMIO.
> > > > +			 *
> > > > +			 * pat_pfn_is_uc() works only when PAT is enabled,
> > > > +			 * so check pat_enabled() as well.
> > > > +			 */
> > > > +			(!pat_enabled() ||
> > > > +			 pat_pfn_is_uc(kvm_pfn_t_to_pfn_t(pfn)));
> > > 
> > > Can it be compiled if !CONFIG_PAT?
> > 
> > Yes.
> > 
> > What I check via pat_enabled() is not only whether PAT support is
> > compiled, but also whether PAT is enabled at runtime.
> 
> The issue is about pat_pfn_is_uc() which is implemented only if CONFIG_PAT is
> enabled, but you used it here unconditionally.
> 
> I am not sure if gcc is smart enough to omit pat_pfn_is_uc() completely under
> this case. If you really have done the test to compile kernel and KVM module
> with CONFIG_PAT disabled, it is fine.
> 

I've done the test and it can compile.

arch/x86/mm/Makefile shows pat.c is compiled regardless of CONFIG_X86_PAT,
and pat_pfn_is_uc() is defined out of  #ifdef CONFIG_X86_PAT ... #endif.

Haozhong

Re: [PATCH v4 3/3] KVM: MMU: consider host cache mode in MMIO page check

[Haozhong Zhang]

On 11/03/17 17:24 +0800, Xiao Guangrong wrote:
> 
> 
> On 11/03/2017 05:02 PM, Haozhong Zhang wrote:
> > On 11/03/17 16:51 +0800, Haozhong Zhang wrote:
> > > On 11/03/17 14:54 +0800, Xiao Guangrong wrote:
> > > > 
> > > > 
> > > > On 11/03/2017 01:53 PM, Haozhong Zhang wrote:
> > > > > Some reserved pages, such as those from NVDIMM DAX devices, are
> > > > > not for MMIO, and can be mapped with cached memory type for better
> > > > > performance. However, the above check misconceives those pages as
> > > > > MMIO.  Because KVM maps MMIO pages with UC memory type, the
> > > > > performance of guest accesses to those pages would be harmed.
> > > > > Therefore, we check the host memory type by lookup_memtype() in
> > > > > addition and only treat UC/UC- pages as MMIO.
> > > > > 
> > > > > Signed-off-by: Haozhong Zhang <haozhong.zhang@intel.com>
> > > > > Reported-by: Cuevas Escareno, Ivan D <ivan.d.cuevas.escareno@intel.com>
> > > > > Reported-by: Kumar, Karthik <karthik.kumar@intel.com>
> > > > > ---
> > > > >    arch/x86/kvm/mmu.c | 19 ++++++++++++++++++-
> > > > >    1 file changed, 18 insertions(+), 1 deletion(-)
> > > > > 
> > > > > diff --git a/arch/x86/kvm/mmu.c b/arch/x86/kvm/mmu.c
> > > > > index 0b481cc9c725..e9ed0e666a83 100644
> > > > > --- a/arch/x86/kvm/mmu.c
> > > > > +++ b/arch/x86/kvm/mmu.c
> > > > > @@ -2708,7 +2708,24 @@ static bool mmu_need_write_protect(struct kvm_vcpu *vcpu, gfn_t gfn,
> > > > >    static bool kvm_is_mmio_pfn(kvm_pfn_t pfn)
> > > > >    {
> > > > >    	if (pfn_valid(pfn))
> > > > > -		return !is_zero_pfn(pfn) && PageReserved(pfn_to_page(pfn));
> > > > > +		return !is_zero_pfn(pfn) && PageReserved(pfn_to_page(pfn)) &&
> > > > > +			/*
> > > > > +			 * Some reserved pages, such as those from
> > > > > +			 * NVDIMM DAX devices, are not for MMIO, and
> > > > > +			 * can be mapped with cached memory type for
> > > > > +			 * better performance. However, the above
> > > > > +			 * check misconceives those pages as MMIO.
> > > > > +			 * Because KVM maps MMIO pages with UC memory
> > > > > +			 * type, the performance of guest accesses to
> > > > > +			 * those pages would be harmed. Therefore, we
> > > > > +			 * check the host memory type in addition and
> > > > > +			 * only treat UC/UC- pages as MMIO.
> > > > > +			 *
> > > > > +			 * pat_pfn_is_uc() works only when PAT is enabled,
> > > > > +			 * so check pat_enabled() as well.
> > > > > +			 */
> > > > > +			(!pat_enabled() ||
> > > > > +			 pat_pfn_is_uc(kvm_pfn_t_to_pfn_t(pfn)));
> > > > 
> > > > Can it be compiled if !CONFIG_PAT?
> > > 
> > > Yes.
> > > 
> > > What I check via pat_enabled() is not only whether PAT support is
> > > compiled, but also whether PAT is enabled at runtime.
> > > 
> > > > 
> > > > It would be better if we move pat_enabled out of kvm as well,
> > > 
> > > Surely I can combine them in one function like
> > > 
> > > bool pat_pfn_is_uc(pfn_t pfn)
> > > {
> > > 	enum page_cache_mode cm;
> > > 
> > > 	if (!pat_enabled())
> > > 		return false;
> > > 
> > > 	cm = lookup_memtype(pfn_t_to_phys(pfn));
> > > 
> > > 	return cm == _PAGE_CACHE_MODE_UC || cm == _PAGE_CACHE_MODE_UC_MINUS;
> > > }
> > 
> > In addition, I think it's better to split this function into
> > pat_pfn_is_uc() and pat_pfn_is_uc_minus() to avoid additional
> > confusion.
> 
> Why not use pat_pfn_is_uc_or_uc_minus(). :)

Just in case that other places other than KVM do not need both of them.

Re: [PATCH v4 2/3] KVM: add converters between pfn_t and kvm_pfn_t

[Dan Williams]

On Thu, Nov 2, 2017 at 10:53 PM, Haozhong Zhang
<haozhong.zhang@intel.com> wrote:
> Signed-off-by: Haozhong Zhang <haozhong.zhang@intel.com>

Can you also add some text to the changelog saying why we need these converters?

> ---
>  include/linux/kvm_host.h | 4 ++++
>  1 file changed, 4 insertions(+)

Re: [PATCH v4 2/3] KVM: add converters between pfn_t and kvm_pfn_t

[Haozhong Zhang]

On 11/03/17 06:21 -0700, Dan Williams wrote:
> On Thu, Nov 2, 2017 at 10:53 PM, Haozhong Zhang
> <haozhong.zhang@intel.com> wrote:
> > Signed-off-by: Haozhong Zhang <haozhong.zhang@intel.com>
> 
> Can you also add some text to the changelog saying why we need these converters?

I'm going to drop these converters in the next version. The function
introduced in patch 1 does not need flags in pfn_t, so unsigned long
is enough. I'll change pfn_t to unsigned long in patch 1 as well.

Haozhong

Re: [PATCH v4 3/3] KVM: MMU: consider host cache mode in MMIO page check

[Haozhong Zhang]

On 11/03/17 16:51 +0800, Haozhong Zhang wrote:
> On 11/03/17 14:54 +0800, Xiao Guangrong wrote:
> > 
> > 
> > On 11/03/2017 01:53 PM, Haozhong Zhang wrote:
> > > Some reserved pages, such as those from NVDIMM DAX devices, are
> > > not for MMIO, and can be mapped with cached memory type for better
> > > performance. However, the above check misconceives those pages as
> > > MMIO.  Because KVM maps MMIO pages with UC memory type, the
> > > performance of guest accesses to those pages would be harmed.
> > > Therefore, we check the host memory type by lookup_memtype() in
> > > addition and only treat UC/UC- pages as MMIO.
> > > 
> > > Signed-off-by: Haozhong Zhang <haozhong.zhang@intel.com>
> > > Reported-by: Cuevas Escareno, Ivan D <ivan.d.cuevas.escareno@intel.com>
> > > Reported-by: Kumar, Karthik <karthik.kumar@intel.com>
> > > ---
> > >   arch/x86/kvm/mmu.c | 19 ++++++++++++++++++-
> > >   1 file changed, 18 insertions(+), 1 deletion(-)
> > > 
> > > diff --git a/arch/x86/kvm/mmu.c b/arch/x86/kvm/mmu.c
> > > index 0b481cc9c725..e9ed0e666a83 100644
> > > --- a/arch/x86/kvm/mmu.c
> > > +++ b/arch/x86/kvm/mmu.c
> > > @@ -2708,7 +2708,24 @@ static bool mmu_need_write_protect(struct kvm_vcpu *vcpu, gfn_t gfn,
> > >   static bool kvm_is_mmio_pfn(kvm_pfn_t pfn)
> > >   {
> > >   	if (pfn_valid(pfn))
> > > -		return !is_zero_pfn(pfn) && PageReserved(pfn_to_page(pfn));
> > > +		return !is_zero_pfn(pfn) && PageReserved(pfn_to_page(pfn)) &&
> > > +			/*
> > > +			 * Some reserved pages, such as those from
> > > +			 * NVDIMM DAX devices, are not for MMIO, and
> > > +			 * can be mapped with cached memory type for
> > > +			 * better performance. However, the above
> > > +			 * check misconceives those pages as MMIO.
> > > +			 * Because KVM maps MMIO pages with UC memory
> > > +			 * type, the performance of guest accesses to
> > > +			 * those pages would be harmed. Therefore, we
> > > +			 * check the host memory type in addition and
> > > +			 * only treat UC/UC- pages as MMIO.
> > > +			 *
> > > +			 * pat_pfn_is_uc() works only when PAT is enabled,
> > > +			 * so check pat_enabled() as well.
> > > +			 */
> > > +			(!pat_enabled() ||
> > > +			 pat_pfn_is_uc(kvm_pfn_t_to_pfn_t(pfn)));
> > 
> > Can it be compiled if !CONFIG_PAT?
> 
> Yes.
> 
> What I check via pat_enabled() is not only whether PAT support is
> compiled, but also whether PAT is enabled at runtime.
> 
> > 
> > It would be better if we move pat_enabled out of kvm as well,
> 
> Surely I can combine them in one function like
> 
> bool pat_pfn_is_uc(pfn_t pfn)
> {
> 	enum page_cache_mode cm;
> 
> 	if (!pat_enabled())
> 		return false;

I made a mistake: it should return true here. Then the semantics of
this function is confused. I think it's still better to leave
!pat_enabled() check in KVM.

Haozhong

> 
> 	cm = lookup_memtype(pfn_t_to_phys(pfn));
> 
> 	return cm == _PAGE_CACHE_MODE_UC || cm == _PAGE_CACHE_MODE_UC_MINUS;
> }
> 
> but I need a good name to make its semantics clear, or is it enough to
> just leave a comment like?
> 
> /*
>  * Check via PAT whether the cache mode of a page if UC or UC-.
>  *
>  * Returns true, if PAT is enabled and the cache mode is UC or UC-.
>  * Returns false otherwise.
>  */
> 
> 
> > please refer
> > to pgprot_writecombine() which is implemented in pat.c and in
> > include\asm-generic\pgtable.h:
> > 
> > #ifndef pgprot_writecombine
> > #define pgprot_writecombine pgprot_noncached
> > #endif
> >
> 
> 
> 

Re: [PATCH v4 3/3] KVM: MMU: consider host cache mode in MMIO page check

[Xiao Guangrong]

On 11/03/2017 05:29 PM, Haozhong Zhang wrote:
> On 11/03/17 17:24 +0800, Xiao Guangrong wrote:
>>
>>
>> On 11/03/2017 05:02 PM, Haozhong Zhang wrote:
>>> On 11/03/17 16:51 +0800, Haozhong Zhang wrote:
>>>> On 11/03/17 14:54 +0800, Xiao Guangrong wrote:
>>>>>
>>>>>
>>>>> On 11/03/2017 01:53 PM, Haozhong Zhang wrote:
>>>>>> Some reserved pages, such as those from NVDIMM DAX devices, are
>>>>>> not for MMIO, and can be mapped with cached memory type for better
>>>>>> performance. However, the above check misconceives those pages as
>>>>>> MMIO.  Because KVM maps MMIO pages with UC memory type, the
>>>>>> performance of guest accesses to those pages would be harmed.
>>>>>> Therefore, we check the host memory type by lookup_memtype() in
>>>>>> addition and only treat UC/UC- pages as MMIO.
>>>>>>
>>>>>> Signed-off-by: Haozhong Zhang <haozhong.zhang@intel.com>
>>>>>> Reported-by: Cuevas Escareno, Ivan D <ivan.d.cuevas.escareno@intel.com>
>>>>>> Reported-by: Kumar, Karthik <karthik.kumar@intel.com>
>>>>>> ---
>>>>>>     arch/x86/kvm/mmu.c | 19 ++++++++++++++++++-
>>>>>>     1 file changed, 18 insertions(+), 1 deletion(-)
>>>>>>
>>>>>> diff --git a/arch/x86/kvm/mmu.c b/arch/x86/kvm/mmu.c
>>>>>> index 0b481cc9c725..e9ed0e666a83 100644
>>>>>> --- a/arch/x86/kvm/mmu.c
>>>>>> +++ b/arch/x86/kvm/mmu.c
>>>>>> @@ -2708,7 +2708,24 @@ static bool mmu_need_write_protect(struct kvm_vcpu *vcpu, gfn_t gfn,
>>>>>>     static bool kvm_is_mmio_pfn(kvm_pfn_t pfn)
>>>>>>     {
>>>>>>     	if (pfn_valid(pfn))
>>>>>> -		return !is_zero_pfn(pfn) && PageReserved(pfn_to_page(pfn));
>>>>>> +		return !is_zero_pfn(pfn) && PageReserved(pfn_to_page(pfn)) &&
>>>>>> +			/*
>>>>>> +			 * Some reserved pages, such as those from
>>>>>> +			 * NVDIMM DAX devices, are not for MMIO, and
>>>>>> +			 * can be mapped with cached memory type for
>>>>>> +			 * better performance. However, the above
>>>>>> +			 * check misconceives those pages as MMIO.
>>>>>> +			 * Because KVM maps MMIO pages with UC memory
>>>>>> +			 * type, the performance of guest accesses to
>>>>>> +			 * those pages would be harmed. Therefore, we
>>>>>> +			 * check the host memory type in addition and
>>>>>> +			 * only treat UC/UC- pages as MMIO.
>>>>>> +			 *
>>>>>> +			 * pat_pfn_is_uc() works only when PAT is enabled,
>>>>>> +			 * so check pat_enabled() as well.
>>>>>> +			 */
>>>>>> +			(!pat_enabled() ||
>>>>>> +			 pat_pfn_is_uc(kvm_pfn_t_to_pfn_t(pfn)));
>>>>>
>>>>> Can it be compiled if !CONFIG_PAT?
>>>>
>>>> Yes.
>>>>
>>>> What I check via pat_enabled() is not only whether PAT support is
>>>> compiled, but also whether PAT is enabled at runtime.
>>>>
>>>>>
>>>>> It would be better if we move pat_enabled out of kvm as well,
>>>>
>>>> Surely I can combine them in one function like
>>>>
>>>> bool pat_pfn_is_uc(pfn_t pfn)
>>>> {
>>>> 	enum page_cache_mode cm;
>>>>
>>>> 	if (!pat_enabled())
>>>> 		return false;
>>>>
>>>> 	cm = lookup_memtype(pfn_t_to_phys(pfn));
>>>>
>>>> 	return cm == _PAGE_CACHE_MODE_UC || cm == _PAGE_CACHE_MODE_UC_MINUS;
>>>> }
>>>
>>> In addition, I think it's better to split this function into
>>> pat_pfn_is_uc() and pat_pfn_is_uc_minus() to avoid additional
>>> confusion.
>>
>> Why not use pat_pfn_is_uc_or_uc_minus(). :)
> 
> Just in case that other places other than KVM do not need both of them.
> 

I think we need not care the future case too much, it is not ABI
anyway. i.e, it can be easily adjusted if it is really needed
in the future.

But it is not a big deal...