[PATCH] mm/dev_pfn: Exclude MEMORY_DEVICE_PRIVATE while computing virtual address

[PATCH] mm/dev_pfn: Exclude MEMORY_DEVICE_PRIVATE while computing virtual address

[Anshuman Khandual]

The presence of struct page does not guarantee linear mapping for the pfn
physical range. Device private memory which is non-coherent is excluded
from linear mapping during devm_memremap_pages() though they will still
have struct page coverage. Just check for device private memory before
giving out virtual address for a given pfn.

Signed-off-by: Anshuman Khandual <anshuman.khandual@arm.com>
---
All these helper functions are all pfn_t related but could not figure out
another way of determining a private pfn without looking into it's struct
page. pfn_t_to_virt() is not getting used any where in mainline kernel.Is
it used by out of tree drivers ? Should we then drop it completely ?

 include/linux/pfn_t.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/include/linux/pfn_t.h b/include/linux/pfn_t.h
index 7bb7785..3c202a1 100644
--- a/include/linux/pfn_t.h
+++ b/include/linux/pfn_t.h
@@ -68,7 +68,7 @@ static inline phys_addr_t pfn_t_to_phys(pfn_t pfn)
 
 static inline void *pfn_t_to_virt(pfn_t pfn)
 {
-	if (pfn_t_has_page(pfn))
+	if (pfn_t_has_page(pfn) && !is_device_private_page(pfn_t_to_page(pfn)))
 		return __va(pfn_t_to_phys(pfn));
 	return NULL;
 }
-- 
2.7.4

Re: [PATCH] mm/dev_pfn: Exclude MEMORY_DEVICE_PRIVATE while computing virtual address

[Andrew Morton]

On Fri, 17 May 2019 16:08:34 +0530 Anshuman Khandual <anshuman.khandual@arm.com> wrote:

> The presence of struct page does not guarantee linear mapping for the pfn
> physical range. Device private memory which is non-coherent is excluded
> from linear mapping during devm_memremap_pages() though they will still
> have struct page coverage. Just check for device private memory before
> giving out virtual address for a given pfn.

I was going to give my standard "what are the user-visible runtime
effects of this change?", but...

> All these helper functions are all pfn_t related but could not figure out
> another way of determining a private pfn without looking into it's struct
> page. pfn_t_to_virt() is not getting used any where in mainline kernel.Is
> it used by out of tree drivers ? Should we then drop it completely ?

Yeah, let's kill it.

But first, let's fix it so that if someone brings it back, they bring
back a non-buggy version.

So...  what (would be) the user-visible runtime effects of this change?

Re: [PATCH] mm/dev_pfn: Exclude MEMORY_DEVICE_PRIVATE while computing virtual address

[Anshuman Khandual]

On 05/18/2019 03:20 AM, Andrew Morton wrote:
> On Fri, 17 May 2019 16:08:34 +0530 Anshuman Khandual <anshuman.khandual@arm.com> wrote:
> 
>> The presence of struct page does not guarantee linear mapping for the pfn
>> physical range. Device private memory which is non-coherent is excluded
>> from linear mapping during devm_memremap_pages() though they will still
>> have struct page coverage. Just check for device private memory before
>> giving out virtual address for a given pfn.
> 
> I was going to give my standard "what are the user-visible runtime
> effects of this change?", but...
> 
>> All these helper functions are all pfn_t related but could not figure out
>> another way of determining a private pfn without looking into it's struct
>> page. pfn_t_to_virt() is not getting used any where in mainline kernel.Is
>> it used by out of tree drivers ? Should we then drop it completely ?
> 
> Yeah, let's kill it.
> 
> But first, let's fix it so that if someone brings it back, they bring
> back a non-buggy version.

Makes sense.

> 
> So...  what (would be) the user-visible runtime effects of this change?

I am not very well aware about the user interaction with the drivers which
hotplug and manage ZONE_DEVICE memory in general. Hence will not be able to
comment on it's user visible runtime impact. I just figured this out from
code audit while testing ZONE_DEVICE on arm64 platform. But the fix makes
the function bit more expensive as it now involve some additional memory
references.

Re: [PATCH] mm/dev_pfn: Exclude MEMORY_DEVICE_PRIVATE while computing virtual address

[Dan Williams]

On Sun, May 19, 2019 at 10:37 PM Anshuman Khandual
<anshuman.khandual@arm.com> wrote:
>
> On 05/18/2019 03:20 AM, Andrew Morton wrote:
> > On Fri, 17 May 2019 16:08:34 +0530 Anshuman Khandual <anshuman.khandual@arm.com> wrote:
> >
> >> The presence of struct page does not guarantee linear mapping for the pfn
> >> physical range. Device private memory which is non-coherent is excluded
> >> from linear mapping during devm_memremap_pages() though they will still
> >> have struct page coverage. Just check for device private memory before
> >> giving out virtual address for a given pfn.
> >
> > I was going to give my standard "what are the user-visible runtime
> > effects of this change?", but...
> >
> >> All these helper functions are all pfn_t related but could not figure out
> >> another way of determining a private pfn without looking into it's struct
> >> page. pfn_t_to_virt() is not getting used any where in mainline kernel.Is
> >> it used by out of tree drivers ? Should we then drop it completely ?
> >
> > Yeah, let's kill it.

+1 to killing it, since there has been a paucity of 'unsigned long
pfn' code path conversions to 'pfn_t', and it continues to go unused.

> > But first, let's fix it so that if someone brings it back, they bring
> > back a non-buggy version.

Not sure this can be solved without a rethink of who owns the virtual
address space corresponding to MEMORY_DEVICE_PRIVATE, and clawing back
some of the special-ness of HMM.

>
> Makes sense.
>
> >
> > So...  what (would be) the user-visible runtime effects of this change?
>
> I am not very well aware about the user interaction with the drivers which
> hotplug and manage ZONE_DEVICE memory in general. Hence will not be able to
> comment on it's user visible runtime impact. I just figured this out from
> code audit while testing ZONE_DEVICE on arm64 platform. But the fix makes
> the function bit more expensive as it now involve some additional memory
> references.

MEMORY_DEVICE_PRIVATE semantics were part of the package of the
initial HMM submission that landed in the kernel without an upstream
user. While pfn_t_to_virt() also does not have an upstream user it was
at least modeled after the existing pfn_to_virt() api to allow for
future 'unsigned long pfn' to 'pfn_t' conversions. As for what a fix
might look like, it seems to me that we should try to unify 'pfn_t'
and 'hmm_pfn's. I don't see why 'hmm_pfn's need to exist as their own
concept vs trying consume more flag space out of pfn_t. That would at
least allow the pfn_t_has_page() helper to detect the HMM case.

Re: [PATCH] mm/dev_pfn: Exclude MEMORY_DEVICE_PRIVATE while computing virtual address

[Jerome Glisse]

On Mon, May 20, 2019 at 11:07:38AM +0530, Anshuman Khandual wrote:
> On 05/18/2019 03:20 AM, Andrew Morton wrote:
> > On Fri, 17 May 2019 16:08:34 +0530 Anshuman Khandual <anshuman.khandual@arm.com> wrote:
> > 
> >> The presence of struct page does not guarantee linear mapping for the pfn
> >> physical range. Device private memory which is non-coherent is excluded
> >> from linear mapping during devm_memremap_pages() though they will still
> >> have struct page coverage. Just check for device private memory before
> >> giving out virtual address for a given pfn.
> > 
> > I was going to give my standard "what are the user-visible runtime
> > effects of this change?", but...
> > 
> >> All these helper functions are all pfn_t related but could not figure out
> >> another way of determining a private pfn without looking into it's struct
> >> page. pfn_t_to_virt() is not getting used any where in mainline kernel.Is
> >> it used by out of tree drivers ? Should we then drop it completely ?
> > 
> > Yeah, let's kill it.
> > 
> > But first, let's fix it so that if someone brings it back, they bring
> > back a non-buggy version.
> 
> Makes sense.
> 
> > 
> > So...  what (would be) the user-visible runtime effects of this change?
> 
> I am not very well aware about the user interaction with the drivers which
> hotplug and manage ZONE_DEVICE memory in general. Hence will not be able to
> comment on it's user visible runtime impact. I just figured this out from
> code audit while testing ZONE_DEVICE on arm64 platform. But the fix makes
> the function bit more expensive as it now involve some additional memory
> references.

A device private pfn can never leak outside code that does not understand it
So this change is useless for any existing users and i would like to keep the
existing behavior ie never leak device private pfn.

Cheers,
Jérôme

Re: [PATCH] mm/dev_pfn: Exclude MEMORY_DEVICE_PRIVATE while computing virtual address

[Dan Williams]

On Mon, May 20, 2019 at 12:27 PM Jerome Glisse <jglisse@redhat.com> wrote:
>
> On Mon, May 20, 2019 at 11:07:38AM +0530, Anshuman Khandual wrote:
> > On 05/18/2019 03:20 AM, Andrew Morton wrote:
> > > On Fri, 17 May 2019 16:08:34 +0530 Anshuman Khandual <anshuman.khandual@arm.com> wrote:
> > >
> > >> The presence of struct page does not guarantee linear mapping for the pfn
> > >> physical range. Device private memory which is non-coherent is excluded
> > >> from linear mapping during devm_memremap_pages() though they will still
> > >> have struct page coverage. Just check for device private memory before
> > >> giving out virtual address for a given pfn.
> > >
> > > I was going to give my standard "what are the user-visible runtime
> > > effects of this change?", but...
> > >
> > >> All these helper functions are all pfn_t related but could not figure out
> > >> another way of determining a private pfn without looking into it's struct
> > >> page. pfn_t_to_virt() is not getting used any where in mainline kernel.Is
> > >> it used by out of tree drivers ? Should we then drop it completely ?
> > >
> > > Yeah, let's kill it.
> > >
> > > But first, let's fix it so that if someone brings it back, they bring
> > > back a non-buggy version.
> >
> > Makes sense.
> >
> > >
> > > So...  what (would be) the user-visible runtime effects of this change?
> >
> > I am not very well aware about the user interaction with the drivers which
> > hotplug and manage ZONE_DEVICE memory in general. Hence will not be able to
> > comment on it's user visible runtime impact. I just figured this out from
> > code audit while testing ZONE_DEVICE on arm64 platform. But the fix makes
> > the function bit more expensive as it now involve some additional memory
> > references.
>
> A device private pfn can never leak outside code that does not understand it
> So this change is useless for any existing users and i would like to keep the
> existing behavior ie never leak device private pfn.

The issue is that only an HMM expert might know that such a pfn can
never leak, in other words the pfn concept from a code perspective is
already leaked / widespread. Ideally any developer familiar with a pfn
and the core-mm pfn helpers need only worry about pfn semantics
without being required to go audit HMM users.

Re: [PATCH] mm/dev_pfn: Exclude MEMORY_DEVICE_PRIVATE while computing virtual address

[Dan Williams]

On Mon, May 20, 2019 at 12:27 PM Jerome Glisse <jglisse@redhat.com> wrote:
>
> On Mon, May 20, 2019 at 11:07:38AM +0530, Anshuman Khandual wrote:
> > On 05/18/2019 03:20 AM, Andrew Morton wrote:
> > > On Fri, 17 May 2019 16:08:34 +0530 Anshuman Khandual <anshuman.khandual@arm.com> wrote:
> > >
> > >> The presence of struct page does not guarantee linear mapping for the pfn
> > >> physical range. Device private memory which is non-coherent is excluded
> > >> from linear mapping during devm_memremap_pages() though they will still
> > >> have struct page coverage. Just check for device private memory before
> > >> giving out virtual address for a given pfn.
> > >
> > > I was going to give my standard "what are the user-visible runtime
> > > effects of this change?", but...
> > >
> > >> All these helper functions are all pfn_t related but could not figure out
> > >> another way of determining a private pfn without looking into it's struct
> > >> page. pfn_t_to_virt() is not getting used any where in mainline kernel.Is
> > >> it used by out of tree drivers ? Should we then drop it completely ?
> > >
> > > Yeah, let's kill it.
> > >
> > > But first, let's fix it so that if someone brings it back, they bring
> > > back a non-buggy version.
> >
> > Makes sense.
> >
> > >
> > > So...  what (would be) the user-visible runtime effects of this change?
> >
> > I am not very well aware about the user interaction with the drivers which
> > hotplug and manage ZONE_DEVICE memory in general. Hence will not be able to
> > comment on it's user visible runtime impact. I just figured this out from
> > code audit while testing ZONE_DEVICE on arm64 platform. But the fix makes
> > the function bit more expensive as it now involve some additional memory
> > references.
>
> A device private pfn can never leak outside code that does not understand it
> So this change is useless for any existing users and i would like to keep the
> existing behavior ie never leak device private pfn.

The issue is that only an HMM expert might know that such a pfn can
never leak, in other words the pfn concept from a code perspective is
already leaked / widespread. Ideally any developer familiar with a pfn
and the core-mm pfn helpers need only worry about pfn semantics
without being required to go audit HMM users.

Re: [PATCH] mm/dev_pfn: Exclude MEMORY_DEVICE_PRIVATE while computing virtual address

[Anshuman Khandual]

On 05/21/2019 01:03 AM, Dan Williams wrote:
> On Mon, May 20, 2019 at 12:27 PM Jerome Glisse <jglisse@redhat.com> wrote:
>>
>> On Mon, May 20, 2019 at 11:07:38AM +0530, Anshuman Khandual wrote:
>>> On 05/18/2019 03:20 AM, Andrew Morton wrote:
>>>> On Fri, 17 May 2019 16:08:34 +0530 Anshuman Khandual <anshuman.khandual@arm.com> wrote:
>>>>
>>>>> The presence of struct page does not guarantee linear mapping for the pfn
>>>>> physical range. Device private memory which is non-coherent is excluded
>>>>> from linear mapping during devm_memremap_pages() though they will still
>>>>> have struct page coverage. Just check for device private memory before
>>>>> giving out virtual address for a given pfn.
>>>>
>>>> I was going to give my standard "what are the user-visible runtime
>>>> effects of this change?", but...
>>>>
>>>>> All these helper functions are all pfn_t related but could not figure out
>>>>> another way of determining a private pfn without looking into it's struct
>>>>> page. pfn_t_to_virt() is not getting used any where in mainline kernel.Is
>>>>> it used by out of tree drivers ? Should we then drop it completely ?
>>>>
>>>> Yeah, let's kill it.
>>>>
>>>> But first, let's fix it so that if someone brings it back, they bring
>>>> back a non-buggy version.
>>>
>>> Makes sense.
>>>
>>>>
>>>> So...  what (would be) the user-visible runtime effects of this change?
>>>
>>> I am not very well aware about the user interaction with the drivers which
>>> hotplug and manage ZONE_DEVICE memory in general. Hence will not be able to
>>> comment on it's user visible runtime impact. I just figured this out from
>>> code audit while testing ZONE_DEVICE on arm64 platform. But the fix makes
>>> the function bit more expensive as it now involve some additional memory
>>> references.
>>
>> A device private pfn can never leak outside code that does not understand it
>> So this change is useless for any existing users and i would like to keep the
>> existing behavior ie never leak device private pfn.
> 
> The issue is that only an HMM expert might know that such a pfn can
> never leak, in other words the pfn concept from a code perspective is
> already leaked / widespread. Ideally any developer familiar with a pfn
> and the core-mm pfn helpers need only worry about pfn semantics
> without being required to go audit HMM users.

Agreed.