[rfc 1/2] mm/hmm: Allow smaps to see zone device public pages

[rfc 1/2] mm/hmm: Allow smaps to see zone device public pages

[Balbir Singh]

vm_normal_page() normally does not return zone device public
pages. In the absence of the visibility the output from smaps
is limited and confusing. It's hard to figure out where the
pages are. This patch uses _vm_normal_page() to expose them
for accounting

Signed-off-by: Balbir Singh <bsingharora@gmail.com>
---
 fs/proc/task_mmu.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c
index 5589b4bd4b85..9f1e2b2b5f5a 100644
--- a/fs/proc/task_mmu.c
+++ b/fs/proc/task_mmu.c
@@ -528,7 +528,7 @@ static void smaps_pte_entry(pte_t *pte, unsigned long addr,
 	struct page *page = NULL;
 
 	if (pte_present(*pte)) {
-		page = vm_normal_page(vma, addr, *pte);
+		page = _vm_normal_page(vma, addr, *pte, true);
 	} else if (is_swap_pte(*pte)) {
 		swp_entry_t swpent = pte_to_swp_entry(*pte);
 
@@ -708,7 +708,7 @@ static int smaps_hugetlb_range(pte_t *pte, unsigned long hmask,
 	struct page *page = NULL;
 
 	if (pte_present(*pte)) {
-		page = vm_normal_page(vma, addr, *pte);
+		page = _vm_normal_page(vma, addr, *pte, true);
 	} else if (is_swap_pte(*pte)) {
 		swp_entry_t swpent = pte_to_swp_entry(*pte);
 
-- 
2.13.6

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

[rfc 2/2] smaps: Show zone device memory used

[Balbir Singh]

With HMM, we can have either public or private zone
device pages. With private zone device pages, they should
show up as swapped entities. For public zone device pages
the smaps output can be confusing and incomplete.

This patch adds a new attribute to just smaps to show
device memory usage.

Signed-off-by: Balbir Singh <bsingharora@gmail.com>
---
 fs/proc/task_mmu.c | 17 +++++++++++++++--
 1 file changed, 15 insertions(+), 2 deletions(-)

diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c
index 9f1e2b2b5f5a..b7f32f42ee93 100644
--- a/fs/proc/task_mmu.c
+++ b/fs/proc/task_mmu.c
@@ -451,6 +451,7 @@ struct mem_size_stats {
 	unsigned long shared_hugetlb;
 	unsigned long private_hugetlb;
 	unsigned long first_vma_start;
+	unsigned long device_memory;
 	u64 pss;
 	u64 pss_locked;
 	u64 swap_pss;
@@ -463,12 +464,22 @@ static void smaps_account(struct mem_size_stats *mss, struct page *page,
 	int i, nr = compound ? 1 << compound_order(page) : 1;
 	unsigned long size = nr * PAGE_SIZE;
 
+	/*
+	 * We don't want to process public zone device pages further
+	 * than just showing how much device memory we have
+	 */
+	if (is_zone_device_page(page)) {
+		mss->device_memory += size;
+		return;
+	}
+
 	if (PageAnon(page)) {
 		mss->anonymous += size;
 		if (!PageSwapBacked(page) && !dirty && !PageDirty(page))
 			mss->lazyfree += size;
 	}
 
+
 	mss->resident += size;
 	/* Accumulate the size in pages that have been accessed. */
 	if (young || page_is_young(page) || PageReferenced(page))
@@ -833,7 +844,8 @@ static int show_smap(struct seq_file *m, void *v, int is_pid)
 			   "Private_Hugetlb: %7lu kB\n"
 			   "Swap:           %8lu kB\n"
 			   "SwapPss:        %8lu kB\n"
-			   "Locked:         %8lu kB\n",
+			   "Locked:         %8lu kB\n"
+			   "DeviceMem:      %8lu kB\n",
 			   mss->resident >> 10,
 			   (unsigned long)(mss->pss >> (10 + PSS_SHIFT)),
 			   mss->shared_clean  >> 10,
@@ -849,7 +861,8 @@ static int show_smap(struct seq_file *m, void *v, int is_pid)
 			   mss->private_hugetlb >> 10,
 			   mss->swap >> 10,
 			   (unsigned long)(mss->swap_pss >> (10 + PSS_SHIFT)),
-			   (unsigned long)(mss->pss >> (10 + PSS_SHIFT)));
+			   (unsigned long)(mss->pss >> (10 + PSS_SHIFT)),
+			   mss->device_memory >> 10);
 
 	if (!rollup_mode) {
 		arch_show_smap(m, vma);
-- 
2.13.6

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [rfc 1/2] mm/hmm: Allow smaps to see zone device public pages

[Anshuman Khandual]

On 10/18/2017 12:01 PM, Balbir Singh wrote:
> vm_normal_page() normally does not return zone device public
> pages. In the absence of the visibility the output from smaps

It never does, as it calls _vm_normal_page() with with_public
_device = false, which skips all ZONE_DEVICE pages which are
MEMORY_DEVICE_PUBLIC.

> is limited and confusing. It's hard to figure out where the
> pages are. This patch uses _vm_normal_page() to expose them

Just a small nit, 'uses _vm_normal_page() with with_public_
device as true'.

> for accounting

Makes sense. It will help to have a small snippet of smaps output
with and without this patch demonstrating the difference. That
apart change looks good.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [rfc 2/2] smaps: Show zone device memory used

[Anshuman Khandual]

On 10/18/2017 12:01 PM, Balbir Singh wrote:
> With HMM, we can have either public or private zone
> device pages. With private zone device pages, they should
> show up as swapped entities. For public zone device pages

Might be missing something here but why they should show up
as swapped entities ? Could you please elaborate.

> the smaps output can be confusing and incomplete.
> 
> This patch adds a new attribute to just smaps to show
> device memory usage.

If we are any way adding a new entry here then why not one
more for private device memory pages as well. Just being
curious.

> 
> Signed-off-by: Balbir Singh <bsingharora@gmail.com>
> ---
>  fs/proc/task_mmu.c | 17 +++++++++++++++--
>  1 file changed, 15 insertions(+), 2 deletions(-)
> 
> diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c
> index 9f1e2b2b5f5a..b7f32f42ee93 100644
> --- a/fs/proc/task_mmu.c
> +++ b/fs/proc/task_mmu.c
> @@ -451,6 +451,7 @@ struct mem_size_stats {
>  	unsigned long shared_hugetlb;
>  	unsigned long private_hugetlb;
>  	unsigned long first_vma_start;
> +	unsigned long device_memory;
>  	u64 pss;
>  	u64 pss_locked;
>  	u64 swap_pss;
> @@ -463,12 +464,22 @@ static void smaps_account(struct mem_size_stats *mss, struct page *page,
>  	int i, nr = compound ? 1 << compound_order(page) : 1;
>  	unsigned long size = nr * PAGE_SIZE;
>  
> +	/*
> +	 * We don't want to process public zone device pages further
> +	 * than just showing how much device memory we have
> +	 */
> +	if (is_zone_device_page(page)) {

Should not this contain both public and private device pages.

> +		mss->device_memory += size;
> +		return;
> +	}
> +
>  	if (PageAnon(page)) {
>  		mss->anonymous += size;
>  		if (!PageSwapBacked(page) && !dirty && !PageDirty(page))
>  			mss->lazyfree += size;
>  	}
>  
> +

Stray new line.

>  	mss->resident += size;
>  	/* Accumulate the size in pages that have been accessed. */
>  	if (young || page_is_young(page) || PageReferenced(page))
> @@ -833,7 +844,8 @@ static int show_smap(struct seq_file *m, void *v, int is_pid)
>  			   "Private_Hugetlb: %7lu kB\n"
>  			   "Swap:           %8lu kB\n"
>  			   "SwapPss:        %8lu kB\n"
> -			   "Locked:         %8lu kB\n",
> +			   "Locked:         %8lu kB\n"

Stray changed line.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [rfc 1/2] mm/hmm: Allow smaps to see zone device public pages

[Balbir Singh]

On Wed, 18 Oct 2017 12:26:25 +0530
Anshuman Khandual <khandual@linux.vnet.ibm.com> wrote:

> On 10/18/2017 12:01 PM, Balbir Singh wrote:
> > vm_normal_page() normally does not return zone device public
> > pages. In the absence of the visibility the output from smaps  
> 
> It never does, as it calls _vm_normal_page() with with_public
> _device = false, which skips all ZONE_DEVICE pages which are
> MEMORY_DEVICE_PUBLIC.

Yes, probably the use of normally is not required.

> 
> > is limited and confusing. It's hard to figure out where the
> > pages are. This patch uses _vm_normal_page() to expose them  
> 
> Just a small nit, 'uses _vm_normal_page() with with_public_
> device as true'.

OK, I'll reword.

> 
> > for accounting  
> 
> Makes sense. It will help to have a small snippet of smaps output
> with and without this patch demonstrating the difference. That
> apart change looks good.
> 

I can do that if its helpful

Balbir Singh.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [rfc 2/2] smaps: Show zone device memory used

[Balbir Singh]

On Wed, 18 Oct 2017 12:40:43 +0530
Anshuman Khandual <khandual@linux.vnet.ibm.com> wrote:

> On 10/18/2017 12:01 PM, Balbir Singh wrote:
> > With HMM, we can have either public or private zone
> > device pages. With private zone device pages, they should
> > show up as swapped entities. For public zone device pages  
> 
> Might be missing something here but why they should show up
> as swapped entities ? Could you please elaborate.
>

For migrated entries, my use case is to

1. malloc()/mmap() memory
2. call migrate_vma()
3. Look at smaps

It's probably not clear in the changelog.

> > the smaps output can be confusing and incomplete.
> > 
> > This patch adds a new attribute to just smaps to show
> > device memory usage.  
> 
> If we are any way adding a new entry here then why not one
> more for private device memory pages as well. Just being
> curious.
> 

Well, how do you define visibility of device private memory?
Device private is either seen as swapped out or when migrated
back is visible as a part of the mm. Am I missing anything?

> > 
> > Signed-off-by: Balbir Singh <bsingharora@gmail.com>
> > ---
> >  fs/proc/task_mmu.c | 17 +++++++++++++++--
> >  1 file changed, 15 insertions(+), 2 deletions(-)
> > 
> > diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c
> > index 9f1e2b2b5f5a..b7f32f42ee93 100644
> > --- a/fs/proc/task_mmu.c
> > +++ b/fs/proc/task_mmu.c
> > @@ -451,6 +451,7 @@ struct mem_size_stats {
> >  	unsigned long shared_hugetlb;
> >  	unsigned long private_hugetlb;
> >  	unsigned long first_vma_start;
> > +	unsigned long device_memory;
> >  	u64 pss;
> >  	u64 pss_locked;
> >  	u64 swap_pss;
> > @@ -463,12 +464,22 @@ static void smaps_account(struct mem_size_stats *mss, struct page *page,
> >  	int i, nr = compound ? 1 << compound_order(page) : 1;
> >  	unsigned long size = nr * PAGE_SIZE;
> >  
> > +	/*
> > +	 * We don't want to process public zone device pages further
> > +	 * than just showing how much device memory we have
> > +	 */
> > +	if (is_zone_device_page(page)) {  
> 
> Should not this contain both public and private device pages.
> 

This page is received from _vm_normal_page(.., true), I don't
think device private pages show up here.

> > +		mss->device_memory += size;
> > +		return;
> > +	}
> > +
> >  	if (PageAnon(page)) {
> >  		mss->anonymous += size;
> >  		if (!PageSwapBacked(page) && !dirty && !PageDirty(page))
> >  			mss->lazyfree += size;
> >  	}
> >  
> > +  
> 
> Stray new line.
> 

I can remove it

> >  	mss->resident += size;
> >  	/* Accumulate the size in pages that have been accessed. */
> >  	if (young || page_is_young(page) || PageReferenced(page))
> > @@ -833,7 +844,8 @@ static int show_smap(struct seq_file *m, void *v, int is_pid)
> >  			   "Private_Hugetlb: %7lu kB\n"
> >  			   "Swap:           %8lu kB\n"
> >  			   "SwapPss:        %8lu kB\n"
> > -			   "Locked:         %8lu kB\n",
> > +			   "Locked:         %8lu kB\n"  
> 
> Stray changed line.

?? The line has changed


Thanks for the review!

Balbir Singh.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [rfc 2/2] smaps: Show zone device memory used

[Jerome Glisse]

On Thu, Oct 19, 2017 at 06:48:58AM +1100, Balbir Singh wrote:
> On Wed, 18 Oct 2017 12:40:43 +0530
> Anshuman Khandual <khandual@linux.vnet.ibm.com> wrote:
> 
> > On 10/18/2017 12:01 PM, Balbir Singh wrote:
> > > With HMM, we can have either public or private zone
> > > device pages. With private zone device pages, they should
> > > show up as swapped entities. For public zone device pages  
> > 
> > Might be missing something here but why they should show up
> > as swapped entities ? Could you please elaborate.
> >
> 
> For migrated entries, my use case is to
> 
> 1. malloc()/mmap() memory
> 2. call migrate_vma()
> 3. Look at smaps
> 
> It's probably not clear in the changelog.

My only worry is about API, is smaps consider as userspace API ?
My fear here is that maybe we will want to report device memory
differently in the future and have different category of device
memory. Even thought right now i can only think of wanting to
differentiate between public and private device memory but right
now as you pointed out this is reported as swap out.

Otherwise patches looks good and you got:

Reviewed-by: Jerome Glisse <jglisse@redhat.com>

> 
> > > the smaps output can be confusing and incomplete.
> > > 
> > > This patch adds a new attribute to just smaps to show
> > > device memory usage.  
> > 
> > If we are any way adding a new entry here then why not one
> > more for private device memory pages as well. Just being
> > curious.
> > 
> 
> Well, how do you define visibility of device private memory?
> Device private is either seen as swapped out or when migrated
> back is visible as a part of the mm. Am I missing anything?
> 
> > > 
> > > Signed-off-by: Balbir Singh <bsingharora@gmail.com>
> > > ---
> > >  fs/proc/task_mmu.c | 17 +++++++++++++++--
> > >  1 file changed, 15 insertions(+), 2 deletions(-)
> > > 
> > > diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c
> > > index 9f1e2b2b5f5a..b7f32f42ee93 100644
> > > --- a/fs/proc/task_mmu.c
> > > +++ b/fs/proc/task_mmu.c
> > > @@ -451,6 +451,7 @@ struct mem_size_stats {
> > >  	unsigned long shared_hugetlb;
> > >  	unsigned long private_hugetlb;
> > >  	unsigned long first_vma_start;
> > > +	unsigned long device_memory;
> > >  	u64 pss;
> > >  	u64 pss_locked;
> > >  	u64 swap_pss;
> > > @@ -463,12 +464,22 @@ static void smaps_account(struct mem_size_stats *mss, struct page *page,
> > >  	int i, nr = compound ? 1 << compound_order(page) : 1;
> > >  	unsigned long size = nr * PAGE_SIZE;
> > >  
> > > +	/*
> > > +	 * We don't want to process public zone device pages further
> > > +	 * than just showing how much device memory we have
> > > +	 */
> > > +	if (is_zone_device_page(page)) {  
> > 
> > Should not this contain both public and private device pages.
> > 
> 
> This page is received from _vm_normal_page(.., true), I don't
> think device private pages show up here.
> 
> > > +		mss->device_memory += size;
> > > +		return;
> > > +	}
> > > +
> > >  	if (PageAnon(page)) {
> > >  		mss->anonymous += size;
> > >  		if (!PageSwapBacked(page) && !dirty && !PageDirty(page))
> > >  			mss->lazyfree += size;
> > >  	}
> > >  
> > > +  
> > 
> > Stray new line.
> > 
> 
> I can remove it
> 
> > >  	mss->resident += size;
> > >  	/* Accumulate the size in pages that have been accessed. */
> > >  	if (young || page_is_young(page) || PageReferenced(page))
> > > @@ -833,7 +844,8 @@ static int show_smap(struct seq_file *m, void *v, int is_pid)
> > >  			   "Private_Hugetlb: %7lu kB\n"
> > >  			   "Swap:           %8lu kB\n"
> > >  			   "SwapPss:        %8lu kB\n"
> > > -			   "Locked:         %8lu kB\n",
> > > +			   "Locked:         %8lu kB\n"  
> > 
> > Stray changed line.
> 
> ?? The line has changed
> 
> 
> Thanks for the review!
> 
> Balbir Singh.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [rfc 2/2] smaps: Show zone device memory used

[Michal Hocko]

On Wed 18-10-17 17:31:23, Balbir Singh wrote:
> With HMM, we can have either public or private zone
> device pages. With private zone device pages, they should
> show up as swapped entities. For public zone device pages
> the smaps output can be confusing and incomplete.
> 
> This patch adds a new attribute to just smaps to show
> device memory usage.

As this will become user API which we will have to maintain for ever I
would really like to hear about who is going to use this information and
what for.
-- 
Michal Hocko
SUSE Labs

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [rfc 1/2] mm/hmm: Allow smaps to see zone device public pages

[Michal Hocko]

On Wed 18-10-17 17:31:22, Balbir Singh wrote:
> vm_normal_page() normally does not return zone device public
> pages. In the absence of the visibility the output from smaps
> is limited and confusing. It's hard to figure out where the
> pages are. This patch uses _vm_normal_page() to expose them
> for accounting

Maybe I am missing something but does this patch make any sense without
patch 2? If no why they are not folded into a single one?
-- 
Michal Hocko
SUSE Labs

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [rfc 1/2] mm/hmm: Allow smaps to see zone device public pages

[Balbir Singh]

On Sat, Oct 21, 2017 at 12:11 AM, Michal Hocko <mhocko@suse.com> wrote:
> On Wed 18-10-17 17:31:22, Balbir Singh wrote:
>> vm_normal_page() normally does not return zone device public
>> pages. In the absence of the visibility the output from smaps
>> is limited and confusing. It's hard to figure out where the
>> pages are. This patch uses _vm_normal_page() to expose them
>> for accounting
>
> Maybe I am missing something but does this patch make any sense without
> patch 2? If no why they are not folded into a single one?


I can fold them into one patch. The first patch when applied will just provide
visibility and they'll show as regular resident pages. The second patch
then accounts only for them being device memory.

Balbir Singh.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [rfc 2/2] smaps: Show zone device memory used

[Balbir Singh]

On Sat, Oct 21, 2017 at 12:08 AM, Michal Hocko <mhocko@suse.com> wrote:
> On Wed 18-10-17 17:31:23, Balbir Singh wrote:
>> With HMM, we can have either public or private zone
>> device pages. With private zone device pages, they should
>> show up as swapped entities. For public zone device pages
>> the smaps output can be confusing and incomplete.
>>
>> This patch adds a new attribute to just smaps to show
>> device memory usage.
>
> As this will become user API which we will have to maintain for ever I
> would really like to hear about who is going to use this information and
> what for.

This is something I observed when running some tests with HMM/CDM.
The issue I had was that there was no visibility of what happened to the
pages after the following sequence

1. malloc/mmap pages
2. migrate_vma() to ZONE_DEVICE (hmm/cdm space)
3. look at smaps

If we look at smaps after 1 and the pages are faulted in we can see the
pages for the region, but at point 3, there is absolutely no visibility of
what happened to the pages. I thought smaps is a good way to provide
the visibility as most developers use that interface. It's more to fix the
inconsistency I saw w.r.t visibility and accounting.

Balbir Singh.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [rfc 2/2] smaps: Show zone device memory used

[Balbir Singh]

On Thu, 2017-10-19 at 13:02 -0400, Jerome Glisse wrote:
> On Thu, Oct 19, 2017 at 06:48:58AM +1100, Balbir Singh wrote:
> > On Wed, 18 Oct 2017 12:40:43 +0530
> > Anshuman Khandual <khandual@linux.vnet.ibm.com> wrote:
> > 
> > > On 10/18/2017 12:01 PM, Balbir Singh wrote:
> > > > With HMM, we can have either public or private zone
> > > > device pages. With private zone device pages, they should
> > > > show up as swapped entities. For public zone device pages  
> > > 
> > > Might be missing something here but why they should show up
> > > as swapped entities ? Could you please elaborate.
> > > 
> > 
> > For migrated entries, my use case is to
> > 
> > 1. malloc()/mmap() memory
> > 2. call migrate_vma()
> > 3. Look at smaps
> > 
> > It's probably not clear in the changelog.
> 
> My only worry is about API, is smaps consider as userspace API ?

Yes, do you think choosing DevicePublicMemory would help?

> My fear here is that maybe we will want to report device memory
> differently in the future and have different category of device

You are right, things will change and we'll probably see more things
in ZONE_DEVICE, but I am not sure how they'd show up in smaps or
can't think of it at the moment. The reason for my patch is was
that I expect only device public memory to have a need to be
visible in smaps as we do migration from regular memory to device
public memory and vice-versa.

> memory. Even thought right now i can only think of wanting to
> differentiate between public and private device memory but right
> now as you pointed out this is reported as swap out.
> 
> Otherwise patches looks good and you got:
> 
> Reviewed-by: JA(C)rA'me Glisse <jglisse@redhat.com>

Thank you.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [rfc 2/2] smaps: Show zone device memory used

[Michal Hocko]

On Sat 21-10-17 08:55:59, Balbir Singh wrote:
> On Sat, Oct 21, 2017 at 12:08 AM, Michal Hocko <mhocko@suse.com> wrote:
> > On Wed 18-10-17 17:31:23, Balbir Singh wrote:
> >> With HMM, we can have either public or private zone
> >> device pages. With private zone device pages, they should
> >> show up as swapped entities. For public zone device pages
> >> the smaps output can be confusing and incomplete.
> >>
> >> This patch adds a new attribute to just smaps to show
> >> device memory usage.
> >
> > As this will become user API which we will have to maintain for ever I
> > would really like to hear about who is going to use this information and
> > what for.
> 
> This is something I observed when running some tests with HMM/CDM.
> The issue I had was that there was no visibility of what happened to the
> pages after the following sequence
> 
> 1. malloc/mmap pages
> 2. migrate_vma() to ZONE_DEVICE (hmm/cdm space)
> 3. look at smaps
> 
> If we look at smaps after 1 and the pages are faulted in we can see the
> pages for the region, but at point 3, there is absolutely no visibility of
> what happened to the pages. I thought smaps is a good way to provide
> the visibility as most developers use that interface. It's more to fix the
> inconsistency I saw w.r.t visibility and accounting.

Yes I can see how this can be confusing. But, well, I have grown overly
cautious regarding user APIs over time. So I would rather not add
something new until we have a real user with a usecase in mind. We can
always add this later but once we have exposed the accounting we are
bound to maintain it for ever.
-- 
Michal Hocko
SUSE Labs

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [rfc 1/2] mm/hmm: Allow smaps to see zone device public pages

[Michal Hocko]

On Sat 21-10-17 08:52:27, Balbir Singh wrote:
> On Sat, Oct 21, 2017 at 12:11 AM, Michal Hocko <mhocko@suse.com> wrote:
> > On Wed 18-10-17 17:31:22, Balbir Singh wrote:
> >> vm_normal_page() normally does not return zone device public
> >> pages. In the absence of the visibility the output from smaps
> >> is limited and confusing. It's hard to figure out where the
> >> pages are. This patch uses _vm_normal_page() to expose them
> >> for accounting
> >
> > Maybe I am missing something but does this patch make any sense without
> > patch 2? If no why they are not folded into a single one?
> 
> 
> I can fold them into one patch. The first patch when applied will just provide
> visibility and they'll show as regular resident pages. The second patch
> then accounts only for them being device memory.

Hmm, I am not really sure. It makes some sense to account mapped HMM
pages as RSS but then I am wondering how HMM differs from other special
mappings (like VM_PFNMAP or VM_MIXEDMAP)?
-- 
Michal Hocko
SUSE Labs

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [rfc 2/2] smaps: Show zone device memory used

[Balbir Singh]

On Mon, Oct 23, 2017 at 7:49 PM, Michal Hocko <mhocko@suse.com> wrote:
> On Sat 21-10-17 08:55:59, Balbir Singh wrote:
>> On Sat, Oct 21, 2017 at 12:08 AM, Michal Hocko <mhocko@suse.com> wrote:
>> > On Wed 18-10-17 17:31:23, Balbir Singh wrote:
>> >> With HMM, we can have either public or private zone
>> >> device pages. With private zone device pages, they should
>> >> show up as swapped entities. For public zone device pages
>> >> the smaps output can be confusing and incomplete.
>> >>
>> >> This patch adds a new attribute to just smaps to show
>> >> device memory usage.
>> >
>> > As this will become user API which we will have to maintain for ever I
>> > would really like to hear about who is going to use this information and
>> > what for.
>>
>> This is something I observed when running some tests with HMM/CDM.
>> The issue I had was that there was no visibility of what happened to the
>> pages after the following sequence
>>
>> 1. malloc/mmap pages
>> 2. migrate_vma() to ZONE_DEVICE (hmm/cdm space)
>> 3. look at smaps
>>
>> If we look at smaps after 1 and the pages are faulted in we can see the
>> pages for the region, but at point 3, there is absolutely no visibility of
>> what happened to the pages. I thought smaps is a good way to provide
>> the visibility as most developers use that interface. It's more to fix the
>> inconsistency I saw w.r.t visibility and accounting.
>
> Yes I can see how this can be confusing. But, well, I have grown overly
> cautious regarding user APIs over time. So I would rather not add
> something new until we have a real user with a usecase in mind. We can
> always add this later but once we have exposed the accounting we are
> bound to maintain it for ever.

I see your point. But we are beginning to build on top of this. I'll just add
it as a private patch in my patchset. But soon, we'll need to address HMM/CDM
pages of different sizes as well. My problem right now is to ensure correctness
of the design and expectations and a large part of it is tracking
where the pages
are.

Balbir Singh.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [rfc 2/2] smaps: Show zone device memory used

[Michal Hocko]

On Mon 23-10-17 20:32:25, Balbir Singh wrote:
> On Mon, Oct 23, 2017 at 7:49 PM, Michal Hocko <mhocko@suse.com> wrote:
> > On Sat 21-10-17 08:55:59, Balbir Singh wrote:
> >> On Sat, Oct 21, 2017 at 12:08 AM, Michal Hocko <mhocko@suse.com> wrote:
> >> > On Wed 18-10-17 17:31:23, Balbir Singh wrote:
> >> >> With HMM, we can have either public or private zone
> >> >> device pages. With private zone device pages, they should
> >> >> show up as swapped entities. For public zone device pages
> >> >> the smaps output can be confusing and incomplete.
> >> >>
> >> >> This patch adds a new attribute to just smaps to show
> >> >> device memory usage.
> >> >
> >> > As this will become user API which we will have to maintain for ever I
> >> > would really like to hear about who is going to use this information and
> >> > what for.
> >>
> >> This is something I observed when running some tests with HMM/CDM.
> >> The issue I had was that there was no visibility of what happened to the
> >> pages after the following sequence
> >>
> >> 1. malloc/mmap pages
> >> 2. migrate_vma() to ZONE_DEVICE (hmm/cdm space)
> >> 3. look at smaps
> >>
> >> If we look at smaps after 1 and the pages are faulted in we can see the
> >> pages for the region, but at point 3, there is absolutely no visibility of
> >> what happened to the pages. I thought smaps is a good way to provide
> >> the visibility as most developers use that interface. It's more to fix the
> >> inconsistency I saw w.r.t visibility and accounting.
> >
> > Yes I can see how this can be confusing. But, well, I have grown overly
> > cautious regarding user APIs over time. So I would rather not add
> > something new until we have a real user with a usecase in mind. We can
> > always add this later but once we have exposed the accounting we are
> > bound to maintain it for ever.
> 
> I see your point. But we are beginning to build on top of this. I'll just add
> it as a private patch in my patchset. But soon, we'll need to address HMM/CDM
> pages of different sizes as well. My problem right now is to ensure correctness
> of the design and expectations and a large part of it is tracking
> where the pages are.

I would wait for more experiences to carve a userspace API into stone.
Adding one for debugging purposes is almost _always_ a bad idea that
back fires sooner or later.
-- 
Michal Hocko
SUSE Labs

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [rfc 2/2] smaps: Show zone device memory used

[Balbir Singh]

On Mon, Oct 23, 2017 at 8:36 PM, Michal Hocko <mhocko@suse.com> wrote:
> On Mon 23-10-17 20:32:25, Balbir Singh wrote:
>> On Mon, Oct 23, 2017 at 7:49 PM, Michal Hocko <mhocko@suse.com> wrote:
>> > On Sat 21-10-17 08:55:59, Balbir Singh wrote:
>> >> On Sat, Oct 21, 2017 at 12:08 AM, Michal Hocko <mhocko@suse.com> wrote:
>> >> > On Wed 18-10-17 17:31:23, Balbir Singh wrote:
>> >> >> With HMM, we can have either public or private zone
>> >> >> device pages. With private zone device pages, they should
>> >> >> show up as swapped entities. For public zone device pages
>> >> >> the smaps output can be confusing and incomplete.
>> >> >>
>> >> >> This patch adds a new attribute to just smaps to show
>> >> >> device memory usage.
>> >> >
>> >> > As this will become user API which we will have to maintain for ever I
>> >> > would really like to hear about who is going to use this information and
>> >> > what for.
>> >>
>> >> This is something I observed when running some tests with HMM/CDM.
>> >> The issue I had was that there was no visibility of what happened to the
>> >> pages after the following sequence
>> >>
>> >> 1. malloc/mmap pages
>> >> 2. migrate_vma() to ZONE_DEVICE (hmm/cdm space)
>> >> 3. look at smaps
>> >>
>> >> If we look at smaps after 1 and the pages are faulted in we can see the
>> >> pages for the region, but at point 3, there is absolutely no visibility of
>> >> what happened to the pages. I thought smaps is a good way to provide
>> >> the visibility as most developers use that interface. It's more to fix the
>> >> inconsistency I saw w.r.t visibility and accounting.
>> >
>> > Yes I can see how this can be confusing. But, well, I have grown overly
>> > cautious regarding user APIs over time. So I would rather not add
>> > something new until we have a real user with a usecase in mind. We can
>> > always add this later but once we have exposed the accounting we are
>> > bound to maintain it for ever.
>>
>> I see your point. But we are beginning to build on top of this. I'll just add
>> it as a private patch in my patchset. But soon, we'll need to address HMM/CDM
>> pages of different sizes as well. My problem right now is to ensure correctness
>> of the design and expectations and a large part of it is tracking
>> where the pages are.
>
> I would wait for more experiences to carve a userspace API into stone.
> Adding one for debugging purposes is almost _always_ a bad idea that
> back fires sooner or later.

Fair enough, I am going to keep these in my private branch for now

Balbir Singh.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>