[PATCH 0/4] Memory management patches from Blackfin team

[PATCH 0/4] Memory management patches from Blackfin team

[Bryan Wu]

Hi folks,

Here are some patches related memory managment, especially related
NOMMU code. Could you please review it?

Thanks
-Bryan

[PATCH 1/4] [mm] buddy page allocator: add tunable big order allocation

[Bryan Wu]

From: Michael Hennerich <michael.hennerich@analog.com>

Signed-off-by: Michael Hennerich <michael.hennerich@analog.com>
Signed-off-by: Bryan Wu <cooloney@kernel.org>
---
 init/Kconfig    |    9 +++++++++
 mm/page_alloc.c |    2 +-
 2 files changed, 10 insertions(+), 1 deletions(-)

diff --git a/init/Kconfig b/init/Kconfig
index 6135d07..b6ff75b 100644
--- a/init/Kconfig
+++ b/init/Kconfig
@@ -742,6 +742,15 @@ config SLUB_DEBUG
 	  SLUB sysfs support. /sys/slab will not exist and there will be
 	  no support for cache validation etc.
 
+config BIG_ORDER_ALLOC_NOFAIL_MAGIC
+	int "Big Order Allocation No FAIL Magic"
+	depends on EMBEDDED
+	range 3 10
+	default 3
+	help
+	  Let big-order allocations loop until memory gets free. Specified Value
+	  expresses the order.
+
 choice
 	prompt "Choose SLAB allocator"
 	default SLUB
diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index bdd5c43..71b09b4 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -1631,7 +1631,7 @@ nofail_alloc:
 	pages_reclaimed += did_some_progress;
 	do_retry = 0;
 	if (!(gfp_mask & __GFP_NORETRY)) {
-		if (order <= PAGE_ALLOC_COSTLY_ORDER) {
+		if (order <= CONFIG_BIG_ORDER_ALLOC_NOFAIL_MAGIC) {
 			do_retry = 1;
 		} else {
 			if (gfp_mask & __GFP_REPEAT &&
-- 
1.5.5

[PATCH 2/4] [NOMMU]: include the problematic mapping in the munmap warning

[Bryan Wu]

From: Mike Frysinger <vapier.adi@gmail.com>

Signed-off-by: Mike Frysinger <vapier.adi@gmail.com>
Signed-off-by: Bryan Wu <cooloney@kernel.org>
---
 mm/nommu.c |    5 +++--
 1 files changed, 3 insertions(+), 2 deletions(-)

diff --git a/mm/nommu.c b/mm/nommu.c
index ef8c62c..c11e5cc 100644
--- a/mm/nommu.c
+++ b/mm/nommu.c
@@ -1117,8 +1117,9 @@ int do_munmap(struct mm_struct *mm, unsigned long addr, size_t len)
 			goto found;
 	}
 
-	printk("munmap of non-mmaped memory by process %d (%s): %p\n",
-	       current->pid, current->comm, (void *) addr);
+	printk(KERN_NOTICE "munmap of non-mmaped memory [%p-%p] by process %d (%s)\n",
+	       (void *)addr, (void *)addr+len, current->pid, current->comm);
+
 	return -EINVAL;
 
  found:
-- 
1.5.5

[PATCH 3/4] [mm/nommu]: use copy_to_user_page to call flush icache for [#811] toolchain old bug

[Bryan Wu]

From: Jie Zhang <jie.zhang@analog.com>

access_process_vm in mm/memory.c uses copy_to_user_page and
copy_from_user_page. So for !MMU we'd better do the same thing.
Other archs with mmu do the cache flush in copy_to_user_page.
It gives me hint that copy_to_user_page is designed to flush
the cache. On other side, no archs do the cache flush ptrace.

Signed-off-by: Jie Zhang <jie.zhang@analog.com>
Signed-off-by: Bryan Wu <cooloney@kernel.org>
---
 mm/nommu.c |    6 ++++--
 1 files changed, 4 insertions(+), 2 deletions(-)

diff --git a/mm/nommu.c b/mm/nommu.c
index c11e5cc..56bb447 100644
--- a/mm/nommu.c
+++ b/mm/nommu.c
@@ -1458,9 +1458,11 @@ int access_process_vm(struct task_struct *tsk, unsigned long addr, void *buf, in
 
 		/* only read or write mappings where it is permitted */
 		if (write && vma->vm_flags & VM_MAYWRITE)
-			len -= copy_to_user((void *) addr, buf, len);
+			copy_to_user_page(vma, NULL, NULL,
+					  (void *) addr, buf, len);
 		else if (!write && vma->vm_flags & VM_MAYREAD)
-			len -= copy_from_user(buf, (void *) addr, len);
+			copy_from_user_page(vma, NULL, NULL,
+					    buf, (void *) addr, len);
 		else
 			len = 0;
 	} else {
-- 
1.5.5

[PATCH 4/4] [MM/NOMMU]: Export two symbols in nommu.c for mmap test

[Bryan Wu]

From: Vivi Li <vivi.li@analog.com>

http://blackfin.uclinux.org/gf/project/uclinux-dist/tracker/?action=TrackerItemEdit&tracker_item_id=2312

Signed-off-by: Vivi Li <vivi.li@analog.com>
Signed-off-by: Bryan Wu <cooloney@kernel.org>
---
 mm/nommu.c |    2 ++
 1 files changed, 2 insertions(+), 0 deletions(-)

diff --git a/mm/nommu.c b/mm/nommu.c
index 56bb447..cef9d75 100644
--- a/mm/nommu.c
+++ b/mm/nommu.c
@@ -47,6 +47,8 @@ int heap_stack_gap = 0;
 
 EXPORT_SYMBOL(mem_map);
 EXPORT_SYMBOL(num_physpages);
+EXPORT_SYMBOL(high_memory);
+EXPORT_SYMBOL(max_mapnr);
 
 /* list of shareable VMAs */
 struct rb_root nommu_vma_tree = RB_ROOT;
-- 
1.5.5

Re: [PATCH 4/4] [MM/NOMMU]: Export two symbols in nommu.c for mmap test

[Mike Frysinger]

On Mon, May 12, 2008 at 6:32 AM, Bryan Wu <cooloney@kernel.org> wrote:
> From: Vivi Li <vivi.li@analog.com>
>
> http://blackfin.uclinux.org/gf/project/uclinux-dist/tracker/?action=TrackerItemEdit&tracker_item_id=2312

i dont think URLs to our tracker is a good substitute for log
messages.  our tracker URLs have known to break in the past (due to
gforge changes), and it's a pain for people reading changelogs to open
up a browser just to see what the issue is about.
-mike

Re: [PATCH 4/4] [MM/NOMMU]: Export two symbols in nommu.c for mmap test

[Christoph Hellwig]

On Mon, May 12, 2008 at 06:38:55AM -0400, Mike Frysinger wrote:
> On Mon, May 12, 2008 at 6:32 AM, Bryan Wu <cooloney@kernel.org> wrote:
> > From: Vivi Li <vivi.li@analog.com>
> >
> > http://blackfin.uclinux.org/gf/project/uclinux-dist/tracker/?action=TrackerItemEdit&tracker_item_id=2312
> 
> i dont think URLs to our tracker is a good substitute for log
> messages.  our tracker URLs have known to break in the past (due to
> gforge changes), and it's a pain for people reading changelogs to open
> up a browser just to see what the issue is about.

Yeah, even the url pointer to there is totally unreadable.  I still
don't understand why you want to export this symbols.  Also please make
sure to always submit the code actually using the exports in the same
patchkit, that makes it a lot easier to figure.

Re: [PATCH 4/4] [MM/NOMMU]: Export two symbols in nommu.c for mmap test

[Mike Frysinger]

On Mon, May 12, 2008 at 9:00 AM, Christoph Hellwig <hch@infradead.org> wrote:
> Yeah, even the url pointer to there is totally unreadable.  I still
> don't understand why you want to export this symbols.  Also please make
> sure to always submit the code actually using the exports in the same
> patchkit, that makes it a lot easier to figure.

the symbols are exported for MMU already.  this brings the no-MMU code in line.
-mike

Re: [PATCH 1/4] [mm] buddy page allocator: add tunable big order allocation

[KAMEZAWA Hiroyuki]

On Mon, 12 May 2008 18:32:02 +0800
Bryan Wu <cooloney@kernel.org> wrote:

> From: Michael Hennerich <michael.hennerich@analog.com>
> 
> Signed-off-by: Michael Hennerich <michael.hennerich@analog.com>
> Signed-off-by: Bryan Wu <cooloney@kernel.org>

Does this really solve your problem ? possible hang-up is better than
page allocation failure ?

> ---
>  init/Kconfig    |    9 +++++++++
>  mm/page_alloc.c |    2 +-
>  2 files changed, 10 insertions(+), 1 deletions(-)
> 
> diff --git a/init/Kconfig b/init/Kconfig
> index 6135d07..b6ff75b 100644
> --- a/init/Kconfig
> +++ b/init/Kconfig
> @@ -742,6 +742,15 @@ config SLUB_DEBUG
>  	  SLUB sysfs support. /sys/slab will not exist and there will be
>  	  no support for cache validation etc.
>  
> +config BIG_ORDER_ALLOC_NOFAIL_MAGIC
> +	int "Big Order Allocation No FAIL Magic"
> +	depends on EMBEDDED
> +	range 3 10
> +	default 3
> +	help
> +	  Let big-order allocations loop until memory gets free. Specified Value
> +	  expresses the order.
> +
I think it's better to add a text to explain why this is for EMBEDED.

Thanks,
-Kame

Re: [PATCH 4/4] [MM/NOMMU]: Export two symbols in nommu.c for mmap test

[Bryan Wu]

On Mon, May 12, 2008 at 9:02 PM, Mike Frysinger <vapier.adi@gmail.com> wrote:
> On Mon, May 12, 2008 at 9:00 AM, Christoph Hellwig <hch@infradead.org> wrote:
>  > Yeah, even the url pointer to there is totally unreadable.  I still
>  > don't understand why you want to export this symbols.  Also please make
>  > sure to always submit the code actually using the exports in the same
>  > patchkit, that makes it a lot easier to figure.
>
>  the symbols are exported for MMU already.  this brings the no-MMU code in line.
>  -mike
>

Right, the same symbols are exported in mm/memory.c for MMU arch.
While for NOMMU arch, Vivi (one of our tester) asked for this exported
symbols for some mmap test case.
And maybe they are useful for others.

Sorry for the misleading URL, I add to wrong patches. If you think
this patch is OK, I will rewrite the git log.

Thanks
-Bryan

RE: [PATCH 1/4] [mm] buddy page allocator: add tunable big order allocation

[Hennerich, Michael]

>-----Original Message-----
>From: KAMEZAWA Hiroyuki [mailto:kamezawa.hiroyu@jp.fujitsu.com]
>Sent: Dienstag, 13. Mai 2008 04:09
>To: Bryan Wu
>Cc: linux-kernel@vger.kernel.org; linux-mm@kvack.org;
dwmw2@infradead.org;
>Michael Hennerich
>Subject: Re: [PATCH 1/4] [mm] buddy page allocator: add tunable big
order
>allocation
>
>On Mon, 12 May 2008 18:32:02 +0800
>Bryan Wu <cooloney@kernel.org> wrote:
>
>> From: Michael Hennerich <michael.hennerich@analog.com>
>>
>> Signed-off-by: Michael Hennerich <michael.hennerich@analog.com>
>> Signed-off-by: Bryan Wu <cooloney@kernel.org>
>
>Does this really solve your problem ? possible hang-up is better than
>page allocation failure ?

On nommu this helped quite a bit, when we run out of memory, eaten up by
the page cache. But yes - with this option it's likely that we sit there
and wait form memory that might never get available.

We now use a better workaround for freeing up "available" memory
currently used as page cache.

I think we should drop this patch.

Best regards,
Michael 

>
>> ---
>>  init/Kconfig    |    9 +++++++++
>>  mm/page_alloc.c |    2 +-
>>  2 files changed, 10 insertions(+), 1 deletions(-)
>>
>> diff --git a/init/Kconfig b/init/Kconfig
>> index 6135d07..b6ff75b 100644
>> --- a/init/Kconfig
>> +++ b/init/Kconfig
>> @@ -742,6 +742,15 @@ config SLUB_DEBUG
>>  	  SLUB sysfs support. /sys/slab will not exist and there will be
>>  	  no support for cache validation etc.
>>
>> +config BIG_ORDER_ALLOC_NOFAIL_MAGIC
>> +	int "Big Order Allocation No FAIL Magic"
>> +	depends on EMBEDDED
>> +	range 3 10
>> +	default 3
>> +	help
>> +	  Let big-order allocations loop until memory gets free.
Specified
>Value
>> +	  expresses the order.
>> +
>I think it's better to add a text to explain why this is for EMBEDED.
>
>Thanks,
>-Kame

Re: [PATCH 1/4] [mm] buddy page allocator: add tunable big order allocation

[Bryan Wu]

On Tue, May 13, 2008 at 7:42 PM, Hennerich, Michael
<Michael.Hennerich@analog.com> wrote:
>
>
>  >-----Original Message-----
>  >From: KAMEZAWA Hiroyuki [mailto:kamezawa.hiroyu@jp.fujitsu.com]
>  >Sent: Dienstag, 13. Mai 2008 04:09
>  >To: Bryan Wu
>  >Cc: linux-kernel@vger.kernel.org; linux-mm@kvack.org;
>  dwmw2@infradead.org;
>  >Michael Hennerich
>  >Subject: Re: [PATCH 1/4] [mm] buddy page allocator: add tunable big
>  order
>  >allocation
>  >
>  >On Mon, 12 May 2008 18:32:02 +0800
>  >Bryan Wu <cooloney@kernel.org> wrote:
>  >
>  >> From: Michael Hennerich <michael.hennerich@analog.com>
>  >>
>  >> Signed-off-by: Michael Hennerich <michael.hennerich@analog.com>
>  >> Signed-off-by: Bryan Wu <cooloney@kernel.org>
>  >
>  >Does this really solve your problem ? possible hang-up is better than
>  >page allocation failure ?
>
>  On nommu this helped quite a bit, when we run out of memory, eaten up by
>  the page cache. But yes - with this option it's likely that we sit there
>  and wait form memory that might never get available.
>
>  We now use a better workaround for freeing up "available" memory
>  currently used as page cache.
>
>  I think we should drop this patch.
>

OK, I dropped it. And do you think the limited page_cache patch is the
replacement of this patch?

-Bryan

Re: [PATCH 1/4] [mm] buddy page allocator: add tunable big order allocation

[KAMEZAWA Hiroyuki]

On Wed, 14 May 2008 12:22:35 +0800
"Bryan Wu" <cooloney@kernel.org> wrote:

> On Tue, May 13, 2008 at 7:42 PM, Hennerich, Michael
> <Michael.Hennerich@analog.com> wrote:
> >
> >
> >  >-----Original Message-----
> >  >From: KAMEZAWA Hiroyuki [mailto:kamezawa.hiroyu@jp.fujitsu.com]
> >  >Sent: Dienstag, 13. Mai 2008 04:09
> >  >To: Bryan Wu
> >  >Cc: linux-kernel@vger.kernel.org; linux-mm@kvack.org;
> >  dwmw2@infradead.org;
> >  >Michael Hennerich
> >  >Subject: Re: [PATCH 1/4] [mm] buddy page allocator: add tunable big
> >  order
> >  >allocation
> >  >
> >  >On Mon, 12 May 2008 18:32:02 +0800
> >  >Bryan Wu <cooloney@kernel.org> wrote:
> >  >
> >  >> From: Michael Hennerich <michael.hennerich@analog.com>
> >  >>
> >  >> Signed-off-by: Michael Hennerich <michael.hennerich@analog.com>
> >  >> Signed-off-by: Bryan Wu <cooloney@kernel.org>
> >  >
> >  >Does this really solve your problem ? possible hang-up is better than
> >  >page allocation failure ?
> >
> >  On nommu this helped quite a bit, when we run out of memory, eaten up by
> >  the page cache. But yes - with this option it's likely that we sit there
> >  and wait form memory that might never get available.
> >
> >  We now use a better workaround for freeing up "available" memory
> >  currently used as page cache.
> >
> >  I think we should drop this patch.
> >
> 
> OK, I dropped it. And do you think the limited page_cache patch is the
> replacement of this patch?
> 

I'm not so familiar with nommu environments but have some thoughts.

one idea is 
 - use memory resource controller.
   but this eats much amount of GFP_KERNEL memory and maybe not useful ;)
 - use ZONE_MOVABLE and set lowmem_reserve_ratio value to be suitable value.
   then, the page cache just uses MOVABLE zone...(maybe)

Thanks,
-Kame

Re: [PATCH 3/4] [mm/nommu]: use copy_to_user_page to call flush icache for [#811] toolchain old bug

[Bryan Wu]

Is this patch OK for the nommu, David?

Thanks
-Bryan

On Mon, May 12, 2008 at 6:32 PM, Bryan Wu <cooloney@kernel.org> wrote:
> From: Jie Zhang <jie.zhang@analog.com>
>
> access_process_vm in mm/memory.c uses copy_to_user_page and
> copy_from_user_page. So for !MMU we'd better do the same thing.
> Other archs with mmu do the cache flush in copy_to_user_page.
> It gives me hint that copy_to_user_page is designed to flush
> the cache. On other side, no archs do the cache flush ptrace.
>
> Signed-off-by: Jie Zhang <jie.zhang@analog.com>
> Signed-off-by: Bryan Wu <cooloney@kernel.org>
> ---
>  mm/nommu.c |    6 ++++--
>  1 files changed, 4 insertions(+), 2 deletions(-)
>
> diff --git a/mm/nommu.c b/mm/nommu.c
> index c11e5cc..56bb447 100644
> --- a/mm/nommu.c
> +++ b/mm/nommu.c
> @@ -1458,9 +1458,11 @@ int access_process_vm(struct task_struct *tsk, unsigned long addr, void *buf, in
>
>                /* only read or write mappings where it is permitted */
>                if (write && vma->vm_flags & VM_MAYWRITE)
> -                       len -= copy_to_user((void *) addr, buf, len);
> +                       copy_to_user_page(vma, NULL, NULL,
> +                                         (void *) addr, buf, len);
>                else if (!write && vma->vm_flags & VM_MAYREAD)
> -                       len -= copy_from_user(buf, (void *) addr, len);
> +                       copy_from_user_page(vma, NULL, NULL,
> +                                           buf, (void *) addr, len);
>                else
>                        len = 0;
>        } else {
> --
> 1.5.5
>
>

Re: [PATCH 2/4] [NOMMU]: include the problematic mapping in the munmap warning

[Bryan Wu]

IMO, this is useful for nommu stuff.
Is there any chance to merge this?

-Bryan

On Mon, May 12, 2008 at 6:32 PM, Bryan Wu <cooloney@kernel.org> wrote:
> From: Mike Frysinger <vapier.adi@gmail.com>
>
> Signed-off-by: Mike Frysinger <vapier.adi@gmail.com>
> Signed-off-by: Bryan Wu <cooloney@kernel.org>
> ---
>  mm/nommu.c |    5 +++--
>  1 files changed, 3 insertions(+), 2 deletions(-)
>
> diff --git a/mm/nommu.c b/mm/nommu.c
> index ef8c62c..c11e5cc 100644
> --- a/mm/nommu.c
> +++ b/mm/nommu.c
> @@ -1117,8 +1117,9 @@ int do_munmap(struct mm_struct *mm, unsigned long addr, size_t len)
>                        goto found;
>        }
>
> -       printk("munmap of non-mmaped memory by process %d (%s): %p\n",
> -              current->pid, current->comm, (void *) addr);
> +       printk(KERN_NOTICE "munmap of non-mmaped memory [%p-%p] by process %d (%s)\n",
> +              (void *)addr, (void *)addr+len, current->pid, current->comm);
> +
>        return -EINVAL;
>
>  found:
> --
> 1.5.5
>
>