[RFC][PATCH RESEND] mm: vmalloc: remove ioremap align constraint

[RFC][PATCH RESEND] mm: vmalloc: remove ioremap align constraint

[Dmitry Safonov]

ioremap uses __get_vm_area_node which sets alignment to fls of requested size.
I couldn't find any reason for such big align. Does it decrease TLB misses?
I tested it on custom ARM board with 200+ Mb of ioremap and it works.
What am I missing?

Alignment restriction for ioremap region was introduced with the commit:

> Author: James Bottomley <jejb@mulgrave.(none)>
> Date:   Wed Jun 30 11:11:14 2004 -0500
> 
>     Add vmalloc alignment constraints
> 
>     vmalloc is used by ioremap() to get regions for
>     remapping I/O space.  To feed these regions back
>     into a __get_free_pages() type memory allocator,
>     they are expected to have more alignment than
>     get_vm_area() proves.  So add additional alignment
>     constraints for VM_IOREMAP.
> 
>     Signed-off-by: James Bottomley <James.Bottomley@SteelEye.com>

Cc: Russell King <linux@arm.linux.org.uk>
Cc: Guan Xuetao <gxt@mprc.pku.edu.cn>
Cc: Nicolas Pitre <nicolas.pitre@linaro.org>
Cc: James Bottomley <JBottomley@parallels.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Arnd Bergmann <arnd.bergmann@linaro.org>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Dyasly Sergey <s.dyasly@samsung.com>
Signed-off-by: Dmitry Safonov <d.safonov@partner.samsung.com>
---
 arch/arm/include/asm/memory.h       | 5 -----
 arch/unicore32/include/asm/memory.h | 5 -----
 include/linux/vmalloc.h             | 8 --------
 mm/vmalloc.c                        | 2 --
 4 files changed, 20 deletions(-)

diff --git a/arch/arm/include/asm/memory.h b/arch/arm/include/asm/memory.h
index 184def0..b333245 100644
--- a/arch/arm/include/asm/memory.h
+++ b/arch/arm/include/asm/memory.h
@@ -78,11 +78,6 @@
  */
 #define XIP_VIRT_ADDR(physaddr)  (MODULES_VADDR + ((physaddr) & 0x000fffff))
 
-/*
- * Allow 16MB-aligned ioremap pages
- */
-#define IOREMAP_MAX_ORDER	24
-
 #else /* CONFIG_MMU */
 
 /*
diff --git a/arch/unicore32/include/asm/memory.h b/arch/unicore32/include/asm/memory.h
index debafc4..ffae189 100644
--- a/arch/unicore32/include/asm/memory.h
+++ b/arch/unicore32/include/asm/memory.h
@@ -46,11 +46,6 @@
 #define MODULES_END		(PAGE_OFFSET)
 
 /*
- * Allow 16MB-aligned ioremap pages
- */
-#define IOREMAP_MAX_ORDER	24
-
-/*
  * Physical vs virtual RAM address space conversion.  These are
  * private definitions which should NOT be used outside memory.h
  * files.  Use virt_to_phys/phys_to_virt/__pa/__va instead.
diff --git a/include/linux/vmalloc.h b/include/linux/vmalloc.h
index b87696f..2f428e8 100644
--- a/include/linux/vmalloc.h
+++ b/include/linux/vmalloc.h
@@ -18,14 +18,6 @@ struct vm_area_struct;		/* vma defining user mapping in mm_types.h */
 #define VM_UNINITIALIZED	0x00000020	/* vm_struct is not fully initialized */
 /* bits [20..32] reserved for arch specific ioremap internals */
 
-/*
- * Maximum alignment for ioremap() regions.
- * Can be overriden by arch-specific value.
- */
-#ifndef IOREMAP_MAX_ORDER
-#define IOREMAP_MAX_ORDER	(7 + PAGE_SHIFT)	/* 128 pages */
-#endif
-
 struct vm_struct {
 	struct vm_struct	*next;
 	void			*addr;
diff --git a/mm/vmalloc.c b/mm/vmalloc.c
index 39c3388..c4f480dd 100644
--- a/mm/vmalloc.c
+++ b/mm/vmalloc.c
@@ -1313,8 +1313,6 @@ static struct vm_struct *__get_vm_area_node(unsigned long size,
 	struct vm_struct *area;
 
 	BUG_ON(in_interrupt());
-	if (flags & VM_IOREMAP)
-		align = 1ul << clamp(fls(size), PAGE_SHIFT, IOREMAP_MAX_ORDER);
 
 	size = PAGE_ALIGN(size);
 	if (unlikely(!size))
-- 
1.9.1

[RFC][PATCH RESEND] mm: vmalloc: remove ioremap align constraint

[Dmitry Safonov]

ioremap uses __get_vm_area_node which sets alignment to fls of requested size.
I couldn't find any reason for such big align. Does it decrease TLB misses?
I tested it on custom ARM board with 200+ Mb of ioremap and it works.
What am I missing?

Alignment restriction for ioremap region was introduced with the commit:

> Author: James Bottomley <jejb@mulgrave.(none)>
> Date:   Wed Jun 30 11:11:14 2004 -0500
> 
>     Add vmalloc alignment constraints
> 
>     vmalloc is used by ioremap() to get regions for
>     remapping I/O space.  To feed these regions back
>     into a __get_free_pages() type memory allocator,
>     they are expected to have more alignment than
>     get_vm_area() proves.  So add additional alignment
>     constraints for VM_IOREMAP.
> 
>     Signed-off-by: James Bottomley <James.Bottomley@SteelEye.com>

Cc: Russell King <linux@arm.linux.org.uk>
Cc: Guan Xuetao <gxt@mprc.pku.edu.cn>
Cc: Nicolas Pitre <nicolas.pitre@linaro.org>
Cc: James Bottomley <JBottomley@parallels.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Arnd Bergmann <arnd.bergmann@linaro.org>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Dyasly Sergey <s.dyasly@samsung.com>
Signed-off-by: Dmitry Safonov <d.safonov@partner.samsung.com>
---
 arch/arm/include/asm/memory.h       | 5 -----
 arch/unicore32/include/asm/memory.h | 5 -----
 include/linux/vmalloc.h             | 8 --------
 mm/vmalloc.c                        | 2 --
 4 files changed, 20 deletions(-)

diff --git a/arch/arm/include/asm/memory.h b/arch/arm/include/asm/memory.h
index 184def0..b333245 100644
--- a/arch/arm/include/asm/memory.h
+++ b/arch/arm/include/asm/memory.h
@@ -78,11 +78,6 @@
  */
 #define XIP_VIRT_ADDR(physaddr)  (MODULES_VADDR + ((physaddr) & 0x000fffff))
 
-/*
- * Allow 16MB-aligned ioremap pages
- */
-#define IOREMAP_MAX_ORDER	24
-
 #else /* CONFIG_MMU */
 
 /*
diff --git a/arch/unicore32/include/asm/memory.h b/arch/unicore32/include/asm/memory.h
index debafc4..ffae189 100644
--- a/arch/unicore32/include/asm/memory.h
+++ b/arch/unicore32/include/asm/memory.h
@@ -46,11 +46,6 @@
 #define MODULES_END		(PAGE_OFFSET)
 
 /*
- * Allow 16MB-aligned ioremap pages
- */
-#define IOREMAP_MAX_ORDER	24
-
-/*
  * Physical vs virtual RAM address space conversion.  These are
  * private definitions which should NOT be used outside memory.h
  * files.  Use virt_to_phys/phys_to_virt/__pa/__va instead.
diff --git a/include/linux/vmalloc.h b/include/linux/vmalloc.h
index b87696f..2f428e8 100644
--- a/include/linux/vmalloc.h
+++ b/include/linux/vmalloc.h
@@ -18,14 +18,6 @@ struct vm_area_struct;		/* vma defining user mapping in mm_types.h */
 #define VM_UNINITIALIZED	0x00000020	/* vm_struct is not fully initialized */
 /* bits [20..32] reserved for arch specific ioremap internals */
 
-/*
- * Maximum alignment for ioremap() regions.
- * Can be overriden by arch-specific value.
- */
-#ifndef IOREMAP_MAX_ORDER
-#define IOREMAP_MAX_ORDER	(7 + PAGE_SHIFT)	/* 128 pages */
-#endif
-
 struct vm_struct {
 	struct vm_struct	*next;
 	void			*addr;
diff --git a/mm/vmalloc.c b/mm/vmalloc.c
index 39c3388..c4f480dd 100644
--- a/mm/vmalloc.c
+++ b/mm/vmalloc.c
@@ -1313,8 +1313,6 @@ static struct vm_struct *__get_vm_area_node(unsigned long size,
 	struct vm_struct *area;
 
 	BUG_ON(in_interrupt());
-	if (flags & VM_IOREMAP)
-		align = 1ul << clamp(fls(size), PAGE_SHIFT, IOREMAP_MAX_ORDER);
 
 	size = PAGE_ALIGN(size);
 	if (unlikely(!size))
-- 
1.9.1

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

[RFC][PATCH RESEND] mm: vmalloc: remove ioremap align constraint

[Dmitry Safonov]

ioremap uses __get_vm_area_node which sets alignment to fls of requested size.
I couldn't find any reason for such big align. Does it decrease TLB misses?
I tested it on custom ARM board with 200+ Mb of ioremap and it works.
What am I missing?

Alignment restriction for ioremap region was introduced with the commit:

> Author: James Bottomley <jejb@mulgrave.(none)>
> Date:   Wed Jun 30 11:11:14 2004 -0500
> 
>     Add vmalloc alignment constraints
> 
>     vmalloc is used by ioremap() to get regions for
>     remapping I/O space.  To feed these regions back
>     into a __get_free_pages() type memory allocator,
>     they are expected to have more alignment than
>     get_vm_area() proves.  So add additional alignment
>     constraints for VM_IOREMAP.
> 
>     Signed-off-by: James Bottomley <James.Bottomley@SteelEye.com>

Cc: Russell King <linux@arm.linux.org.uk>
Cc: Guan Xuetao <gxt@mprc.pku.edu.cn>
Cc: Nicolas Pitre <nicolas.pitre@linaro.org>
Cc: James Bottomley <JBottomley@parallels.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Arnd Bergmann <arnd.bergmann@linaro.org>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Dyasly Sergey <s.dyasly@samsung.com>
Signed-off-by: Dmitry Safonov <d.safonov@partner.samsung.com>
---
 arch/arm/include/asm/memory.h       | 5 -----
 arch/unicore32/include/asm/memory.h | 5 -----
 include/linux/vmalloc.h             | 8 --------
 mm/vmalloc.c                        | 2 --
 4 files changed, 20 deletions(-)

diff --git a/arch/arm/include/asm/memory.h b/arch/arm/include/asm/memory.h
index 184def0..b333245 100644
--- a/arch/arm/include/asm/memory.h
+++ b/arch/arm/include/asm/memory.h
@@ -78,11 +78,6 @@
  */
 #define XIP_VIRT_ADDR(physaddr)  (MODULES_VADDR + ((physaddr) & 0x000fffff))
 
-/*
- * Allow 16MB-aligned ioremap pages
- */
-#define IOREMAP_MAX_ORDER	24
-
 #else /* CONFIG_MMU */
 
 /*
diff --git a/arch/unicore32/include/asm/memory.h b/arch/unicore32/include/asm/memory.h
index debafc4..ffae189 100644
--- a/arch/unicore32/include/asm/memory.h
+++ b/arch/unicore32/include/asm/memory.h
@@ -46,11 +46,6 @@
 #define MODULES_END		(PAGE_OFFSET)
 
 /*
- * Allow 16MB-aligned ioremap pages
- */
-#define IOREMAP_MAX_ORDER	24
-
-/*
  * Physical vs virtual RAM address space conversion.  These are
  * private definitions which should NOT be used outside memory.h
  * files.  Use virt_to_phys/phys_to_virt/__pa/__va instead.
diff --git a/include/linux/vmalloc.h b/include/linux/vmalloc.h
index b87696f..2f428e8 100644
--- a/include/linux/vmalloc.h
+++ b/include/linux/vmalloc.h
@@ -18,14 +18,6 @@ struct vm_area_struct;		/* vma defining user mapping in mm_types.h */
 #define VM_UNINITIALIZED	0x00000020	/* vm_struct is not fully initialized */
 /* bits [20..32] reserved for arch specific ioremap internals */
 
-/*
- * Maximum alignment for ioremap() regions.
- * Can be overriden by arch-specific value.
- */
-#ifndef IOREMAP_MAX_ORDER
-#define IOREMAP_MAX_ORDER	(7 + PAGE_SHIFT)	/* 128 pages */
-#endif
-
 struct vm_struct {
 	struct vm_struct	*next;
 	void			*addr;
diff --git a/mm/vmalloc.c b/mm/vmalloc.c
index 39c3388..c4f480dd 100644
--- a/mm/vmalloc.c
+++ b/mm/vmalloc.c
@@ -1313,8 +1313,6 @@ static struct vm_struct *__get_vm_area_node(unsigned long size,
 	struct vm_struct *area;
 
 	BUG_ON(in_interrupt());
-	if (flags & VM_IOREMAP)
-		align = 1ul << clamp(fls(size), PAGE_SHIFT, IOREMAP_MAX_ORDER);
 
 	size = PAGE_ALIGN(size);
 	if (unlikely(!size))
-- 
1.9.1

Re: [RFC][PATCH RESEND] mm: vmalloc: remove ioremap align constraint

[Arnd Bergmann]

On Tuesday 23 December 2014 13:00:13 Dmitry Safonov wrote:
> ioremap uses __get_vm_area_node which sets alignment to fls of requested size.
> I couldn't find any reason for such big align. Does it decrease TLB misses?
> I tested it on custom ARM board with 200+ Mb of ioremap and it works.
> What am I missing?

The alignment was originally introduced in this commit:

commit ff0daca525dde796382b9ccd563f169df2571211
Author: Russell King <rmk@dyn-67.arm.linux.org.uk>
Date:   Thu Jun 29 20:17:15 2006 +0100

    [ARM] Add section support to ioremap
    
    Allow section mappings to be setup using ioremap() and torn down
    with iounmap().  This requires additional support in the MM
    context switch to ensure that mappings are properly synchronised
    when mapped in.
    
    Based an original implementation by Deepak Saxena, reworked and
    ARMv6 support added by rmk.
    
    Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>

and then later extended to 16MB supersection mappings, which indeed
is used to reduce TLB pressure.

I don't see any downsides to it, why change it?

	Arnd

Re: [RFC][PATCH RESEND] mm: vmalloc: remove ioremap align constraint

[Arnd Bergmann]

On Tuesday 23 December 2014 13:00:13 Dmitry Safonov wrote:
> ioremap uses __get_vm_area_node which sets alignment to fls of requested size.
> I couldn't find any reason for such big align. Does it decrease TLB misses?
> I tested it on custom ARM board with 200+ Mb of ioremap and it works.
> What am I missing?

The alignment was originally introduced in this commit:

commit ff0daca525dde796382b9ccd563f169df2571211
Author: Russell King <rmk@dyn-67.arm.linux.org.uk>
Date:   Thu Jun 29 20:17:15 2006 +0100

    [ARM] Add section support to ioremap
    
    Allow section mappings to be setup using ioremap() and torn down
    with iounmap().  This requires additional support in the MM
    context switch to ensure that mappings are properly synchronised
    when mapped in.
    
    Based an original implementation by Deepak Saxena, reworked and
    ARMv6 support added by rmk.
    
    Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>

and then later extended to 16MB supersection mappings, which indeed
is used to reduce TLB pressure.

I don't see any downsides to it, why change it?

	Arnd

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

[RFC][PATCH RESEND] mm: vmalloc: remove ioremap align constraint

[Arnd Bergmann]

On Tuesday 23 December 2014 13:00:13 Dmitry Safonov wrote:
> ioremap uses __get_vm_area_node which sets alignment to fls of requested size.
> I couldn't find any reason for such big align. Does it decrease TLB misses?
> I tested it on custom ARM board with 200+ Mb of ioremap and it works.
> What am I missing?

The alignment was originally introduced in this commit:

commit ff0daca525dde796382b9ccd563f169df2571211
Author: Russell King <rmk@dyn-67.arm.linux.org.uk>
Date:   Thu Jun 29 20:17:15 2006 +0100

    [ARM] Add section support to ioremap
    
    Allow section mappings to be setup using ioremap() and torn down
    with iounmap().  This requires additional support in the MM
    context switch to ensure that mappings are properly synchronised
    when mapped in.
    
    Based an original implementation by Deepak Saxena, reworked and
    ARMv6 support added by rmk.
    
    Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>

and then later extended to 16MB supersection mappings, which indeed
is used to reduce TLB pressure.

I don't see any downsides to it, why change it?

	Arnd

Re: [RFC][PATCH RESEND] mm: vmalloc: remove ioremap align constraint

[Sergey Dyasly]

Hi Arnd,

First, some background information. We originally encountered high fragmentation
issue in vmalloc area:

	1. Total size of vmalloc area was 400 MB.
	2. 200 MB of vmalloc area was consumed by ioremaps of various sizes.
	3. Largest contiguous chunk of vmalloc area was 12 MB.
	4. ioremap of 10 MB failed due to 8 MB alignment requirement.

It was decided to further increase the size of vmalloc area to resolve the above
issue. And I don't like that solution because it decreases the amount of lowmem.

Now let's see how ioremap uses supersections. Judging from current implementation
of __arm_ioremap_pfn_caller:

	#if !defined(CONFIG_SMP) && !defined(CONFIG_ARM_LPAE)
		if (pfn >= 0x100000 && !((paddr | size | addr) & ~SUPERSECTION_MASK)) {
			remap_area_supersections();
		} else if (!((paddr | size | addr) & ~PMD_MASK)) {
			remap_area_sections();
		} else
	#endif
			err = ioremap_page_range();

supersections and sections mappings are used only in !SMP && !LPAE case.
Otherwise, mapping is created using the usual 4K pages (and we are using SMP).
The suggested patch removes alignment requirements for ioremap but it means that
sections will not be used in !SMP case. So another solution is required.

__get_vm_area_node has align parameter, maybe it can be used to specify the
required alignment of ioremap operation? Because I find current generic fls
algorithm to be very restrictive in cases when it's not necessary to use such
a big alignment.


On Tue, 23 Dec 2014 21:58:49 +0100
Arnd Bergmann <arnd@arndb.de> wrote:

> On Tuesday 23 December 2014 13:00:13 Dmitry Safonov wrote:
> > ioremap uses __get_vm_area_node which sets alignment to fls of requested size.
> > I couldn't find any reason for such big align. Does it decrease TLB misses?
> > I tested it on custom ARM board with 200+ Mb of ioremap and it works.
> > What am I missing?
> 
> The alignment was originally introduced in this commit:
> 
> commit ff0daca525dde796382b9ccd563f169df2571211
> Author: Russell King <rmk@dyn-67.arm.linux.org.uk>
> Date:   Thu Jun 29 20:17:15 2006 +0100
> 
>     [ARM] Add section support to ioremap
>     
>     Allow section mappings to be setup using ioremap() and torn down
>     with iounmap().  This requires additional support in the MM
>     context switch to ensure that mappings are properly synchronised
>     when mapped in.
>     
>     Based an original implementation by Deepak Saxena, reworked and
>     ARMv6 support added by rmk.
>     
>     Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
> 
> and then later extended to 16MB supersection mappings, which indeed
> is used to reduce TLB pressure.
> 
> I don't see any downsides to it, why change it?
> 
> 	Arnd

-- 
Sergey Dyasly <dserrg@gmail.com>

Re: [RFC][PATCH RESEND] mm: vmalloc: remove ioremap align constraint

[Sergey Dyasly]

Hi Arnd,

First, some background information. We originally encountered high fragmentation
issue in vmalloc area:

	1. Total size of vmalloc area was 400 MB.
	2. 200 MB of vmalloc area was consumed by ioremaps of various sizes.
	3. Largest contiguous chunk of vmalloc area was 12 MB.
	4. ioremap of 10 MB failed due to 8 MB alignment requirement.

It was decided to further increase the size of vmalloc area to resolve the above
issue. And I don't like that solution because it decreases the amount of lowmem.

Now let's see how ioremap uses supersections. Judging from current implementation
of __arm_ioremap_pfn_caller:

	#if !defined(CONFIG_SMP) && !defined(CONFIG_ARM_LPAE)
		if (pfn >= 0x100000 && !((paddr | size | addr) & ~SUPERSECTION_MASK)) {
			remap_area_supersections();
		} else if (!((paddr | size | addr) & ~PMD_MASK)) {
			remap_area_sections();
		} else
	#endif
			err = ioremap_page_range();

supersections and sections mappings are used only in !SMP && !LPAE case.
Otherwise, mapping is created using the usual 4K pages (and we are using SMP).
The suggested patch removes alignment requirements for ioremap but it means that
sections will not be used in !SMP case. So another solution is required.

__get_vm_area_node has align parameter, maybe it can be used to specify the
required alignment of ioremap operation? Because I find current generic fls
algorithm to be very restrictive in cases when it's not necessary to use such
a big alignment.


On Tue, 23 Dec 2014 21:58:49 +0100
Arnd Bergmann <arnd@arndb.de> wrote:

> On Tuesday 23 December 2014 13:00:13 Dmitry Safonov wrote:
> > ioremap uses __get_vm_area_node which sets alignment to fls of requested size.
> > I couldn't find any reason for such big align. Does it decrease TLB misses?
> > I tested it on custom ARM board with 200+ Mb of ioremap and it works.
> > What am I missing?
> 
> The alignment was originally introduced in this commit:
> 
> commit ff0daca525dde796382b9ccd563f169df2571211
> Author: Russell King <rmk@dyn-67.arm.linux.org.uk>
> Date:   Thu Jun 29 20:17:15 2006 +0100
> 
>     [ARM] Add section support to ioremap
>     
>     Allow section mappings to be setup using ioremap() and torn down
>     with iounmap().  This requires additional support in the MM
>     context switch to ensure that mappings are properly synchronised
>     when mapped in.
>     
>     Based an original implementation by Deepak Saxena, reworked and
>     ARMv6 support added by rmk.
>     
>     Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
> 
> and then later extended to 16MB supersection mappings, which indeed
> is used to reduce TLB pressure.
> 
> I don't see any downsides to it, why change it?
> 
> 	Arnd

-- 
Sergey Dyasly <dserrg@gmail.com>

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

[RFC][PATCH RESEND] mm: vmalloc: remove ioremap align constraint

[Sergey Dyasly]

Hi Arnd,

First, some background information. We originally encountered high fragmentation
issue in vmalloc area:

	1. Total size of vmalloc area was 400 MB.
	2. 200 MB of vmalloc area was consumed by ioremaps of various sizes.
	3. Largest contiguous chunk of vmalloc area was 12 MB.
	4. ioremap of 10 MB failed due to 8 MB alignment requirement.

It was decided to further increase the size of vmalloc area to resolve the above
issue. And I don't like that solution because it decreases the amount of lowmem.

Now let's see how ioremap uses supersections. Judging from current implementation
of __arm_ioremap_pfn_caller:

	#if !defined(CONFIG_SMP) && !defined(CONFIG_ARM_LPAE)
		if (pfn >= 0x100000 && !((paddr | size | addr) & ~SUPERSECTION_MASK)) {
			remap_area_supersections();
		} else if (!((paddr | size | addr) & ~PMD_MASK)) {
			remap_area_sections();
		} else
	#endif
			err = ioremap_page_range();

supersections and sections mappings are used only in !SMP && !LPAE case.
Otherwise, mapping is created using the usual 4K pages (and we are using SMP).
The suggested patch removes alignment requirements for ioremap but it means that
sections will not be used in !SMP case. So another solution is required.

__get_vm_area_node has align parameter, maybe it can be used to specify the
required alignment of ioremap operation? Because I find current generic fls
algorithm to be very restrictive in cases when it's not necessary to use such
a big alignment.


On Tue, 23 Dec 2014 21:58:49 +0100
Arnd Bergmann <arnd@arndb.de> wrote:

> On Tuesday 23 December 2014 13:00:13 Dmitry Safonov wrote:
> > ioremap uses __get_vm_area_node which sets alignment to fls of requested size.
> > I couldn't find any reason for such big align. Does it decrease TLB misses?
> > I tested it on custom ARM board with 200+ Mb of ioremap and it works.
> > What am I missing?
> 
> The alignment was originally introduced in this commit:
> 
> commit ff0daca525dde796382b9ccd563f169df2571211
> Author: Russell King <rmk@dyn-67.arm.linux.org.uk>
> Date:   Thu Jun 29 20:17:15 2006 +0100
> 
>     [ARM] Add section support to ioremap
>     
>     Allow section mappings to be setup using ioremap() and torn down
>     with iounmap().  This requires additional support in the MM
>     context switch to ensure that mappings are properly synchronised
>     when mapped in.
>     
>     Based an original implementation by Deepak Saxena, reworked and
>     ARMv6 support added by rmk.
>     
>     Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
> 
> and then later extended to 16MB supersection mappings, which indeed
> is used to reduce TLB pressure.
> 
> I don't see any downsides to it, why change it?
> 
> 	Arnd

-- 
Sergey Dyasly <dserrg@gmail.com>

Re: [RFC][PATCH RESEND] mm: vmalloc: remove ioremap align constraint

[Arnd Bergmann]

On Saturday 03 January 2015 18:59:46 Sergey Dyasly wrote:
> Hi Arnd,
> 
> First, some background information. We originally encountered high fragmentation
> issue in vmalloc area:
> 
> 	1. Total size of vmalloc area was 400 MB.
> 	2. 200 MB of vmalloc area was consumed by ioremaps of various sizes.
> 	3. Largest contiguous chunk of vmalloc area was 12 MB.
> 	4. ioremap of 10 MB failed due to 8 MB alignment requirement.

Interesting, can you describe how you end up with that many ioremap mappings?
200MB seems like a lot. Do you perhaps get a lot of duplicate entries for the
same hardware registers, or maybe a leak?

Can you send the output of /proc/vmallocinfo?
 
> It was decided to further increase the size of vmalloc area to resolve the above
> issue. And I don't like that solution because it decreases the amount of lowmem.

If all the mappings are in fact required, have you considered using
CONFIG_VMSPLIT_2G split to avoid the use of highmem?

> Now let's see how ioremap uses supersections. Judging from current implementation
> of __arm_ioremap_pfn_caller:
> 
> 	#if !defined(CONFIG_SMP) && !defined(CONFIG_ARM_LPAE)
> 		if (pfn >= 0x100000 && !((paddr | size | addr) & ~SUPERSECTION_MASK)) {
> 			remap_area_supersections();
> 		} else if (!((paddr | size | addr) & ~PMD_MASK)) {
> 			remap_area_sections();
> 		} else
> 	#endif
> 			err = ioremap_page_range();
> 
> supersections and sections mappings are used only in !SMP && !LPAE case.
> Otherwise, mapping is created using the usual 4K pages (and we are using SMP).
> The suggested patch removes alignment requirements for ioremap but it means that
> sections will not be used in !SMP case. So another solution is required.
> 
> __get_vm_area_node has align parameter, maybe it can be used to specify the
> required alignment of ioremap operation? Because I find current generic fls
> algorithm to be very restrictive in cases when it's not necessary to use such
> a big alignment.

I think using next-power-of-two alignment generally helps limit the effects of
fragmentation the same way that the buddy allocator works.

Since the section and supersection maps are only used with non-SMP non-LPAE
(why is that the case btw?), it would however make sense to use the default
(7 + PAGE_SHIFT) instead of the ARM-specific 24 here if one of them is set,
I don't see any downsides to that.

	Arnd

Re: [RFC][PATCH RESEND] mm: vmalloc: remove ioremap align constraint

[Arnd Bergmann]

On Saturday 03 January 2015 18:59:46 Sergey Dyasly wrote:
> Hi Arnd,
> 
> First, some background information. We originally encountered high fragmentation
> issue in vmalloc area:
> 
> 	1. Total size of vmalloc area was 400 MB.
> 	2. 200 MB of vmalloc area was consumed by ioremaps of various sizes.
> 	3. Largest contiguous chunk of vmalloc area was 12 MB.
> 	4. ioremap of 10 MB failed due to 8 MB alignment requirement.

Interesting, can you describe how you end up with that many ioremap mappings?
200MB seems like a lot. Do you perhaps get a lot of duplicate entries for the
same hardware registers, or maybe a leak?

Can you send the output of /proc/vmallocinfo?
 
> It was decided to further increase the size of vmalloc area to resolve the above
> issue. And I don't like that solution because it decreases the amount of lowmem.

If all the mappings are in fact required, have you considered using
CONFIG_VMSPLIT_2G split to avoid the use of highmem?

> Now let's see how ioremap uses supersections. Judging from current implementation
> of __arm_ioremap_pfn_caller:
> 
> 	#if !defined(CONFIG_SMP) && !defined(CONFIG_ARM_LPAE)
> 		if (pfn >= 0x100000 && !((paddr | size | addr) & ~SUPERSECTION_MASK)) {
> 			remap_area_supersections();
> 		} else if (!((paddr | size | addr) & ~PMD_MASK)) {
> 			remap_area_sections();
> 		} else
> 	#endif
> 			err = ioremap_page_range();
> 
> supersections and sections mappings are used only in !SMP && !LPAE case.
> Otherwise, mapping is created using the usual 4K pages (and we are using SMP).
> The suggested patch removes alignment requirements for ioremap but it means that
> sections will not be used in !SMP case. So another solution is required.
> 
> __get_vm_area_node has align parameter, maybe it can be used to specify the
> required alignment of ioremap operation? Because I find current generic fls
> algorithm to be very restrictive in cases when it's not necessary to use such
> a big alignment.

I think using next-power-of-two alignment generally helps limit the effects of
fragmentation the same way that the buddy allocator works.

Since the section and supersection maps are only used with non-SMP non-LPAE
(why is that the case btw?), it would however make sense to use the default
(7 + PAGE_SHIFT) instead of the ARM-specific 24 here if one of them is set,
I don't see any downsides to that.

	Arnd

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

[RFC][PATCH RESEND] mm: vmalloc: remove ioremap align constraint

[Arnd Bergmann]

On Saturday 03 January 2015 18:59:46 Sergey Dyasly wrote:
> Hi Arnd,
> 
> First, some background information. We originally encountered high fragmentation
> issue in vmalloc area:
> 
> 	1. Total size of vmalloc area was 400 MB.
> 	2. 200 MB of vmalloc area was consumed by ioremaps of various sizes.
> 	3. Largest contiguous chunk of vmalloc area was 12 MB.
> 	4. ioremap of 10 MB failed due to 8 MB alignment requirement.

Interesting, can you describe how you end up with that many ioremap mappings?
200MB seems like a lot. Do you perhaps get a lot of duplicate entries for the
same hardware registers, or maybe a leak?

Can you send the output of /proc/vmallocinfo?
 
> It was decided to further increase the size of vmalloc area to resolve the above
> issue. And I don't like that solution because it decreases the amount of lowmem.

If all the mappings are in fact required, have you considered using
CONFIG_VMSPLIT_2G split to avoid the use of highmem?

> Now let's see how ioremap uses supersections. Judging from current implementation
> of __arm_ioremap_pfn_caller:
> 
> 	#if !defined(CONFIG_SMP) && !defined(CONFIG_ARM_LPAE)
> 		if (pfn >= 0x100000 && !((paddr | size | addr) & ~SUPERSECTION_MASK)) {
> 			remap_area_supersections();
> 		} else if (!((paddr | size | addr) & ~PMD_MASK)) {
> 			remap_area_sections();
> 		} else
> 	#endif
> 			err = ioremap_page_range();
> 
> supersections and sections mappings are used only in !SMP && !LPAE case.
> Otherwise, mapping is created using the usual 4K pages (and we are using SMP).
> The suggested patch removes alignment requirements for ioremap but it means that
> sections will not be used in !SMP case. So another solution is required.
> 
> __get_vm_area_node has align parameter, maybe it can be used to specify the
> required alignment of ioremap operation? Because I find current generic fls
> algorithm to be very restrictive in cases when it's not necessary to use such
> a big alignment.

I think using next-power-of-two alignment generally helps limit the effects of
fragmentation the same way that the buddy allocator works.

Since the section and supersection maps are only used with non-SMP non-LPAE
(why is that the case btw?), it would however make sense to use the default
(7 + PAGE_SHIFT) instead of the ARM-specific 24 here if one of them is set,
I don't see any downsides to that.

	Arnd

Re: [RFC][PATCH RESEND] mm: vmalloc: remove ioremap align constraint

[Sergey Dyasly]

On Sun, 04 Jan 2015 17:38:06 +0100
Arnd Bergmann <arnd@arndb.de> wrote:

> On Saturday 03 January 2015 18:59:46 Sergey Dyasly wrote:
> > Hi Arnd,
> > 
> > First, some background information. We originally encountered high fragmentation
> > issue in vmalloc area:
> > 
> > 	1. Total size of vmalloc area was 400 MB.
> > 	2. 200 MB of vmalloc area was consumed by ioremaps of various sizes.
> > 	3. Largest contiguous chunk of vmalloc area was 12 MB.
> > 	4. ioremap of 10 MB failed due to 8 MB alignment requirement.
> 
> Interesting, can you describe how you end up with that many ioremap mappings?
> 200MB seems like a lot. Do you perhaps get a lot of duplicate entries for the
> same hardware registers, or maybe a leak?
> 
> Can you send the output of /proc/vmallocinfo?
>  
> > It was decided to further increase the size of vmalloc area to resolve the above
> > issue. And I don't like that solution because it decreases the amount of lowmem.
> 
> If all the mappings are in fact required, have you considered using
> CONFIG_VMSPLIT_2G split to avoid the use of highmem?
> 
> > Now let's see how ioremap uses supersections. Judging from current implementation
> > of __arm_ioremap_pfn_caller:
> > 
> > 	#if !defined(CONFIG_SMP) && !defined(CONFIG_ARM_LPAE)
> > 		if (pfn >= 0x100000 && !((paddr | size | addr) & ~SUPERSECTION_MASK)) {
> > 			remap_area_supersections();
> > 		} else if (!((paddr | size | addr) & ~PMD_MASK)) {
> > 			remap_area_sections();
> > 		} else
> > 	#endif
> > 			err = ioremap_page_range();
> > 
> > supersections and sections mappings are used only in !SMP && !LPAE case.
> > Otherwise, mapping is created using the usual 4K pages (and we are using SMP).
> > The suggested patch removes alignment requirements for ioremap but it means that
> > sections will not be used in !SMP case. So another solution is required.
> > 
> > __get_vm_area_node has align parameter, maybe it can be used to specify the
> > required alignment of ioremap operation? Because I find current generic fls
> > algorithm to be very restrictive in cases when it's not necessary to use such
> > a big alignment.
> 
> I think using next-power-of-two alignment generally helps limit the effects of
> fragmentation the same way that the buddy allocator works.
> 
> Since the section and supersection maps are only used with non-SMP non-LPAE
> (why is that the case btw?),

vmap/vunmap mechanism works that way. ARM is using 2 levels of page tables:
PGD and PTE; and that provides the needed level of indirection. Every mm
contains a copy of init_mm's pgd mappings for kernel and they point to the same
set of PTEs. vmap/vunmap manipulates only with *pgd->pte and the change becomes
visible to every mm. This is impossible to do for sections because they use
PGD entries directly.

> it would however make sense to use the default
> (7 + PAGE_SHIFT) instead of the ARM-specific 24 here if one of them is set,
> I don't see any downsides to that.

This makes sense. I'll prepare a patch for that.

> 
> 	Arnd


-- 
Sergey Dyasly <s.dyasly@samsung.com>

Re: [RFC][PATCH RESEND] mm: vmalloc: remove ioremap align constraint

[Sergey Dyasly]

On Sun, 04 Jan 2015 17:38:06 +0100
Arnd Bergmann <arnd@arndb.de> wrote:

> On Saturday 03 January 2015 18:59:46 Sergey Dyasly wrote:
> > Hi Arnd,
> > 
> > First, some background information. We originally encountered high fragmentation
> > issue in vmalloc area:
> > 
> > 	1. Total size of vmalloc area was 400 MB.
> > 	2. 200 MB of vmalloc area was consumed by ioremaps of various sizes.
> > 	3. Largest contiguous chunk of vmalloc area was 12 MB.
> > 	4. ioremap of 10 MB failed due to 8 MB alignment requirement.
> 
> Interesting, can you describe how you end up with that many ioremap mappings?
> 200MB seems like a lot. Do you perhaps get a lot of duplicate entries for the
> same hardware registers, or maybe a leak?
> 
> Can you send the output of /proc/vmallocinfo?
>  
> > It was decided to further increase the size of vmalloc area to resolve the above
> > issue. And I don't like that solution because it decreases the amount of lowmem.
> 
> If all the mappings are in fact required, have you considered using
> CONFIG_VMSPLIT_2G split to avoid the use of highmem?
> 
> > Now let's see how ioremap uses supersections. Judging from current implementation
> > of __arm_ioremap_pfn_caller:
> > 
> > 	#if !defined(CONFIG_SMP) && !defined(CONFIG_ARM_LPAE)
> > 		if (pfn >= 0x100000 && !((paddr | size | addr) & ~SUPERSECTION_MASK)) {
> > 			remap_area_supersections();
> > 		} else if (!((paddr | size | addr) & ~PMD_MASK)) {
> > 			remap_area_sections();
> > 		} else
> > 	#endif
> > 			err = ioremap_page_range();
> > 
> > supersections and sections mappings are used only in !SMP && !LPAE case.
> > Otherwise, mapping is created using the usual 4K pages (and we are using SMP).
> > The suggested patch removes alignment requirements for ioremap but it means that
> > sections will not be used in !SMP case. So another solution is required.
> > 
> > __get_vm_area_node has align parameter, maybe it can be used to specify the
> > required alignment of ioremap operation? Because I find current generic fls
> > algorithm to be very restrictive in cases when it's not necessary to use such
> > a big alignment.
> 
> I think using next-power-of-two alignment generally helps limit the effects of
> fragmentation the same way that the buddy allocator works.
> 
> Since the section and supersection maps are only used with non-SMP non-LPAE
> (why is that the case btw?),

vmap/vunmap mechanism works that way. ARM is using 2 levels of page tables:
PGD and PTE; and that provides the needed level of indirection. Every mm
contains a copy of init_mm's pgd mappings for kernel and they point to the same
set of PTEs. vmap/vunmap manipulates only with *pgd->pte and the change becomes
visible to every mm. This is impossible to do for sections because they use
PGD entries directly.

> it would however make sense to use the default
> (7 + PAGE_SHIFT) instead of the ARM-specific 24 here if one of them is set,
> I don't see any downsides to that.

This makes sense. I'll prepare a patch for that.

> 
> 	Arnd


-- 
Sergey Dyasly <s.dyasly@samsung.com>

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

[RFC][PATCH RESEND] mm: vmalloc: remove ioremap align constraint

[Sergey Dyasly]

On Sun, 04 Jan 2015 17:38:06 +0100
Arnd Bergmann <arnd@arndb.de> wrote:

> On Saturday 03 January 2015 18:59:46 Sergey Dyasly wrote:
> > Hi Arnd,
> > 
> > First, some background information. We originally encountered high fragmentation
> > issue in vmalloc area:
> > 
> > 	1. Total size of vmalloc area was 400 MB.
> > 	2. 200 MB of vmalloc area was consumed by ioremaps of various sizes.
> > 	3. Largest contiguous chunk of vmalloc area was 12 MB.
> > 	4. ioremap of 10 MB failed due to 8 MB alignment requirement.
> 
> Interesting, can you describe how you end up with that many ioremap mappings?
> 200MB seems like a lot. Do you perhaps get a lot of duplicate entries for the
> same hardware registers, or maybe a leak?
> 
> Can you send the output of /proc/vmallocinfo?
>  
> > It was decided to further increase the size of vmalloc area to resolve the above
> > issue. And I don't like that solution because it decreases the amount of lowmem.
> 
> If all the mappings are in fact required, have you considered using
> CONFIG_VMSPLIT_2G split to avoid the use of highmem?
> 
> > Now let's see how ioremap uses supersections. Judging from current implementation
> > of __arm_ioremap_pfn_caller:
> > 
> > 	#if !defined(CONFIG_SMP) && !defined(CONFIG_ARM_LPAE)
> > 		if (pfn >= 0x100000 && !((paddr | size | addr) & ~SUPERSECTION_MASK)) {
> > 			remap_area_supersections();
> > 		} else if (!((paddr | size | addr) & ~PMD_MASK)) {
> > 			remap_area_sections();
> > 		} else
> > 	#endif
> > 			err = ioremap_page_range();
> > 
> > supersections and sections mappings are used only in !SMP && !LPAE case.
> > Otherwise, mapping is created using the usual 4K pages (and we are using SMP).
> > The suggested patch removes alignment requirements for ioremap but it means that
> > sections will not be used in !SMP case. So another solution is required.
> > 
> > __get_vm_area_node has align parameter, maybe it can be used to specify the
> > required alignment of ioremap operation? Because I find current generic fls
> > algorithm to be very restrictive in cases when it's not necessary to use such
> > a big alignment.
> 
> I think using next-power-of-two alignment generally helps limit the effects of
> fragmentation the same way that the buddy allocator works.
> 
> Since the section and supersection maps are only used with non-SMP non-LPAE
> (why is that the case btw?),

vmap/vunmap mechanism works that way. ARM is using 2 levels of page tables:
PGD and PTE; and that provides the needed level of indirection. Every mm
contains a copy of init_mm's pgd mappings for kernel and they point to the same
set of PTEs. vmap/vunmap manipulates only with *pgd->pte and the change becomes
visible to every mm. This is impossible to do for sections because they use
PGD entries directly.

> it would however make sense to use the default
> (7 + PAGE_SHIFT) instead of the ARM-specific 24 here if one of them is set,
> I don't see any downsides to that.

This makes sense. I'll prepare a patch for that.

> 
> 	Arnd


-- 
Sergey Dyasly <s.dyasly@samsung.com>