[RFCv1 0/6] PASR: Partial Array Self-Refresh Framework

[RFCv1 0/6] PASR: Partial Array Self-Refresh Framework

[Maxime Coquelin]

PASR Frameworks brings support for the Partial Array Self-Refresh DDR power
management feature. PASR has been introduced in LP-DDR2, and is also present
in DDR3.

PASR provides 4 modes:

* Single-Ended: Only 1/1, 1/2, 1/4 or 1/8 are refreshed, masking starting at
  the end of the DDR die.

* Double-Ended: Same as Single-Ended, but refresh-masking does not start
  necessairly at the end of the DDR die.

* Bank-Selective: Refresh of each bank of a die can be masked or unmasked via
  a dedicated DDR register (MR16). This mode is convenient for DDR configured
  in BRC (Bank-Row-Column) mode.

* Segment-Selective: Refresh of each segment of a die can be masked or unmasked
  via a dedicated DDR register (MR17). This mode is convenient for DDR configured
  in RBC (Row-Bank-Column) mode.

The role of this framework is to stop the refresh of unused memory to enhance
DDR power consumption.

It supports Bank-Selective and Segment-Selective modes, as the more adapted to
modern OSes.

At early boot stage, a representation of the physical DDR layout is built:

             Die 0
_______________________________
| I--------------------------I |
| I    Bank or Segment 0     I |
| I--------------------------I |
| I--------------------------I |
| I    Bank or Segment 1     I |
| I--------------------------I |
| I--------------------------I |
| I    Bank or Segment ...   I |
| I--------------------------I |
| I--------------------------I |
| I    Bank or Segment n     I |
| I--------------------------I |
|______________________________|
             ...

             Die n
_______________________________
| I--------------------------I |
| I    Bank or Segment 0     I |
| I--------------------------I |
| I--------------------------I |
| I    Bank or Segment 1     I |
| I--------------------------I |
| I--------------------------I |
| I    Bank or Segment ...   I |
| I--------------------------I |
| I--------------------------I |
| I    Bank or Segment n     I |
| I--------------------------I |
|______________________________|

The first level is a table where elements represent a die:
* Base address,
* Number of segments,
* Table representing banks/segments,
* MR16/MR17 refresh mask,
* DDR Controller callback to update MR16/MR17 refresh mask.

The second level is the section tables representing the banks or segments,
depending on hardware configuration:
* Base address,
* Unused memory size counter,
* Possible pointer to another section it depends on (E.g. Interleaving)

When some memory becomes unused, the allocator owning this memory calls the PASR
Framework's pasr_put(phys_addr, size) function. The framework finds the
sections impacted and updates their counters accordingly.
If a section counter reach the section size, the refresh of the section is
masked. If the corresponding section has a dependency with another section
(E.g. because of DDR interleaving, see figure below), it checks the "paired" section
is also unused before updating the refresh mask.

When some unused memory is requested by the allocator, the allocator owning
this memory calls the PASR Framework's pasr_get(phys_addr, size) function. The
framework find the section impacted and updates their counters accordingly.
If before the update, the section counter was to the section size, the refrewh
of the section is unmasked. If the corresponding section has a dependency with
another section, it also unmask the refresh of the other section.

Patch 3/6 contains modifications for the Buddy allocator. Overhead induced is
very low because the PASR framework is notified only on "MAX_ORDER" pageblocs.
Any allocator support(PMEM, HWMEM...) and Memory Hotplug would be added in next
patch set revisions.

Maxime Coquelin (6):
  PASR: Initialize DDR layout
  PASR: Add core Framework
  PASR: mm: Integrate PASR in Buddy allocator
  PASR: Call PASR initialization
  PASR: Add Documentation
  PASR: Ux500: Add PASR support

 Documentation/pasr.txt                      |  183 ++++++++++++
 arch/arm/Kconfig                            |    1 +
 arch/arm/kernel/setup.c                     |    1 +
 arch/arm/mach-ux500/include/mach/hardware.h |   11 +
 arch/arm/mach-ux500/include/mach/memory.h   |    8 +
 drivers/mfd/db8500-prcmu.c                  |   67 +++++
 drivers/staging/Kconfig                     |    2 +
 drivers/staging/Makefile                    |    1 +
 drivers/staging/pasr/Kconfig                |   19 ++
 drivers/staging/pasr/Makefile               |    6 +
 drivers/staging/pasr/core.c                 |  168 +++++++++++
 drivers/staging/pasr/helper.c               |   84 ++++++
 drivers/staging/pasr/helper.h               |   16 +
 drivers/staging/pasr/init.c                 |  403 +++++++++++++++++++++++++++
 drivers/staging/pasr/ux500.c                |   58 ++++
 include/linux/pasr.h                        |  143 ++++++++++
 include/linux/ux500-pasr.h                  |   11 +
 init/main.c                                 |    8 +
 mm/page_alloc.c                             |    9 +
 19 files changed, 1199 insertions(+), 0 deletions(-)
 create mode 100644 Documentation/pasr.txt
 create mode 100644 drivers/staging/pasr/Kconfig
 create mode 100644 drivers/staging/pasr/Makefile
 create mode 100644 drivers/staging/pasr/core.c
 create mode 100644 drivers/staging/pasr/helper.c
 create mode 100644 drivers/staging/pasr/helper.h
 create mode 100644 drivers/staging/pasr/init.c
 create mode 100644 drivers/staging/pasr/ux500.c
 create mode 100644 include/linux/pasr.h
 create mode 100644 include/linux/ux500-pasr.h

-- 
1.7.8

[RFCv1 1/6] PASR: Initialize DDR layout

[Maxime Coquelin]

Build the DDR layout representation at early init.

To build the PASR MAP, two parameters are provided:

* ddr_die (mandatory): Should be added for every DDR dies present in the system.
   - Usage: ddr_die=xxx[M|G]@yyy[M|G] where xxx represents the size and yyy
     the base address of the die. E.g.: ddr_die=512M@0 ddr_die=512M@512M

* interleaved (optionnal): Should be added for every interleaved dependencies.
   - Usage: interleaved=xxx[M|G]@yyy[M|G]:zzz[M|G] where xxx is the size of
     the interleaved area between the adresses yyy and zzz. E.g
     interleaved=256M@0:512M

Signed-off-by: Maxime Coquelin <maxime.coquelin@stericsson.com>
---
 drivers/staging/Kconfig       |    2 +
 drivers/staging/Makefile      |    1 +
 drivers/staging/pasr/Kconfig  |   14 ++
 drivers/staging/pasr/Makefile |    4 +
 drivers/staging/pasr/helper.c |   84 +++++++++
 drivers/staging/pasr/helper.h |   16 ++
 drivers/staging/pasr/init.c   |  403 +++++++++++++++++++++++++++++++++++++++++
 include/linux/pasr.h          |   73 ++++++++
 8 files changed, 597 insertions(+), 0 deletions(-)
 create mode 100644 drivers/staging/pasr/Kconfig
 create mode 100644 drivers/staging/pasr/Makefile
 create mode 100644 drivers/staging/pasr/helper.c
 create mode 100644 drivers/staging/pasr/helper.h
 create mode 100644 drivers/staging/pasr/init.c
 create mode 100644 include/linux/pasr.h

diff --git a/drivers/staging/Kconfig b/drivers/staging/Kconfig
index d061318..ddb56aa 100644
--- a/drivers/staging/Kconfig
+++ b/drivers/staging/Kconfig
@@ -182,4 +182,6 @@ source "drivers/staging/nmf-cm/Kconfig"
 
 source "drivers/staging/camera_flash/Kconfig"
 
+source "drivers/staging/pasr/Kconfig"
+
 endif # STAGING
diff --git a/drivers/staging/Makefile b/drivers/staging/Makefile
index f0c7417..20a692d 100644
--- a/drivers/staging/Makefile
+++ b/drivers/staging/Makefile
@@ -5,6 +5,7 @@ obj-$(CONFIG_STAGING)		+= staging.o
 
 obj-y += tty/
 obj-y += generic_serial/
+obj-y += pasr/
 obj-$(CONFIG_ET131X)		+= et131x/
 obj-$(CONFIG_SLICOSS)		+= slicoss/
 obj-$(CONFIG_VIDEO_GO7007)	+= go7007/
diff --git a/drivers/staging/pasr/Kconfig b/drivers/staging/pasr/Kconfig
new file mode 100644
index 0000000..6bd2421
--- /dev/null
+++ b/drivers/staging/pasr/Kconfig
@@ -0,0 +1,14 @@
+config ARCH_HAS_PASR
+	bool
+
+config PASR
+	bool "DDR Partial Array Self-Refresh"
+	depends on ARCH_HAS_PASR
+	---help---
+	  PASR consists on masking the refresh of unused segments or banks
+	  when DDR is in self-refresh state.
+
+config PASR_DEBUG
+	bool "Add PASR debug prints"
+	def_bool n
+	depends on PASR
diff --git a/drivers/staging/pasr/Makefile b/drivers/staging/pasr/Makefile
new file mode 100644
index 0000000..72f7c27
--- /dev/null
+++ b/drivers/staging/pasr/Makefile
@@ -0,0 +1,4 @@
+pasr-objs := helper.o init.o
+obj-$(CONFIG_PASR) += pasr.o
+
+ccflags-$(CONFIG_PASR_DEBUG) := -DDEBUG
diff --git a/drivers/staging/pasr/helper.c b/drivers/staging/pasr/helper.c
new file mode 100644
index 0000000..7c48051
--- /dev/null
+++ b/drivers/staging/pasr/helper.c
@@ -0,0 +1,84 @@
+/*
+ * Copyright (C) ST-Ericsson SA 2011
+ * Author: Maxime Coquelin <maxime.coquelin@stericsson.com> for ST-Ericsson.
+ * License terms:  GNU General Public License (GPL), version 2
+ */
+
+#include <linux/pasr.h>
+
+
+struct pasr_die *pasr_addr2die(struct pasr_map *map, phys_addr_t addr)
+{
+	unsigned int left, right, mid;
+
+	if (!map)
+		return NULL;
+
+	left = 0;
+	right = map->nr_dies;
+
+	addr &= ~(PASR_SECTION_SZ - 1);
+
+	while (left != right) {
+		struct pasr_die *d;
+		phys_addr_t start;
+
+		mid = (left + right) >> 1;
+		d = &map->die[mid];
+		start = addr & ~((PASR_SECTION_SZ * d->nr_sections) - 1);
+
+		if (start == d->start)
+			return d;
+		else if (start > d->start)
+			left = mid;
+		else
+			right = mid;
+	}
+
+	pr_err("%s: No die found for address %#x",
+			__func__, addr);
+	return NULL;
+}
+
+struct pasr_section *pasr_addr2section(struct pasr_map *map
+				, phys_addr_t addr)
+{
+	unsigned int left, right, mid;
+	struct pasr_die *die;
+
+	/* Find the die the address it is located in */
+	die = pasr_addr2die(map, addr);
+	if (!die)
+		goto err;
+
+	left = 0;
+	right = die->nr_sections;
+
+	addr &= ~(PASR_SECTION_SZ - 1);
+
+	 while (left != right) {
+		struct pasr_section *s;
+
+		mid = (left + right) >> 1;
+		s = &die->section[mid];
+
+		if (addr == s->start)
+			return s;
+		else if (addr > s->start)
+			left = mid;
+		else
+			right = mid;
+	}
+
+err:
+	/* Provided address isn't in any declared section */
+	pr_err("%s: No section found for address %#x",
+			__func__, addr);
+
+	return NULL;
+}
+
+phys_addr_t pasr_section2addr(struct pasr_section *s)
+{
+	return s->start;
+}
diff --git a/drivers/staging/pasr/helper.h b/drivers/staging/pasr/helper.h
new file mode 100644
index 0000000..6488f2f
--- /dev/null
+++ b/drivers/staging/pasr/helper.h
@@ -0,0 +1,16 @@
+/*
+ * Copyright (C) ST-Ericsson SA 2011
+ * Author: Maxime Coquelin <maxime.coquelin@stericsson.com> for ST-Ericsson.
+ * License terms:  GNU General Public License (GPL), version 2
+ */
+
+#ifndef _PASR_HELPER_H
+#define _PASR_HELPER_H
+
+#include <linux/pasr.h>
+
+struct pasr_die *pasr_addr2die(struct pasr_map *map, phys_addr_t addr);
+struct pasr_section *pasr_addr2section(struct pasr_map *map, phys_addr_t addr);
+phys_addr_t pasr_section2addr(struct pasr_section *s);
+
+#endif /* _PASR_HELPER_H */
diff --git a/drivers/staging/pasr/init.c b/drivers/staging/pasr/init.c
new file mode 100644
index 0000000..2c7c280
--- /dev/null
+++ b/drivers/staging/pasr/init.c
@@ -0,0 +1,403 @@
+/*
+ * Copyright (C) ST-Ericsson SA 2011
+ * Author: Maxime Coquelin <maxime.coquelin@stericsson.com> for ST-Ericsson.
+ * License terms:  GNU General Public License (GPL), version 2
+ */
+
+#include <linux/mm.h>
+#include <linux/slab.h>
+#include <linux/spinlock.h>
+#include <linux/sort.h>
+#include <linux/pasr.h>
+
+#include "helper.h"
+
+#define NR_DIES 8
+#define NR_INT 8
+
+struct ddr_die {
+	phys_addr_t addr;
+	unsigned long size;
+};
+
+struct interleaved_area {
+	phys_addr_t addr1;
+	phys_addr_t addr2;
+	unsigned long size;
+};
+
+struct pasr_info {
+	int nr_dies;
+	struct ddr_die die[NR_DIES];
+
+	int nr_int;
+	struct interleaved_area int_area[NR_INT];
+};
+
+static struct pasr_info __initdata pasr_info;
+static struct pasr_map pasr_map;
+
+static void add_ddr_die(phys_addr_t addr, unsigned long size);
+static void add_interleaved_area(phys_addr_t a1,
+		phys_addr_t a2, unsigned long size);
+
+static int __init ddr_die_param(char *p)
+{
+	phys_addr_t start;
+	unsigned long size;
+
+	size = memparse(p, &p);
+
+	if (*p != '@')
+		goto err;
+
+	start = memparse(p + 1, &p);
+
+	add_ddr_die(start, size);
+
+	return 0;
+err:
+	return -EINVAL;
+}
+early_param("ddr_die", ddr_die_param);
+
+static int __init interleaved_param(char *p)
+{
+	phys_addr_t start1, start2;
+	unsigned long size;
+
+	size = memparse(p, &p);
+
+	if (*p != '@')
+		goto err;
+
+	start1 = memparse(p + 1, &p);
+
+	if (*p != ':')
+		goto err;
+
+	start2 = memparse(p + 1, &p);
+
+	add_interleaved_area(start1, start2, size);
+
+	return 0;
+err:
+	return -EINVAL;
+}
+early_param("interleaved", interleaved_param);
+
+void __init add_ddr_die(phys_addr_t addr, unsigned long size)
+{
+	BUG_ON(pasr_info.nr_dies >= NR_DIES);
+
+	pasr_info.die[pasr_info.nr_dies].addr = addr;
+	pasr_info.die[pasr_info.nr_dies++].size = size;
+}
+
+void __init add_interleaved_area(phys_addr_t a1, phys_addr_t a2,
+		unsigned long size)
+{
+	BUG_ON(pasr_info.nr_int >= NR_INT);
+
+	pasr_info.int_area[pasr_info.nr_int].addr1 = a1;
+	pasr_info.int_area[pasr_info.nr_int].addr2 = a2;
+	pasr_info.int_area[pasr_info.nr_int++].size = size;
+}
+
+#ifdef DEBUG
+static void __init pasr_print_info(struct pasr_info *info)
+{
+	int i;
+
+	pr_info("PASR information coherent\n");
+
+
+	pr_info("DDR Dies layout:\n");
+	pr_info("\tid - start address - end address\n");
+	for (i = 0; i < info->nr_dies; i++)
+		pr_info("\t- %d : %#08x - %#08x\n",
+			i, (unsigned int)info->die[i].addr,
+			(unsigned int)(info->die[i].addr
+				+ info->die[i].size - 1));
+
+	if (info->nr_int == 0) {
+		pr_info("No interleaved areas declared\n");
+		return;
+	}
+
+	pr_info("Interleaving layout:\n");
+	pr_info("\tid - start @1 - end @2 : start @2 - end @2\n");
+	for (i = 0; i < info->nr_int; i++)
+		pr_info("\t-%d - %#08x - %#08x : %#08x - %#08x\n"
+			, i
+			, (unsigned int)info->int_area[i].addr1
+			, (unsigned int)(info->int_area[i].addr1
+				+ info->int_area[i].size - 1)
+			, (unsigned int)info->int_area[i].addr2
+			, (unsigned int)(info->int_area[i].addr2
+				+ info->int_area[i].size - 1));
+}
+#else
+#define pasr_print_info(info) do {} while (0)
+#endif /* DEBUG */
+
+static int __init is_in_physmem(phys_addr_t addr, struct ddr_die *d)
+{
+	return ((addr >= d->addr) && (addr <= d->addr + d->size));
+}
+
+static int __init pasr_check_interleave_in_physmem(struct pasr_info *info,
+						struct interleaved_area *i)
+{
+	struct ddr_die *d;
+	int j;
+	int err = 4;
+
+	for (j = 0; j < info->nr_dies; j++) {
+		d = &info->die[j];
+		if (is_in_physmem(i->addr1, d))
+			err--;
+		if (is_in_physmem(i->addr1 + i->size, d))
+			err--;
+		if (is_in_physmem(i->addr2, d))
+			err--;
+		if (is_in_physmem(i->addr2 + i->size, d))
+			err--;
+	}
+
+	return err;
+}
+
+static int __init ddrdie_cmp(const void *_a, const void *_b)
+{
+	const struct ddr_die *a = _a, *b = _b;
+
+	return a->addr < b->addr ? -1 : a->addr > b->addr ? 1 : 0;
+}
+
+static int __init interleaved_cmp(const void *_a, const void *_b)
+{
+	const struct interleaved_area *a = _a, *b = _b;
+
+	return a->addr1 < b->addr1 ? -1 : a->addr1 > b->addr1 ? 1 : 0;
+}
+
+static int __init pasr_info_sanity_check(struct pasr_info *info)
+{
+	int i;
+
+	/* Check at least one physical chunk is defined */
+	if (info->nr_dies == 0) {
+		pr_err("%s: No DDR dies declared in command line\n", __func__);
+		return -EINVAL;
+	}
+
+	/* Sort DDR dies areas */
+	sort(&info->die, info->nr_dies,
+			sizeof(info->die[0]), ddrdie_cmp, NULL);
+
+	/* Physical layout checking */
+	for (i = 0; i < info->nr_dies; i++) {
+		struct ddr_die *d1, *d2;
+
+		d1 = &info->die[i];
+
+		if (d1->size == 0) {
+			pr_err("%s: DDR die at %#x has 0 size\n",
+					__func__, d1->addr);
+			return -EINVAL;
+		}
+
+		/*  Check die is aligned on section boundaries */
+		if (((d1->addr & ~(PASR_SECTION_SZ - 1)) != d1->addr)
+			|| ((d1->size & ~(PASR_SECTION_SZ - 1)) != d1->size)) {
+			pr_err("%s: DDR die at %#x (size %#lx) is not aligned"
+					"on section boundaries %#x\n",
+					__func__, d1->addr,
+					d1->size, PASR_SECTION_SZ);
+			return -EINVAL;
+		}
+
+		if (i == 0)
+			continue;
+
+		/* Check areas are not overlapping */
+		d2 = d1;
+		d1 = &info->die[i-1];
+		if ((d1->addr + d1->size - 1) >= d2->addr) {
+			pr_err("%s: DDR dies at %#x and %#x are overlapping\n",
+					__func__, d1->addr, d2->addr);
+			return -EINVAL;
+		}
+	}
+
+	/* Interleave layout checking */
+	if (info->nr_int == 0)
+		goto out;
+
+	/* Sort interleaved areas */
+	sort(&info->int_area, info->nr_int,
+			sizeof(info->int_area[0]), interleaved_cmp, NULL);
+
+	for (i = 0; i < info->nr_int; i++) {
+		struct interleaved_area *i1;
+
+		i1 = &info->int_area[i];
+		if (i1->size == 0) {
+			pr_err("%s: Interleaved area %#x/%#x  has 0 size\n",
+					__func__, i1->addr1, i1->addr2);
+			return -EINVAL;
+		}
+
+		/* Check area is aligned on section boundaries */
+		if (((i1->addr1 & ~(PASR_SECTION_SZ - 1)) != i1->addr1)
+			|| ((i1->addr2 & ~(PASR_SECTION_SZ - 1)) != i1->addr2)
+			|| ((i1->size & ~(PASR_SECTION_SZ - 1)) != i1->size)) {
+			pr_err("%s: Interleaved area at %#x/%#x (size %#lx) is not"
+					"aligned on section boundaries %#x\n",
+					__func__, i1->addr1, i1->addr2,
+					i1->size, PASR_SECTION_SZ);
+			return -EINVAL;
+		}
+
+		/* Check interleaved areas are not overlapping */
+		if ((i1->addr1 + i1->size - 1) >= i1->addr2) {
+			pr_err("%s: Interleaved areas %#x"
+					"and %#x are overlapping\n",
+					__func__, i1->addr1, i1->addr2);
+			return -EINVAL;
+		}
+
+		/* Check the interleaved areas are in the physical areas */
+		if (pasr_check_interleave_in_physmem(info, i1)) {
+			pr_err("%s: Interleaved area %#x/%#x"
+					"not in physical memory\n",
+					__func__, i1->addr1, i1->addr2);
+			return -EINVAL;
+		}
+	}
+
+out:
+	return 0;
+}
+
+#ifdef DEBUG
+static void __init pasr_print_map(struct pasr_map *map)
+{
+	int i, j;
+
+	if (!map)
+		goto out;
+
+	pr_info("PASR map:\n");
+
+	for (i = 0; i < map->nr_dies; i++) {
+		struct pasr_die *die = &map->die[i];
+
+		pr_info("Die %d:\n", i);
+		for (j = 0; j < die->nr_sections; j++) {
+			struct pasr_section *s = &die->section[j];
+			pr_info("\tSection %d: @ = %#08x, Pair = %s\n"
+					, j, s->start, s->pair ? "Yes" : "No");
+		}
+	}
+out:
+	return;
+}
+#else
+#define pasr_print_map(map) do {} while (0)
+#endif /* DEBUG */
+
+static int __init pasr_build_map(struct pasr_info *info, struct pasr_map *map)
+{
+	int i, j;
+	struct pasr_die *die;
+
+	map->nr_dies = info->nr_dies;
+	die = map->die;
+
+	for (i = 0; i < info->nr_dies; i++) {
+		phys_addr_t addr = info->die[i].addr;
+		struct pasr_section *section = die[i].section;
+
+		die[i].start = addr;
+		die[i].idx = i;
+		die[i].nr_sections = info->die[i].size >> PASR_SECTION_SZ_BITS;
+
+		for (j = 0; j < die[i].nr_sections; j++) {
+			section[j].start = addr;
+			addr += PASR_SECTION_SZ;
+			section[j].die = &die[i];
+		}
+	}
+
+	for (i = 0; i < info->nr_int; i++) {
+		struct interleaved_area *ia = &info->int_area[i];
+		struct pasr_section *s1, *s2;
+		unsigned long offset = 0;
+
+		for (j = 0; j < (ia->size >> PASR_SECTION_SZ_BITS); j++) {
+			s1 = pasr_addr2section(map, ia->addr1 + offset);
+			s2 = pasr_addr2section(map, ia->addr2 + offset);
+			if (!s1 || !s2)
+				return -EINVAL;
+
+			offset += PASR_SECTION_SZ;
+
+			s1->pair = s2;
+			s2->pair = s1;
+		}
+	}
+	return 0;
+}
+
+int __init early_pasr_setup(void)
+{
+	int ret;
+
+	ret = pasr_info_sanity_check(&pasr_info);
+	if (ret) {
+		pr_err("PASR info sanity check failed (err %d)\n", ret);
+		return ret;
+	}
+
+	pasr_print_info(&pasr_info);
+
+	ret = pasr_build_map(&pasr_info, &pasr_map);
+	if (ret) {
+		pr_err("PASR build map failed (err %d)\n", ret);
+		return ret;
+	}
+
+	pasr_print_map(&pasr_map);
+
+	ret = pasr_init_core(&pasr_map);
+
+	pr_debug("PASR: First stage init done.\n");
+
+	return ret;
+}
+
+/*
+ * late_pasr_setup() has to be called after Linux allocator is
+ * initialized but before other CPUs are launched.
+ */
+int __init late_pasr_setup(void)
+{
+	int i, j;
+	struct pasr_section *s;
+
+	for_each_pasr_section(i, j, pasr_map, s) {
+		if (!s->lock) {
+			s->lock = kzalloc(sizeof(spinlock_t), GFP_KERNEL);
+			BUG_ON(!s->lock);
+			spin_lock_init(s->lock);
+			if (s->pair)
+				s->pair->lock = s->lock;
+		}
+	}
+
+	pr_debug("PASR Second stage init done.\n");
+
+	return 0;
+}
diff --git a/include/linux/pasr.h b/include/linux/pasr.h
new file mode 100644
index 0000000..93867f0
--- /dev/null
+++ b/include/linux/pasr.h
@@ -0,0 +1,73 @@
+/*
+ * Copyright (C) ST-Ericsson SA 2011
+ * Author: Maxime Coquelin <maxime.coquelin@stericsson.com> for ST-Ericsson.
+ * License terms:  GNU General Public License (GPL), version 2
+ */
+#ifndef _LINUX_PASR_H
+#define _LINUX_PASR_H
+
+#include <linux/mm.h>
+#include <linux/spinlock.h>
+#include <mach/memory.h>
+
+#ifdef CONFIG_PASR
+
+/**
+ * struct pasr_section - Represent either a DDR Bank or Segment depending on
+ * the DDR configuration (Bank-Row-Column or Row-Bank-Coloumn)
+ *
+ * @start: Start address of the segment.
+ * @pair: Pointer on another segment in case of dependency (e.g. interleaving).
+ *	Masking of the dependant segments have to be done accordingly.
+ * @free_size: Represents the free memory size in the segment.
+ * @lock: Protect the free_size counter
+ * @die: Pointer to the Die the segment is part of.
+ */
+struct pasr_section {
+	phys_addr_t start;
+	struct pasr_section *pair;
+	unsigned long free_size;
+	spinlock_t *lock;
+	struct pasr_die *die;
+};
+
+/**
+ * struct pasr_die - Represent a DDR die
+ *
+ * @start: Start address of the die.
+ * @idx: Index of the die.
+ * @nr_sections: Number of Bank or Segment in the die.
+ * @section: Table of the die's segments.
+ * @mem_reg: Represents the PASR mask of the die. It is either MR16 or MR17,
+ *	depending on the addressing configuration (RBC or BRC).
+ * @apply_mask: Callback registred by the platform's PASR driver to apply the
+ *	calculated PASR mask.
+ * @cookie: Private data for the platform's PASR driver.
+ */
+struct pasr_die {
+	phys_addr_t start;
+	int idx;
+	int nr_sections;
+	struct pasr_section section[PASR_MAX_SECTION_NR_PER_DIE];
+};
+
+/**
+ * struct pasr_map - Represent the DDR physical map
+ *
+ * @nr_dies: Number of DDR dies.
+ * @die: Table of the dies.
+ */
+struct pasr_map {
+	int nr_dies;
+	struct pasr_die die[PASR_MAX_DIE_NR];
+};
+
+#define for_each_pasr_section(i, j, map, s) \
+        for (i = 0; i < map.nr_dies; i++) \
+                for (s = &map.die[i].section[0], j = 0; \
+                                j < map.die[i].nr_sections; \
+                                j++, s = &map.die[i].section[j])
+
+#endif /* CONFIG_PASR */
+
+#endif /* _LINUX_PASR_H */
-- 
1.7.8

[RFCv1 2/6] PASR: Add core Framework

[Maxime Coquelin]

This patch introduces the core of the PASR Framework, whose role is to update
sections counters and Self-Refresh masks when sections become free/used.

Signed-off-by: Maxime Coquelin <maxime.coquelin@stericsson.com>
---
 drivers/staging/pasr/Makefile |    3 +-
 drivers/staging/pasr/core.c   |  168 +++++++++++++++++++++++++++++++++++++++++
 include/linux/pasr.h          |   70 +++++++++++++++++
 3 files changed, 240 insertions(+), 1 deletions(-)
 create mode 100644 drivers/staging/pasr/core.c

diff --git a/drivers/staging/pasr/Makefile b/drivers/staging/pasr/Makefile
index 72f7c27..d172294 100644
--- a/drivers/staging/pasr/Makefile
+++ b/drivers/staging/pasr/Makefile
@@ -1,4 +1,5 @@
-pasr-objs := helper.o init.o
+pasr-objs := helper.o init.o core.o
+
 obj-$(CONFIG_PASR) += pasr.o
 
 ccflags-$(CONFIG_PASR_DEBUG) := -DDEBUG
diff --git a/drivers/staging/pasr/core.c b/drivers/staging/pasr/core.c
new file mode 100644
index 0000000..49bacb9
--- /dev/null
+++ b/drivers/staging/pasr/core.c
@@ -0,0 +1,168 @@
+/*
+ * Copyright (C) ST-Ericsson SA 2011
+ * Author: Maxime Coquelin <maxime.coquelin@stericsson.com> for ST-Ericsson.
+ * License terms:  GNU General Public License (GPL), version 2
+ */
+
+#include <linux/mm.h>
+#include <linux/spinlock.h>
+#include <linux/pasr.h>
+
+#include "helper.h"
+
+enum pasr_state {
+	PASR_REFRESH,
+	PASR_NO_REFRESH,
+};
+
+struct pasr_fw {
+	struct pasr_map *map;
+};
+
+static struct pasr_fw pasr;
+
+void pasr_update_mask(struct pasr_section *section, enum pasr_state state)
+{
+	struct pasr_die *die = section->die;
+	phys_addr_t addr = section->start - die->start;
+	u8 bit = addr >> PASR_SECTION_SZ_BITS;
+
+	if (state == PASR_REFRESH)
+		die->mem_reg &= ~(1 << bit);
+	else
+		die->mem_reg |= (1 << bit);
+
+	pr_debug("%s(): %s refresh section 0x%08x. Die%d mem_reg = 0x%02x\n"
+			, __func__, state == PASR_REFRESH ? "Start" : "Stop"
+			, section->start, die->idx, die->mem_reg);
+
+	if (die->apply_mask)
+		die->apply_mask(&die->mem_reg, die->cookie);
+
+	return;
+}
+
+void pasr_put(phys_addr_t paddr, unsigned long size)
+{
+	struct pasr_section *s;
+	unsigned long cur_sz;
+	unsigned long flags;
+
+	if (!pasr.map) {
+		WARN_ONCE(1, KERN_INFO"%s(): Map not initialized.\n"
+			"\tCommand line parameters missing or incorrect\n"
+			, __func__);
+		goto out;
+	}
+
+	do {
+		s = pasr_addr2section(pasr.map, paddr);
+		if (!s)
+			goto out;
+
+		cur_sz = ((paddr + size) < (s->start + PASR_SECTION_SZ)) ?
+			size : s->start + PASR_SECTION_SZ - paddr;
+
+		if (s->lock)
+			spin_lock_irqsave(s->lock, flags);
+
+		s->free_size += cur_sz;
+		BUG_ON(s->free_size > PASR_SECTION_SZ);
+
+		if (s->free_size < PASR_SECTION_SZ)
+			goto unlock;
+
+		if (!s->pair)
+			pasr_update_mask(s, PASR_NO_REFRESH);
+		else if (s->pair->free_size == PASR_SECTION_SZ) {
+				pasr_update_mask(s, PASR_NO_REFRESH);
+				pasr_update_mask(s->pair, PASR_NO_REFRESH);
+		}
+unlock:
+		if (s->lock)
+			spin_unlock_irqrestore(s->lock, flags);
+
+		paddr += cur_sz;
+		size -= cur_sz;
+	} while (size);
+
+out:
+	return;
+}
+
+void pasr_get(phys_addr_t paddr, unsigned long size)
+{
+	unsigned long flags;
+	unsigned long cur_sz;
+	struct pasr_section *s;
+
+	if (!pasr.map) {
+		WARN_ONCE(1, KERN_INFO"%s(): Map not initialized.\n"
+			"\tCommand line parameters missing or incorrect\n"
+			, __func__);
+		return;
+	}
+
+	do {
+		s = pasr_addr2section(pasr.map, paddr);
+		if (!s)
+			goto out;
+
+		cur_sz = ((paddr + size) < (s->start + PASR_SECTION_SZ)) ?
+			size : s->start + PASR_SECTION_SZ - paddr;
+
+		if (s->lock)
+			spin_lock_irqsave(s->lock, flags);
+
+		if (s->free_size < PASR_SECTION_SZ)
+			goto unlock;
+
+		if (!s->pair)
+			pasr_update_mask(s, PASR_REFRESH);
+		else if (s->pair->free_size == PASR_SECTION_SZ) {
+				pasr_update_mask(s, PASR_REFRESH);
+				pasr_update_mask(s->pair, PASR_REFRESH);
+		}
+unlock:
+		BUG_ON(cur_sz > s->free_size);
+		s->free_size -= cur_sz;
+
+		if (s->lock)
+			spin_unlock_irqrestore(s->lock, flags);
+
+		paddr += cur_sz;
+		size -= cur_sz;
+	} while (size);
+
+out:
+	return;
+}
+
+int pasr_register_mask_function(phys_addr_t addr, void *function, void *cookie)
+{
+	struct pasr_die *die = pasr_addr2die(pasr.map, addr);
+
+	if (!die) {
+		pr_err("%s: No DDR die corresponding to address 0x%08x\n",
+				__func__, addr);
+		return -EINVAL;
+	}
+
+	if (addr != die->start)
+		pr_warning("%s: Addresses mismatch (Die = 0x%08x, addr = 0x%08x\n"
+				, __func__, die->start, addr);
+
+	die->cookie = cookie;
+	die->apply_mask = function;
+
+	die->apply_mask(&die->mem_reg, die->cookie);
+
+	return 0;
+}
+
+int __init pasr_init_core(struct pasr_map *map)
+{
+	pasr.map = map;
+	return 0;
+}
+
diff --git a/include/linux/pasr.h b/include/linux/pasr.h
index 93867f0..e85be73 100644
--- a/include/linux/pasr.h
+++ b/include/linux/pasr.h
@@ -49,6 +49,10 @@ struct pasr_die {
 	int idx;
 	int nr_sections;
 	struct pasr_section section[PASR_MAX_SECTION_NR_PER_DIE];
+	u16 mem_reg; /* Either MR16 or MR17 */
+
+	void (*apply_mask)(u16 *mem_reg, void *cookie);
+	void *cookie;
 };
 
 /**
@@ -68,6 +72,72 @@ struct pasr_map {
                                 j < map.die[i].nr_sections; \
                                 j++, s = &map.die[i].section[j])
 
+/**
+ * pasr_register_mask_function()
+ *
+ * @die_addr: Physical base address of the die.
+ * @function: Callback function for applying the DDR PASR mask.
+ * @cookie: Private data called with the callback function.
+ *
+ * This function is to be called by the platform specific PASR driver in
+ * charge of application of the PASR masks.
+ */
+int pasr_register_mask_function(phys_addr_t die_addr,
+		void *function, void *cookie);
+
+/**
+ * pasr_put()
+ *
+ * @paddr: Physical address of the freed memory chunk.
+ * @size: Size of the freed memory chunk.
+ *
+ * This function is to be placed in the allocators when memory chunks are
+ * inserted in the free memory pool.
+ * This function has only to be called for unused memory, otherwise retention
+ * cannot be guaranteed.
+ */
+void pasr_put(phys_addr_t paddr, unsigned long size);
+
+/**
+ * pasr_get()
+ *
+ * @paddr: Physical address of the allocated memory chunk.
+ * @size: Size of the allocated memory chunk.
+ *
+ * This function is to be placed in the allocators when memory chunks are
+ * removed from the free memory pool.
+ * If pasr_put() is used by the allocator, using this function is mandatory to
+ * guarantee retention.
+ */
+void pasr_get(phys_addr_t paddr, unsigned long size);
+
+
+static inline void pasr_kput(struct page *page, int order)
+{
+	if (order != MAX_ORDER - 1)
+		return;
+
+	pasr_put(page_to_phys(page), PAGE_SIZE << (MAX_ORDER - 1));
+}
+
+static inline void pasr_kget(struct page *page, int order)
+{
+	if (order != MAX_ORDER - 1)
+		return;
+
+	pasr_get(page_to_phys(page), PAGE_SIZE << (MAX_ORDER - 1));
+}
+
+int __init early_pasr_setup(void);
+int __init late_pasr_setup(void);
+int __init pasr_init_core(struct pasr_map *);
+
+#else
+#define pasr_kput(page, order) do {} while (0)
+#define pasr_kget(page, order) do {} while (0)
+
+#define pasr_put(paddr, size) do {} while (0)
+#define pasr_get(paddr, size) do {} while (0)
 #endif /* CONFIG_PASR */
 
 #endif /* _LINUX_PASR_H */
-- 
1.7.8

[RFCv1 3/6] PASR: mm: Integrate PASR in Buddy allocator

[Maxime Coquelin]

Any allocators might call the PASR Framework for DDR power savings. Currently,
only Linux Buddy allocator is patched, but HWMEM and PMEM physically
contiguous memory allocators will follow.

Linux Buddy allocator porting uses Buddy specificities to reduce the overhead
induced by the PASR Framework counter updates. Indeed, the PASR Framework is
called only when MAX_ORDER (4MB page blocs by default) buddies are
inserted/removed from the free lists.

To port PASR FW into a new allocator:

* Call pasr_put(phys_addr, size) each time a memory chunk becomes unused.
* Call pasr_get(phys_addr, size) each time a memory chunk becomes used.


Signed-off-by: Maxime Coquelin <maxime.coquelin@stericsson.com>
---
 mm/page_alloc.c |    9 +++++++++
 1 files changed, 9 insertions(+), 0 deletions(-)

diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 03d8c48..c62fe11 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -57,6 +57,7 @@
 #include <linux/ftrace_event.h>
 #include <linux/memcontrol.h>
 #include <linux/prefetch.h>
+#include <linux/pasr.h>
 
 #include <asm/tlbflush.h>
 #include <asm/div64.h>
@@ -534,6 +535,7 @@ static inline void __free_one_page(struct page *page,
 		/* Our buddy is free, merge with it and move up one order. */
 		list_del(&buddy->lru);
 		zone->free_area[order].nr_free--;
+		pasr_kget(buddy, order);
 		rmv_page_order(buddy);
 		combined_idx = buddy_idx & page_idx;
 		page = page + (combined_idx - page_idx);
@@ -566,6 +568,7 @@ static inline void __free_one_page(struct page *page,
 	list_add(&page->lru, &zone->free_area[order].free_list[migratetype]);
 out:
 	zone->free_area[order].nr_free++;
+	pasr_kput(page, order);
 }
 
 /*
@@ -762,6 +765,7 @@ static inline void expand(struct zone *zone, struct page *page,
 		VM_BUG_ON(bad_range(zone, &page[size]));
 		list_add(&page[size].lru, &area->free_list[migratetype]);
 		area->nr_free++;
+		pasr_kput(page, high);
 		set_page_order(&page[size], high);
 	}
 }
@@ -830,6 +834,7 @@ struct page *__rmqueue_smallest(struct zone *zone, unsigned int order,
 		list_del(&page->lru);
 		rmv_page_order(page);
 		area->nr_free--;
+		pasr_kget(page, current_order);
 		expand(zone, page, order, current_order, area, migratetype);
 		return page;
 	}
@@ -955,6 +960,7 @@ __rmqueue_fallback(struct zone *zone, int order, int start_migratetype)
 			page = list_entry(area->free_list[migratetype].next,
 					struct page, lru);
 			area->nr_free--;
+			pasr_kget(page, current_order);
 
 			/*
 			 * If breaking a large block of pages, move all free
@@ -1281,6 +1287,8 @@ int split_free_page(struct page *page)
 	/* Remove page from free list */
 	list_del(&page->lru);
 	zone->free_area[order].nr_free--;
+	pasr_kget(page, order);
+
 	rmv_page_order(page);
 	__mod_zone_page_state(zone, NR_FREE_PAGES, -(1UL << order));
 
@@ -5692,6 +5700,7 @@ __offline_isolated_pages(unsigned long start_pfn, unsigned long end_pfn)
 		list_del(&page->lru);
 		rmv_page_order(page);
 		zone->free_area[order].nr_free--;
+		pasr_kget(page, order);
 		__mod_zone_page_state(zone, NR_FREE_PAGES,
 				      - (1UL << order));
 		for (i = 0; i < (1 << order); i++)
-- 
1.7.8

[RFCv1 4/6] PASR: Call PASR initialization

[Maxime Coquelin]

Signed-off-by: Maxime Coquelin <maxime.coquelin@stericsson.com>
---
 init/main.c             |    8 ++++++++
 1 file changed, 8 insertions(+), 0 deletions(-)

diff --git a/init/main.c b/init/main.c
index 9fd91c3..5e0aeb7 100644
--- a/init/main.c
+++ b/init/main.c
@@ -69,6 +69,7 @@
 #include <linux/slab.h>
 #include <linux/perf_event.h>
 #include <linux/boottime.h>
+#include <linux/pasr.h>
 
 #include <asm/io.h>
 #include <asm/bugs.h>
@@ -487,6 +488,9 @@ asmlinkage void __init start_kernel(void)
 	page_address_init();
 	printk(KERN_NOTICE "%s", linux_banner);
 	setup_arch(&command_line);
+#ifdef CONFIG_PASR
+	early_pasr_setup();
+#endif
 	mm_init_owner(&init_mm, &init_task);
 	mm_init_cpumask(&init_mm);
 	setup_command_line(command_line);
@@ -555,6 +559,10 @@ asmlinkage void __init start_kernel(void)
 
 	kmem_cache_init_late();
 
+#ifdef CONFIG_PASR
+	late_pasr_setup();
+#endif
+
 	/*
 	 * HACK ALERT! This is early. We're enabling the console before
 	 * we've done PCI setups etc, and console_init() must be aware of
-- 
1.7.8

[RFCv1 5/6] PASR: Add Documentation

[Maxime Coquelin]

Signed-off-by: Maxime Coquelin <maxime.coquelin@stericsson.com>
---
 Documentation/pasr.txt |  183 ++++++++++++++++++++++++++++++++++++++++++++++++
 1 files changed, 183 insertions(+), 0 deletions(-)
 create mode 100644 Documentation/pasr.txt

diff --git a/Documentation/pasr.txt b/Documentation/pasr.txt
new file mode 100644
index 0000000..d40e3f6
--- /dev/null
+++ b/Documentation/pasr.txt
@@ -0,0 +1,183 @@
+Partial Array Self-Refresh Framework
+
+(C) 2012 Maxime Coquelin <maxime.coquelin@stericsson.com>, ST-Ericsson.
+
+CONTENT
+1. Introduction
+2. Command-line parameters
+3. Allocators patching
+4. PASR platform drivers
+
+
+1. Introduction
+
+PASR Frameworks brings support for the Partial Array Self-Refresh DDR power
+management feature. PASR has been introduced in LP-DDR2, and is also present
+in DDR3.
+
+PASR provides 4 modes:
+
+* Single-Ended: Only 1/1, 1/2, 1/4 or 1/8 are refreshed, masking starting at
+  the end of the DDR die.
+
+* Double-Ended: Same as Single-Ended, but refresh-masking does not start
+  necessairly at the end of the DDR die.
+
+* Bank-Selective: Refresh of each bank of a die can be masked or unmasked via
+  a dedicated DDR register (MR16). This mode is convenient for DDR configured
+  in BRC (Bank-Row-Column) mode.
+
+* Segment-Selective: Refresh of each segment of a die can be masked or unmasked
+  via a dedicated DDR register (MR17). This mode is convenient for DDR configured
+  in RBC (Row-Bank-Column) mode.
+
+The role of this framework is to stop the refresh of unused memory to enhance
+DDR power consumption.
+
+It supports Bank-Selective and Segment-Selective modes, as the more adapted to
+modern OSes.
+
+At early boot stage, a representation of the physical DDR layout is built:
+
+             Die 0
+_______________________________
+| I--------------------------I |
+| I    Bank or Segment 0     I |
+| I--------------------------I |
+| I--------------------------I |
+| I    Bank or Segment 1     I |
+| I--------------------------I |
+| I--------------------------I |
+| I    Bank or Segment ...   I |
+| I--------------------------I |
+| I--------------------------I |
+| I    Bank or Segment n     I |
+| I--------------------------I |
+|______________________________|
+             ...
+
+             Die n
+_______________________________
+| I--------------------------I |
+| I    Bank or Segment 0     I |
+| I--------------------------I |
+| I--------------------------I |
+| I    Bank or Segment 1     I |
+| I--------------------------I |
+| I--------------------------I |
+| I    Bank or Segment ...   I |
+| I--------------------------I |
+| I--------------------------I |
+| I    Bank or Segment n     I |
+| I--------------------------I |
+|______________________________|
+
+The first level is a table where elements represent a die:
+* Base address,
+* Number of segments,
+* Table representing banks/segments,
+* MR16/MR17 refresh mask,
+* DDR Controller callback to update MR16/MR17 refresh mask.
+
+The second level is the section tables representing the banks or segments,
+depending on hardware configuration:
+* Base address,
+* Unused memory size counter,
+* Possible pointer to another section it depends on (E.g. Interleaving)
+
+When some memory becomes unused, the allocator owning this memory calls the PASR
+Framework's pasr_put(phys_addr, size) function. The framework finds the
+sections impacted and updates their counters accordingly.
+If a section counter reach the section size, the refresh of the section is
+masked. If the corresponding section has a dependency with another section
+(E.g. because of DDR interleaving, see figure below), it checks the "paired" section is also
+unused before updating the refresh mask.
+
+When some unused memory is requested by the allocator, the allocator owning
+this memory calls the PASR Framework's pasr_get(phys_addr, size) function. The
+framework find the section impacted and updates their counters accordingly.
+If before the update, the section counter was to the section size, the refrewh
+of the section is unmasked. If the corresponding section has a dependency with
+another section, it also unmask the refresh of the other section.
+
+Interleaving example:
+
+             Die 0
+_______________________________
+| I--------------------------I |
+| I    Bank or Segment 0     I |<----|
+| I--------------------------I |     |
+| I--------------------------I |     |
+| I    Bank or Segment 1     I |     |
+| I--------------------------I |     |
+| I--------------------------I |     |
+| I    Bank or Segment ...   I |     |
+| I--------------------------I |     |
+| I--------------------------I |     |
+| I    Bank or Segment n     I |     |
+| I--------------------------I |     |
+|______________________________|     |
+                                     |
+             Die 1                   |
+_______________________________      |
+| I--------------------------I |     |
+| I    Bank or Segment 0     I |<----|
+| I--------------------------I |
+| I--------------------------I |
+| I    Bank or Segment 1     I |
+| I--------------------------I |
+| I--------------------------I |
+| I    Bank or Segment ...   I |
+| I--------------------------I |
+| I--------------------------I |
+| I    Bank or Segment n     I |
+| I--------------------------I |
+|______________________________|
+
+In the above example, bank 0 of die 0 is interleaved with bank0 of die 0.
+The interleaving is done in HW by inverting some addresses lines. The goal is
+to improve DDR bandwidth.
+Practically, one buffer seen as contiguous by the kernel might be spread
+into two DDR dies physically.
+
+
+2. Command-line parameters
+
+To buid the DDR physical layout representation, two parameters are requested:
+
+* ddr_die (mandatory): Should be added for every DDR dies present in the system.
+   - Usage: ddr_die=xxx[M|G]@yyy[M|G] where xxx represents the size and yyy
+     the base address of the die. E.g.: ddr_die=512M@0 ddr_die=512M@512M
+
+* interleaved (optionnal): Should be added for every interleaved dependencies.
+   - Usage: interleaved=xxx[M|G]@yyy[M|G]:zzz[M|G] where xxx is the size of
+     the interleaved area between the adresses yyy and zzz. E.g
+     interleaved=256M@0:512M
+
+
+3. Allocator patching
+
+Any allocators might call the PASR Framework for DDR power savings. Currently,
+only Linux Buddy allocator is patched, but HWMEM and PMEM physically
+contiguous memory allocators will follow.
+
+Linux Buddy allocator porting uses Buddy specificities to reduce the overhead
+induced by the PASR Framework counter updates. Indeed, the PASR Framework is
+called only when MAX_ORDER (4MB page blocs by default) buddies are
+inserted/removed from the free lists.
+
+To port PASR FW into a new allocator:
+
+* Call pasr_put(phys_addr, size) each time a memory chunk becomes unused.
+* Call pasr_get(phys_addr, size) each time a memory chunk becomes used.
+
+4. PASR platform drivers
+
+The MR16/MR17 PASR mask registers are generally accessible through the DDR
+controller. At probe time, the DDR controller driver should register the
+callback used by PASR Framework to apply the refresh mask for every DDR die
+using pasr_register_mask_function(die_addr, callback, cookie).
+
+The callback passed to apply mask must not sleep since it can me called in
+interrupt contexts.
+
-- 
1.7.8

[RFCv1 6/6] PASR: Ux500: Add PASR support

[Maxime Coquelin]

The MR16/MR17 PASR mask registers are generally accessible through the DDR
controller. At probe time, the DDR controller driver should register the
callback used by PASR Framework to apply the refresh mask for every DDR die
using pasr_register_mask_function(die_addr, callback, cookie).

The callback passed to apply mask must not sleep since it can me called in
interrupt contexts.

This example creates a new PASR stubbed driver for Nova platforms.

Signed-off-by: Maxime Coquelin <maxime.coquelin@stericsson.com>
---
 arch/arm/Kconfig                            |    1 +
 arch/arm/mach-ux500/include/mach/hardware.h |   11 ++++
 arch/arm/mach-ux500/include/mach/memory.h   |    8 +++
 drivers/mfd/db8500-prcmu.c                  |   67 +++++++++++++++++++++++++++
 drivers/staging/pasr/Kconfig                |    5 ++
 drivers/staging/pasr/Makefile               |    1 +
 drivers/staging/pasr/ux500.c                |   58 +++++++++++++++++++++++
 include/linux/ux500-pasr.h                  |   11 ++++
 8 files changed, 162 insertions(+), 0 deletions(-)
 create mode 100644 drivers/staging/pasr/ux500.c
 create mode 100644 include/linux/ux500-pasr.h

diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig
index 3df3573..b8981ee 100644
--- a/arch/arm/Kconfig
+++ b/arch/arm/Kconfig
@@ -826,6 +826,7 @@ config ARCH_U8500
 	select HAVE_CLK
 	select ARCH_HAS_CPUFREQ
 	select NOMADIK_GPIO
+	select ARCH_HAS_PASR
 	help
 	  Support for ST-Ericsson's Ux500 architecture
 
diff --git a/arch/arm/mach-ux500/include/mach/hardware.h b/arch/arm/mach-ux500/include/mach/hardware.h
index d8f218b..11c23b1 100644
--- a/arch/arm/mach-ux500/include/mach/hardware.h
+++ b/arch/arm/mach-ux500/include/mach/hardware.h
@@ -39,6 +39,17 @@
 #include <mach/db5500-regs.h>
 
 /*
+ * DDR Dies base addresses for PASR
+ */
+#define U8500_CS0_BASE_ADDR	0x00000000
+#define U8500_CS1_BASE_ADDR	0x10000000
+
+#define U9540_DDR0_CS0_BASE_ADDR 0x00000000
+#define U9540_DDR0_CS1_BASE_ADDR 0x20000000
+#define U9540_DDR1_CS0_BASE_ADDR 0xC0000000
+#define U9540_DDR1_CS1_BASE_ADDR 0xE0000000
+
+/*
  * FIFO offsets for IPs
  */
 #define MSP_TX_RX_REG_OFFSET	0
diff --git a/arch/arm/mach-ux500/include/mach/memory.h b/arch/arm/mach-ux500/include/mach/memory.h
index ada8ad0..5f5c339 100644
--- a/arch/arm/mach-ux500/include/mach/memory.h
+++ b/arch/arm/mach-ux500/include/mach/memory.h
@@ -15,6 +15,14 @@
 #define PLAT_PHYS_OFFSET	UL(0x00000000)
 #define BUS_OFFSET	UL(0x00000000)
 
+
+#ifdef CONFIG_UX500_PASR
+#define PASR_SECTION_SZ_BITS	26 /* 64MB sections */
+#define PASR_SECTION_SZ	(1 << PASR_SECTION_SZ_BITS)
+#define PASR_MAX_DIE_NR		4
+#define PASR_MAX_SECTION_NR_PER_DIE	8 /* 32 * 64MB = 2GB */
+#endif
+
 #ifdef CONFIG_UX500_SOC_DB8500
 /*
  * STE NMF CM driver only used on the U8500 allocate using dma_alloc_coherent:
diff --git a/drivers/mfd/db8500-prcmu.c b/drivers/mfd/db8500-prcmu.c
index 65a644d..db4ebd8 100644
--- a/drivers/mfd/db8500-prcmu.c
+++ b/drivers/mfd/db8500-prcmu.c
@@ -30,6 +30,7 @@
 #include <linux/mfd/dbx500-prcmu.h>
 #include <linux/regulator/db8500-prcmu.h>
 #include <linux/regulator/machine.h>
+#include <linux/ux500-pasr.h>
 #include <mach/hardware.h>
 #include <mach/irqs.h>
 #include <mach/db8500-regs.h>
@@ -105,6 +106,10 @@
 #define MB0H_CONFIG_WAKEUPS_EXE		1
 #define MB0H_READ_WAKEUP_ACK		3
 #define MB0H_CONFIG_WAKEUPS_SLEEP	4
+#define MB0H_SET_PASR_DDR0_CS0		5
+#define MB0H_SET_PASR_DDR0_CS1		6
+#define MB0H_SET_PASR_DDR1_CS0		7
+#define MB0H_SET_PASR_DDR1_CS1		8
 
 #define MB0H_WAKEUP_EXE 2
 #define MB0H_WAKEUP_SLEEP 5
@@ -116,6 +121,8 @@
 #define PRCM_REQ_MB0_DO_NOT_WFI		(PRCM_REQ_MB0 + 0x3)
 #define PRCM_REQ_MB0_WAKEUP_8500	(PRCM_REQ_MB0 + 0x4)
 #define PRCM_REQ_MB0_WAKEUP_4500	(PRCM_REQ_MB0 + 0x8)
+#define PRCM_REQ_MB0_PASR_MR16		(PRCM_REQ_MB0 + 0x0)
+#define PRCM_REQ_MB0_PASR_MR17		(PRCM_REQ_MB0 + 0x2)
 
 /* Mailbox 0 ACKs */
 #define PRCM_ACK_MB0_AP_PWRSTTR_STATUS	(PRCM_ACK_MB0 + 0x0)
@@ -3909,6 +3916,52 @@ static struct mfd_cell db8500_prcmu_devs[] = {
 	},
 };
 
+static struct ux500_pasr_data u9540_pasr_pdata[] = {
+	{
+		.base_addr = U9540_DDR0_CS0_BASE_ADDR,
+		.mailbox = MB0H_SET_PASR_DDR0_CS0,
+	},
+	{
+		.base_addr = U9540_DDR0_CS1_BASE_ADDR,
+		.mailbox = MB0H_SET_PASR_DDR0_CS1,
+	},
+	{
+		.base_addr = U9540_DDR1_CS0_BASE_ADDR,
+		.mailbox = MB0H_SET_PASR_DDR1_CS0,
+	},
+	{
+		.base_addr = U9540_DDR1_CS1_BASE_ADDR,
+		.mailbox = MB0H_SET_PASR_DDR1_CS1,
+	},
+	{
+		/*  End marker */
+		.base_addr = 0xFFFFFFFF
+	},
+};
+
+static struct ux500_pasr_data u8500_pasr_pdata[] = {
+	{
+		.base_addr = U8500_CS0_BASE_ADDR,
+		.mailbox = MB0H_SET_PASR_DDR0_CS0,
+	},
+	{
+		.base_addr = U8500_CS1_BASE_ADDR,
+		.mailbox = MB0H_SET_PASR_DDR0_CS1,
+	},
+	{
+		/*  End marker */
+		.base_addr = 0xFFFFFFFF
+	},
+};
+
+
+static struct mfd_cell ux500_pasr_devs[] = {
+	{
+		.name = "ux500-pasr",
+	},
+};
+
+
 /**
  * prcmu_fw_init - arch init call for the Linux PRCMU fw init logic
  *
@@ -3951,6 +4004,20 @@ static int __init db8500_prcmu_probe(struct platform_device *pdev)
 	else
 		pr_info("DB8500 PRCMU initialized\n");
 
+	if (cpu_is_u9540()) {
+		ux500_pasr_devs[0].platform_data = u9540_pasr_pdata;
+		ux500_pasr_devs[0].pdata_size = sizeof(u9540_pasr_pdata);
+	} else {
+		ux500_pasr_devs[0].platform_data = u8500_pasr_pdata;
+		ux500_pasr_devs[0].pdata_size = sizeof(u8500_pasr_pdata);
+	}
+
+	err = mfd_add_devices(&pdev->dev, 0, ux500_pasr_devs,
+			      ARRAY_SIZE(ux500_pasr_devs), NULL,
+			      0);
+	if (err)
+		pr_err("prcmu: Failed to add PASR subdevice\n");
+
 	/*
 	 * Temporary U9540 bringup code - Enable all clock gates.
 	 * Write 1 to all bits of PRCM_YYCLKEN0_MGT_SET and
diff --git a/drivers/staging/pasr/Kconfig b/drivers/staging/pasr/Kconfig
index 6bd2421..b8145e0 100644
--- a/drivers/staging/pasr/Kconfig
+++ b/drivers/staging/pasr/Kconfig
@@ -12,3 +12,8 @@ config PASR_DEBUG
 	bool "Add PASR debug prints"
 	def_bool n
 	depends on PASR
+
+config UX500_PASR
+	bool "Ux500 Family PASR driver"
+	def_bool n
+	depends on (PASR && UX500_SOC_DB8500)
diff --git a/drivers/staging/pasr/Makefile b/drivers/staging/pasr/Makefile
index d172294..0b18a79 100644
--- a/drivers/staging/pasr/Makefile
+++ b/drivers/staging/pasr/Makefile
@@ -1,5 +1,6 @@
 pasr-objs := helper.o init.o core.o
 
 obj-$(CONFIG_PASR) += pasr.o
+obj-$(CONFIG_UX500_PASR) += ux500.o
 
 ccflags-$(CONFIG_PASR_DEBUG) := -DDEBUG
diff --git a/drivers/staging/pasr/ux500.c b/drivers/staging/pasr/ux500.c
new file mode 100644
index 0000000..ce5df0c
--- /dev/null
+++ b/drivers/staging/pasr/ux500.c
@@ -0,0 +1,58 @@
+/*
+ * Copyright (C) ST-Ericsson SA 2012
+ * Author: Maxime Coquelin <maxime.coquelin@stericsson.com> for ST-Ericsson.
+ * License terms:  GNU General Public License (GPL), version 2
+ */
+#include <linux/module.h>
+#include <linux/platform_device.h>
+#include <linux/mfd/dbx500-prcmu.h>
+#include <linux/pasr.h>
+#include <linux/ux500-pasr.h>
+
+
+static void ux500_pasr_apply_mask(u16 *mem_reg, void *cookie)
+{
+	printk(KERN_INFO"%s: cookie = %d, mem_reg = 0x%04x\n",
+			__func__, (int)cookie, *mem_reg);
+}
+
+static int ux500_pasr_probe(struct platform_device *pdev)
+{
+	int i;
+	struct ux500_pasr_data *pasr_data = dev_get_platdata(&pdev->dev);
+
+	if (!pasr_data)
+		return -ENODEV;
+
+	for (i = 0; pasr_data[i].base_addr != 0xFFFFFFFF; i++) {
+		phys_addr_t base = pasr_data[i].base_addr;
+
+		/*
+		 * We don't have specific structure pointer to pass, but only
+		 * DDR die channel in PRCMU. This may change in future
+		 * version.
+		 */
+		void *cookie = (void *)(int)pasr_data[i].mailbox;
+
+		if (pasr_register_mask_function(base,
+				&ux500_pasr_apply_mask,
+				cookie))
+			printk(KERN_ERR"Pasr register failed\n");
+	}
+
+	return 0;
+}
+
+static struct platform_driver ux500_pasr_driver = {
+	.probe = ux500_pasr_probe,
+	.driver = {
+		.name = "ux500-pasr",
+		.owner = THIS_MODULE,
+	},
+};
+
+static int __init ux500_pasr_init(void)
+{
+	return platform_driver_register(&ux500_pasr_driver);
+}
+module_init(ux500_pasr_init);
diff --git a/include/linux/ux500-pasr.h b/include/linux/ux500-pasr.h
new file mode 100644
index 0000000..c62d961
--- /dev/null
+++ b/include/linux/ux500-pasr.h
@@ -0,0 +1,11 @@
+/*
+ * Copyright (C) ST-Ericsson SA 2012
+ * Author: Maxime Coquelin <maxime.coquelin@stericsson.com> for ST-Ericsson.
+ * License terms:  GNU General Public License (GPL), version 2
+ */
+
+struct ux500_pasr_data {
+	phys_addr_t base_addr;
+	u8 mailbox;
+};
+
-- 
1.7.8

Re: [RFCv1 0/6] PASR: Partial Array Self-Refresh Framework

[Ingo Molnar]

* Maxime Coquelin <maxime.coquelin@stericsson.com> wrote:

> The role of this framework is to stop the refresh of unused 
> memory to enhance DDR power consumption.

I'm wondering in what scenarios this is useful, and how 
consistently it is useful.

The primary concern I can see is that on most Linux systems with 
an uptime more than a couple of minutes RAM gets used up by the 
Linux page-cache:

 $ uptime
  14:46:39 up 11 days,  2:04, 19 users,  load average: 0.11, 0.29, 0.80
 $ free
              total       used       free     shared    buffers     cached
 Mem:      12255096   12030152     224944          0     651560    6000452
 -/+ buffers/cache:    5378140    6876956

Even mobile phones easily have days of uptime - quite often 
weeks of uptime. I'd expect the page-cache to fill up RAM on 
such systems.

So how will this actually end up saving power consistently? Does 
it have to be combined with a VM policy that more aggressively 
flushes cached pages from the page-cache?

A secondary concern is fragmentation: right now we fragment 
memory rather significantly. For the Ux500 PASR driver you've 
implemented the section size is 64 MB. Do I interpret the code 
correctly in that a continuous, 64MB physical block of RAM has 
to be 100% free for us to be able to turn off refresh and power 
for this block of RAM?

Thanks,

	Ingo

Re: [RFCv1 0/6] PASR: Partial Array Self-Refresh Framework

[Maxime Coquelin]

Dear Ingo,

On 01/30/2012 02:53 PM, Ingo Molnar wrote:
> * Maxime Coquelin<maxime.coquelin@stericsson.com>  wrote:
>
>> The role of this framework is to stop the refresh of unused
>> memory to enhance DDR power consumption.
> I'm wondering in what scenarios this is useful, and how
> consistently it is useful.
>
> The primary concern I can see is that on most Linux systems with
> an uptime more than a couple of minutes RAM gets used up by the
> Linux page-cache:
>
>   $ uptime
>    14:46:39 up 11 days,  2:04, 19 users,  load average: 0.11, 0.29, 0.80
>   $ free
>                total       used       free     shared    buffers     cached
>   Mem:      12255096   12030152     224944          0     651560    6000452
>   -/+ buffers/cache:    5378140    6876956
>
> Even mobile phones easily have days of uptime - quite often
> weeks of uptime. I'd expect the page-cache to fill up RAM on
> such systems.
>
> So how will this actually end up saving power consistently? Does
> it have to be combined with a VM policy that more aggressively
> flushes cached pages from the page-cache?
You're right Ingo, page-cache fills up the RAM.
This framework is to be used in combination with a page-cache flush 
governor.
In the case of a mobile phone, we can imagine dropping the cache when 
system's
screen is off for a while, in order to preserve user's experience.

>
> A secondary concern is fragmentation: right now we fragment
> memory rather significantly.
Yes, I think fragmentation is the main challenge.
This is the same problem faced for Memory Hotplug feature.
The solution I see is to add a significant Movable zone in the system and
use the Compaction feature from Mel Gorman.
The problem of course remains for the Normal zone.

> For the Ux500 PASR driver you've
> implemented the section size is 64 MB. Do I interpret the code
> correctly in that a continuous, 64MB physical block of RAM has
> to be 100% free for us to be able to turn off refresh and power
> for this block of RAM?
Current DDR (2Gb/4Gb dies) used in mobile platform have 64MB banks and 
segments.
This is the lower granularity for Partial Array Self-refresh.

Thanks for your comments,
Maxime
> Thanks,
>
> 	Ingo

Re: [RFCv1 3/6] PASR: mm: Integrate PASR in Buddy allocator

[Mel Gorman]

On Mon, Jan 30, 2012 at 02:33:53PM +0100, Maxime Coquelin wrote:
> Any allocators might call the PASR Framework for DDR power savings. Currently,
> only Linux Buddy allocator is patched, but HWMEM and PMEM physically
> contiguous memory allocators will follow.
> 
> Linux Buddy allocator porting uses Buddy specificities to reduce the overhead
> induced by the PASR Framework counter updates. Indeed, the PASR Framework is
> called only when MAX_ORDER (4MB page blocs by default) buddies are
> inserted/removed from the free lists.
> 
> To port PASR FW into a new allocator:
> 
> * Call pasr_put(phys_addr, size) each time a memory chunk becomes unused.
> * Call pasr_get(phys_addr, size) each time a memory chunk becomes used.
> 
> 
> Signed-off-by: Maxime Coquelin <maxime.coquelin@stericsson.com>
> ---
>  mm/page_alloc.c |    9 +++++++++
>  1 files changed, 9 insertions(+), 0 deletions(-)
> 
> diff --git a/mm/page_alloc.c b/mm/page_alloc.c
> index 03d8c48..c62fe11 100644
> --- a/mm/page_alloc.c
> +++ b/mm/page_alloc.c
> @@ -57,6 +57,7 @@
>  #include <linux/ftrace_event.h>
>  #include <linux/memcontrol.h>
>  #include <linux/prefetch.h>
> +#include <linux/pasr.h>
>  
>  #include <asm/tlbflush.h>
>  #include <asm/div64.h>
> @@ -534,6 +535,7 @@ static inline void __free_one_page(struct page *page,
>  		/* Our buddy is free, merge with it and move up one order. */
>  		list_del(&buddy->lru);
>  		zone->free_area[order].nr_free--;
> +		pasr_kget(buddy, order);
>  		rmv_page_order(buddy);
>  		combined_idx = buddy_idx & page_idx;
>  		page = page + (combined_idx - page_idx);

I did not review this series carefully and I know nothing about
how you implemented PASR support but driver hooks like this in the
page allocator are heavily frowned upon. It is subject to abuse but
it adds overhead to the allocator although I note that you avoiding
putting hooks in the per-cpu page allocator. I note that you hardcode
it so only PASR can use the hook but it looks like there is no way
of avoiding that overhead on platforms that do not have PASR if
it is enabled in the config. At a glance, it appears to be doing a
fair amount of work too - looking up maps, taking locks etc. This
potentially creates a new hot lock because in this paths, we have
per-zone locking but you are adding a PASR lock into the mix that
may be more coarse than zone->lock (I didn't check).

You may be able to use the existing arch_alloc_page() hook and
call PASR on architectures that support it if and only if PASR is
present and enabled by the administrator but even this is likely to be
unpopular as it'll have a measurable performance impact on platforms
with PASR (not to mention the PASR lock will be even heavier as it'll
now be also used for per-cpu page allocations). To get the hook you
want, you'd need to show significant benefit before they were happy with
the hook.

What is more likely is that you will get pushed to doing something like
periodically scanning memory as part of a separate power management
module and calling into PASR if regions of memory that are found that
can be powered down in some ways.

-- 
Mel Gorman
SUSE Labs

Re: [RFCv1 3/6] PASR: mm: Integrate PASR in Buddy allocator

[Maxime Coquelin]

Hello Mel,

Thanks for your comments,

On 01/30/2012 04:22 PM, Mel Gorman wrote:
> On Mon, Jan 30, 2012 at 02:33:53PM +0100, Maxime Coquelin wrote:
>> Any allocators might call the PASR Framework for DDR power savings. Currently,
>> only Linux Buddy allocator is patched, but HWMEM and PMEM physically
>> contiguous memory allocators will follow.
>>
>> Linux Buddy allocator porting uses Buddy specificities to reduce the overhead
>> induced by the PASR Framework counter updates. Indeed, the PASR Framework is
>> called only when MAX_ORDER (4MB page blocs by default) buddies are
>> inserted/removed from the free lists.
>>
>> To port PASR FW into a new allocator:
>>
>> * Call pasr_put(phys_addr, size) each time a memory chunk becomes unused.
>> * Call pasr_get(phys_addr, size) each time a memory chunk becomes used.
>>
>>
>> Signed-off-by: Maxime Coquelin<maxime.coquelin@stericsson.com>
>> ---
>>   mm/page_alloc.c |    9 +++++++++
>>   1 files changed, 9 insertions(+), 0 deletions(-)
>>
>> diff --git a/mm/page_alloc.c b/mm/page_alloc.c
>> index 03d8c48..c62fe11 100644
>> --- a/mm/page_alloc.c
>> +++ b/mm/page_alloc.c
>> @@ -57,6 +57,7 @@
>>   #include<linux/ftrace_event.h>
>>   #include<linux/memcontrol.h>
>>   #include<linux/prefetch.h>
>> +#include<linux/pasr.h>
>>
>>   #include<asm/tlbflush.h>
>>   #include<asm/div64.h>
>> @@ -534,6 +535,7 @@ static inline void __free_one_page(struct page *page,
>>   		/* Our buddy is free, merge with it and move up one order. */
>>   		list_del(&buddy->lru);
>>   		zone->free_area[order].nr_free--;
>> +		pasr_kget(buddy, order);
>>   		rmv_page_order(buddy);
>>   		combined_idx = buddy_idx&  page_idx;
>>   		page = page + (combined_idx - page_idx);
> I did not review this series carefully and I know nothing about
> how you implemented PASR support but driver hooks like this in the
> page allocator are heavily frowned upon. It is subject to abuse but
> it adds overhead to the allocator although I note that you avoiding
> putting hooks in the per-cpu page allocator.
I catch your point.
However, adding hooks in the page allocator is the only way I see to 
ensure memory that is accessed is refreshed.

> I note that you hardcode
> it so only PASR can use the hook but it looks like there is no way
> of avoiding that overhead on platforms that do not have PASR if
> it is enabled in the config.
In that RFC patch set, I assumed the PASR would be enabled in the config 
only in case it is used.
If not activated, there is no overhead.

This could of course be improved in next patch set.
>   At a glance, it appears to be doing a
> fair amount of work too - looking up maps, taking locks etc.
Note that we do that work only on MAX_ORDER pages, so it limits the 
overhead.
>   This
> potentially creates a new hot lock because in this paths, we have
> per-zone locking but you are adding a PASR lock into the mix that
> may be more coarse than zone->lock (I didn't check).
Ok.
We might fall in a deadlock if the underlying PASR driver allocates 
something in its apply_mask callback.
However, this callback should be used only to write a register of the 
DDR controller.

> You may be able to use the existing arch_alloc_page() hook and
> call PASR on architectures that support it if and only if PASR is
> present and enabled by the administrator but even this is likely to be
> unpopular as it'll have a measurable performance impact on platforms
> with PASR (not to mention the PASR lock will be even heavier as it'll
> now be also used for per-cpu page allocations). To get the hook you
> want, you'd need to show significant benefit before they were happy with
> the hook.
Your proposal sounds good.
AFAIK, per-cpu allocation maximum size is 32KB. Please correct me if I'm 
wrong.
Since pasr_kget/kput() calls the PASR framework only on MAX_ORDER 
allocations, we wouldn't add any locking risks nor contention compared 
to current patch.
I will update the patch set using  arch_alloc/free_page().

>
> What is more likely is that you will get pushed to doing something like
> periodically scanning memory as part of a separate power management
> module and calling into PASR if regions of memory that are found that
> can be powered down in some ways.
With this solution, we need in any case to add some hooks in the 
allocator to ensure the pages being allocated are refreshed.

Best regards,
Maxime

Re: [RFCv1 3/6] PASR: mm: Integrate PASR in Buddy allocator

[Maxime Coquelin]

Hello Mel,
On 01/30/2012 05:52 PM, Maxime Coquelin wrote:
>
> On 01/30/2012 04:22 PM, Mel Gorman wrote:
>
>> You may be able to use the existing arch_alloc_page() hook and
>> call PASR on architectures that support it if and only if PASR is
>> present and enabled by the administrator but even this is likely to be
>> unpopular as it'll have a measurable performance impact on platforms
>> with PASR (not to mention the PASR lock will be even heavier as it'll
>> now be also used for per-cpu page allocations). To get the hook you
>> want, you'd need to show significant benefit before they were happy with
>> the hook.
> Your proposal sounds good.
> AFAIK, per-cpu allocation maximum size is 32KB. Please correct me if 
> I'm wrong.
> Since pasr_kget/kput() calls the PASR framework only on MAX_ORDER 
> allocations, we wouldn't add any locking risks nor contention compared 
> to current patch.
> I will update the patch set using  arch_alloc/free_page().
>
I just had a deeper look at when arch_alloc_page() is called. I think it 
does not fit with PASR framework needs.
pasr_kget() calls pasr_get() only for max order pages (same for 
pasr_kput()) to avoid overhead.

In current patch set, pasr_kget() is called when pages are removed from 
the free lists, and pasr_kput() when pages are inserted in the free lists.
So, pasr_get() is called in case of :
     - allocation of a max order page
     - split of a max order page into lower order pages to fulfill 
allocation of pages smaller than max order
And pasr_put() is called in case of:
     - release of a max order page
     - coalescence of two "max order -1" pages when smaller pages are 
released

If we call the PASR framework in arch_alloc_page(), we have two 
possibilities:
     1) using pasr_kget(): the PASR framework will only be notified of 
max order allocations, so the coalesce/split of free pages case will not 
be taken into account.
     2) using pasr_get(): the PASR framework will be called for every 
orders of page allocation/release. The induced overhead is not acceptable.

To avoid calling pasr_kget/kput() directly in page_alloc.c, do you think 
adding some arch specific hooks when a page is inserted or removed from 
the free lists could be acceptable?
Something like arch_insert_freepage(struct page *page, int order) and 
arch_remove_freepage(struct page *page, int order).

Regards,
Maxime

Re: [RFCv1 0/6] PASR: Partial Array Self-Refresh Framework

[Ingo Molnar]

* Maxime Coquelin <maxime.coquelin@stericsson.com> wrote:

> Dear Ingo,
> 
> On 01/30/2012 02:53 PM, Ingo Molnar wrote:
> >* Maxime Coquelin<maxime.coquelin@stericsson.com>  wrote:
> >
> >>The role of this framework is to stop the refresh of unused
> >>memory to enhance DDR power consumption.
> >I'm wondering in what scenarios this is useful, and how
> >consistently it is useful.
> >
> >The primary concern I can see is that on most Linux systems with
> >an uptime more than a couple of minutes RAM gets used up by the
> >Linux page-cache:
> >
> >  $ uptime
> >   14:46:39 up 11 days,  2:04, 19 users,  load average: 0.11, 0.29, 0.80
> >  $ free
> >               total       used       free     shared    buffers     cached
> >  Mem:      12255096   12030152     224944          0     651560    6000452
> >  -/+ buffers/cache:    5378140    6876956
> >
> > Even mobile phones easily have days of uptime - quite often 
> > weeks of uptime. I'd expect the page-cache to fill up RAM on 
> > such systems.
> >
> > So how will this actually end up saving power consistently? 
> > Does it have to be combined with a VM policy that more 
> > aggressively flushes cached pages from the page-cache?
>
> You're right Ingo, page-cache fills up the RAM. This framework 
> is to be used in combination with a page-cache flush governor. 
> In the case of a mobile phone, we can imagine dropping the 
> cache when system's screen is off for a while, in order to 
> preserve user's experience.

Is this "page-cache flush governor" some existing code?
How does it work and does it need upstream patches?

> > A secondary concern is fragmentation: right now we fragment 
> > memory rather significantly.
>
> Yes, I think fragmentation is the main challenge. This is the 
> same problem faced for Memory Hotplug feature. The solution I 
> see is to add a significant Movable zone in the system and use 
> the Compaction feature from Mel Gorman. The problem of course 
> remains for the Normal zone.

Ok. I guess phones/appliances can generally live with a 
relatively large movable zone as they don't have serious
memory pressure issues.

> > For the Ux500 PASR driver you've implemented the section 
> > size is 64 MB. Do I interpret the code correctly in that a 
> > continuous, 64MB physical block of RAM has to be 100% free 
> > for us to be able to turn off refresh and power for this 
> > block of RAM?
>
> Current DDR (2Gb/4Gb dies) used in mobile platform have 64MB 
> banks and segments. This is the lower granularity for Partial 
> Array Self-refresh.

Ok, so do you see real, consistent power savings with a large 
movable zone, with page cache governor patches applied (assuming 
it's a kernel mechanism) and CONFIG_COMPACTION=y enabled, on an 
upstream kernel with all these patches applied?

Thanks,

	Ingo

Re: [RFCv1 3/6] PASR: mm: Integrate PASR in Buddy allocator

[Pekka Enberg]

On Mon, Jan 30, 2012 at 6:52 PM, Maxime Coquelin
<maxime.coquelin@stericsson.com> wrote:
>> What is more likely is that you will get pushed to doing something like
>> periodically scanning memory as part of a separate power management
>> module and calling into PASR if regions of memory that are found that
>> can be powered down in some ways.
>
> With this solution, we need in any case to add some hooks in the allocator
> to ensure the pages being allocated are refreshed.

Why do you insist on making this happen at page level when you're only
able to power off *much* larger chunks?

Re: [RFCv1 3/6] PASR: mm: Integrate PASR in Buddy allocator

[Mel Gorman]

On Tue, Jan 31, 2012 at 01:15:55PM +0100, Maxime Coquelin wrote:
> Hello Mel,
> On 01/30/2012 05:52 PM, Maxime Coquelin wrote:
> >
> >On 01/30/2012 04:22 PM, Mel Gorman wrote:
> >
> >>You may be able to use the existing arch_alloc_page() hook and
> >>call PASR on architectures that support it if and only if PASR is
> >>present and enabled by the administrator but even this is likely to be
> >>unpopular as it'll have a measurable performance impact on platforms
> >>with PASR (not to mention the PASR lock will be even heavier as it'll
> >>now be also used for per-cpu page allocations). To get the hook you
> >>want, you'd need to show significant benefit before they were happy with
> >>the hook.
> >Your proposal sounds good.
> >AFAIK, per-cpu allocation maximum size is 32KB. Please correct me
> >if I'm wrong.
> >Since pasr_kget/kput() calls the PASR framework only on MAX_ORDER
> >allocations, we wouldn't add any locking risks nor contention
> >compared to current patch.
> >I will update the patch set using  arch_alloc/free_page().
> >
> I just had a deeper look at when arch_alloc_page() is called. I
> think it does not fit with PASR framework needs.
> pasr_kget() calls pasr_get() only for max order pages (same for
> pasr_kput()) to avoid overhead.
> 

I see. My bad.

> In current patch set, pasr_kget() is called when pages are removed
> from the free lists, and pasr_kput() when pages are inserted in the
> free lists.
> So, pasr_get() is called in case of :
>     - allocation of a max order page
>     - split of a max order page into lower order pages to fulfill
> allocation of pages smaller than max order
> And pasr_put() is called in case of:
>     - release of a max order page
>     - coalescence of two "max order -1" pages when smaller pages are
> released
> 
> If we call the PASR framework in arch_alloc_page(), we have two
> possibilities:
>     1) using pasr_kget(): the PASR framework will only be notified
> of max order allocations, so the coalesce/split of free pages case
> will not be taken into account.
>     2) using pasr_get(): the PASR framework will be called for every
> orders of page allocation/release. The induced overhead is not
> acceptable.
> 
> To avoid calling pasr_kget/kput() directly in page_alloc.c, do you
> think adding some arch specific hooks when a page is inserted or
> removed from the free lists could be acceptable?

It's not the name that is the problem, I'm strongly against any hook
that can delay the page allocator for arbitrary lengths of time like
this. I am open to being convinced otherwise but for me PASR would
need to demonstrate large savings for a wide variety of machines and
the alternatives would have to be considered and explained why they
would be far inferior or unsuitable.

For example - it seems like this could be also be done with a
balloon driver instead of page allocator hooks. A governer would
identify when the machine was under no memory pressure or triggered
from userspace. To power down memory, it would use page reclaim and
page migration to allocate large contiguous ranges of memory - CMA
could potentially be adapted when it gets merged to save a lot of
implementation work. The governer should register a slab shrinker
so that under memory pressure it gets called so it can shrink the
ballon, power the DIMMS back up and free the memory back to the
buddy allocator. This would keep all the cost out of the allocator
paths and move the cost to when the machine is either idle (in the
case of powering down) or under memory pressure (where the cost of
powering up will be small in comparison to the overall cost of the
page reclaim operation).

-- 
Mel Gorman
SUSE Labs

Re: [RFCv1 0/6] PASR: Partial Array Self-Refresh Framework

[Maxime Coquelin]

On 01/31/2012 01:39 PM, Ingo Molnar wrote:
> * Maxime Coquelin<maxime.coquelin@stericsson.com>  wrote:
>
>> Dear Ingo,
>>
>> On 01/30/2012 02:53 PM, Ingo Molnar wrote:
>>> * Maxime Coquelin<maxime.coquelin@stericsson.com>   wrote:
>>>
>>>> The role of this framework is to stop the refresh of unused
>>>> memory to enhance DDR power consumption.
>>> I'm wondering in what scenarios this is useful, and how
>>> consistently it is useful.
>>>
>>> The primary concern I can see is that on most Linux systems with
>>> an uptime more than a couple of minutes RAM gets used up by the
>>> Linux page-cache:
>>>
>>>   $ uptime
>>>    14:46:39 up 11 days,  2:04, 19 users,  load average: 0.11, 0.29, 0.80
>>>   $ free
>>>                total       used       free     shared    buffers     cached
>>>   Mem:      12255096   12030152     224944          0     651560    6000452
>>>   -/+ buffers/cache:    5378140    6876956
>>>
>>> Even mobile phones easily have days of uptime - quite often
>>> weeks of uptime. I'd expect the page-cache to fill up RAM on
>>> such systems.
>>>
>>> So how will this actually end up saving power consistently?
>>> Does it have to be combined with a VM policy that more
>>> aggressively flushes cached pages from the page-cache?
>> You're right Ingo, page-cache fills up the RAM. This framework
>> is to be used in combination with a page-cache flush governor.
>> In the case of a mobile phone, we can imagine dropping the
>> cache when system's screen is off for a while, in order to
>> preserve user's experience.
> Is this "page-cache flush governor" some existing code?
> How does it work and does it need upstream patches?
For now, such a governor has not been implemented.
I use the dedicated ProcFS interface to test the framework (echo 3 > 
/proc/sys/vm/drop_caches).


>>> A secondary concern is fragmentation: right now we fragment
>>> memory rather significantly.
>> Yes, I think fragmentation is the main challenge. This is the
>> same problem faced for Memory Hotplug feature. The solution I
>> see is to add a significant Movable zone in the system and use
>> the Compaction feature from Mel Gorman. The problem of course
>> remains for the Normal zone.
> Ok. I guess phones/appliances can generally live with a
> relatively large movable zone as they don't have serious
> memory pressure issues.
Actually, current high-end smartphones and tablets have 1GB DDR.
Smartphones and tablets arriving later this year should have up to 2GB DDR.
For example, my Android phone running for 2 days has only 230MB are used 
in idle once the page-caches dropped.
So I think having a 1GB movable zone on a 2GB DDR phone is conceivable.

>>> For the Ux500 PASR driver you've implemented the section
>>> size is 64 MB. Do I interpret the code correctly in that a
>>> continuous, 64MB physical block of RAM has to be 100% free
>>> for us to be able to turn off refresh and power for this
>>> block of RAM?
>> Current DDR (2Gb/4Gb dies) used in mobile platform have 64MB
>> banks and segments. This is the lower granularity for Partial
>> Array Self-refresh.
> Ok, so do you see real, consistent power savings with a large
> movable zone, with page cache governor patches applied (assuming
> it's a kernel mechanism) and CONFIG_COMPACTION=y enabled, on an
> upstream kernel with all these patches applied?
I don't have consistent figures for now as it is being prototyped.
 From the DDR datasheet I gathered, the DDR power savings is about 33% 
when half of the die is in self-refresh, compared to the full die in 
self-refresh.



Thanks for your comments,
Maxime

>
> Thanks,
>
> 	Ingo
>
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@kvack.org.  For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
> Don't email:<a href=mailto:"dont@kvack.org">  email@kvack.org</a>

Re: [RFCv1 3/6] PASR: mm: Integrate PASR in Buddy allocator

[Maxime Coquelin]

On 01/31/2012 03:01 PM, Mel Gorman wrote:
> On Tue, Jan 31, 2012 at 01:15:55PM +0100, Maxime Coquelin wrote:
>> In current patch set, pasr_kget() is called when pages are removed
>> from the free lists, and pasr_kput() when pages are inserted in the
>> free lists.
>> So, pasr_get() is called in case of :
>>      - allocation of a max order page
>>      - split of a max order page into lower order pages to fulfill
>> allocation of pages smaller than max order
>> And pasr_put() is called in case of:
>>      - release of a max order page
>>      - coalescence of two "max order -1" pages when smaller pages are
>> released
>>
>> If we call the PASR framework in arch_alloc_page(), we have two
>> possibilities:
>>      1) using pasr_kget(): the PASR framework will only be notified
>> of max order allocations, so the coalesce/split of free pages case
>> will not be taken into account.
>>      2) using pasr_get(): the PASR framework will be called for every
>> orders of page allocation/release. The induced overhead is not
>> acceptable.
>>
>> To avoid calling pasr_kget/kput() directly in page_alloc.c, do you
>> think adding some arch specific hooks when a page is inserted or
>> removed from the free lists could be acceptable?
> It's not the name that is the problem, I'm strongly against any hook
> that can delay the page allocator for arbitrary lengths of time like
> this. I am open to being convinced otherwise but for me PASR would
> need to demonstrate large savings for a wide variety of machines and
> the alternatives would have to be considered and explained why they
> would be far inferior or unsuitable.
Ok Mel, I understand your point of view.

The goal of this RFC patch set was to collect comments, so I'm glad to 
get your opinion.
I propose to forget the patch in the Buddy allocator.

> For example - it seems like this could be also be done with a
> balloon driver instead of page allocator hooks. A governer would
> identify when the machine was under no memory pressure or triggered
> from userspace. To power down memory, it would use page reclaim and
> page migration to allocate large contiguous ranges of memory - CMA
> could potentially be adapted when it gets merged to save a lot of
> implementation work. The governer should register a slab shrinker
> so that under memory pressure it gets called so it can shrink the
> ballon, power the DIMMS back up and free the memory back to the
> buddy allocator. This would keep all the cost out of the allocator
> paths and move the cost to when the machine is either idle (in the
> case of powering down) or under memory pressure (where the cost of
> powering up will be small in comparison to the overall cost of the
> page reclaim operation).
>
This is very interesting.
I know Linaro plans to work on DDR power management topic.
One of the options they envisage is to use the Memory Hotplug feature.
However, the main problem with Memory Hotplug is to handle the memory 
pressure, i.e. when to re-plug the memory sections.
Your proposal address this issue. I don't know if such a driver could be 
done in the Linaro scope.

Anyway, even with a balloon driver, I think the PASR framework could be 
suitable to keep an "hardware" view of the memory layout (dies, banks, 
segments...).
Moreover, this framework is designed to also support some physically 
contiguous memory allocators (such as hwmem and pmem).

Best regards,
Maxime

Re: [RFCv1 5/6] PASR: Add Documentation

[Randy Dunlap]

On 01/30/2012 05:33 AM, Maxime Coquelin wrote:
> Signed-off-by: Maxime Coquelin <maxime.coquelin@stericsson.com>
> ---
>  Documentation/pasr.txt |  183 ++++++++++++++++++++++++++++++++++++++++++++++++
>  1 files changed, 183 insertions(+), 0 deletions(-)
>  create mode 100644 Documentation/pasr.txt
> 
> diff --git a/Documentation/pasr.txt b/Documentation/pasr.txt
> new file mode 100644
> index 0000000..d40e3f6
> --- /dev/null
> +++ b/Documentation/pasr.txt
> @@ -0,0 +1,183 @@
> +Partial Array Self-Refresh Framework
> +
> +(C) 2012 Maxime Coquelin <maxime.coquelin@stericsson.com>, ST-Ericsson.
> +
> +CONTENT
> +1. Introduction
> +2. Command-line parameters
> +3. Allocators patching
> +4. PASR platform drivers
> +
> +
> +1. Introduction
> +
> +PASR Frameworks brings support for the Partial Array Self-Refresh DDR power

  The PASR framework brings support

> +management feature. PASR has been introduced in LP-DDR2, and is also present

                            was introduced in LP-DDR2 and is also present


> +in DDR3.
> +
> +PASR provides 4 modes:
> +
> +* Single-Ended: Only 1/1, 1/2, 1/4 or 1/8 are refreshed, masking starting at
> +  the end of the DDR die.
> +
> +* Double-Ended: Same as Single-Ended, but refresh-masking does not start
> +  necessairly at the end of the DDR die.

     necessarily

> +
> +* Bank-Selective: Refresh of each bank of a die can be masked or unmasked via
> +  a dedicated DDR register (MR16). This mode is convenient for DDR configured
> +  in BRC (Bank-Row-Column) mode.
> +
> +* Segment-Selective: Refresh of each segment of a die can be masked or unmasked
> +  via a dedicated DDR register (MR17). This mode is convenient for DDR configured
> +  in RBC (Row-Bank-Column) mode.
> +
> +The role of this framework is to stop the refresh of unused memory to enhance
> +DDR power consumption.
> +
> +It supports Bank-Selective and Segment-Selective modes, as the more adapted to
> +modern OSes.

huh?  parse error above.

> +
> +At early boot stage, a representation of the physical DDR layout is built:
> +
> +             Die 0
> +_______________________________
> +| I--------------------------I |
> +| I    Bank or Segment 0     I |
> +| I--------------------------I |
> +| I--------------------------I |
> +| I    Bank or Segment 1     I |
> +| I--------------------------I |
> +| I--------------------------I |
> +| I    Bank or Segment ...   I |
> +| I--------------------------I |
> +| I--------------------------I |
> +| I    Bank or Segment n     I |
> +| I--------------------------I |
> +|______________________________|
> +             ...
> +
> +             Die n
> +_______________________________
> +| I--------------------------I |
> +| I    Bank or Segment 0     I |
> +| I--------------------------I |
> +| I--------------------------I |
> +| I    Bank or Segment 1     I |
> +| I--------------------------I |
> +| I--------------------------I |
> +| I    Bank or Segment ...   I |
> +| I--------------------------I |
> +| I--------------------------I |
> +| I    Bank or Segment n     I |
> +| I--------------------------I |
> +|______________________________|
> +
> +The first level is a table where elements represent a die:
> +* Base address,
> +* Number of segments,
> +* Table representing banks/segments,
> +* MR16/MR17 refresh mask,
> +* DDR Controller callback to update MR16/MR17 refresh mask.
> +
> +The second level is the section tables representing the banks or segments,
> +depending on hardware configuration:
> +* Base address,
> +* Unused memory size counter,
> +* Possible pointer to another section it depends on (E.g. Interleaving)
> +
> +When some memory becomes unused, the allocator owning this memory calls the PASR
> +Framework's pasr_put(phys_addr, size) function. The framework finds the
> +sections impacted and updates their counters accordingly.
> +If a section counter reach the section size, the refresh of the section is

                        reaches

> +masked. If the corresponding section has a dependency with another section
> +(E.g. because of DDR interleaving, see figure below), it checks the "paired" section is also

                                                         it checks if the "paired" section is also

> +unused before updating the refresh mask.
> +
> +When some unused memory is requested by the allocator, the allocator owning
> +this memory calls the PASR Framework's pasr_get(phys_addr, size) function. The
> +framework find the section impacted and updates their counters accordingly.

             finds                     and updates its counter accordingly.
or
             find the sections impacted and updates their counters accordingly.


> +If before the update, the section counter was to the section size, the refrewh

                                             was equal to the section size, the refresh

> +of the section is unmasked. If the corresponding section has a dependency with
> +another section, it also unmask the refresh of the other section.

                            unmasks

> +
> +Interleaving example:
> +
> +             Die 0
> +_______________________________
> +| I--------------------------I |
> +| I    Bank or Segment 0     I |<----|
> +| I--------------------------I |     |
> +| I--------------------------I |     |
> +| I    Bank or Segment 1     I |     |
> +| I--------------------------I |     |
> +| I--------------------------I |     |
> +| I    Bank or Segment ...   I |     |
> +| I--------------------------I |     |
> +| I--------------------------I |     |
> +| I    Bank or Segment n     I |     |
> +| I--------------------------I |     |
> +|______________________________|     |
> +                                     |
> +             Die 1                   |
> +_______________________________      |
> +| I--------------------------I |     |
> +| I    Bank or Segment 0     I |<----|
> +| I--------------------------I |
> +| I--------------------------I |
> +| I    Bank or Segment 1     I |
> +| I--------------------------I |
> +| I--------------------------I |
> +| I    Bank or Segment ...   I |
> +| I--------------------------I |
> +| I--------------------------I |
> +| I    Bank or Segment n     I |
> +| I--------------------------I |
> +|______________________________|
> +
> +In the above example, bank 0 of die 0 is interleaved with bank0 of die 0.

                                                             bank 0 of die 1.

> +The interleaving is done in HW by inverting some addresses lines. The goal is

                            in hardware

> +to improve DDR bandwidth.
> +Practically, one buffer seen as contiguous by the kernel might be spread
> +into two DDR dies physically.
> +
> +
> +2. Command-line parameters
> +
> +To buid the DDR physical layout representation, two parameters are requested:

      build

> +
> +* ddr_die (mandatory): Should be added for every DDR dies present in the system.

                                                        die

> +   - Usage: ddr_die=xxx[M|G]@yyy[M|G] where xxx represents the size and yyy
> +     the base address of the die. E.g.: ddr_die=512M@0 ddr_die=512M@512M
> +
> +* interleaved (optionnal): Should be added for every interleaved dependencies.

                 (optional):                  for all interleaved dependencies.


> +   - Usage: interleaved=xxx[M|G]@yyy[M|G]:zzz[M|G] where xxx is the size of
> +     the interleaved area between the adresses yyy and zzz. E.g
> +     interleaved=256M@0:512M
> +
> +
> +3. Allocator patching
> +
> +Any allocators might call the PASR Framework for DDR power savings. Currently,
> +only Linux Buddy allocator is patched, but HWMEM and PMEM physically

   only the Linux Buddy

> +contiguous memory allocators will follow.
> +
> +Linux Buddy allocator porting uses Buddy specificities to reduce the overhead
> +induced by the PASR Framework counter updates. Indeed, the PASR Framework is
> +called only when MAX_ORDER (4MB page blocs by default) buddies are

                                        blocks

> +inserted/removed from the free lists.
> +
> +To port PASR FW into a new allocator:

           the PASR framework

> +
> +* Call pasr_put(phys_addr, size) each time a memory chunk becomes unused.
> +* Call pasr_get(phys_addr, size) each time a memory chunk becomes used.
> +
> +4. PASR platform drivers
> +
> +The MR16/MR17 PASR mask registers are generally accessible through the DDR
> +controller. At probe time, the DDR controller driver should register the
> +callback used by PASR Framework to apply the refresh mask for every DDR die
> +using pasr_register_mask_function(die_addr, callback, cookie).
> +
> +The callback passed to apply mask must not sleep since it can me called in

                                                             can be

> +interrupt contexts.
> +


-- 
~Randy
*** Remember to use Documentation/SubmitChecklist when testing your code ***

Re: [RFCv1 5/6] PASR: Add Documentation

[Maxime Coquelin]

On 02/02/2012 04:51 AM, Randy Dunlap wrote:
> On 01/30/2012 05:33 AM, Maxime Coquelin wrote:
>> Signed-off-by: Maxime Coquelin<maxime.coquelin@stericsson.com>
>> ---
>>   Documentation/pasr.txt |  183 ++++++++++++++++++++++++++++++++++++++++++++++++
>>   1 files changed, 183 insertions(+), 0 deletions(-)
>>   create mode 100644 Documentation/pasr.txt
>>
>> diff --git a/Documentation/pasr.txt b/Documentation/pasr.txt
>> new file mode 100644
>> index 0000000..d40e3f6
>> --- /dev/null
>> +++ b/Documentation/pasr.txt
>> @@ -0,0 +1,183 @@
>> +Partial Array Self-Refresh Framework
>> +
>> +(C) 2012 Maxime Coquelin<maxime.coquelin@stericsson.com>, ST-Ericsson.
>> +
>> +CONTENT
>> +1. Introduction
>> +2. Command-line parameters
>> +3. Allocators patching
>> +4. PASR platform drivers
>> +
>> +
>> +1. Introduction
>> +
>> +PASR Frameworks brings support for the Partial Array Self-Refresh DDR power
>    The PASR framework brings support
>
>> +management feature. PASR has been introduced in LP-DDR2, and is also present
>                              was introduced in LP-DDR2 and is also present
>
>
>> +in DDR3.
>> +
>> +PASR provides 4 modes:
>> +
>> +* Single-Ended: Only 1/1, 1/2, 1/4 or 1/8 are refreshed, masking starting at
>> +  the end of the DDR die.
>> +
>> +* Double-Ended: Same as Single-Ended, but refresh-masking does not start
>> +  necessairly at the end of the DDR die.
>       necessarily
>
>> +
>> +* Bank-Selective: Refresh of each bank of a die can be masked or unmasked via
>> +  a dedicated DDR register (MR16). This mode is convenient for DDR configured
>> +  in BRC (Bank-Row-Column) mode.
>> +
>> +* Segment-Selective: Refresh of each segment of a die can be masked or unmasked
>> +  via a dedicated DDR register (MR17). This mode is convenient for DDR configured
>> +  in RBC (Row-Bank-Column) mode.
>> +
>> +The role of this framework is to stop the refresh of unused memory to enhance
>> +DDR power consumption.
>> +
>> +It supports Bank-Selective and Segment-Selective modes, as the more adapted to
>> +modern OSes.
> huh?  parse error above.
>
>> +
>> +At early boot stage, a representation of the physical DDR layout is built:
>> +
>> +             Die 0
>> +_______________________________
>> +| I--------------------------I |
>> +| I    Bank or Segment 0     I |
>> +| I--------------------------I |
>> +| I--------------------------I |
>> +| I    Bank or Segment 1     I |
>> +| I--------------------------I |
>> +| I--------------------------I |
>> +| I    Bank or Segment ...   I |
>> +| I--------------------------I |
>> +| I--------------------------I |
>> +| I    Bank or Segment n     I |
>> +| I--------------------------I |
>> +|______________________________|
>> +             ...
>> +
>> +             Die n
>> +_______________________________
>> +| I--------------------------I |
>> +| I    Bank or Segment 0     I |
>> +| I--------------------------I |
>> +| I--------------------------I |
>> +| I    Bank or Segment 1     I |
>> +| I--------------------------I |
>> +| I--------------------------I |
>> +| I    Bank or Segment ...   I |
>> +| I--------------------------I |
>> +| I--------------------------I |
>> +| I    Bank or Segment n     I |
>> +| I--------------------------I |
>> +|______________________________|
>> +
>> +The first level is a table where elements represent a die:
>> +* Base address,
>> +* Number of segments,
>> +* Table representing banks/segments,
>> +* MR16/MR17 refresh mask,
>> +* DDR Controller callback to update MR16/MR17 refresh mask.
>> +
>> +The second level is the section tables representing the banks or segments,
>> +depending on hardware configuration:
>> +* Base address,
>> +* Unused memory size counter,
>> +* Possible pointer to another section it depends on (E.g. Interleaving)
>> +
>> +When some memory becomes unused, the allocator owning this memory calls the PASR
>> +Framework's pasr_put(phys_addr, size) function. The framework finds the
>> +sections impacted and updates their counters accordingly.
>> +If a section counter reach the section size, the refresh of the section is
>                          reaches
>
>> +masked. If the corresponding section has a dependency with another section
>> +(E.g. because of DDR interleaving, see figure below), it checks the "paired" section is also
>                                                           it checks if the "paired" section is also
>
>> +unused before updating the refresh mask.
>> +
>> +When some unused memory is requested by the allocator, the allocator owning
>> +this memory calls the PASR Framework's pasr_get(phys_addr, size) function. The
>> +framework find the section impacted and updates their counters accordingly.
>               finds                     and updates its counter accordingly.
> or
>               find the sections impacted and updates their counters accordingly.
>
>
>> +If before the update, the section counter was to the section size, the refrewh
>                                               was equal to the section size, the refresh
>
>> +of the section is unmasked. If the corresponding section has a dependency with
>> +another section, it also unmask the refresh of the other section.
>                              unmasks
>
>> +
>> +Interleaving example:
>> +
>> +             Die 0
>> +_______________________________
>> +| I--------------------------I |
>> +| I    Bank or Segment 0     I |<----|
>> +| I--------------------------I |     |
>> +| I--------------------------I |     |
>> +| I    Bank or Segment 1     I |     |
>> +| I--------------------------I |     |
>> +| I--------------------------I |     |
>> +| I    Bank or Segment ...   I |     |
>> +| I--------------------------I |     |
>> +| I--------------------------I |     |
>> +| I    Bank or Segment n     I |     |
>> +| I--------------------------I |     |
>> +|______________________________|     |
>> +                                     |
>> +             Die 1                   |
>> +_______________________________      |
>> +| I--------------------------I |     |
>> +| I    Bank or Segment 0     I |<----|
>> +| I--------------------------I |
>> +| I--------------------------I |
>> +| I    Bank or Segment 1     I |
>> +| I--------------------------I |
>> +| I--------------------------I |
>> +| I    Bank or Segment ...   I |
>> +| I--------------------------I |
>> +| I--------------------------I |
>> +| I    Bank or Segment n     I |
>> +| I--------------------------I |
>> +|______________________________|
>> +
>> +In the above example, bank 0 of die 0 is interleaved with bank0 of die 0.
>                                                               bank 0 of die 1.
>
>> +The interleaving is done in HW by inverting some addresses lines. The goal is
>                              in hardware
>
>> +to improve DDR bandwidth.
>> +Practically, one buffer seen as contiguous by the kernel might be spread
>> +into two DDR dies physically.
>> +
>> +
>> +2. Command-line parameters
>> +
>> +To buid the DDR physical layout representation, two parameters are requested:
>        build
>
>> +
>> +* ddr_die (mandatory): Should be added for every DDR dies present in the system.
>                                                          die
>
>> +   - Usage: ddr_die=xxx[M|G]@yyy[M|G] where xxx represents the size and yyy
>> +     the base address of the die. E.g.: ddr_die=512M@0 ddr_die=512M@512M
>> +
>> +* interleaved (optionnal): Should be added for every interleaved dependencies.
>                   (optional):                  for all interleaved dependencies.
>
>
>> +   - Usage: interleaved=xxx[M|G]@yyy[M|G]:zzz[M|G] where xxx is the size of
>> +     the interleaved area between the adresses yyy and zzz. E.g
>> +     interleaved=256M@0:512M
>> +
>> +
>> +3. Allocator patching
>> +
>> +Any allocators might call the PASR Framework for DDR power savings. Currently,
>> +only Linux Buddy allocator is patched, but HWMEM and PMEM physically
>     only the Linux Buddy
>
>> +contiguous memory allocators will follow.
>> +
>> +Linux Buddy allocator porting uses Buddy specificities to reduce the overhead
>> +induced by the PASR Framework counter updates. Indeed, the PASR Framework is
>> +called only when MAX_ORDER (4MB page blocs by default) buddies are
>                                          blocks
>
>> +inserted/removed from the free lists.
>> +
>> +To port PASR FW into a new allocator:
>             the PASR framework
>
>> +
>> +* Call pasr_put(phys_addr, size) each time a memory chunk becomes unused.
>> +* Call pasr_get(phys_addr, size) each time a memory chunk becomes used.
>> +
>> +4. PASR platform drivers
>> +
>> +The MR16/MR17 PASR mask registers are generally accessible through the DDR
>> +controller. At probe time, the DDR controller driver should register the
>> +callback used by PASR Framework to apply the refresh mask for every DDR die
>> +using pasr_register_mask_function(die_addr, callback, cookie).
>> +
>> +The callback passed to apply mask must not sleep since it can me called in
>                                                               can be
>
>> +interrupt contexts.
>> +
>

Thanks Randy for the review.

Regards,
Maxime

Re: [RFCv1 3/6] PASR: mm: Integrate PASR in Buddy allocator

[Mel Gorman]

On Tue, Jan 31, 2012 at 07:55:47PM +0100, Maxime Coquelin wrote:
> >>To avoid calling pasr_kget/kput() directly in page_alloc.c, do you
> >>think adding some arch specific hooks when a page is inserted or
> >>removed from the free lists could be acceptable?
> >It's not the name that is the problem, I'm strongly against any hook
> >that can delay the page allocator for arbitrary lengths of time like
> >this. I am open to being convinced otherwise but for me PASR would
> >need to demonstrate large savings for a wide variety of machines and
> >the alternatives would have to be considered and explained why they
> >would be far inferior or unsuitable.
>
> Ok Mel, I understand your point of view.
> 
> The goal of this RFC patch set was to collect comments, so I'm glad
> to get your opinion.
> I propose to forget the patch in the Buddy allocator.
> 

Or at least tag it as "this should have an alternative"

> >For example - it seems like this could be also be done with a
> >balloon driver instead of page allocator hooks. A governer would
> >identify when the machine was under no memory pressure or triggered
> >from userspace. To power down memory, it would use page reclaim and
> >page migration to allocate large contiguous ranges of memory - CMA
> >could potentially be adapted when it gets merged to save a lot of
> >implementation work. The governer should register a slab shrinker
> >so that under memory pressure it gets called so it can shrink the
> >ballon, power the DIMMS back up and free the memory back to the
> >buddy allocator. This would keep all the cost out of the allocator
> >paths and move the cost to when the machine is either idle (in the
> >case of powering down) or under memory pressure (where the cost of
> >powering up will be small in comparison to the overall cost of the
> >page reclaim operation).
> >
>
> This is very interesting.
> I know Linaro plans to work on DDR power management topic.
> One of the options they envisage is to use the Memory Hotplug feature.
> However, the main problem with Memory Hotplug is to handle the
> memory pressure, i.e. when to re-plug the memory sections.

heh, I was originally going to suggest the Memory Hotplug feature until
I ran into the problem of how to bring the memory back. Technically,
it could *also* use a shrinker and keep track of the memory it
offlined. It would work on a similar principal to the balloon but I
was worried that the cost of offlining and onlining memory would be
too high for PASR. There was also the problem that it would require
SPARSEMEM and would also require that you grew/shrunk memory in ranges
of the section size which might be totally different to PASR.

> Your proposal address this issue. I don't know if such a driver
> could be done in the Linaro scope.
> 

Beats me.

> Anyway, even with a balloon driver, I think the PASR framework could
> be suitable to keep an "hardware" view of the memory layout (dies,
> banks, segments...).

Oh yes, the balloon driver would still need this information!

> Moreover, this framework is designed to also support some physically
> contiguous memory allocators (such as hwmem and pmem).
> 

Not being familiar with hwmem or pmem, I can't be 100% certain but
superficially, I would expect that the same balloon driver could be used
for hwmem and pmem. The main difference between this balloon driver and
others will be how it selects pages to add to the balloon.

There are existing balloon drivers that you may or may not be able to
leverage. There is some talk that KVM people want to be able to balloon
2M contiguous pages. If this was ever implemented, it's possible that
you could reuse it for PASR so keep an eye out for it.

-- 
Mel Gorman
SUSE Labs