[PATCH v8 0/7] powerpc/powernv: Nest Instrumentation support

[PATCH v8 0/7] powerpc/powernv: Nest Instrumentation support

[Madhavan Srinivasan]

This patchset enables Nest Instrumentation support on powerpc.
POWER8 has per-chip Nest Intrumentation which provides various
per-chip metrics like memory, powerbus, Xlink and Alink
bandwidth.

Nest Instrumentation provides an interface (via PORE Engine)
to configure and move the nest counter data to memory. From
kernel side, OPAL Call interface is used to activate/deactivate
PORE Engine for nest data collection.

OPAL at boot, detects the feature, initializes it and pass on
the nest units and other related information such as memory
region, events supported so on, to kernel via device-tree.

Kernel code then, parses the device-tree for nest pmu support
and registers nest pmu with the events available. PORE Engine collects
and accumulate nest counter data in per-chip reserved memory region, hence
device-tree also exports per-chip nest accumulation memory region.
And individual event offset are used as event configuration values.

Here is sample perf usage to explain the interface.

#./perf list
....
  iTLB-load-misses                                   [Hardware cache event]

  Nest_Alink_BW/Alink0/                              [Kernel PMU event]
  Nest_Alink_BW/Alink1/                              [Kernel PMU event]
  Nest_Alink_BW/Alink2/                              [Kernel PMU event]
  Nest_MCS_Read_BW/MCS_00/                           [Kernel PMU event]
  Nest_MCS_Read_BW/MCS_01/                           [Kernel PMU event]
  Nest_MCS_Read_BW/MCS_02/                           [Kernel PMU event]
  Nest_MCS_Read_BW/MCS_03/                           [Kernel PMU event]
  Nest_MCS_Write_BW/MCS_00/                          [Kernel PMU event]
  Nest_MCS_Write_BW/MCS_01/                          [Kernel PMU event]
  Nest_MCS_Write_BW/MCS_02/                          [Kernel PMU event]
  Nest_MCS_Write_BW/MCS_03/                          [Kernel PMU event]
  Nest_PowerBus_BW/External/                         [Kernel PMU event]
  Nest_PowerBus_BW/Internal/                         [Kernel PMU event]
  Nest_Xlink_BW/Xlink0/                              [Kernel PMU event]
  Nest_Xlink_BW/Xlink1/                              [Kernel PMU event]
  Nest_Xlink_BW/Xlink2/                              [Kernel PMU event]

  rNNN                                               [Raw hardware event descriptor]
  cpu/t1=v1[,t2=v2,t3 ...]/modifier                  [Raw hardware event descriptor]
.....

# ./perf stat -e 'Nest_Xlink_BW/Xlink1/' -a -A sleep 1

 Performance counter stats for 'system wide':

CPU0             15,913.18 MiB  Nest_Xlink_BW/Xlink1/
CPU32            11,955.88 MiB  Nest_Xlink_BW/Xlink1/
CPU64            11,042.43 MiB  Nest_Xlink_BW/Xlink1/
CPU96            14,065.27 MiB  Nest_Xlink_BW/Xlink1/

       1.001062038 seconds time elapsed


# ./perf stat -e 'Nest_Alink_BW/Alink0/,Nest_Alink_BW/Alink1/,Nest_Alink_BW/Alink2/' -a -A -I 1000 sleep 5

 Performance counter stats for 'system wide':

CPU0                  0.00 MiB  Nest_Alink_BW/Alink0/                                         (100.00%)
CPU32                 0.00 MiB  Nest_Alink_BW/Alink0/                                         (100.00%)
CPU64                 0.00 MiB  Nest_Alink_BW/Alink0/                                         (100.00%)
CPU96                 0.00 MiB  Nest_Alink_BW/Alink0/                                         (100.00%)
CPU0              1,430.43 MiB  Nest_Alink_BW/Alink1/                                         (100.00%)
CPU32               320.99 MiB  Nest_Alink_BW/Alink1/                                         (100.00%)
CPU64             3,443.83 MiB  Nest_Alink_BW/Alink1/                                         (100.00%)
CPU96             1,904.41 MiB  Nest_Alink_BW/Alink1/                                         (100.00%)
CPU0              2,856.85 MiB  Nest_Alink_BW/Alink2/
CPU32                 7.50 MiB  Nest_Alink_BW/Alink2/
CPU64             4,034.29 MiB  Nest_Alink_BW/Alink2/
CPU96               288.49 MiB  Nest_Alink_BW/Alink2/
.....

OPAL side patches (v6) are posted in the skiboot mailing list.

https://lists.ozlabs.org/pipermail/skiboot/2016-February/002824.html
https://lists.ozlabs.org/pipermail/skiboot/2016-February/002825.html
https://lists.ozlabs.org/pipermail/skiboot/2016-February/002826.html
https://lists.ozlabs.org/pipermail/skiboot/2016-February/002833.html
https://lists.ozlabs.org/pipermail/skiboot/2016-February/002832.html
https://lists.ozlabs.org/pipermail/skiboot/2016-February/002831.html
https://lists.ozlabs.org/pipermail/skiboot/2016-February/002829.html
https://lists.ozlabs.org/pipermail/skiboot/2016-February/002827.html
https://lists.ozlabs.org/pipermail/skiboot/2016-February/002828.html
https://lists.ozlabs.org/pipermail/skiboot/2016-February/002830.html

Changelog from v7:
1)Rebased to latest upstream code.
2)Separated the device tree parser code and have created a new file "opal-nest.c"
in powernv/ folder. Have added a autoload function using a compatible string as match id.
3)Renamed variables and function names to skip "P8", since the nest pmu code is not
specific to POWER8 processor.
4)Updated comments and commit messages

Changelog from v6:

1) Modified the OPAL call number to sync with OPAL side changes.
2) Rebased to latest upstream.

Changelog from v5:

1) Replaced be32_to_cpup with of_property_read_u32 when reading from device tree
2) Replaced uintxx_t to u[32/64] variable types
3) Broken the memory location for event attribute group and events into
   two alloc functions to have it simple.
4) Added separate functions for event string and event value as suggested.
5) Added P8_NEST_MAX_PMUS check for array overflow
6) Made changes to commit messages.

Changelog from v4:

1) Variable name changes for consistency and added more comments
2) Added sysfs_att_init to have lockdep happy
3) Updated OPAL Call interface changes and added code to handle
   failure case.
4) Added new macro "P8_NEST_MODE_PRODUCTION" to specify PORE Engine mode
5) Modified nest_pmu_cpumask_init function to return value to
   nest pmu init function incase of OPAL call failure.

Changelog from v3:

No logic change, just a rebase to latest upstream kernel.

Changelog from v2:

1) Changed variable and macro names to be consistent.
2) Made changes to commit message and code comment messages
3) Moved "format attribute" related code from patch 6 to 5
4) Added check for pmu register function
5) Changed cpu_init and cpu_exit functions to use first online
   cpu of the chip, there by making code lot simplier.

Changelog from v1:

1) No logic changes, re-ordered patches make each patch compile
   without errors
2) Added comments based on the review feedback.
3) removed perf_event_del function and replaced it with perf_event_stop.
4) Moved Nest feature detection code out of parser function.
5) Optimized functions and removed some variables.
6) squashed the makefile changes, instead of the separate patch
7) squashed the cpumask and hotplug patches as single patch
8) Added cpu checks in nest_change_cpu_context and nest_exit_cpu functions
9) Made changes to commit messages.

Changelog from RFC:

1) Removed "uncore" code and made each Nest Unit a separate PMU.
2) Removed uncore type abstraction and uncore related functions.
3) Added simple cpumask function since these are per-chip counters
4) Redesigned device-tree parser based on the latest platform enablement code
5) Made changes to commit message

Thanks to input from Sukadev Bhattiprolu, Preeti Murthy, Daniel Axtens,
Suzuki Poulose and Michael Ellerman

Kindly let me know you comments and feedback.

Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Anton Blanchard <anton@samba.org>
Cc: Daniel Axtens <dja@axtens.net>
Cc: Stephane Eranian <eranian@google.com>
Cc: Sukadev Bhattiprolu <sukadev@linux.vnet.ibm.com>
Signed-off-by: Madhavan Srinivasan <maddy@linux.vnet.ibm.com>


Madhavan Srinivasan (7):
  powerpc/powernv: Data structure and macros definition
  powerpc/powernv: Add OPAL support for Nest PMU
  powerpc/powernv: autoload nest unit driver module
  powerpc/powernv: detect supported nest units and its events
  powerpc/perf: Add event attribute and group to nest pmu
  powerpc/perf: generic nest pmu event functions
  powerpc/perf: nest pmu cpumask and cpu hotplug support

 arch/powerpc/include/asm/nest-pmu.h            |  55 ++++
 arch/powerpc/include/asm/opal-api.h            |   3 +-
 arch/powerpc/include/asm/opal.h                |   3 +
 arch/powerpc/perf/Makefile                     |   2 +-
 arch/powerpc/perf/nest-pmu.c                   | 354 +++++++++++++++++++++++++
 arch/powerpc/platforms/powernv/Makefile        |   2 +-
 arch/powerpc/platforms/powernv/opal-nest.c     | 293 ++++++++++++++++++++
 arch/powerpc/platforms/powernv/opal-wrappers.S |   1 +
 arch/powerpc/platforms/powernv/opal.c          |  12 +
 9 files changed, 722 insertions(+), 3 deletions(-)
 create mode 100644 arch/powerpc/include/asm/nest-pmu.h
 create mode 100644 arch/powerpc/perf/nest-pmu.c
 create mode 100644 arch/powerpc/platforms/powernv/opal-nest.c

-- 
1.9.1

[PATCH v8 1/7] powerpc/powernv: Data structure and macros definition

[Madhavan Srinivasan]

Create new header file "nest-pmu.h" to add the data structures
and macros needed for the nest pmu support.

Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Anton Blanchard <anton@samba.org>
Cc: Daniel Axtens <dja@axtens.net>
Cc: Stephane Eranian <eranian@google.com>
Cc: Sukadev Bhattiprolu <sukadev@linux.vnet.ibm.com>
Signed-off-by: Madhavan Srinivasan <maddy@linux.vnet.ibm.com>
---
 arch/powerpc/include/asm/nest-pmu.h | 55 +++++++++++++++++++++++++++++++++++++
 1 file changed, 55 insertions(+)
 create mode 100644 arch/powerpc/include/asm/nest-pmu.h

diff --git a/arch/powerpc/include/asm/nest-pmu.h b/arch/powerpc/include/asm/nest-pmu.h
new file mode 100644
index 000000000000..13c21817e3cc
--- /dev/null
+++ b/arch/powerpc/include/asm/nest-pmu.h
@@ -0,0 +1,55 @@
+/*
+ * Nest Performance Monitor counter support.
+ *
+ * Copyright (C) 2016 Madhavan Srinivasan, IBM Corporation.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License version 2 as published
+ * by the Free Software Foundation.
+ */
+
+#include <linux/perf_event.h>
+#include <linux/slab.h>
+#include <linux/of.h>
+#include <linux/io.h>
+#include <asm/opal.h>
+
+#define NEST_MAX_CHIPS			32
+#define NEST_MAX_PMUS			32
+#define NEST_MAX_PMU_NAME_LEN		256
+#define NEST_MAX_EVENTS_SUPPORTED	64
+#define NEST_ENGINE_START		1
+#define NEST_ENGINE_STOP		0
+#define NEST_MODE_PRODUCTION		1
+#define NEST_MAX_PAGES			16
+
+/*
+ * Structure to hold per chip specific memory address
+ * information for nest pmus. Nest Counter data are exported
+ * in per-chip reserved memory region by the PORE Engine.
+ */
+struct perchip_nest_info {
+	u32 chip_id;
+	u64 pbase;
+	u64 vbase[NEST_MAX_PAGES];
+	u32 size;
+};
+
+/*
+ * Place holder for nest pmu events and values.
+ */
+struct nest_ima_events {
+	char *ev_name;
+	char *ev_value;
+};
+
+/*
+ * Device tree parser code detects nest pmu support and
+ * registers new nest pmus. This structure will
+ * hold the pmu functions and attrs for each nest pmu and
+ * will be referenced at the time of pmu registration.
+ */
+struct nest_pmu {
+	struct pmu pmu;
+	const struct attribute_group *attr_groups[4];
+};
-- 
1.9.1

[PATCH v8 2/7] powerpc/powernv: Add OPAL support for Nest PMU

[Madhavan Srinivasan]

Nest Counters can be configured via PORE Engine and OPAL
provides an interface to start/stop it.

Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Anton Blanchard <anton@samba.org>
Cc: Daniel Axtens <dja@axtens.net>
Cc: Stephane Eranian <eranian@google.com>
Cc: Sukadev Bhattiprolu <sukadev@linux.vnet.ibm.com>
Signed-off-by: Madhavan Srinivasan <maddy@linux.vnet.ibm.com>
---
OPAL side patches are posted in the skiboot mailing.

 arch/powerpc/include/asm/opal-api.h            | 3 ++-
 arch/powerpc/include/asm/opal.h                | 3 +++
 arch/powerpc/platforms/powernv/opal-wrappers.S | 1 +
 3 files changed, 6 insertions(+), 1 deletion(-)

diff --git a/arch/powerpc/include/asm/opal-api.h b/arch/powerpc/include/asm/opal-api.h
index f8faaaeeca1e..1845af7917a4 100644
--- a/arch/powerpc/include/asm/opal-api.h
+++ b/arch/powerpc/include/asm/opal-api.h
@@ -158,7 +158,8 @@
 #define OPAL_LEDS_SET_INDICATOR			115
 #define OPAL_CEC_REBOOT2			116
 #define OPAL_CONSOLE_FLUSH			117
-#define OPAL_LAST				117
+#define OPAL_NEST_COUNTERS_CONTROL		118
+#define OPAL_LAST				118
 
 /* Device tree flags */
 
diff --git a/arch/powerpc/include/asm/opal.h b/arch/powerpc/include/asm/opal.h
index 07a99e638449..6556e2a2141c 100644
--- a/arch/powerpc/include/asm/opal.h
+++ b/arch/powerpc/include/asm/opal.h
@@ -210,6 +210,9 @@ int64_t opal_flash_write(uint64_t id, uint64_t offset, uint64_t buf,
 int64_t opal_flash_erase(uint64_t id, uint64_t offset, uint64_t size,
 		uint64_t token);
 
+int64_t opal_nest_counters_control(uint64_t mode, uint64_t value1,
+		uint64_t value2, uint64_t value3);
+
 /* Internal functions */
 extern int early_init_dt_scan_opal(unsigned long node, const char *uname,
 				   int depth, void *data);
diff --git a/arch/powerpc/platforms/powernv/opal-wrappers.S b/arch/powerpc/platforms/powernv/opal-wrappers.S
index e45b88a5d7e0..193fed741a53 100644
--- a/arch/powerpc/platforms/powernv/opal-wrappers.S
+++ b/arch/powerpc/platforms/powernv/opal-wrappers.S
@@ -302,3 +302,4 @@ OPAL_CALL(opal_prd_msg,				OPAL_PRD_MSG);
 OPAL_CALL(opal_leds_get_ind,			OPAL_LEDS_GET_INDICATOR);
 OPAL_CALL(opal_leds_set_ind,			OPAL_LEDS_SET_INDICATOR);
 OPAL_CALL(opal_console_flush,			OPAL_CONSOLE_FLUSH);
+OPAL_CALL(opal_nest_counters_control,		OPAL_NEST_COUNTERS_CONTROL);
-- 
1.9.1

[PATCH v8 3/7] powerpc/powernv: autoload nest unit driver module

[Madhavan Srinivasan]

Device tree parsing of nest units and events are
separated from pmu code and are placed in powernv/
as "opal-nest.c". Makefile is updated to up pick
the file.

Code added to "opal.c" to create a platform device
for the nest unit. Nest device module code with a
probe function to get the nest counter memory address
information for each chip is added in this patch.

"ibm,opal-in-memory-counters" compatible string is
used for autoload device_id match.

Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Anton Blanchard <anton@samba.org>
Cc: Daniel Axtens <dja@axtens.net>
Cc: Stephane Eranian <eranian@google.com>
Cc: Sukadev Bhattiprolu <sukadev@linux.vnet.ibm.com>
Signed-off-by: Madhavan Srinivasan <maddy@linux.vnet.ibm.com>
---
 arch/powerpc/platforms/powernv/Makefile    |  2 +-
 arch/powerpc/platforms/powernv/opal-nest.c | 97 ++++++++++++++++++++++++++++++
 arch/powerpc/platforms/powernv/opal.c      | 12 ++++
 3 files changed, 110 insertions(+), 1 deletion(-)
 create mode 100644 arch/powerpc/platforms/powernv/opal-nest.c

diff --git a/arch/powerpc/platforms/powernv/Makefile b/arch/powerpc/platforms/powernv/Makefile
index f1516b5ecec9..ffc54cd49656 100644
--- a/arch/powerpc/platforms/powernv/Makefile
+++ b/arch/powerpc/platforms/powernv/Makefile
@@ -2,7 +2,7 @@ obj-y			+= setup.o opal-wrappers.o opal.o opal-async.o idle.o
 obj-y			+= opal-rtc.o opal-nvram.o opal-lpc.o opal-flash.o
 obj-y			+= rng.o opal-elog.o opal-dump.o opal-sysparam.o opal-sensor.o
 obj-y			+= opal-msglog.o opal-hmi.o opal-power.o opal-irqchip.o
-obj-y			+= opal-kmsg.o
+obj-y			+= opal-kmsg.o opal-nest.o
 
 obj-$(CONFIG_SMP)	+= smp.o subcore.o subcore-asm.o
 obj-$(CONFIG_PCI)	+= pci.o pci-p5ioc2.o pci-ioda.o npu-dma.o
diff --git a/arch/powerpc/platforms/powernv/opal-nest.c b/arch/powerpc/platforms/powernv/opal-nest.c
new file mode 100644
index 000000000000..2ca879fdf159
--- /dev/null
+++ b/arch/powerpc/platforms/powernv/opal-nest.c
@@ -0,0 +1,97 @@
+/*
+ * OPAL Nest detection interface driver
+ * Supported on POWERNV platform
+ *
+ * Copyright IBM Corporation 2016
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/platform_device.h>
+#include <linux/miscdevice.h>
+#include <linux/fs.h>
+#include <linux/of.h>
+#include <linux/of_address.h>
+#include <linux/of_platform.h>
+#include <linux/poll.h>
+#include <linux/mm.h>
+#include <linux/slab.h>
+#include <asm/opal.h>
+#include <asm/io.h>
+#include <asm/uaccess.h>
+#include <asm/nest-pmu.h>
+
+extern struct perchip_nest_info nest_perchip_info[NEST_MAX_CHIPS];
+
+static int opal_nest_counters_probe(struct platform_device *pdev)
+{
+	struct device_node *child, *parent;
+	struct perchip_nest_info *pcni;
+	u32 idx, range[4], pages;
+	int rc=0, i=0;
+
+	if (!pdev || !pdev->dev.of_node)
+		return -ENODEV;
+
+	/*
+	 * "nest-counters" folder contains two things,
+	 * a) per-chip reserved memory region for Nest PMU Counter data
+	 * b) Support Nest PMU units and their event files
+	 */
+	parent = pdev->dev.of_node;
+	for_each_child_of_node(parent, child) {
+		if (of_property_read_u32(child, "ibm,chip-id", &idx)) {
+			pr_err("opal-nest-counters: device %s missing property\n",
+							child->full_name);
+			return -ENODEV;
+		}
+
+		/*
+		 *"ranges" property will have four u32 cells.
+		 */
+		if (of_property_read_u32_array(child, "ranges", range, 4)) {
+			pr_err("opal-nest-counters: range property value wrong\n");
+			return -1;
+		}
+
+		pcni = &nest_perchip_info[idx];
+		pcni->pbase = range[1];
+		pcni->pbase = pcni->pbase << 32 | range[2];
+		pcni->size = range[3];
+
+		do
+		{
+			pages = PAGE_SIZE * i;
+			pcni->vbase[i++] = (u64)phys_to_virt(pcni->pbase + pages);
+		} while( i < (pcni->size/PAGE_SIZE));
+	}
+
+	return rc;
+}
+
+static const struct of_device_id opal_nest_match[] = {
+	{ .compatible = "ibm,opal-in-memory-counters" },
+	{ },
+};
+
+static struct platform_driver opal_nest_driver = {
+	.driver = {
+		.name           = "opal-nest-counters",
+		.of_match_table = opal_nest_match,
+	},
+	.probe  = opal_nest_counters_probe,
+};
+
+MODULE_DEVICE_TABLE(of, opal_nest_match);
+module_platform_driver(opal_nest_driver);
+MODULE_DESCRIPTION("PowerNV OPAL Nest Counters driver");
+MODULE_LICENSE("GPL");
diff --git a/arch/powerpc/platforms/powernv/opal.c b/arch/powerpc/platforms/powernv/opal.c
index 4e0da5af94a1..60b576d1fc94 100644
--- a/arch/powerpc/platforms/powernv/opal.c
+++ b/arch/powerpc/platforms/powernv/opal.c
@@ -651,6 +651,15 @@ static void opal_i2c_create_devs(void)
 		of_platform_device_create(np, NULL, NULL);
 }
 
+static void opal_nest_create_dev(void)
+{
+	struct device_node *np;
+
+	np = of_find_compatible_node(NULL, NULL, "ibm,opal-in-memory-counters");
+	if (np)
+		of_platform_device_create(np, NULL, NULL);
+}
+
 static int kopald(void *unused)
 {
 	__be64 events;
@@ -717,6 +726,9 @@ static int __init opal_init(void)
 	/* Setup a heatbeat thread if requested by OPAL */
 	opal_init_heartbeat();
 
+	/* Create nest platform devices */
+	opal_nest_create_dev();
+
 	/* Create leds platform devices */
 	leds = of_find_node_by_path("/ibm,opal/leds");
 	if (leds) {
-- 
1.9.1

[PATCH v8 4/7] powerpc/powernv: detect supported nest units and its events

[Madhavan Srinivasan]

Parse device tree to detect nest units. Traverse through
each nest unit folder to find supported events and
corresponding unit/scale files (if any).

The nest unit event file from Device Tree will contain the offset in the
reserved memory region to get the counter data for a given event.
Kernel code uses this offset as event configuration value.

Device tree parser code also looks for scale/unit in the file name and
passes on the file as an event attr for perf tool to use in the post
processing.

Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Anton Blanchard <anton@samba.org>
Cc: Daniel Axtens <dja@axtens.net>
Cc: Stephane Eranian <eranian@google.com>
Cc: Sukadev Bhattiprolu <sukadev@linux.vnet.ibm.com>
Signed-off-by: Madhavan Srinivasan <maddy@linux.vnet.ibm.com>
---
 arch/powerpc/platforms/powernv/opal-nest.c | 190 ++++++++++++++++++++++++++++-
 1 file changed, 189 insertions(+), 1 deletion(-)

diff --git a/arch/powerpc/platforms/powernv/opal-nest.c b/arch/powerpc/platforms/powernv/opal-nest.c
index 2ca879fdf159..548a8c0236e4 100644
--- a/arch/powerpc/platforms/powernv/opal-nest.c
+++ b/arch/powerpc/platforms/powernv/opal-nest.c
@@ -31,13 +31,194 @@
 #include <asm/nest-pmu.h>
 
 extern struct perchip_nest_info nest_perchip_info[NEST_MAX_CHIPS];
+extern struct nest_pmu *per_nest_pmu_arr[NEST_MAX_PMUS];
+
+static int nest_event_info(char *name, struct nest_ima_events *nest_events)
+{
+	char *buf;
+
+	/* memory for content */
+	buf = kzalloc(NEST_MAX_PMU_NAME_LEN, GFP_KERNEL);
+	if (!buf)
+		return -ENOMEM;
+
+	nest_events->ev_name = name;
+	nest_events->ev_value = buf;
+	return 0;
+}
+
+static int nest_event_info_str(struct property *pp, char *name,
+					struct nest_ima_events *nest_events)
+{
+	if (nest_event_info(name, nest_events))
+		return -ENOMEM;
+
+	if (!pp->value || (strnlen(pp->value, pp->length) == pp->length) ||
+	   (pp->length > NEST_MAX_PMU_NAME_LEN))
+		return -EINVAL;
+
+	strncpy(nest_events->ev_value, (const char *)pp->value, pp->length);
+	return 0;
+}
+
+static int nest_event_info_val(char *name, u32 val,
+					struct nest_ima_events *nest_events)
+{
+	if (nest_event_info(name, nest_events))
+		return -ENOMEM;
+
+	sprintf(nest_events->ev_value, "event=0x%x", val);
+	return 0;
+}
+
+static int nest_events_node_parser(struct device_node *dev,
+					struct nest_ima_events *nest_events)
+{
+	struct property *name, *pp, *id;
+	char *buf, *start, *ev_name;
+	u32 val;
+	int idx = 0, ret;
+
+	if (!dev)
+		return -EINVAL;
+
+	/*
+	* Loop through each property
+	*/
+	name = of_find_property(dev, "name", NULL);
+	if (!name) {
+		printk(KERN_INFO "No property by name\n");
+		return -1;
+	}
+
+	if (!name->value ||
+	  (strnlen(name->value, name->length) == name->length) ||
+	  (name->length > NEST_MAX_PMU_NAME_LEN))
+		return -EINVAL;
+
+	ev_name = kzalloc(NEST_MAX_PMU_NAME_LEN, GFP_KERNEL);
+	if (!ev_name)
+		return -ENOMEM;
+
+	/* Now that we got the event name, look for id */
+	id = of_find_property(dev, "id", NULL);
+	if (!id) {
+		strncpy(ev_name, name->value, (int)strlen(name->value));
+		printk(KERN_INFO "No property by id = %s\n", ev_name);
+	} else {
+		if (!id->value ||
+		  (strnlen(id->value, id->length) == id->length) ||
+		  (id->length > NEST_MAX_PMU_NAME_LEN))
+			return -EINVAL;
+
+		of_property_read_u32(dev, id->name, &val);
+		sprintf(ev_name, "%s_%x", (char *)name->value, val);
+	}
+
+	for_each_property_of_node(dev, pp) {
+		start = pp->name;
+
+		/* Skip these, we don't need it */
+		if (!strcmp(pp->name, "phandle") ||
+		   !strcmp(pp->name, "linux,phandle") ||
+		   !strcmp(pp->name, "name"))
+			continue;
+
+		if (strncmp(pp->name, "reg", 3) == 0) {
+			of_property_read_u32(dev, pp->name, &val);
+			ret = nest_event_info_val(ev_name, val, &nest_events[idx]);
+			idx++;
+		} else if (strncmp(pp->name, "unit", 4) == 0) {
+			buf = kzalloc(NEST_MAX_PMU_NAME_LEN, GFP_KERNEL);
+			if (!buf)
+				return -ENOMEM;
+			sprintf(buf,"%s.unit", ev_name);
+			ret = nest_event_info_str(pp, buf, &nest_events[idx]);
+			idx++;
+		} else if (strncmp(pp->name, "scale", 5) == 0) {
+			buf = kzalloc(NEST_MAX_PMU_NAME_LEN, GFP_KERNEL);
+			if (!buf)
+				return -ENOMEM;
+			sprintf(buf,"%s.scale", ev_name);
+			ret = nest_event_info_str(pp, buf, &nest_events[idx]);
+			idx++;
+		}
+
+		if (ret)
+			return ret;
+	}
+
+	return idx;
+}
+
+static int nest_pmu_create(struct device_node *parent, int pmu_index)
+{
+	struct device_node *ev_node;
+	struct nest_ima_events *nest_events;
+	struct nest_pmu *pmu_ptr;
+	struct property *pp;
+	char *buf;
+	int idx = 0, ret;
+
+	if (!parent)
+		return -EINVAL;
+
+	/* memory for nest pmus */
+	pmu_ptr = kzalloc(sizeof(struct nest_pmu), GFP_KERNEL);
+	if (!pmu_ptr)
+		return -ENOMEM;
+
+	/* Needed for hotplug/migration */
+	per_nest_pmu_arr[pmu_index] = pmu_ptr;
+
+	/* memory for nest pmu events */
+	nest_events = kzalloc((sizeof(struct nest_ima_events) *
+					NEST_MAX_EVENTS_SUPPORTED), GFP_KERNEL);
+	if (!nest_events)
+		return -ENOMEM;
+
+	pp = of_find_property(parent, "name", NULL);
+	if (!pp) {
+		printk(KERN_INFO "No property by name\n");
+		return -1;
+	}
+
+	if (!pp->value ||
+	  (strnlen(pp->value, pp->length) == pp->length) ||
+	  (pp->length > NEST_MAX_PMU_NAME_LEN))
+		return -EINVAL;
+
+	buf = kzalloc(NEST_MAX_PMU_NAME_LEN, GFP_KERNEL);
+	if (!buf)
+		return -ENOMEM;
+
+	/* Save the name to register it later */
+	sprintf(buf, "nest_%s", (char *)pp->value);
+	pmu_ptr->pmu.name = (char *)buf;
+
+	/* Loop through event nodes */
+	for_each_child_of_node(parent, ev_node) {
+		ret = nest_events_node_parser(ev_node, &nest_events[idx]);
+		if (ret < 0)
+			return -1;
+
+		/*
+		 * nest_event_node_parser will return number of
+		 * event entried created for this. This could include
+		 * event scale and unit files also.
+		 */
+		idx += ret;
+	}
+
+	return 0;
+}
 
 static int opal_nest_counters_probe(struct platform_device *pdev)
 {
 	struct device_node *child, *parent;
 	struct perchip_nest_info *pcni;
 	u32 idx, range[4], pages;
-	int rc=0, i=0;
+	int rc=0, i=0, pmu_count=0;
 
 	if (!pdev || !pdev->dev.of_node)
 		return -ENODEV;
@@ -75,6 +256,13 @@ static int opal_nest_counters_probe(struct platform_device *pdev)
 		} while( i < (pcni->size/PAGE_SIZE));
 	}
 
+	parent = of_get_next_child(pdev->dev.of_node, NULL);
+	for_each_child_of_node(parent, child) {
+		rc = nest_pmu_create(child, pmu_count++);
+		if (rc)
+			return rc;
+	}
+
 	return rc;
 }
 
-- 
1.9.1

[PATCH v8 5/7] powerpc/perf: Add event attribute and group to nest pmu

[Madhavan Srinivasan]

Device tree nest driver module parses the nest unit and its events.
It pass the information to nest pmu code which is placed in
perf/ as "nest-pmu.c".

Inthis patch code added to create only event attributes and
attribute groups for the nest pmu.

Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Anton Blanchard <anton@samba.org>
Cc: Daniel Axtens <dja@axtens.net>
Cc: Stephane Eranian <eranian@google.com>
Cc: Sukadev Bhattiprolu <sukadev@linux.vnet.ibm.com>
Signed-off-by: Madhavan Srinivasan <maddy@linux.vnet.ibm.com>
---
 arch/powerpc/perf/Makefile                 |  2 +-
 arch/powerpc/perf/nest-pmu.c               | 69 ++++++++++++++++++++++++++++++
 arch/powerpc/platforms/powernv/opal-nest.c |  8 ++++
 3 files changed, 78 insertions(+), 1 deletion(-)
 create mode 100644 arch/powerpc/perf/nest-pmu.c

diff --git a/arch/powerpc/perf/Makefile b/arch/powerpc/perf/Makefile
index f9c083a5652a..6da656b50e3c 100644
--- a/arch/powerpc/perf/Makefile
+++ b/arch/powerpc/perf/Makefile
@@ -5,7 +5,7 @@ obj-$(CONFIG_PERF_EVENTS)	+= callchain.o
 obj-$(CONFIG_PPC_PERF_CTRS)	+= core-book3s.o bhrb.o
 obj64-$(CONFIG_PPC_PERF_CTRS)	+= power4-pmu.o ppc970-pmu.o power5-pmu.o \
 				   power5+-pmu.o power6-pmu.o power7-pmu.o \
-				   power8-pmu.o
+				   power8-pmu.o nest-pmu.o
 obj32-$(CONFIG_PPC_PERF_CTRS)	+= mpc7450-pmu.o
 
 obj-$(CONFIG_FSL_EMB_PERF_EVENT) += core-fsl-emb.o
diff --git a/arch/powerpc/perf/nest-pmu.c b/arch/powerpc/perf/nest-pmu.c
new file mode 100644
index 000000000000..b9b44147c9b8
--- /dev/null
+++ b/arch/powerpc/perf/nest-pmu.c
@@ -0,0 +1,69 @@
+/*
+ * Nest Performance Monitor counter support.
+ *
+ * Copyright (C) 2016 Madhavan Srinivasan, IBM Corporation.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License version 2 as published
+ * by the Free Software Foundation.
+ */
+
+#include <asm/nest-pmu.h>
+
+struct perchip_nest_info nest_perchip_info[NEST_MAX_CHIPS];
+struct nest_pmu *per_nest_pmu_arr[NEST_MAX_PMUS];
+
+/*
+ * Populate event name and string in attribute
+ */
+static struct attribute *dev_str_attr(const char *name, const char *str)
+{
+	struct perf_pmu_events_attr *attr;
+
+	attr = kzalloc(sizeof(*attr), GFP_KERNEL);
+
+	sysfs_attr_init(&attr->attr.attr);
+
+	attr->event_str = str;
+	attr->attr.attr.name = name;
+	attr->attr.attr.mode = 0444;
+	attr->attr.show = perf_event_sysfs_show;
+
+	return &attr->attr.attr;
+}
+
+static int update_events_in_group(
+	struct nest_ima_events *nest_events, int idx, struct nest_pmu *pmu)
+{
+	struct attribute_group *attr_group;
+	struct attribute **attrs;
+	int i;
+
+	/* Allocate memory for attribute group */
+	attr_group = kzalloc(sizeof(*attr_group), GFP_KERNEL);
+	if (!attr_group)
+		return -ENOMEM;
+
+	/* Allocate memory for attributes */
+	attrs = kzalloc((sizeof(struct attribute *) * (idx + 1)), GFP_KERNEL);
+	if (!attrs)
+		return -ENOMEM;
+
+	attr_group->name = "events";
+	attr_group->attrs = attrs;
+	for (i = 0; i < idx; i++, nest_events++)
+		attrs[i] = dev_str_attr((char *)nest_events->ev_name,
+						(char *)nest_events->ev_value);
+
+	pmu->attr_groups[0] = attr_group;
+	return 0;
+}
+
+int init_nest_pmu(struct nest_ima_events *nest_events,
+					int idx, struct nest_pmu *pmu_ptr)
+{
+
+	update_events_in_group(nest_events, idx, pmu_ptr);
+	return 0;
+}
+
diff --git a/arch/powerpc/platforms/powernv/opal-nest.c b/arch/powerpc/platforms/powernv/opal-nest.c
index 548a8c0236e4..6ad260353d04 100644
--- a/arch/powerpc/platforms/powernv/opal-nest.c
+++ b/arch/powerpc/platforms/powernv/opal-nest.c
@@ -32,6 +32,8 @@
 
 extern struct perchip_nest_info nest_perchip_info[NEST_MAX_CHIPS];
 extern struct nest_pmu *per_nest_pmu_arr[NEST_MAX_PMUS];
+extern int init_nest_pmu(struct nest_ima_events *nest_events,
+				int idx, struct nest_pmu *pmu_ptr);
 
 static int nest_event_info(char *name, struct nest_ima_events *nest_events)
 {
@@ -210,6 +212,12 @@ static int nest_pmu_create(struct device_node *parent, int pmu_index)
 		idx += ret;
 	}
 
+	ret = init_nest_pmu(nest_events, idx, pmu_ptr);
+	if (ret) {
+		pr_err("Nest PMU %s Register failed\n", pmu_ptr->pmu.name);
+		return ret;
+	}
+
 	return 0;
 }
 
-- 
1.9.1

[PATCH v8 6/7] powerpc/perf: generic nest pmu event functions

[Madhavan Srinivasan]

Add set of generic nest pmu related event functions to be used by
each nest pmu. Add code to setup format attribute and to register
nest pmus.

Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Anton Blanchard <anton@samba.org>
Cc: Daniel Axtens <dja@axtens.net>
Cc: Stephane Eranian <eranian@google.com>
Cc: Sukadev Bhattiprolu <sukadev@linux.vnet.ibm.com>
Signed-off-by: Madhavan Srinivasan <maddy@linux.vnet.ibm.com>
---
 arch/powerpc/perf/nest-pmu.c | 114 +++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 114 insertions(+)

diff --git a/arch/powerpc/perf/nest-pmu.c b/arch/powerpc/perf/nest-pmu.c
index b9b44147c9b8..772e4d2d7a04 100644
--- a/arch/powerpc/perf/nest-pmu.c
+++ b/arch/powerpc/perf/nest-pmu.c
@@ -13,6 +13,111 @@
 struct perchip_nest_info nest_perchip_info[NEST_MAX_CHIPS];
 struct nest_pmu *per_nest_pmu_arr[NEST_MAX_PMUS];
 
+PMU_FORMAT_ATTR(event, "config:0-20");
+static struct attribute *nest_format_attrs[] = {
+	&format_attr_event.attr,
+	NULL,
+};
+
+static struct attribute_group nest_format_group = {
+	.name = "format",
+	.attrs = nest_format_attrs,
+};
+
+static int nest_event_init(struct perf_event *event)
+{
+	int chip_id;
+	u32 config = event->attr.config;
+	struct perchip_nest_info *pcni;
+
+	if (event->attr.type != event->pmu->type)
+		return -ENOENT;
+
+	/* Sampling not supported yet */
+	if (event->hw.sample_period)
+		return -EINVAL;
+
+	/* unsupported modes and filters */
+	if (event->attr.exclude_user   ||
+	  event->attr.exclude_kernel ||
+	  event->attr.exclude_hv     ||
+	  event->attr.exclude_idle   ||
+	  event->attr.exclude_host   ||
+	  event->attr.exclude_guest)
+		return -EINVAL;
+
+	if (event->cpu < 0)
+		return -EINVAL;
+
+	chip_id = topology_physical_package_id(event->cpu);
+	pcni = &nest_perchip_info[chip_id];
+	event->hw.event_base = pcni->vbase[config/PAGE_SIZE] +
+							(config & ~PAGE_MASK );
+
+	return 0;
+}
+
+static void nest_read_counter(struct perf_event *event)
+{
+	u64 *addr, data;
+
+	addr = (u64 *)event->hw.event_base;
+	data = __be64_to_cpu(*addr);
+	local64_set(&event->hw.prev_count, data);
+}
+
+static void nest_perf_event_update(struct perf_event *event)
+{
+	u64 counter_prev, counter_new, final_count, *addr;
+
+	addr = (u64 *)event->hw.event_base;
+	counter_prev = local64_read(&event->hw.prev_count);
+	counter_new = __be64_to_cpu(*addr);
+	final_count = counter_new - counter_prev;
+
+	local64_set(&event->hw.prev_count, counter_new);
+	local64_add(final_count, &event->count);
+}
+
+static void nest_event_start(struct perf_event *event, int flags)
+{
+	nest_read_counter(event);
+}
+
+static void nest_event_stop(struct perf_event *event, int flags)
+{
+	if (flags & PERF_EF_UPDATE)
+		nest_perf_event_update(event);
+}
+
+static int nest_event_add(struct perf_event *event, int flags)
+{
+       if (flags & PERF_EF_START)
+               nest_event_start(event, flags);
+
+       return 0;
+}
+
+/*
+ * Populate pmu ops in the structure
+ */
+static int update_pmu_ops(struct nest_pmu *pmu)
+{
+	if (!pmu)
+		return -EINVAL;
+
+	pmu->pmu.task_ctx_nr = perf_invalid_context;
+	pmu->pmu.event_init = nest_event_init;
+	pmu->pmu.add = nest_event_add;
+	pmu->pmu.del = nest_event_stop;
+	pmu->pmu.start = nest_event_start;
+	pmu->pmu.stop = nest_event_stop;
+	pmu->pmu.read = nest_perf_event_update;
+	pmu->attr_groups[1] = &nest_format_group;
+	pmu->pmu.attr_groups = pmu->attr_groups;
+
+	return 0;
+}
 /*
  * Populate event name and string in attribute
  */
@@ -62,8 +167,17 @@ static int update_events_in_group(
 int init_nest_pmu(struct nest_ima_events *nest_events,
 					int idx, struct nest_pmu *pmu_ptr)
 {
+	int ret = -ENODEV;
 
 	update_events_in_group(nest_events, idx, pmu_ptr);
+	update_pmu_ops(pmu_ptr);
+
+	ret = perf_pmu_register(&pmu_ptr->pmu, pmu_ptr->pmu.name, -1);
+	if (ret)
+		return ret;
+
+	pr_info("%s performance monitor hardware support registered\n",
+							pmu_ptr->pmu.name);
 	return 0;
 }
 
-- 
1.9.1

[PATCH v8 7/7] powerpc/perf: nest pmu cpumask and cpu hotplug support

[Madhavan Srinivasan]

Adds cpumask attribute to be used by each nest pmu since nest
units are per-chip. Only one cpu (first online cpu) from each chip
is designated to read counters.

On cpu hotplug, dying cpu is checked to see whether it is one of the
designated cpus, if yes, next online cpu from the same chip is
designated as new cpu to read counters.

Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Anton Blanchard <anton@samba.org>
Cc: Daniel Axtens <dja@axtens.net>
Cc: Stephane Eranian <eranian@google.com>
Cc: Sukadev Bhattiprolu <sukadev@linux.vnet.ibm.com>
Cc: Preeti U Murthy <preetium@andrew.cmu.edu>
Signed-off-by: Madhavan Srinivasan <maddy@linux.vnet.ibm.com>
---
 arch/powerpc/perf/nest-pmu.c | 171 +++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 171 insertions(+)

diff --git a/arch/powerpc/perf/nest-pmu.c b/arch/powerpc/perf/nest-pmu.c
index 772e4d2d7a04..4550de859f36 100644
--- a/arch/powerpc/perf/nest-pmu.c
+++ b/arch/powerpc/perf/nest-pmu.c
@@ -12,6 +12,7 @@
 
 struct perchip_nest_info nest_perchip_info[NEST_MAX_CHIPS];
 struct nest_pmu *per_nest_pmu_arr[NEST_MAX_PMUS];
+static cpumask_t nest_pmu_cpu_mask;
 
 PMU_FORMAT_ATTR(event, "config:0-20");
 static struct attribute *nest_format_attrs[] = {
@@ -24,6 +25,171 @@ static struct attribute_group nest_format_group = {
 	.attrs = nest_format_attrs,
 };
 
+static ssize_t nest_pmu_cpumask_get_attr(struct device *dev,
+				struct device_attribute *attr, char *buf)
+{
+	return cpumap_print_to_pagebuf(true, buf, &nest_pmu_cpu_mask);
+}
+
+static DEVICE_ATTR(cpumask, S_IRUGO, nest_pmu_cpumask_get_attr, NULL);
+
+static struct attribute *nest_pmu_cpumask_attrs[] = {
+	&dev_attr_cpumask.attr,
+	NULL,
+};
+
+static struct attribute_group nest_pmu_cpumask_attr_group = {
+	.attrs = nest_pmu_cpumask_attrs,
+};
+
+static void nest_init(int *loc)
+{
+	int rc;
+
+	rc = opal_nest_counters_control(
+				NEST_MODE_PRODUCTION, NEST_ENGINE_START, 0, 0);
+	if (rc)
+		loc[smp_processor_id()] = 1;
+}
+
+static void nest_change_cpu_context(int old_cpu, int new_cpu)
+{
+	int i;
+
+	for (i = 0;
+		(per_nest_pmu_arr[i] != NULL) && (i < NEST_MAX_PMUS); i++)
+		perf_pmu_migrate_context(&per_nest_pmu_arr[i]->pmu,
+							old_cpu, new_cpu);
+}
+
+static void nest_exit_cpu(int cpu)
+{
+	int nid, target = -1;
+	struct cpumask *l_cpumask;
+
+	/*
+	 * Check in the designated list for this cpu. Dont bother
+	 * if not one of them.
+	 */
+	if (!cpumask_test_and_clear_cpu(cpu, &nest_pmu_cpu_mask))
+		return;
+
+	/*
+	 * Now that this cpu is one of the designated,
+	 * find a next cpu a) which is online and b) in same chip.
+	 */
+	nid = cpu_to_node(cpu);
+	l_cpumask = cpumask_of_node(nid);
+	target = cpumask_next(cpu, l_cpumask);
+
+	/*
+	 * Update the cpumask with the target cpu and
+	 * migrate the context if needed
+	 */
+	if (target >= 0 && target <= nr_cpu_ids) {
+		cpumask_set_cpu(target, &nest_pmu_cpu_mask);
+		nest_change_cpu_context(cpu, target);
+	}
+}
+
+static void nest_init_cpu(int cpu)
+{
+	int nid, fcpu, ncpu;
+	struct cpumask *l_cpumask, tmp_mask;
+
+	nid = cpu_to_node(cpu);
+	l_cpumask = cpumask_of_node(nid);
+
+	if (!cpumask_and(&tmp_mask, l_cpumask, &nest_pmu_cpu_mask)) {
+		cpumask_set_cpu(cpu, &nest_pmu_cpu_mask);
+		return;
+	}
+
+	fcpu = cpumask_first(l_cpumask);
+	ncpu = cpumask_next(cpu, l_cpumask);
+	if (cpu == fcpu) {
+		if (cpumask_test_and_clear_cpu(ncpu, &nest_pmu_cpu_mask)) {
+			cpumask_set_cpu(cpu, &nest_pmu_cpu_mask);
+			nest_change_cpu_context(ncpu, cpu);
+		}
+	}
+}
+
+static int nest_pmu_cpu_notifier(struct notifier_block *self,
+					unsigned long action, void *hcpu)
+{
+	long cpu = (long)hcpu;
+
+	switch (action & ~CPU_TASKS_FROZEN) {
+	case CPU_ONLINE:
+		nest_init_cpu(cpu);
+		break;
+	case CPU_DOWN_PREPARE:
+		nest_exit_cpu(cpu);
+		break;
+	default:
+		break;
+	}
+
+	return NOTIFY_OK;
+}
+
+static struct notifier_block nest_pmu_cpu_nb = {
+	.notifier_call  = nest_pmu_cpu_notifier,
+	.priority       = CPU_PRI_PERF + 1,
+};
+
+static int nest_pmu_cpumask_init(void)
+{
+	const struct cpumask *l_cpumask;
+	int cpu, nid;
+	int *cpus_opal_rc;
+
+	if (!cpumask_empty(&nest_pmu_cpu_mask))
+		return 0;
+
+	cpu_notifier_register_begin();
+
+	/*
+	 * Nest PMUs are per-chip counters. So designate a cpu
+	 * from each chip for counter collection.
+	 */
+	for_each_online_node(nid) {
+		l_cpumask = cpumask_of_node(nid);
+
+		/* designate first online cpu in this node */
+		cpu = cpumask_first(l_cpumask);
+		cpumask_set_cpu(cpu, &nest_pmu_cpu_mask);
+	}
+
+	/*
+	 * Memory for OPAL call return value.
+	 */
+	cpus_opal_rc = kzalloc((sizeof(int) * nr_cpu_ids), GFP_KERNEL);
+	if (!cpus_opal_rc)
+		goto fail;
+
+	/* Initialize Nest PMUs in each node using designated cpus */
+	on_each_cpu_mask(&nest_pmu_cpu_mask, (smp_call_func_t)nest_init,
+						(void *)cpus_opal_rc, 1);
+
+	/* Check return value array for any OPAL call failure */
+	for_each_cpu(cpu, &nest_pmu_cpu_mask) {
+		if (cpus_opal_rc[cpu])
+			goto fail;
+	}
+
+
+	__register_cpu_notifier(&nest_pmu_cpu_nb);
+
+	cpu_notifier_register_done();
+	return 0;
+
+fail:
+	cpu_notifier_register_done();
+	return -ENODEV;
+}
+
 static int nest_event_init(struct perf_event *event)
 {
 	int chip_id;
@@ -114,6 +280,7 @@ static int update_pmu_ops(struct nest_pmu *pmu)
 	pmu->pmu.stop = nest_event_stop;
 	pmu->pmu.read = nest_perf_event_update;
 	pmu->attr_groups[1] = &nest_format_group;
+	pmu->attr_groups[2] = &nest_pmu_cpumask_attr_group;
 	pmu->pmu.attr_groups = pmu->attr_groups;
 
 	return 0;
@@ -169,6 +336,10 @@ int init_nest_pmu(struct nest_ima_events *nest_events,
 {
 	int ret = -ENODEV;
 
+	/* Add cpumask and register for hotplug notification */
+	if (nest_pmu_cpumask_init())
+		return ret;
+
 	update_events_in_group(nest_events, idx, pmu_ptr);
 	update_pmu_ops(pmu_ptr);
 
-- 
1.9.1

Re: [PATCH v8 7/7] powerpc/perf: nest pmu cpumask and cpu hotplug support

[kbuild test robot]

[-- Attachment #1: Type: text/plain, Size: 1763 bytes --]

Hi Madhavan,

[auto build test ERROR on v4.5-rc5]
[also build test ERROR on next-20160229]
[cannot apply to powerpc/next]
[if your patch is applied to the wrong git tree, please drop us a note to help improving the system]

url:    https://github.com/0day-ci/linux/commits/Madhavan-Srinivasan/powerpc-powernv-Nest-Instrumentation-support/20160223-120036
config: powerpc-maple_defconfig (attached as .config)
reproduce:
        wget https://git.kernel.org/cgit/linux/kernel/git/wfg/lkp-tests.git/plain/sbin/make.cross -O ~/bin/make.cross
        chmod +x ~/bin/make.cross
        # save the attached .config to linux build tree
        make.cross ARCH=powerpc 

All errors (new ones prefixed by >>):

   arch/powerpc/perf/nest-pmu.c: In function 'nest_exit_cpu':
>> arch/powerpc/perf/nest-pmu.c:82:12: error: assignment discards 'const' qualifier from pointer target type [-Werror=discarded-qualifiers]
     l_cpumask = cpumask_of_node(nid);
               ^
   arch/powerpc/perf/nest-pmu.c: In function 'nest_init_cpu':
   arch/powerpc/perf/nest-pmu.c:101:12: error: assignment discards 'const' qualifier from pointer target type [-Werror=discarded-qualifiers]
     l_cpumask = cpumask_of_node(nid);
               ^
   cc1: all warnings being treated as errors

vim +/const +82 arch/powerpc/perf/nest-pmu.c

    76	
    77		/*
    78		 * Now that this cpu is one of the designated,
    79		 * find a next cpu a) which is online and b) in same chip.
    80		 */
    81		nid = cpu_to_node(cpu);
  > 82		l_cpumask = cpumask_of_node(nid);
    83		target = cpumask_next(cpu, l_cpumask);
    84	
    85		/*

---
0-DAY kernel test infrastructure                Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all                   Intel Corporation

[-- Attachment #2: .config.gz --]
[-- Type: application/octet-stream, Size: 15965 bytes --]