[PATCH 0/2] powerpc: define and implement MPIC message register support

[PATCH 0/2] powerpc: define and implement MPIC message register support

[Meador Inge]

This patch set defines a binding for FSL MPIC message registers and implements
an API for accessing those message registers.  Testing was done on a MPC8572DS
in an Linux-Linux AMP setup using OpenMCAPI (www.openmcapi.org) to communicate
between OS instances.  The message register API is used by the OpenMCAPI shared 
memory driver to send notifications between cores.

Signed-off-by: Meador Inge <meador_inge@mentor.com>
Cc: Hollis Blanchard <hollis_blanchard@mentor.com>

Meador Inge (2):
  powerpc: document the FSL MPIC message register binding
  powerpc: add support for MPIC message register API

 .../devicetree/bindings/powerpc/fsl/mpic-msgr.txt  |   71 +++++
 arch/powerpc/include/asm/mpic_msgr.h               |   35 +++
 arch/powerpc/platforms/Kconfig                     |    8 +
 arch/powerpc/sysdev/Makefile                       |    3 +-
 arch/powerpc/sysdev/mpic_msgr.c                    |  279 ++++++++++++++++++++
 5 files changed, 395 insertions(+), 1 deletions(-)
 create mode 100644 Documentation/devicetree/bindings/powerpc/fsl/mpic-msgr.txt
 create mode 100644 arch/powerpc/include/asm/mpic_msgr.h
 create mode 100644 arch/powerpc/sysdev/mpic_msgr.c

[PATCH 1/2] powerpc: document the FSL MPIC message register binding

[Meador Inge]

This binding documents how the message register blocks found in some FSL
MPIC implementations shall be represented in a device tree.

Signed-off-by: Meador Inge <meador_inge@mentor.com>
Cc: Hollis Blanchard <hollis_blanchard@mentor.com>
Cc: Grant Likely <grant.likely@secretlab.ca>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
---
 .../devicetree/bindings/powerpc/fsl/mpic-msgr.txt  |   71 ++++++++++++++++++++
 1 files changed, 71 insertions(+), 0 deletions(-)
 create mode 100644 Documentation/devicetree/bindings/powerpc/fsl/mpic-msgr.txt

diff --git a/Documentation/devicetree/bindings/powerpc/fsl/mpic-msgr.txt b/Documentation/devicetree/bindings/powerpc/fsl/mpic-msgr.txt
new file mode 100644
index 0000000..41f1965
--- /dev/null
+++ b/Documentation/devicetree/bindings/powerpc/fsl/mpic-msgr.txt
@@ -0,0 +1,71 @@
+* FSL MPIC Message Registers
+
+This binding specifies what properties must be available in the device tree
+representation of the message register blocks found in some FSL MPIC
+implementations.
+
+Required properties:
+
+    - compatible: Specifies the compatibility list for the message register
+      block.  The type shall be <string> and the value shall be of the form
+      "fsl,mpic-v<version>-msgr", where <version> is the version number of
+      the MPIC containing the message registers.
+
+    - reg: Specifies the base physical address(s) and size(s) of the
+      message register block's addressable register space.  The type shall be
+      <prop-encoded-array>.
+
+    - interrupts: Specifies a list of interrupt source and level-sense pairs.
+      The type shall be <prop-encoded-array>.  The length shall be equal to
+      the number of bits set in the 'msg-receive-mask' property value.
+
+    - interrupt-parent: Specifies the interrupt parent of the message register
+      block.  The type shall be a <phandle> and the value of that <phandle>
+      shall point to the interrupt parent.
+
+Optional properties:
+
+    - msg-receive-mask: Specifies what registers in the containing block are
+      allowed to receive interrupts.  The value is a bit mask where a set bit
+      at bit 'n' indicates that message register 'n' can receive interrupts.
+      The type shall be <prop-encoded-array>.  The default value shall be
+      all a string of consecutive ones where the length of the run is equal
+      to the number of registers in the block.  For example, a block with
+      four registers shall default to 0xF.
+
+Required alias:
+
+    In order for a message register block to be discovered it *must* define
+    an alias in the 'aliases' node.  Aliases are of the form 'msgr-block<n>',
+    where <n> is an integer specifying the block's number.  Numbers shall start
+    at 0.
+
+Example:
+
+	/* The aliases needed to define an order on the message register blocks.
+	 */
+	aliases {
+		msgr-block0 = &msgr_block0;
+		msgr-block1 = &msgr_block1;
+	};
+
+	msgr_block0: msgr-block@41400 {
+		compatible = "fsl,mpic-v3.1-msgr";
+		reg = <0x41400 0x200>;
+		// Message registers 0 and 3 in this block can receive interrupts on
+		// sources 0xb0 and 0xb2, respectively.
+		interrupts = <0xb0 2 0xb2 2>;
+		msg-receive-mask = <0x5>;
+		interrupt-parent = <&mpic>;
+	};
+
+	msgr_block1: msgr-block@42400 {
+		compatible = "fsl,mpic-v3.1-msgr";
+		reg = <0x42400 0x200>;
+		// Message registers 0 and 3 in this block can receive interrupts on
+		// sources 0xb4 and 0xb6, respectively.
+		interrupts = <0xb4 2 0xb6 2>;
+		msg-receive-mask = <0x5>;
+		interrupt-parent = <&mpic>;
+	};
+
-- 
1.6.3.3

Re: [PATCH 1/2] powerpc: document the FSL MPIC message register binding

[Scott Wood]

On Tue, 19 Apr 2011 11:59:34 -0500
Meador Inge <meador_inge@mentor.com> wrote:

> +    - interrupt-parent: Specifies the interrupt parent of the message register
> +      block.  The type shall be a <phandle> and the value of that <phandle>
> +      shall point to the interrupt parent.

interrupt-parent is not required; it can be inherited from an ancestor.  In
any case, this description doesn't say anything specifically about MPIC
message nodes.

>  The default value shall be
> +      all a string of consecutive ones where the length of the run is equal
> +      to the number of registers in the block.  For example, a block with
> +      four registers shall default to 0xF.

Could be more simply worded as, "If not present, all message registers in
the group are available."

> +Required alias:
> +
> +    In order for a message register block to be discovered it *must* define
> +    an alias in the 'aliases' node.

I think the "in order to be discovered" statement is specific to your use
case.

>  Aliases are of the form 'msgr-block<n>',
> +    where <n> is an integer specifying the block's number.  Numbers shall start
> +    at 0.

The hw docs refer to "group A" and "group B", not "block 0" and "block 1".

Plus, I'd put "mpic-" in the alias name.

> +Example:
> +
> +	/* The aliases needed to define an order on the message register blocks.
> +	 */
> +	aliases {
> +		msgr-block0 = &msgr_block0;
> +		msgr-block1 = &msgr_block1;
> +	};
> +
> +	msgr_block0: msgr-block@41400 {
> +		compatible = "fsl,mpic-v3.1-msgr";
> +		reg = <0x41400 0x200>;
> +		// Message registers 0 and 3 in this block can receive interrupts on
> +		// sources 0xb0 and 0xb2, respectively.
> +		interrupts = <0xb0 2 0xb2 2>;
> +		msg-receive-mask = <0x5>;
> +		interrupt-parent = <&mpic>;
> +	};

A mask of 0x5 specifies message registers 0 and 2 (as do interrupts 0xb0
and 0xb2), not 0 and 3.

-Scott

Re: [PATCH 1/2] powerpc: document the FSL MPIC message register binding

[Meador Inge]

On 04/19/2011 12:52 PM, Scott Wood wrote:
> On Tue, 19 Apr 2011 11:59:34 -0500
> Meador Inge <meador_inge@mentor.com> wrote:
> 
>> +    - interrupt-parent: Specifies the interrupt parent of the message register
>> +      block.  The type shall be a <phandle> and the value of that <phandle>
>> +      shall point to the interrupt parent.
> 
> interrupt-parent is not required; it can be inherited from an ancestor.  In
> any case, this description doesn't say anything specifically about MPIC
> message nodes.

Removed.

>>  The default value shall be
>> +      all a string of consecutive ones where the length of the run is equal
>> +      to the number of registers in the block.  For example, a block with
>> +      four registers shall default to 0xF.
> 
> Could be more simply worded as, "If not present, all message registers in
> the group are available."

That is much better.

>> +Required alias:
>> +
>> +    In order for a message register block to be discovered it *must* define
>> +    an alias in the 'aliases' node.
> 
> I think the "in order to be discovered" statement is specific to your use
> case.
> 
>>  Aliases are of the form 'msgr-block<n>',
>> +    where <n> is an integer specifying the block's number.  Numbers shall start
>> +    at 0.
> 
> The hw docs refer to "group A" and "group B", not "block 0" and "block 1".
> 
> Plus, I'd put "mpic-" in the alias name.

Are you suggesting that the alias should be called: 'mpic-groupA',
'mpic-groupB', 'mpic-groupC', etc... ?  Or just that I should use the terminology
group instead of block where discussing things in the binding?


>> +Example:
>> +
>> +	/* The aliases needed to define an order on the message register blocks.
>> +	 */
>> +	aliases {
>> +		msgr-block0 = &msgr_block0;
>> +		msgr-block1 = &msgr_block1;
>> +	};
>> +
>> +	msgr_block0: msgr-block@41400 {
>> +		compatible = "fsl,mpic-v3.1-msgr";
>> +		reg = <0x41400 0x200>;
>> +		// Message registers 0 and 3 in this block can receive interrupts on
>> +		// sources 0xb0 and 0xb2, respectively.
>> +		interrupts = <0xb0 2 0xb2 2>;
>> +		msg-receive-mask = <0x5>;
>> +		interrupt-parent = <&mpic>;
>> +	};
> 
> A mask of 0x5 specifies message registers 0 and 2 (as do interrupts 0xb0
> and 0xb2), not 0 and 3.
> 

Fixed.  Thanks.

-- 
Meador Inge     | meador_inge AT mentor.com
Mentor Embedded | http://www.mentor.com/embedded-software

Re: [PATCH 1/2] powerpc: document the FSL MPIC message register binding

[Scott Wood]

On Tue, 19 Apr 2011 13:26:26 -0500
Meador Inge <meador_inge@mentor.com> wrote:

> On 04/19/2011 12:52 PM, Scott Wood wrote:
> > On Tue, 19 Apr 2011 11:59:34 -0500
> > Meador Inge <meador_inge@mentor.com> wrote:
> > 
> >>  Aliases are of the form 'msgr-block<n>',
> >> +    where <n> is an integer specifying the block's number.  Numbers shall start
> >> +    at 0.
> > 
> > The hw docs refer to "group A" and "group B", not "block 0" and "block 1".
> > 
> > Plus, I'd put "mpic-" in the alias name.
> 
> Are you suggesting that the alias should be called: 'mpic-groupA',
> 'mpic-groupB', 'mpic-groupC', etc... ?

I was thinking something like "mpic-msgr-group-a", "mpic-msgr-group-b" --
though if you want to use numbers instead to more easily map to potential
APIs, that's OK.

-Scott

Re: [PATCH 1/2] powerpc: document the FSL MPIC message register binding

[Meador Inge]

On 04/19/2011 01:33 PM, Scott Wood wrote:
> On Tue, 19 Apr 2011 13:26:26 -0500
> Meador Inge <meador_inge@mentor.com> wrote:
> 
>> On 04/19/2011 12:52 PM, Scott Wood wrote:
>>> On Tue, 19 Apr 2011 11:59:34 -0500
>>> Meador Inge <meador_inge@mentor.com> wrote:
>>>
>>>>  Aliases are of the form 'msgr-block<n>',
>>>> +    where <n> is an integer specifying the block's number.  Numbers shall start
>>>> +    at 0.
>>>
>>> The hw docs refer to "group A" and "group B", not "block 0" and "block 1".
>>>
>>> Plus, I'd put "mpic-" in the alias name.
>>
>> Are you suggesting that the alias should be called: 'mpic-groupA',
>> 'mpic-groupB', 'mpic-groupC', etc... ?
> 
> I was thinking something like "mpic-msgr-group-a", "mpic-msgr-group-b" --
> though if you want to use numbers instead to more easily map to potential
> APIs, that's OK.
> 

Hmmm ...  In the MPC8572E and P1022DS manuals I don't see the terminology
group used for message registers.  The MPIC global timers on the other hand
do use group A and group B.  I will definitely add the 'mpic-' prefix, but
unless I am looking in the wrong place in the manuals, then I am not so
sure there are MPIC message register groups.  As a side note,
the term "register block" pops up in a few places not related to
message registers.

If you feel like group is a more idiomatic term, I can change it.

-- 
Meador Inge     | meador_inge AT mentor.com
Mentor Embedded | http://www.mentor.com/embedded-software

Re: [PATCH 1/2] powerpc: document the FSL MPIC message register binding

[Scott Wood]

On Thu, 21 Apr 2011 14:26:46 -0500
Meador Inge <meador_inge@mentor.com> wrote:

> Hmmm ...  In the MPC8572E and P1022DS manuals I don't see the terminology
> group used for message registers.

I was looking at the P4080 manual, which does use it.  It looks like some
other chip manuals just use MSGR0-MSGR7.

> If you feel like group is a more idiomatic term, I can change it.

Since the docs aren't consistent, I don't really have a strong opinion
here.  Just make clear in the binding what you're referring to.

-Scott

[PATCH 2/2] powerpc: add support for MPIC message register API

[Meador Inge]

Some MPIC implementations contain one or more blocks of message registers
that are used to send messages between cores via IPIs.  A simple API has
been added to access (get/put, read, write, etc ...) these message registers.
The available message registers are initially discovered via nodes in the
device tree.  A separate commit contains a binding for the message register
nodes.

Signed-off-by: Meador Inge <meador_inge@mentor.com>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Hollis Blanchard <hollis_blanchard@mentor.com>
---
 arch/powerpc/include/asm/mpic_msgr.h |   35 +++++
 arch/powerpc/platforms/Kconfig       |    8 +
 arch/powerpc/sysdev/Makefile         |    3 +-
 arch/powerpc/sysdev/mpic_msgr.c      |  279 ++++++++++++++++++++++++++++++++++
 4 files changed, 324 insertions(+), 1 deletions(-)
 create mode 100644 arch/powerpc/include/asm/mpic_msgr.h
 create mode 100644 arch/powerpc/sysdev/mpic_msgr.c

diff --git a/arch/powerpc/include/asm/mpic_msgr.h b/arch/powerpc/include/asm/mpic_msgr.h
new file mode 100644
index 0000000..370dcb4
--- /dev/null
+++ b/arch/powerpc/include/asm/mpic_msgr.h
@@ -0,0 +1,35 @@
+/*
+ * Copyright 2011-2012, Meador Inge, Mentor Graphics Corporation.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; version 2 of the
+ * License.
+ *
+ */
+
+#ifndef _ASM_MPIC_MSGR_H
+#define _ASM_MPIC_MSGR_H
+
+#include <linux/types.h>
+
+struct mpic_msgr {
+	u32 __iomem *addr;
+	u32 __iomem *mer;
+	u32 __iomem	*msr;
+	int irq;
+	atomic_t in_use;
+	int num;
+};
+
+extern struct mpic_msgr* mpic_msgr_get(unsigned int reg_num);
+extern void mpic_msgr_put(struct mpic_msgr* msgr);
+extern void mpic_msgr_enable(struct mpic_msgr *msgr);
+extern void mpic_msgr_disable(struct mpic_msgr *msgr);
+extern void mpic_msgr_write(struct mpic_msgr *msgr, u32 message);
+extern u32 mpic_msgr_read(struct mpic_msgr *msgr);
+extern void mpic_msgr_clear(struct mpic_msgr *msgr);
+extern void mpic_msgr_set_destination(struct mpic_msgr *msgr, u32 cpu_num);
+extern int mpic_msgr_get_irq(struct mpic_msgr *msgr);
+
+#endif
diff --git a/arch/powerpc/platforms/Kconfig b/arch/powerpc/platforms/Kconfig
index f7b0772..4d65593 100644
--- a/arch/powerpc/platforms/Kconfig
+++ b/arch/powerpc/platforms/Kconfig
@@ -78,6 +78,14 @@ config MPIC_WEIRD
 	bool
 	default n
 
+config MPIC_MSGR
+	bool "MPIC message register support"
+	depends on MPIC
+	default n
+	help
+	  Enables support for the MPIC message registers.  These
+	  registers are used for inter-processor communication.
+
 config PPC_I8259
 	bool
 	default n
diff --git a/arch/powerpc/sysdev/Makefile b/arch/powerpc/sysdev/Makefile
index 1e0c933..6d40185 100644
--- a/arch/powerpc/sysdev/Makefile
+++ b/arch/powerpc/sysdev/Makefile
@@ -3,7 +3,8 @@ subdir-ccflags-$(CONFIG_PPC_WERROR) := -Werror
 ccflags-$(CONFIG_PPC64)		:= -mno-minimal-toc
 
 mpic-msi-obj-$(CONFIG_PCI_MSI)	+= mpic_msi.o mpic_u3msi.o mpic_pasemi_msi.o
-obj-$(CONFIG_MPIC)		+= mpic.o $(mpic-msi-obj-y)
+mpic-msgr-obj-$(CONFIG_MPIC_MSGR)	+= mpic_msgr.o
+obj-$(CONFIG_MPIC)		+= mpic.o $(mpic-msi-obj-y) $(mpic-msgr-obj-y)
 fsl-msi-obj-$(CONFIG_PCI_MSI)	+= fsl_msi.o
 obj-$(CONFIG_PPC_MSI_BITMAP)	+= msi_bitmap.o
 
diff --git a/arch/powerpc/sysdev/mpic_msgr.c b/arch/powerpc/sysdev/mpic_msgr.c
new file mode 100644
index 0000000..352bfa6
--- /dev/null
+++ b/arch/powerpc/sysdev/mpic_msgr.c
@@ -0,0 +1,279 @@
+/*
+ * Copyright 2011-2012, Meador Inge, Mentor Graphics Corporation.
+ *
+ * Some ideas based on un-pushed work done by Vivek Mahajan, Jason Jin, and
+ * Mingkai Hu from Freescale Semiconductor, Inc.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; version 2 of the
+ * License.
+ *
+ */
+
+#include <linux/list.h>
+#include <linux/of_platform.h>
+#include <linux/errno.h>
+#include <asm/prom.h>
+#include <asm/hw_irq.h>
+#include <asm/ppc-pci.h>
+#include <asm/mpic_msgr.h>
+
+#define MPIC_MSGR_REGISTERS_PER_BLOCK 4
+#define MSGR_INUSE 0
+#define MSGR_FREE 1
+
+/* Internal structure used *only* for IO mapping register blocks. */
+struct mpic_msgr_block {
+	struct msgr {
+		u32 msgr;
+		u8 res[12];
+	} msgrs[MPIC_MSGR_REGISTERS_PER_BLOCK];
+	u8 res0[192];
+	u32 mer;
+	u8 res1[12];
+	u32 msr;
+};
+
+static struct mpic_msgr **mpic_msgrs = 0;
+static unsigned int mpic_msgr_count = 0;
+
+struct mpic_msgr* mpic_msgr_get(unsigned int reg_num)
+{
+	struct mpic_msgr* msgr;
+
+	if (reg_num >= mpic_msgr_count)
+		return ERR_PTR(-ENODEV);
+
+	msgr = mpic_msgrs[reg_num];
+
+	if (atomic_cmpxchg(&msgr->in_use, MSGR_FREE, MSGR_INUSE) == MSGR_FREE)
+		return msgr;
+
+	return ERR_PTR(-EBUSY);
+}
+EXPORT_SYMBOL(mpic_msgr_get);
+
+void mpic_msgr_put(struct mpic_msgr* msgr)
+{
+	atomic_set(&msgr->in_use, MSGR_FREE);
+}
+EXPORT_SYMBOL(mpic_msgr_put);
+
+void mpic_msgr_enable(struct mpic_msgr *msgr)
+{
+	out_be32(msgr->mer, in_be32(msgr->mer) | (1 << msgr->num));
+}
+EXPORT_SYMBOL(mpic_msgr_enable);
+
+void mpic_msgr_disable(struct mpic_msgr *msgr)
+{
+	out_be32(msgr->mer, in_be32(msgr->mer) & ~(1 << msgr->num));
+}
+EXPORT_SYMBOL(mpic_msgr_disable);
+
+void mpic_msgr_write(struct mpic_msgr *msgr, u32 message)
+{
+	out_be32(msgr->addr, message);
+}
+EXPORT_SYMBOL(mpic_msgr_write);
+
+u32 mpic_msgr_read(struct mpic_msgr *msgr)
+{
+	return in_be32(msgr->addr);
+}
+EXPORT_SYMBOL(mpic_msgr_read);
+
+void mpic_msgr_clear(struct mpic_msgr *msgr)
+{
+	(void) mpic_msgr_read(msgr);
+}
+EXPORT_SYMBOL(mpic_msgr_clear);
+
+void mpic_msgr_set_destination(struct mpic_msgr *msgr, u32 cpu_num)
+{
+	out_be32(msgr->addr, 1 << cpu_num);
+}
+EXPORT_SYMBOL(mpic_msgr_set_destination);
+
+int mpic_msgr_get_irq(struct mpic_msgr *msgr)
+{
+	return msgr->irq;
+}
+EXPORT_SYMBOL(mpic_msgr_get_irq);
+
+/* The following three functions are used to compute the order and number of
+ * the message register blocks.  They are clearly very inefficent.  However,
+ * they are called *only* a few times during device initialization.
+ */
+static unsigned int mpic_msgr_number_of_blocks(void)
+{
+	unsigned int count;
+	struct device_node *aliases;
+
+	count = 0;
+	aliases = of_find_node_by_name(NULL, "aliases");
+
+	if (aliases) {
+		char buf[32];
+
+		for (;;) {
+			snprintf(buf, sizeof(buf), "msgr-block%d", count);
+			if (!of_find_property(aliases, buf, NULL))
+				break;
+
+			count += 1;
+		}
+	}
+
+	return count;
+}
+
+static unsigned int mpic_msgr_number_of_registers(void)
+{
+	return mpic_msgr_number_of_blocks() * MPIC_MSGR_REGISTERS_PER_BLOCK;
+}
+
+static int mpic_msgr_block_number(struct device_node *node)
+{
+	struct device_node *aliases;
+	unsigned int index, number_of_blocks;
+	char buf[64];
+
+	number_of_blocks = mpic_msgr_number_of_blocks();
+	aliases = of_find_node_by_name(NULL, "aliases");
+	if (!aliases)
+		return -1;
+
+	for (index = 0; index < number_of_blocks; ++index) {
+		struct property *prop;
+
+		snprintf(buf, sizeof(buf), "msgr-block%d", index);
+		prop = of_find_property(aliases, buf, NULL);
+		if (node == of_find_node_by_path(prop->value))
+			break;
+	}
+
+	return index == number_of_blocks ? -1 : index;
+}
+
+/* The probe function for a single message register block.
+ */
+static __devinit int mpic_msgr_probe(struct platform_device *dev)
+{
+	struct mpic_msgr_block __iomem *msgr_block;
+	int block_number;
+	struct resource rsrc;
+	unsigned int i;
+	unsigned int irq_index;
+	struct device_node *np = dev->dev.of_node;
+	unsigned int receive_mask;
+	const unsigned int *prop;
+
+	if (!np) {
+		dev_err(&dev->dev, "Device OF-Node is NULL");
+		return -EFAULT;
+	}
+
+	/* Allocate the message register array upon the first device
+	 * registered.
+	 */
+	if (!mpic_msgrs) {
+		mpic_msgr_count = mpic_msgr_number_of_registers();
+		dev_info(&dev->dev, "Found %d message registers\n", mpic_msgr_count);
+
+		mpic_msgrs = kzalloc(sizeof(struct mpic_msgr) * mpic_msgr_count,
+							 GFP_KERNEL);
+		if (!mpic_msgrs) {
+			dev_err(&dev->dev, "No memory for message register blocks\n");
+			return -ENOMEM;
+		}
+	}
+	dev_info(&dev->dev, "Of-device full name %s\n", np->full_name);
+
+	/* IO map the message register block. */
+	of_address_to_resource(np, 0, &rsrc);
+	msgr_block = ioremap(rsrc.start, rsrc.end - rsrc.start);
+	if (!msgr_block) {
+		dev_err(&dev->dev, "Failed to iomap MPIC message registers");
+		return -EFAULT;
+	}
+
+	/* Ensure the block has a defined order. */
+	block_number = mpic_msgr_block_number(np);
+	if (block_number < 0) {
+		dev_err(&dev->dev, "Failed to find message register block alias\n");
+		return -ENODEV;
+	}
+	dev_info(&dev->dev, "Setting up message register block %d\n", block_number);
+
+	/* Grab the receive mask which specifies what registers can receive
+	 * interrupts.
+	 */
+	prop = of_get_property(np, "msg-receive-mask", NULL);
+	receive_mask = (prop) ? *prop : 0xF;
+
+	/* Build up the appropriate message register data structures. */
+	for (i = 0, irq_index = 0; i < MPIC_MSGR_REGISTERS_PER_BLOCK; ++i) {
+		struct mpic_msgr *msgr;
+		unsigned int reg_number;
+
+		msgr = kzalloc(sizeof(struct mpic_msgr), GFP_KERNEL);
+		if (!msgr) {
+			dev_err(&dev->dev, "No memory for message register\n");
+			return -ENOMEM;
+		}
+
+		reg_number = block_number * MPIC_MSGR_REGISTERS_PER_BLOCK + i;
+		msgr->addr = &msgr_block->msgrs[i].msgr;
+		msgr->mer = &msgr_block->mer;
+		msgr->msr = &msgr_block->msr;
+		atomic_set(&msgr->in_use, MSGR_FREE);
+		msgr->num = reg_number;
+
+		if (receive_mask & (1 << i)) {
+			struct resource irq;
+
+			if (of_irq_to_resource(np, irq_index, &irq) == NO_IRQ) {
+				dev_err(&dev->dev, "Missing interrupt specifier");
+				kfree(msgr);
+				return -EFAULT;
+			}
+			msgr->irq = irq.start;
+			irq_index += 1;
+		} else {
+			msgr->irq = NO_IRQ;
+		}
+
+		mpic_msgrs[reg_number] = msgr;
+		mpic_msgr_disable(msgr);
+		dev_info(&dev->dev, "Register %d initialized: irq %d\n",
+				 msgr->num, msgr->irq);
+	
+	}
+
+	return 0;
+}
+
+static const struct of_device_id mpic_msgr_ids[] = {
+	{
+		.compatible = "fsl,mpic-v3.1-msgr",
+		.data = NULL,
+	},
+	{}
+};
+
+static struct platform_driver mpic_msgr_driver = {
+	.driver = {
+		.name = "mpic-msgr",
+		.owner = THIS_MODULE,
+		.of_match_table = mpic_msgr_ids,
+	},
+	.probe = mpic_msgr_probe,
+};
+
+static __init int mpic_msgr_init(void)
+{
+	return platform_driver_register(&mpic_msgr_driver);
+}
+subsys_initcall(mpic_msgr_init);
-- 
1.6.3.3

RE: [PATCH 2/2] powerpc: add support for MPIC message register API

[Kushwaha Prabhakar-B32579]

Hi,

I have no comments about coding and architecture. It looks fine.

Only have a query about its use case..
  "Any application intended to use message interrupt requires to know reg_n=
um because of struct mpic_msgr* mpic_msgr_get(unsigned int reg_num) API"

It will be good to search available unit internally and provide its pointer=
. It will make application more flexible.=20

Regards,
Prabhakar

> -----Original Message-----
> From: devicetree-discuss-bounces+b32579=3Dfreescale.com@lists.ozlabs.org
> [mailto:devicetree-discuss-bounces+b32579=3Dfreescale.com@lists.ozlabs.or=
g]
> On Behalf Of Meador Inge
> Sent: Tuesday, April 19, 2011 10:30 PM
> To: linuxppc-dev@lists.ozlabs.org
> Cc: openmcapi-dev@googlegroups.com; devicetree-discuss@lists.ozlabs.org;
> Hollis Blanchard
> Subject: [PATCH 2/2] powerpc: add support for MPIC message register API
>=20
> Some MPIC implementations contain one or more blocks of message registers
> that are used to send messages between cores via IPIs.  A simple API has
> been added to access (get/put, read, write, etc ...) these message
> registers.
> The available message registers are initially discovered via nodes in the
> device tree.  A separate commit contains a binding for the message
> register nodes.
>=20
> Signed-off-by: Meador Inge <meador_inge@mentor.com>
> Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
> Cc: Hollis Blanchard <hollis_blanchard@mentor.com>
> ---
>  arch/powerpc/include/asm/mpic_msgr.h |   35 +++++
>  arch/powerpc/platforms/Kconfig       |    8 +
>  arch/powerpc/sysdev/Makefile         |    3 +-
>  arch/powerpc/sysdev/mpic_msgr.c      |  279
> ++++++++++++++++++++++++++++++++++
>  4 files changed, 324 insertions(+), 1 deletions(-)  create mode 100644
> arch/powerpc/include/asm/mpic_msgr.h
>  create mode 100644 arch/powerpc/sysdev/mpic_msgr.c
>=20
> diff --git a/arch/powerpc/include/asm/mpic_msgr.h
> b/arch/powerpc/include/asm/mpic_msgr.h
> new file mode 100644
> index 0000000..370dcb4
> --- /dev/null
> +++ b/arch/powerpc/include/asm/mpic_msgr.h
> @@ -0,0 +1,35 @@
> +/*
> + * Copyright 2011-2012, Meador Inge, Mentor Graphics Corporation.
> + *
> + * This program is free software; you can redistribute it and/or
> + * modify it under the terms of the GNU General Public License
> + * as published by the Free Software Foundation; version 2 of the
> + * License.
> + *
> + */
> +
> +#ifndef _ASM_MPIC_MSGR_H
> +#define _ASM_MPIC_MSGR_H
> +
> +#include <linux/types.h>
> +
> +struct mpic_msgr {
> +	u32 __iomem *addr;
> +	u32 __iomem *mer;
> +	u32 __iomem	*msr;
> +	int irq;
> +	atomic_t in_use;
> +	int num;
> +};
> +
> +extern struct mpic_msgr* mpic_msgr_get(unsigned int reg_num); extern
> +void mpic_msgr_put(struct mpic_msgr* msgr); extern void
> +mpic_msgr_enable(struct mpic_msgr *msgr); extern void
> +mpic_msgr_disable(struct mpic_msgr *msgr); extern void
> +mpic_msgr_write(struct mpic_msgr *msgr, u32 message); extern u32
> +mpic_msgr_read(struct mpic_msgr *msgr); extern void
> +mpic_msgr_clear(struct mpic_msgr *msgr); extern void
> +mpic_msgr_set_destination(struct mpic_msgr *msgr, u32 cpu_num); extern
> +int mpic_msgr_get_irq(struct mpic_msgr *msgr);
> +
> +#endif
> diff --git a/arch/powerpc/platforms/Kconfig
> b/arch/powerpc/platforms/Kconfig index f7b0772..4d65593 100644
> --- a/arch/powerpc/platforms/Kconfig
> +++ b/arch/powerpc/platforms/Kconfig
> @@ -78,6 +78,14 @@ config MPIC_WEIRD
>  	bool
>  	default n
>=20
> +config MPIC_MSGR
> +	bool "MPIC message register support"
> +	depends on MPIC
> +	default n
> +	help
> +	  Enables support for the MPIC message registers.  These
> +	  registers are used for inter-processor communication.
> +
>  config PPC_I8259
>  	bool
>  	default n
> diff --git a/arch/powerpc/sysdev/Makefile b/arch/powerpc/sysdev/Makefile
> index 1e0c933..6d40185 100644
> --- a/arch/powerpc/sysdev/Makefile
> +++ b/arch/powerpc/sysdev/Makefile
> @@ -3,7 +3,8 @@ subdir-ccflags-$(CONFIG_PPC_WERROR) :=3D -Werror
>  ccflags-$(CONFIG_PPC64)		:=3D -mno-minimal-toc
>=20
>  mpic-msi-obj-$(CONFIG_PCI_MSI)	+=3D mpic_msi.o mpic_u3msi.o
> mpic_pasemi_msi.o
> -obj-$(CONFIG_MPIC)		+=3D mpic.o $(mpic-msi-obj-y)
> +mpic-msgr-obj-$(CONFIG_MPIC_MSGR)	+=3D mpic_msgr.o
> +obj-$(CONFIG_MPIC)		+=3D mpic.o $(mpic-msi-obj-y) $(mpic-msgr-
> obj-y)
>  fsl-msi-obj-$(CONFIG_PCI_MSI)	+=3D fsl_msi.o
>  obj-$(CONFIG_PPC_MSI_BITMAP)	+=3D msi_bitmap.o
>=20
> diff --git a/arch/powerpc/sysdev/mpic_msgr.c
> b/arch/powerpc/sysdev/mpic_msgr.c new file mode 100644 index
> 0000000..352bfa6
> --- /dev/null
> +++ b/arch/powerpc/sysdev/mpic_msgr.c
> @@ -0,0 +1,279 @@
> +/*
> + * Copyright 2011-2012, Meador Inge, Mentor Graphics Corporation.
> + *
> + * Some ideas based on un-pushed work done by Vivek Mahajan, Jason Jin,
> +and
> + * Mingkai Hu from Freescale Semiconductor, Inc.
> + *
> + * This program is free software; you can redistribute it and/or
> + * modify it under the terms of the GNU General Public License
> + * as published by the Free Software Foundation; version 2 of the
> + * License.
> + *
> + */
> +
> +#include <linux/list.h>
> +#include <linux/of_platform.h>
> +#include <linux/errno.h>
> +#include <asm/prom.h>
> +#include <asm/hw_irq.h>
> +#include <asm/ppc-pci.h>
> +#include <asm/mpic_msgr.h>
> +
> +#define MPIC_MSGR_REGISTERS_PER_BLOCK 4 #define MSGR_INUSE 0 #define
> +MSGR_FREE 1
> +
> +/* Internal structure used *only* for IO mapping register blocks. */
> +struct mpic_msgr_block {
> +	struct msgr {
> +		u32 msgr;
> +		u8 res[12];
> +	} msgrs[MPIC_MSGR_REGISTERS_PER_BLOCK];
> +	u8 res0[192];
> +	u32 mer;
> +	u8 res1[12];
> +	u32 msr;
> +};
> +
> +static struct mpic_msgr **mpic_msgrs =3D 0; static unsigned int
> +mpic_msgr_count =3D 0;
> +
> +struct mpic_msgr* mpic_msgr_get(unsigned int reg_num) {
> +	struct mpic_msgr* msgr;
> +
> +	if (reg_num >=3D mpic_msgr_count)
> +		return ERR_PTR(-ENODEV);
> +
> +	msgr =3D mpic_msgrs[reg_num];
> +
> +	if (atomic_cmpxchg(&msgr->in_use, MSGR_FREE, MSGR_INUSE) =3D=3D
> MSGR_FREE)
> +		return msgr;
> +
> +	return ERR_PTR(-EBUSY);
> +}
> +EXPORT_SYMBOL(mpic_msgr_get);
> +
> +void mpic_msgr_put(struct mpic_msgr* msgr) {
> +	atomic_set(&msgr->in_use, MSGR_FREE);
> +}
> +EXPORT_SYMBOL(mpic_msgr_put);
> +
> +void mpic_msgr_enable(struct mpic_msgr *msgr) {
> +	out_be32(msgr->mer, in_be32(msgr->mer) | (1 << msgr->num)); }
> +EXPORT_SYMBOL(mpic_msgr_enable);
> +
> +void mpic_msgr_disable(struct mpic_msgr *msgr) {
> +	out_be32(msgr->mer, in_be32(msgr->mer) & ~(1 << msgr->num)); }
> +EXPORT_SYMBOL(mpic_msgr_disable);
> +
> +void mpic_msgr_write(struct mpic_msgr *msgr, u32 message) {
> +	out_be32(msgr->addr, message);
> +}
> +EXPORT_SYMBOL(mpic_msgr_write);
> +
> +u32 mpic_msgr_read(struct mpic_msgr *msgr) {
> +	return in_be32(msgr->addr);
> +}
> +EXPORT_SYMBOL(mpic_msgr_read);
> +
> +void mpic_msgr_clear(struct mpic_msgr *msgr) {
> +	(void) mpic_msgr_read(msgr);
> +}
> +EXPORT_SYMBOL(mpic_msgr_clear);
> +
> +void mpic_msgr_set_destination(struct mpic_msgr *msgr, u32 cpu_num) {
> +	out_be32(msgr->addr, 1 << cpu_num);
> +}
> +EXPORT_SYMBOL(mpic_msgr_set_destination);
> +
> +int mpic_msgr_get_irq(struct mpic_msgr *msgr) {
> +	return msgr->irq;
> +}
> +EXPORT_SYMBOL(mpic_msgr_get_irq);
> +
> +/* The following three functions are used to compute the order and
> +number of
> + * the message register blocks.  They are clearly very inefficent.
> +However,
> + * they are called *only* a few times during device initialization.
> + */
> +static unsigned int mpic_msgr_number_of_blocks(void) {
> +	unsigned int count;
> +	struct device_node *aliases;
> +
> +	count =3D 0;
> +	aliases =3D of_find_node_by_name(NULL, "aliases");
> +
> +	if (aliases) {
> +		char buf[32];
> +
> +		for (;;) {
> +			snprintf(buf, sizeof(buf), "msgr-block%d", count);
> +			if (!of_find_property(aliases, buf, NULL))
> +				break;
> +
> +			count +=3D 1;
> +		}
> +	}
> +
> +	return count;
> +}
> +
> +static unsigned int mpic_msgr_number_of_registers(void)
> +{
> +	return mpic_msgr_number_of_blocks() *
> MPIC_MSGR_REGISTERS_PER_BLOCK; }
> +
> +static int mpic_msgr_block_number(struct device_node *node) {
> +	struct device_node *aliases;
> +	unsigned int index, number_of_blocks;
> +	char buf[64];
> +
> +	number_of_blocks =3D mpic_msgr_number_of_blocks();
> +	aliases =3D of_find_node_by_name(NULL, "aliases");
> +	if (!aliases)
> +		return -1;
> +
> +	for (index =3D 0; index < number_of_blocks; ++index) {
> +		struct property *prop;
> +
> +		snprintf(buf, sizeof(buf), "msgr-block%d", index);
> +		prop =3D of_find_property(aliases, buf, NULL);
> +		if (node =3D=3D of_find_node_by_path(prop->value))
> +			break;
> +	}
> +
> +	return index =3D=3D number_of_blocks ? -1 : index; }
> +
> +/* The probe function for a single message register block.
> + */
> +static __devinit int mpic_msgr_probe(struct platform_device *dev) {
> +	struct mpic_msgr_block __iomem *msgr_block;
> +	int block_number;
> +	struct resource rsrc;
> +	unsigned int i;
> +	unsigned int irq_index;
> +	struct device_node *np =3D dev->dev.of_node;
> +	unsigned int receive_mask;
> +	const unsigned int *prop;
> +
> +	if (!np) {
> +		dev_err(&dev->dev, "Device OF-Node is NULL");
> +		return -EFAULT;
> +	}
> +
> +	/* Allocate the message register array upon the first device
> +	 * registered.
> +	 */
> +	if (!mpic_msgrs) {
> +		mpic_msgr_count =3D mpic_msgr_number_of_registers();
> +		dev_info(&dev->dev, "Found %d message registers\n",
> mpic_msgr_count);
> +
> +		mpic_msgrs =3D kzalloc(sizeof(struct mpic_msgr) *
> mpic_msgr_count,
> +							 GFP_KERNEL);
> +		if (!mpic_msgrs) {
> +			dev_err(&dev->dev, "No memory for message register
> blocks\n");
> +			return -ENOMEM;
> +		}
> +	}
> +	dev_info(&dev->dev, "Of-device full name %s\n", np->full_name);
> +
> +	/* IO map the message register block. */
> +	of_address_to_resource(np, 0, &rsrc);
> +	msgr_block =3D ioremap(rsrc.start, rsrc.end - rsrc.start);
> +	if (!msgr_block) {
> +		dev_err(&dev->dev, "Failed to iomap MPIC message registers");
> +		return -EFAULT;
> +	}
> +
> +	/* Ensure the block has a defined order. */
> +	block_number =3D mpic_msgr_block_number(np);
> +	if (block_number < 0) {
> +		dev_err(&dev->dev, "Failed to find message register block
> alias\n");
> +		return -ENODEV;
> +	}
> +	dev_info(&dev->dev, "Setting up message register block %d\n",
> +block_number);
> +
> +	/* Grab the receive mask which specifies what registers can receive
> +	 * interrupts.
> +	 */
> +	prop =3D of_get_property(np, "msg-receive-mask", NULL);
> +	receive_mask =3D (prop) ? *prop : 0xF;
> +
> +	/* Build up the appropriate message register data structures. */
> +	for (i =3D 0, irq_index =3D 0; i < MPIC_MSGR_REGISTERS_PER_BLOCK; ++i)
> {
> +		struct mpic_msgr *msgr;
> +		unsigned int reg_number;
> +
> +		msgr =3D kzalloc(sizeof(struct mpic_msgr), GFP_KERNEL);
> +		if (!msgr) {
> +			dev_err(&dev->dev, "No memory for message register\n");
> +			return -ENOMEM;
> +		}
> +
> +		reg_number =3D block_number * MPIC_MSGR_REGISTERS_PER_BLOCK +
> i;
> +		msgr->addr =3D &msgr_block->msgrs[i].msgr;
> +		msgr->mer =3D &msgr_block->mer;
> +		msgr->msr =3D &msgr_block->msr;
> +		atomic_set(&msgr->in_use, MSGR_FREE);
> +		msgr->num =3D reg_number;
> +
> +		if (receive_mask & (1 << i)) {
> +			struct resource irq;
> +
> +			if (of_irq_to_resource(np, irq_index, &irq) =3D=3D NO_IRQ)
> {
> +				dev_err(&dev->dev, "Missing interrupt
> specifier");
> +				kfree(msgr);
> +				return -EFAULT;
> +			}
> +			msgr->irq =3D irq.start;
> +			irq_index +=3D 1;
> +		} else {
> +			msgr->irq =3D NO_IRQ;
> +		}
> +
> +		mpic_msgrs[reg_number] =3D msgr;
> +		mpic_msgr_disable(msgr);
> +		dev_info(&dev->dev, "Register %d initialized: irq %d\n",
> +				 msgr->num, msgr->irq);
> +
> +	}
> +
> +	return 0;
> +}
> +
> +static const struct of_device_id mpic_msgr_ids[] =3D {
> +	{
> +		.compatible =3D "fsl,mpic-v3.1-msgr",
> +		.data =3D NULL,
> +	},
> +	{}
> +};
> +
> +static struct platform_driver mpic_msgr_driver =3D {
> +	.driver =3D {
> +		.name =3D "mpic-msgr",
> +		.owner =3D THIS_MODULE,
> +		.of_match_table =3D mpic_msgr_ids,
> +	},
> +	.probe =3D mpic_msgr_probe,
> +};
> +
> +static __init int mpic_msgr_init(void)
> +{
> +	return platform_driver_register(&mpic_msgr_driver);
> +}
> +subsys_initcall(mpic_msgr_init);
> --
> 1.6.3.3
>=20
> _______________________________________________
> devicetree-discuss mailing list
> devicetree-discuss@lists.ozlabs.org
> https://lists.ozlabs.org/listinfo/devicetree-discuss

Re: [PATCH 2/2] powerpc: add support for MPIC message register API

[Hollis Blanchard]

On 04/28/2011 10:00 PM, Kushwaha Prabhakar-B32579 wrote:
> Hi,
>
> I have no comments about coding and architecture. It looks fine.
>
> Only have a query about its use case..
>    "Any application intended to use message interrupt requires to know reg_num because of struct mpic_msgr* mpic_msgr_get(unsigned int reg_num) API"
>
> It will be good to search available unit internally and provide its pointer. It will make application more flexible.
>

The problem is that you fundamentally cannot implement an allocator for 
MSG registers if you're going to communicate with another kernel (how 
would both kernels' allocators be synchronized?). So the message 
register allocation must be decided at design time, not run time.

Hollis Blanchard
Mentor Graphics, Embedded Systems Division

RE: [PATCH 2/2] powerpc: add support for MPIC message register API

[Kushwaha Prabhakar-B32579]

> > Hi,
> >
> > I have no comments about coding and architecture. It looks fine.
> >
> > Only have a query about its use case..
> >    "Any application intended to use message interrupt requires to know
> reg_num because of struct mpic_msgr* mpic_msgr_get(unsigned int reg_num)
> API"
> >
> > It will be good to search available unit internally and provide its
> pointer. It will make application more flexible.
> >
>=20
> The problem is that you fundamentally cannot implement an allocator for
> MSG registers if you're going to communicate with another kernel (how
> would both kernels' allocators be synchronized?). So the message register
> allocation must be decided at design time, not run time.
>=20

I agree with you..  It is true while communicating with another kernel.
But message interrupts can be used by different independent drivers within =
same kernel. For eg. PCIe and Ethernet driver.=20
As per current design both drivers needs to be in sync before requesting an=
y message unit for avoiding any conflict. As these drivers are completely i=
ndependent. It is very difficult.=20

Can it be possible to provide new API to take care it.

--Prabhakar

Re: [PATCH 2/2] powerpc: add support for MPIC message register API

[Scott Wood]

On Fri, 29 Apr 2011 17:27:06 +0000
Kushwaha Prabhakar-B32579 <B32579@freescale.com> wrote:

> 
> 
> > > Hi,
> > >
> > > I have no comments about coding and architecture. It looks fine.
> > >
> > > Only have a query about its use case..
> > >    "Any application intended to use message interrupt requires to know
> > reg_num because of struct mpic_msgr* mpic_msgr_get(unsigned int reg_num)
> > API"
> > >
> > > It will be good to search available unit internally and provide its
> > pointer. It will make application more flexible.
> > >
> > 
> > The problem is that you fundamentally cannot implement an allocator for
> > MSG registers if you're going to communicate with another kernel (how
> > would both kernels' allocators be synchronized?). So the message register
> > allocation must be decided at design time, not run time.
> > 
> 
> I agree with you..  It is true while communicating with another kernel.
> But message interrupts can be used by different independent drivers within same kernel. For eg. PCIe and Ethernet driver. 
> As per current design both drivers needs to be in sync before requesting any message unit for avoiding any conflict. As these drivers are completely independent. It is very difficult. 
> 
> Can it be possible to provide new API to take care it.

Do you have a real use case in mind where these message registers (not
MSIs) are used internally in this manner?

Perhaps an allocator could be added in the same patchset that adds such a
user.

-Scott

RE: [PATCH 2/2] powerpc: add support for MPIC message register API

[Kushwaha Prabhakar-B32579]

>=20
> >
> >
> > > > Hi,
> > > >
> > > > I have no comments about coding and architecture. It looks fine.
> > > >
> > > > Only have a query about its use case..
> > > >    "Any application intended to use message interrupt requires to
> > > > know
> > > reg_num because of struct mpic_msgr* mpic_msgr_get(unsigned int
> > > reg_num) API"
> > > >
> > > > It will be good to search available unit internally and provide
> > > > its
> > > pointer. It will make application more flexible.
> > > >
> > >
> > > The problem is that you fundamentally cannot implement an allocator
> > > for MSG registers if you're going to communicate with another kernel
> > > (how would both kernels' allocators be synchronized?). So the
> > > message register allocation must be decided at design time, not run
> time.
> > >
> >
> > I agree with you..  It is true while communicating with another kernel.
> > But message interrupts can be used by different independent drivers
> within same kernel. For eg. PCIe and Ethernet driver.
> > As per current design both drivers needs to be in sync before
> requesting any message unit for avoiding any conflict. As these drivers
> are completely independent. It is very difficult.
> >
> > Can it be possible to provide new API to take care it.
>=20
> Do you have a real use case in mind where these message registers (not
> MSIs) are used internally in this manner?

Yes, we use for PCIe host/agent test case scenario.
Host usage message register to interrupt Agent...
Agent uses message register to generate irq_out (automatically generate MSI=
) to interrupt master. Please see RM for more details about irq_out
=20

Note: PCIe host/agent test scenario is used internally and we are working o=
n pushing it out..
=20
> Perhaps an allocator could be added in the same patchset that adds such a
> user.
>=20

Yaa. It can be done. Otherwise module has to query each message unit for it=
s availability.=20

--Prabhakar

Re: [PATCH 2/2] powerpc: add support for MPIC message register API

[Hollis Blanchard]

On 05/01/2011 08:41 PM, Kushwaha Prabhakar-B32579 wrote:
>>>>> Hi,
>>>>>
>>>>> I have no comments about coding and architecture. It looks fine.
>>>>>
>>>>> Only have a query about its use case..
>>>>>     "Any application intended to use message interrupt requires to
>>>>> know
>>>> reg_num because of struct mpic_msgr* mpic_msgr_get(unsigned int
>>>> reg_num) API"
>>>>> It will be good to search available unit internally and provide
>>>>> its
>>>> pointer. It will make application more flexible.
>>>> The problem is that you fundamentally cannot implement an allocator
>>>> for MSG registers if you're going to communicate with another kernel
>>>> (how would both kernels' allocators be synchronized?). So the
>>>> message register allocation must be decided at design time, not run time.
>>> I agree with you..  It is true while communicating with another kernel.
>>> But message interrupts can be used by different independent drivers
>> within same kernel. For eg. PCIe and Ethernet driver.
>>> As per current design both drivers needs to be in sync before
>> requesting any message unit for avoiding any conflict. As these drivers
>> are completely independent. It is very difficult.
>>> Can it be possible to provide new API to take care it.
>> Do you have a real use case in mind where these message registers (not
>> MSIs) are used internally in this manner?
> Yes, we use for PCIe host/agent test case scenario.
> Host usage message register to interrupt Agent...
> Agent uses message register to generate irq_out (automatically generate MSI) to interrupt master. Please see RM for more details about irq_out
>
>
> Note: PCIe host/agent test scenario is used internally and we are working on pushing it out..

I believe this has been true for several years.

>> Perhaps an allocator could be added in the same patchset that adds such a
>> user.
> Yaa. It can be done. Otherwise module has to query each message unit for its availability.

No, instead the system designer should pick one. If it doesn't matter 
which one, then the designer is free to pick any.

An allocator can't work if you're going to mix drivers. For example, 
driver A needs MSRG0, and driver B doesn't care. Driver B loads first, 
the allocator selects MSGR0; driver A loads and fails. Having an 
allocator at all will create this conflict.

To prevent this scenario, either don't use a MSGR (can you configure 
anything else for irq_out?), or have the system designer choose all MSGRs.

Hollis Blanchard
Mentor Graphics, Embedded Systems Division

Re: [PATCH 2/2] powerpc: add support for MPIC message register API

[Scott Wood]

On Mon, 2 May 2011 09:03:53 -0700
Hollis Blanchard <hollis_blanchard@mentor.com> wrote:

> On 05/01/2011 08:41 PM, Kushwaha Prabhakar-B32579 wrote:
> >> Perhaps an allocator could be added in the same patchset that adds such a
> >> user.
> > Yaa. It can be done. Otherwise module has to query each message unit for its availability.
> 
> No, instead the system designer should pick one. If it doesn't matter 
> which one, then the designer is free to pick any.

We should avoid resorting to the notion of a "system designer" if possible
-- it's hard to avoid it when dealing with partitioning, but it shouldn't
be needed for unpartitioned basic driver operation.

> An allocator can't work if you're going to mix drivers. For example, 
> driver A needs MSRG0, and driver B doesn't care. Driver B loads first, 
> the allocator selects MSGR0; driver A loads and fails. Having an 
> allocator at all will create this conflict.

In the absence of partitioning, no driver should need a specific one.  With
partitioning, let the system designer mark those resources as reserved so
they don't get allocated. :-)

-Scott

Re: [PATCH 2/2] powerpc: add support for MPIC message register API

[Meador Inge]

On 05/03/2011 10:19 AM, Scott Wood wrote:
> In the absence of partitioning, no driver should need a specific one.  With
> partitioning, let the system designer mark those resources as reserved so
> they don't get allocated. :-)

That seem reasonable.  Back to the device tree then.  One option is to bring
back the sister-send property 'mpic-msgr-send-mask', which was discussed some
months back.  For example, consider:

	/* OS 1 */
	mpic_msgr_block0: mpic-msgr-block@41400 {
		compatible = "fsl,mpic-v3.1-msgr";
		reg = <0x41400 0x200>;
		interrupts = <0xb0 2 0xb2 2>;
		mpic-msgr-receive-mask = <0x5>;
		mpic-msgr-send-mask = <0xa>;
	};

	mpic_msgr_block1: mpic-msgr-block@42400 {
		compatible = "fsl,mpic-v3.1-msgr";
		reg = <0x42400 0x200>;
		interrupts = <0xb4 2 0xb6 2>;
		mpic-msgr-receive-mask = <0x5>;
	};

	/* OS 2 */
	mpic_msgr_block0: mpic-msgr-block@41400 {
		compatible = "fsl,mpic-v3.1-msgr";
		reg = <0x41400 0x200>;
		interrupts = <0xb0 2 0xb2 2>;
		mpic-msgr-receive-mask = <0xa>;
		mpic-msgr-send-mask = <0x5>;
	};

	mpic_msgr_block1: mpic-msgr-block@42400 {
		compatible = "fsl,mpic-v3.1-msgr";
		reg = <0x42400 0x200>;
		interrupts = <0xb4 2 0xb6 2>;
		mpic-msgr-send-mask = <0x5>;
	};

In block0 for both OSes, all registers are partitioned and are thus not
available for allocation.  In block1 for both OSes, registers 0 and 2 are
reserved and registers 1 and 3 are available for general allocation.

So any register mentioned in one of 'mpic-msgr-receive-mask' or
'mpic-msgr-send-mask' is out of the running for general allocation.
If neither of the properties appear, then all registers are available
for general allocation.

You could get into trouble with this method with cases like:

	/* OS 1 */
	mpic_msgr_block0: mpic-msgr-block@41400 {
		compatible = "fsl,mpic-v3.1-msgr";
		reg = <0x41400 0x200>;
		interrupts = <0xb0 2 0xb2 2>;
		mpic-msgr-send-mask = <0xa>;
	};

	/* OS 2 */
	mpic_msgr_block0: mpic-msgr-block@41400 {
		compatible = "fsl,mpic-v3.1-msgr";
		reg = <0x41400 0x200>;
		interrupts = <0xb0 2 0xb2 2>;
		mpic-msgr-receive-mask = <0x5>;
	};

Now OS 1 has registers 0 and 2 available for general allocation, which
OS 2 is receiving on.  However, we already have that problem if someone
botches the masks.  So I am not very worried about that.

Clearly this is just one method, but I think tagging what is available from the
device tree is a must.


Thoughts?


-- 
Meador Inge     | meador_inge AT mentor.com
Mentor Embedded | http://www.mentor.com/embedded-software

Re: [PATCH 2/2] powerpc: add support for MPIC message register API

[Scott Wood]

On Thu, 5 May 2011 16:41:29 -0500
Meador Inge <meador_inge@mentor.com> wrote:

> 	/* OS 1 */
> 	mpic_msgr_block0: mpic-msgr-block@41400 {
> 		compatible = "fsl,mpic-v3.1-msgr";
> 		reg = <0x41400 0x200>;
> 		interrupts = <0xb0 2 0xb2 2>;
> 		mpic-msgr-receive-mask = <0x5>;
> 		mpic-msgr-send-mask = <0xa>;
> 	};
> 
> 	mpic_msgr_block1: mpic-msgr-block@42400 {
> 		compatible = "fsl,mpic-v3.1-msgr";
> 		reg = <0x42400 0x200>;
> 		interrupts = <0xb4 2 0xb6 2>;
> 		mpic-msgr-receive-mask = <0x5>;
> 	};
> 
> 	/* OS 2 */
> 	mpic_msgr_block0: mpic-msgr-block@41400 {
> 		compatible = "fsl,mpic-v3.1-msgr";
> 		reg = <0x41400 0x200>;
> 		interrupts = <0xb0 2 0xb2 2>;
> 		mpic-msgr-receive-mask = <0xa>;
> 		mpic-msgr-send-mask = <0x5>;
> 	};
> 
> 	mpic_msgr_block1: mpic-msgr-block@42400 {
> 		compatible = "fsl,mpic-v3.1-msgr";
> 		reg = <0x42400 0x200>;
> 		interrupts = <0xb4 2 0xb6 2>;
> 		mpic-msgr-send-mask = <0x5>;
> 	};
> 
> In block0 for both OSes, all registers are partitioned and are thus not
> available for allocation.  In block1 for both OSes, registers 0 and 2 are
> reserved and registers 1 and 3 are available for general allocation.

How can both OSes independently own registers 1 and 3 for alloction?  And
where are the interrupt specifiers for these registers?

> So any register mentioned in one of 'mpic-msgr-receive-mask' or
> 'mpic-msgr-send-mask' is out of the running for general allocation.

mpic-msgr-receive-mask has to match interrupts -- it's not intended to be
an indication of usage, just that this partition is granted those
interrupts.

Plus, a dynamically allocated message register must be owned for both
sending and receiving, so it doesn't make sense to separate it.  I'd have
an "mpic-msgr-free-mask" property, which must be a subset of
"mpic-msgr-receive-mask".  If the register is not in free-mask, it is
reserved for a fixed purpose.  If free-mask is absent, all registers in the
receive-mask can be allocated.

So the above example would be:

	/* OS 1 */
	mpic_msgr_block0: mpic-msgr-block@41400 {
		compatible = "fsl,mpic-v3.1-msgr";
		reg = <0x41400 0x200>;
		interrupts = <0xb0 2 0xb2 2>;
		mpic-msgr-receive-mask = <0x5>;
		mpic-msgr-free-mask = <0>;
	};

	mpic_msgr_block1: mpic-msgr-block@42400 {
		compatible = "fsl,mpic-v3.1-msgr";
		reg = <0x42400 0x200>;
		interrupts = <0xb4 2 0xb5 2>;
		mpic-msgr-receive-mask = <0x3>;
		mpic-msgr-free-mask = <0x2>;
	};

	/* OS 2 */
	mpic_msgr_block0: mpic-msgr-block@41400 {
		compatible = "fsl,mpic-v3.1-msgr";
		reg = <0x41400 0x200>;
		interrupts = <0xb1 2 0xb3 2>;
		mpic-msgr-receive-mask = <0xa>;
		mpic-msgr-free-mask = <0>;
	};

	mpic_msgr_block1: mpic-msgr-block@42400 {
		compatible = "fsl,mpic-v3.1-msgr";
		reg = <0x42400 0x200>;
		interrupts = <0xb6 2 0xb7 2>;
		mpic-msgr-receive-mask = <0xc>;
		mpic-msgr-free-mask = <0x8>;
	};

mpic-msgr-send-mask could be added as well, as a permissions mechanism to
serve as extra protection against an improperly specified non-free message
register -- especially if the interface is exposed to a less-trusted realm
such as userspace, or if a hypervisor is reading the device tree to
determine what to allow guests to do.  In this case, just like
mpic-msgr-receive-mask, it would list both free and non-free message
registers that the partition can send to, and mpic-msgr-free-mask would be
a subset of both the send and receive masks.

> You could get into trouble with this method with cases like:
> 
> 	/* OS 1 */
> 	mpic_msgr_block0: mpic-msgr-block@41400 {
> 		compatible = "fsl,mpic-v3.1-msgr";
> 		reg = <0x41400 0x200>;
> 		interrupts = <0xb0 2 0xb2 2>;
> 		mpic-msgr-send-mask = <0xa>;
> 	};
> 
> 	/* OS 2 */
> 	mpic_msgr_block0: mpic-msgr-block@41400 {
> 		compatible = "fsl,mpic-v3.1-msgr";
> 		reg = <0x41400 0x200>;
> 		interrupts = <0xb0 2 0xb2 2>;
> 		mpic-msgr-receive-mask = <0x5>;
> 	};
> 
> Now OS 1 has registers 0 and 2 available for general allocation, which
> OS 2 is receiving on.  However, we already have that problem if someone
> botches the masks.  So I am not very worried about that.

There's a big difference between "botching the masks" and having no way to
express the situation properly.

BTW, the above fragment has the two OSes inappropriately sharing
interrupts, and OS1 has only two interrupts but no receive mask
(and therefore owns all 4 message registers for receive).  Only one OS
should be able to receive any given interrupt.

Consider this:

	/* OS 1 */
	mpic_msgr_block0: mpic-msgr-block@41400 {
		compatible = "fsl,mpic-v3.1-msgr";
		reg = <0x41400 0x200>;
		interrupts = <0xb0 2 0xb1 2>;
		mpic-msgr-receive-mask = <0x3>;
		mpic-msgr-send-mask = <0xc>;
	};

	/* OS 2 */
	mpic_msgr_block0: mpic-msgr-block@41400 {
		compatible = "fsl,mpic-v3.1-msgr";
		reg = <0x41400 0x200>;
		interrupts = <0xb2 2>;
		mpic-msgr-receive-mask = <0x4>;
		mpic-msgr-send-mask = <0x1>;
	};

	/* OS 3 */
	mpic_msgr_block0: mpic-msgr-block@41400 {
		compatible = "fsl,mpic-v3.1-msgr";
		reg = <0x41400 0x200>;
		interrupts = <0xb3 2>;
		mpic-msgr-receive-mask = <0x8>;
		mpic-msgr-send-mask = <0x2>;
	};

None of the message registers are actually free for allocation, but with
your scheme OS 2 would think it could allocate 1 and 3, and OS 3 would
think it could allocate 0 and 2.  Even if a register were actually free
from a reserved use, it would have to be owned by one partition.

-Scott

Re: [PATCH 2/2] powerpc: add support for MPIC message register API

[Meador Inge]

On 05/06/2011 02:29 PM, Scott Wood wrote:
> On Thu, 5 May 2011 16:41:29 -0500
> 
> How can both OSes independently own registers 1 and 3 for alloction?

They can't.  I just choose a horrible example.  It does point to a serious
flaw (which you allude to later) in inferring free registers from send/receive:
in some cases one will have to arbitrarily add registers to send/receive masks
just to keep registers out of the free allocation pool.  Your free-mask
proposal is better.

>> So any register mentioned in one of 'mpic-msgr-receive-mask' or
>> 'mpic-msgr-send-mask' is out of the running for general allocation.
> 
> mpic-msgr-receive-mask has to match interrupts -- it's not intended to be
> an indication of usage, just that this partition is granted those
> interrupts.
> 
> Plus, a dynamically allocated message register must be owned for both
> sending and receiving, so it doesn't make sense to separate it.  I'd have
> an "mpic-msgr-free-mask" property, which must be a subset of
> "mpic-msgr-receive-mask".  If the register is not in free-mask, it is
> reserved for a fixed purpose.  If free-mask is absent, all registers in the
> receive-mask can be allocated.
> 
> So the above example would be:
> 
> 	/* OS 1 */
> 	mpic_msgr_block0: mpic-msgr-block@41400 {
> 		compatible = "fsl,mpic-v3.1-msgr";
> 		reg = <0x41400 0x200>;
> 		interrupts = <0xb0 2 0xb2 2>;
> 		mpic-msgr-receive-mask = <0x5>;
> 		mpic-msgr-free-mask = <0>;
> 	};
> 
> 	mpic_msgr_block1: mpic-msgr-block@42400 {
> 		compatible = "fsl,mpic-v3.1-msgr";
> 		reg = <0x42400 0x200>;
> 		interrupts = <0xb4 2 0xb5 2>;
> 		mpic-msgr-receive-mask = <0x3>;
> 		mpic-msgr-free-mask = <0x2>;
> 	};
> 
> 	/* OS 2 */
> 	mpic_msgr_block0: mpic-msgr-block@41400 {
> 		compatible = "fsl,mpic-v3.1-msgr";
> 		reg = <0x41400 0x200>;
> 		interrupts = <0xb1 2 0xb3 2>;
> 		mpic-msgr-receive-mask = <0xa>;
> 		mpic-msgr-free-mask = <0>;
> 	};
> 
> 	mpic_msgr_block1: mpic-msgr-block@42400 {
> 		compatible = "fsl,mpic-v3.1-msgr";
> 		reg = <0x42400 0x200>;
> 		interrupts = <0xb6 2 0xb7 2>;
> 		mpic-msgr-receive-mask = <0xc>;
> 		mpic-msgr-free-mask = <0x8>;
> 	};
> 
> mpic-msgr-send-mask could be added as well, as a permissions mechanism to
> serve as extra protection against an improperly specified non-free message
> register -- especially if the interface is exposed to a less-trusted realm
> such as userspace, or if a hypervisor is reading the device tree to
> determine what to allow guests to do.  In this case, just like
> mpic-msgr-receive-mask, it would list both free and non-free message
> registers that the partition can send to, and mpic-msgr-free-mask would be
> a subset of both the send and receive masks.

free-mask seems reasonable.  Although, all of these masks are starting to get
rather complicated :-)

Anyway, I am going to cut a v2 patch without the dynamic allocation.  All
of this is getting complicated without a public use case.  I agree with your
previous suggestion that the dynamic allocation can be added as a part of the
patch set that actually uses it.

Thanks Scott.

-- 
Meador Inge     | meador_inge AT mentor.com
Mentor Embedded | http://www.mentor.com/embedded-software

[PATCH 1/2] powerpc: document the FSL MPIC message register binding

[Jia Hongtao]

This binding documents how the message register blocks found in some FSL
MPIC implementations shall be represented in a device tree.

Signed-off-by: Meador Inge <meador_inge@mentor.com>
Signed-off-by: Jia Hongtao <B38951@freescale.com>
Signed-off-by: Li Yang <leoli@freescale.com>
---
 .../devicetree/bindings/powerpc/fsl/mpic-msgr.txt  |   62 ++++++++++++++++++++
 1 files changed, 62 insertions(+), 0 deletions(-)
 create mode 100644 Documentation/devicetree/bindings/powerpc/fsl/mpic-msgr.txt

diff --git a/Documentation/devicetree/bindings/powerpc/fsl/mpic-msgr.txt b/Documentation/devicetree/bindings/powerpc/fsl/mpic-msgr.txt
new file mode 100644
index 0000000..b4ae70e
--- /dev/null
+++ b/Documentation/devicetree/bindings/powerpc/fsl/mpic-msgr.txt
@@ -0,0 +1,62 @@
+* FSL MPIC Message Registers
+
+This binding specifies what properties must be available in the device tree
+representation of the message register blocks found in some FSL MPIC
+implementations.
+
+Required properties:
+
+    - compatible: Specifies the compatibility list for the message register
+      block.  The type shall be <string> and the value shall be of the form
+      "fsl,mpic-v<version>-msgr", where <version> is the version number of
+      the MPIC containing the message registers.
+
+    - reg: Specifies the base physical address(s) and size(s) of the
+      message register block's addressable register space.  The type shall be
+      <prop-encoded-array>.
+
+    - interrupts: Specifies a list of interrupt source and level-sense pairs.
+      The type shall be <prop-encoded-array>.  The length shall be equal to
+      the number of registers that are available for receiving interrupts.
+
+Optional properties:
+
+    - mpic-msgr-receive-mask: Specifies what registers in the containing block
+      are allowed to receive interrupts.  The value is a bit mask where a set
+      bit at bit 'n' indicates that message register 'n' can receive interrupts.
+      The type shall be <prop-encoded-array>.  If not present, then all of
+      the message registers in the block are available.
+
+Aliases:
+
+    An alias should be created for every message register block.  They are not
+    required, though.  However, a particular implementation of this binding
+    may require aliases to be present.  Aliases are of the form
+    'mpic-msgr-block<n>', where <n> is an integer specifying the block's number.
+    Numbers shall start at 0.
+
+Example:
+
+	aliases {
+		mpic-msgr-block0 = &mpic_msgr_block0;
+		mpic-msgr-block1 = &mpic_msgr_block1;
+	};
+
+	mpic_msgr_block0: mpic-msgr-block@41400 {
+		compatible = "fsl,mpic-v3.1-msgr";
+		reg = <0x41400 0x200>;
+		// Message registers 0 and 2 in this block can receive interrupts on
+		// sources 0xb0 and 0xb2, respectively.
+		interrupts = <0xb0 2 0xb2 2>;
+		mpic-msgr-receive-mask = <0x5>;
+	};
+
+	mpic_msgr_block1: mpic-msgr-block@42400 {
+		compatible = "fsl,mpic-v3.1-msgr";
+		reg = <0x42400 0x200>;
+		// Message registers 0 and 2 in this block can receive interrupts on
+		// sources 0xb4 and 0xb6, respectively.
+		interrupts = <0xb4 2 0xb6 2>;
+		mpic-msgr-receive-mask = <0x5>;
+	};
+
-- 
1.7.5.1

[PATCH 2/2] powerpc: add support for MPIC message register API

[Jia Hongtao]

Some MPIC implementations contain one or more blocks of message registers
that are used to send messages between cores via IPIs.  A simple API has
been added to access (get/put, read, write, etc ...) these message registers.
The available message registers are initially discovered via nodes in the
device tree.  A separate commit contains a binding for the message register
nodes.

Signed-off-by: Meador Inge <meador_inge@mentor.com>
Signed-off-by: Jia Hongtao <B38951@freescale.com>
Signed-off-by: Li Yang <leoli@freescale.com>
---
This patch is an update for : http://patchwork.ozlabs.org/patch/98075/

Meador Inge addressed the following points from Ben's feedback:
* Drop the 'mpic_msgr.msr' field.
* Drop the 'mpic_msgr.mer' field in favor of address arithmetic off of  'mpic_msgr.addr'.
* Document the API.
* Disable MPIC register in 'mpic_msgr_put'.
* Put locking in 'mpic_msgr_disable'.
* s/EXPORT_SYMBOL/EXPORT_SYMBOL_GPL/g.
* Make 'mpic_msgr_write' and 'mpic_msgr_read' 'static inline'.

I just fixed the checkpatch errors and addressed the following item:
* In 'mpic_msgr_set_destination' have a wrapper that goes from Linux
  CPU number to HW CPU number.

 arch/powerpc/include/asm/mpic_msgr.h |  132 ++++++++++++++++
 arch/powerpc/platforms/Kconfig       |    8 +
 arch/powerpc/sysdev/Makefile         |    2 +
 arch/powerpc/sysdev/mpic_msgr.c      |  282 ++++++++++++++++++++++++++++++++++
 4 files changed, 424 insertions(+), 0 deletions(-)
 create mode 100644 arch/powerpc/include/asm/mpic_msgr.h
 create mode 100644 arch/powerpc/sysdev/mpic_msgr.c

diff --git a/arch/powerpc/include/asm/mpic_msgr.h b/arch/powerpc/include/asm/mpic_msgr.h
new file mode 100644
index 0000000..3ec37dc
--- /dev/null
+++ b/arch/powerpc/include/asm/mpic_msgr.h
@@ -0,0 +1,132 @@
+/*
+ * Copyright 2011-2012, Meador Inge, Mentor Graphics Corporation.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; version 2 of the
+ * License.
+ *
+ */
+
+#ifndef _ASM_MPIC_MSGR_H
+#define _ASM_MPIC_MSGR_H
+
+#include <linux/types.h>
+#include <linux/spinlock.h>
+
+struct mpic_msgr {
+	u32 __iomem *base;
+	u32 __iomem *mer;
+	int irq;
+	unsigned char in_use;
+	raw_spinlock_t lock;
+	int num;
+};
+
+/* Get a message register
+ *
+ * @reg_num:	the MPIC message register to get
+ *
+ * A pointer to the message register is returned.  If
+ * the message register asked for is already in use, then
+ * EBUSY is returned.  If the number given is not associated
+ * with an actual message register, then ENODEV is returned.
+ * Successfully getting the register marks it as in use.
+ */
+extern struct mpic_msgr *mpic_msgr_get(unsigned int reg_num);
+
+/* Relinquish a message register
+ *
+ * @msgr:	the message register to return
+ *
+ * Disables the given message register and marks it as free.
+ * After this call has completed successully the message
+ * register is available to be acquired by a call to
+ * mpic_msgr_get.
+ */
+extern void mpic_msgr_put(struct mpic_msgr *msgr);
+
+/* Enable a message register
+ *
+ * @msgr:	the message register to enable
+ *
+ * The given message register is enabled for sending
+ * messages.
+ */
+extern void mpic_msgr_enable(struct mpic_msgr *msgr);
+
+/* Disable a message register
+ *
+ * @msgr:	the message register to disable
+ *
+ * The given message register is disabled for sending
+ * messages.
+ */
+extern void mpic_msgr_disable(struct mpic_msgr *msgr);
+
+/* Write a message to a message register
+ *
+ * @msgr:	the message register to write to
+ * @message:	the message to write
+ *
+ * The given 32-bit message is written to the given message
+ * register.  Writing to an enabled message registers fires
+ * an interrupt.
+ */
+static inline void mpic_msgr_write(struct mpic_msgr *msgr, u32 message)
+{
+	out_be32(msgr->base, message);
+}
+
+/* Read a message from a message register
+ *
+ * @msgr:	the message register to read from
+ *
+ * Returns the 32-bit value currently in the given message register.
+ * Upon reading the register any interrupts for that register are
+ * cleared.
+ */
+static inline u32 mpic_msgr_read(struct mpic_msgr *msgr)
+{
+	return in_be32(msgr->base);
+}
+
+/* Clear a message register
+ *
+ * @msgr:	the message register to clear
+ *
+ * Clears any interrupts associated with the given message register.
+ */
+static inline void mpic_msgr_clear(struct mpic_msgr *msgr)
+{
+	(void) mpic_msgr_read(msgr);
+}
+
+/* Set the destination CPU for the message register
+ *
+ * @msgr:	the message register whose destination is to be set
+ * @cpu_num:	the Linux CPU number to bind the message register to
+ *
+ * Note that the CPU number given is the CPU number used by the kernel
+ * and *not* the actual hardware CPU number.
+ */
+static inline void mpic_msgr_set_destination(struct mpic_msgr *msgr,
+					     u32 cpu_num)
+{
+	out_be32(msgr->base, 1 << get_hard_smp_processor_id(cpu_num));
+}
+
+/* Get the IRQ number for the message register
+ * @msgr:	the message register whose IRQ is to be returned
+ *
+ * Returns the IRQ number associated with the given message register.
+ * NO_IRQ is returned if this message register is not capable of
+ * receiving interrupts.  What message register can and cannot receive
+ * interrupts is specified in the device tree for the system.
+ */
+static inline int mpic_msgr_get_irq(struct mpic_msgr *msgr)
+{
+	return msgr->irq;
+}
+
+#endif
diff --git a/arch/powerpc/platforms/Kconfig b/arch/powerpc/platforms/Kconfig
index eb294a6..fc4f5580 100644
--- a/arch/powerpc/platforms/Kconfig
+++ b/arch/powerpc/platforms/Kconfig
@@ -87,6 +87,14 @@ config MPIC_WEIRD
 	bool
 	default n
 
+config MPIC_MSGR
+	bool "MPIC message register support"
+	depends on MPIC
+	default n
+	help
+	  Enables support for the MPIC message registers.  These
+	  registers are used for inter-processor communication.
+
 config PPC_I8259
 	bool
 	default n
diff --git a/arch/powerpc/sysdev/Makefile b/arch/powerpc/sysdev/Makefile
index 5e37b47..1951dc7 100644
--- a/arch/powerpc/sysdev/Makefile
+++ b/arch/powerpc/sysdev/Makefile
@@ -4,6 +4,8 @@ ccflags-$(CONFIG_PPC64)		:= -mno-minimal-toc
 
 mpic-msi-obj-$(CONFIG_PCI_MSI)	+= mpic_msi.o mpic_u3msi.o mpic_pasemi_msi.o
 obj-$(CONFIG_MPIC)		+= mpic.o $(mpic-msi-obj-y)
+mpic-msgr-obj-$(CONFIG_MPIC_MSGR)	+= mpic_msgr.o
+obj-$(CONFIG_MPIC)		+= mpic.o $(mpic-msi-obj-y) $(mpic-msgr-obj-y)
 obj-$(CONFIG_PPC_EPAPR_HV_PIC)	+= ehv_pic.o
 fsl-msi-obj-$(CONFIG_PCI_MSI)	+= fsl_msi.o
 obj-$(CONFIG_PPC_MSI_BITMAP)	+= msi_bitmap.o
diff --git a/arch/powerpc/sysdev/mpic_msgr.c b/arch/powerpc/sysdev/mpic_msgr.c
new file mode 100644
index 0000000..6e7fa38
--- /dev/null
+++ b/arch/powerpc/sysdev/mpic_msgr.c
@@ -0,0 +1,282 @@
+/*
+ * Copyright 2011-2012, Meador Inge, Mentor Graphics Corporation.
+ *
+ * Some ideas based on un-pushed work done by Vivek Mahajan, Jason Jin, and
+ * Mingkai Hu from Freescale Semiconductor, Inc.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; version 2 of the
+ * License.
+ *
+ */
+
+#include <linux/list.h>
+#include <linux/of_platform.h>
+#include <linux/errno.h>
+#include <asm/prom.h>
+#include <asm/hw_irq.h>
+#include <asm/ppc-pci.h>
+#include <asm/mpic_msgr.h>
+
+#define MPIC_MSGR_REGISTERS_PER_BLOCK	4
+#define MPIC_MSGR_STRIDE		0x10
+#define MPIC_MSGR_MER_OFFSET		0x100
+#define MSGR_INUSE			0
+#define MSGR_FREE			1
+
+static struct mpic_msgr **mpic_msgrs;
+static unsigned int mpic_msgr_count;
+
+static inline void _mpic_msgr_mer_write(struct mpic_msgr *msgr, u32 value)
+{
+	out_be32(msgr->mer, value);
+}
+
+static inline u32 _mpic_msgr_mer_read(struct mpic_msgr *msgr)
+{
+	return in_be32(msgr->mer);
+}
+
+static inline void _mpic_msgr_disable(struct mpic_msgr *msgr)
+{
+	u32 mer = _mpic_msgr_mer_read(msgr);
+
+	_mpic_msgr_mer_write(msgr, mer & ~(1 << msgr->num));
+}
+
+struct mpic_msgr *mpic_msgr_get(unsigned int reg_num)
+{
+	unsigned long flags;
+	struct mpic_msgr *msgr;
+
+	/* Assume busy until proven otherwise.  */
+	msgr = ERR_PTR(-EBUSY);
+
+	if (reg_num >= mpic_msgr_count)
+		return ERR_PTR(-ENODEV);
+
+	raw_spin_lock_irqsave(&msgr->lock, flags);
+	if (mpic_msgrs[reg_num]->in_use == MSGR_FREE) {
+		msgr = mpic_msgrs[reg_num];
+		msgr->in_use = MSGR_INUSE;
+	}
+	raw_spin_unlock_irqrestore(&msgr->lock, flags);
+
+	return msgr;
+}
+EXPORT_SYMBOL_GPL(mpic_msgr_get);
+
+void mpic_msgr_put(struct mpic_msgr *msgr)
+{
+	unsigned long flags;
+
+	raw_spin_lock_irqsave(&msgr->lock, flags);
+	msgr->in_use = MSGR_FREE;
+	_mpic_msgr_disable(msgr);
+	raw_spin_unlock_irqrestore(&msgr->lock, flags);
+}
+EXPORT_SYMBOL_GPL(mpic_msgr_put);
+
+void mpic_msgr_enable(struct mpic_msgr *msgr)
+{
+	unsigned long flags;
+	u32 mer;
+
+	raw_spin_lock_irqsave(&msgr->lock, flags);
+	mer = _mpic_msgr_mer_read(msgr);
+	_mpic_msgr_mer_write(msgr, mer | (1 << msgr->num));
+	raw_spin_unlock_irqrestore(&msgr->lock, flags);
+}
+EXPORT_SYMBOL_GPL(mpic_msgr_enable);
+
+void mpic_msgr_disable(struct mpic_msgr *msgr)
+{
+	unsigned long flags;
+
+	raw_spin_lock_irqsave(&msgr->lock, flags);
+	_mpic_msgr_disable(msgr);
+	raw_spin_unlock_irqrestore(&msgr->lock, flags);
+}
+EXPORT_SYMBOL_GPL(mpic_msgr_disable);
+
+/* The following three functions are used to compute the order and number of
+ * the message register blocks.  They are clearly very inefficent.  However,
+ * they are called *only* a few times during device initialization.
+ */
+static unsigned int mpic_msgr_number_of_blocks(void)
+{
+	unsigned int count;
+	struct device_node *aliases;
+
+	count = 0;
+	aliases = of_find_node_by_name(NULL, "aliases");
+
+	if (aliases) {
+		char buf[32];
+
+		for (;;) {
+			snprintf(buf, sizeof(buf), "mpic-msgr-block%d", count);
+			if (!of_find_property(aliases, buf, NULL))
+				break;
+
+			count += 1;
+		}
+	}
+
+	return count;
+}
+
+static unsigned int mpic_msgr_number_of_registers(void)
+{
+	return mpic_msgr_number_of_blocks() * MPIC_MSGR_REGISTERS_PER_BLOCK;
+}
+
+static int mpic_msgr_block_number(struct device_node *node)
+{
+	struct device_node *aliases;
+	unsigned int index, number_of_blocks;
+	char buf[64];
+
+	number_of_blocks = mpic_msgr_number_of_blocks();
+	aliases = of_find_node_by_name(NULL, "aliases");
+	if (!aliases)
+		return -1;
+
+	for (index = 0; index < number_of_blocks; ++index) {
+		struct property *prop;
+
+		snprintf(buf, sizeof(buf), "mpic-msgr-block%d", index);
+		prop = of_find_property(aliases, buf, NULL);
+		if (node == of_find_node_by_path(prop->value))
+			break;
+	}
+
+	return index == number_of_blocks ? -1 : index;
+}
+
+/* The probe function for a single message register block.
+ */
+static __devinit int mpic_msgr_probe(struct platform_device *dev)
+{
+	void __iomem *msgr_block_addr;
+	int block_number;
+	struct resource rsrc;
+	unsigned int i;
+	unsigned int irq_index;
+	struct device_node *np = dev->dev.of_node;
+	unsigned int receive_mask;
+	const unsigned int *prop;
+
+	if (!np) {
+		dev_err(&dev->dev, "Device OF-Node is NULL");
+		return -EFAULT;
+	}
+
+	/* Allocate the message register array upon the first device
+	 * registered.
+	 */
+	if (!mpic_msgrs) {
+		mpic_msgr_count = mpic_msgr_number_of_registers();
+		dev_info(&dev->dev, "Found %d message registers\n",
+				mpic_msgr_count);
+
+		mpic_msgrs = kzalloc(sizeof(struct mpic_msgr) * mpic_msgr_count,
+							 GFP_KERNEL);
+		if (!mpic_msgrs) {
+			dev_err(&dev->dev,
+				"No memory for message register blocks\n");
+			return -ENOMEM;
+		}
+	}
+	dev_info(&dev->dev, "Of-device full name %s\n", np->full_name);
+
+	/* IO map the message register block. */
+	of_address_to_resource(np, 0, &rsrc);
+	msgr_block_addr = ioremap(rsrc.start, rsrc.end - rsrc.start);
+	if (!msgr_block_addr) {
+		dev_err(&dev->dev, "Failed to iomap MPIC message registers");
+		return -EFAULT;
+	}
+
+	/* Ensure the block has a defined order. */
+	block_number = mpic_msgr_block_number(np);
+	if (block_number < 0) {
+		dev_err(&dev->dev,
+			"Failed to find message register block alias\n");
+		return -ENODEV;
+	}
+	dev_info(&dev->dev, "Setting up message register block %d\n",
+			block_number);
+
+	/* Grab the receive mask which specifies what registers can receive
+	 * interrupts.
+	 */
+	prop = of_get_property(np, "mpic-msgr-receive-mask", NULL);
+	receive_mask = (prop) ? *prop : 0xF;
+
+	/* Build up the appropriate message register data structures. */
+	for (i = 0, irq_index = 0; i < MPIC_MSGR_REGISTERS_PER_BLOCK; ++i) {
+		struct mpic_msgr *msgr;
+		unsigned int reg_number;
+
+		msgr = kzalloc(sizeof(struct mpic_msgr), GFP_KERNEL);
+		if (!msgr) {
+			dev_err(&dev->dev, "No memory for message register\n");
+			return -ENOMEM;
+		}
+
+		reg_number = block_number * MPIC_MSGR_REGISTERS_PER_BLOCK + i;
+		msgr->base = msgr_block_addr + i * MPIC_MSGR_STRIDE;
+		msgr->mer = msgr->base + MPIC_MSGR_MER_OFFSET;
+		msgr->in_use = MSGR_FREE;
+		msgr->num = i;
+		raw_spin_lock_init(&msgr->lock);
+
+		if (receive_mask & (1 << i)) {
+			struct resource irq;
+
+			if (of_irq_to_resource(np, irq_index, &irq) == NO_IRQ) {
+				dev_err(&dev->dev,
+						"Missing interrupt specifier");
+				kfree(msgr);
+				return -EFAULT;
+			}
+			msgr->irq = irq.start;
+			irq_index += 1;
+		} else {
+			msgr->irq = NO_IRQ;
+		}
+
+		mpic_msgrs[reg_number] = msgr;
+		mpic_msgr_disable(msgr);
+		dev_info(&dev->dev, "Register %d initialized: irq %d\n",
+				reg_number, msgr->irq);
+
+	}
+
+	return 0;
+}
+
+static const struct of_device_id mpic_msgr_ids[] = {
+	{
+		.compatible = "fsl,mpic-v3.1-msgr",
+		.data = NULL,
+	},
+	{}
+};
+
+static struct platform_driver mpic_msgr_driver = {
+	.driver = {
+		.name = "mpic-msgr",
+		.owner = THIS_MODULE,
+		.of_match_table = mpic_msgr_ids,
+	},
+	.probe = mpic_msgr_probe,
+};
+
+static __init int mpic_msgr_init(void)
+{
+	return platform_driver_register(&mpic_msgr_driver);
+}
+subsys_initcall(mpic_msgr_init);
-- 
1.7.5.1

Re: [PATCH 2/2] powerpc: add support for MPIC message register API

[Kumar Gala]

On Feb 16, 2012, at 8:49 PM, Jia Hongtao wrote:

> Some MPIC implementations contain one or more blocks of message =
registers
> that are used to send messages between cores via IPIs.  A simple API =
has
> been added to access (get/put, read, write, etc ...) these message =
registers.
> The available message registers are initially discovered via nodes in =
the
> device tree.  A separate commit contains a binding for the message =
register
> nodes.
>=20
> Signed-off-by: Meador Inge <meador_inge@mentor.com>
> Signed-off-by: Jia Hongtao <B38951@freescale.com>
> Signed-off-by: Li Yang <leoli@freescale.com>
> ---
> This patch is an update for : http://patchwork.ozlabs.org/patch/98075/
>=20
> Meador Inge addressed the following points from Ben's feedback:
> * Drop the 'mpic_msgr.msr' field.
> * Drop the 'mpic_msgr.mer' field in favor of address arithmetic off of =
 'mpic_msgr.addr'.
> * Document the API.
> * Disable MPIC register in 'mpic_msgr_put'.
> * Put locking in 'mpic_msgr_disable'.
> * s/EXPORT_SYMBOL/EXPORT_SYMBOL_GPL/g.
> * Make 'mpic_msgr_write' and 'mpic_msgr_read' 'static inline'.
>=20
> I just fixed the checkpatch errors and addressed the following item:
> * In 'mpic_msgr_set_destination' have a wrapper that goes from Linux
>  CPU number to HW CPU number.
>=20
> arch/powerpc/include/asm/mpic_msgr.h |  132 ++++++++++++++++
> arch/powerpc/platforms/Kconfig       |    8 +
> arch/powerpc/sysdev/Makefile         |    2 +
> arch/powerpc/sysdev/mpic_msgr.c      |  282 =
++++++++++++++++++++++++++++++++++
> 4 files changed, 424 insertions(+), 0 deletions(-)
> create mode 100644 arch/powerpc/include/asm/mpic_msgr.h
> create mode 100644 arch/powerpc/sysdev/mpic_msgr.c

applied

- k=