[PATCH v2 00/26] x86, irq: support ioapic device hotplug

[PATCH v2 00/26] x86, irq: support ioapic device hotplug

[Yinghai Lu]

Hi,

Current x86 code does not support iapic hotplug yet.

This patcheset will try to pre-reserve irq block in allocated_irqs bitmap.
for hot add ioapic controller, also record irq_base in gsi_config, so later
could use it to convert gsi to irq for pci device using that ioapic controller.
Need to new realloc_irq to allocate irq_desc/cfg later.

It is based on x86/apic and pci/next that is supporting pci root bus hotplug now.

could get them from:
        git://git.kernel.org/pub/scm/linux/kernel/git/yinghai/linux-yinghai.git for-x86-irq

Yinghai Lu (26):
  x86, irq: Change irq_remap_modify_chip_defaults to static
  x86, irq: Modify irq chip once for irq remapping
  x86, irq: Print out MSI/MSI-X clearly
  x86, irq: Show MSI-X in /proc/interrupt
  x86, irq: Show pci device name for msi in /proc/interrupt
  x86, irq: Make dmar_msi/hpet_msi irq_chip name consistent
  ia64, irq: Add dummy create_irq_nr()
  iommu, irq: Allocate irq_desc for dmar_msi with local node
  x86, irq: kill create_irq()
  x86, irq: Convert irq_2_pin list to generic list
  genirq: Split __irq_reserve_irqs from irq_alloc_descs
  x86, irq: Add realloc_irq_and_cfg_at()
  x86, irq: Move down arch_early_irq_init()
  x86, irq: Split out alloc_ioapic_save_registers()
  xen, irq: call irq_realloc_desc_at() at first
  x86, irq: pre-reserve irq range/realloc for booting path
  x86, irq: Add ioapic_gsi_to_irq
  genirq: Bail out early in free_desc()
  x86, irq: More strict checking about registering ioapic
  x86, irq: Make mp_register_ioapic handle hotadd ioapic
  x86, irq: Add mp_unregister_ioapic to handle hot-remove ioapic
  x86, irq: Make ioapics loop skip blank slots
  x86, ioapic: Find usable ioapic id for 64bit.
  x86: Move declaration for mp_register_ioapic()
  PCI: Disable mem in the ioapic removing path
  PCI, x86, ACPI: Add ioapic hotplug support with acpi host bridge.

 arch/ia64/kernel/irq_ia64.c          |   10 +
 arch/x86/include/asm/hw_irq.h        |    2 +-
 arch/x86/include/asm/io_apic.h       |    7 +
 arch/x86/include/asm/irq_remapping.h |    6 -
 arch/x86/include/asm/mpspec.h        |   21 +-
 arch/x86/kernel/acpi/boot.c          |   32 +--
 arch/x86/kernel/apic/apic.c          |    9 +-
 arch/x86/kernel/apic/io_apic.c       |  446 +++++++++++++++++++++++++---------
 drivers/acpi/pci_root.c              |    4 +
 drivers/iommu/dmar.c                 |    2 +-
 drivers/iommu/irq_remapping.c        |   24 +-
 drivers/pci/Kconfig                  |    3 +-
 drivers/pci/ioapic.c                 |  243 ++++++++++++------
 drivers/xen/events.c                 |    8 +-
 include/linux/irq.h                  |    6 +
 include/linux/pci-acpi.h             |    8 +
 kernel/irq/irqdesc.c                 |   83 +++++--
 17 files changed, 673 insertions(+), 241 deletions(-)

-- 
1.7.10.4

[PATCH v2 01/26] x86, irq: Change irq_remap_modify_chip_defaults to static

[Yinghai Lu]

Change irq_remap_modify_chip_defaults() to static, as we have no
outside user.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
Cc: Joerg Roedel <joro@8bytes.org>
Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Cc: Sebastian Andrzej Siewior <sebastian@breakpoint.cc>
---
 arch/x86/include/asm/irq_remapping.h |    6 ------
 drivers/iommu/irq_remapping.c        |    2 +-
 2 files changed, 1 insertion(+), 7 deletions(-)

diff --git a/arch/x86/include/asm/irq_remapping.h b/arch/x86/include/asm/irq_remapping.h
index 95fd352..06279a6 100644
--- a/arch/x86/include/asm/irq_remapping.h
+++ b/arch/x86/include/asm/irq_remapping.h
@@ -48,8 +48,6 @@ extern bool setup_remapped_irq(int irq,
 			       struct irq_cfg *cfg,
 			       struct irq_chip *chip);
 
-void irq_remap_modify_chip_defaults(struct irq_chip *chip);
-
 #else  /* CONFIG_IRQ_REMAP */
 
 static inline void setup_irq_remapping_ops(void) { }
@@ -82,10 +80,6 @@ static inline void panic_if_irq_remap(const char *msg)
 {
 }
 
-static inline void irq_remap_modify_chip_defaults(struct irq_chip *chip)
-{
-}
-
 static inline bool setup_remapped_irq(int irq,
 				      struct irq_cfg *cfg,
 				      struct irq_chip *chip)
diff --git a/drivers/iommu/irq_remapping.c b/drivers/iommu/irq_remapping.c
index d56f8c1..01ed477 100644
--- a/drivers/iommu/irq_remapping.c
+++ b/drivers/iommu/irq_remapping.c
@@ -368,7 +368,7 @@ static void ir_print_prefix(struct irq_data *data, struct seq_file *p)
 	seq_printf(p, " IR-%s", data->chip->name);
 }
 
-void irq_remap_modify_chip_defaults(struct irq_chip *chip)
+static void irq_remap_modify_chip_defaults(struct irq_chip *chip)
 {
 	chip->irq_print_chip = ir_print_prefix;
 	chip->irq_ack = ir_ack_apic_edge;
-- 
1.7.10.4

[PATCH v2 02/26] x86, irq: Modify irq chip once for irq remapping

[Yinghai Lu]

Now after irq remapping is enabled, irq_chip fields are modified during
every irq setup.

Only need to change them one during we enable irq mapping.

Also change irq_remap_modify_chip_defaults() to __init.

We don't need to use #ifdef in irq_remap_modify_chips(), as
IRQ_REMAP only support x86_64 AND X86_IO_APIC AND PCI_MSI,
also HPET_TIMER is set when x86_64 is set.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Cc: Sebastian Andrzej Siewior <sebastian@breakpoint.cc>
Cc: Joerg Roedel <joro@8bytes.org>
---
 arch/x86/include/asm/io_apic.h |    5 +++++
 arch/x86/kernel/apic/apic.c    |    9 ++++++++-
 arch/x86/kernel/apic/io_apic.c |    8 +++-----
 drivers/iommu/irq_remapping.c  |   11 +++++++++--
 4 files changed, 25 insertions(+), 8 deletions(-)

diff --git a/arch/x86/include/asm/io_apic.h b/arch/x86/include/asm/io_apic.h
index 459e50a..eaff3ad 100644
--- a/arch/x86/include/asm/io_apic.h
+++ b/arch/x86/include/asm/io_apic.h
@@ -215,6 +215,11 @@ static inline void io_apic_modify(unsigned int apic, unsigned int reg, unsigned
 
 extern void io_apic_eoi(unsigned int apic, unsigned int vector);
 
+extern struct irq_chip ioapic_chip;
+extern struct irq_chip msi_chip;
+extern struct irq_chip hpet_msi_type;
+void irq_remap_modify_chips(void);
+
 #else  /* !CONFIG_X86_IO_APIC */
 
 #define io_apic_assign_pci_irqs 0
diff --git a/arch/x86/kernel/apic/apic.c b/arch/x86/kernel/apic/apic.c
index a5b4dce..84815b1 100644
--- a/arch/x86/kernel/apic/apic.c
+++ b/arch/x86/kernel/apic/apic.c
@@ -1552,6 +1552,8 @@ void enable_x2apic(void)
 int __init enable_IR(void)
 {
 #ifdef CONFIG_IRQ_REMAP
+	int ret;
+
 	if (!irq_remapping_supported()) {
 		pr_debug("intr-remapping not supported\n");
 		return -1;
@@ -1563,7 +1565,12 @@ int __init enable_IR(void)
 		return -1;
 	}
 
-	return irq_remapping_enable();
+	ret = irq_remapping_enable();
+
+	if (ret >= 0)
+		irq_remap_modify_chips();
+
+	return ret;
 #endif
 	return -1;
 }
diff --git a/arch/x86/kernel/apic/io_apic.c b/arch/x86/kernel/apic/io_apic.c
index 9ed796c..f78f7f6 100644
--- a/arch/x86/kernel/apic/io_apic.c
+++ b/arch/x86/kernel/apic/io_apic.c
@@ -1238,8 +1238,6 @@ void __setup_vector_irq(int cpu)
 	raw_spin_unlock(&vector_lock);
 }
 
-static struct irq_chip ioapic_chip;
-
 #ifdef CONFIG_X86_32
 static inline int IO_APIC_irq_trigger(int irq)
 {
@@ -2511,7 +2509,7 @@ static void ack_apic_level(struct irq_data *data)
 	ioapic_irqd_unmask(data, cfg, masked);
 }
 
-static struct irq_chip ioapic_chip __read_mostly = {
+struct irq_chip ioapic_chip __read_mostly = {
 	.name			= "IO-APIC",
 	.irq_startup		= startup_ioapic_irq,
 	.irq_mask		= mask_ioapic_irq,
@@ -3093,7 +3091,7 @@ msi_set_affinity(struct irq_data *data, const struct cpumask *mask, bool force)
  * IRQ Chip for MSI PCI/PCI-X/PCI-Express Devices,
  * which implement the MSI or MSI-X Capability Structure.
  */
-static struct irq_chip msi_chip = {
+struct irq_chip msi_chip = {
 	.name			= "PCI-MSI",
 	.irq_unmask		= unmask_msi_irq,
 	.irq_mask		= mask_msi_irq,
@@ -3240,7 +3238,7 @@ static int hpet_msi_set_affinity(struct irq_data *data,
 	return IRQ_SET_MASK_OK_NOCOPY;
 }
 
-static struct irq_chip hpet_msi_type = {
+struct irq_chip hpet_msi_type = {
 	.name = "HPET_MSI",
 	.irq_unmask = hpet_msi_unmask,
 	.irq_mask = hpet_msi_mask,
diff --git a/drivers/iommu/irq_remapping.c b/drivers/iommu/irq_remapping.c
index 01ed477..33918c3 100644
--- a/drivers/iommu/irq_remapping.c
+++ b/drivers/iommu/irq_remapping.c
@@ -368,19 +368,26 @@ static void ir_print_prefix(struct irq_data *data, struct seq_file *p)
 	seq_printf(p, " IR-%s", data->chip->name);
 }
 
-static void irq_remap_modify_chip_defaults(struct irq_chip *chip)
+static void __init irq_remap_modify_chip_defaults(struct irq_chip *chip)
 {
+	printk(KERN_DEBUG "irq_chip: %s ==> IR-%s", chip->name, chip->name);
 	chip->irq_print_chip = ir_print_prefix;
 	chip->irq_ack = ir_ack_apic_edge;
 	chip->irq_eoi = ir_ack_apic_level;
 	chip->irq_set_affinity = x86_io_apic_ops.set_affinity;
 }
 
+void __init irq_remap_modify_chips(void)
+{
+	irq_remap_modify_chip_defaults(&ioapic_chip);
+	irq_remap_modify_chip_defaults(&msi_chip);
+	irq_remap_modify_chip_defaults(&hpet_msi_type);
+}
+
 bool setup_remapped_irq(int irq, struct irq_cfg *cfg, struct irq_chip *chip)
 {
 	if (!irq_remapped(cfg))
 		return false;
 	irq_set_status_flags(irq, IRQ_MOVE_PCNTXT);
-	irq_remap_modify_chip_defaults(chip);
 	return true;
 }
-- 
1.7.10.4

Re: [PATCH v2 02/26] x86, irq: Modify irq chip once for irq remapping

[Thomas Gleixner]

On Fri, 8 Feb 2013, Yinghai Lu wrote:

> Now after irq remapping is enabled, irq_chip fields are modified during
> every irq setup.
> 
> Only need to change them one during we enable irq mapping.

Can you please start to write sensible changelogs? I'm really sick of
your sloppyness.

> Also change irq_remap_modify_chip_defaults() to __init.

Why can you do that ?
  
> We don't need to use #ifdef in irq_remap_modify_chips(), as
> IRQ_REMAP only support x86_64 AND X86_IO_APIC AND PCI_MSI,
> also HPET_TIMER is set when x86_64 is set.

You should not describe what the patch is doing, you are again missing
to describe why the patch is doing this. 

Sigh. 

      tglx

Re: [PATCH v2 02/26] x86, irq: Modify irq chip once for irq remapping

[Thomas Gleixner]

On Fri, 8 Mar 2013, Thomas Gleixner wrote:
> On Fri, 8 Feb 2013, Yinghai Lu wrote:
> 
> > Now after irq remapping is enabled, irq_chip fields are modified during
> > every irq setup.
> > 
> > Only need to change them one during we enable irq mapping.
> 
> Can you please start to write sensible changelogs? I'm really sick of
> your sloppyness.
> 
> > Also change irq_remap_modify_chip_defaults() to __init.
> 
> Why can you do that ?
>   
> > We don't need to use #ifdef in irq_remap_modify_chips(), as
> > IRQ_REMAP only support x86_64 AND X86_IO_APIC AND PCI_MSI,
> > also HPET_TIMER is set when x86_64 is set.
> 
> You should not describe what the patch is doing, you are again missing
> to describe why the patch is doing this. 

And just for clarification. If you are not going to provide proper
changelogs for _all_ patches of the series, this stuff is going
directly towards /dev/null. I'm not wasting my time to review any of
that otherwise.

Thanks,

	tglx

Re: [PATCH v2 02/26] x86, irq: Modify irq chip once for irq remapping

[Yinghai Lu]

On Fri, Mar 8, 2013 at 12:10 PM, Thomas Gleixner <tglx@linutronix.de> wrote:
>
> And just for clarification. If you are not going to provide proper
> changelogs for _all_ patches of the series, this stuff is going
> directly towards /dev/null. I'm not wasting my time to review any of
> that otherwise.

ok, will try to update change logs and resend them next week.

Thanks

Yinghai

[PATCH v2 03/26] x86, irq: Print out MSI/MSI-X clearly

[Yinghai Lu]

Print out exact MSI or MSI-X instead of MSI/MSI-X.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
Cc: Joerg Roedel <joro@8bytes.org>
Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Cc: Sebastian Andrzej Siewior <sebastian@breakpoint.cc>
---
 arch/x86/kernel/apic/io_apic.c |    3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/arch/x86/kernel/apic/io_apic.c b/arch/x86/kernel/apic/io_apic.c
index f78f7f6..059589f 100644
--- a/arch/x86/kernel/apic/io_apic.c
+++ b/arch/x86/kernel/apic/io_apic.c
@@ -3125,7 +3125,8 @@ int setup_msi_irq(struct pci_dev *dev, struct msi_desc *msidesc,
 
 	irq_set_chip_and_handler_name(irq, chip, handle_edge_irq, "edge");
 
-	dev_printk(KERN_DEBUG, &dev->dev, "irq %d for MSI/MSI-X\n", irq);
+	dev_printk(KERN_DEBUG, &dev->dev, "irq %d for MSI%s\n", irq,
+			 msidesc->msi_attrib.is_msix ? "-X" : "");
 
 	return 0;
 }
-- 
1.7.10.4

[PATCH v2 04/26] x86, irq: Show MSI-X in /proc/interrupt

[Yinghai Lu]

Now MSI-X is shown as MSI in /proc/interrupt.

We could use new added irq_print_chip() interface to append -X for MSI-X.

-v2: do not need to check if msi_desc is null in msi_irq_print_chip().

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
Cc: Joerg Roedel <joro@8bytes.org>
Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Cc: Sebastian Andrzej Siewior <sebastian@breakpoint.cc>
---
 arch/x86/kernel/apic/io_apic.c |    7 +++++++
 drivers/iommu/irq_remapping.c  |    5 ++++-
 2 files changed, 11 insertions(+), 1 deletion(-)

diff --git a/arch/x86/kernel/apic/io_apic.c b/arch/x86/kernel/apic/io_apic.c
index 059589f..f8d4d8d 100644
--- a/arch/x86/kernel/apic/io_apic.c
+++ b/arch/x86/kernel/apic/io_apic.c
@@ -3087,6 +3087,12 @@ msi_set_affinity(struct irq_data *data, const struct cpumask *mask, bool force)
 	return IRQ_SET_MASK_OK_NOCOPY;
 }
 
+static void msi_irq_print_chip(struct irq_data *data, struct seq_file *p)
+{
+	seq_printf(p, " %s%s", data->chip->name,
+			data->msi_desc->msi_attrib.is_msix ? "-X" : "");
+}
+
 /*
  * IRQ Chip for MSI PCI/PCI-X/PCI-Express Devices,
  * which implement the MSI or MSI-X Capability Structure.
@@ -3098,6 +3104,7 @@ struct irq_chip msi_chip = {
 	.irq_ack		= ack_apic_edge,
 	.irq_set_affinity	= msi_set_affinity,
 	.irq_retrigger		= ioapic_retrigger_irq,
+	.irq_print_chip		= msi_irq_print_chip,
 };
 
 int setup_msi_irq(struct pci_dev *dev, struct msi_desc *msidesc,
diff --git a/drivers/iommu/irq_remapping.c b/drivers/iommu/irq_remapping.c
index 33918c3..e245ab0 100644
--- a/drivers/iommu/irq_remapping.c
+++ b/drivers/iommu/irq_remapping.c
@@ -365,7 +365,10 @@ static void ir_ack_apic_level(struct irq_data *data)
 
 static void ir_print_prefix(struct irq_data *data, struct seq_file *p)
 {
-	seq_printf(p, " IR-%s", data->chip->name);
+	seq_printf(p, " IR-%s%s", data->chip->name,
+			data->msi_desc ?
+			 (data->msi_desc->msi_attrib.is_msix ? "-X" : "")
+			 : "");
 }
 
 static void __init irq_remap_modify_chip_defaults(struct irq_chip *chip)
-- 
1.7.10.4

[PATCH v2 05/26] x86, irq: Show pci device name for msi in /proc/interrupt

[Yinghai Lu]

We could find out which device is using that MSI/MSI-X.
Use irq_print_chip() to append pci device name.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
Cc: Joerg Roedel <joro@8bytes.org>
Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Cc: Sebastian Andrzej Siewior <sebastian@breakpoint.cc>
---
 arch/x86/kernel/apic/io_apic.c |    9 ++++++---
 drivers/iommu/irq_remapping.c  |    8 ++++++++
 2 files changed, 14 insertions(+), 3 deletions(-)

diff --git a/arch/x86/kernel/apic/io_apic.c b/arch/x86/kernel/apic/io_apic.c
index f8d4d8d..8b03875 100644
--- a/arch/x86/kernel/apic/io_apic.c
+++ b/arch/x86/kernel/apic/io_apic.c
@@ -3089,8 +3089,11 @@ msi_set_affinity(struct irq_data *data, const struct cpumask *mask, bool force)
 
 static void msi_irq_print_chip(struct irq_data *data, struct seq_file *p)
 {
-	seq_printf(p, " %s%s", data->chip->name,
-			data->msi_desc->msi_attrib.is_msix ? "-X" : "");
+	struct pci_dev *dev = data->msi_desc->dev;
+
+	seq_printf(p, " %s%s-edge@%s", data->chip->name,
+			data->msi_desc->msi_attrib.is_msix ? "-X" : "",
+			dev_name(&dev->dev));
 }
 
 /*
@@ -3130,7 +3133,7 @@ int setup_msi_irq(struct pci_dev *dev, struct msi_desc *msidesc,
 
 	setup_remapped_irq(irq, irq_get_chip_data(irq), chip);
 
-	irq_set_chip_and_handler_name(irq, chip, handle_edge_irq, "edge");
+	irq_set_chip_and_handler(irq, chip, handle_edge_irq);
 
 	dev_printk(KERN_DEBUG, &dev->dev, "irq %d for MSI%s\n", irq,
 			 msidesc->msi_attrib.is_msix ? "-X" : "");
diff --git a/drivers/iommu/irq_remapping.c b/drivers/iommu/irq_remapping.c
index e245ab0..ca595f7 100644
--- a/drivers/iommu/irq_remapping.c
+++ b/drivers/iommu/irq_remapping.c
@@ -365,10 +365,18 @@ static void ir_ack_apic_level(struct irq_data *data)
 
 static void ir_print_prefix(struct irq_data *data, struct seq_file *p)
 {
+	struct pci_dev *dev = NULL;
+
 	seq_printf(p, " IR-%s%s", data->chip->name,
 			data->msi_desc ?
 			 (data->msi_desc->msi_attrib.is_msix ? "-X" : "")
 			 : "");
+
+	if (data->msi_desc)
+		dev = data->msi_desc->dev;
+
+	if (dev)
+		seq_printf(p, "-edge@%s", dev_name(&dev->dev));
 }
 
 static void __init irq_remap_modify_chip_defaults(struct irq_chip *chip)
-- 
1.7.10.4

Re: [PATCH v2 05/26] x86, irq: Show pci device name for msi in /proc/interrupt

[Konrad Rzeszutek Wilk]

On Fri, Feb 08, 2013 at 11:28:02AM -0800, Yinghai Lu wrote:
> We could find out which device is using that MSI/MSI-X.
> Use irq_print_chip() to append pci device name.

What would the end result be? For example for:

80:	some values PCI-MSI-edge      xhci_hcd

Is it going to be:

80:	some values PCI-MSI-edge-xhci_hcd	xhci_hcd

?

> 
> Signed-off-by: Yinghai Lu <yinghai@kernel.org>
> Cc: Joerg Roedel <joro@8bytes.org>
> Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
> Cc: Sebastian Andrzej Siewior <sebastian@breakpoint.cc>
> ---
>  arch/x86/kernel/apic/io_apic.c |    9 ++++++---
>  drivers/iommu/irq_remapping.c  |    8 ++++++++
>  2 files changed, 14 insertions(+), 3 deletions(-)
> 
> diff --git a/arch/x86/kernel/apic/io_apic.c b/arch/x86/kernel/apic/io_apic.c
> index f8d4d8d..8b03875 100644
> --- a/arch/x86/kernel/apic/io_apic.c
> +++ b/arch/x86/kernel/apic/io_apic.c
> @@ -3089,8 +3089,11 @@ msi_set_affinity(struct irq_data *data, const struct cpumask *mask, bool force)
>  
>  static void msi_irq_print_chip(struct irq_data *data, struct seq_file *p)
>  {
> -	seq_printf(p, " %s%s", data->chip->name,
> -			data->msi_desc->msi_attrib.is_msix ? "-X" : "");
> +	struct pci_dev *dev = data->msi_desc->dev;
> +
> +	seq_printf(p, " %s%s-edge@%s", data->chip->name,
> +			data->msi_desc->msi_attrib.is_msix ? "-X" : "",
> +			dev_name(&dev->dev));
>  }
>  
>  /*
> @@ -3130,7 +3133,7 @@ int setup_msi_irq(struct pci_dev *dev, struct msi_desc *msidesc,
>  
>  	setup_remapped_irq(irq, irq_get_chip_data(irq), chip);
>  
> -	irq_set_chip_and_handler_name(irq, chip, handle_edge_irq, "edge");
> +	irq_set_chip_and_handler(irq, chip, handle_edge_irq);
>  
>  	dev_printk(KERN_DEBUG, &dev->dev, "irq %d for MSI%s\n", irq,
>  			 msidesc->msi_attrib.is_msix ? "-X" : "");
> diff --git a/drivers/iommu/irq_remapping.c b/drivers/iommu/irq_remapping.c
> index e245ab0..ca595f7 100644
> --- a/drivers/iommu/irq_remapping.c
> +++ b/drivers/iommu/irq_remapping.c
> @@ -365,10 +365,18 @@ static void ir_ack_apic_level(struct irq_data *data)
>  
>  static void ir_print_prefix(struct irq_data *data, struct seq_file *p)
>  {
> +	struct pci_dev *dev = NULL;
> +
>  	seq_printf(p, " IR-%s%s", data->chip->name,
>  			data->msi_desc ?
>  			 (data->msi_desc->msi_attrib.is_msix ? "-X" : "")
>  			 : "");
> +
> +	if (data->msi_desc)
> +		dev = data->msi_desc->dev;
> +
> +	if (dev)
> +		seq_printf(p, "-edge@%s", dev_name(&dev->dev));
>  }
>  
>  static void __init irq_remap_modify_chip_defaults(struct irq_chip *chip)
> -- 
> 1.7.10.4
> 

Re: [PATCH v2 05/26] x86, irq: Show pci device name for msi in /proc/interrupt

[Yinghai Lu]

On Fri, Mar 8, 2013 at 11:46 AM, Konrad Rzeszutek Wilk
<konrad.wilk@oracle.com> wrote:
> On Fri, Feb 08, 2013 at 11:28:02AM -0800, Yinghai Lu wrote:
>> We could find out which device is using that MSI/MSI-X.
>> Use irq_print_chip() to append pci device name.
>
> What would the end result be? For example for:
>
> 80:     some values PCI-MSI-edge      xhci_hcd
>
> Is it going to be:
>
> 80:     some values PCI-MSI-edge-xhci_hcd       xhci_hcd
>
> ?

should be PCI-MSI-edge@0000:01:00.0  xhci_hcd

[PATCH v2 06/26] x86, irq: Make dmar_msi/hpet_msi irq_chip name consistent

[Yinghai Lu]

All others are using "-" instead of "_".

Change dmar_msi and hpet_msi to use "-".

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
Cc: Joerg Roedel <joro@8bytes.org>
Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Cc: Sebastian Andrzej Siewior <sebastian@breakpoint.cc>
---
 arch/x86/kernel/apic/io_apic.c |    4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/arch/x86/kernel/apic/io_apic.c b/arch/x86/kernel/apic/io_apic.c
index 8b03875..c703092 100644
--- a/arch/x86/kernel/apic/io_apic.c
+++ b/arch/x86/kernel/apic/io_apic.c
@@ -3202,7 +3202,7 @@ dmar_msi_set_affinity(struct irq_data *data, const struct cpumask *mask,
 }
 
 static struct irq_chip dmar_msi_type = {
-	.name			= "DMAR_MSI",
+	.name			= "DMAR-MSI",
 	.irq_unmask		= dmar_msi_unmask,
 	.irq_mask		= dmar_msi_mask,
 	.irq_ack		= ack_apic_edge,
@@ -3250,7 +3250,7 @@ static int hpet_msi_set_affinity(struct irq_data *data,
 }
 
 struct irq_chip hpet_msi_type = {
-	.name = "HPET_MSI",
+	.name = "HPET-MSI",
 	.irq_unmask = hpet_msi_unmask,
 	.irq_mask = hpet_msi_mask,
 	.irq_ack = ack_apic_edge,
-- 
1.7.10.4

[PATCH v2 07/26] ia64, irq: Add dummy create_irq_nr()

[Yinghai Lu]

create_irq() will return -1 when fail to allocate.
create_irq_nr() will return 0 when fail to allocate.

Will use it to fix one return vaule checking for dmar_msi irq.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
Cc: Tony Luck <tony.luck@intel.com>
Cc: Fenghua Yu <fenghua.yu@intel.com>
Cc: linux-ia64@vger.kernel.org
---
 arch/ia64/kernel/irq_ia64.c |   10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/arch/ia64/kernel/irq_ia64.c b/arch/ia64/kernel/irq_ia64.c
index 1034884..38e46df 100644
--- a/arch/ia64/kernel/irq_ia64.c
+++ b/arch/ia64/kernel/irq_ia64.c
@@ -429,6 +429,16 @@ int create_irq(void)
 	return irq;
 }
 
+unsigned int create_irq_nr(unsigned int from, int node)
+{
+	int irq = create_irq();
+
+	if (irq < 0)
+		irq = 0;
+
+	return irq;
+}
+
 void destroy_irq(unsigned int irq)
 {
 	dynamic_irq_cleanup(irq);
-- 
1.7.10.4

[PATCH v2 08/26] iommu, irq: Allocate irq_desc for dmar_msi with local node

[Yinghai Lu]

iommu irq's irq_desc should be on local node ram.

Fix the return value checking problem.

create_irq() will return -1 when fail to allocate.
create_irq_nr() will return 0 when fail to allocate.

here only check !irq, so need to change it to use create_irq_nr instead.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
Cc: Joerg Roedel <joro@8bytes.org>
Cc: Donald Dutile <ddutile@redhat.com>
---
 drivers/iommu/dmar.c |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/iommu/dmar.c b/drivers/iommu/dmar.c
index 174bb65..316b5a2 100644
--- a/drivers/iommu/dmar.c
+++ b/drivers/iommu/dmar.c
@@ -1263,7 +1263,7 @@ int dmar_set_interrupt(struct intel_iommu *iommu)
 	if (iommu->irq)
 		return 0;
 
-	irq = create_irq();
+	irq = create_irq_nr(0, iommu->node);
 	if (!irq) {
 		pr_err("IOMMU: no free vectors\n");
 		return -EINVAL;
-- 
1.7.10.4

Re: [PATCH v2 08/26] iommu, irq: Allocate irq_desc for dmar_msi with local node

[Don Dutile]

On 02/08/2013 02:28 PM, Yinghai Lu wrote:
> iommu irq's irq_desc should be on local node ram.
>
> Fix the return value checking problem.
>
> create_irq() will return -1 when fail to allocate.
> create_irq_nr() will return 0 when fail to allocate.
>
> here only check !irq, so need to change it to use create_irq_nr instead.
>
> Signed-off-by: Yinghai Lu<yinghai@kernel.org>
> Cc: Joerg Roedel<joro@8bytes.org>
> Cc: Donald Dutile<ddutile@redhat.com>
> ---
>   drivers/iommu/dmar.c |    2 +-
>   1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/drivers/iommu/dmar.c b/drivers/iommu/dmar.c
> index 174bb65..316b5a2 100644
> --- a/drivers/iommu/dmar.c
> +++ b/drivers/iommu/dmar.c
> @@ -1263,7 +1263,7 @@ int dmar_set_interrupt(struct intel_iommu *iommu)
>   	if (iommu->irq)
>   		return 0;
>
> -	irq = create_irq();
> +	irq = create_irq_nr(0, iommu->node);
>   	if (!irq) {
>   		pr_err("IOMMU: no free vectors\n");
>   		return -EINVAL;
Good catch!

you can add my ack to it.

Re: [PATCH v2 08/26] iommu, irq: Allocate irq_desc for dmar_msi with local node

[Yinghai Lu]

On Fri, Feb 8, 2013 at 12:45 PM, Don Dutile <ddutile@redhat.com> wrote:
> On 02/08/2013 02:28 PM, Yinghai Lu wrote:
>>
>> iommu irq's irq_desc should be on local node ram.
>>
>> Fix the return value checking problem.
>>
>> create_irq() will return -1 when fail to allocate.
>> create_irq_nr() will return 0 when fail to allocate.
>>
>> here only check !irq, so need to change it to use create_irq_nr instead.
>>
>> Signed-off-by: Yinghai Lu<yinghai@kernel.org>
>> Cc: Joerg Roedel<joro@8bytes.org>
>> Cc: Donald Dutile<ddutile@redhat.com>
>> ---
>>   drivers/iommu/dmar.c |    2 +-
>>   1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/drivers/iommu/dmar.c b/drivers/iommu/dmar.c
>> index 174bb65..316b5a2 100644
>> --- a/drivers/iommu/dmar.c
>> +++ b/drivers/iommu/dmar.c
>> @@ -1263,7 +1263,7 @@ int dmar_set_interrupt(struct intel_iommu *iommu)
>>         if (iommu->irq)
>>                 return 0;
>>
>> -       irq = create_irq();
>> +       irq = create_irq_nr(0, iommu->node);
>>         if (!irq) {
>>                 pr_err("IOMMU: no free vectors\n");
>>                 return -EINVAL;
>
> Good catch!
>
> you can add my ack to it.
>

Thanks.

[PATCH v2 09/26] x86, irq: kill create_irq()

[Yinghai Lu]

create_irq() will return -1 when fail to allocate.
create_irq_nr() will return 0 when fail to allocate.

It only causes confusing.

Now we have no user for create_irq(), so remove create_irq() for x86.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
Cc: Joerg Roedel <joro@8bytes.org>
Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Cc: Sebastian Andrzej Siewior <sebastian@breakpoint.cc>
---
 arch/x86/kernel/apic/io_apic.c |   15 ---------------
 1 file changed, 15 deletions(-)

diff --git a/arch/x86/kernel/apic/io_apic.c b/arch/x86/kernel/apic/io_apic.c
index c703092..4c1a726 100644
--- a/arch/x86/kernel/apic/io_apic.c
+++ b/arch/x86/kernel/apic/io_apic.c
@@ -2968,21 +2968,6 @@ unsigned int create_irq_nr(unsigned int from, int node)
 	return __create_irqs(from, 1, node);
 }
 
-int create_irq(void)
-{
-	int node = cpu_to_node(0);
-	unsigned int irq_want;
-	int irq;
-
-	irq_want = nr_irqs_gsi;
-	irq = create_irq_nr(irq_want, node);
-
-	if (irq == 0)
-		irq = -1;
-
-	return irq;
-}
-
 void destroy_irq(unsigned int irq)
 {
 	struct irq_cfg *cfg = irq_get_chip_data(irq);
-- 
1.7.10.4

Re: [PATCH v2 09/26] x86, irq: kill create_irq()

[Konrad Rzeszutek Wilk]

On Fri, Feb 08, 2013 at 11:28:06AM -0800, Yinghai Lu wrote:
> create_irq() will return -1 when fail to allocate.

.. failing to allocate
> create_irq_nr() will return 0 when fail to allocate.

.. failing..
> 
> It only causes confusing.

confusion.

> 
> Now we have no user for create_irq(), so remove create_irq() for x86.

> 
> Signed-off-by: Yinghai Lu <yinghai@kernel.org>
> Cc: Joerg Roedel <joro@8bytes.org>
> Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
> Cc: Sebastian Andrzej Siewior <sebastian@breakpoint.cc>
> ---
>  arch/x86/kernel/apic/io_apic.c |   15 ---------------
>  1 file changed, 15 deletions(-)
> 
> diff --git a/arch/x86/kernel/apic/io_apic.c b/arch/x86/kernel/apic/io_apic.c
> index c703092..4c1a726 100644
> --- a/arch/x86/kernel/apic/io_apic.c
> +++ b/arch/x86/kernel/apic/io_apic.c
> @@ -2968,21 +2968,6 @@ unsigned int create_irq_nr(unsigned int from, int node)
>  	return __create_irqs(from, 1, node);
>  }
>  
> -int create_irq(void)
> -{
> -	int node = cpu_to_node(0);
> -	unsigned int irq_want;
> -	int irq;
> -
> -	irq_want = nr_irqs_gsi;
> -	irq = create_irq_nr(irq_want, node);
> -
> -	if (irq == 0)
> -		irq = -1;
> -
> -	return irq;
> -}
> -
>  void destroy_irq(unsigned int irq)
>  {
>  	struct irq_cfg *cfg = irq_get_chip_data(irq);
> -- 
> 1.7.10.4
> 

[PATCH v2 10/26] x86, irq: Convert irq_2_pin list to generic list

[Yinghai Lu]

Now irq_2_pin list is own grown list.
We can use generic list to replace it so we could generic helper
functions to operate it.

Also make free_irq_cfg() free irq_2_pin list to support coming ioapic
hotplug.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Cc: Sebastian Andrzej Siewior <sebastian@breakpoint.cc>
Cc: Joerg Roedel <joro@8bytes.org>
---
 arch/x86/include/asm/hw_irq.h  |    2 +-
 arch/x86/kernel/apic/io_apic.c |   22 ++++++++++++----------
 2 files changed, 13 insertions(+), 11 deletions(-)

diff --git a/arch/x86/include/asm/hw_irq.h b/arch/x86/include/asm/hw_irq.h
index 10a78c3..69743f8 100644
--- a/arch/x86/include/asm/hw_irq.h
+++ b/arch/x86/include/asm/hw_irq.h
@@ -121,7 +121,7 @@ struct irq_2_irte {
  * Most irqs are mapped 1:1 with pins.
  */
 struct irq_cfg {
-	struct irq_pin_list	*irq_2_pin;
+	struct list_head	irq_2_pin;
 	cpumask_var_t		domain;
 	cpumask_var_t		old_domain;
 	u8			vector;
diff --git a/arch/x86/kernel/apic/io_apic.c b/arch/x86/kernel/apic/io_apic.c
index 4c1a726..d2b4988 100644
--- a/arch/x86/kernel/apic/io_apic.c
+++ b/arch/x86/kernel/apic/io_apic.c
@@ -66,7 +66,7 @@
 #define __apicdebuginit(type) static type __init
 
 #define for_each_irq_pin(entry, head) \
-	for (entry = head; entry; entry = entry->next)
+	list_for_each_entry(entry, &head, list)
 
 /*
  *      Is the SiS APIC rmw bug present ?
@@ -176,8 +176,8 @@ void mp_save_irq(struct mpc_intsrc *m)
 }
 
 struct irq_pin_list {
+	struct list_head list;
 	int apic, pin;
-	struct irq_pin_list *next;
 };
 
 static struct irq_pin_list *alloc_irq_pin_list(int node)
@@ -213,6 +213,7 @@ int __init arch_early_irq_init(void)
 	irq_reserve_irqs(0, legacy_pic->nr_legacy_irqs);
 
 	for (i = 0; i < count; i++) {
+		INIT_LIST_HEAD(&cfg[i].irq_2_pin);
 		irq_set_chip_data(i, &cfg[i]);
 		zalloc_cpumask_var_node(&cfg[i].domain, GFP_KERNEL, node);
 		zalloc_cpumask_var_node(&cfg[i].old_domain, GFP_KERNEL, node);
@@ -245,6 +246,7 @@ static struct irq_cfg *alloc_irq_cfg(unsigned int irq, int node)
 		goto out_cfg;
 	if (!zalloc_cpumask_var_node(&cfg->old_domain, GFP_KERNEL, node))
 		goto out_domain;
+	INIT_LIST_HEAD(&cfg->irq_2_pin);
 	return cfg;
 out_domain:
 	free_cpumask_var(cfg->domain);
@@ -255,11 +257,15 @@ out_cfg:
 
 static void free_irq_cfg(unsigned int at, struct irq_cfg *cfg)
 {
+	struct irq_pin_list *entry, *tmp;
+
 	if (!cfg)
 		return;
 	irq_set_chip_data(at, NULL);
 	free_cpumask_var(cfg->domain);
 	free_cpumask_var(cfg->old_domain);
+	list_for_each_entry_safe(entry, tmp, &cfg->irq_2_pin, list)
+		kfree(entry);
 	kfree(cfg);
 }
 
@@ -420,15 +426,12 @@ static void ioapic_mask_entry(int apic, int pin)
  */
 static int __add_pin_to_irq_node(struct irq_cfg *cfg, int node, int apic, int pin)
 {
-	struct irq_pin_list **last, *entry;
+	struct irq_pin_list *entry;
 
 	/* don't allow duplicates */
-	last = &cfg->irq_2_pin;
-	for_each_irq_pin(entry, cfg->irq_2_pin) {
+	for_each_irq_pin(entry, cfg->irq_2_pin)
 		if (entry->apic == apic && entry->pin == pin)
 			return 0;
-		last = &entry->next;
-	}
 
 	entry = alloc_irq_pin_list(node);
 	if (!entry) {
@@ -439,7 +442,7 @@ static int __add_pin_to_irq_node(struct irq_cfg *cfg, int node, int apic, int pi
 	entry->apic = apic;
 	entry->pin = pin;
 
-	*last = entry;
+	list_add_tail(&entry->list, &cfg->irq_2_pin);
 	return 0;
 }
 
@@ -1622,8 +1625,7 @@ __apicdebuginit(void) print_IO_APICs(void)
 		cfg = irq_get_chip_data(irq);
 		if (!cfg)
 			continue;
-		entry = cfg->irq_2_pin;
-		if (!entry)
+		if (list_empty(&cfg->irq_2_pin))
 			continue;
 		printk(KERN_DEBUG "IRQ%d ", irq);
 		for_each_irq_pin(entry, cfg->irq_2_pin)
-- 
1.7.10.4

Re: [PATCH v2 10/26] x86, irq: Convert irq_2_pin list to generic list

[Konrad Rzeszutek Wilk]

On Fri, Feb 08, 2013 at 11:28:07AM -0800, Yinghai Lu wrote:
> Now irq_2_pin list is own grown list.

I think you meant to say 'has its own list'. Not sure how
its growing?

> We can use generic list to replace it so we could generic helper
> functions to operate it.

You need to put the verb a bit earlier. Perhaps:

".. generic list to replace so that we can operate using
generic helper functions."

> 
> Also make free_irq_cfg() free irq_2_pin list to support coming ioapic
> hotplug.

If you have them ready - could you say the titles of the patches here?

> 
> Signed-off-by: Yinghai Lu <yinghai@kernel.org>
> Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
> Cc: Sebastian Andrzej Siewior <sebastian@breakpoint.cc>
> Cc: Joerg Roedel <joro@8bytes.org>
> ---
>  arch/x86/include/asm/hw_irq.h  |    2 +-
>  arch/x86/kernel/apic/io_apic.c |   22 ++++++++++++----------
>  2 files changed, 13 insertions(+), 11 deletions(-)
> 
> diff --git a/arch/x86/include/asm/hw_irq.h b/arch/x86/include/asm/hw_irq.h
> index 10a78c3..69743f8 100644
> --- a/arch/x86/include/asm/hw_irq.h
> +++ b/arch/x86/include/asm/hw_irq.h
> @@ -121,7 +121,7 @@ struct irq_2_irte {
>   * Most irqs are mapped 1:1 with pins.
>   */
>  struct irq_cfg {
> -	struct irq_pin_list	*irq_2_pin;
> +	struct list_head	irq_2_pin;
>  	cpumask_var_t		domain;
>  	cpumask_var_t		old_domain;
>  	u8			vector;
> diff --git a/arch/x86/kernel/apic/io_apic.c b/arch/x86/kernel/apic/io_apic.c
> index 4c1a726..d2b4988 100644
> --- a/arch/x86/kernel/apic/io_apic.c
> +++ b/arch/x86/kernel/apic/io_apic.c
> @@ -66,7 +66,7 @@
>  #define __apicdebuginit(type) static type __init
>  
>  #define for_each_irq_pin(entry, head) \
> -	for (entry = head; entry; entry = entry->next)
> +	list_for_each_entry(entry, &head, list)
>  
>  /*
>   *      Is the SiS APIC rmw bug present ?
> @@ -176,8 +176,8 @@ void mp_save_irq(struct mpc_intsrc *m)
>  }
>  
>  struct irq_pin_list {
> +	struct list_head list;
>  	int apic, pin;
> -	struct irq_pin_list *next;
>  };
>  
>  static struct irq_pin_list *alloc_irq_pin_list(int node)
> @@ -213,6 +213,7 @@ int __init arch_early_irq_init(void)
>  	irq_reserve_irqs(0, legacy_pic->nr_legacy_irqs);
>  
>  	for (i = 0; i < count; i++) {
> +		INIT_LIST_HEAD(&cfg[i].irq_2_pin);
>  		irq_set_chip_data(i, &cfg[i]);
>  		zalloc_cpumask_var_node(&cfg[i].domain, GFP_KERNEL, node);
>  		zalloc_cpumask_var_node(&cfg[i].old_domain, GFP_KERNEL, node);
> @@ -245,6 +246,7 @@ static struct irq_cfg *alloc_irq_cfg(unsigned int irq, int node)
>  		goto out_cfg;
>  	if (!zalloc_cpumask_var_node(&cfg->old_domain, GFP_KERNEL, node))
>  		goto out_domain;
> +	INIT_LIST_HEAD(&cfg->irq_2_pin);
>  	return cfg;
>  out_domain:
>  	free_cpumask_var(cfg->domain);
> @@ -255,11 +257,15 @@ out_cfg:
>  
>  static void free_irq_cfg(unsigned int at, struct irq_cfg *cfg)
>  {
> +	struct irq_pin_list *entry, *tmp;
> +
>  	if (!cfg)
>  		return;
>  	irq_set_chip_data(at, NULL);
>  	free_cpumask_var(cfg->domain);
>  	free_cpumask_var(cfg->old_domain);
> +	list_for_each_entry_safe(entry, tmp, &cfg->irq_2_pin, list)
> +		kfree(entry);
>  	kfree(cfg);
>  }
>  
> @@ -420,15 +426,12 @@ static void ioapic_mask_entry(int apic, int pin)
>   */
>  static int __add_pin_to_irq_node(struct irq_cfg *cfg, int node, int apic, int pin)
>  {
> -	struct irq_pin_list **last, *entry;
> +	struct irq_pin_list *entry;
>  
>  	/* don't allow duplicates */
> -	last = &cfg->irq_2_pin;
> -	for_each_irq_pin(entry, cfg->irq_2_pin) {
> +	for_each_irq_pin(entry, cfg->irq_2_pin)
>  		if (entry->apic == apic && entry->pin == pin)
>  			return 0;
> -		last = &entry->next;
> -	}
>  
>  	entry = alloc_irq_pin_list(node);
>  	if (!entry) {
> @@ -439,7 +442,7 @@ static int __add_pin_to_irq_node(struct irq_cfg *cfg, int node, int apic, int pi
>  	entry->apic = apic;
>  	entry->pin = pin;
>  
> -	*last = entry;
> +	list_add_tail(&entry->list, &cfg->irq_2_pin);
>  	return 0;
>  }
>  
> @@ -1622,8 +1625,7 @@ __apicdebuginit(void) print_IO_APICs(void)
>  		cfg = irq_get_chip_data(irq);
>  		if (!cfg)
>  			continue;
> -		entry = cfg->irq_2_pin;
> -		if (!entry)
> +		if (list_empty(&cfg->irq_2_pin))
>  			continue;
>  		printk(KERN_DEBUG "IRQ%d ", irq);
>  		for_each_irq_pin(entry, cfg->irq_2_pin)
> -- 
> 1.7.10.4
> 

Re: [PATCH v2 10/26] x86, irq: Convert irq_2_pin list to generic list

[Yinghai Lu]

On Fri, Mar 8, 2013 at 11:50 AM, Konrad Rzeszutek Wilk
<konrad.wilk@oracle.com> wrote:
> On Fri, Feb 08, 2013 at 11:28:07AM -0800, Yinghai Lu wrote:
>> Now irq_2_pin list is own grown list.
>
> I think you meant to say 'has its own list'. Not sure how
> its growing?
>
>> We can use generic list to replace it so we could generic helper
>> functions to operate it.
>
> You need to put the verb a bit earlier. Perhaps:
>
> ".. generic list to replace so that we can operate using
> generic helper functions."

missed use before generic helper.

>
>>
>> Also make free_irq_cfg() free irq_2_pin list to support coming ioapic
>> hotplug.
>
> If you have them ready - could you say the titles of the patches here?

list them all ?

Re: [PATCH v2 10/26] x86, irq: Convert irq_2_pin list to generic list

[Konrad Rzeszutek Wilk]

On Fri, Mar 08, 2013 at 12:00:43PM -0800, Yinghai Lu wrote:
> On Fri, Mar 8, 2013 at 11:50 AM, Konrad Rzeszutek Wilk
> <konrad.wilk@oracle.com> wrote:
> > On Fri, Feb 08, 2013 at 11:28:07AM -0800, Yinghai Lu wrote:
> >> Now irq_2_pin list is own grown list.
> >
> > I think you meant to say 'has its own list'. Not sure how
> > its growing?
> >
> >> We can use generic list to replace it so we could generic helper
> >> functions to operate it.
> >
> > You need to put the verb a bit earlier. Perhaps:
> >
> > ".. generic list to replace so that we can operate using
> > generic helper functions."
> 
> missed use before generic helper.

That would work as well.

> 
> >
> >>
> >> Also make free_irq_cfg() free irq_2_pin list to support coming ioapic
> >> hotplug.
> >
> > If you have them ready - could you say the titles of the patches here?
> 
> list them all ?

No that would not work. Perhaps the first one that introduces the code
that uses irq_2_pin ?

[PATCH v2 11/26] genirq: Split __irq_reserve_irqs from irq_alloc_descs

[Yinghai Lu]

irq_alloc_descs and irq_reserve_irqs are almost the same.
Separate code out to __irq_reserved_irqs, and other two reuse
__irq_reserve_irqs.

And will use __irq_reserve_irqs for coming ioapic hotplug support.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
---
 include/linux/irq.h  |    1 +
 kernel/irq/irqdesc.c |   54 +++++++++++++++++++++++++++++---------------------
 2 files changed, 32 insertions(+), 23 deletions(-)

diff --git a/include/linux/irq.h b/include/linux/irq.h
index bc4e066..4e0fcbb 100644
--- a/include/linux/irq.h
+++ b/include/linux/irq.h
@@ -600,6 +600,7 @@ int __irq_alloc_descs(int irq, unsigned int from, unsigned int cnt, int node,
 
 void irq_free_descs(unsigned int irq, unsigned int cnt);
 int irq_reserve_irqs(unsigned int from, unsigned int cnt);
+int __irq_reserve_irqs(int irq, unsigned int from, unsigned int cnt);
 
 static inline void irq_free_desc(unsigned int irq)
 {
diff --git a/kernel/irq/irqdesc.c b/kernel/irq/irqdesc.c
index 192a302..3b9fb92 100644
--- a/kernel/irq/irqdesc.c
+++ b/kernel/irq/irqdesc.c
@@ -340,18 +340,15 @@ void irq_free_descs(unsigned int from, unsigned int cnt)
 EXPORT_SYMBOL_GPL(irq_free_descs);
 
 /**
- * irq_alloc_descs - allocate and initialize a range of irq descriptors
- * @irq:	Allocate for specific irq number if irq >= 0
+ * __irq_reserve_descs - reserve and initialize a range of irq descriptors
+ * @irq:	Reserve for specific irq number if irq >= 0
  * @from:	Start the search from this irq number
- * @cnt:	Number of consecutive irqs to allocate.
- * @node:	Preferred node on which the irq descriptor should be allocated
- * @owner:	Owning module (can be NULL)
+ * @cnt:	Number of consecutive irqs to reserve.
  *
  * Returns the first irq number or error code
  */
 int __ref
-__irq_alloc_descs(int irq, unsigned int from, unsigned int cnt, int node,
-		  struct module *owner)
+__irq_reserve_irqs(int irq, unsigned int from, unsigned int cnt)
 {
 	int start, ret;
 
@@ -369,7 +366,7 @@ __irq_alloc_descs(int irq, unsigned int from, unsigned int cnt, int node,
 	start = bitmap_find_next_zero_area(allocated_irqs, IRQ_BITMAP_BITS,
 					   from, cnt, 0);
 	ret = -EEXIST;
-	if (irq >=0 && start != irq)
+	if (irq >= 0 && start != irq)
 		goto err;
 
 	if (start + cnt > nr_irqs) {
@@ -380,12 +377,36 @@ __irq_alloc_descs(int irq, unsigned int from, unsigned int cnt, int node,
 
 	bitmap_set(allocated_irqs, start, cnt);
 	mutex_unlock(&sparse_irq_lock);
-	return alloc_descs(start, cnt, node, owner);
+	return start;
 
 err:
 	mutex_unlock(&sparse_irq_lock);
 	return ret;
 }
+
+/**
+ * irq_alloc_descs - allocate and initialize a range of irq descriptors
+ * @irq:	Allocate for specific irq number if irq >= 0
+ * @from:	Start the search from this irq number
+ * @cnt:	Number of consecutive irqs to allocate.
+ * @node:	Preferred node on which the irq descriptor should be allocated
+ * @owner:	Owning module (can be NULL)
+ *
+ * Returns the first irq number or error code
+ */
+int __ref
+__irq_alloc_descs(int irq, unsigned int from, unsigned int cnt, int node,
+		  struct module *owner)
+{
+	int start;
+
+	start = __irq_reserve_irqs(irq, from, cnt);
+
+	if (start < 0)
+		return start;
+
+	return alloc_descs(start, cnt, node, owner);
+}
 EXPORT_SYMBOL_GPL(__irq_alloc_descs);
 
 /**
@@ -397,20 +418,7 @@ EXPORT_SYMBOL_GPL(__irq_alloc_descs);
  */
 int irq_reserve_irqs(unsigned int from, unsigned int cnt)
 {
-	unsigned int start;
-	int ret = 0;
-
-	if (!cnt || (from + cnt) > nr_irqs)
-		return -EINVAL;
-
-	mutex_lock(&sparse_irq_lock);
-	start = bitmap_find_next_zero_area(allocated_irqs, nr_irqs, from, cnt, 0);
-	if (start == from)
-		bitmap_set(allocated_irqs, start, cnt);
-	else
-		ret = -EEXIST;
-	mutex_unlock(&sparse_irq_lock);
-	return ret;
+	return __irq_reserve_irqs(from, from, cnt);
 }
 
 /**
-- 
1.7.10.4

[PATCH v2 12/26] x86, irq: Add realloc_irq_and_cfg_at()

[Yinghai Lu]

For ioapic hotplug support, we need to reserve irq range for hot
added ioapic controller. After that we need to allocate those
pre-reserved one.

Add realloc_irq_and_cfg_at() to really allocate irq_desc and cfg
as pre-reserved only hold bit in allocate_irqs bit maps.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
Cc: Joerg Roedel <joro@8bytes.org>
Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Cc: Sebastian Andrzej Siewior <sebastian@breakpoint.cc>
---
 arch/x86/kernel/apic/io_apic.c |   32 +++++++++++++++++++++++++++++++-
 include/linux/irq.h            |    5 +++++
 kernel/irq/irqdesc.c           |   26 ++++++++++++++++++++++++++
 3 files changed, 62 insertions(+), 1 deletion(-)

diff --git a/arch/x86/kernel/apic/io_apic.c b/arch/x86/kernel/apic/io_apic.c
index d2b4988..72a1351 100644
--- a/arch/x86/kernel/apic/io_apic.c
+++ b/arch/x86/kernel/apic/io_apic.c
@@ -301,6 +301,36 @@ static void free_irq_at(unsigned int at, struct irq_cfg *cfg)
 	irq_free_desc(at);
 }
 
+static struct irq_cfg *realloc_irq_and_cfg_at(unsigned int at, int node)
+{
+	struct irq_desc *desc = irq_to_desc(at);
+	struct irq_cfg *cfg;
+	int res;
+
+	if (desc) {
+		if (irq_desc_get_irq_data(desc)->node == node)
+			return alloc_irq_and_cfg_at(at, node);
+
+		cfg = irq_desc_get_chip_data(desc);
+		if (cfg) {
+			/* shared irq */
+			if (!list_empty(&cfg->irq_2_pin))
+				return cfg;
+			free_irq_cfg(at, cfg);
+		}
+	}
+
+	res = irq_realloc_desc_at(at, node);
+	if (res >= 0) {
+		cfg = alloc_irq_cfg(at, node);
+		if (cfg) {
+			irq_set_chip_data(at, cfg);
+			return cfg;
+		}
+	}
+
+	return alloc_irq_and_cfg_at(at, node);
+}
 
 struct io_apic {
 	unsigned int index;
@@ -3355,7 +3385,7 @@ int arch_setup_ht_irq(unsigned int irq, struct pci_dev *dev)
 static int
 io_apic_setup_irq_pin(unsigned int irq, int node, struct io_apic_irq_attr *attr)
 {
-	struct irq_cfg *cfg = alloc_irq_and_cfg_at(irq, node);
+	struct irq_cfg *cfg = realloc_irq_and_cfg_at(irq, node);
 	int ret;
 
 	if (!cfg)
diff --git a/include/linux/irq.h b/include/linux/irq.h
index 4e0fcbb..9c6c047 100644
--- a/include/linux/irq.h
+++ b/include/linux/irq.h
@@ -602,6 +602,11 @@ void irq_free_descs(unsigned int irq, unsigned int cnt);
 int irq_reserve_irqs(unsigned int from, unsigned int cnt);
 int __irq_reserve_irqs(int irq, unsigned int from, unsigned int cnt);
 
+int __irq_realloc_desc(int at, int node, struct module *owner);
+/* use macros to avoid needing export.h for THIS_MODULE */
+#define irq_realloc_desc_at(at, node)	\
+	__irq_realloc_desc(at, node, THIS_MODULE)
+
 static inline void irq_free_desc(unsigned int irq)
 {
 	irq_free_descs(irq, 1);
diff --git a/kernel/irq/irqdesc.c b/kernel/irq/irqdesc.c
index 3b9fb92..b48f65b 100644
--- a/kernel/irq/irqdesc.c
+++ b/kernel/irq/irqdesc.c
@@ -99,6 +99,11 @@ EXPORT_SYMBOL_GPL(nr_irqs);
 static DEFINE_MUTEX(sparse_irq_lock);
 static DECLARE_BITMAP(allocated_irqs, IRQ_BITMAP_BITS);
 
+static bool __irq_is_reserved(int irq)
+{
+	return !!test_bit(irq, allocated_irqs);
+}
+
 #ifdef CONFIG_SPARSE_IRQ
 
 static RADIX_TREE(irq_desc_tree, GFP_KERNEL);
@@ -410,6 +415,27 @@ __irq_alloc_descs(int irq, unsigned int from, unsigned int cnt, int node,
 EXPORT_SYMBOL_GPL(__irq_alloc_descs);
 
 /**
+ * irq_realloc_desc - allocate irq descriptor for irq that is already reserved
+ * @irq:	Allocate for specific irq number if irq >= 0
+ * @node:	Preferred node on which the irq descriptor should be allocated
+ * @owner:	Owning module (can be NULL)
+ *
+ * Returns the irq number or error code
+ */
+int __ref
+__irq_realloc_desc(int irq, int node, struct module *owner)
+{
+	if (!__irq_is_reserved(irq))
+		return -EINVAL;
+
+	if (irq_to_desc(irq))
+		free_desc(irq);
+
+	return alloc_descs(irq, 1, node, owner);
+}
+EXPORT_SYMBOL_GPL(__irq_realloc_desc);
+
+/**
  * irq_reserve_irqs - mark irqs allocated
  * @from:	mark from irq number
  * @cnt:	number of irqs to mark
-- 
1.7.10.4

Re: [PATCH v2 12/26] x86, irq: Add realloc_irq_and_cfg_at()

[Konrad Rzeszutek Wilk]

On Fri, Feb 08, 2013 at 11:28:09AM -0800, Yinghai Lu wrote:
> For ioapic hotplug support, we need to reserve irq range for hot
> added ioapic controller. After that we need to allocate those
> pre-reserved one.
> 
> Add realloc_irq_and_cfg_at() to really allocate irq_desc and cfg

Put a period at 'cfg'.
> as pre-reserved only hold bit in allocate_irqs bit maps.

And then start the new sentence here saying 'Pre-reserved (XX function
name) only holds bits in .."

> 
> Signed-off-by: Yinghai Lu <yinghai@kernel.org>
> Cc: Joerg Roedel <joro@8bytes.org>
> Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
> Cc: Sebastian Andrzej Siewior <sebastian@breakpoint.cc>
> ---
>  arch/x86/kernel/apic/io_apic.c |   32 +++++++++++++++++++++++++++++++-
>  include/linux/irq.h            |    5 +++++
>  kernel/irq/irqdesc.c           |   26 ++++++++++++++++++++++++++
>  3 files changed, 62 insertions(+), 1 deletion(-)
> 
> diff --git a/arch/x86/kernel/apic/io_apic.c b/arch/x86/kernel/apic/io_apic.c
> index d2b4988..72a1351 100644
> --- a/arch/x86/kernel/apic/io_apic.c
> +++ b/arch/x86/kernel/apic/io_apic.c
> @@ -301,6 +301,36 @@ static void free_irq_at(unsigned int at, struct irq_cfg *cfg)
>  	irq_free_desc(at);
>  }
>  
> +static struct irq_cfg *realloc_irq_and_cfg_at(unsigned int at, int node)
> +{
> +	struct irq_desc *desc = irq_to_desc(at);
> +	struct irq_cfg *cfg;
> +	int res;
> +
> +	if (desc) {
> +		if (irq_desc_get_irq_data(desc)->node == node)
> +			return alloc_irq_and_cfg_at(at, node);
> +
> +		cfg = irq_desc_get_chip_data(desc);
> +		if (cfg) {
> +			/* shared irq */
> +			if (!list_empty(&cfg->irq_2_pin))
> +				return cfg;
> +			free_irq_cfg(at, cfg);
> +		}
> +	}
> +
> +	res = irq_realloc_desc_at(at, node);
> +	if (res >= 0) {
> +		cfg = alloc_irq_cfg(at, node);
> +		if (cfg) {
> +			irq_set_chip_data(at, cfg);
> +			return cfg;
> +		}
> +	}
> +
> +	return alloc_irq_and_cfg_at(at, node);
> +}
>  
>  struct io_apic {
>  	unsigned int index;
> @@ -3355,7 +3385,7 @@ int arch_setup_ht_irq(unsigned int irq, struct pci_dev *dev)
>  static int
>  io_apic_setup_irq_pin(unsigned int irq, int node, struct io_apic_irq_attr *attr)
>  {
> -	struct irq_cfg *cfg = alloc_irq_and_cfg_at(irq, node);
> +	struct irq_cfg *cfg = realloc_irq_and_cfg_at(irq, node);
>  	int ret;
>  
>  	if (!cfg)
> diff --git a/include/linux/irq.h b/include/linux/irq.h
> index 4e0fcbb..9c6c047 100644
> --- a/include/linux/irq.h
> +++ b/include/linux/irq.h
> @@ -602,6 +602,11 @@ void irq_free_descs(unsigned int irq, unsigned int cnt);
>  int irq_reserve_irqs(unsigned int from, unsigned int cnt);
>  int __irq_reserve_irqs(int irq, unsigned int from, unsigned int cnt);
>  
> +int __irq_realloc_desc(int at, int node, struct module *owner);
> +/* use macros to avoid needing export.h for THIS_MODULE */
> +#define irq_realloc_desc_at(at, node)	\
> +	__irq_realloc_desc(at, node, THIS_MODULE)
> +
>  static inline void irq_free_desc(unsigned int irq)
>  {
>  	irq_free_descs(irq, 1);
> diff --git a/kernel/irq/irqdesc.c b/kernel/irq/irqdesc.c
> index 3b9fb92..b48f65b 100644
> --- a/kernel/irq/irqdesc.c
> +++ b/kernel/irq/irqdesc.c
> @@ -99,6 +99,11 @@ EXPORT_SYMBOL_GPL(nr_irqs);
>  static DEFINE_MUTEX(sparse_irq_lock);
>  static DECLARE_BITMAP(allocated_irqs, IRQ_BITMAP_BITS);
>  
> +static bool __irq_is_reserved(int irq)
> +{
> +	return !!test_bit(irq, allocated_irqs);
> +}
> +
>  #ifdef CONFIG_SPARSE_IRQ
>  
>  static RADIX_TREE(irq_desc_tree, GFP_KERNEL);
> @@ -410,6 +415,27 @@ __irq_alloc_descs(int irq, unsigned int from, unsigned int cnt, int node,
>  EXPORT_SYMBOL_GPL(__irq_alloc_descs);
>  
>  /**
> + * irq_realloc_desc - allocate irq descriptor for irq that is already reserved

Which begs the question - why was it not allocated when it was reserved?

> + * @irq:	Allocate for specific irq number if irq >= 0
> + * @node:	Preferred node on which the irq descriptor should be allocated
> + * @owner:	Owning module (can be NULL)
> + *
> + * Returns the irq number or error code
> + */
> +int __ref
> +__irq_realloc_desc(int irq, int node, struct module *owner)
> +{
> +	if (!__irq_is_reserved(irq))
> +		return -EINVAL;
> +
> +	if (irq_to_desc(irq))
> +		free_desc(irq);
> +
> +	return alloc_descs(irq, 1, node, owner);
> +}
> +EXPORT_SYMBOL_GPL(__irq_realloc_desc);
> +
> +/**
>   * irq_reserve_irqs - mark irqs allocated
>   * @from:	mark from irq number
>   * @cnt:	number of irqs to mark
> -- 
> 1.7.10.4
> 

Re: [PATCH v2 12/26] x86, irq: Add realloc_irq_and_cfg_at()

[Yinghai Lu]

On Fri, Mar 8, 2013 at 11:53 AM, Konrad Rzeszutek Wilk
<konrad.wilk@oracle.com> wrote:
>> + * irq_realloc_desc - allocate irq descriptor for irq that is already reserved
>
> Which begs the question - why was it not allocated when it was reserved?

The reasons for not allocating them during reserving:
1. only several pins in ioapic are used, allocate for them pin, will
   waste memory for not used one.
2. relocate later could make sure irq_desc is allocated on local node ram.

Thanks

Yinghai

[PATCH v2 13/26] x86, irq: Move down arch_early_irq_init()

[Yinghai Lu]

Change position only.

Prepare to update arch_early_irq_init(), it needs call some static functions.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
Cc: Joerg Roedel <joro@8bytes.org>
Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Cc: Sebastian Andrzej Siewior <sebastian@breakpoint.cc>
---
 arch/x86/kernel/apic/io_apic.c |   89 ++++++++++++++++++++--------------------
 1 file changed, 44 insertions(+), 45 deletions(-)

diff --git a/arch/x86/kernel/apic/io_apic.c b/arch/x86/kernel/apic/io_apic.c
index 72a1351..deda981 100644
--- a/arch/x86/kernel/apic/io_apic.c
+++ b/arch/x86/kernel/apic/io_apic.c
@@ -185,51 +185,6 @@ static struct irq_pin_list *alloc_irq_pin_list(int node)
 	return kzalloc_node(sizeof(struct irq_pin_list), GFP_KERNEL, node);
 }
 
-
-/* irq_cfg is indexed by the sum of all RTEs in all I/O APICs. */
-static struct irq_cfg irq_cfgx[NR_IRQS_LEGACY];
-
-int __init arch_early_irq_init(void)
-{
-	struct irq_cfg *cfg;
-	int count, node, i;
-
-	if (!legacy_pic->nr_legacy_irqs)
-		io_apic_irqs = ~0UL;
-
-	for (i = 0; i < nr_ioapics; i++) {
-		ioapics[i].saved_registers =
-			kzalloc(sizeof(struct IO_APIC_route_entry) *
-				ioapics[i].nr_registers, GFP_KERNEL);
-		if (!ioapics[i].saved_registers)
-			pr_err("IOAPIC %d: suspend/resume impossible!\n", i);
-	}
-
-	cfg = irq_cfgx;
-	count = ARRAY_SIZE(irq_cfgx);
-	node = cpu_to_node(0);
-
-	/* Make sure the legacy interrupts are marked in the bitmap */
-	irq_reserve_irqs(0, legacy_pic->nr_legacy_irqs);
-
-	for (i = 0; i < count; i++) {
-		INIT_LIST_HEAD(&cfg[i].irq_2_pin);
-		irq_set_chip_data(i, &cfg[i]);
-		zalloc_cpumask_var_node(&cfg[i].domain, GFP_KERNEL, node);
-		zalloc_cpumask_var_node(&cfg[i].old_domain, GFP_KERNEL, node);
-		/*
-		 * For legacy IRQ's, start with assigning irq0 to irq15 to
-		 * IRQ0_VECTOR to IRQ15_VECTOR for all cpu's.
-		 */
-		if (i < legacy_pic->nr_legacy_irqs) {
-			cfg[i].vector = IRQ0_VECTOR + i;
-			cpumask_setall(cfg[i].domain);
-		}
-	}
-
-	return 0;
-}
-
 static struct irq_cfg *irq_cfg(unsigned int irq)
 {
 	return irq_get_chip_data(irq);
@@ -332,6 +287,50 @@ static struct irq_cfg *realloc_irq_and_cfg_at(unsigned int at, int node)
 	return alloc_irq_and_cfg_at(at, node);
 }
 
+/* irq_cfg is indexed by the sum of all RTEs in all I/O APICs. */
+static struct irq_cfg irq_cfgx[NR_IRQS_LEGACY];
+
+int __init arch_early_irq_init(void)
+{
+	struct irq_cfg *cfg;
+	int count, node, i;
+
+	if (!legacy_pic->nr_legacy_irqs)
+		io_apic_irqs = ~0UL;
+
+	for (i = 0; i < nr_ioapics; i++) {
+		ioapics[i].saved_registers =
+			kzalloc(sizeof(struct IO_APIC_route_entry) *
+				ioapics[i].nr_registers, GFP_KERNEL);
+		if (!ioapics[i].saved_registers)
+			pr_err("IOAPIC %d: suspend/resume impossible!\n", i);
+	}
+
+	cfg = irq_cfgx;
+	count = ARRAY_SIZE(irq_cfgx);
+	node = cpu_to_node(0);
+
+	/* Make sure the legacy interrupts are marked in the bitmap */
+	irq_reserve_irqs(0, legacy_pic->nr_legacy_irqs);
+
+	for (i = 0; i < count; i++) {
+		INIT_LIST_HEAD(&cfg[i].irq_2_pin);
+		irq_set_chip_data(i, &cfg[i]);
+		zalloc_cpumask_var_node(&cfg[i].domain, GFP_KERNEL, node);
+		zalloc_cpumask_var_node(&cfg[i].old_domain, GFP_KERNEL, node);
+		/*
+		 * For legacy IRQ's, start with assigning irq0 to irq15 to
+		 * IRQ0_VECTOR to IRQ15_VECTOR for all cpu's.
+		 */
+		if (i < legacy_pic->nr_legacy_irqs) {
+			cfg[i].vector = IRQ0_VECTOR + i;
+			cpumask_setall(cfg[i].domain);
+		}
+	}
+
+	return 0;
+}
+
 struct io_apic {
 	unsigned int index;
 	unsigned int unused[3];
-- 
1.7.10.4

Re: [PATCH v2 13/26] x86, irq: Move down arch_early_irq_init()

[Konrad Rzeszutek Wilk]

On Fri, Feb 08, 2013 at 11:28:10AM -0800, Yinghai Lu wrote:
> Change position only.
> 
> Prepare to update arch_early_irq_init(), it needs call some static functions.
                                                   ^^-> to
> 
> Signed-off-by: Yinghai Lu <yinghai@kernel.org>
> Cc: Joerg Roedel <joro@8bytes.org>
> Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
> Cc: Sebastian Andrzej Siewior <sebastian@breakpoint.cc>
> ---
>  arch/x86/kernel/apic/io_apic.c |   89 ++++++++++++++++++++--------------------
>  1 file changed, 44 insertions(+), 45 deletions(-)
> 
> diff --git a/arch/x86/kernel/apic/io_apic.c b/arch/x86/kernel/apic/io_apic.c
> index 72a1351..deda981 100644
> --- a/arch/x86/kernel/apic/io_apic.c
> +++ b/arch/x86/kernel/apic/io_apic.c
> @@ -185,51 +185,6 @@ static struct irq_pin_list *alloc_irq_pin_list(int node)
>  	return kzalloc_node(sizeof(struct irq_pin_list), GFP_KERNEL, node);
>  }
>  
> -
> -/* irq_cfg is indexed by the sum of all RTEs in all I/O APICs. */
> -static struct irq_cfg irq_cfgx[NR_IRQS_LEGACY];
> -
> -int __init arch_early_irq_init(void)
> -{
> -	struct irq_cfg *cfg;
> -	int count, node, i;
> -
> -	if (!legacy_pic->nr_legacy_irqs)
> -		io_apic_irqs = ~0UL;
> -
> -	for (i = 0; i < nr_ioapics; i++) {
> -		ioapics[i].saved_registers =
> -			kzalloc(sizeof(struct IO_APIC_route_entry) *
> -				ioapics[i].nr_registers, GFP_KERNEL);
> -		if (!ioapics[i].saved_registers)
> -			pr_err("IOAPIC %d: suspend/resume impossible!\n", i);
> -	}
> -
> -	cfg = irq_cfgx;
> -	count = ARRAY_SIZE(irq_cfgx);
> -	node = cpu_to_node(0);
> -
> -	/* Make sure the legacy interrupts are marked in the bitmap */
> -	irq_reserve_irqs(0, legacy_pic->nr_legacy_irqs);
> -
> -	for (i = 0; i < count; i++) {
> -		INIT_LIST_HEAD(&cfg[i].irq_2_pin);
> -		irq_set_chip_data(i, &cfg[i]);
> -		zalloc_cpumask_var_node(&cfg[i].domain, GFP_KERNEL, node);
> -		zalloc_cpumask_var_node(&cfg[i].old_domain, GFP_KERNEL, node);
> -		/*
> -		 * For legacy IRQ's, start with assigning irq0 to irq15 to
> -		 * IRQ0_VECTOR to IRQ15_VECTOR for all cpu's.
> -		 */
> -		if (i < legacy_pic->nr_legacy_irqs) {
> -			cfg[i].vector = IRQ0_VECTOR + i;
> -			cpumask_setall(cfg[i].domain);
> -		}
> -	}
> -
> -	return 0;
> -}
> -
>  static struct irq_cfg *irq_cfg(unsigned int irq)
>  {
>  	return irq_get_chip_data(irq);
> @@ -332,6 +287,50 @@ static struct irq_cfg *realloc_irq_and_cfg_at(unsigned int at, int node)
>  	return alloc_irq_and_cfg_at(at, node);
>  }
>  
> +/* irq_cfg is indexed by the sum of all RTEs in all I/O APICs. */
> +static struct irq_cfg irq_cfgx[NR_IRQS_LEGACY];
> +
> +int __init arch_early_irq_init(void)
> +{
> +	struct irq_cfg *cfg;
> +	int count, node, i;
> +
> +	if (!legacy_pic->nr_legacy_irqs)
> +		io_apic_irqs = ~0UL;
> +
> +	for (i = 0; i < nr_ioapics; i++) {
> +		ioapics[i].saved_registers =
> +			kzalloc(sizeof(struct IO_APIC_route_entry) *
> +				ioapics[i].nr_registers, GFP_KERNEL);
> +		if (!ioapics[i].saved_registers)
> +			pr_err("IOAPIC %d: suspend/resume impossible!\n", i);
> +	}
> +
> +	cfg = irq_cfgx;
> +	count = ARRAY_SIZE(irq_cfgx);
> +	node = cpu_to_node(0);
> +
> +	/* Make sure the legacy interrupts are marked in the bitmap */
> +	irq_reserve_irqs(0, legacy_pic->nr_legacy_irqs);
> +
> +	for (i = 0; i < count; i++) {
> +		INIT_LIST_HEAD(&cfg[i].irq_2_pin);
> +		irq_set_chip_data(i, &cfg[i]);
> +		zalloc_cpumask_var_node(&cfg[i].domain, GFP_KERNEL, node);
> +		zalloc_cpumask_var_node(&cfg[i].old_domain, GFP_KERNEL, node);
> +		/*
> +		 * For legacy IRQ's, start with assigning irq0 to irq15 to
> +		 * IRQ0_VECTOR to IRQ15_VECTOR for all cpu's.
> +		 */
> +		if (i < legacy_pic->nr_legacy_irqs) {
> +			cfg[i].vector = IRQ0_VECTOR + i;
> +			cpumask_setall(cfg[i].domain);
> +		}
> +	}
> +
> +	return 0;
> +}
> +
>  struct io_apic {
>  	unsigned int index;
>  	unsigned int unused[3];
> -- 
> 1.7.10.4
> 

[PATCH v2 14/26] x86, irq: Split out alloc_ioapic_save_registers()

[Yinghai Lu]

Split alloc_ioapic_save_registers() from early_irq_init(),
so it will be per ioapic.

Will call that later for hot added ioapic.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
Cc: Joerg Roedel <joro@8bytes.org>
Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Cc: Sebastian Andrzej Siewior <sebastian@breakpoint.cc>
---
 arch/x86/kernel/apic/io_apic.c |   22 +++++++++++++++-------
 1 file changed, 15 insertions(+), 7 deletions(-)

diff --git a/arch/x86/kernel/apic/io_apic.c b/arch/x86/kernel/apic/io_apic.c
index deda981..714ed20 100644
--- a/arch/x86/kernel/apic/io_apic.c
+++ b/arch/x86/kernel/apic/io_apic.c
@@ -290,6 +290,19 @@ static struct irq_cfg *realloc_irq_and_cfg_at(unsigned int at, int node)
 /* irq_cfg is indexed by the sum of all RTEs in all I/O APICs. */
 static struct irq_cfg irq_cfgx[NR_IRQS_LEGACY];
 
+static void alloc_ioapic_saved_registers(int idx)
+{
+	if (ioapics[idx].saved_registers)
+		return;
+
+	ioapics[idx].saved_registers =
+			kzalloc(sizeof(struct IO_APIC_route_entry) *
+				ioapics[idx].nr_registers, GFP_KERNEL);
+
+	if (!ioapics[idx].saved_registers)
+		pr_err("IOAPIC %d: suspend/resume impossible!\n", idx);
+}
+
 int __init arch_early_irq_init(void)
 {
 	struct irq_cfg *cfg;
@@ -298,13 +311,8 @@ int __init arch_early_irq_init(void)
 	if (!legacy_pic->nr_legacy_irqs)
 		io_apic_irqs = ~0UL;
 
-	for (i = 0; i < nr_ioapics; i++) {
-		ioapics[i].saved_registers =
-			kzalloc(sizeof(struct IO_APIC_route_entry) *
-				ioapics[i].nr_registers, GFP_KERNEL);
-		if (!ioapics[i].saved_registers)
-			pr_err("IOAPIC %d: suspend/resume impossible!\n", i);
-	}
+	for (i = 0; i < nr_ioapics; i++)
+		alloc_ioapic_saved_registers(i);
 
 	cfg = irq_cfgx;
 	count = ARRAY_SIZE(irq_cfgx);
-- 
1.7.10.4

[PATCH v2 15/26] xen, irq: call irq_realloc_desc_at() at first

[Yinghai Lu]

We will pre-reserve irq for all gsi at first for x86, so we have to
use realloc with it.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Cc: xen-devel@lists.xensource.com
---
 drivers/xen/events.c |    8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/drivers/xen/events.c b/drivers/xen/events.c
index 0be4df3..dda38db 100644
--- a/drivers/xen/events.c
+++ b/drivers/xen/events.c
@@ -482,8 +482,12 @@ static int __must_check xen_allocate_irq_gsi(unsigned gsi)
 	/* Legacy IRQ descriptors are already allocated by the arch. */
 	if (gsi < NR_IRQS_LEGACY)
 		irq = gsi;
-	else
-		irq = irq_alloc_desc_at(gsi, -1);
+	else {
+		/* for x86, irq already get reserved for gsi */
+		irq = irq_realloc_desc_at(gsi, -1);
+		if (irq < 0)
+			irq = irq_alloc_desc_at(gsi, -1);
+	}
 
 	xen_irq_init(irq);
 
-- 
1.7.10.4

Re: [PATCH v2 15/26] xen, irq: call irq_realloc_desc_at() at first

[Konrad Rzeszutek Wilk]

On Fri, Feb 08, 2013 at 11:28:12AM -0800, Yinghai Lu wrote:
> We will pre-reserve irq for all gsi at first for x86, so we have to
> use realloc with it.

Where are the GSI's pre-allocated?

> 
> Signed-off-by: Yinghai Lu <yinghai@kernel.org>
> Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
> Cc: xen-devel@lists.xensource.com
> ---
>  drivers/xen/events.c |    8 ++++++--
>  1 file changed, 6 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/xen/events.c b/drivers/xen/events.c
> index 0be4df3..dda38db 100644
> --- a/drivers/xen/events.c
> +++ b/drivers/xen/events.c
> @@ -482,8 +482,12 @@ static int __must_check xen_allocate_irq_gsi(unsigned gsi)
>  	/* Legacy IRQ descriptors are already allocated by the arch. */
>  	if (gsi < NR_IRQS_LEGACY)
>  		irq = gsi;
> -	else
> -		irq = irq_alloc_desc_at(gsi, -1);
> +	else {
> +		/* for x86, irq already get reserved for gsi */
> +		irq = irq_realloc_desc_at(gsi, -1);
> +		if (irq < 0)
> +			irq = irq_alloc_desc_at(gsi, -1);
> +	}
>  
>  	xen_irq_init(irq);
>  
> -- 
> 1.7.10.4
> 

Re: [PATCH v2 15/26] xen, irq: call irq_realloc_desc_at() at first

[Yinghai Lu]

On Fri, Mar 8, 2013 at 11:43 AM, Konrad Rzeszutek Wilk
<konrad.wilk@oracle.com> wrote:
> On Fri, Feb 08, 2013 at 11:28:12AM -0800, Yinghai Lu wrote:
>> We will pre-reserve irq for all gsi at first for x86, so we have to
>> use realloc with it.
>
> Where are the GSI's pre-allocated?

In following patch.

I put this one forward, so it will not break git bisecting.

Re: [PATCH v2 15/26] xen, irq: call irq_realloc_desc_at() at first

[Konrad Rzeszutek Wilk]

On Fri, Mar 08, 2013 at 02:43:57PM -0500, Konrad Rzeszutek Wilk wrote:
> On Fri, Feb 08, 2013 at 11:28:12AM -0800, Yinghai Lu wrote:
> > We will pre-reserve irq for all gsi at first for x86, so we have to
> > use realloc with it.
> 
> Where are the GSI's pre-allocated?

And the answer is in arch_probe_nr_irqs and determined by nr_irqs_gsi.
That data then feeds in early_irq_init which sets the bitmap.

Could you add that please in the git commit if it is not too much
trouble?
> 
> > 
> > Signed-off-by: Yinghai Lu <yinghai@kernel.org>
> > Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
> > Cc: xen-devel@lists.xensource.com
> > ---
> >  drivers/xen/events.c |    8 ++++++--
> >  1 file changed, 6 insertions(+), 2 deletions(-)
> > 
> > diff --git a/drivers/xen/events.c b/drivers/xen/events.c
> > index 0be4df3..dda38db 100644
> > --- a/drivers/xen/events.c
> > +++ b/drivers/xen/events.c
> > @@ -482,8 +482,12 @@ static int __must_check xen_allocate_irq_gsi(unsigned gsi)
> >  	/* Legacy IRQ descriptors are already allocated by the arch. */
> >  	if (gsi < NR_IRQS_LEGACY)
> >  		irq = gsi;
> > -	else
> > -		irq = irq_alloc_desc_at(gsi, -1);
> > +	else {
> > +		/* for x86, irq already get reserved for gsi */
> > +		irq = irq_realloc_desc_at(gsi, -1);
> > +		if (irq < 0)
> > +			irq = irq_alloc_desc_at(gsi, -1);
> > +	}
> >  
> >  	xen_irq_init(irq);
> >  
> > -- 
> > 1.7.10.4
> > 

[PATCH v2 16/26] x86, irq: pre-reserve irq range/realloc for booting path

[Yinghai Lu]

We will use reserve/realloc_irq_and_cfg_at for hotplug ioapic path.

To make thing simple, we could make booting path use same code.
So all gsi range will be reserved at first, and realloc will really
allocated those irq_desc/cfg when it is used.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
Cc: Joerg Roedel <joro@8bytes.org>
Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Cc: Sebastian Andrzej Siewior <sebastian@breakpoint.cc>
---
 arch/x86/include/asm/io_apic.h |    1 +
 arch/x86/kernel/apic/io_apic.c |   78 ++++++++++++++++++++++++++++------------
 2 files changed, 57 insertions(+), 22 deletions(-)

diff --git a/arch/x86/include/asm/io_apic.h b/arch/x86/include/asm/io_apic.h
index eaff3ad..8181fd8 100644
--- a/arch/x86/include/asm/io_apic.h
+++ b/arch/x86/include/asm/io_apic.h
@@ -176,6 +176,7 @@ extern void setup_ioapic_ids_from_mpc_nocheck(void);
 struct mp_ioapic_gsi{
 	u32 gsi_base;
 	u32 gsi_end;
+	u32 irq_base;
 };
 extern struct mp_ioapic_gsi  mp_gsi_routing[];
 extern u32 gsi_top;
diff --git a/arch/x86/kernel/apic/io_apic.c b/arch/x86/kernel/apic/io_apic.c
index 714ed20..c499c88 100644
--- a/arch/x86/kernel/apic/io_apic.c
+++ b/arch/x86/kernel/apic/io_apic.c
@@ -287,8 +287,47 @@ static struct irq_cfg *realloc_irq_and_cfg_at(unsigned int at, int node)
 	return alloc_irq_and_cfg_at(at, node);
 }
 
-/* irq_cfg is indexed by the sum of all RTEs in all I/O APICs. */
-static struct irq_cfg irq_cfgx[NR_IRQS_LEGACY];
+static int reserve_ioapic_gsi_irq_base(int idx)
+{
+	int irq;
+	struct mp_ioapic_gsi *gsi_cfg = mp_ioapic_gsi_routing(idx);
+	int cnt = gsi_cfg->gsi_end - gsi_cfg->gsi_base + 1;
+
+	irq = __irq_reserve_irqs(-1, gsi_cfg->gsi_base, cnt);
+	if (irq >= 0) {
+		gsi_cfg->irq_base = irq;
+		apic_printk(APIC_VERBOSE, KERN_INFO
+			"IOAPIC[%d]: apic_id %d, GSI %d-%d ==> irq %d-%d reserved\n",
+			idx, mpc_ioapic_id(idx),
+			gsi_cfg->gsi_base, gsi_cfg->gsi_end,
+			irq, irq + cnt - 1);
+	} else
+		apic_printk(APIC_VERBOSE, KERN_WARNING
+			"IOAPIC[%d]: apic_id %d, GSI %d-%d ==> irq reserve failed\n",
+			idx, mpc_ioapic_id(idx),
+			gsi_cfg->gsi_base, gsi_cfg->gsi_end);
+
+	return irq;
+}
+
+static void __init reserve_ioapic_gsi_irq_extra(void)
+{
+	int irq;
+
+	/* to prevent hot add ioapic taking those slots */
+	if (gsi_top) {
+		irq = irq_reserve_irqs(gsi_top, NR_IRQS_LEGACY);
+		if (irq >= 0)
+			apic_printk(APIC_VERBOSE, KERN_INFO
+				"IOAPIC[extra]: GSI %d-%d ==> irq %d-%d reserved\n",
+				gsi_top, gsi_top + NR_IRQS_LEGACY - 1,
+				irq, irq + NR_IRQS_LEGACY - 1);
+		else
+			apic_printk(APIC_VERBOSE, KERN_WARNING
+				"IOAPIC[extra]: GSI %d-%d ==> irq reserve failed\n",
+				gsi_top, gsi_top + NR_IRQS_LEGACY - 1);
+	}
+}
 
 static void alloc_ioapic_saved_registers(int idx)
 {
@@ -305,8 +344,9 @@ static void alloc_ioapic_saved_registers(int idx)
 
 int __init arch_early_irq_init(void)
 {
+	int node = cpu_to_node(0);
 	struct irq_cfg *cfg;
-	int count, node, i;
+	int i;
 
 	if (!legacy_pic->nr_legacy_irqs)
 		io_apic_irqs = ~0UL;
@@ -314,26 +354,19 @@ int __init arch_early_irq_init(void)
 	for (i = 0; i < nr_ioapics; i++)
 		alloc_ioapic_saved_registers(i);
 
-	cfg = irq_cfgx;
-	count = ARRAY_SIZE(irq_cfgx);
-	node = cpu_to_node(0);
+	for (i = 0; i < nr_ioapics; i++)
+		reserve_ioapic_gsi_irq_base(i);
 
-	/* Make sure the legacy interrupts are marked in the bitmap */
-	irq_reserve_irqs(0, legacy_pic->nr_legacy_irqs);
+	reserve_ioapic_gsi_irq_extra();
 
-	for (i = 0; i < count; i++) {
-		INIT_LIST_HEAD(&cfg[i].irq_2_pin);
-		irq_set_chip_data(i, &cfg[i]);
-		zalloc_cpumask_var_node(&cfg[i].domain, GFP_KERNEL, node);
-		zalloc_cpumask_var_node(&cfg[i].old_domain, GFP_KERNEL, node);
-		/*
-		 * For legacy IRQ's, start with assigning irq0 to irq15 to
-		 * IRQ0_VECTOR to IRQ15_VECTOR for all cpu's.
-		 */
-		if (i < legacy_pic->nr_legacy_irqs) {
-			cfg[i].vector = IRQ0_VECTOR + i;
-			cpumask_setall(cfg[i].domain);
-		}
+	/*
+	 * For legacy IRQ's, start with assigning irq0 to irq15 to
+	 * IRQ0_VECTOR to IRQ15_VECTOR for all cpu's.
+	 */
+	for (i = 0; i < legacy_pic->nr_legacy_irqs; i++) {
+		cfg = realloc_irq_and_cfg_at(i, node);
+		cfg->vector = IRQ0_VECTOR + i;
+		cpumask_setall(cfg->domain);
 	}
 
 	return 0;
@@ -3470,7 +3503,8 @@ int __init arch_probe_nr_irqs(void)
 	if (nr < nr_irqs)
 		nr_irqs = nr;
 
-	return NR_IRQS_LEGACY;
+	/* x86 arch code will allocate irq_desc/cfg */
+	return 0;
 }
 
 int io_apic_set_pci_routing(struct device *dev, int irq,
-- 
1.7.10.4

[PATCH v2 17/26] x86, irq: Add ioapic_gsi_to_irq

[Yinghai Lu]

For hot add ioapic, irq_base is not equal to gsi_base.

So we need a way to do mapping between gsi and irq.

Also remove irq_to_gsi that is causing confusing, just use that array
directly as only caller already check input irq before.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
Cc: Pavel Machek <pavel@ucw.cz>
Cc: Joerg Roedel <joro@8bytes.org>
Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Cc: Sebastian Andrzej Siewior <sebastian@breakpoint.cc>
---
 arch/x86/include/asm/io_apic.h |    1 +
 arch/x86/kernel/acpi/boot.c    |   22 +++++-----------------
 arch/x86/kernel/apic/io_apic.c |   29 ++++++++++++++++++++++++++++-
 3 files changed, 34 insertions(+), 18 deletions(-)

diff --git a/arch/x86/include/asm/io_apic.h b/arch/x86/include/asm/io_apic.h
index 8181fd8..02ac411 100644
--- a/arch/x86/include/asm/io_apic.h
+++ b/arch/x86/include/asm/io_apic.h
@@ -180,6 +180,7 @@ struct mp_ioapic_gsi{
 };
 extern struct mp_ioapic_gsi  mp_gsi_routing[];
 extern u32 gsi_top;
+int ioapic_gsi_to_irq(u32 gsi);
 int mp_find_ioapic(u32 gsi);
 int mp_find_ioapic_pin(int ioapic, u32 gsi);
 void __init mp_register_ioapic(int id, u32 address, u32 gsi_base);
diff --git a/arch/x86/kernel/acpi/boot.c b/arch/x86/kernel/acpi/boot.c
index bacf4b0..350879f 100644
--- a/arch/x86/kernel/acpi/boot.c
+++ b/arch/x86/kernel/acpi/boot.c
@@ -112,6 +112,10 @@ static unsigned int gsi_to_irq(unsigned int gsi)
 		}
 	}
 
+#ifdef CONFIG_X86_IO_APIC
+	if (acpi_irq_model == ACPI_IRQ_MODEL_IOAPIC)
+		return ioapic_gsi_to_irq(gsi);
+#endif
 	/* Provide an identity mapping of gsi == irq
 	 * except on truly weird platforms that have
 	 * non isa irqs in the first 16 gsis.
@@ -124,22 +128,6 @@ static unsigned int gsi_to_irq(unsigned int gsi)
 	return irq;
 }
 
-static u32 irq_to_gsi(int irq)
-{
-	unsigned int gsi;
-
-	if (irq < NR_IRQS_LEGACY)
-		gsi = isa_irq_to_gsi[irq];
-	else if (irq < gsi_top)
-		gsi = irq;
-	else if (irq < (gsi_top + NR_IRQS_LEGACY))
-		gsi = irq - gsi_top;
-	else
-		gsi = 0xffffffff;
-
-	return gsi;
-}
-
 /*
  * Temporarily use the virtual area starting from FIX_IO_APIC_BASE_END,
  * to map the target physical address. The problem is that set_fixmap()
@@ -529,7 +517,7 @@ int acpi_isa_irq_to_gsi(unsigned isa_irq, u32 *gsi)
 {
 	if (isa_irq >= 16)
 		return -1;
-	*gsi = irq_to_gsi(isa_irq);
+	*gsi = isa_irq_to_gsi[isa_irq];
 	return 0;
 }
 
diff --git a/arch/x86/kernel/apic/io_apic.c b/arch/x86/kernel/apic/io_apic.c
index c499c88..9514728 100644
--- a/arch/x86/kernel/apic/io_apic.c
+++ b/arch/x86/kernel/apic/io_apic.c
@@ -1044,13 +1044,16 @@ static int pin_2_irq(int idx, int apic, int pin)
 
 	if (test_bit(bus, mp_bus_not_pci)) {
 		irq = mp_irqs[idx].srcbusirq;
-	} else {
+	} else if (gsi_cfg->gsi_base == gsi_cfg->irq_base) {
 		u32 gsi = gsi_cfg->gsi_base + pin;
 
 		if (gsi >= NR_IRQS_LEGACY)
 			irq = gsi;
 		else
 			irq = gsi_top + gsi;
+	} else {
+		/* hotadd ioapic */
+		irq = gsi_cfg->irq_base + pin;
 	}
 
 #ifdef CONFIG_X86_32
@@ -1476,6 +1479,30 @@ static void __init setup_IO_APIC_irqs(void)
 		__io_apic_setup_irqs(ioapic_idx);
 }
 
+int ioapic_gsi_to_irq(u32 gsi)
+{
+	int ioapic_idx = 0, irq = gsi;
+	struct mp_ioapic_gsi *gsi_cfg;
+
+	ioapic_idx = mp_find_ioapic(gsi);
+	if (ioapic_idx < 0)
+		return -1;
+
+	gsi_cfg = mp_ioapic_gsi_routing(ioapic_idx);
+	if (gsi_cfg->gsi_base == gsi_cfg->irq_base) {
+		if (gsi < NR_IRQS_LEGACY)
+			irq = gsi_top + gsi;
+	} else {
+		int pin = mp_find_ioapic_pin(ioapic_idx, gsi);
+
+		if (pin < 0)
+			return -1;
+		/* hotadd ioapic */
+		irq = gsi_cfg->irq_base + pin;
+	}
+
+	return irq;
+}
 /*
  * for the gsit that is not in first ioapic
  * but could not use acpi_register_gsi()
-- 
1.7.10.4

[PATCH v2 18/26] genirq: Bail out early in free_desc()

[Yinghai Lu]

We pre-reserve irq range for hot-added ioapic, and later only
some are used via realloc.
So during hot-remove, we need to clear bits in allocated_irqs
for both case.

Check if the irq_desc is there at first, and bail out early if
irq_desc is not allocated yet.

So we can use irq_free_descs to clear allocated_irqs bits for
preserved irqs.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
---
 kernel/irq/irqdesc.c |    3 +++
 1 file changed, 3 insertions(+)

diff --git a/kernel/irq/irqdesc.c b/kernel/irq/irqdesc.c
index b48f65b..32f099e 100644
--- a/kernel/irq/irqdesc.c
+++ b/kernel/irq/irqdesc.c
@@ -170,6 +170,9 @@ static void free_desc(unsigned int irq)
 {
 	struct irq_desc *desc = irq_to_desc(irq);
 
+	if (!desc)
+		return;
+
 	unregister_irq_proc(irq, desc);
 
 	mutex_lock(&sparse_irq_lock);
-- 
1.7.10.4

[PATCH v2 19/26] x86, irq: More strict checking about registering ioapic

[Yinghai Lu]

1. check overlaping gsi range
for hotplug ioapic case, BIOS may have some entries in MADT
and also have setting in pci root bus with _GSB of DSDT.

2. make bad_ioapics check idx instead of nr_ioapics.
for hotadd ioapic could find spare slot in the middle later.

3. check if entries is in right range.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
Cc: Joerg Roedel <joro@8bytes.org>
Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Cc: Sebastian Andrzej Siewior <sebastian@breakpoint.cc>
---
 arch/x86/kernel/apic/io_apic.c |   29 ++++++++++++++++++++++-------
 1 file changed, 22 insertions(+), 7 deletions(-)

diff --git a/arch/x86/kernel/apic/io_apic.c b/arch/x86/kernel/apic/io_apic.c
index 9514728..8e92ba0 100644
--- a/arch/x86/kernel/apic/io_apic.c
+++ b/arch/x86/kernel/apic/io_apic.c
@@ -3821,7 +3821,7 @@ void __init ioapic_insert_resources(void)
 	}
 }
 
-int mp_find_ioapic(u32 gsi)
+static int __mp_find_ioapic(u32 gsi, bool quiet)
 {
 	int i = 0;
 
@@ -3836,10 +3836,16 @@ int mp_find_ioapic(u32 gsi)
 			return i;
 	}
 
-	printk(KERN_ERR "ERROR: Unable to locate IOAPIC for GSI %d\n", gsi);
+	if (!quiet)
+		pr_err("ERROR: Unable to locate IOAPIC for GSI %d\n", gsi);
 	return -1;
 }
 
+int mp_find_ioapic(u32 gsi)
+{
+	return __mp_find_ioapic(gsi, false);
+}
+
 int mp_find_ioapic_pin(int ioapic, u32 gsi)
 {
 	struct mp_ioapic_gsi *gsi_cfg;
@@ -3854,11 +3860,11 @@ int mp_find_ioapic_pin(int ioapic, u32 gsi)
 	return gsi - gsi_cfg->gsi_base;
 }
 
-static __init int bad_ioapic(unsigned long address)
+static __init int bad_ioapic(int idx, unsigned long address)
 {
-	if (nr_ioapics >= MAX_IO_APICS) {
+	if (idx >= MAX_IO_APICS) {
 		pr_warn("WARNING: Max # of I/O APICs (%d) exceeded (found %d), skipping\n",
-			MAX_IO_APICS, nr_ioapics);
+			MAX_IO_APICS, idx);
 		return 1;
 	}
 	if (!address) {
@@ -3889,14 +3895,17 @@ static __init int bad_ioapic_register(int idx)
 
 void __init mp_register_ioapic(int id, u32 address, u32 gsi_base)
 {
-	int idx = 0;
+	int idx;
 	int entries;
 	struct mp_ioapic_gsi *gsi_cfg;
 
-	if (bad_ioapic(address))
+	idx = __mp_find_ioapic(gsi_base, true);
+	if (idx >= 0)
 		return;
 
 	idx = nr_ioapics;
+	if (bad_ioapic(idx, address))
+		return;
 
 	ioapics[idx].mp_config.type = MP_IOAPIC;
 	ioapics[idx].mp_config.flags = MPC_APIC_USABLE;
@@ -3917,6 +3926,12 @@ void __init mp_register_ioapic(int id, u32 address, u32 gsi_base)
 	 * and to prevent reprogramming of IOAPIC pins (PCI GSIs).
 	 */
 	entries = io_apic_get_redir_entries(idx);
+
+	if (!entries || entries > MP_MAX_IOAPIC_PIN) {
+		clear_fixmap(FIX_IO_APIC_BASE_0 + idx);
+		return;
+	}
+
 	gsi_cfg = mp_ioapic_gsi_routing(idx);
 	gsi_cfg->gsi_base = gsi_base;
 	gsi_cfg->gsi_end = gsi_base + entries - 1;
-- 
1.7.10.4

[PATCH v2 20/26] x86, irq: Make mp_register_ioapic handle hotadd ioapic

[Yinghai Lu]

It needs to reserve irq range in allocated_irqs bitmaps
and irq_base will be used to get right irq for ioapic/pin or gsi.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
Cc: Paul Gortmaker <paul.gortmaker@windriver.com>
Cc: Joerg Roedel <joro@8bytes.org>
Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Cc: Sebastian Andrzej Siewior <sebastian@breakpoint.cc>
---
 arch/x86/include/asm/mpspec.h  |    1 +
 arch/x86/kernel/apic/io_apic.c |   59 ++++++++++++++++++++++++++++++----------
 2 files changed, 46 insertions(+), 14 deletions(-)

diff --git a/arch/x86/include/asm/mpspec.h b/arch/x86/include/asm/mpspec.h
index 3e2f42a..11bb6ea 100644
--- a/arch/x86/include/asm/mpspec.h
+++ b/arch/x86/include/asm/mpspec.h
@@ -96,6 +96,7 @@ static inline void early_reserve_e820_mpc_new(void) { }
 
 void __cpuinit generic_processor_info(int apicid, int version);
 #ifdef CONFIG_ACPI
+int __mp_register_ioapic(int id, u32 address, u32 gsi_base, bool hot);
 extern void mp_register_ioapic(int id, u32 address, u32 gsi_base);
 extern void mp_override_legacy_irq(u8 bus_irq, u8 polarity, u8 trigger,
 				   u32 gsi);
diff --git a/arch/x86/kernel/apic/io_apic.c b/arch/x86/kernel/apic/io_apic.c
index 8e92ba0..d3b3c14 100644
--- a/arch/x86/kernel/apic/io_apic.c
+++ b/arch/x86/kernel/apic/io_apic.c
@@ -3893,7 +3893,7 @@ static __init int bad_ioapic_register(int idx)
 	return 0;
 }
 
-void __init mp_register_ioapic(int id, u32 address, u32 gsi_base)
+int __mp_register_ioapic(int id, u32 address, u32 gsi_base, bool hotadd)
 {
 	int idx;
 	int entries;
@@ -3901,11 +3901,19 @@ void __init mp_register_ioapic(int id, u32 address, u32 gsi_base)
 
 	idx = __mp_find_ioapic(gsi_base, true);
 	if (idx >= 0)
-		return;
+		return -EINVAL;
 
 	idx = nr_ioapics;
+	if (hotadd) {
+		/* find free spot */
+		for (idx = 0; idx < nr_ioapics; idx++)
+			if (!ioapics[idx].nr_registers &&
+			    ioapics[idx].mp_config.apicid == 0xff)
+				break;
+	}
+
 	if (bad_ioapic(idx, address))
-		return;
+		return -EINVAL;
 
 	ioapics[idx].mp_config.type = MP_IOAPIC;
 	ioapics[idx].mp_config.flags = MPC_APIC_USABLE;
@@ -3913,10 +3921,8 @@ void __init mp_register_ioapic(int id, u32 address, u32 gsi_base)
 
 	set_fixmap_nocache(FIX_IO_APIC_BASE_0 + idx, address);
 
-	if (bad_ioapic_register(idx)) {
-		clear_fixmap(FIX_IO_APIC_BASE_0 + idx);
-		return;
-	}
+	if (bad_ioapic_register(idx))
+		goto failed;
 
 	ioapics[idx].mp_config.apicid = io_apic_unique_id(id);
 	ioapics[idx].mp_config.apicver = io_apic_get_version(idx);
@@ -3927,10 +3933,8 @@ void __init mp_register_ioapic(int id, u32 address, u32 gsi_base)
 	 */
 	entries = io_apic_get_redir_entries(idx);
 
-	if (!entries || entries > MP_MAX_IOAPIC_PIN) {
-		clear_fixmap(FIX_IO_APIC_BASE_0 + idx);
-		return;
-	}
+	if (!entries || entries > MP_MAX_IOAPIC_PIN)
+		goto failed;
 
 	gsi_cfg = mp_ioapic_gsi_routing(idx);
 	gsi_cfg->gsi_base = gsi_base;
@@ -3941,15 +3945,42 @@ void __init mp_register_ioapic(int id, u32 address, u32 gsi_base)
 	 */
 	ioapics[idx].nr_registers = entries;
 
-	if (gsi_cfg->gsi_end >= gsi_top)
-		gsi_top = gsi_cfg->gsi_end + 1;
+	if (!hotadd) {
+		/*
+		 * irqs will be reserved in arch_early_irq_init()
+		 * don't need to update gsi_top for hot add case
+		 */
+		if (gsi_cfg->gsi_end >= gsi_top)
+			gsi_top = gsi_cfg->gsi_end + 1;
+	} else {
+		int irq = reserve_ioapic_gsi_irq_base(idx);
+
+		if (irq < 0)
+			goto failed;
+
+		alloc_ioapic_saved_registers(idx);
+	}
 
 	pr_info("IOAPIC[%d]: apic_id %d, version %d, address 0x%x, GSI %d-%d\n",
 		idx, mpc_ioapic_id(idx),
 		mpc_ioapic_ver(idx), mpc_ioapic_addr(idx),
 		gsi_cfg->gsi_base, gsi_cfg->gsi_end);
 
-	nr_ioapics++;
+	if (idx == nr_ioapics)
+		nr_ioapics++;
+
+	return 0;
+
+failed:
+	clear_fixmap(FIX_IO_APIC_BASE_0 + idx);
+	memset(&ioapics[idx], 0, sizeof(struct ioapic));
+	ioapics[idx].mp_config.apicid = 0xff;
+	return -EINVAL;
+}
+
+void mp_register_ioapic(int id, u32 address, u32 gsi_base)
+{
+	__mp_register_ioapic(id, address, gsi_base, false);
 }
 
 /* Enable IOAPIC early just for system timer */
-- 
1.7.10.4

[PATCH v2 21/26] x86, irq: Add mp_unregister_ioapic to handle hot-remove ioapic

[Yinghai Lu]

It will free ioapic related irq_desc and also clear allocated_irqs bits.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
Cc: Joerg Roedel <joro@8bytes.org>
Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Cc: Sebastian Andrzej Siewior <sebastian@breakpoint.cc>
---
 arch/x86/include/asm/mpspec.h  |    1 +
 arch/x86/kernel/apic/io_apic.c |   42 ++++++++++++++++++++++++++++++++++++++++
 2 files changed, 43 insertions(+)

diff --git a/arch/x86/include/asm/mpspec.h b/arch/x86/include/asm/mpspec.h
index 11bb6ea..e70b7e5 100644
--- a/arch/x86/include/asm/mpspec.h
+++ b/arch/x86/include/asm/mpspec.h
@@ -97,6 +97,7 @@ static inline void early_reserve_e820_mpc_new(void) { }
 void __cpuinit generic_processor_info(int apicid, int version);
 #ifdef CONFIG_ACPI
 int __mp_register_ioapic(int id, u32 address, u32 gsi_base, bool hot);
+int mp_unregister_ioapic(u32 gsi_base);
 extern void mp_register_ioapic(int id, u32 address, u32 gsi_base);
 extern void mp_override_legacy_irq(u8 bus_irq, u8 polarity, u8 trigger,
 				   u32 gsi);
diff --git a/arch/x86/kernel/apic/io_apic.c b/arch/x86/kernel/apic/io_apic.c
index d3b3c14..4a9ccf6 100644
--- a/arch/x86/kernel/apic/io_apic.c
+++ b/arch/x86/kernel/apic/io_apic.c
@@ -256,6 +256,14 @@ static void free_irq_at(unsigned int at, struct irq_cfg *cfg)
 	irq_free_desc(at);
 }
 
+static void free_irqs(unsigned int from, int cnt)
+{
+	int i;
+
+	for (i = from; i < from + cnt; i++)
+		free_irq_at(i, irq_get_chip_data(i));
+}
+
 static struct irq_cfg *realloc_irq_and_cfg_at(unsigned int at, int node)
 {
 	struct irq_desc *desc = irq_to_desc(at);
@@ -329,6 +337,16 @@ static void __init reserve_ioapic_gsi_irq_extra(void)
 	}
 }
 
+static void free_ioapic_gsi_irq_base(int idx)
+{
+	struct mp_ioapic_gsi *gsi_cfg = mp_ioapic_gsi_routing(idx);
+	int irq_base = gsi_cfg->irq_base;
+	int irq_cnt = gsi_cfg->gsi_end - gsi_cfg->gsi_base + 1;
+
+	if (irq_base > 0)
+		free_irqs(irq_base, irq_cnt);
+}
+
 static void alloc_ioapic_saved_registers(int idx)
 {
 	if (ioapics[idx].saved_registers)
@@ -342,6 +360,11 @@ static void alloc_ioapic_saved_registers(int idx)
 		pr_err("IOAPIC %d: suspend/resume impossible!\n", idx);
 }
 
+static void free_ioapic_saved_registers(int idx)
+{
+	kfree(ioapics[idx].saved_registers);
+}
+
 int __init arch_early_irq_init(void)
 {
 	int node = cpu_to_node(0);
@@ -3983,6 +4006,25 @@ void mp_register_ioapic(int id, u32 address, u32 gsi_base)
 	__mp_register_ioapic(id, address, gsi_base, false);
 }
 
+int mp_unregister_ioapic(u32 gsi_base)
+{
+	int idx;
+
+	idx = mp_find_ioapic(gsi_base);
+	if (idx < 0)
+		return -EINVAL;
+
+	free_ioapic_saved_registers(idx);
+
+	free_ioapic_gsi_irq_base(idx);
+
+	clear_fixmap(FIX_IO_APIC_BASE_0 + idx);
+	memset(&ioapics[idx], 0, sizeof(struct ioapic));
+	ioapics[idx].mp_config.apicid = 0xff;
+
+	return 0;
+}
+
 /* Enable IOAPIC early just for system timer */
 void __init pre_init_apic_IRQ0(void)
 {
-- 
1.7.10.4

[PATCH v2 22/26] x86, irq: Make ioapics loop skip blank slots

[Yinghai Lu]

When multiple ioapics get added and removed, we could have blank slots
in middle of ioapics array.

Add skip code for this case by checking nr_registers.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
Cc: Paul Gortmaker <paul.gortmaker@windriver.com>
Cc: Joerg Roedel <joro@8bytes.org>
Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Cc: Sebastian Andrzej Siewior <sebastian@breakpoint.cc>
---
 arch/x86/kernel/apic/io_apic.c |   29 +++++++++++++++++++++++------
 1 file changed, 23 insertions(+), 6 deletions(-)

diff --git a/arch/x86/kernel/apic/io_apic.c b/arch/x86/kernel/apic/io_apic.c
index 4a9ccf6..d4c68d9 100644
--- a/arch/x86/kernel/apic/io_apic.c
+++ b/arch/x86/kernel/apic/io_apic.c
@@ -793,7 +793,8 @@ int save_ioapic_entries(void)
 	int err = 0;
 
 	for (apic = 0; apic < nr_ioapics; apic++) {
-		if (!ioapics[apic].saved_registers) {
+		if (!ioapics[apic].saved_registers &&
+		    ioapics[apic].nr_registers) {
 			err = -ENOMEM;
 			continue;
 		}
@@ -899,9 +900,12 @@ static int __init find_isa_irq_apic(int irq, int type)
 	if (i < mp_irq_entries) {
 		int ioapic_idx;
 
-		for (ioapic_idx = 0; ioapic_idx < nr_ioapics; ioapic_idx++)
+		for (ioapic_idx = 0; ioapic_idx < nr_ioapics; ioapic_idx++) {
+			if (!ioapics[ioapic_idx].nr_registers)
+				continue;
 			if (mpc_ioapic_id(ioapic_idx) == mp_irqs[i].dstapic)
 				return ioapic_idx;
+		}
 	}
 
 	return -1;
@@ -1121,10 +1125,13 @@ int IO_APIC_get_PCI_irq_vector(int bus, int slot, int pin,
 	for (i = 0; i < mp_irq_entries; i++) {
 		int lbus = mp_irqs[i].srcbus;
 
-		for (ioapic_idx = 0; ioapic_idx < nr_ioapics; ioapic_idx++)
+		for (ioapic_idx = 0; ioapic_idx < nr_ioapics; ioapic_idx++) {
+			if (!ioapics[ioapic_idx].nr_registers)
+				continue;
 			if (mpc_ioapic_id(ioapic_idx) == mp_irqs[i].dstapic ||
 			    mp_irqs[i].dstapic == MP_APIC_ALL)
 				break;
+		}
 
 		if (!test_bit(lbus, mp_bus_not_pci) &&
 		    !mp_irqs[i].irqtype &&
@@ -2084,6 +2091,9 @@ void __init setup_ioapic_ids_from_mpc_nocheck(void)
 	 * Set the IOAPIC ID to the value stored in the MPC table.
 	 */
 	for (ioapic_idx = 0; ioapic_idx < nr_ioapics; ioapic_idx++) {
+		if (!ioapics[ioapic_idx].nr_registers)
+			continue;
+
 		/* Read the register 0 value */
 		raw_spin_lock_irqsave(&ioapic_lock, flags);
 		reg_00.raw = io_apic_read(ioapic_idx, 0);
@@ -3013,8 +3023,12 @@ static void ioapic_resume(void)
 {
 	int ioapic_idx;
 
-	for (ioapic_idx = nr_ioapics - 1; ioapic_idx >= 0; ioapic_idx--)
+	for (ioapic_idx = nr_ioapics - 1; ioapic_idx >= 0; ioapic_idx--) {
+		if (!ioapics[ioapic_idx].nr_registers)
+			continue;
+
 		resume_ioapic_id(ioapic_idx);
+	}
 
 	restore_ioapic_entries();
 }
@@ -3854,8 +3868,11 @@ static int __mp_find_ioapic(u32 gsi, bool quiet)
 	/* Find the IOAPIC that manages this GSI. */
 	for (i = 0; i < nr_ioapics; i++) {
 		struct mp_ioapic_gsi *gsi_cfg = mp_ioapic_gsi_routing(i);
-		if ((gsi >= gsi_cfg->gsi_base)
-		    && (gsi <= gsi_cfg->gsi_end))
+
+		if (!ioapics[i].nr_registers)
+			continue;
+
+		if ((gsi >= gsi_cfg->gsi_base) && (gsi <= gsi_cfg->gsi_end))
 			return i;
 	}
 
-- 
1.7.10.4

[PATCH v2 23/26] x86, ioapic: Find usable ioapic id for 64bit.

[Yinghai Lu]

Checking the id in register, if that is duplicated, will pick one and
update that to register.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
Cc: Joerg Roedel <joro@8bytes.org>
Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Cc: Sebastian Andrzej Siewior <sebastian@breakpoint.cc>
---
 arch/x86/kernel/apic/io_apic.c |   41 +++++++++++++++++++++++++++++++++-------
 1 file changed, 34 insertions(+), 7 deletions(-)

diff --git a/arch/x86/kernel/apic/io_apic.c b/arch/x86/kernel/apic/io_apic.c
index d4c68d9..42f273b 100644
--- a/arch/x86/kernel/apic/io_apic.c
+++ b/arch/x86/kernel/apic/io_apic.c
@@ -3663,27 +3663,54 @@ static int __init io_apic_get_unique_id(int ioapic, int apic_id)
 	return apic_id;
 }
 
-static u8 __init io_apic_unique_id(u8 id)
+static u8 __init io_apic_unique_id(int idx, u8 id)
 {
 	if ((boot_cpu_data.x86_vendor == X86_VENDOR_INTEL) &&
 	    !APIC_XAPIC(apic_version[boot_cpu_physical_apicid]))
-		return io_apic_get_unique_id(nr_ioapics, id);
+		return io_apic_get_unique_id(idx, id);
 	else
 		return id;
 }
 #else
-static u8 __init io_apic_unique_id(u8 id)
+static u8 io_apic_unique_id(int idx, u8 id)
 {
 	int i;
+	u8 new_id;
+	unsigned long flags;
 	DECLARE_BITMAP(used, 256);
+	union IO_APIC_reg_00 reg_00;
 
 	bitmap_zero(used, 256);
-	for (i = 0; i < nr_ioapics; i++) {
+	for (i = 0; i < nr_ioapics; i++)
 		__set_bit(mpc_ioapic_id(i), used);
-	}
 	if (!test_bit(id, used))
 		return id;
-	return find_first_zero_bit(used, 256);
+
+	/* check register at first */
+	raw_spin_lock_irqsave(&ioapic_lock, flags);
+	reg_00.raw = io_apic_read(idx, 0);
+	raw_spin_unlock_irqrestore(&ioapic_lock, flags);
+	new_id = reg_00.bits.ID;
+	if (!test_bit(new_id, used)) {
+		apic_printk(APIC_VERBOSE, KERN_INFO
+			"IOAPIC[%d]: Using reg apic_id %d instead of %d\n",
+			 idx, new_id, id);
+		return new_id;
+	}
+
+	new_id = find_first_zero_bit(used, 256);
+	reg_00.bits.ID = new_id;
+	raw_spin_lock_irqsave(&ioapic_lock, flags);
+	io_apic_write(idx, 0, reg_00.raw);
+	reg_00.raw = io_apic_read(idx, 0);
+	raw_spin_unlock_irqrestore(&ioapic_lock, flags);
+
+	/* Sanity check */
+	if (reg_00.bits.ID != new_id)
+		pr_warn("IOAPIC[%d]: Unable to change apic_id to %d!\n",
+			idx, new_id);
+
+	return new_id;
 }
 #endif
 
@@ -3964,7 +3991,7 @@ int __mp_register_ioapic(int id, u32 address, u32 gsi_base, bool hotadd)
 	if (bad_ioapic_register(idx))
 		goto failed;
 
-	ioapics[idx].mp_config.apicid = io_apic_unique_id(id);
+	ioapics[idx].mp_config.apicid = io_apic_unique_id(idx, id);
 	ioapics[idx].mp_config.apicver = io_apic_get_version(idx);
 
 	/*
-- 
1.7.10.4

[PATCH v2 24/26] x86: Move declaration for mp_register_ioapic()

[Yinghai Lu]

Address compiling problem that Fengguang report.

Reported-by: Fengguang Wu <wfg@linux.intel.com>
Signed-off-by: Yinghai Lu <yinghai@kernel.org>
---
 arch/x86/include/asm/mpspec.h |   19 ++++++++++++++++++-
 1 file changed, 18 insertions(+), 1 deletion(-)

diff --git a/arch/x86/include/asm/mpspec.h b/arch/x86/include/asm/mpspec.h
index e70b7e5..13439c5 100644
--- a/arch/x86/include/asm/mpspec.h
+++ b/arch/x86/include/asm/mpspec.h
@@ -95,10 +95,27 @@ static inline void early_reserve_e820_mpc_new(void) { }
 #endif
 
 void __cpuinit generic_processor_info(int apicid, int version);
-#ifdef CONFIG_ACPI
+
+#ifdef CONFIG_X86_IO_APIC
 int __mp_register_ioapic(int id, u32 address, u32 gsi_base, bool hot);
 int mp_unregister_ioapic(u32 gsi_base);
 extern void mp_register_ioapic(int id, u32 address, u32 gsi_base);
+#else
+static inline int __mp_register_ioapic(int id, u32 address, u32 gsi_base,
+					 bool hot)
+{
+	return 0;
+}
+static inline int mp_unregister_ioapic(u32 gsi_base)
+{
+	return 0;
+}
+static inline void mp_register_ioapic(int id, u32 address, u32 gsi_base)
+{
+}
+#endif
+
+#ifdef CONFIG_ACPI
 extern void mp_override_legacy_irq(u8 bus_irq, u8 polarity, u8 trigger,
 				   u32 gsi);
 extern void mp_config_acpi_legacy_irqs(void);
-- 
1.7.10.4

[PATCH v2 25/26] PCI: Disable mem in the ioapic removing path

[Yinghai Lu]

For physical hot plug should be ok, but for remove/rescan path will
need us to disable that.

otherwise rescan mmio resource for pci ioapic device will not be
sized and allocated, aka skiped.
For ioapic_probe:pci_enable_device will not enable the device
correctly, and will bail out early.

So we can just disable mmio for all removing case, and that will not hurt
real hotplug path.

Signed-off-by: <yinghai@kernel.org>
---
 drivers/pci/ioapic.c |   13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/drivers/pci/ioapic.c b/drivers/pci/ioapic.c
index 3c6bbdd..8dacfd0 100644
--- a/drivers/pci/ioapic.c
+++ b/drivers/pci/ioapic.c
@@ -88,6 +88,17 @@ exit_free:
 	return -ENODEV;
 }
 
+static void pci_disable_device_mem(struct pci_dev *dev)
+{
+	u16 pci_command;
+
+	pci_read_config_word(dev, PCI_COMMAND, &pci_command);
+	if (pci_command & PCI_COMMAND_MEMORY) {
+		pci_command &= ~PCI_COMMAND_MEMORY;
+		pci_write_config_word(dev, PCI_COMMAND, pci_command);
+	}
+}
+
 static void ioapic_remove(struct pci_dev *dev)
 {
 	struct ioapic *ioapic = pci_get_drvdata(dev);
@@ -95,6 +106,8 @@ static void ioapic_remove(struct pci_dev *dev)
 	acpi_unregister_ioapic(ioapic->handle, ioapic->gsi_base);
 	pci_release_region(dev, 0);
 	pci_disable_device(dev);
+	/* need to disable it, otherwise remove/rescan will not work */
+	pci_disable_device_mem(dev);
 	kfree(ioapic);
 }
 
-- 
1.7.10.4

Re: [PATCH v2 25/26] PCI: Disable mem in the ioapic removing path

[Bjorn Helgaas]

On Fri, Feb 8, 2013 at 12:28 PM, Yinghai Lu <yinghai@kernel.org> wrote:
> For physical hot plug should be ok, but for remove/rescan path will
> need us to disable that.
>
> otherwise rescan mmio resource for pci ioapic device will not be
> sized and allocated, aka skiped.

When we scan other PCI devices, we can size memory BARs even if
PCI_COMMAND_MEMORY is already set.  So there must be something
different about IOAPICs?  Or maybe it's something different about
rescan vs. the initial scan?

> For ioapic_probe:pci_enable_device will not enable the device
> correctly, and will bail out early.

Exactly where and why do we bail out early?  The only early bail out I
see is where __pci_enable_device_flags() returns if "dev->enable_cnt >
1".  Is that what you mean?

> So we can just disable mmio for all removing case, and that will not hurt
> real hotplug path.

> Signed-off-by: <yinghai@kernel.org>
> ---
>  drivers/pci/ioapic.c |   13 +++++++++++++
>  1 file changed, 13 insertions(+)
>
> diff --git a/drivers/pci/ioapic.c b/drivers/pci/ioapic.c
> index 3c6bbdd..8dacfd0 100644
> --- a/drivers/pci/ioapic.c
> +++ b/drivers/pci/ioapic.c
> @@ -88,6 +88,17 @@ exit_free:
>         return -ENODEV;
>  }
>
> +static void pci_disable_device_mem(struct pci_dev *dev)
> +{
> +       u16 pci_command;
> +
> +       pci_read_config_word(dev, PCI_COMMAND, &pci_command);
> +       if (pci_command & PCI_COMMAND_MEMORY) {
> +               pci_command &= ~PCI_COMMAND_MEMORY;
> +               pci_write_config_word(dev, PCI_COMMAND, pci_command);
> +       }
> +}
> +
>  static void ioapic_remove(struct pci_dev *dev)
>  {
>         struct ioapic *ioapic = pci_get_drvdata(dev);
> @@ -95,6 +106,8 @@ static void ioapic_remove(struct pci_dev *dev)
>         acpi_unregister_ioapic(ioapic->handle, ioapic->gsi_base);
>         pci_release_region(dev, 0);
>         pci_disable_device(dev);
> +       /* need to disable it, otherwise remove/rescan will not work */
> +       pci_disable_device_mem(dev);
>         kfree(ioapic);
>  }
>
> --
> 1.7.10.4
>

Re: [PATCH v2 25/26] PCI: Disable mem in the ioapic removing path

[Yinghai Lu]

On Fri, Feb 8, 2013 at 1:14 PM, Bjorn Helgaas <bhelgaas@google.com> wrote:
> On Fri, Feb 8, 2013 at 12:28 PM, Yinghai Lu <yinghai@kernel.org> wrote:
>> For physical hot plug should be ok, but for remove/rescan path will
>> need us to disable that.
>>
>> otherwise rescan mmio resource for pci ioapic device will not be
>> sized and allocated, aka skiped.
>
> When we scan other PCI devices, we can size memory BARs even if
> PCI_COMMAND_MEMORY is already set.  So there must be something
> different about IOAPICs?  Or maybe it's something different about
> rescan vs. the initial scan?

it is in drivers/pci/setup-bus.c::__dev_sort_resources()
it will skip the reallocation for ioapic controller.

...
        /* Don't touch ioapic devices already enabled by firmware */
        if (class == PCI_CLASS_SYSTEM_PIC) {
                u16 command;
                pci_read_config_word(dev, PCI_COMMAND, &command);
                if (command & (PCI_COMMAND_IO | PCI_COMMAND_MEMORY))
                        return;
        }

>
>> For ioapic_probe:pci_enable_device will not enable the device
>> correctly, and will bail out early.
>
> Exactly where and why do we bail out early?  The only early bail out I
> see is where __pci_enable_device_flags() returns if "dev->enable_cnt >
> 1".  Is that what you mean?

pci_enable_device==>pci_enable_device_flags
==>do_pci_enable_device==>pcibios_enable_device==>pci_enable_resources
...
                if (!r->parent) {
                        dev_err(&dev->dev, "device not available "
                                "(can't reserve %pR)\n", r);
                        return -EINVAL;
                }
...

only reassign one will have get it's parent.

Thanks

Yinghai

Re: [PATCH v2 25/26] PCI: Disable mem in the ioapic removing path

[Bjorn Helgaas]

On Fri, Feb 8, 2013 at 3:33 PM, Yinghai Lu <yinghai@kernel.org> wrote:
> On Fri, Feb 8, 2013 at 1:14 PM, Bjorn Helgaas <bhelgaas@google.com> wrote:
>> On Fri, Feb 8, 2013 at 12:28 PM, Yinghai Lu <yinghai@kernel.org> wrote:
>>> For physical hot plug should be ok, but for remove/rescan path will
>>> need us to disable that.
>>>
>>> otherwise rescan mmio resource for pci ioapic device will not be
>>> sized and allocated, aka skiped.
>>
>> When we scan other PCI devices, we can size memory BARs even if
>> PCI_COMMAND_MEMORY is already set.  So there must be something
>> different about IOAPICs?  Or maybe it's something different about
>> rescan vs. the initial scan?
>
> it is in drivers/pci/setup-bus.c::__dev_sort_resources()
> it will skip the reallocation for ioapic controller.
>
> ...
>         /* Don't touch ioapic devices already enabled by firmware */
>         if (class == PCI_CLASS_SYSTEM_PIC) {
>                 u16 command;
>                 pci_read_config_word(dev, PCI_COMMAND, &command);
>                 if (command & (PCI_COMMAND_IO | PCI_COMMAND_MEMORY))
>                         return;
>         }
>
>>
>>> For ioapic_probe:pci_enable_device will not enable the device
>>> correctly, and will bail out early.
>>
>> Exactly where and why do we bail out early?  The only early bail out I
>> see is where __pci_enable_device_flags() returns if "dev->enable_cnt >
>> 1".  Is that what you mean?
>
> pci_enable_device==>pci_enable_device_flags
> ==>do_pci_enable_device==>pcibios_enable_device==>pci_enable_resources
> ...
>                 if (!r->parent) {
>                         dev_err(&dev->dev, "device not available "
>                                 "(can't reserve %pR)\n", r);
>                         return -EINVAL;
>                 }
> ...
>
> only reassign one will have get it's parent.

Hmmm, OK.  So basically this patch is a hack to work around previous
hacks elsewhere.

We are like flies in a spider's web, bound tighter and tighter by
every loop of silk.

[PATCH v2 26/26] PCI, x86, ACPI: Add ioapic hotplug support with acpi host bridge.

[Yinghai Lu]

We need to have ioapic setup before normal pci drivers.
otherwise other pci driver can not setup irq.

Make ioapic built-in, so can call add/remove during host-bridge add/remove
the same as the booting path.

Also need to make it depends on X86_IO_APIC.

Signed-off-by: <yinghai@kernel.org>
---
 arch/x86/kernel/acpi/boot.c |   10 +-
 drivers/acpi/pci_root.c     |    4 +
 drivers/pci/Kconfig         |    3 +-
 drivers/pci/ioapic.c        |  216 ++++++++++++++++++++++++++++++-------------
 include/linux/pci-acpi.h    |    8 ++
 5 files changed, 172 insertions(+), 69 deletions(-)

diff --git a/arch/x86/kernel/acpi/boot.c b/arch/x86/kernel/acpi/boot.c
index 350879f..338163b 100644
--- a/arch/x86/kernel/acpi/boot.c
+++ b/arch/x86/kernel/acpi/boot.c
@@ -697,16 +697,18 @@ EXPORT_SYMBOL(acpi_unmap_lsapic);
 
 int acpi_register_ioapic(acpi_handle handle, u64 phys_addr, u32 gsi_base)
 {
-	/* TBD */
-	return -EINVAL;
+	unsigned long long id = 0;
+
+	acpi_evaluate_integer(handle, "_UID", NULL, &id);
+
+	return	__mp_register_ioapic(id, phys_addr, gsi_base, true);
 }
 
 EXPORT_SYMBOL(acpi_register_ioapic);
 
 int acpi_unregister_ioapic(acpi_handle handle, u32 gsi_base)
 {
-	/* TBD */
-	return -EINVAL;
+	return mp_unregister_ioapic(gsi_base);
 }
 
 EXPORT_SYMBOL(acpi_unregister_ioapic);
diff --git a/drivers/acpi/pci_root.c b/drivers/acpi/pci_root.c
index 417487a..54d61ce 100644
--- a/drivers/acpi/pci_root.c
+++ b/drivers/acpi/pci_root.c
@@ -587,6 +587,8 @@ static int acpi_pci_root_add(struct acpi_device *device)
 		pci_assign_unassigned_bus_resources(root->bus);
 	}
 
+	acpi_pci_ioapic_add(root);
+
 	mutex_lock(&acpi_pci_root_lock);
 	list_for_each_entry(driver, &acpi_pci_drivers, node)
 		if (driver->add)
@@ -626,6 +628,8 @@ static int acpi_pci_root_remove(struct acpi_device *device, int type)
 			driver->remove(root);
 	mutex_unlock(&acpi_pci_root_lock);
 
+	acpi_pci_ioapic_remove(root);
+
 	device_set_run_wake(root->bus->bridge, false);
 	pci_acpi_remove_bus_pm_notifier(device);
 
diff --git a/drivers/pci/Kconfig b/drivers/pci/Kconfig
index 6d51aa6..720989f 100644
--- a/drivers/pci/Kconfig
+++ b/drivers/pci/Kconfig
@@ -110,10 +110,11 @@ config PCI_PASID
 	  If unsure, say N.
 
 config PCI_IOAPIC
-	tristate "PCI IO-APIC hotplug support" if X86
+	bool "PCI IO-APIC hotplug support" if X86
 	depends on PCI
 	depends on ACPI
 	depends on HOTPLUG
+	depends on X86_IO_APIC
 	default !X86
 
 config PCI_LABEL
diff --git a/drivers/pci/ioapic.c b/drivers/pci/ioapic.c
index 8dacfd0..12e56c7 100644
--- a/drivers/pci/ioapic.c
+++ b/drivers/pci/ioapic.c
@@ -22,70 +22,128 @@
 #include <linux/slab.h>
 #include <acpi/acpi_bus.h>
 
-struct ioapic {
-	acpi_handle	handle;
+struct acpi_pci_ioapic {
+	acpi_handle	root_handle;
 	u32		gsi_base;
+	struct pci_dev *pdev;
+	struct list_head list;
 };
 
-static int ioapic_probe(struct pci_dev *dev, const struct pci_device_id *ent)
+static LIST_HEAD(ioapic_list);
+static DEFINE_MUTEX(ioapic_list_lock);
+
+static acpi_status setup_res(struct acpi_resource *acpi_res, void *data)
+{
+	struct resource *res;
+	struct acpi_resource_address64 addr;
+	acpi_status status;
+	unsigned long flags;
+	u64 start, end;
+
+	status = acpi_resource_to_address64(acpi_res, &addr);
+	if (!ACPI_SUCCESS(status))
+		return AE_OK;
+
+	if (addr.resource_type == ACPI_MEMORY_RANGE) {
+		if (addr.info.mem.caching == ACPI_PREFETCHABLE_MEMORY)
+			return AE_OK;
+		flags = IORESOURCE_MEM;
+	} else
+		return AE_OK;
+
+	start = addr.minimum + addr.translation_offset;
+	end = addr.maximum + addr.translation_offset;
+
+	res = data;
+	res->flags = flags;
+	res->start = start;
+	res->end = end;
+
+	return AE_OK;
+}
+
+static void handle_ioapic_add(acpi_handle handle, struct pci_dev **pdev,
+				 u32 *pgsi_base)
 {
-	acpi_handle handle;
 	acpi_status status;
 	unsigned long long gsb;
-	struct ioapic *ioapic;
+	struct pci_dev *dev;
+	u32 gsi_base;
 	int ret;
 	char *type;
-	struct resource *res;
+	struct resource r;
+	struct resource *res = &r;
+	char objname[64];
+	struct acpi_buffer buffer = {sizeof(objname), objname};
 
-	handle = DEVICE_ACPI_HANDLE(&dev->dev);
-	if (!handle)
-		return -EINVAL;
+	*pdev = NULL;
+	*pgsi_base = 0;
 
 	status = acpi_evaluate_integer(handle, "_GSB", NULL, &gsb);
-	if (ACPI_FAILURE(status))
-		return -EINVAL;
+	if (ACPI_FAILURE(status) || !gsb)
+		return;
+
+	dev = acpi_get_pci_dev(handle);
+	if (!dev) {
+		struct acpi_device_info *info;
+		char *hid = NULL;
+
+		status = acpi_get_object_info(handle, &info);
+		if (ACPI_FAILURE(status))
+			return;
+		if (info->valid & ACPI_VALID_HID)
+			hid = info->hardware_id.string;
+		if (!hid || strcmp(hid, "ACPI0009")) {
+			kfree(info);
+			return;
+		}
+		kfree(info);
+		memset(res, 0, sizeof(*res));
+		acpi_walk_resources(handle, METHOD_NAME__CRS, setup_res, res);
+		if (!res->flags)
+			return;
+	}
 
-	/*
-	 * The previous code in acpiphp evaluated _MAT if _GSB failed, but
-	 * ACPI spec 4.0 sec 6.2.2 requires _GSB for hot-pluggable I/O APICs.
-	 */
+	acpi_get_name(handle, ACPI_FULL_PATHNAME, &buffer);
 
-	ioapic = kzalloc(sizeof(*ioapic), GFP_KERNEL);
-	if (!ioapic)
-		return -ENOMEM;
+	gsi_base = gsb;
+	type = "IOxAPIC";
+	if (dev) {
+		ret = pci_enable_device(dev);
+		if (ret < 0)
+			goto exit_put;
 
-	ioapic->handle = handle;
-	ioapic->gsi_base = (u32) gsb;
+		pci_set_master(dev);
 
-	if (dev->class == PCI_CLASS_SYSTEM_PIC_IOAPIC)
-		type = "IOAPIC";
-	else
-		type = "IOxAPIC";
+		if (dev->class == PCI_CLASS_SYSTEM_PIC_IOAPIC)
+			type = "IOAPIC";
 
-	ret = pci_enable_device(dev);
-	if (ret < 0)
-		goto exit_free;
+		if (pci_request_region(dev, 0, type))
+			goto exit_disable;
 
-	pci_set_master(dev);
+		res = &dev->resource[0];
+	}
 
-	if (pci_request_region(dev, 0, type))
-		goto exit_disable;
+	if (acpi_register_ioapic(handle, res->start, gsi_base)) {
+		if (dev)
+			goto exit_release;
+		return;
+	}
 
-	res = &dev->resource[0];
-	if (acpi_register_ioapic(ioapic->handle, res->start, ioapic->gsi_base))
-		goto exit_release;
+	pr_info("%s %s %s at %pR, GSI %u\n",
+		dev ? dev_name(&dev->dev) : "", objname, type,
+		res, gsi_base);
 
-	pci_set_drvdata(dev, ioapic);
-	dev_info(&dev->dev, "%s at %pR, GSI %u\n", type, res, ioapic->gsi_base);
-	return 0;
+	*pdev = dev;
+	*pgsi_base = gsi_base;
+	return;
 
 exit_release:
 	pci_release_region(dev, 0);
 exit_disable:
 	pci_disable_device(dev);
-exit_free:
-	kfree(ioapic);
-	return -ENODEV;
+exit_put:
+	pci_dev_put(dev);
 }
 
 static void pci_disable_device_mem(struct pci_dev *dev)
@@ -99,44 +157,74 @@ static void pci_disable_device_mem(struct pci_dev *dev)
 	}
 }
 
-static void ioapic_remove(struct pci_dev *dev)
+static void handle_ioapic_remove(acpi_handle handle, struct pci_dev *dev,
+				  u32 gsi_base)
 {
-	struct ioapic *ioapic = pci_get_drvdata(dev);
+	acpi_unregister_ioapic(handle, gsi_base);
+
+	if (!dev)
+		return;
 
-	acpi_unregister_ioapic(ioapic->handle, ioapic->gsi_base);
 	pci_release_region(dev, 0);
 	pci_disable_device(dev);
 	/* need to disable it, otherwise remove/rescan will not work */
 	pci_disable_device_mem(dev);
-	kfree(ioapic);
+	pci_dev_put(dev);
 }
 
+static acpi_status register_ioapic(acpi_handle handle, u32 lvl,
+					void *context, void **rv)
+{
+	acpi_handle root_handle = context;
+	struct pci_dev *pdev;
+	u32 gsi_base;
+	struct acpi_pci_ioapic *ioapic;
 
-static DEFINE_PCI_DEVICE_TABLE(ioapic_devices) = {
-	{ PCI_DEVICE_CLASS(PCI_CLASS_SYSTEM_PIC_IOAPIC, ~0) },
-	{ PCI_DEVICE_CLASS(PCI_CLASS_SYSTEM_PIC_IOXAPIC, ~0) },
-	{ }
-};
-MODULE_DEVICE_TABLE(pci, ioapic_devices);
+	handle_ioapic_add(handle, &pdev, &gsi_base);
+	if (!gsi_base)
+		return AE_OK;
 
-static struct pci_driver ioapic_driver = {
-	.name		= "ioapic",
-	.id_table	= ioapic_devices,
-	.probe		= ioapic_probe,
-	.remove		= ioapic_remove,
-};
+	ioapic = kzalloc(sizeof(*ioapic), GFP_KERNEL);
+	if (!ioapic) {
+		pr_err("%s: cannot allocate memory\n", __func__);
+		handle_ioapic_remove(root_handle, pdev, gsi_base);
+		return AE_OK;
+	}
+	ioapic->root_handle = root_handle;
+	ioapic->pdev = pdev;
+	ioapic->gsi_base = gsi_base;
 
-static int __init ioapic_init(void)
-{
-	return pci_register_driver(&ioapic_driver);
+	mutex_lock(&ioapic_list_lock);
+	list_add(&ioapic->list, &ioapic_list);
+	mutex_unlock(&ioapic_list_lock);
+
+	return AE_OK;
 }
 
-static void __exit ioapic_exit(void)
+void acpi_pci_ioapic_add(struct acpi_pci_root *root)
 {
-	pci_unregister_driver(&ioapic_driver);
-}
+	acpi_status status;
 
-module_init(ioapic_init);
-module_exit(ioapic_exit);
+	status = acpi_walk_namespace(ACPI_TYPE_DEVICE, root->device->handle,
+				     (u32)1, register_ioapic, NULL,
+				     root->device->handle,
+				     NULL);
+	if (ACPI_FAILURE(status))
+		pr_err("%s: register_ioapic failure - %d", __func__, status);
+}
 
-MODULE_LICENSE("GPL");
+void acpi_pci_ioapic_remove(struct acpi_pci_root *root)
+{
+	struct acpi_pci_ioapic *ioapic, *tmp;
+
+	mutex_lock(&ioapic_list_lock);
+	list_for_each_entry_safe(ioapic, tmp, &ioapic_list, list) {
+		if (root->device->handle != ioapic->root_handle)
+			continue;
+		list_del(&ioapic->list);
+		handle_ioapic_remove(ioapic->root_handle, ioapic->pdev,
+					ioapic->gsi_base);
+		kfree(ioapic);
+	}
+	mutex_unlock(&ioapic_list_lock);
+}
diff --git a/include/linux/pci-acpi.h b/include/linux/pci-acpi.h
index 9a22b5e..6f83039 100644
--- a/include/linux/pci-acpi.h
+++ b/include/linux/pci-acpi.h
@@ -49,4 +49,12 @@ extern bool aer_acpi_firmware_first(void);
 static inline bool aer_acpi_firmware_first(void) { return false; }
 #endif
 
+#ifdef CONFIG_PCI_IOAPIC
+void acpi_pci_ioapic_add(struct acpi_pci_root *root);
+void acpi_pci_ioapic_remove(struct acpi_pci_root *root);
+#else
+static inline void acpi_pci_ioapic_add(struct acpi_pci_root *root) { }
+static inline void acpi_pci_ioapic_remove(struct acpi_pci_root *root) { }
+#endif
+
 #endif	/* _PCI_ACPI_H_ */
-- 
1.7.10.4

Re: [PATCH v2 26/26] PCI, x86, ACPI: Add ioapic hotplug support with acpi host bridge.

[Bjorn Helgaas]

On Fri, Feb 8, 2013 at 12:28 PM, Yinghai Lu <yinghai@kernel.org> wrote:
> We need to have ioapic setup before normal pci drivers.
> otherwise other pci driver can not setup irq.
>
> Make ioapic built-in, so can call add/remove during host-bridge add/remove
> the same as the booting path.
>
> Also need to make it depends on X86_IO_APIC.

There are a lot of different things happening in this patch.  I'd
prefer if they were split out so we can see what's happening.  For
example, the Kconfig changes could be their own patch.  The removal of
the PCI device table is worthy of mention and its own patch (if
possible), since it's a pretty major change to how the driver binds to
devices.

> Signed-off-by: <yinghai@kernel.org>
> ---
>  arch/x86/kernel/acpi/boot.c |   10 +-
>  drivers/acpi/pci_root.c     |    4 +
>  drivers/pci/Kconfig         |    3 +-
>  drivers/pci/ioapic.c        |  216 ++++++++++++++++++++++++++++++-------------
>  include/linux/pci-acpi.h    |    8 ++
>  5 files changed, 172 insertions(+), 69 deletions(-)
>
> diff --git a/arch/x86/kernel/acpi/boot.c b/arch/x86/kernel/acpi/boot.c
> index 350879f..338163b 100644
> --- a/arch/x86/kernel/acpi/boot.c
> +++ b/arch/x86/kernel/acpi/boot.c
> @@ -697,16 +697,18 @@ EXPORT_SYMBOL(acpi_unmap_lsapic);
>
>  int acpi_register_ioapic(acpi_handle handle, u64 phys_addr, u32 gsi_base)
>  {
> -       /* TBD */
> -       return -EINVAL;
> +       unsigned long long id = 0;
> +
> +       acpi_evaluate_integer(handle, "_UID", NULL, &id);
> +
> +       return  __mp_register_ioapic(id, phys_addr, gsi_base, true);
>  }
>
>  EXPORT_SYMBOL(acpi_register_ioapic);
>
>  int acpi_unregister_ioapic(acpi_handle handle, u32 gsi_base)
>  {
> -       /* TBD */
> -       return -EINVAL;
> +       return mp_unregister_ioapic(gsi_base);
>  }
>
>  EXPORT_SYMBOL(acpi_unregister_ioapic);
> diff --git a/drivers/acpi/pci_root.c b/drivers/acpi/pci_root.c
> index 417487a..54d61ce 100644
> --- a/drivers/acpi/pci_root.c
> +++ b/drivers/acpi/pci_root.c
> @@ -587,6 +587,8 @@ static int acpi_pci_root_add(struct acpi_device *device)
>                 pci_assign_unassigned_bus_resources(root->bus);
>         }
>
> +       acpi_pci_ioapic_add(root);
> +
>         mutex_lock(&acpi_pci_root_lock);
>         list_for_each_entry(driver, &acpi_pci_drivers, node)
>                 if (driver->add)
> @@ -626,6 +628,8 @@ static int acpi_pci_root_remove(struct acpi_device *device, int type)
>                         driver->remove(root);
>         mutex_unlock(&acpi_pci_root_lock);
>
> +       acpi_pci_ioapic_remove(root);
> +
>         device_set_run_wake(root->bus->bridge, false);
>         pci_acpi_remove_bus_pm_notifier(device);
>
> diff --git a/drivers/pci/Kconfig b/drivers/pci/Kconfig
> index 6d51aa6..720989f 100644
> --- a/drivers/pci/Kconfig
> +++ b/drivers/pci/Kconfig
> @@ -110,10 +110,11 @@ config PCI_PASID
>           If unsure, say N.
>
>  config PCI_IOAPIC
> -       tristate "PCI IO-APIC hotplug support" if X86
> +       bool "PCI IO-APIC hotplug support" if X86
>         depends on PCI
>         depends on ACPI
>         depends on HOTPLUG
> +       depends on X86_IO_APIC
>         default !X86
>
>  config PCI_LABEL
> diff --git a/drivers/pci/ioapic.c b/drivers/pci/ioapic.c
> index 8dacfd0..12e56c7 100644
> --- a/drivers/pci/ioapic.c
> +++ b/drivers/pci/ioapic.c
> @@ -22,70 +22,128 @@
>  #include <linux/slab.h>
>  #include <acpi/acpi_bus.h>
>
> -struct ioapic {
> -       acpi_handle     handle;
> +struct acpi_pci_ioapic {
> +       acpi_handle     root_handle;
>         u32             gsi_base;
> +       struct pci_dev *pdev;
> +       struct list_head list;
>  };
>
> -static int ioapic_probe(struct pci_dev *dev, const struct pci_device_id *ent)
> +static LIST_HEAD(ioapic_list);
> +static DEFINE_MUTEX(ioapic_list_lock);
> +
> +static acpi_status setup_res(struct acpi_resource *acpi_res, void *data)
> +{
> +       struct resource *res;
> +       struct acpi_resource_address64 addr;
> +       acpi_status status;
> +       unsigned long flags;
> +       u64 start, end;
> +
> +       status = acpi_resource_to_address64(acpi_res, &addr);
> +       if (!ACPI_SUCCESS(status))
> +               return AE_OK;
> +
> +       if (addr.resource_type == ACPI_MEMORY_RANGE) {
> +               if (addr.info.mem.caching == ACPI_PREFETCHABLE_MEMORY)
> +                       return AE_OK;
> +               flags = IORESOURCE_MEM;
> +       } else
> +               return AE_OK;
> +
> +       start = addr.minimum + addr.translation_offset;
> +       end = addr.maximum + addr.translation_offset;
> +
> +       res = data;
> +       res->flags = flags;
> +       res->start = start;
> +       res->end = end;
> +
> +       return AE_OK;
> +}
> +
> +static void handle_ioapic_add(acpi_handle handle, struct pci_dev **pdev,
> +                                u32 *pgsi_base)
>  {
> -       acpi_handle handle;
>         acpi_status status;
>         unsigned long long gsb;
> -       struct ioapic *ioapic;
> +       struct pci_dev *dev;
> +       u32 gsi_base;
>         int ret;
>         char *type;
> -       struct resource *res;
> +       struct resource r;
> +       struct resource *res = &r;
> +       char objname[64];
> +       struct acpi_buffer buffer = {sizeof(objname), objname};
>
> -       handle = DEVICE_ACPI_HANDLE(&dev->dev);
> -       if (!handle)
> -               return -EINVAL;
> +       *pdev = NULL;
> +       *pgsi_base = 0;
>
>         status = acpi_evaluate_integer(handle, "_GSB", NULL, &gsb);
> -       if (ACPI_FAILURE(status))
> -               return -EINVAL;
> +       if (ACPI_FAILURE(status) || !gsb)
> +               return;
> +
> +       dev = acpi_get_pci_dev(handle);
> +       if (!dev) {
> +               struct acpi_device_info *info;
> +               char *hid = NULL;
> +
> +               status = acpi_get_object_info(handle, &info);
> +               if (ACPI_FAILURE(status))
> +                       return;
> +               if (info->valid & ACPI_VALID_HID)
> +                       hid = info->hardware_id.string;
> +               if (!hid || strcmp(hid, "ACPI0009")) {
> +                       kfree(info);
> +                       return;
> +               }
> +               kfree(info);
> +               memset(res, 0, sizeof(*res));
> +               acpi_walk_resources(handle, METHOD_NAME__CRS, setup_res, res);
> +               if (!res->flags)
> +                       return;
> +       }
>
> -       /*
> -        * The previous code in acpiphp evaluated _MAT if _GSB failed, but
> -        * ACPI spec 4.0 sec 6.2.2 requires _GSB for hot-pluggable I/O APICs.
> -        */
> +       acpi_get_name(handle, ACPI_FULL_PATHNAME, &buffer);
>
> -       ioapic = kzalloc(sizeof(*ioapic), GFP_KERNEL);
> -       if (!ioapic)
> -               return -ENOMEM;
> +       gsi_base = gsb;
> +       type = "IOxAPIC";
> +       if (dev) {
> +               ret = pci_enable_device(dev);
> +               if (ret < 0)
> +                       goto exit_put;
>
> -       ioapic->handle = handle;
> -       ioapic->gsi_base = (u32) gsb;
> +               pci_set_master(dev);
>
> -       if (dev->class == PCI_CLASS_SYSTEM_PIC_IOAPIC)
> -               type = "IOAPIC";
> -       else
> -               type = "IOxAPIC";
> +               if (dev->class == PCI_CLASS_SYSTEM_PIC_IOAPIC)
> +                       type = "IOAPIC";
>
> -       ret = pci_enable_device(dev);
> -       if (ret < 0)
> -               goto exit_free;
> +               if (pci_request_region(dev, 0, type))
> +                       goto exit_disable;
>
> -       pci_set_master(dev);
> +               res = &dev->resource[0];
> +       }
>
> -       if (pci_request_region(dev, 0, type))
> -               goto exit_disable;
> +       if (acpi_register_ioapic(handle, res->start, gsi_base)) {
> +               if (dev)
> +                       goto exit_release;
> +               return;
> +       }
>
> -       res = &dev->resource[0];
> -       if (acpi_register_ioapic(ioapic->handle, res->start, ioapic->gsi_base))
> -               goto exit_release;
> +       pr_info("%s %s %s at %pR, GSI %u\n",
> +               dev ? dev_name(&dev->dev) : "", objname, type,
> +               res, gsi_base);
>
> -       pci_set_drvdata(dev, ioapic);
> -       dev_info(&dev->dev, "%s at %pR, GSI %u\n", type, res, ioapic->gsi_base);
> -       return 0;
> +       *pdev = dev;
> +       *pgsi_base = gsi_base;
> +       return;
>
>  exit_release:
>         pci_release_region(dev, 0);
>  exit_disable:
>         pci_disable_device(dev);
> -exit_free:
> -       kfree(ioapic);
> -       return -ENODEV;
> +exit_put:
> +       pci_dev_put(dev);
>  }
>
>  static void pci_disable_device_mem(struct pci_dev *dev)
> @@ -99,44 +157,74 @@ static void pci_disable_device_mem(struct pci_dev *dev)
>         }
>  }
>
> -static void ioapic_remove(struct pci_dev *dev)
> +static void handle_ioapic_remove(acpi_handle handle, struct pci_dev *dev,
> +                                 u32 gsi_base)
>  {
> -       struct ioapic *ioapic = pci_get_drvdata(dev);
> +       acpi_unregister_ioapic(handle, gsi_base);
> +
> +       if (!dev)
> +               return;
>
> -       acpi_unregister_ioapic(ioapic->handle, ioapic->gsi_base);
>         pci_release_region(dev, 0);
>         pci_disable_device(dev);
>         /* need to disable it, otherwise remove/rescan will not work */
>         pci_disable_device_mem(dev);
> -       kfree(ioapic);
> +       pci_dev_put(dev);
>  }
>
> +static acpi_status register_ioapic(acpi_handle handle, u32 lvl,
> +                                       void *context, void **rv)
> +{
> +       acpi_handle root_handle = context;
> +       struct pci_dev *pdev;
> +       u32 gsi_base;
> +       struct acpi_pci_ioapic *ioapic;
>
> -static DEFINE_PCI_DEVICE_TABLE(ioapic_devices) = {
> -       { PCI_DEVICE_CLASS(PCI_CLASS_SYSTEM_PIC_IOAPIC, ~0) },
> -       { PCI_DEVICE_CLASS(PCI_CLASS_SYSTEM_PIC_IOXAPIC, ~0) },
> -       { }
> -};
> -MODULE_DEVICE_TABLE(pci, ioapic_devices);
> +       handle_ioapic_add(handle, &pdev, &gsi_base);
> +       if (!gsi_base)
> +               return AE_OK;
>
> -static struct pci_driver ioapic_driver = {
> -       .name           = "ioapic",
> -       .id_table       = ioapic_devices,
> -       .probe          = ioapic_probe,
> -       .remove         = ioapic_remove,
> -};
> +       ioapic = kzalloc(sizeof(*ioapic), GFP_KERNEL);
> +       if (!ioapic) {
> +               pr_err("%s: cannot allocate memory\n", __func__);
> +               handle_ioapic_remove(root_handle, pdev, gsi_base);
> +               return AE_OK;
> +       }
> +       ioapic->root_handle = root_handle;
> +       ioapic->pdev = pdev;
> +       ioapic->gsi_base = gsi_base;
>
> -static int __init ioapic_init(void)
> -{
> -       return pci_register_driver(&ioapic_driver);
> +       mutex_lock(&ioapic_list_lock);
> +       list_add(&ioapic->list, &ioapic_list);
> +       mutex_unlock(&ioapic_list_lock);
> +
> +       return AE_OK;
>  }
>
> -static void __exit ioapic_exit(void)
> +void acpi_pci_ioapic_add(struct acpi_pci_root *root)
>  {
> -       pci_unregister_driver(&ioapic_driver);
> -}
> +       acpi_status status;
>
> -module_init(ioapic_init);
> -module_exit(ioapic_exit);
> +       status = acpi_walk_namespace(ACPI_TYPE_DEVICE, root->device->handle,
> +                                    (u32)1, register_ioapic, NULL,
> +                                    root->device->handle,
> +                                    NULL);
> +       if (ACPI_FAILURE(status))
> +               pr_err("%s: register_ioapic failure - %d", __func__, status);
> +}
>
> -MODULE_LICENSE("GPL");
> +void acpi_pci_ioapic_remove(struct acpi_pci_root *root)
> +{
> +       struct acpi_pci_ioapic *ioapic, *tmp;
> +
> +       mutex_lock(&ioapic_list_lock);
> +       list_for_each_entry_safe(ioapic, tmp, &ioapic_list, list) {
> +               if (root->device->handle != ioapic->root_handle)
> +                       continue;
> +               list_del(&ioapic->list);
> +               handle_ioapic_remove(ioapic->root_handle, ioapic->pdev,
> +                                       ioapic->gsi_base);
> +               kfree(ioapic);
> +       }
> +       mutex_unlock(&ioapic_list_lock);
> +}
> diff --git a/include/linux/pci-acpi.h b/include/linux/pci-acpi.h
> index 9a22b5e..6f83039 100644
> --- a/include/linux/pci-acpi.h
> +++ b/include/linux/pci-acpi.h
> @@ -49,4 +49,12 @@ extern bool aer_acpi_firmware_first(void);
>  static inline bool aer_acpi_firmware_first(void) { return false; }
>  #endif
>
> +#ifdef CONFIG_PCI_IOAPIC
> +void acpi_pci_ioapic_add(struct acpi_pci_root *root);
> +void acpi_pci_ioapic_remove(struct acpi_pci_root *root);
> +#else
> +static inline void acpi_pci_ioapic_add(struct acpi_pci_root *root) { }
> +static inline void acpi_pci_ioapic_remove(struct acpi_pci_root *root) { }
> +#endif
> +
>  #endif /* _PCI_ACPI_H_ */
> --
> 1.7.10.4
>

Re: [PATCH v2 26/26] PCI, x86, ACPI: Add ioapic hotplug support with acpi host bridge.

[Yinghai Lu]

On Fri, Feb 8, 2013 at 1:24 PM, Bjorn Helgaas <bhelgaas@google.com> wrote:
> On Fri, Feb 8, 2013 at 12:28 PM, Yinghai Lu <yinghai@kernel.org> wrote:
>> We need to have ioapic setup before normal pci drivers.
>> otherwise other pci driver can not setup irq.
>>
>> Make ioapic built-in, so can call add/remove during host-bridge add/remove
>> the same as the booting path.
>>
>> Also need to make it depends on X86_IO_APIC.
>
> There are a lot of different things happening in this patch.  I'd
> prefer if they were split out so we can see what's happening.  For
> example, the Kconfig changes could be their own patch.  The removal of
> the PCI device table is worthy of mention and its own patch (if
> possible), since it's a pretty major change to how the driver binds to
> devices.

ok, will try to split it small patches..

also I'd like to leave you and Rafael to decide if we need to move that file to
drivers/acpi/pci_ioapic.c or something else.

Thanks

Yinghai

[PATCH v2 26-1/26] PCI, x86: Make ioapic hotplug support built-in

[Yinghai Lu]

ioapic hotplug should be built-in by nature.

Also need to make it depends on X86_IO_APIC.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>

---
 drivers/pci/Kconfig  |    3 ++-
 drivers/pci/ioapic.c |    7 -------
 2 files changed, 2 insertions(+), 8 deletions(-)

Index: linux-2.6/drivers/pci/Kconfig
===================================================================
--- linux-2.6.orig/drivers/pci/Kconfig
+++ linux-2.6/drivers/pci/Kconfig
@@ -110,10 +110,11 @@ config PCI_PASID
 	  If unsure, say N.
 
 config PCI_IOAPIC
-	tristate "PCI IO-APIC hotplug support" if X86
+	bool "PCI IO-APIC hotplug support" if X86
 	depends on PCI
 	depends on ACPI
 	depends on HOTPLUG
+	depends on X86_IO_APIC
 	default !X86
 
 config PCI_LABEL
Index: linux-2.6/drivers/pci/ioapic.c
===================================================================
--- linux-2.6.orig/drivers/pci/ioapic.c
+++ linux-2.6/drivers/pci/ioapic.c
@@ -130,13 +130,6 @@ static int __init ioapic_init(void)
 {
 	return pci_register_driver(&ioapic_driver);
 }
-
-static void __exit ioapic_exit(void)
-{
-	pci_unregister_driver(&ioapic_driver);
-}
-
 module_init(ioapic_init);
-module_exit(ioapic_exit);
 
 MODULE_LICENSE("GPL");

[PATCH v2 26-2/26] PCI, x86, ACPI: Link acpi ioapic register to ioapic

[Yinghai Lu]

During ioapic hotplug, acpi_register_ioapic will be called.

Now for x86, that function is blank.

Fill that will update __mp_register_ioapic to use those ioapic.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>

---
 arch/x86/kernel/acpi/boot.c |   10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)

Index: linux-2.6/arch/x86/kernel/acpi/boot.c
===================================================================
--- linux-2.6.orig/arch/x86/kernel/acpi/boot.c
+++ linux-2.6/arch/x86/kernel/acpi/boot.c
@@ -696,16 +696,18 @@ EXPORT_SYMBOL(acpi_unmap_lsapic);
 
 int acpi_register_ioapic(acpi_handle handle, u64 phys_addr, u32 gsi_base)
 {
-	/* TBD */
-	return -EINVAL;
+	unsigned long long id = 0;
+
+	acpi_evaluate_integer(handle, "_UID", NULL, &id);
+
+	return	__mp_register_ioapic(id, phys_addr, gsi_base, true);
 }
 
 EXPORT_SYMBOL(acpi_register_ioapic);
 
 int acpi_unregister_ioapic(acpi_handle handle, u32 gsi_base)
 {
-	/* TBD */
-	return -EINVAL;
+	return mp_unregister_ioapic(gsi_base);
 }
 
 EXPORT_SYMBOL(acpi_unregister_ioapic);

[PATCH v2 26-3/26] PCI, x86, ACPI: Enable ioapic hotplug support with acpi host bridge.

[Yinghai Lu]

We need to have ioapic setup before normal pci drivers.
otherwise other pci driver can not setup irq.

So we should not treat them as normal pci devices.
Also we will need to support ioapic hotplug without pci device around.

We need to call ioapic add/remove during host-bridge add/remove.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>

---
 drivers/acpi/pci_root.c  |    4 +
 drivers/pci/ioapic.c     |  147 ++++++++++++++++++++++++++++++-----------------
 include/linux/pci-acpi.h |    9 ++
 3 files changed, 107 insertions(+), 53 deletions(-)

Index: linux-2.6/drivers/pci/ioapic.c
===================================================================
--- linux-2.6.orig/drivers/pci/ioapic.c
+++ linux-2.6/drivers/pci/ioapic.c
@@ -22,70 +22,77 @@
 #include <linux/slab.h>
 #include <acpi/acpi_bus.h>
 
-struct ioapic {
-	acpi_handle	handle;
+struct acpi_pci_ioapic {
+	acpi_handle	root_handle;
 	u32		gsi_base;
+	struct pci_dev *pdev;
+	struct list_head list;
 };
 
-static int ioapic_probe(struct pci_dev *dev, const struct pci_device_id *ent)
+static LIST_HEAD(ioapic_list);
+static DEFINE_MUTEX(ioapic_list_lock);
+
+static void handle_ioapic_add(acpi_handle handle, struct pci_dev **pdev,
+				 u32 *pgsi_base)
 {
-	acpi_handle handle;
 	acpi_status status;
 	unsigned long long gsb;
-	struct ioapic *ioapic;
+	struct pci_dev *dev;
+	u32 gsi_base;
 	int ret;
 	char *type;
-	struct resource *res;
+	struct resource r;
+	struct resource *res = &r;
+	char objname[64];
+	struct acpi_buffer buffer = {sizeof(objname), objname};
 
-	handle = DEVICE_ACPI_HANDLE(&dev->dev);
-	if (!handle)
-		return -EINVAL;
+	*pdev = NULL;
+	*pgsi_base = 0;
 
 	status = acpi_evaluate_integer(handle, "_GSB", NULL, &gsb);
-	if (ACPI_FAILURE(status))
-		return -EINVAL;
-
-	/*
-	 * The previous code in acpiphp evaluated _MAT if _GSB failed, but
-	 * ACPI spec 4.0 sec 6.2.2 requires _GSB for hot-pluggable I/O APICs.
-	 */
+	if (ACPI_FAILURE(status) || !gsb)
+		return;
 
-	ioapic = kzalloc(sizeof(*ioapic), GFP_KERNEL);
-	if (!ioapic)
-		return -ENOMEM;
+	dev = acpi_get_pci_dev(handle);
+	if (!dev)
+		return;
 
-	ioapic->handle = handle;
-	ioapic->gsi_base = (u32) gsb;
+	acpi_get_name(handle, ACPI_FULL_PATHNAME, &buffer);
 
-	if (dev->class == PCI_CLASS_SYSTEM_PIC_IOAPIC)
-		type = "IOAPIC";
-	else
-		type = "IOxAPIC";
+	gsi_base = gsb;
+	type = "IOxAPIC";
 
 	ret = pci_enable_device(dev);
 	if (ret < 0)
-		goto exit_free;
+		goto exit_put;
 
 	pci_set_master(dev);
 
+	if (dev->class == PCI_CLASS_SYSTEM_PIC_IOAPIC)
+		type = "IOAPIC";
+
 	if (pci_request_region(dev, 0, type))
 		goto exit_disable;
 
 	res = &dev->resource[0];
-	if (acpi_register_ioapic(ioapic->handle, res->start, ioapic->gsi_base))
+
+	if (acpi_register_ioapic(handle, res->start, gsi_base))
 		goto exit_release;
 
-	pci_set_drvdata(dev, ioapic);
-	dev_info(&dev->dev, "%s at %pR, GSI %u\n", type, res, ioapic->gsi_base);
-	return 0;
+	pr_info("%s %s %s at %pR, GSI %u\n",
+		dev_name(&dev->dev), objname, type,
+		res, gsi_base);
+
+	*pdev = dev;
+	*pgsi_base = gsi_base;
+	return;
 
 exit_release:
 	pci_release_region(dev, 0);
 exit_disable:
 	pci_disable_device(dev);
-exit_free:
-	kfree(ioapic);
-	return -ENODEV;
+exit_put:
+	pci_dev_put(dev);
 }
 
 static void pci_disable_device_mem(struct pci_dev *dev)
@@ -99,37 +106,71 @@ static void pci_disable_device_mem(struc
 	}
 }
 
-static void ioapic_remove(struct pci_dev *dev)
+static void handle_ioapic_remove(acpi_handle handle, struct pci_dev *dev,
+				  u32 gsi_base)
 {
-	struct ioapic *ioapic = pci_get_drvdata(dev);
+	acpi_unregister_ioapic(handle, gsi_base);
 
-	acpi_unregister_ioapic(ioapic->handle, ioapic->gsi_base);
 	pci_release_region(dev, 0);
 	pci_disable_device(dev);
 	/* need to disable it, otherwise remove/rescan will not work */
 	pci_disable_device_mem(dev);
-	kfree(ioapic);
+	pci_dev_put(dev);
 }
 
+static acpi_status register_ioapic(acpi_handle handle, u32 lvl,
+					void *context, void **rv)
+{
+	acpi_handle root_handle = context;
+	struct pci_dev *pdev;
+	u32 gsi_base;
+	struct acpi_pci_ioapic *ioapic;
+
+	handle_ioapic_add(handle, &pdev, &gsi_base);
+	if (!gsi_base)
+		return AE_OK;
 
-static DEFINE_PCI_DEVICE_TABLE(ioapic_devices) = {
-	{ PCI_DEVICE_CLASS(PCI_CLASS_SYSTEM_PIC_IOAPIC, ~0) },
-	{ PCI_DEVICE_CLASS(PCI_CLASS_SYSTEM_PIC_IOXAPIC, ~0) },
-	{ }
-};
-MODULE_DEVICE_TABLE(pci, ioapic_devices);
+	ioapic = kzalloc(sizeof(*ioapic), GFP_KERNEL);
+	if (!ioapic) {
+		pr_err("%s: cannot allocate memory\n", __func__);
+		handle_ioapic_remove(root_handle, pdev, gsi_base);
+		return AE_OK;
+	}
+	ioapic->root_handle = root_handle;
+	ioapic->pdev = pdev;
+	ioapic->gsi_base = gsi_base;
+
+	mutex_lock(&ioapic_list_lock);
+	list_add(&ioapic->list, &ioapic_list);
+	mutex_unlock(&ioapic_list_lock);
 
-static struct pci_driver ioapic_driver = {
-	.name		= "ioapic",
-	.id_table	= ioapic_devices,
-	.probe		= ioapic_probe,
-	.remove		= ioapic_remove,
-};
+	return AE_OK;
+}
 
-static int __init ioapic_init(void)
+void acpi_pci_ioapic_add(struct acpi_pci_root *root)
 {
-	return pci_register_driver(&ioapic_driver);
+	acpi_status status;
+
+	status = acpi_walk_namespace(ACPI_TYPE_DEVICE, root->device->handle,
+				     (u32)1, register_ioapic, NULL,
+				     root->device->handle,
+				     NULL);
+	if (ACPI_FAILURE(status))
+		pr_err("%s: register_ioapic failure - %d", __func__, status);
 }
-module_init(ioapic_init);
 
-MODULE_LICENSE("GPL");
+void acpi_pci_ioapic_remove(struct acpi_pci_root *root)
+{
+	struct acpi_pci_ioapic *ioapic, *tmp;
+
+	mutex_lock(&ioapic_list_lock);
+	list_for_each_entry_safe(ioapic, tmp, &ioapic_list, list) {
+		if (root->device->handle != ioapic->root_handle)
+			continue;
+		list_del(&ioapic->list);
+		handle_ioapic_remove(ioapic->root_handle, ioapic->pdev,
+					ioapic->gsi_base);
+		kfree(ioapic);
+	}
+	mutex_unlock(&ioapic_list_lock);
+}
Index: linux-2.6/drivers/acpi/pci_root.c
===================================================================
--- linux-2.6.orig/drivers/acpi/pci_root.c
+++ linux-2.6/drivers/acpi/pci_root.c
@@ -584,6 +584,8 @@ static int acpi_pci_root_add(struct acpi
 		pci_assign_unassigned_bus_resources(root->bus);
 	}
 
+	acpi_pci_ioapic_add(root);
+
 	mutex_lock(&acpi_pci_root_lock);
 	list_for_each_entry(driver, &acpi_pci_drivers, node)
 		if (driver->add)
@@ -623,6 +625,8 @@ static void acpi_pci_root_remove(struct
 			driver->remove(root);
 	mutex_unlock(&acpi_pci_root_lock);
 
+	acpi_pci_ioapic_remove(root);
+
 	device_set_run_wake(root->bus->bridge, false);
 	pci_acpi_remove_bus_pm_notifier(device);
 
Index: linux-2.6/include/linux/pci-acpi.h
===================================================================
--- linux-2.6.orig/include/linux/pci-acpi.h
+++ linux-2.6/include/linux/pci-acpi.h
@@ -41,8 +41,17 @@ static inline acpi_handle acpi_pci_get_b
 
 	return DEVICE_ACPI_HANDLE(dev);
 }
+
+#ifdef CONFIG_PCI_IOAPIC
+void acpi_pci_ioapic_add(struct acpi_pci_root *root);
+void acpi_pci_ioapic_remove(struct acpi_pci_root *root);
+#else
+static inline void acpi_pci_ioapic_add(struct acpi_pci_root *root) { }
+static inline void acpi_pci_ioapic_remove(struct acpi_pci_root *root) { }
 #endif
 
+#endif /* CONFIG_ACPI */
+
 #ifdef CONFIG_ACPI_APEI
 extern bool aer_acpi_firmware_first(void);
 #else

[PATCH v2 26-4/26] PCI, x86, ACPI: get ioapic address from acpi device

[Yinghai Lu]

Some ioapic controller does not show up on pci config space,
or pci device is there but no bar is used and is set by firmare in
other place.

We need to get ioapic address from ACPI0009's _CRS.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>

---
 drivers/pci/ioapic.c |   92 +++++++++++++++++++++++++++++++++++++++++----------
 1 file changed, 74 insertions(+), 18 deletions(-)

Index: linux-2.6/drivers/pci/ioapic.c
===================================================================
--- linux-2.6.orig/drivers/pci/ioapic.c
+++ linux-2.6/drivers/pci/ioapic.c
@@ -32,6 +32,36 @@ struct acpi_pci_ioapic {
 static LIST_HEAD(ioapic_list);
 static DEFINE_MUTEX(ioapic_list_lock);
 
+static acpi_status setup_res(struct acpi_resource *acpi_res, void *data)
+{
+	struct resource *res;
+	struct acpi_resource_address64 addr;
+	acpi_status status;
+	unsigned long flags;
+	u64 start, end;
+
+	status = acpi_resource_to_address64(acpi_res, &addr);
+	if (!ACPI_SUCCESS(status))
+		return AE_OK;
+
+	if (addr.resource_type == ACPI_MEMORY_RANGE) {
+		if (addr.info.mem.caching == ACPI_PREFETCHABLE_MEMORY)
+			return AE_OK;
+		flags = IORESOURCE_MEM;
+	} else
+		return AE_OK;
+
+	start = addr.minimum + addr.translation_offset;
+	end = addr.maximum + addr.translation_offset;
+
+	res = data;
+	res->flags = flags;
+	res->start = start;
+	res->end = end;
+
+	return AE_OK;
+}
+
 static void handle_ioapic_add(acpi_handle handle, struct pci_dev **pdev,
 				 u32 *pgsi_base)
 {
@@ -54,33 +84,56 @@ static void handle_ioapic_add(acpi_handl
 		return;
 
 	dev = acpi_get_pci_dev(handle);
-	if (!dev)
-		return;
+	if (!dev || !pci_resource_len(dev, 0)) {
+		struct acpi_device_info *info;
+		char *hid = NULL;
+
+		status = acpi_get_object_info(handle, &info);
+		if (ACPI_FAILURE(status))
+			goto exit_put;
+		if (info->valid & ACPI_VALID_HID)
+			hid = info->hardware_id.string;
+		if (!hid || strcmp(hid, "ACPI0009")) {
+			kfree(info);
+			goto exit_put;
+		}
+		kfree(info);
+		memset(res, 0, sizeof(*res));
+		acpi_walk_resources(handle, METHOD_NAME__CRS, setup_res, res);
+		if (!res->flags)
+			goto exit_put;
+	}
 
 	acpi_get_name(handle, ACPI_FULL_PATHNAME, &buffer);
 
 	gsi_base = gsb;
 	type = "IOxAPIC";
+	if (dev) {
+		ret = pci_enable_device(dev);
+		if (ret < 0)
+			goto exit_put;
+
+		pci_set_master(dev);
+
+		if (dev->class == PCI_CLASS_SYSTEM_PIC_IOAPIC)
+			type = "IOAPIC";
+
+		if (pci_resource_len(dev, 0)) {
+			if (pci_request_region(dev, 0, type))
+				goto exit_disable;
 
-	ret = pci_enable_device(dev);
-	if (ret < 0)
-		goto exit_put;
-
-	pci_set_master(dev);
-
-	if (dev->class == PCI_CLASS_SYSTEM_PIC_IOAPIC)
-		type = "IOAPIC";
-
-	if (pci_request_region(dev, 0, type))
-		goto exit_disable;
-
-	res = &dev->resource[0];
+			res = &dev->resource[0];
+		}
+	}
 
-	if (acpi_register_ioapic(handle, res->start, gsi_base))
-		goto exit_release;
+	if (acpi_register_ioapic(handle, res->start, gsi_base)) {
+		if (dev)
+			goto exit_release;
+		return;
+	}
 
 	pr_info("%s %s %s at %pR, GSI %u\n",
-		dev_name(&dev->dev), objname, type,
+		dev ? dev_name(&dev->dev) : "", objname, type,
 		res, gsi_base);
 
 	*pdev = dev;
@@ -111,6 +164,9 @@ static void handle_ioapic_remove(acpi_ha
 {
 	acpi_unregister_ioapic(handle, gsi_base);
 
+	if (!dev)
+		return;
+
 	pci_release_region(dev, 0);
 	pci_disable_device(dev);
 	/* need to disable it, otherwise remove/rescan will not work */

Re: [PATCH v2 00/26] x86, irq: support ioapic device hotplug

[Ingo Molnar]

* Yinghai Lu <yinghai@kernel.org> wrote:

> Hi,
> 
> Current x86 code does not support iapic hotplug yet.

Please give a better high-level description: outline how an 
IO-APIC will be hotplugged physically and what changes the 
user will experience.

That needs at least 2 paragraphs, not one cursory sentence. Then 
should come technical descriptions - but even those need 
improvements, please first explain the current status quo, then 
explain how that is inadequate for IO-APIC hot-plugging, *then* 
only explain what you are doing in the patch ...

Thanks,

	Ingo

Re: [PATCH v2 00/26] x86, irq: support ioapic device hotplug

[H. Peter Anvin]

On 02/11/2013 01:34 AM, Ingo Molnar wrote:
> 
> * Yinghai Lu <yinghai@kernel.org> wrote:
> 
>> Hi,
>>
>> Current x86 code does not support iapic hotplug yet.
> 
> Please give a better high-level description: outline how an 
> IO-APIC will be hotplugged physically and what changes the 
> user will experience.
> 
> That needs at least 2 paragraphs, not one cursory sentence. Then 
> should come technical descriptions - but even those need 
> improvements, please first explain the current status quo, then 
> explain how that is inadequate for IO-APIC hot-plugging, *then* 
> only explain what you are doing in the patch ...
> 

A few more notes:

Please explain what you mean with pre-reserve IRQ blocks, and in
particular, how do you know what to reserve and for what?

Second, some of the things in this patchset seems to overlap with what
we would need to support MSI (as opposed to MSI-X), and you have several
patches related to MSI vs MSI-X.  Are you also supporting multivector
MSI (I believe the answer is no) or could some of this work make
multivector MSI practical (possible)?

	-hpa

Re: [PATCH v2 00/26] x86, irq: support ioapic device hotplug

[Yinghai Lu]

On Mon, Feb 11, 2013 at 10:10 PM, H. Peter Anvin <hpa@zytor.com> wrote:
> On 02/11/2013 01:34 AM, Ingo Molnar wrote:
>>
>> * Yinghai Lu <yinghai@kernel.org> wrote:
>>
>>> Hi,
>>>
>>> Current x86 code does not support iapic hotplug yet.
>>
>> Please give a better high-level description: outline how an
>> IO-APIC will be hotplugged physically and what changes the
>> user will experience.

Intel cpu (from IVB) include cpu, mem controller, and IIO.
When hotadd cpu physically, it will involve cpu hotplug, mem hotplug,
and pci root hotplug.

IIO includes pci host bridge, ioapic controller and iommu.
pci devices will need to use ioapic and iommu.
So to make pci hotplug working, we need to add ioapic hotplug support.

>>
>> That needs at least 2 paragraphs, not one cursory sentence. Then
>> should come technical descriptions - but even those need
>> improvements, please first explain the current status quo, then
>> explain how that is inadequate for IO-APIC hot-plugging, *then*
>> only explain what you are doing in the patch ...

Now ioapics are with ACPI MADT table. During booting, kernel will parse
MADT and put info into ioapic arrays.
Also Bjorn added one pci device based driver, but it is not wired in yet,
as it need to call acpi_register_ioapic, and that is TBD.

This patchset will
1. extend genirq to support reserve/realloc method.
   because we want to irqs for one ioapic controller to be linear mapping
   to the gsi range.
2. change ioapic array operation code so we could insert new ioapic and
   remove one leave the blank slot.
3. record irq_base in gsi_config in ioapic struct, and use it to convert gsi
    to irq for pci device using that ioapic controller
4. make static ioapic path (from MADT) use same code as hotadd path,
   with reserve/realloc.
5. change ioapic add/removing to acpi way, as that is only need when
   pci root bus hotplug. Also it will make it support the case that ioapic
   controller is hiding in pci controller space or ioapic address is not
   managed by pci reallocation system.

>>
>
> A few more notes:
>
> Please explain what you mean with pre-reserve IRQ blocks, and in
> particular, how do you know what to reserve and for what?

For each ioapic controller, and we should get gsi range already.
So we could keep irq to gsi mapping linear for each ioapic controller.

>
> Second, some of the things in this patchset seems to overlap with what
> we would need to support MSI (as opposed to MSI-X), and you have several
> patches related to MSI vs MSI-X.  Are you also supporting multivector
> MSI (I believe the answer is no) or could some of this work make
> multivector MSI practical (possible)?

Patches that is related to MSI/MSI-X, is just print out clean up.
Now we don't tell if it is MSI or MSI-X in /proc/interrupts or dmesg.

Patches from multivector MSI support from Alex are already put in
x86/apic branch by Ingo.

And my v2 patches are on top of x86/apic and pci/next.

Thanks

Yinghai