LKML Archive on lore.kernel.org
 help / color / Atom feed
From: Thomas Gleixner <tglx@linutronix.de>
To: LKML <linux-kernel@vger.kernel.org>
Cc: x86@kernel.org, Jiang Liu <jiang.liu@linux.intel.com>,
	Joerg Roedel <joro@8bytes.org>, Borislav Petkov <bp@alien8.de>
Subject: [patch 0/5] x86/iommu: Bootup stage cleanups
Date: Fri, 05 Dec 2014 08:48:28 -0000
Message-ID: <20141204234737.728961990@linutronix.de> (raw)

Boris reported that the conversion to stacked irqdomains causes a
GFP_KERNEL allocation from atomic context splat.

The callchain is:

kernel_init_freeable
 smp_prepare_cpus
  default_setup_apic_routing
   enable_IR_x2apic
    irq_remapping_enable
     intel_enable_irq_remapping
       intel_setup_irq_remapping
         irq_domain_add_hierarchy
	   __irq_domain_add

The reason for the splat is that smp_prepare_cpus() is pointlessly
disabling preemption and enable_IR_x2apic is calling
irq_remapping_enable with interrupts disabled.

Removing the completely useless preemption disable from
smp_prepare_cpus() is a no brainer, but resolving the interrupt
disabled issue in enable_IR_x2apic/irq_remapping_enable forced me to
look deeper into the iommu setup code.

Convoluted is probably an euphemism for that maze.

The early detection of iommus happens in mem_init() which looks for
the availability of ACPI tables describing iommus.

The actual parsing of the tables happens for AMD in
setup_irq_remapping_ops() which calls remap_ops.prepare() and for
Intel in irq_remapping_enable().

The AMD parsing allocates data structures, memory pages and the irq
domain from the prepare callback, which makes a lot of sense as this
has nothing to do with the actual enablement. That AMD code uses
GFP_KERNEL allocations inside of a preempt disabled region. That issue
is solved by removing the silly preempt_disable() from
smp_prepare_cpus().

For Intel the actual parsing and allocation is moved from the enable
to the prepare callback. That cures the reported splat and allows to
use GFP_KERNEL allocations for the iommu internal stuff as well. While
in the early bootup it does not matter much, for physical hotplug
GFP_ATOMIC allocations are not desired at all.

That solves the issue at hand, but the iommu setup stays a tangled
maze with completely non obvious setup mechanisms depending on the
options (iommu on/off - irq remapping on/off).

Thanks,

	tglx
---
 arch/x86/include/asm/irq_remapping.h |    2 -
 arch/x86/kernel/apic/apic.c          |    3 -
 arch/x86/kernel/smpboot.c            |    8 ----
 drivers/iommu/intel_irq_remapping.c  |   68 +++++++++++++++++++++++------------
 drivers/iommu/irq_remapping.c        |   19 +++------
 kernel/irq/irqdomain.c               |   21 +---------
 6 files changed, 56 insertions(+), 65 deletions(-)






             reply index

Thread overview: 56+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2014-12-05  8:48 Thomas Gleixner [this message]
2014-12-05  8:48 ` [patch 1/5] x86, smpboot: Remove pointless preempt_disable() in native_smp_prepare_cpus() Thomas Gleixner
2014-12-05 23:25   ` [tip:x86/apic] " tip-bot for Thomas Gleixner
2014-12-19 14:02   ` tip-bot for Thomas Gleixner
2014-12-05  8:48 ` [patch 2/5] iommu, x86: Restructure setup of the irq remapping feature Thomas Gleixner
2014-12-05 23:25   ` [tip:x86/apic] " tip-bot for Thomas Gleixner
2014-12-05  8:48 ` [patch 3/5] iommu/vt-d: Move iommu preparatory allocations to irq_remap_ops.prepare Thomas Gleixner
2014-12-05 23:26   ` [tip:x86/apic] " tip-bot for Thomas Gleixner
2014-12-11  7:35     ` Yinghai Lu
2014-12-11 14:33       ` Jiang Liu
2014-12-11 17:57         ` Yinghai Lu
2014-12-11 20:30           ` Thomas Gleixner
2014-12-12  2:04             ` Yinghai Lu
2015-04-27 22:46               ` Yinghai Lu
2015-04-29  8:15                 ` Jiang Liu
2014-12-05  8:48 ` [patch 4/5] irqdomain: Revert gfp hackery Thomas Gleixner
2014-12-05 23:26   ` [tip:x86/apic] " tip-bot for Thomas Gleixner
2014-12-05  8:48 ` [patch 5/5] iommu/vt-d: Convert allocations to GFP_KERNEL Thomas Gleixner
2014-12-05 23:26   ` [tip:x86/apic] " tip-bot for Thomas Gleixner
2014-12-05 12:22 ` [patch 0/5] x86/iommu: Bootup stage cleanups Joerg Roedel
2015-01-07  7:31 [Patch v2 00/16] Refine IR initialization flow and fixes bugs related to X2APIC Jiang Liu
2015-01-07  7:31 ` [Patch v2 01/16] iommu, x86: Restructure setup of the irq remapping feature Jiang Liu
2015-01-15 12:58   ` [tip:x86/apic] " tip-bot for Thomas Gleixner
2015-01-07  7:31 ` [Patch v2 02/16] iommu/vt-d: Move iommu preparatory allocations to irq_remap_ops.prepare Jiang Liu
2015-01-15 12:58   ` [tip:x86/apic] " tip-bot for Thomas Gleixner
2015-01-07  7:31 ` [Patch v2 03/16] iommu/vt-d: Convert allocations to GFP_KERNEL Jiang Liu
2015-01-15 12:59   ` [tip:x86/apic] " tip-bot for Thomas Gleixner
2015-01-07  7:31 ` [Patch v2 04/16] x86/apic: Panic if kernel doesn't support x2apic but BIOS has enabled x2apic Jiang Liu
2015-01-15 12:59   ` [tip:x86/apic] x86/apic: Panic if kernel doesn' t " tip-bot for Jiang Liu
2015-01-07  7:31 ` [Patch v2 05/16] x86/apic: Kill useless variable x2apic_enabled in function enable_IR_x2apic() Jiang Liu
2015-01-15 12:59   ` [tip:x86/apic] " tip-bot for Jiang Liu
2015-01-07  7:31 ` [Patch v2 06/16] x86/apic: Correctly detect X2APIC status in function enable_IR() Jiang Liu
2015-01-15 13:00   ` [tip:x86/apic] " tip-bot for Jiang Liu
2015-01-07  7:31 ` [Patch v2 07/16] x86/apic: Refine enable_IR_x2apic() and related functions Jiang Liu
2015-01-15  9:24   ` Thomas Gleixner
2015-01-15 13:00   ` [tip:x86/apic] " tip-bot for Jiang Liu
2015-01-15 13:00   ` [tip:x86/apic] x86/apic: Handle XAPIC remap mode proper tip-bot for Jiang Liu
2015-01-07  7:31 ` [Patch v2 08/16] iommu/vt-d: Prepare for killing function irq_remapping_supported() Jiang Liu
2015-01-15 13:01   ` [tip:x86/apic] " tip-bot for Jiang Liu
2015-01-07  7:31 ` [Patch v2 09/16] iommu/vt-d: Allocate IRQ remapping data structures only for all IOMMUs Jiang Liu
2015-01-15 13:01   ` [tip:x86/apic] " tip-bot for Joerg Roedel
2015-01-07  7:31 ` [Patch v2 10/16] iommu/vt-d: Allow IR works in XAPIC mode though CPU works in X2APIC mode Jiang Liu
2015-01-15 13:01   ` [tip:x86/apic] " tip-bot for Jiang Liu
2015-01-07  7:31 ` [Patch v2 11/16] x86/apic: Only disable CPU x2apic mode when necessary Jiang Liu
2015-01-15 13:02   ` [tip:x86/apic] " tip-bot for Jiang Liu
2015-01-07  7:31 ` [Patch v2 12/16] iommu/amd: Check for irq-remap support amd_iommu_prepare() Jiang Liu
2015-01-15 13:02   ` [tip:x86/apic] " tip-bot for Joerg Roedel
2015-01-07  7:31 ` [Patch v2 13/16] iommu/irq_remapping: Kill function irq_remapping_supported() and related code Jiang Liu
2015-01-15 13:02   ` [tip:x86/apic] " tip-bot for Jiang Liu
2015-01-07  7:31 ` [Patch v2 14/16] iommu/irq_remapping: Refine function irq_remapping_prepare() for maintenance Jiang Liu
2015-01-15 13:03   ` [tip:x86/apic] " tip-bot for Jiang Liu
2015-01-07  7:31 ` [Patch v2 15/16] iommu/irq_remapping: Change variable disable_irq_remap to be static Jiang Liu
2015-01-15 13:03   ` [tip:x86/apic] " tip-bot for Jiang Liu
2015-01-07  7:31 ` [Patch v2 16/16] iommu/irq_remapping: Normailize the way to detect whether IR is enabled Jiang Liu
2015-01-15 13:03   ` [tip:x86/apic] " tip-bot for Jiang Liu
2015-01-09 12:03 ` [Patch v2 00/16] Refine IR initialization flow and fixes bugs related to X2APIC Joerg Roedel

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20141204234737.728961990@linutronix.de \
    --to=tglx@linutronix.de \
    --cc=bp@alien8.de \
    --cc=jiang.liu@linux.intel.com \
    --cc=joro@8bytes.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=x86@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

LKML Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/lkml/0 lkml/git/0.git
	git clone --mirror https://lore.kernel.org/lkml/1 lkml/git/1.git
	git clone --mirror https://lore.kernel.org/lkml/2 lkml/git/2.git
	git clone --mirror https://lore.kernel.org/lkml/3 lkml/git/3.git
	git clone --mirror https://lore.kernel.org/lkml/4 lkml/git/4.git
	git clone --mirror https://lore.kernel.org/lkml/5 lkml/git/5.git
	git clone --mirror https://lore.kernel.org/lkml/6 lkml/git/6.git
	git clone --mirror https://lore.kernel.org/lkml/7 lkml/git/7.git
	git clone --mirror https://lore.kernel.org/lkml/8 lkml/git/8.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 lkml lkml/ https://lore.kernel.org/lkml \
		linux-kernel@vger.kernel.org
	public-inbox-index lkml

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-kernel


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git