linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Ricardo Neri <ricardo.neri-calderon@linux.intel.com>
To: Thomas Gleixner <tglx@linutronix.de>,
	Ingo Molnar <mingo@kernel.org>, "H. Peter Anvin" <hpa@zytor.com>
Cc: Andi Kleen <andi.kleen@intel.com>,
	Ashok Raj <ashok.raj@intel.com>, Borislav Petkov <bp@suse.de>,
	Tony Luck <tony.luck@intel.com>,
	"Ravi V. Shankar" <ravi.v.shankar@intel.com>,
	x86@kernel.org, sparclinux@vger.kernel.org,
	linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org,
	Ricardo Neri <ricardo.neri-calderon@linux.intel.com>
Subject: [RFC PATCH 00/23] Implement an HPET-based hardlockup detector
Date: Tue, 12 Jun 2018 17:57:20 -0700	[thread overview]
Message-ID: <1528851463-21140-1-git-send-email-ricardo.neri-calderon@linux.intel.com> (raw)

Hi,

This patchset demonstrates the implementation of a hardlockup detector
driven by the High-Precision Event Timer.

== Introduction ==

In CPU architectures that do not have an NMI watchdog, one can be
constructed using a counter of the Performance Monitoring Unit (PMU).
Counters in the PMU have high granularity and high visibility of the CPU.
These capabilities and their limited number make these counters precious
resources. Unfortunately, the perf-based hardlockup detector permanently
consumes one of these counters per CPU.

These counters could be freed for profiling purposes if the hardlockup
detector were driven by another timer.

The hardlockup detector runs relatively infrequently and does not require
visibility of the CPU activity (in addition to detect locked-up CPUs). A
timer that is external to the CPU (e.g., in the chipset) can be used to
drive the detector.

A key requirement is that the timer needs to be capable of issuing a
non-maskable interrupt to the CPU. In most cases, this can be achieved
by tweaking the delivery mode of the interrupt in the interrupt controller
chip (the exception is the IO APIC).

== Parts of this series ==

Several parts of Linux need to be updated to operate the aforementioned
detector.

   1) Update the interrupt subsystem to accept requests of interrupts as
      non-maskable. Likewise, handle irqchips that have this capability.
      Patches 1-5
   
   2) Rework the x86 HPET platform code to reserve, configure a timer
      and its interrupt, and expose the needed interfaces and definitions.
      Patches 6-11

   3) Rework the hardlockup detector to decouple its generic part from
      perf. This adds definitions to be implemented using other sources
      of non-maskable interrupts. Patches 12-14

   4) Add an HPET-based hardlockup detector. This includes probing the
      hardware resources, configure the interrupt and rotate the
      destination of the interrupts among all monitored CPUs.

== Details on the HPET-based hardlockup detector

Unlike the the perf-based hardlockup detector, this implementation is
driven by a single timer. The timer targets one CPU at a time in a round-
robin manner. This means that if a CPU must be monitored every watch_thresh
seconds, in a system with N monitored CPUs the timer must expire every
watch_thresh/N. A timer expiration per CPU attribute is maintained.

The timer expiration time per CPU is updated every time CPUs are put
online or offline (a CPU hotplug thread enables and disables the watchdog
in these events).

Also, given that a single timer drives the detector, a cpumask is needed
to keep track of which online CPUs are allowed to be monitored. This mask
is updated every time a CPU is put online or offline as well as when the
user modifies the mask in /proc/sys/kernel/watchdog_cpumask. This mask
is needed to keep the current behavior of the lockup detector.


Thanks and BR,
Ricardo

Ricardo Neri (23):
  x86/apic: Add a parameter for the APIC delivery mode
  genirq: Introduce IRQD_DELIVER_AS_NMI
  genirq: Introduce IRQF_DELIVER_AS_NMI
  iommu/vt-d/irq_remapping: Add support for IRQCHIP_CAN_DELIVER_AS_NMI
  x86/msi: Add support for IRQCHIP_CAN_DELIVER_AS_NMI
  x86/ioapic: Add support for IRQCHIP_CAN_DELIVER_AS_NMI with interrupt
    remapping
  x86/hpet: Expose more functions to read and write registers
  x86/hpet: Calculate ticks-per-second in a separate function
  x86/hpet: Reserve timer for the HPET hardlockup detector
  x86/hpet: Relocate flag definitions to a header file
  x86/hpet: Configure the timer used by the hardlockup detector
  kernel/watchdog: Introduce a struct for NMI watchdog operations
  watchdog/hardlockup: Define a generic function to detect hardlockups
  watchdog/hardlockup: Decouple the hardlockup detector from perf
  kernel/watchdog: Add a function to obtain the watchdog_allowed_mask
  watchdog/hardlockup: Add an HPET-based hardlockup detector
  watchdog/hardlockup/hpet: Convert the timer's interrupt to NMI
  watchdog/hardlockup/hpet: Add the NMI watchdog operations
  watchdog/hardlockup: Make arch_touch_nmi_watchdog() to hpet-based
    implementation
  watchdog/hardlockup/hpet: Rotate interrupt among all monitored CPUs
  watchdog/hardlockup/hpet: Adjust timer expiration on the number of
    monitored CPUs
  watchdog/hardlockup/hpet: Only enable the HPET watchdog via a boot
    parameter
  watchdog/hardlockup: Activate the HPET-based lockup detector

 Documentation/admin-guide/kernel-parameters.txt |   5 +-
 arch/x86/include/asm/hpet.h                     |  38 ++
 arch/x86/include/asm/hw_irq.h                   |   5 +-
 arch/x86/include/asm/msidef.h                   |   3 +
 arch/x86/kernel/apic/io_apic.c                  |   5 +-
 arch/x86/kernel/apic/msi.c                      |   7 +-
 arch/x86/kernel/apic/vector.c                   |   8 +
 arch/x86/kernel/hpet.c                          | 149 ++++++-
 arch/x86/platform/uv/uv_irq.c                   |   2 +-
 drivers/char/hpet.c                             |  31 +-
 drivers/iommu/intel_irq_remapping.c             |  18 +-
 include/linux/hpet.h                            |   1 +
 include/linux/interrupt.h                       |   3 +
 include/linux/irq.h                             |  15 +
 include/linux/nmi.h                             |  56 ++-
 kernel/Makefile                                 |   3 +-
 kernel/irq/manage.c                             |  22 +-
 kernel/watchdog.c                               |  78 +++-
 kernel/watchdog_hld.c                           | 152 +------
 kernel/watchdog_hld_hpet.c                      | 557 ++++++++++++++++++++++++
 kernel/watchdog_hld_perf.c                      | 182 ++++++++
 lib/Kconfig.debug                               |  10 +
 22 files changed, 1145 insertions(+), 205 deletions(-)
 create mode 100644 kernel/watchdog_hld_hpet.c
 create mode 100644 kernel/watchdog_hld_perf.c

-- 
2.7.4


             reply	other threads:[~2018-06-13  1:07 UTC|newest]

Thread overview: 70+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-06-13  0:57 Ricardo Neri [this message]
2018-06-13  0:57 ` [RFC PATCH 01/23] x86/apic: Add a parameter for the APIC delivery mode Ricardo Neri
2018-06-13  0:57 ` [RFC PATCH 02/23] genirq: Introduce IRQD_DELIVER_AS_NMI Ricardo Neri
2018-06-13  0:57 ` [RFC PATCH 03/23] genirq: Introduce IRQF_DELIVER_AS_NMI Ricardo Neri
2018-06-13  8:34   ` Peter Zijlstra
2018-06-13  8:59     ` Julien Thierry
2018-06-13  9:20       ` Thomas Gleixner
2018-06-13  9:36         ` Julien Thierry
2018-06-13  9:49           ` Julien Thierry
2018-06-13  9:57           ` Thomas Gleixner
2018-06-13 10:25             ` Julien Thierry
2018-06-13 10:06         ` Marc Zyngier
2018-06-15  2:12           ` Ricardo Neri
2018-06-15  8:01             ` Julien Thierry
2018-06-16  0:39               ` Ricardo Neri
2018-06-16 13:36                 ` Thomas Gleixner
2018-06-13  0:57 ` [RFC PATCH 04/23] iommu/vt-d/irq_remapping: Add support for IRQCHIP_CAN_DELIVER_AS_NMI Ricardo Neri
2018-06-13  0:57 ` [RFC PATCH 05/23] x86/msi: " Ricardo Neri
2018-06-13  0:57 ` [RFC PATCH 06/23] x86/ioapic: Add support for IRQCHIP_CAN_DELIVER_AS_NMI with interrupt remapping Ricardo Neri
2018-06-13  0:57 ` [RFC PATCH 07/23] x86/hpet: Expose more functions to read and write registers Ricardo Neri
2018-06-13  0:57 ` [RFC PATCH 08/23] x86/hpet: Calculate ticks-per-second in a separate function Ricardo Neri
2018-06-13  0:57 ` [RFC PATCH 09/23] x86/hpet: Reserve timer for the HPET hardlockup detector Ricardo Neri
2018-06-13  0:57 ` [RFC PATCH 10/23] x86/hpet: Relocate flag definitions to a header file Ricardo Neri
2018-06-13  0:57 ` [RFC PATCH 11/23] x86/hpet: Configure the timer used by the hardlockup detector Ricardo Neri
2018-06-13  0:57 ` [RFC PATCH 12/23] kernel/watchdog: Introduce a struct for NMI watchdog operations Ricardo Neri
2018-06-13  7:41   ` Nicholas Piggin
2018-06-13  8:42     ` Peter Zijlstra
2018-06-13  9:26       ` Thomas Gleixner
2018-06-13 11:52         ` Nicholas Piggin
2018-06-14  1:31           ` Ricardo Neri
2018-06-14  2:32             ` Nicholas Piggin
2018-06-14  8:32               ` Thomas Gleixner
2018-06-15  2:21               ` Ricardo Neri
2018-06-14  1:26       ` Ricardo Neri
2018-06-13  0:57 ` [RFC PATCH 13/23] watchdog/hardlockup: Define a generic function to detect hardlockups Ricardo Neri
2018-06-13  0:57 ` [RFC PATCH 14/23] watchdog/hardlockup: Decouple the hardlockup detector from perf Ricardo Neri
2018-06-13  8:43   ` Peter Zijlstra
2018-06-14  1:19     ` Ricardo Neri
2018-06-14  1:41       ` Nicholas Piggin
2018-06-15  2:23         ` Ricardo Neri
2018-06-13  0:57 ` [RFC PATCH 15/23] kernel/watchdog: Add a function to obtain the watchdog_allowed_mask Ricardo Neri
2018-06-13  0:57 ` [RFC PATCH 16/23] watchdog/hardlockup: Add an HPET-based hardlockup detector Ricardo Neri
2018-06-13  5:23   ` Randy Dunlap
2018-06-14  1:00     ` Ricardo Neri
2018-06-13  0:57 ` [RFC PATCH 17/23] watchdog/hardlockup/hpet: Convert the timer's interrupt to NMI Ricardo Neri
2018-06-13  9:07   ` Peter Zijlstra
2018-06-15  2:07     ` Ricardo Neri
2018-06-13  9:40   ` Thomas Gleixner
2018-06-15  2:03     ` Ricardo Neri
2018-06-15  9:19       ` Thomas Gleixner
2018-06-16  0:51         ` Ricardo Neri
2018-06-16 13:24           ` Thomas Gleixner
2018-06-20  0:15             ` Ricardo Neri
2018-06-20  0:25               ` Randy Dunlap
2018-06-21  0:25                 ` Ricardo Neri
2018-06-20  7:47               ` Thomas Gleixner
2018-06-13  0:57 ` [RFC PATCH 18/23] watchdog/hardlockup/hpet: Add the NMI watchdog operations Ricardo Neri
2018-06-13  0:57 ` [RFC PATCH 19/23] watchdog/hardlockup: Make arch_touch_nmi_watchdog() to hpet-based implementation Ricardo Neri
2018-06-13  0:57 ` [RFC PATCH 20/23] watchdog/hardlockup/hpet: Rotate interrupt among all monitored CPUs Ricardo Neri
2018-06-13  9:48   ` Thomas Gleixner
2018-06-15  2:16     ` Ricardo Neri
2018-06-15 10:29       ` Thomas Gleixner
2018-06-16  0:46         ` Ricardo Neri
2018-06-16 13:27           ` Thomas Gleixner
2018-06-13  0:57 ` [RFC PATCH 21/23] watchdog/hardlockup/hpet: Adjust timer expiration on the number of " Ricardo Neri
2018-06-13  0:57 ` [RFC PATCH 22/23] watchdog/hardlockup/hpet: Only enable the HPET watchdog via a boot parameter Ricardo Neri
2018-06-13  5:26   ` Randy Dunlap
2018-06-14  0:58     ` Ricardo Neri
2018-06-14  3:30       ` Randy Dunlap
2018-06-13  0:57 ` [RFC PATCH 23/23] watchdog/hardlockup: Activate the HPET-based lockup detector Ricardo Neri

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1528851463-21140-1-git-send-email-ricardo.neri-calderon@linux.intel.com \
    --to=ricardo.neri-calderon@linux.intel.com \
    --cc=andi.kleen@intel.com \
    --cc=ashok.raj@intel.com \
    --cc=bp@suse.de \
    --cc=hpa@zytor.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linuxppc-dev@lists.ozlabs.org \
    --cc=mingo@kernel.org \
    --cc=ravi.v.shankar@intel.com \
    --cc=sparclinux@vger.kernel.org \
    --cc=tglx@linutronix.de \
    --cc=tony.luck@intel.com \
    --cc=x86@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).