* [PATCH v7 0/2] Detect stalls on guest vCPUS @ 2022-06-21 8:03 Sebastian Ene 2022-06-21 8:03 ` [PATCH v7 1/2] dt-bindings: vcpu_stall_detector: Add qemu,vcpu-stall-detector compatible Sebastian Ene 2022-06-21 8:03 ` [PATCH v7 2/2] misc: Add a mechanism to detect stalls on guest vCPUs Sebastian Ene 0 siblings, 2 replies; 8+ messages in thread From: Sebastian Ene @ 2022-06-21 8:03 UTC (permalink / raw) To: Rob Herring, Greg Kroah-Hartman, Arnd Bergmann, Dragan Cvetic Cc: linux-kernel, devicetree, maz, will, vdonnefort, Guenter Roeck, Sebastian Ene This adds a mechanism to detect stalls on the guest vCPUS by creating a per CPU hrtimer which periodically 'pets' the host backend driver. On a conventional watchdog-core driver, the userspace is responsible for delivering the 'pet' events by writing to the particular /dev/watchdogN node. In this case we require a strong thread affinity to be able to account for lost time on a per vCPU basis. This device driver acts as a soft lockup detector by relying on the host backend driver to measure the elapesed time between subsequent 'pet' events. If the elapsed time doesn't match an expected value, the backend driver decides that the guest vCPU is locked and resets the guest. The host backend driver takes into account the time that the guest is not running. The communication with the backend driver is done through MMIO and the register layout of the virtual watchdog is described as part of the backend driver changes. The host backend driver is implemented as part of: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3548817 Changelog v7: - fix the dtschema warnings for 'timeout-sec' property - rename vcpu_stall_detector.yaml to qemu,vcpu_stall_detector.yaml and place the file under misc - improve the Kconfig description for the driver by making it KVM specific Changelog v6: - fix issues reported by lkp@intel robot: building for ARCH=h8300 incorrect type in assignment (different address spaces) Sebastian Ene (2): dt-bindings: vcpu_stall_detector: Add qemu,vcpu-stall-detector compatible misc: Add a mechanism to detect stalls on guest vCPUs .../misc/qemu,vcpu-stall-detector.yaml | 50 ++++ drivers/misc/Kconfig | 12 + drivers/misc/Makefile | 1 + drivers/misc/vcpu_stall_detector.c | 222 ++++++++++++++++++ 4 files changed, 285 insertions(+) create mode 100644 Documentation/devicetree/bindings/misc/qemu,vcpu-stall-detector.yaml create mode 100644 drivers/misc/vcpu_stall_detector.c -- 2.36.1.476.g0c4daa206d-goog ^ permalink raw reply [flat|nested] 8+ messages in thread
* [PATCH v7 1/2] dt-bindings: vcpu_stall_detector: Add qemu,vcpu-stall-detector compatible 2022-06-21 8:03 [PATCH v7 0/2] Detect stalls on guest vCPUS Sebastian Ene @ 2022-06-21 8:03 ` Sebastian Ene 2022-06-24 17:26 ` Rob Herring 2022-06-21 8:03 ` [PATCH v7 2/2] misc: Add a mechanism to detect stalls on guest vCPUs Sebastian Ene 1 sibling, 1 reply; 8+ messages in thread From: Sebastian Ene @ 2022-06-21 8:03 UTC (permalink / raw) To: Rob Herring, Greg Kroah-Hartman, Arnd Bergmann, Dragan Cvetic Cc: linux-kernel, devicetree, maz, will, vdonnefort, Guenter Roeck, Sebastian Ene The VCPU stall detection mechanism allows to configure the expiration duration and the internal counter clock frequency measured in Hz. Add these properties in the schema. While this is a memory mapped virtual device, it is expected to be loaded when the DT contains the compatible: "qemu,vcpu-stall-detector" node. In a protected VM we trust the generated DT nodes and we don't rely on the host to present the hardware peripherals. Signed-off-by: Sebastian Ene <sebastianene@google.com> --- .../misc/qemu,vcpu-stall-detector.yaml | 50 +++++++++++++++++++ 1 file changed, 50 insertions(+) create mode 100644 Documentation/devicetree/bindings/misc/qemu,vcpu-stall-detector.yaml diff --git a/Documentation/devicetree/bindings/misc/qemu,vcpu-stall-detector.yaml b/Documentation/devicetree/bindings/misc/qemu,vcpu-stall-detector.yaml new file mode 100644 index 000000000000..91f012d2a382 --- /dev/null +++ b/Documentation/devicetree/bindings/misc/qemu,vcpu-stall-detector.yaml @@ -0,0 +1,50 @@ +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) +%YAML 1.2 +--- +$id: http://devicetree.org/schemas/misc/qemu,vcpu-stall-detector.yaml# +$schema: http://devicetree.org/meta-schemas/core.yaml# + +title: VCPU stall detector + +description: + This binding describes a CPU stall detector mechanism for virtual CPUs + which is accessed through MMIO. + +maintainers: + - Sebastian Ene <sebastianene@google.com> + +properties: + compatible: + enum: + - qemu,vcpu-stall-detector + + clock-frequency: + $ref: /schemas/types.yaml#/definitions/uint32 + description: | + The internal clock of the stall detector peripheral measure in Hz used + to decrement its internal counter register on each tick. + Defaults to 10 if unset. + default: 10 + + timeout-sec: + description: | + The stall detector expiration timeout measured in seconds. + Defaults to 8 if unset. Please note that it also takes into account the + time spent while the VCPU is not running. + default: 8 + +required: + - compatible + +additionalProperties: false + +examples: + - | + vmwdt@9030000 { + compatible = "qemu,vcpu-stall-detector"; + clock-frequency = <10>; + timeout-sec = <8>; + reg = <0x0 0x9030000 0x0 0x10000>; + }; + +... -- 2.36.1.476.g0c4daa206d-goog ^ permalink raw reply related [flat|nested] 8+ messages in thread
* Re: [PATCH v7 1/2] dt-bindings: vcpu_stall_detector: Add qemu,vcpu-stall-detector compatible 2022-06-21 8:03 ` [PATCH v7 1/2] dt-bindings: vcpu_stall_detector: Add qemu,vcpu-stall-detector compatible Sebastian Ene @ 2022-06-24 17:26 ` Rob Herring 0 siblings, 0 replies; 8+ messages in thread From: Rob Herring @ 2022-06-24 17:26 UTC (permalink / raw) To: Sebastian Ene Cc: Greg Kroah-Hartman, will, vdonnefort, maz, Arnd Bergmann, Guenter Roeck, Dragan Cvetic, Rob Herring, devicetree, linux-kernel On Tue, 21 Jun 2022 08:03:08 +0000, Sebastian Ene wrote: > The VCPU stall detection mechanism allows to configure the expiration > duration and the internal counter clock frequency measured in Hz. > Add these properties in the schema. > > While this is a memory mapped virtual device, it is expected to be loaded > when the DT contains the compatible: "qemu,vcpu-stall-detector" node. > In a protected VM we trust the generated DT nodes and we don't rely on > the host to present the hardware peripherals. > > Signed-off-by: Sebastian Ene <sebastianene@google.com> > --- > .../misc/qemu,vcpu-stall-detector.yaml | 50 +++++++++++++++++++ > 1 file changed, 50 insertions(+) > create mode 100644 Documentation/devicetree/bindings/misc/qemu,vcpu-stall-detector.yaml > My bot found errors running 'make DT_CHECKER_FLAGS=-m dt_binding_check' on your patch (DT_CHECKER_FLAGS is new in v5.13): yamllint warnings/errors: dtschema/dtc warnings/errors: /builds/robherring/linux-dt-review/Documentation/devicetree/bindings/misc/qemu,vcpu-stall-detector.example.dtb: vmwdt@9030000: 'reg' does not match any of the regexes: 'pinctrl-[0-9]+' From schema: /builds/robherring/linux-dt-review/Documentation/devicetree/bindings/misc/qemu,vcpu-stall-detector.yaml doc reference errors (make refcheckdocs): See https://patchwork.ozlabs.org/patch/ This check can fail if there are any dependencies. The base for a patch series is generally the most recent rc1. If you already ran 'make dt_binding_check' and didn't see the above error(s), then make sure 'yamllint' is installed and dt-schema is up to date: pip3 install dtschema --upgrade Please check and re-submit. ^ permalink raw reply [flat|nested] 8+ messages in thread
* [PATCH v7 2/2] misc: Add a mechanism to detect stalls on guest vCPUs 2022-06-21 8:03 [PATCH v7 0/2] Detect stalls on guest vCPUS Sebastian Ene 2022-06-21 8:03 ` [PATCH v7 1/2] dt-bindings: vcpu_stall_detector: Add qemu,vcpu-stall-detector compatible Sebastian Ene @ 2022-06-21 8:03 ` Sebastian Ene 2022-06-21 8:27 ` Greg Kroah-Hartman 1 sibling, 1 reply; 8+ messages in thread From: Sebastian Ene @ 2022-06-21 8:03 UTC (permalink / raw) To: Rob Herring, Greg Kroah-Hartman, Arnd Bergmann, Dragan Cvetic Cc: linux-kernel, devicetree, maz, will, vdonnefort, Guenter Roeck, Sebastian Ene This driver creates per-cpu hrtimers which are required to do the periodic 'pet' operation. On a conventional watchdog-core driver, the userspace is responsible for delivering the 'pet' events by writing to the particular /dev/watchdogN node. In this case we require a strong thread affinity to be able to account for lost time on a per vCPU. This part of the driver is the 'frontend' which is reponsible for delivering the periodic 'pet' events, configuring the virtual peripheral and listening for cpu hotplug events. The other part of the driver handles the peripheral emulation and this part accounts for lost time by looking at the /proc/{}/task/{}/stat entries and is located here: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3548817 Signed-off-by: Sebastian Ene <sebastianene@google.com> --- drivers/misc/Kconfig | 12 ++ drivers/misc/Makefile | 1 + drivers/misc/vcpu_stall_detector.c | 222 +++++++++++++++++++++++++++++ 3 files changed, 235 insertions(+) create mode 100644 drivers/misc/vcpu_stall_detector.c diff --git a/drivers/misc/Kconfig b/drivers/misc/Kconfig index 41d2bb0ae23a..e15c85d74c4b 100644 --- a/drivers/misc/Kconfig +++ b/drivers/misc/Kconfig @@ -483,6 +483,18 @@ config OPEN_DICE If unsure, say N. +config VCPU_STALL_DETECTOR + tristate "VCPU stall detector" + select LOCKUP_DETECTOR + help + Detect CPU locks on a kvm virtual machine. This driver relies on + the hrtimers which are CPU-binded to do the 'pet' operation. When a + vCPU has to do a 'pet', it exits the guest through MMIO write and + the backend driver takes into account the lost ticks for this + particular CPU. + To compile this driver as a module, choose M here: the + module will be called vcpu_stall_detector. + source "drivers/misc/c2port/Kconfig" source "drivers/misc/eeprom/Kconfig" source "drivers/misc/cb710/Kconfig" diff --git a/drivers/misc/Makefile b/drivers/misc/Makefile index 70e800e9127f..2be8542616dd 100644 --- a/drivers/misc/Makefile +++ b/drivers/misc/Makefile @@ -60,3 +60,4 @@ obj-$(CONFIG_XILINX_SDFEC) += xilinx_sdfec.o obj-$(CONFIG_HISI_HIKEY_USB) += hisi_hikey_usb.o obj-$(CONFIG_HI6421V600_IRQ) += hi6421v600-irq.o obj-$(CONFIG_OPEN_DICE) += open-dice.o +obj-$(CONFIG_VCPU_STALL_DETECTOR) += vcpu_stall_detector.o \ No newline at end of file diff --git a/drivers/misc/vcpu_stall_detector.c b/drivers/misc/vcpu_stall_detector.c new file mode 100644 index 000000000000..8b33f04a9719 --- /dev/null +++ b/drivers/misc/vcpu_stall_detector.c @@ -0,0 +1,222 @@ +// SPDX-License-Identifier: GPL-2.0 +// +// VCPU stall detector. +// Copyright (C) Google, 2022 + +#include <linux/cpu.h> +#include <linux/init.h> +#include <linux/io.h> +#include <linux/kernel.h> + +#include <linux/device.h> +#include <linux/interrupt.h> +#include <linux/module.h> +#include <linux/nmi.h> +#include <linux/of.h> +#include <linux/of_device.h> +#include <linux/param.h> +#include <linux/percpu.h> +#include <linux/platform_device.h> +#include <linux/slab.h> + +#define REG_STATUS (0x00) +#define REG_LOAD_CNT (0x04) +#define REG_CURRENT_CNT (0x08) +#define REG_CLOCK_FREQ_HZ (0x0C) +#define REG_LEN (0x10) + +#define DEFAULT_CLOCK_HZ (10) +#define DEFAULT_TIMEOT_SEC (8) + +struct vm_stall_detect_s { + void __iomem *membase; + u32 clock_freq; + u32 expiration_sec; + u32 ping_timeout_ms; + struct hrtimer per_cpu_hrtimer; + struct platform_device *dev; +}; + +#define vcpu_stall_detect_reg_write(stall_detect, reg, value) \ + iowrite32((value), (stall_detect)->membase + (reg)) +#define vcpu_stall_detect_reg_read(stall_detect, reg) \ + io32read((stall_detect)->membase + (reg)) + +static struct platform_device *virt_dev; + +static enum hrtimer_restart +vcpu_stall_detect_timer_fn(struct hrtimer *hrtimer) +{ + struct vm_stall_detect_s *cpu_stall_detect; + u32 ticks; + + cpu_stall_detect = container_of(hrtimer, struct vm_stall_detect_s, + per_cpu_hrtimer); + ticks = cpu_stall_detect->clock_freq * + cpu_stall_detect->expiration_sec; + vcpu_stall_detect_reg_write(cpu_stall_detect, REG_LOAD_CNT, ticks); + hrtimer_forward_now(hrtimer, + ms_to_ktime(cpu_stall_detect->ping_timeout_ms)); + + return HRTIMER_RESTART; +} + +static void vcpu_stall_detect_start(void *arg) +{ + u32 ticks; + struct vm_stall_detect_s *cpu_stall_detect = arg; + struct hrtimer *hrtimer = &cpu_stall_detect->per_cpu_hrtimer; + + vcpu_stall_detect_reg_write(cpu_stall_detect, REG_CLOCK_FREQ_HZ, + cpu_stall_detect->clock_freq); + + /* Compute the number of ticks required for the stall detector counter + * register based on the internal clock frequency and the timeout + * value given from the device tree. + */ + ticks = cpu_stall_detect->clock_freq * + cpu_stall_detect->expiration_sec; + vcpu_stall_detect_reg_write(cpu_stall_detect, REG_LOAD_CNT, ticks); + + /* Enable the internal clock and start the stall detector */ + vcpu_stall_detect_reg_write(cpu_stall_detect, REG_STATUS, 1); + + hrtimer_init(hrtimer, CLOCK_MONOTONIC, HRTIMER_MODE_REL); + hrtimer->function = vcpu_stall_detect_timer_fn; + hrtimer_start(hrtimer, ms_to_ktime(cpu_stall_detect->ping_timeout_ms), + HRTIMER_MODE_REL_PINNED); +} + +static void vcpu_stall_detect_stop(void *arg) +{ + struct vm_stall_detect_s *cpu_stall_detect = arg; + struct hrtimer *hrtimer = &cpu_stall_detect->per_cpu_hrtimer; + + hrtimer_cancel(hrtimer); + + /* Disable the stall detector */ + vcpu_stall_detect_reg_write(cpu_stall_detect, REG_STATUS, 0); +} + +static int start_stall_detector_on_cpu(unsigned int cpu) +{ + struct vm_stall_detect_s __percpu *vm_stall_detect; + + vm_stall_detect = (struct vm_stall_detect_s __percpu *) + platform_get_drvdata(virt_dev); + vcpu_stall_detect_start(this_cpu_ptr(vm_stall_detect)); + return 0; +} + +static int stop_stall_detector_on_cpu(unsigned int cpu) +{ + struct vm_stall_detect_s __percpu *vm_stall_detect; + + vm_stall_detect = (struct vm_stall_detect_s __percpu *) + platform_get_drvdata(virt_dev); + vcpu_stall_detect_stop(this_cpu_ptr(vm_stall_detect)); + return 0; +} + +static int vcpu_stall_detect_probe(struct platform_device *dev) +{ + int cpu, ret, err; + void __iomem *membase; + struct resource *r; + struct vm_stall_detect_s __percpu *vm_stall_detect; + u32 stall_detect_clock, stall_detect_timeout_sec = 0; + + r = platform_get_resource(dev, IORESOURCE_MEM, 0); + if (r == NULL) + return -ENOENT; + + vm_stall_detect = alloc_percpu(typeof(struct vm_stall_detect_s)); + if (!vm_stall_detect) + return -ENOMEM; + + membase = ioremap(r->start, resource_size(r)); + if (!membase) { + ret = -ENXIO; + goto err_withmem; + } + + virt_dev = dev; + platform_set_drvdata(dev, vm_stall_detect); + if (of_property_read_u32(dev->dev.of_node, "clock-frequency", + &stall_detect_clock)) + stall_detect_clock = DEFAULT_CLOCK_HZ; + + if (of_property_read_u32(dev->dev.of_node, "timeout-sec", + &stall_detect_timeout_sec)) + stall_detect_timeout_sec = DEFAULT_TIMEOT_SEC; + + for_each_cpu_and(cpu, cpu_online_mask, &watchdog_cpumask) { + struct vm_stall_detect_s *cpu_stall_detect; + + cpu_stall_detect = per_cpu_ptr(vm_stall_detect, cpu); + cpu_stall_detect->membase = membase + cpu * REG_LEN; + cpu_stall_detect->clock_freq = stall_detect_clock; + cpu_stall_detect->expiration_sec = stall_detect_timeout_sec; + cpu_stall_detect->ping_timeout_ms = stall_detect_timeout_sec * + MSEC_PER_SEC / 2; + smp_call_function_single(cpu, vcpu_stall_detect_start, + cpu_stall_detect, true); + } + + err = cpuhp_setup_state_nocalls(CPUHP_AP_ONLINE_DYN, + "virt/vcpu_stall_detector:online", + start_stall_detector_on_cpu, + stop_stall_detector_on_cpu); + if (err < 0) { + dev_warn(&dev->dev, "failed to install cpu hotplug"); + ret = err; + goto err_withmem; + } + + return 0; + +err_withmem: + free_percpu(vm_stall_detect); + return ret; +} + +static int vcpu_stall_detect_remove(struct platform_device *dev) +{ + int cpu; + struct vm_stall_detect_s __percpu *vm_stall_detect; + + vm_stall_detect = (struct vm_stall_detect_s __percpu *) + platform_get_drvdata(dev); + for_each_cpu_and(cpu, cpu_online_mask, &watchdog_cpumask) { + struct vm_stall_detect_s *cpu_stall_detect; + + cpu_stall_detect = per_cpu_ptr(vm_stall_detect, cpu); + smp_call_function_single(cpu, vcpu_stall_detect_stop, + cpu_stall_detect, true); + } + + free_percpu(vm_stall_detect); + return 0; +} + +static const struct of_device_id vcpu_stall_detect_of_match[] = { + { .compatible = "qemu,vcpu-stall-detector", }, + {} +}; + +MODULE_DEVICE_TABLE(of, vcpu_stall_detect_of_match); + +static struct platform_driver vcpu_stall_detect_driver = { + .probe = vcpu_stall_detect_probe, + .remove = vcpu_stall_detect_remove, + .driver = { + .name = KBUILD_MODNAME, + .of_match_table = vcpu_stall_detect_of_match, + }, +}; + +module_platform_driver(vcpu_stall_detect_driver); + +MODULE_LICENSE("GPL"); +MODULE_AUTHOR("Sebastian Ene <sebastianene@google.com>"); +MODULE_DESCRIPTION("VCPU stall detector"); -- 2.36.1.476.g0c4daa206d-goog ^ permalink raw reply related [flat|nested] 8+ messages in thread
* Re: [PATCH v7 2/2] misc: Add a mechanism to detect stalls on guest vCPUs 2022-06-21 8:03 ` [PATCH v7 2/2] misc: Add a mechanism to detect stalls on guest vCPUs Sebastian Ene @ 2022-06-21 8:27 ` Greg Kroah-Hartman 2022-06-21 8:44 ` Marc Zyngier 0 siblings, 1 reply; 8+ messages in thread From: Greg Kroah-Hartman @ 2022-06-21 8:27 UTC (permalink / raw) To: Sebastian Ene Cc: Rob Herring, Arnd Bergmann, Dragan Cvetic, linux-kernel, devicetree, maz, will, vdonnefort, Guenter Roeck On Tue, Jun 21, 2022 at 08:03:09AM +0000, Sebastian Ene wrote: > This driver creates per-cpu hrtimers which are required to do the > periodic 'pet' operation. On a conventional watchdog-core driver, the > userspace is responsible for delivering the 'pet' events by writing to > the particular /dev/watchdogN node. In this case we require a strong > thread affinity to be able to account for lost time on a per vCPU. > > This part of the driver is the 'frontend' which is reponsible for > delivering the periodic 'pet' events, configuring the virtual peripheral > and listening for cpu hotplug events. The other part of the driver > handles the peripheral emulation and this part accounts for lost time by > looking at the /proc/{}/task/{}/stat entries and is located here: > https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3548817 > > Signed-off-by: Sebastian Ene <sebastianene@google.com> > --- > drivers/misc/Kconfig | 12 ++ > drivers/misc/Makefile | 1 + > drivers/misc/vcpu_stall_detector.c | 222 +++++++++++++++++++++++++++++ > 3 files changed, 235 insertions(+) > create mode 100644 drivers/misc/vcpu_stall_detector.c > > diff --git a/drivers/misc/Kconfig b/drivers/misc/Kconfig > index 41d2bb0ae23a..e15c85d74c4b 100644 > --- a/drivers/misc/Kconfig > +++ b/drivers/misc/Kconfig > @@ -483,6 +483,18 @@ config OPEN_DICE > > If unsure, say N. > > +config VCPU_STALL_DETECTOR > + tristate "VCPU stall detector" > + select LOCKUP_DETECTOR > + help > + Detect CPU locks on a kvm virtual machine. This driver relies on > + the hrtimers which are CPU-binded to do the 'pet' operation. When a > + vCPU has to do a 'pet', it exits the guest through MMIO write and > + the backend driver takes into account the lost ticks for this > + particular CPU. > + To compile this driver as a module, choose M here: the > + module will be called vcpu_stall_detector. Should this depend on KVM_GUEST? thanks, greg k-h ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH v7 2/2] misc: Add a mechanism to detect stalls on guest vCPUs 2022-06-21 8:27 ` Greg Kroah-Hartman @ 2022-06-21 8:44 ` Marc Zyngier 2022-06-21 8:54 ` Greg Kroah-Hartman 0 siblings, 1 reply; 8+ messages in thread From: Marc Zyngier @ 2022-06-21 8:44 UTC (permalink / raw) To: Greg Kroah-Hartman Cc: Sebastian Ene, Rob Herring, Arnd Bergmann, Dragan Cvetic, linux-kernel, devicetree, will, vdonnefort, Guenter Roeck On 2022-06-21 09:27, Greg Kroah-Hartman wrote: > On Tue, Jun 21, 2022 at 08:03:09AM +0000, Sebastian Ene wrote: >> This driver creates per-cpu hrtimers which are required to do the >> periodic 'pet' operation. On a conventional watchdog-core driver, the >> userspace is responsible for delivering the 'pet' events by writing to >> the particular /dev/watchdogN node. In this case we require a strong >> thread affinity to be able to account for lost time on a per vCPU. >> >> This part of the driver is the 'frontend' which is reponsible for >> delivering the periodic 'pet' events, configuring the virtual >> peripheral >> and listening for cpu hotplug events. The other part of the driver >> handles the peripheral emulation and this part accounts for lost time >> by >> looking at the /proc/{}/task/{}/stat entries and is located here: >> https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3548817 >> >> Signed-off-by: Sebastian Ene <sebastianene@google.com> >> --- >> drivers/misc/Kconfig | 12 ++ >> drivers/misc/Makefile | 1 + >> drivers/misc/vcpu_stall_detector.c | 222 >> +++++++++++++++++++++++++++++ >> 3 files changed, 235 insertions(+) >> create mode 100644 drivers/misc/vcpu_stall_detector.c >> >> diff --git a/drivers/misc/Kconfig b/drivers/misc/Kconfig >> index 41d2bb0ae23a..e15c85d74c4b 100644 >> --- a/drivers/misc/Kconfig >> +++ b/drivers/misc/Kconfig >> @@ -483,6 +483,18 @@ config OPEN_DICE >> >> If unsure, say N. >> >> +config VCPU_STALL_DETECTOR >> + tristate "VCPU stall detector" >> + select LOCKUP_DETECTOR >> + help >> + Detect CPU locks on a kvm virtual machine. This driver relies on >> + the hrtimers which are CPU-binded to do the 'pet' operation. When >> a >> + vCPU has to do a 'pet', it exits the guest through MMIO write and >> + the backend driver takes into account the lost ticks for this >> + particular CPU. >> + To compile this driver as a module, choose M here: the >> + module will be called vcpu_stall_detector. > > Should this depend on KVM_GUEST? Not all architectures have KVM_GUEST, and arm64 has no use for it. Thanks, M. -- Jazz is not dead. It just smells funny... ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH v7 2/2] misc: Add a mechanism to detect stalls on guest vCPUs 2022-06-21 8:44 ` Marc Zyngier @ 2022-06-21 8:54 ` Greg Kroah-Hartman 2022-06-21 9:24 ` Marc Zyngier 0 siblings, 1 reply; 8+ messages in thread From: Greg Kroah-Hartman @ 2022-06-21 8:54 UTC (permalink / raw) To: Marc Zyngier Cc: Sebastian Ene, Rob Herring, Arnd Bergmann, Dragan Cvetic, linux-kernel, devicetree, will, vdonnefort, Guenter Roeck On Tue, Jun 21, 2022 at 09:44:35AM +0100, Marc Zyngier wrote: > On 2022-06-21 09:27, Greg Kroah-Hartman wrote: > > On Tue, Jun 21, 2022 at 08:03:09AM +0000, Sebastian Ene wrote: > > > This driver creates per-cpu hrtimers which are required to do the > > > periodic 'pet' operation. On a conventional watchdog-core driver, the > > > userspace is responsible for delivering the 'pet' events by writing to > > > the particular /dev/watchdogN node. In this case we require a strong > > > thread affinity to be able to account for lost time on a per vCPU. > > > > > > This part of the driver is the 'frontend' which is reponsible for > > > delivering the periodic 'pet' events, configuring the virtual > > > peripheral > > > and listening for cpu hotplug events. The other part of the driver > > > handles the peripheral emulation and this part accounts for lost > > > time by > > > looking at the /proc/{}/task/{}/stat entries and is located here: > > > https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3548817 > > > > > > Signed-off-by: Sebastian Ene <sebastianene@google.com> > > > --- > > > drivers/misc/Kconfig | 12 ++ > > > drivers/misc/Makefile | 1 + > > > drivers/misc/vcpu_stall_detector.c | 222 > > > +++++++++++++++++++++++++++++ > > > 3 files changed, 235 insertions(+) > > > create mode 100644 drivers/misc/vcpu_stall_detector.c > > > > > > diff --git a/drivers/misc/Kconfig b/drivers/misc/Kconfig > > > index 41d2bb0ae23a..e15c85d74c4b 100644 > > > --- a/drivers/misc/Kconfig > > > +++ b/drivers/misc/Kconfig > > > @@ -483,6 +483,18 @@ config OPEN_DICE > > > > > > If unsure, say N. > > > > > > +config VCPU_STALL_DETECTOR > > > + tristate "VCPU stall detector" > > > + select LOCKUP_DETECTOR > > > + help > > > + Detect CPU locks on a kvm virtual machine. This driver relies on > > > + the hrtimers which are CPU-binded to do the 'pet' operation. > > > When a > > > + vCPU has to do a 'pet', it exits the guest through MMIO write and > > > + the backend driver takes into account the lost ticks for this > > > + particular CPU. > > > + To compile this driver as a module, choose M here: the > > > + module will be called vcpu_stall_detector. > > > > Should this depend on KVM_GUEST? > > Not all architectures have KVM_GUEST, and arm64 has no use for it. Ah, I thought this was a requirement (or created a better guest image) for use under KVM. Nevermind then... thanks, greg k-h ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH v7 2/2] misc: Add a mechanism to detect stalls on guest vCPUs 2022-06-21 8:54 ` Greg Kroah-Hartman @ 2022-06-21 9:24 ` Marc Zyngier 0 siblings, 0 replies; 8+ messages in thread From: Marc Zyngier @ 2022-06-21 9:24 UTC (permalink / raw) To: Greg Kroah-Hartman Cc: Sebastian Ene, Rob Herring, Arnd Bergmann, Dragan Cvetic, linux-kernel, devicetree, will, vdonnefort, Guenter Roeck On Tue, 21 Jun 2022 09:54:35 +0100, Greg Kroah-Hartman <gregkh@linuxfoundation.org> wrote: > > On Tue, Jun 21, 2022 at 09:44:35AM +0100, Marc Zyngier wrote: > > On 2022-06-21 09:27, Greg Kroah-Hartman wrote: > > > On Tue, Jun 21, 2022 at 08:03:09AM +0000, Sebastian Ene wrote: > > > > This driver creates per-cpu hrtimers which are required to do the > > > > periodic 'pet' operation. On a conventional watchdog-core driver, the > > > > userspace is responsible for delivering the 'pet' events by writing to > > > > the particular /dev/watchdogN node. In this case we require a strong > > > > thread affinity to be able to account for lost time on a per vCPU. > > > > > > > > This part of the driver is the 'frontend' which is reponsible for > > > > delivering the periodic 'pet' events, configuring the virtual > > > > peripheral > > > > and listening for cpu hotplug events. The other part of the driver > > > > handles the peripheral emulation and this part accounts for lost > > > > time by > > > > looking at the /proc/{}/task/{}/stat entries and is located here: > > > > https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3548817 > > > > > > > > Signed-off-by: Sebastian Ene <sebastianene@google.com> > > > > --- > > > > drivers/misc/Kconfig | 12 ++ > > > > drivers/misc/Makefile | 1 + > > > > drivers/misc/vcpu_stall_detector.c | 222 > > > > +++++++++++++++++++++++++++++ > > > > 3 files changed, 235 insertions(+) > > > > create mode 100644 drivers/misc/vcpu_stall_detector.c > > > > > > > > diff --git a/drivers/misc/Kconfig b/drivers/misc/Kconfig > > > > index 41d2bb0ae23a..e15c85d74c4b 100644 > > > > --- a/drivers/misc/Kconfig > > > > +++ b/drivers/misc/Kconfig > > > > @@ -483,6 +483,18 @@ config OPEN_DICE > > > > > > > > If unsure, say N. > > > > > > > > +config VCPU_STALL_DETECTOR > > > > + tristate "VCPU stall detector" > > > > + select LOCKUP_DETECTOR > > > > + help > > > > + Detect CPU locks on a kvm virtual machine. This driver relies on > > > > + the hrtimers which are CPU-binded to do the 'pet' operation. > > > > When a > > > > + vCPU has to do a 'pet', it exits the guest through MMIO write and > > > > + the backend driver takes into account the lost ticks for this > > > > + particular CPU. > > > > + To compile this driver as a module, choose M here: the > > > > + module will be called vcpu_stall_detector. > > > > > > Should this depend on KVM_GUEST? > > > > Not all architectures have KVM_GUEST, and arm64 has no use for it. > > Ah, I thought this was a requirement (or created a better guest image) > for use under KVM. Nevermind then... It really depends whether an architecture relies on non-architectural extensions to support KVM guests. PPC does most of the time, x86 certainly works better with the knowledge that this is a KVM guest. KVM on arm64 implements the architecture itself, and hardly anything else (if something sucks in virt, it also likely sucks bare metal). The couple of KVM-specific options we support are definitely not worth a KVM_GUEST, as they only cover pretty esoteric stuff that nobody enables, such as PTP_1588_CLOCK_KVM. M. -- Without deviation from the norm, progress is not possible. ^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2022-06-24 17:27 UTC | newest] Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2022-06-21 8:03 [PATCH v7 0/2] Detect stalls on guest vCPUS Sebastian Ene 2022-06-21 8:03 ` [PATCH v7 1/2] dt-bindings: vcpu_stall_detector: Add qemu,vcpu-stall-detector compatible Sebastian Ene 2022-06-24 17:26 ` Rob Herring 2022-06-21 8:03 ` [PATCH v7 2/2] misc: Add a mechanism to detect stalls on guest vCPUs Sebastian Ene 2022-06-21 8:27 ` Greg Kroah-Hartman 2022-06-21 8:44 ` Marc Zyngier 2022-06-21 8:54 ` Greg Kroah-Hartman 2022-06-21 9:24 ` Marc Zyngier
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.