* [ANNOUNCE] v4.16.18-rt11 @ 2018-07-27 21:57 Sebastian Andrzej Siewior 2018-07-28 9:07 ` candidates for @devel-rt localversion-rt++ Mike Galbraith [not found] ` <1532764179.9882.14.camel@gmx.de> 0 siblings, 2 replies; 28+ messages in thread From: Sebastian Andrzej Siewior @ 2018-07-27 21:57 UTC (permalink / raw) To: Thomas Gleixner; +Cc: LKML, linux-rt-users, Steven Rostedt Dear RT folks! I'm pleased to announce the v4.16.18-rt11 patch set. Changes since v4.16.18-rt10: - Finally fix the SIMD locking on arm64. - Revert the srcu based notifier in crypto code. This is no longer required since the rwsem on RT allows multiple reader. - Add a per-CPU lock in crypto's cryptd per-CPU queue. Reported by Mike Galbraith. - Add a per-CPU lock in crypto's scompress code. Patch by Mike Galbraith. - Disable runtime EFI services on ARM64. It can be enabled again via command line. The getTime function needs around 10ms to complete on one of my machines. The setVariable might take even longer. It is disabled by default, it can be enabled via `efi=runtime' on the command line. Known issues - A warning triggered in "rcu_note_context_switch" originated from SyS_timer_gettime(). The issue was always there, it is now visible. Reported by Grygorii Strashko and Daniel Wagner. The delta patch against v4.16.18-rt10 is appended below and can be found here: https://cdn.kernel.org/pub/linux/kernel/projects/rt/4.16/incr/patch-4.16.18-rt10-rt11.patch.xz You can get this release via the git tree at: git://git.kernel.org/pub/scm/linux/kernel/git/rt/linux-rt-devel.git v4.16.18-rt11 The RT patch against v4.16.18 can be found here: https://cdn.kernel.org/pub/linux/kernel/projects/rt/4.16/older/patch-4.16.18-rt11.patch.xz The split quilt queue is available at: https://cdn.kernel.org/pub/linux/kernel/projects/rt/4.16/older/patches-4.16.18-rt11.tar.xz Sebastian ^ permalink raw reply [flat|nested] 28+ messages in thread
* candidates for @devel-rt localversion-rt++ 2018-07-27 21:57 [ANNOUNCE] v4.16.18-rt11 Sebastian Andrzej Siewior @ 2018-07-28 9:07 ` Mike Galbraith 2018-07-28 9:13 ` [rt-patch 1/3] arm64/acpi/perf: move pmu allocation to an early CPU up hook Mike Galbraith 2018-07-29 6:55 ` candidates for @devel-rt localversion-rt++ Mike Galbraith [not found] ` <1532764179.9882.14.camel@gmx.de> 1 sibling, 2 replies; 28+ messages in thread From: Mike Galbraith @ 2018-07-28 9:07 UTC (permalink / raw) To: Sebastian Andrzej Siewior, Thomas Gleixner Cc: LKML, linux-rt-users, Steven Rostedt 1. arm64/acpi/perf: move pmu allocation to an early CPU up hook 2. sched: Introduce raw_cond_resched_lock() 3. arm, KVM: convert vgic_irq.irq_lock to raw_spinlock_t With these applied, 4 socket TaiShan 2280 box boots shiny new -rt11 gripe free, and has been tossed into SUSE's kvm build-bot slave pit, where it is presumably performing acceptably, given its boss keeps giving it more work to do. (I see only lack of smoke/flame) 1 should fly, 2 and 3 may well die.. as box does without them. -Mike ^ permalink raw reply [flat|nested] 28+ messages in thread
* [rt-patch 1/3] arm64/acpi/perf: move pmu allocation to an early CPU up hook 2018-07-28 9:07 ` candidates for @devel-rt localversion-rt++ Mike Galbraith @ 2018-07-28 9:13 ` Mike Galbraith 2018-08-02 13:45 ` [rt-patch 1/3 v2] " Mike Galbraith 2018-07-29 6:55 ` candidates for @devel-rt localversion-rt++ Mike Galbraith 1 sibling, 1 reply; 28+ messages in thread From: Mike Galbraith @ 2018-07-28 9:13 UTC (permalink / raw) To: Sebastian Andrzej Siewior, Thomas Gleixner Cc: LKML, linux-rt-users, Steven Rostedt RT cannot allocate while irqs are disabled. BUG: sleeping function called from invalid context at kernel/locking/rtmutex.c:974 in_atomic(): 0, irqs_disabled(): 128, pid: 25, name: cpuhp/0 CPU: 0 PID: 25 Comm: cpuhp/0 Not tainted 4.16.18-rt10-rt #2 Hardware name: Huawei TaiShan 2280 /BC11SPCD, BIOS 1.32 08/22/2017 Call trace: dump_backtrace+0x0/0x188 show_stack+0x24/0x30 dump_stack+0x9c/0xd0 ___might_sleep+0x124/0x188 rt_spin_lock+0x40/0x80 pcpu_alloc+0x104/0x7a0 __alloc_percpu_gfp+0x38/0x48 __armpmu_alloc+0x44/0x168 armpmu_alloc_atomic+0x1c/0x28 arm_pmu_acpi_cpu_starting+0x1cc/0x210 cpuhp_invoke_callback+0xb8/0x820 cpuhp_thread_fun+0xc0/0x1e0 smpboot_thread_fn+0x1ac/0x2c8 kthread+0x134/0x138 ret_from_fork+0x10/0x18 Move the allocation to CPUHP_BP_PREPARE_DYN, where we'll be preemptible, thus no longer needing GFP_ATOMIC. Signed-off-by: Mike Galbraith <efault@gmx.de> --- drivers/perf/arm_pmu_acpi.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) --- a/drivers/perf/arm_pmu_acpi.c +++ b/drivers/perf/arm_pmu_acpi.c @@ -135,10 +135,10 @@ static struct arm_pmu *arm_pmu_acpi_find return pmu; } - pmu = armpmu_alloc_atomic(); + pmu = armpmu_alloc(); if (!pmu) { pr_warn("Unable to allocate PMU for CPU%d\n", - smp_processor_id()); + raw_smp_processor_id()); return NULL; } @@ -283,7 +283,7 @@ static int arm_pmu_acpi_init(void) if (ret) return ret; - ret = cpuhp_setup_state(CPUHP_AP_PERF_ARM_ACPI_STARTING, + ret = cpuhp_setup_state(CPUHP_BP_PREPARE_DYN, "perf/arm/pmu_acpi:starting", arm_pmu_acpi_cpu_starting, NULL); ^ permalink raw reply [flat|nested] 28+ messages in thread
* [rt-patch 1/3 v2] arm64/acpi/perf: move pmu allocation to an early CPU up hook 2018-07-28 9:13 ` [rt-patch 1/3] arm64/acpi/perf: move pmu allocation to an early CPU up hook Mike Galbraith @ 2018-08-02 13:45 ` Mike Galbraith 0 siblings, 0 replies; 28+ messages in thread From: Mike Galbraith @ 2018-08-02 13:45 UTC (permalink / raw) To: Sebastian Andrzej Siewior, Thomas Gleixner Cc: LKML, linux-rt-users, Steven Rostedt (bah, make a clean The End to adventures in arm-land) RT cannot allocate while irqs are disabled. BUG: sleeping function called from invalid context at kernel/locking/rtmutex.c:974 in_atomic(): 0, irqs_disabled(): 128, pid: 25, name: cpuhp/0 CPU: 0 PID: 25 Comm: cpuhp/0 Not tainted 4.16.18-rt10-rt #2 Hardware name: Huawei TaiShan 2280 /BC11SPCD, BIOS 1.32 08/22/2017 Call trace: dump_backtrace+0x0/0x188 show_stack+0x24/0x30 dump_stack+0x9c/0xd0 ___might_sleep+0x124/0x188 rt_spin_lock+0x40/0x80 pcpu_alloc+0x104/0x7a0 __alloc_percpu_gfp+0x38/0x48 __armpmu_alloc+0x44/0x168 armpmu_alloc_atomic+0x1c/0x28 arm_pmu_acpi_cpu_starting+0x1cc/0x210 cpuhp_invoke_callback+0xb8/0x820 cpuhp_thread_fun+0xc0/0x1e0 smpboot_thread_fn+0x1ac/0x2c8 kthread+0x134/0x138 ret_from_fork+0x10/0x18 Do the allocation and other preparation for probe along with the other CPUHP_PERF_{ARCH}_PREPARE stages, where we'll be preemptible, thus no longer requiring a GFP_ATOMIC allocation either. Signed-off-by: Mike Galbraith <efault@gmx.de> --- drivers/perf/arm_pmu_acpi.c | 12 ++++++------ include/linux/cpuhotplug.h | 2 +- 2 files changed, 7 insertions(+), 7 deletions(-) --- a/drivers/perf/arm_pmu_acpi.c +++ b/drivers/perf/arm_pmu_acpi.c @@ -135,10 +135,10 @@ static struct arm_pmu *arm_pmu_acpi_find return pmu; } - pmu = armpmu_alloc_atomic(); + pmu = armpmu_alloc(); if (!pmu) { pr_warn("Unable to allocate PMU for CPU%d\n", - smp_processor_id()); + raw_smp_processor_id()); return NULL; } @@ -185,7 +185,7 @@ static bool pmu_irq_matches(struct arm_p * coming up. The perf core won't open events while a hotplug event is in * progress. */ -static int arm_pmu_acpi_cpu_starting(unsigned int cpu) +static int arm_pmu_acpi_cpu_prepare(unsigned int cpu) { struct arm_pmu *pmu; struct pmu_hw_events __percpu *hw_events; @@ -283,9 +283,9 @@ static int arm_pmu_acpi_init(void) if (ret) return ret; - ret = cpuhp_setup_state(CPUHP_AP_PERF_ARM_ACPI_STARTING, - "perf/arm/pmu_acpi:starting", - arm_pmu_acpi_cpu_starting, NULL); + ret = cpuhp_setup_state(CPUHP_PERF_ARM_PMU_ACPI_PREPARE, + "perf/arm/pmu_acpi:prepare", + arm_pmu_acpi_cpu_prepare, NULL); return ret; } --- a/include/linux/cpuhotplug.h +++ b/include/linux/cpuhotplug.h @@ -29,6 +29,7 @@ enum cpuhp_state { CPUHP_PERF_PREPARE, CPUHP_PERF_X86_PREPARE, CPUHP_PERF_X86_AMD_UNCORE_PREP, + CPUHP_PERF_ARM_PMU_ACPI_PREPARE, CPUHP_PERF_BFIN, CPUHP_PERF_POWER, CPUHP_PERF_SUPERH, @@ -114,7 +115,6 @@ enum cpuhp_state { CPUHP_AP_ARM_VFP_STARTING, CPUHP_AP_ARM64_DEBUG_MONITORS_STARTING, CPUHP_AP_PERF_ARM_HW_BREAKPOINT_STARTING, - CPUHP_AP_PERF_ARM_ACPI_STARTING, CPUHP_AP_PERF_ARM_STARTING, CPUHP_AP_ARM_L2X0_STARTING, CPUHP_AP_ARM_ARCH_TIMER_STARTING, ^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: candidates for @devel-rt localversion-rt++ 2018-07-28 9:07 ` candidates for @devel-rt localversion-rt++ Mike Galbraith 2018-07-28 9:13 ` [rt-patch 1/3] arm64/acpi/perf: move pmu allocation to an early CPU up hook Mike Galbraith @ 2018-07-29 6:55 ` Mike Galbraith 2018-07-29 11:47 ` Mike Galbraith 1 sibling, 1 reply; 28+ messages in thread From: Mike Galbraith @ 2018-07-29 6:55 UTC (permalink / raw) To: Sebastian Andrzej Siewior, Thomas Gleixner Cc: LKML, linux-rt-users, Steven Rostedt On Sat, 2018-07-28 at 11:07 +0200, Mike Galbraith wrote: > 1. arm64/acpi/perf: move pmu allocation to an early CPU up hook Nope, it's an ex-candidate. Having found/run kvm unit tests, I discovered that while the above fixes boot time splat, it somehow manages to break kvm pmu tests, so needs staring at. -Mike ^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: candidates for @devel-rt localversion-rt++ 2018-07-29 6:55 ` candidates for @devel-rt localversion-rt++ Mike Galbraith @ 2018-07-29 11:47 ` Mike Galbraith 2018-07-30 16:24 ` Mike Galbraith 0 siblings, 1 reply; 28+ messages in thread From: Mike Galbraith @ 2018-07-29 11:47 UTC (permalink / raw) To: Sebastian Andrzej Siewior, Thomas Gleixner Cc: LKML, linux-rt-users, Steven Rostedt FYI, per kvm unit tests, 4.16-rt definitely has more kvm issues. huawei5:/abuild/mike/kvm-unit-tests # uname -r 4.16.18-rt11-rt huawei5:/abuild/mike/kvm-unit-tests # ./run_tests.sh PASS selftest-setup (2 tests) FAIL selftest-vectors-kernel FAIL selftest-vectors-user PASS selftest-smp (65 tests) PASS pci-test (1 tests) PASS pmu (3 tests) FAIL gicv2-ipi FAIL gicv3-ipi FAIL gicv2-active FAIL gicv3-active PASS psci (4 tests) FAIL timer huawei5:/abuild/mike/kvm-unit-tests # 4.14-rt passes all tests. The above is with the kvm raw_spinlock_t conversion patch applied, but the 4.12 based SLERT tree I cloned to explore arm-land in the first place shows only one timer failure, and has/needs it applied as well, which would seem to vindicate it. huawei5:/abuild/mike/kvm-unit-tests # uname -r 4.12.14-0.gec0b559-rt huawei5:/abuild/mike/kvm-unit-tests # ./run_tests.sh PASS selftest-setup (2 tests) PASS selftest-vectors-kernel (2 tests) PASS selftest-vectors-user (2 tests) PASS selftest-smp (65 tests) PASS pci-test (1 tests) PASS pmu (3 tests) PASS gicv2-ipi (3 tests) PASS gicv3-ipi (3 tests) PASS gicv2-active (1 tests) PASS gicv3-active (1 tests) PASS psci (4 tests) FAIL timer (8 tests, 1 unexpected failures) huawei5:/abuild/mike/kvm-unit-tests # ^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: candidates for @devel-rt localversion-rt++ 2018-07-29 11:47 ` Mike Galbraith @ 2018-07-30 16:24 ` Mike Galbraith 2018-07-31 18:28 ` bisected - arm64 kvm unit test failures Mike Galbraith 0 siblings, 1 reply; 28+ messages in thread From: Mike Galbraith @ 2018-07-30 16:24 UTC (permalink / raw) To: Sebastian Andrzej Siewior, Thomas Gleixner Cc: LKML, linux-rt-users, Steven Rostedt On Sun, 2018-07-29 at 13:47 +0200, Mike Galbraith wrote: > FYI, per kvm unit tests, 4.16-rt definitely has more kvm issues. > > huawei5:/abuild/mike/kvm-unit-tests # uname -r > 4.16.18-rt11-rt > huawei5:/abuild/mike/kvm-unit-tests # ./run_tests.sh > PASS selftest-setup (2 tests) > FAIL selftest-vectors-kernel > FAIL selftest-vectors-user > PASS selftest-smp (65 tests) > PASS pci-test (1 tests) > PASS pmu (3 tests) > FAIL gicv2-ipi > FAIL gicv3-ipi > FAIL gicv2-active > FAIL gicv3-active > PASS psci (4 tests) > FAIL timer > huawei5:/abuild/mike/kvm-unit-tests # > > 4.14-rt passes all tests. The above is with the kvm raw_spinlock_t > conversion patch applied, but the 4.12 based SLERT tree I cloned to > explore arm-land in the first place shows only one timer failure, and > has/needs it applied as well, which would seem to vindicate it. > > huawei5:/abuild/mike/kvm-unit-tests # uname -r > 4.12.14-0.gec0b559-rt > huawei5:/abuild/mike/kvm-unit-tests # ./run_tests.sh > PASS selftest-setup (2 tests) > PASS selftest-vectors-kernel (2 tests) > PASS selftest-vectors-user (2 tests) > PASS selftest-smp (65 tests) > PASS pci-test (1 tests) > PASS pmu (3 tests) > PASS gicv2-ipi (3 tests) > PASS gicv3-ipi (3 tests) > PASS gicv2-active (1 tests) > PASS gicv3-active (1 tests) > PASS psci (4 tests) > FAIL timer (8 tests, 1 unexpected failures) FWIW, this single timer failure wass inspired by something in the 4-15 merge window. A 4.14-rt based 4.15-rt updated to include recent fixes reproduces the exact same (ie my colleagues imported it into SLE). The rest landed in 4.16.. staring at which is not proving the least bit enlightening. -Mike ^ permalink raw reply [flat|nested] 28+ messages in thread
* bisected - arm64 kvm unit test failures 2018-07-30 16:24 ` Mike Galbraith @ 2018-07-31 18:28 ` Mike Galbraith 2018-08-01 5:35 ` Marc Zyngier 2018-08-21 15:34 ` Marc Zyngier 0 siblings, 2 replies; 28+ messages in thread From: Mike Galbraith @ 2018-07-31 18:28 UTC (permalink / raw) To: Sebastian Andrzej Siewior, Thomas Gleixner Cc: LKML, linux-rt-users, Steven Rostedt, Marc Zyngier [-- Attachment #1: Type: text/plain, Size: 5652 bytes --] On Mon, 2018-07-30 at 18:24 +0200, Mike Galbraith wrote: > On Sun, 2018-07-29 at 13:47 +0200, Mike Galbraith wrote: > > FYI, per kvm unit tests, 4.16-rt definitely has more kvm issues. But it's not RT, or rather most of it isn't... > > huawei5:/abuild/mike/kvm-unit-tests # uname -r > > 4.16.18-rt11-rt > > huawei5:/abuild/mike/kvm-unit-tests # ./run_tests.sh > > PASS selftest-setup (2 tests) > > FAIL selftest-vectors-kernel > > FAIL selftest-vectors-user > > PASS selftest-smp (65 tests) > > PASS pci-test (1 tests) > > PASS pmu (3 tests) > > FAIL gicv2-ipi > > FAIL gicv3-ipi > > FAIL gicv2-active > > FAIL gicv3-active > > PASS psci (4 tests) > > FAIL timer > > huawei5:/abuild/mike/kvm-unit-tests # > > > > 4.14-rt passes all tests. The above is with the kvm raw_spinlock_t > > conversion patch applied, but the 4.12 based SLERT tree I cloned to > > explore arm-land in the first place shows only one timer failure, and > > has/needs it applied as well, which would seem to vindicate it. > > > > huawei5:/abuild/mike/kvm-unit-tests # uname -r > > 4.12.14-0.gec0b559-rt > > huawei5:/abuild/mike/kvm-unit-tests # ./run_tests.sh > > PASS selftest-setup (2 tests) > > PASS selftest-vectors-kernel (2 tests) > > PASS selftest-vectors-user (2 tests) > > PASS selftest-smp (65 tests) > > PASS pci-test (1 tests) > > PASS pmu (3 tests) > > PASS gicv2-ipi (3 tests) > > PASS gicv3-ipi (3 tests) > > PASS gicv2-active (1 tests) > > PASS gicv3-active (1 tests) > > PASS psci (4 tests) > > FAIL timer (8 tests, 1 unexpected failures) > > FWIW, this single timer failure wass inspired by something in the 4-15 > merge window. As noted, the single timer failure is an RT issue of some sort, and remains. The rest I bisected in @stable with the attached config, and confirmed that revert fixes up 4.16-rt as well (modulo singleton). a9c0e12ebee56ef06b7eccdbc73bab71d0018df8 is the first bad commit commit a9c0e12ebee56ef06b7eccdbc73bab71d0018df8 Author: Marc Zyngier <marc.zyngier@arm.com> Date: Mon Oct 23 17:11:20 2017 +0100 KVM: arm/arm64: Only clean the dcache on translation fault The only case where we actually need to perform a dcache maintenance is when we map the page for the first time, and subsequent permission faults do not require cache maintenance. Let's make it conditional on not being a permission fault (and thus a translation fault). Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com> Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org> :040000 040000 951e77e6ec8df405f4bad59086ad1416c480ce3c 946dd071aa755606eaa5103d870bc471cc07258f M virt git bisect start # good: [a8ec862fd39d9adb88469eb8b9125daccc1c8335] Linux 4.15.18 git bisect good a8ec862fd39d9adb88469eb8b9125daccc1c8335 # bad: [62e9ccfaaedffd057e921fca976f9f7f71c9b254] Linux 4.16.18 git bisect bad 62e9ccfaaedffd057e921fca976f9f7f71c9b254 # good: [d8a5b80568a9cb66810e75b182018e9edb68e8ff] Linux 4.15 git bisect good d8a5b80568a9cb66810e75b182018e9edb68e8ff # good: [fe53d1443a146326b49d57fe6336b5c2a725223f] Merge tag 'armsoc-drivers' of git://git.kernel.org/pub/scm/linux/kernel/git/arm/arm-soc git bisect good fe53d1443a146326b49d57fe6336b5c2a725223f # good: [9a61df9e5f7471fe5be3e02bd0bed726b2761a54] Merge tag 'kbuild-v4.16-2' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild git bisect good 9a61df9e5f7471fe5be3e02bd0bed726b2761a54 # bad: [c4f4d2f917729e9b7b8bb452bf4971be93e7a15f] Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net git bisect bad c4f4d2f917729e9b7b8bb452bf4971be93e7a15f # bad: [97ace515f01439d4cf6e898b4094040dc12d36e7] Merge tag 'armsoc-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/arm/arm-soc git bisect bad 97ace515f01439d4cf6e898b4094040dc12d36e7 # bad: [9f416319f40cd857d2bb517630e5855a905ef3fb] arm64: fix unwind_frame() for filtered out fn for function graph tracing git bisect bad 9f416319f40cd857d2bb517630e5855a905ef3fb # bad: [405cacc947f7b58969b2a8ab1568c2d98b245308] drm/i915/vlv: Add cdclk workaround for DSI git bisect bad 405cacc947f7b58969b2a8ab1568c2d98b245308 # good: [810f4600ec5ee79c68dcbb136ed26a652df46348] Merge tag 'kvm-s390-next-4.16-3' of git://git.kernel.org/pub/scm/linux/kernel/git/kvms390/linux git bisect good 810f4600ec5ee79c68dcbb136ed26a652df46348 # bad: [1ab03c072feb579c9fd116de25be2b211e6bff6a] Merge tag 'kvm-ppc-next-4.16-2' of git://git.kernel.org/pub/scm/linux/kernel/git/paulus/powerpc git bisect bad 1ab03c072feb579c9fd116de25be2b211e6bff6a # bad: [87cedc6be55954c6efd6eca2e694132513f65a2a] kvm: x86: remove efer_reload entry in kvm_vcpu_stat git bisect bad 87cedc6be55954c6efd6eca2e694132513f65a2a # good: [fefb876b9b96fa7e4ed3d906979ea45b4cf07349] arm64: KVM: PTE/PMD S2 XN bit definition git bisect good fefb876b9b96fa7e4ed3d906979ea45b4cf07349 # bad: [b276f1b3b1eb52697f6645bb98f7d32ffb89df69] KVM: arm/arm64: Fix trailing semicolon git bisect bad b276f1b3b1eb52697f6645bb98f7d32ffb89df69 # bad: [17ab9d57debaa53d665651e425a0efc4a893c039] KVM: arm/arm64: Drop vcpu parameter from guest cache maintenance operartions git bisect bad 17ab9d57debaa53d665651e425a0efc4a893c039 # bad: [a9c0e12ebee56ef06b7eccdbc73bab71d0018df8] KVM: arm/arm64: Only clean the dcache on translation fault git bisect bad a9c0e12ebee56ef06b7eccdbc73bab71d0018df8 # good: [d0e22b4ac3ba23c611739f554392bf5e217df49f] KVM: arm/arm64: Limit icache invalidation to prefetch aborts git bisect good d0e22b4ac3ba23c611739f554392bf5e217df49f # first bad commit: [a9c0e12ebee56ef06b7eccdbc73bab71d0018df8] KVM: arm/arm64: Only clean the dcache on translation fault [-- Attachment #2: config.bisect.xz --] [-- Type: application/x-xz, Size: 26848 bytes --] ^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: bisected - arm64 kvm unit test failures 2018-07-31 18:28 ` bisected - arm64 kvm unit test failures Mike Galbraith @ 2018-08-01 5:35 ` Marc Zyngier 2018-08-01 6:02 ` Mike Galbraith 2018-08-21 15:34 ` Marc Zyngier 1 sibling, 1 reply; 28+ messages in thread From: Marc Zyngier @ 2018-08-01 5:35 UTC (permalink / raw) To: Mike Galbraith Cc: Sebastian Andrzej Siewior, Thomas Gleixner, LKML, linux-rt-users, Steven Rostedt On Tue, 31 Jul 2018 19:28:49 +0100, Mike Galbraith <efault@gmx.de> wrote: Hi Mike, > > [1 <text/plain; ISO-8859-15 (7bit)>] > On Mon, 2018-07-30 at 18:24 +0200, Mike Galbraith wrote: > > On Sun, 2018-07-29 at 13:47 +0200, Mike Galbraith wrote: > > > FYI, per kvm unit tests, 4.16-rt definitely has more kvm issues. > > But it's not RT, or rather most of it isn't... > > > > huawei5:/abuild/mike/kvm-unit-tests # uname -r > > > 4.16.18-rt11-rt > > > huawei5:/abuild/mike/kvm-unit-tests # ./run_tests.sh > > > PASS selftest-setup (2 tests) > > > FAIL selftest-vectors-kernel > > > FAIL selftest-vectors-user > > > PASS selftest-smp (65 tests) > > > PASS pci-test (1 tests) > > > PASS pmu (3 tests) > > > FAIL gicv2-ipi > > > FAIL gicv3-ipi > > > FAIL gicv2-active > > > FAIL gicv3-active > > > PASS psci (4 tests) > > > FAIL timer > > > huawei5:/abuild/mike/kvm-unit-tests # > > > > > > 4.14-rt passes all tests. The above is with the kvm raw_spinlock_t > > > conversion patch applied, but the 4.12 based SLERT tree I cloned to > > > explore arm-land in the first place shows only one timer failure, and > > > has/needs it applied as well, which would seem to vindicate it. > > > > > > huawei5:/abuild/mike/kvm-unit-tests # uname -r > > > 4.12.14-0.gec0b559-rt > > > huawei5:/abuild/mike/kvm-unit-tests # ./run_tests.sh > > > PASS selftest-setup (2 tests) > > > PASS selftest-vectors-kernel (2 tests) > > > PASS selftest-vectors-user (2 tests) > > > PASS selftest-smp (65 tests) > > > PASS pci-test (1 tests) > > > PASS pmu (3 tests) > > > PASS gicv2-ipi (3 tests) > > > PASS gicv3-ipi (3 tests) > > > PASS gicv2-active (1 tests) > > > PASS gicv3-active (1 tests) > > > PASS psci (4 tests) > > > FAIL timer (8 tests, 1 unexpected failures) > > > > FWIW, this single timer failure wass inspired by something in the 4-15 > > merge window. > > As noted, the single timer failure is an RT issue of some sort, and > remains. The rest I bisected in @stable with the attached config, and > confirmed that revert fixes up 4.16-rt as well (modulo singleton). Is it something that is reproducible with the current mainline (non-RT)? > > a9c0e12ebee56ef06b7eccdbc73bab71d0018df8 is the first bad commit > commit a9c0e12ebee56ef06b7eccdbc73bab71d0018df8 > Author: Marc Zyngier <marc.zyngier@arm.com> > Date: Mon Oct 23 17:11:20 2017 +0100 > > KVM: arm/arm64: Only clean the dcache on translation fault > > The only case where we actually need to perform a dcache maintenance > is when we map the page for the first time, and subsequent permission > faults do not require cache maintenance. Let's make it conditional > on not being a permission fault (and thus a translation fault). > > Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org> > Signed-off-by: Marc Zyngier <marc.zyngier@arm.com> > Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org> Pretty worrying. What HW is that on? M. -- Jazz is not dead, it just smell funny. ^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: bisected - arm64 kvm unit test failures 2018-08-01 5:35 ` Marc Zyngier @ 2018-08-01 6:02 ` Mike Galbraith 2018-08-01 7:22 ` Marc Zyngier 0 siblings, 1 reply; 28+ messages in thread From: Mike Galbraith @ 2018-08-01 6:02 UTC (permalink / raw) To: Marc Zyngier Cc: Sebastian Andrzej Siewior, Thomas Gleixner, LKML, linux-rt-users, Steven Rostedt [-- Attachment #1: Type: text/plain, Size: 626 bytes --] On Wed, 2018-08-01 at 06:35 +0100, Marc Zyngier wrote: > > Is it something that is reproducible with the current mainline (non-RT)? These waters are a bit muddy, it's config dependent. I'm trying to generate a reproducing !RT config for -rc7 as we speak. If I build openSUSE/master-default, it does NOT reproduce. That with the bisect config just finished building, and.... yup, it reproduced (attached). Every RT tree 4.16..master.today reproduces, and is fixed up (modulo the RT specific failure) by reverting the fingered commit. > Pretty worrying. What HW is that on? Box is a 4 node/64 core TaiShan 2280. -Mike [-- Attachment #2: config-4.18.0-rc7-bisect.xz --] [-- Type: application/x-xz, Size: 27428 bytes --] ^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: bisected - arm64 kvm unit test failures 2018-08-01 6:02 ` Mike Galbraith @ 2018-08-01 7:22 ` Marc Zyngier 2018-08-01 7:48 ` Mike Galbraith 2018-08-01 9:20 ` Mike Galbraith 0 siblings, 2 replies; 28+ messages in thread From: Marc Zyngier @ 2018-08-01 7:22 UTC (permalink / raw) To: Mike Galbraith Cc: Sebastian Andrzej Siewior, Thomas Gleixner, LKML, linux-rt-users, Steven Rostedt On Wed, 01 Aug 2018 07:02:25 +0100, Mike Galbraith <efault@gmx.de> wrote: > > [1 <text/plain; ISO-8859-15 (7bit)>] > On Wed, 2018-08-01 at 06:35 +0100, Marc Zyngier wrote: > > > > Is it something that is reproducible with the current mainline (non-RT)? > > These waters are a bit muddy, it's config dependent. I'm trying to > generate a reproducing !RT config for -rc7 as we speak. If I build > openSUSE/master-default, it does NOT reproduce. That with the bisect > config just finished building, and.... yup, it reproduced > (attached). Thanks for that. > > Every RT tree 4.16..master.today reproduces, and is fixed up (modulo > the RT specific failure) by reverting the fingered commit. > > > Pretty worrying. What HW is that on? > > Box is a 4 node/64 core TaiShan 2280. Is that what is also known as D05/HIP07, with 64 Cortex-A72? If so, I think we have one somewhere... I'll try to reproduce it once I'm back into the office (things seem to run just fine on my A72-based laptop, so it definitely smells of being config dependent). Cheers, M. -- Jazz is not dead, it just smell funny. ^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: bisected - arm64 kvm unit test failures 2018-08-01 7:22 ` Marc Zyngier @ 2018-08-01 7:48 ` Mike Galbraith 2018-08-01 9:20 ` Mike Galbraith 1 sibling, 0 replies; 28+ messages in thread From: Mike Galbraith @ 2018-08-01 7:48 UTC (permalink / raw) To: Marc Zyngier Cc: Sebastian Andrzej Siewior, Thomas Gleixner, LKML, linux-rt-users, Steven Rostedt On Wed, 2018-08-01 at 08:22 +0100, Marc Zyngier wrote: > > > Box is a 4 node/64 core TaiShan 2280. > > Is that what is also known as D05/HIP07, with 64 Cortex-A72? No idea, our rent-a-box web client shows nothing informative. -Mike ^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: bisected - arm64 kvm unit test failures 2018-08-01 7:22 ` Marc Zyngier 2018-08-01 7:48 ` Mike Galbraith @ 2018-08-01 9:20 ` Mike Galbraith 1 sibling, 0 replies; 28+ messages in thread From: Mike Galbraith @ 2018-08-01 9:20 UTC (permalink / raw) To: Marc Zyngier Cc: Sebastian Andrzej Siewior, Thomas Gleixner, LKML, linux-rt-users, Steven Rostedt On Wed, 2018-08-01 at 08:22 +0100, Marc Zyngier wrote: > On Wed, 01 Aug 2018 07:02:25 +0100, > Mike Galbraith <efault@gmx.de> wrote: > > > > [1 <text/plain; ISO-8859-15 (7bit)>] > > On Wed, 2018-08-01 at 06:35 +0100, Marc Zyngier wrote: > > > > > > Is it something that is reproducible with the current mainline (non-RT)? > > > > These waters are a bit muddy, it's config dependent. I'm trying to > > generate a reproducing !RT config for -rc7 as we speak. If I build > > openSUSE/master-default, it does NOT reproduce. That with the bisect > > config just finished building, and.... yup, it reproduced > > (attached). > > Thanks for that. Cheers. The dependency is THP, which is disabled in RT. --- config.save 2018-05-18 17:59:44.729165480 +0200 +++ .config 2018-08-01 11:00:24.484148316 +0200 @@ -632,7 +632,10 @@ CONFIG_DEFAULT_MMAP_MIN_ADDR=65536 CONFIG_ARCH_SUPPORTS_MEMORY_FAILURE=y CONFIG_MEMORY_FAILURE=y # CONFIG_HWPOISON_INJECT is not set -# CONFIG_TRANSPARENT_HUGEPAGE is not set +CONFIG_TRANSPARENT_HUGEPAGE=y +CONFIG_TRANSPARENT_HUGEPAGE_ALWAYS=y +# CONFIG_TRANSPARENT_HUGEPAGE_MADVISE is not set +CONFIG_TRANSPARENT_HUGE_PAGECACHE=y CONFIG_CLEANCACHE=y CONFIG_FRONTSWAP=y CONFIG_CMA=y @@ -4306,6 +4309,7 @@ CONFIG_RAS=y # CONFIG_ANDROID is not set # CONFIG_LIBNVDIMM is not set CONFIG_DAX=y +# CONFIG_DEV_DAX is not set CONFIG_NVMEM=y # CONFIG_MTK_EFUSE is not set # CONFIG_QCOM_QFPROM is not set @@ -5175,6 +5179,7 @@ CONFIG_DECOMPRESS_XZ=y CONFIG_DECOMPRESS_LZO=y CONFIG_DECOMPRESS_LZ4=y CONFIG_GENERIC_ALLOCATOR=y +CONFIG_RADIX_TREE_MULTIORDER=y CONFIG_ASSOCIATIVE_ARRAY=y CONFIG_HAS_IOMEM=y CONFIG_HAS_IOPORT_MAP=y ^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: bisected - arm64 kvm unit test failures 2018-07-31 18:28 ` bisected - arm64 kvm unit test failures Mike Galbraith 2018-08-01 5:35 ` Marc Zyngier @ 2018-08-21 15:34 ` Marc Zyngier 2018-08-22 13:38 ` Mike Galbraith 1 sibling, 1 reply; 28+ messages in thread From: Marc Zyngier @ 2018-08-21 15:34 UTC (permalink / raw) To: Mike Galbraith, Sebastian Andrzej Siewior, Thomas Gleixner Cc: LKML, linux-rt-users, Steven Rostedt Hi Mike, On 31/07/18 19:28, Mike Galbraith wrote: > On Mon, 2018-07-30 at 18:24 +0200, Mike Galbraith wrote: >> On Sun, 2018-07-29 at 13:47 +0200, Mike Galbraith wrote: >>> FYI, per kvm unit tests, 4.16-rt definitely has more kvm issues. > > But it's not RT, or rather most of it isn't... > >>> huawei5:/abuild/mike/kvm-unit-tests # uname -r >>> 4.16.18-rt11-rt >>> huawei5:/abuild/mike/kvm-unit-tests # ./run_tests.sh >>> PASS selftest-setup (2 tests) >>> FAIL selftest-vectors-kernel >>> FAIL selftest-vectors-user >>> PASS selftest-smp (65 tests) >>> PASS pci-test (1 tests) >>> PASS pmu (3 tests) >>> FAIL gicv2-ipi >>> FAIL gicv3-ipi >>> FAIL gicv2-active >>> FAIL gicv3-active >>> PASS psci (4 tests) >>> FAIL timer >>> huawei5:/abuild/mike/kvm-unit-tests # >>> >>> 4.14-rt passes all tests. The above is with the kvm raw_spinlock_t >>> conversion patch applied, but the 4.12 based SLERT tree I cloned to >>> explore arm-land in the first place shows only one timer failure, and >>> has/needs it applied as well, which would seem to vindicate it. >>> >>> huawei5:/abuild/mike/kvm-unit-tests # uname -r >>> 4.12.14-0.gec0b559-rt >>> huawei5:/abuild/mike/kvm-unit-tests # ./run_tests.sh >>> PASS selftest-setup (2 tests) >>> PASS selftest-vectors-kernel (2 tests) >>> PASS selftest-vectors-user (2 tests) >>> PASS selftest-smp (65 tests) >>> PASS pci-test (1 tests) >>> PASS pmu (3 tests) >>> PASS gicv2-ipi (3 tests) >>> PASS gicv3-ipi (3 tests) >>> PASS gicv2-active (1 tests) >>> PASS gicv3-active (1 tests) >>> PASS psci (4 tests) >>> FAIL timer (8 tests, 1 unexpected failures) >> >> FWIW, this single timer failure wass inspired by something in the 4-15 >> merge window. > > As noted, the single timer failure is an RT issue of some sort, and > remains. The rest I bisected in @stable with the attached config, and > confirmed that revert fixes up 4.16-rt as well (modulo singleton). Could you give that patchlet[1] a go? It solves a similar issue for me on a different platform. Thanks, M. [1] https://lists.cs.columbia.edu/pipermail/kvmarm/2018-August/032469.html -- Jazz is not dead. It just smells funny... ^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: bisected - arm64 kvm unit test failures 2018-08-21 15:34 ` Marc Zyngier @ 2018-08-22 13:38 ` Mike Galbraith 2018-08-22 13:50 ` Marc Zyngier 0 siblings, 1 reply; 28+ messages in thread From: Mike Galbraith @ 2018-08-22 13:38 UTC (permalink / raw) To: Marc Zyngier, Sebastian Andrzej Siewior, Thomas Gleixner Cc: LKML, linux-rt-users, Steven Rostedt On Tue, 2018-08-21 at 16:34 +0100, Marc Zyngier wrote: > Could you give that patchlet[1] a go? It solves a similar issue for me > on a different platform. > > [1] https://lists.cs.columbia.edu/pipermail/kvmarm/2018-August/032469.html Yup, all better. -Mike ^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: bisected - arm64 kvm unit test failures 2018-08-22 13:38 ` Mike Galbraith @ 2018-08-22 13:50 ` Marc Zyngier 2018-08-22 13:57 ` Mike Galbraith 0 siblings, 1 reply; 28+ messages in thread From: Marc Zyngier @ 2018-08-22 13:50 UTC (permalink / raw) To: Mike Galbraith, Sebastian Andrzej Siewior, Thomas Gleixner Cc: LKML, linux-rt-users, Steven Rostedt On 22/08/18 14:38, Mike Galbraith wrote: > On Tue, 2018-08-21 at 16:34 +0100, Marc Zyngier wrote: >> Could you give that patchlet[1] a go? It solves a similar issue for me >> on a different platform. >> >> [1] https://lists.cs.columbia.edu/pipermail/kvmarm/2018-August/032469.html > > Yup, all better. Now I'm not sure it actually fixes anything. Alex's test case seem to have opened a new can of worms... In tracing hell at the moment. Thanks, M. -- Jazz is not dead. It just smells funny... ^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: bisected - arm64 kvm unit test failures 2018-08-22 13:50 ` Marc Zyngier @ 2018-08-22 13:57 ` Mike Galbraith 0 siblings, 0 replies; 28+ messages in thread From: Mike Galbraith @ 2018-08-22 13:57 UTC (permalink / raw) To: Marc Zyngier, Sebastian Andrzej Siewior, Thomas Gleixner Cc: LKML, linux-rt-users, Steven Rostedt On Wed, 2018-08-22 at 14:50 +0100, Marc Zyngier wrote: > On 22/08/18 14:38, Mike Galbraith wrote: > > On Tue, 2018-08-21 at 16:34 +0100, Marc Zyngier wrote: > >> Could you give that patchlet[1] a go? It solves a similar issue for me > >> on a different platform. > >> > >> [1] https://lists.cs.columbia.edu/pipermail/kvmarm/2018-August/032469.html > > > > Yup, all better. > > Now I'm not sure it actually fixes anything. Alex's test case seem to > have opened a new can of worms... In tracing hell at the moment. I'll keep the test box reserved a while longer. When you get it's other leg stuffed into the casket, if you want testing, holler. -Mike ^ permalink raw reply [flat|nested] 28+ messages in thread
[parent not found: <1532764179.9882.14.camel@gmx.de>]
* [rt-patch 2/3] sched: Introduce raw_cond_resched_lock() [not found] ` <1532764179.9882.14.camel@gmx.de> @ 2018-07-28 9:07 ` Mike Galbraith 2018-07-28 9:07 ` [rt-patch 3/3] arm, KVM: convert vgic_irq.irq_lock to raw_spinlock_t Mike Galbraith 1 sibling, 0 replies; 28+ messages in thread From: Mike Galbraith @ 2018-07-28 9:07 UTC (permalink / raw) To: Sebastian Andrzej Siewior, Thomas Gleixner Cc: LKML, linux-rt-users, Steven Rostedt, Peter Zijlstra Add raw_cond_resched_lock() infrastructure. Signed-off-by: Mike Galbraith <efault@gmx.de> --- include/linux/sched.h | 15 +++++++++++++++ kernel/sched/core.c | 20 ++++++++++++++++++++ 2 files changed, 35 insertions(+) --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -1779,12 +1779,18 @@ static inline int _cond_resched(void) { }) extern int __cond_resched_lock(spinlock_t *lock); +extern int __raw_cond_resched_lock(raw_spinlock_t *lock); #define cond_resched_lock(lock) ({ \ ___might_sleep(__FILE__, __LINE__, PREEMPT_LOCK_OFFSET);\ __cond_resched_lock(lock); \ }) +#define raw_cond_resched_lock(lock) ({ \ + ___might_sleep(__FILE__, __LINE__, PREEMPT_LOCK_OFFSET);\ + __raw_cond_resched_lock(lock); \ +}) + #ifndef CONFIG_PREEMPT_RT_FULL extern int __cond_resched_softirq(void); @@ -1817,6 +1823,15 @@ static inline int spin_needbreak(spinloc #else return 0; #endif +} + +static inline int raw_spin_needbreak(raw_spinlock_t *lock) +{ +#ifdef CONFIG_PREEMPT + return raw_spin_is_contended(lock); +#else + return 0; +#endif } static __always_inline bool need_resched(void) --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -5065,6 +5065,26 @@ int __cond_resched_lock(spinlock_t *lock } EXPORT_SYMBOL(__cond_resched_lock); +int __raw_cond_resched_lock(raw_spinlock_t *lock) +{ + int resched = should_resched(PREEMPT_LOCK_OFFSET); + int ret = 0; + + lockdep_assert_held(lock); + + if (raw_spin_needbreak(lock) || resched) { + raw_spin_unlock(lock); + if (resched) + preempt_schedule_common(); + else + cpu_relax(); + ret = 1; + raw_spin_lock(lock); + } + return ret; +} +EXPORT_SYMBOL(__raw_cond_resched_lock); + #ifndef CONFIG_PREEMPT_RT_FULL int __sched __cond_resched_softirq(void) { ^ permalink raw reply [flat|nested] 28+ messages in thread
* [rt-patch 3/3] arm, KVM: convert vgic_irq.irq_lock to raw_spinlock_t [not found] ` <1532764179.9882.14.camel@gmx.de> 2018-07-28 9:07 ` [rt-patch 2/3] sched: Introduce raw_cond_resched_lock() Mike Galbraith @ 2018-07-28 9:07 ` Mike Galbraith 2018-07-30 9:27 ` Peter Zijlstra 2018-08-02 6:56 ` [rt-patch 4/3] arm,KVM: Move phys_timer handling to hard irq context Mike Galbraith 1 sibling, 2 replies; 28+ messages in thread From: Mike Galbraith @ 2018-07-28 9:07 UTC (permalink / raw) To: Sebastian Andrzej Siewior, Thomas Gleixner Cc: LKML, linux-rt-users, Steven Rostedt, Peter Zijlstra b103cc3f10c0 ("KVM: arm/arm64: Avoid timer save/restore in vcpu entry/exit") requires vgic_irq.irq_lock be converted to raw_spinlock_t. Problem: kvm_preempt_ops.sched_in = kvm_sched_in; kvm_sched_in() kvm_arch_vcpu_load() kvm_timer_vcpu_load() <- b103cc3f10c0 addition kvm_timer_vcpu_load_gic() kvm_vgic_map_is_active() spin_lock_irqsave(&irq->irq_lock, flags); Quoting virt/kvm/arm/vgic/vgic.c, locking order is... kvm->lock (mutex) its->cmd_lock (mutex) its->its_lock (mutex) vgic_cpu->ap_list_lock must be taken with IRQs disabled kvm->lpi_list_lock must be taken with IRQs disabled vgic_irq->irq_lock must be taken with IRQs disabled ...meaning vgic_dist.lpi_list_lock and vgic_cpu.ap_list_lock must be converted as well. Signed-off-by: Mike Galbraith <efault@gmx.de> --- include/kvm/arm_vgic.h | 6 - virt/kvm/arm/vgic/vgic-debug.c | 4 - virt/kvm/arm/vgic/vgic-init.c | 8 +- virt/kvm/arm/vgic/vgic-its.c | 22 +++---- virt/kvm/arm/vgic/vgic-mmio-v2.c | 14 ++-- virt/kvm/arm/vgic/vgic-mmio-v3.c | 10 +-- virt/kvm/arm/vgic/vgic-mmio.c | 34 +++++------ virt/kvm/arm/vgic/vgic-v2.c | 4 - virt/kvm/arm/vgic/vgic-v3.c | 8 +- virt/kvm/arm/vgic/vgic.c | 120 +++++++++++++++++++-------------------- 10 files changed, 115 insertions(+), 115 deletions(-) --- a/include/kvm/arm_vgic.h +++ b/include/kvm/arm_vgic.h @@ -96,7 +96,7 @@ enum vgic_irq_config { }; struct vgic_irq { - spinlock_t irq_lock; /* Protects the content of the struct */ + raw_spinlock_t irq_lock; /* Protects the content of the struct */ struct list_head lpi_list; /* Used to link all LPIs together */ struct list_head ap_list; @@ -243,7 +243,7 @@ struct vgic_dist { u64 propbaser; /* Protects the lpi_list and the count value below. */ - spinlock_t lpi_list_lock; + raw_spinlock_t lpi_list_lock; struct list_head lpi_list_head; int lpi_list_count; @@ -296,7 +296,7 @@ struct vgic_cpu { unsigned int used_lrs; struct vgic_irq private_irqs[VGIC_NR_PRIVATE_IRQS]; - spinlock_t ap_list_lock; /* Protects the ap_list */ + raw_spinlock_t ap_list_lock; /* Protects the ap_list */ /* * List of IRQs that this VCPU should consider because they are either --- a/virt/kvm/arm/vgic/vgic-debug.c +++ b/virt/kvm/arm/vgic/vgic-debug.c @@ -228,9 +228,9 @@ static int vgic_debug_show(struct seq_fi irq = &kvm->arch.vgic.spis[iter->intid - VGIC_NR_PRIVATE_IRQS]; } - spin_lock_irqsave(&irq->irq_lock, flags); + raw_spin_lock_irqsave(&irq->irq_lock, flags); print_irq_state(s, irq, vcpu); - spin_unlock_irqrestore(&irq->irq_lock, flags); + raw_spin_unlock_irqrestore(&irq->irq_lock, flags); return 0; } --- a/virt/kvm/arm/vgic/vgic-init.c +++ b/virt/kvm/arm/vgic/vgic-init.c @@ -64,7 +64,7 @@ void kvm_vgic_early_init(struct kvm *kvm struct vgic_dist *dist = &kvm->arch.vgic; INIT_LIST_HEAD(&dist->lpi_list_head); - spin_lock_init(&dist->lpi_list_lock); + raw_spin_lock_init(&dist->lpi_list_lock); } /** @@ -80,7 +80,7 @@ void kvm_vgic_vcpu_early_init(struct kvm int i; INIT_LIST_HEAD(&vgic_cpu->ap_list_head); - spin_lock_init(&vgic_cpu->ap_list_lock); + raw_spin_lock_init(&vgic_cpu->ap_list_lock); /* * Enable and configure all SGIs to be edge-triggered and @@ -90,7 +90,7 @@ void kvm_vgic_vcpu_early_init(struct kvm struct vgic_irq *irq = &vgic_cpu->private_irqs[i]; INIT_LIST_HEAD(&irq->ap_list); - spin_lock_init(&irq->irq_lock); + raw_spin_lock_init(&irq->irq_lock); irq->intid = i; irq->vcpu = NULL; irq->target_vcpu = vcpu; @@ -214,7 +214,7 @@ static int kvm_vgic_dist_init(struct kvm irq->intid = i + VGIC_NR_PRIVATE_IRQS; INIT_LIST_HEAD(&irq->ap_list); - spin_lock_init(&irq->irq_lock); + raw_spin_lock_init(&irq->irq_lock); irq->vcpu = NULL; irq->target_vcpu = vcpu0; kref_init(&irq->refcount); --- a/virt/kvm/arm/vgic/vgic-its.c +++ b/virt/kvm/arm/vgic/vgic-its.c @@ -65,14 +65,14 @@ static struct vgic_irq *vgic_add_lpi(str INIT_LIST_HEAD(&irq->lpi_list); INIT_LIST_HEAD(&irq->ap_list); - spin_lock_init(&irq->irq_lock); + raw_spin_lock_init(&irq->irq_lock); irq->config = VGIC_CONFIG_EDGE; kref_init(&irq->refcount); irq->intid = intid; irq->target_vcpu = vcpu; - spin_lock_irqsave(&dist->lpi_list_lock, flags); + raw_spin_lock_irqsave(&dist->lpi_list_lock, flags); /* * There could be a race with another vgic_add_lpi(), so we need to @@ -100,7 +100,7 @@ static struct vgic_irq *vgic_add_lpi(str dist->lpi_list_count++; out_unlock: - spin_unlock_irqrestore(&dist->lpi_list_lock, flags); + raw_spin_unlock_irqrestore(&dist->lpi_list_lock, flags); /* * We "cache" the configuration table entries in our struct vgic_irq's. @@ -287,7 +287,7 @@ static int update_lpi_config(struct kvm if (ret) return ret; - spin_lock_irqsave(&irq->irq_lock, flags); + raw_spin_lock_irqsave(&irq->irq_lock, flags); if (!filter_vcpu || filter_vcpu == irq->target_vcpu) { irq->priority = LPI_PROP_PRIORITY(prop); @@ -299,7 +299,7 @@ static int update_lpi_config(struct kvm } } - spin_unlock_irqrestore(&irq->irq_lock, flags); + raw_spin_unlock_irqrestore(&irq->irq_lock, flags); if (irq->hw) return its_prop_update_vlpi(irq->host_irq, prop, needs_inv); @@ -332,7 +332,7 @@ static int vgic_copy_lpi_list(struct kvm if (!intids) return -ENOMEM; - spin_lock_irqsave(&dist->lpi_list_lock, flags); + raw_spin_lock_irqsave(&dist->lpi_list_lock, flags); list_for_each_entry(irq, &dist->lpi_list_head, lpi_list) { if (i == irq_count) break; @@ -341,7 +341,7 @@ static int vgic_copy_lpi_list(struct kvm continue; intids[i++] = irq->intid; } - spin_unlock_irqrestore(&dist->lpi_list_lock, flags); + raw_spin_unlock_irqrestore(&dist->lpi_list_lock, flags); *intid_ptr = intids; return i; @@ -352,9 +352,9 @@ static int update_affinity(struct vgic_i int ret = 0; unsigned long flags; - spin_lock_irqsave(&irq->irq_lock, flags); + raw_spin_lock_irqsave(&irq->irq_lock, flags); irq->target_vcpu = vcpu; - spin_unlock_irqrestore(&irq->irq_lock, flags); + raw_spin_unlock_irqrestore(&irq->irq_lock, flags); if (irq->hw) { struct its_vlpi_map map; @@ -455,7 +455,7 @@ static int its_sync_lpi_pending_table(st } irq = vgic_get_irq(vcpu->kvm, NULL, intids[i]); - spin_lock_irqsave(&irq->irq_lock, flags); + raw_spin_lock_irqsave(&irq->irq_lock, flags); irq->pending_latch = pendmask & (1U << bit_nr); vgic_queue_irq_unlock(vcpu->kvm, irq, flags); vgic_put_irq(vcpu->kvm, irq); @@ -612,7 +612,7 @@ static int vgic_its_trigger_msi(struct k return irq_set_irqchip_state(irq->host_irq, IRQCHIP_STATE_PENDING, true); - spin_lock_irqsave(&irq->irq_lock, flags); + raw_spin_lock_irqsave(&irq->irq_lock, flags); irq->pending_latch = true; vgic_queue_irq_unlock(kvm, irq, flags); --- a/virt/kvm/arm/vgic/vgic-mmio-v2.c +++ b/virt/kvm/arm/vgic/vgic-mmio-v2.c @@ -100,7 +100,7 @@ static void vgic_mmio_write_sgir(struct irq = vgic_get_irq(source_vcpu->kvm, vcpu, intid); - spin_lock_irqsave(&irq->irq_lock, flags); + raw_spin_lock_irqsave(&irq->irq_lock, flags); irq->pending_latch = true; irq->source |= 1U << source_vcpu->vcpu_id; @@ -144,13 +144,13 @@ static void vgic_mmio_write_target(struc struct vgic_irq *irq = vgic_get_irq(vcpu->kvm, NULL, intid + i); int target; - spin_lock_irqsave(&irq->irq_lock, flags); + raw_spin_lock_irqsave(&irq->irq_lock, flags); irq->targets = (val >> (i * 8)) & cpu_mask; target = irq->targets ? __ffs(irq->targets) : 0; irq->target_vcpu = kvm_get_vcpu(vcpu->kvm, target); - spin_unlock_irqrestore(&irq->irq_lock, flags); + raw_spin_unlock_irqrestore(&irq->irq_lock, flags); vgic_put_irq(vcpu->kvm, irq); } } @@ -183,13 +183,13 @@ static void vgic_mmio_write_sgipendc(str for (i = 0; i < len; i++) { struct vgic_irq *irq = vgic_get_irq(vcpu->kvm, vcpu, intid + i); - spin_lock_irqsave(&irq->irq_lock, flags); + raw_spin_lock_irqsave(&irq->irq_lock, flags); irq->source &= ~((val >> (i * 8)) & 0xff); if (!irq->source) irq->pending_latch = false; - spin_unlock_irqrestore(&irq->irq_lock, flags); + raw_spin_unlock_irqrestore(&irq->irq_lock, flags); vgic_put_irq(vcpu->kvm, irq); } } @@ -205,7 +205,7 @@ static void vgic_mmio_write_sgipends(str for (i = 0; i < len; i++) { struct vgic_irq *irq = vgic_get_irq(vcpu->kvm, vcpu, intid + i); - spin_lock_irqsave(&irq->irq_lock, flags); + raw_spin_lock_irqsave(&irq->irq_lock, flags); irq->source |= (val >> (i * 8)) & 0xff; @@ -213,7 +213,7 @@ static void vgic_mmio_write_sgipends(str irq->pending_latch = true; vgic_queue_irq_unlock(vcpu->kvm, irq, flags); } else { - spin_unlock_irqrestore(&irq->irq_lock, flags); + raw_spin_unlock_irqrestore(&irq->irq_lock, flags); } vgic_put_irq(vcpu->kvm, irq); } --- a/virt/kvm/arm/vgic/vgic-mmio-v3.c +++ b/virt/kvm/arm/vgic/vgic-mmio-v3.c @@ -145,13 +145,13 @@ static void vgic_mmio_write_irouter(stru if (!irq) return; - spin_lock_irqsave(&irq->irq_lock, flags); + raw_spin_lock_irqsave(&irq->irq_lock, flags); /* We only care about and preserve Aff0, Aff1 and Aff2. */ irq->mpidr = val & GENMASK(23, 0); irq->target_vcpu = kvm_mpidr_to_vcpu(vcpu->kvm, irq->mpidr); - spin_unlock_irqrestore(&irq->irq_lock, flags); + raw_spin_unlock_irqrestore(&irq->irq_lock, flags); vgic_put_irq(vcpu->kvm, irq); } @@ -252,7 +252,7 @@ static void vgic_v3_uaccess_write_pendin for (i = 0; i < len * 8; i++) { struct vgic_irq *irq = vgic_get_irq(vcpu->kvm, vcpu, intid + i); - spin_lock_irqsave(&irq->irq_lock, flags); + raw_spin_lock_irqsave(&irq->irq_lock, flags); if (test_bit(i, &val)) { /* * pending_latch is set irrespective of irq type @@ -263,7 +263,7 @@ static void vgic_v3_uaccess_write_pendin vgic_queue_irq_unlock(vcpu->kvm, irq, flags); } else { irq->pending_latch = false; - spin_unlock_irqrestore(&irq->irq_lock, flags); + raw_spin_unlock_irqrestore(&irq->irq_lock, flags); } vgic_put_irq(vcpu->kvm, irq); @@ -845,7 +845,7 @@ void vgic_v3_dispatch_sgi(struct kvm_vcp irq = vgic_get_irq(vcpu->kvm, c_vcpu, sgi); - spin_lock_irqsave(&irq->irq_lock, flags); + raw_spin_lock_irqsave(&irq->irq_lock, flags); irq->pending_latch = true; vgic_queue_irq_unlock(vcpu->kvm, irq, flags); --- a/virt/kvm/arm/vgic/vgic-mmio.c +++ b/virt/kvm/arm/vgic/vgic-mmio.c @@ -75,7 +75,7 @@ void vgic_mmio_write_senable(struct kvm_ for_each_set_bit(i, &val, len * 8) { struct vgic_irq *irq = vgic_get_irq(vcpu->kvm, vcpu, intid + i); - spin_lock_irqsave(&irq->irq_lock, flags); + raw_spin_lock_irqsave(&irq->irq_lock, flags); irq->enabled = true; vgic_queue_irq_unlock(vcpu->kvm, irq, flags); @@ -94,11 +94,11 @@ void vgic_mmio_write_cenable(struct kvm_ for_each_set_bit(i, &val, len * 8) { struct vgic_irq *irq = vgic_get_irq(vcpu->kvm, vcpu, intid + i); - spin_lock_irqsave(&irq->irq_lock, flags); + raw_spin_lock_irqsave(&irq->irq_lock, flags); irq->enabled = false; - spin_unlock_irqrestore(&irq->irq_lock, flags); + raw_spin_unlock_irqrestore(&irq->irq_lock, flags); vgic_put_irq(vcpu->kvm, irq); } } @@ -115,10 +115,10 @@ unsigned long vgic_mmio_read_pending(str struct vgic_irq *irq = vgic_get_irq(vcpu->kvm, vcpu, intid + i); unsigned long flags; - spin_lock_irqsave(&irq->irq_lock, flags); + raw_spin_lock_irqsave(&irq->irq_lock, flags); if (irq_is_pending(irq)) value |= (1U << i); - spin_unlock_irqrestore(&irq->irq_lock, flags); + raw_spin_unlock_irqrestore(&irq->irq_lock, flags); vgic_put_irq(vcpu->kvm, irq); } @@ -170,7 +170,7 @@ void vgic_mmio_write_spending(struct kvm for_each_set_bit(i, &val, len * 8) { struct vgic_irq *irq = vgic_get_irq(vcpu->kvm, vcpu, intid + i); - spin_lock_irqsave(&irq->irq_lock, flags); + raw_spin_lock_irqsave(&irq->irq_lock, flags); if (irq->hw) vgic_hw_irq_spending(vcpu, irq, is_uaccess); else @@ -217,14 +217,14 @@ void vgic_mmio_write_cpending(struct kvm for_each_set_bit(i, &val, len * 8) { struct vgic_irq *irq = vgic_get_irq(vcpu->kvm, vcpu, intid + i); - spin_lock_irqsave(&irq->irq_lock, flags); + raw_spin_lock_irqsave(&irq->irq_lock, flags); if (irq->hw) vgic_hw_irq_cpending(vcpu, irq, is_uaccess); else irq->pending_latch = false; - spin_unlock_irqrestore(&irq->irq_lock, flags); + raw_spin_unlock_irqrestore(&irq->irq_lock, flags); vgic_put_irq(vcpu->kvm, irq); } } @@ -266,7 +266,7 @@ static void vgic_mmio_change_active(stru unsigned long flags; struct kvm_vcpu *requester_vcpu = vgic_get_mmio_requester_vcpu(); - spin_lock_irqsave(&irq->irq_lock, flags); + raw_spin_lock_irqsave(&irq->irq_lock, flags); /* * If this virtual IRQ was written into a list register, we @@ -287,7 +287,7 @@ static void vgic_mmio_change_active(stru while (irq->vcpu && /* IRQ may have state in an LR somewhere */ irq->vcpu != requester_vcpu && /* Current thread is not the VCPU thread */ irq->vcpu->cpu != -1) /* VCPU thread is running */ - cond_resched_lock(&irq->irq_lock); + raw_cond_resched_lock(&irq->irq_lock); if (irq->hw) vgic_hw_irq_change_active(vcpu, irq, active, !requester_vcpu); @@ -297,7 +297,7 @@ static void vgic_mmio_change_active(stru if (irq->active) vgic_queue_irq_unlock(vcpu->kvm, irq, flags); else - spin_unlock_irqrestore(&irq->irq_lock, flags); + raw_spin_unlock_irqrestore(&irq->irq_lock, flags); } /* @@ -436,10 +436,10 @@ void vgic_mmio_write_priority(struct kvm for (i = 0; i < len; i++) { struct vgic_irq *irq = vgic_get_irq(vcpu->kvm, vcpu, intid + i); - spin_lock_irqsave(&irq->irq_lock, flags); + raw_spin_lock_irqsave(&irq->irq_lock, flags); /* Narrow the priority range to what we actually support */ irq->priority = (val >> (i * 8)) & GENMASK(7, 8 - VGIC_PRI_BITS); - spin_unlock_irqrestore(&irq->irq_lock, flags); + raw_spin_unlock_irqrestore(&irq->irq_lock, flags); vgic_put_irq(vcpu->kvm, irq); } @@ -485,14 +485,14 @@ void vgic_mmio_write_config(struct kvm_v continue; irq = vgic_get_irq(vcpu->kvm, vcpu, intid + i); - spin_lock_irqsave(&irq->irq_lock, flags); + raw_spin_lock_irqsave(&irq->irq_lock, flags); if (test_bit(i * 2 + 1, &val)) irq->config = VGIC_CONFIG_EDGE; else irq->config = VGIC_CONFIG_LEVEL; - spin_unlock_irqrestore(&irq->irq_lock, flags); + raw_spin_unlock_irqrestore(&irq->irq_lock, flags); vgic_put_irq(vcpu->kvm, irq); } } @@ -541,12 +541,12 @@ void vgic_write_irq_line_level_info(stru * restore irq config before line level. */ new_level = !!(val & (1U << i)); - spin_lock_irqsave(&irq->irq_lock, flags); + raw_spin_lock_irqsave(&irq->irq_lock, flags); irq->line_level = new_level; if (new_level) vgic_queue_irq_unlock(vcpu->kvm, irq, flags); else - spin_unlock_irqrestore(&irq->irq_lock, flags); + raw_spin_unlock_irqrestore(&irq->irq_lock, flags); vgic_put_irq(vcpu->kvm, irq); } --- a/virt/kvm/arm/vgic/vgic-v2.c +++ b/virt/kvm/arm/vgic/vgic-v2.c @@ -85,7 +85,7 @@ void vgic_v2_fold_lr_state(struct kvm_vc irq = vgic_get_irq(vcpu->kvm, vcpu, intid); - spin_lock_irqsave(&irq->irq_lock, flags); + raw_spin_lock_irqsave(&irq->irq_lock, flags); /* Always preserve the active bit */ irq->active = !!(val & GICH_LR_ACTIVE_BIT); @@ -132,7 +132,7 @@ void vgic_v2_fold_lr_state(struct kvm_vc vgic_irq_set_phys_active(irq, false); } - spin_unlock_irqrestore(&irq->irq_lock, flags); + raw_spin_unlock_irqrestore(&irq->irq_lock, flags); vgic_put_irq(vcpu->kvm, irq); } --- a/virt/kvm/arm/vgic/vgic-v3.c +++ b/virt/kvm/arm/vgic/vgic-v3.c @@ -75,7 +75,7 @@ void vgic_v3_fold_lr_state(struct kvm_vc if (!irq) /* An LPI could have been unmapped. */ continue; - spin_lock_irqsave(&irq->irq_lock, flags); + raw_spin_lock_irqsave(&irq->irq_lock, flags); /* Always preserve the active bit */ irq->active = !!(val & ICH_LR_ACTIVE_BIT); @@ -123,7 +123,7 @@ void vgic_v3_fold_lr_state(struct kvm_vc vgic_irq_set_phys_active(irq, false); } - spin_unlock_irqrestore(&irq->irq_lock, flags); + raw_spin_unlock_irqrestore(&irq->irq_lock, flags); vgic_put_irq(vcpu->kvm, irq); } @@ -335,9 +335,9 @@ int vgic_v3_lpi_sync_pending_status(stru status = val & (1 << bit_nr); - spin_lock_irqsave(&irq->irq_lock, flags); + raw_spin_lock_irqsave(&irq->irq_lock, flags); if (irq->target_vcpu != vcpu) { - spin_unlock_irqrestore(&irq->irq_lock, flags); + raw_spin_unlock_irqrestore(&irq->irq_lock, flags); goto retry; } irq->pending_latch = status; --- a/virt/kvm/arm/vgic/vgic.c +++ b/virt/kvm/arm/vgic/vgic.c @@ -75,7 +75,7 @@ static struct vgic_irq *vgic_get_lpi(str struct vgic_irq *irq = NULL; unsigned long flags; - spin_lock_irqsave(&dist->lpi_list_lock, flags); + raw_spin_lock_irqsave(&dist->lpi_list_lock, flags); list_for_each_entry(irq, &dist->lpi_list_head, lpi_list) { if (irq->intid != intid) @@ -91,7 +91,7 @@ static struct vgic_irq *vgic_get_lpi(str irq = NULL; out_unlock: - spin_unlock_irqrestore(&dist->lpi_list_lock, flags); + raw_spin_unlock_irqrestore(&dist->lpi_list_lock, flags); return irq; } @@ -137,15 +137,15 @@ void vgic_put_irq(struct kvm *kvm, struc if (irq->intid < VGIC_MIN_LPI) return; - spin_lock_irqsave(&dist->lpi_list_lock, flags); + raw_spin_lock_irqsave(&dist->lpi_list_lock, flags); if (!kref_put(&irq->refcount, vgic_irq_release)) { - spin_unlock_irqrestore(&dist->lpi_list_lock, flags); + raw_spin_unlock_irqrestore(&dist->lpi_list_lock, flags); return; }; list_del(&irq->lpi_list); dist->lpi_list_count--; - spin_unlock_irqrestore(&dist->lpi_list_lock, flags); + raw_spin_unlock_irqrestore(&dist->lpi_list_lock, flags); kfree(irq); } @@ -195,7 +195,7 @@ void vgic_irq_set_phys_active(struct vgi */ static struct kvm_vcpu *vgic_target_oracle(struct vgic_irq *irq) { - DEBUG_SPINLOCK_BUG_ON(!spin_is_locked(&irq->irq_lock)); + DEBUG_SPINLOCK_BUG_ON(!raw_spin_is_locked(&irq->irq_lock)); /* If the interrupt is active, it must stay on the current vcpu */ if (irq->active) @@ -243,8 +243,8 @@ static int vgic_irq_cmp(void *priv, stru bool penda, pendb; int ret; - spin_lock(&irqa->irq_lock); - spin_lock_nested(&irqb->irq_lock, SINGLE_DEPTH_NESTING); + raw_spin_lock(&irqa->irq_lock); + raw_spin_lock_nested(&irqb->irq_lock, SINGLE_DEPTH_NESTING); if (irqa->active || irqb->active) { ret = (int)irqb->active - (int)irqa->active; @@ -262,8 +262,8 @@ static int vgic_irq_cmp(void *priv, stru /* Both pending and enabled, sort by priority */ ret = irqa->priority - irqb->priority; out: - spin_unlock(&irqb->irq_lock); - spin_unlock(&irqa->irq_lock); + raw_spin_unlock(&irqb->irq_lock); + raw_spin_unlock(&irqa->irq_lock); return ret; } @@ -272,7 +272,7 @@ static void vgic_sort_ap_list(struct kvm { struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu; - DEBUG_SPINLOCK_BUG_ON(!spin_is_locked(&vgic_cpu->ap_list_lock)); + DEBUG_SPINLOCK_BUG_ON(!raw_spin_is_locked(&vgic_cpu->ap_list_lock)); list_sort(NULL, &vgic_cpu->ap_list_head, vgic_irq_cmp); } @@ -310,7 +310,7 @@ bool vgic_queue_irq_unlock(struct kvm *k { struct kvm_vcpu *vcpu; - DEBUG_SPINLOCK_BUG_ON(!spin_is_locked(&irq->irq_lock)); + DEBUG_SPINLOCK_BUG_ON(!raw_spin_is_locked(&irq->irq_lock)); retry: vcpu = vgic_target_oracle(irq); @@ -324,7 +324,7 @@ bool vgic_queue_irq_unlock(struct kvm *k * not need to be inserted into an ap_list and there is also * no more work for us to do. */ - spin_unlock_irqrestore(&irq->irq_lock, flags); + raw_spin_unlock_irqrestore(&irq->irq_lock, flags); /* * We have to kick the VCPU here, because we could be @@ -346,12 +346,12 @@ bool vgic_queue_irq_unlock(struct kvm *k * We must unlock the irq lock to take the ap_list_lock where * we are going to insert this new pending interrupt. */ - spin_unlock_irqrestore(&irq->irq_lock, flags); + raw_spin_unlock_irqrestore(&irq->irq_lock, flags); /* someone can do stuff here, which we re-check below */ - spin_lock_irqsave(&vcpu->arch.vgic_cpu.ap_list_lock, flags); - spin_lock(&irq->irq_lock); + raw_spin_lock_irqsave(&vcpu->arch.vgic_cpu.ap_list_lock, flags); + raw_spin_lock(&irq->irq_lock); /* * Did something change behind our backs? @@ -366,10 +366,10 @@ bool vgic_queue_irq_unlock(struct kvm *k */ if (unlikely(irq->vcpu || vcpu != vgic_target_oracle(irq))) { - spin_unlock(&irq->irq_lock); - spin_unlock_irqrestore(&vcpu->arch.vgic_cpu.ap_list_lock, flags); + raw_spin_unlock(&irq->irq_lock); + raw_spin_unlock_irqrestore(&vcpu->arch.vgic_cpu.ap_list_lock, flags); - spin_lock_irqsave(&irq->irq_lock, flags); + raw_spin_lock_irqsave(&irq->irq_lock, flags); goto retry; } @@ -381,8 +381,8 @@ bool vgic_queue_irq_unlock(struct kvm *k list_add_tail(&irq->ap_list, &vcpu->arch.vgic_cpu.ap_list_head); irq->vcpu = vcpu; - spin_unlock(&irq->irq_lock); - spin_unlock_irqrestore(&vcpu->arch.vgic_cpu.ap_list_lock, flags); + raw_spin_unlock(&irq->irq_lock); + raw_spin_unlock_irqrestore(&vcpu->arch.vgic_cpu.ap_list_lock, flags); kvm_make_request(KVM_REQ_IRQ_PENDING, vcpu); kvm_vcpu_kick(vcpu); @@ -429,11 +429,11 @@ int kvm_vgic_inject_irq(struct kvm *kvm, if (!irq) return -EINVAL; - spin_lock_irqsave(&irq->irq_lock, flags); + raw_spin_lock_irqsave(&irq->irq_lock, flags); if (!vgic_validate_injection(irq, level, owner)) { /* Nothing to see here, move along... */ - spin_unlock_irqrestore(&irq->irq_lock, flags); + raw_spin_unlock_irqrestore(&irq->irq_lock, flags); vgic_put_irq(kvm, irq); return 0; } @@ -493,9 +493,9 @@ int kvm_vgic_map_phys_irq(struct kvm_vcp BUG_ON(!irq); - spin_lock_irqsave(&irq->irq_lock, flags); + raw_spin_lock_irqsave(&irq->irq_lock, flags); ret = kvm_vgic_map_irq(vcpu, irq, host_irq, get_input_level); - spin_unlock_irqrestore(&irq->irq_lock, flags); + raw_spin_unlock_irqrestore(&irq->irq_lock, flags); vgic_put_irq(vcpu->kvm, irq); return ret; @@ -518,11 +518,11 @@ void kvm_vgic_reset_mapped_irq(struct kv if (!irq->hw) goto out; - spin_lock_irqsave(&irq->irq_lock, flags); + raw_spin_lock_irqsave(&irq->irq_lock, flags); irq->active = false; irq->pending_latch = false; irq->line_level = false; - spin_unlock_irqrestore(&irq->irq_lock, flags); + raw_spin_unlock_irqrestore(&irq->irq_lock, flags); out: vgic_put_irq(vcpu->kvm, irq); } @@ -538,9 +538,9 @@ int kvm_vgic_unmap_phys_irq(struct kvm_v irq = vgic_get_irq(vcpu->kvm, vcpu, vintid); BUG_ON(!irq); - spin_lock_irqsave(&irq->irq_lock, flags); + raw_spin_lock_irqsave(&irq->irq_lock, flags); kvm_vgic_unmap_irq(irq); - spin_unlock_irqrestore(&irq->irq_lock, flags); + raw_spin_unlock_irqrestore(&irq->irq_lock, flags); vgic_put_irq(vcpu->kvm, irq); return 0; @@ -570,12 +570,12 @@ int kvm_vgic_set_owner(struct kvm_vcpu * return -EINVAL; irq = vgic_get_irq(vcpu->kvm, vcpu, intid); - spin_lock_irqsave(&irq->irq_lock, flags); + raw_spin_lock_irqsave(&irq->irq_lock, flags); if (irq->owner && irq->owner != owner) ret = -EEXIST; else irq->owner = owner; - spin_unlock_irqrestore(&irq->irq_lock, flags); + raw_spin_unlock_irqrestore(&irq->irq_lock, flags); return ret; } @@ -595,13 +595,13 @@ static void vgic_prune_ap_list(struct kv unsigned long flags; retry: - spin_lock_irqsave(&vgic_cpu->ap_list_lock, flags); + raw_spin_lock_irqsave(&vgic_cpu->ap_list_lock, flags); list_for_each_entry_safe(irq, tmp, &vgic_cpu->ap_list_head, ap_list) { struct kvm_vcpu *target_vcpu, *vcpuA, *vcpuB; bool target_vcpu_needs_kick = false; - spin_lock(&irq->irq_lock); + raw_spin_lock(&irq->irq_lock); BUG_ON(vcpu != irq->vcpu); @@ -614,7 +614,7 @@ static void vgic_prune_ap_list(struct kv */ list_del(&irq->ap_list); irq->vcpu = NULL; - spin_unlock(&irq->irq_lock); + raw_spin_unlock(&irq->irq_lock); /* * This vgic_put_irq call matches the @@ -629,14 +629,14 @@ static void vgic_prune_ap_list(struct kv if (target_vcpu == vcpu) { /* We're on the right CPU */ - spin_unlock(&irq->irq_lock); + raw_spin_unlock(&irq->irq_lock); continue; } /* This interrupt looks like it has to be migrated. */ - spin_unlock(&irq->irq_lock); - spin_unlock_irqrestore(&vgic_cpu->ap_list_lock, flags); + raw_spin_unlock(&irq->irq_lock); + raw_spin_unlock_irqrestore(&vgic_cpu->ap_list_lock, flags); /* * Ensure locking order by always locking the smallest @@ -650,10 +650,10 @@ static void vgic_prune_ap_list(struct kv vcpuB = vcpu; } - spin_lock_irqsave(&vcpuA->arch.vgic_cpu.ap_list_lock, flags); - spin_lock_nested(&vcpuB->arch.vgic_cpu.ap_list_lock, + raw_spin_lock_irqsave(&vcpuA->arch.vgic_cpu.ap_list_lock, flags); + raw_spin_lock_nested(&vcpuB->arch.vgic_cpu.ap_list_lock, SINGLE_DEPTH_NESTING); - spin_lock(&irq->irq_lock); + raw_spin_lock(&irq->irq_lock); /* * If the affinity has been preserved, move the @@ -673,9 +673,9 @@ static void vgic_prune_ap_list(struct kv target_vcpu_needs_kick = true; } - spin_unlock(&irq->irq_lock); - spin_unlock(&vcpuB->arch.vgic_cpu.ap_list_lock); - spin_unlock_irqrestore(&vcpuA->arch.vgic_cpu.ap_list_lock, flags); + raw_spin_unlock(&irq->irq_lock); + raw_spin_unlock(&vcpuB->arch.vgic_cpu.ap_list_lock); + raw_spin_unlock_irqrestore(&vcpuA->arch.vgic_cpu.ap_list_lock, flags); if (target_vcpu_needs_kick) { kvm_make_request(KVM_REQ_IRQ_PENDING, target_vcpu); @@ -685,7 +685,7 @@ static void vgic_prune_ap_list(struct kv goto retry; } - spin_unlock_irqrestore(&vgic_cpu->ap_list_lock, flags); + raw_spin_unlock_irqrestore(&vgic_cpu->ap_list_lock, flags); } static inline void vgic_fold_lr_state(struct kvm_vcpu *vcpu) @@ -700,7 +700,7 @@ static inline void vgic_fold_lr_state(st static inline void vgic_populate_lr(struct kvm_vcpu *vcpu, struct vgic_irq *irq, int lr) { - DEBUG_SPINLOCK_BUG_ON(!spin_is_locked(&irq->irq_lock)); + DEBUG_SPINLOCK_BUG_ON(!raw_spin_is_locked(&irq->irq_lock)); if (kvm_vgic_global_state.type == VGIC_V2) vgic_v2_populate_lr(vcpu, irq, lr); @@ -742,10 +742,10 @@ static int compute_ap_list_depth(struct *multi_sgi = false; - DEBUG_SPINLOCK_BUG_ON(!spin_is_locked(&vgic_cpu->ap_list_lock)); + DEBUG_SPINLOCK_BUG_ON(!raw_spin_is_locked(&vgic_cpu->ap_list_lock)); list_for_each_entry(irq, &vgic_cpu->ap_list_head, ap_list) { - spin_lock(&irq->irq_lock); + raw_spin_lock(&irq->irq_lock); /* GICv2 SGIs can count for more than one... */ if (vgic_irq_is_sgi(irq->intid) && irq->source) { int w = hweight8(irq->source); @@ -755,7 +755,7 @@ static int compute_ap_list_depth(struct } else { count++; } - spin_unlock(&irq->irq_lock); + raw_spin_unlock(&irq->irq_lock); } return count; } @@ -770,7 +770,7 @@ static void vgic_flush_lr_state(struct k bool multi_sgi; u8 prio = 0xff; - DEBUG_SPINLOCK_BUG_ON(!spin_is_locked(&vgic_cpu->ap_list_lock)); + DEBUG_SPINLOCK_BUG_ON(!raw_spin_is_locked(&vgic_cpu->ap_list_lock)); count = compute_ap_list_depth(vcpu, &multi_sgi); if (count > kvm_vgic_global_state.nr_lr || multi_sgi) @@ -779,7 +779,7 @@ static void vgic_flush_lr_state(struct k count = 0; list_for_each_entry(irq, &vgic_cpu->ap_list_head, ap_list) { - spin_lock(&irq->irq_lock); + raw_spin_lock(&irq->irq_lock); /* * If we have multi-SGIs in the pipeline, we need to @@ -789,7 +789,7 @@ static void vgic_flush_lr_state(struct k * the AP list has been sorted already. */ if (multi_sgi && irq->priority > prio) { - spin_unlock(&irq->irq_lock); + raw_spin_unlock(&irq->irq_lock); break; } @@ -802,7 +802,7 @@ static void vgic_flush_lr_state(struct k } } - spin_unlock(&irq->irq_lock); + raw_spin_unlock(&irq->irq_lock); if (count == kvm_vgic_global_state.nr_lr) { if (!list_is_last(&irq->ap_list, @@ -857,9 +857,9 @@ void kvm_vgic_flush_hwstate(struct kvm_v DEBUG_SPINLOCK_BUG_ON(!irqs_disabled()); - spin_lock(&vcpu->arch.vgic_cpu.ap_list_lock); + raw_spin_lock(&vcpu->arch.vgic_cpu.ap_list_lock); vgic_flush_lr_state(vcpu); - spin_unlock(&vcpu->arch.vgic_cpu.ap_list_lock); + raw_spin_unlock(&vcpu->arch.vgic_cpu.ap_list_lock); } void kvm_vgic_load(struct kvm_vcpu *vcpu) @@ -897,18 +897,18 @@ int kvm_vgic_vcpu_pending_irq(struct kvm if (vcpu->arch.vgic_cpu.vgic_v3.its_vpe.pending_last) return true; - spin_lock_irqsave(&vgic_cpu->ap_list_lock, flags); + raw_spin_lock_irqsave(&vgic_cpu->ap_list_lock, flags); list_for_each_entry(irq, &vgic_cpu->ap_list_head, ap_list) { - spin_lock(&irq->irq_lock); + raw_spin_lock(&irq->irq_lock); pending = irq_is_pending(irq) && irq->enabled; - spin_unlock(&irq->irq_lock); + raw_spin_unlock(&irq->irq_lock); if (pending) break; } - spin_unlock_irqrestore(&vgic_cpu->ap_list_lock, flags); + raw_spin_unlock_irqrestore(&vgic_cpu->ap_list_lock, flags); return pending; } @@ -940,9 +940,9 @@ bool kvm_vgic_map_is_active(struct kvm_v return false; irq = vgic_get_irq(vcpu->kvm, vcpu, vintid); - spin_lock_irqsave(&irq->irq_lock, flags); + raw_spin_lock_irqsave(&irq->irq_lock, flags); map_is_active = irq->hw && irq->active; - spin_unlock_irqrestore(&irq->irq_lock, flags); + raw_spin_unlock_irqrestore(&irq->irq_lock, flags); vgic_put_irq(vcpu->kvm, irq); return map_is_active; ^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [rt-patch 3/3] arm, KVM: convert vgic_irq.irq_lock to raw_spinlock_t 2018-07-28 9:07 ` [rt-patch 3/3] arm, KVM: convert vgic_irq.irq_lock to raw_spinlock_t Mike Galbraith @ 2018-07-30 9:27 ` Peter Zijlstra 2018-07-30 13:34 ` Mike Galbraith 2018-08-02 6:56 ` [rt-patch 4/3] arm,KVM: Move phys_timer handling to hard irq context Mike Galbraith 1 sibling, 1 reply; 28+ messages in thread From: Peter Zijlstra @ 2018-07-30 9:27 UTC (permalink / raw) To: Mike Galbraith Cc: Sebastian Andrzej Siewior, Thomas Gleixner, LKML, linux-rt-users, Steven Rostedt On Sat, Jul 28, 2018 at 11:07:33AM +0200, Mike Galbraith wrote: > > b103cc3f10c0 ("KVM: arm/arm64: Avoid timer save/restore in vcpu entry/exit") > requires vgic_irq.irq_lock be converted to raw_spinlock_t. > > Problem: kvm_preempt_ops.sched_in = kvm_sched_in; > kvm_sched_in() > kvm_arch_vcpu_load() > kvm_timer_vcpu_load() <- b103cc3f10c0 addition > kvm_timer_vcpu_load_gic() > kvm_vgic_map_is_active() > spin_lock_irqsave(&irq->irq_lock, flags); > > Quoting virt/kvm/arm/vgic/vgic.c, locking order is... > > kvm->lock (mutex) > its->cmd_lock (mutex) > its->its_lock (mutex) > vgic_cpu->ap_list_lock must be taken with IRQs disabled > kvm->lpi_list_lock must be taken with IRQs disabled > vgic_irq->irq_lock must be taken with IRQs disabled > > ...meaning vgic_dist.lpi_list_lock and vgic_cpu.ap_list_lock must be > converted as well. The thing missing from the Changelog is the analysis that all the work done under these locks is indeed properly bounded and cannot cause excessive latencies. ^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [rt-patch 3/3] arm, KVM: convert vgic_irq.irq_lock to raw_spinlock_t 2018-07-30 9:27 ` Peter Zijlstra @ 2018-07-30 13:34 ` Mike Galbraith 0 siblings, 0 replies; 28+ messages in thread From: Mike Galbraith @ 2018-07-30 13:34 UTC (permalink / raw) To: Peter Zijlstra Cc: Sebastian Andrzej Siewior, Thomas Gleixner, LKML, linux-rt-users, Steven Rostedt On Mon, 2018-07-30 at 11:27 +0200, Peter Zijlstra wrote: > > The thing missing from the Changelog is the analysis that all the work > done under these locks is indeed properly bounded and cannot cause > excessive latencies. True, I have no idea what worst case hold times are. Nothing poked me dead in the eye when looking around in completely alien code, nor did cyclictest inspire concern running on box with no base of comparison. I do know that latency is now < infinity, a modest improvement ;-) -Mike ^ permalink raw reply [flat|nested] 28+ messages in thread
* [rt-patch 4/3] arm,KVM: Move phys_timer handling to hard irq context 2018-07-28 9:07 ` [rt-patch 3/3] arm, KVM: convert vgic_irq.irq_lock to raw_spinlock_t Mike Galbraith 2018-07-30 9:27 ` Peter Zijlstra @ 2018-08-02 6:56 ` Mike Galbraith 2018-08-02 16:31 ` Steven Rostedt 1 sibling, 1 reply; 28+ messages in thread From: Mike Galbraith @ 2018-08-02 6:56 UTC (permalink / raw) To: Sebastian Andrzej Siewior, Thomas Gleixner Cc: LKML, linux-rt-users, Steven Rostedt, Peter Zijlstra (arm-land adventures 1/3 take2 will have to wait, my cup runeth over) v4.14..v4.15 timer handling changes including calling kvm_timer_vcpu_load() during kvm_preempt_ops.sched_in and taking vgic_dist.lpi_list_lock in the timer interrupt handler required locks for which locking rules/context had been changed be converted to raw_spinlock_t... Quoting virt/kvm/arm/vgic/vgic.c: * Locking order is always: * kvm->lock (mutex) * its->cmd_lock (mutex) * its->its_lock (mutex) * vgic_cpu->ap_list_lock must be taken with IRQs disabled * kvm->lpi_list_lock must be taken with IRQs disabled * vgic_irq->irq_lock must be taken with IRQs disabled * * As the ap_list_lock might be taken from the timer interrupt handler, * we have to disable IRQs before taking this lock and everything lower * than it. ...and fixed the obvious bricking consequence of those changes for RT, but left an RT specific kvm unit test timer failure in its wake. Handling phys_timer in hard interrupt context as expected cures that failure. Pre: PASS selftest-setup (2 tests) PASS selftest-vectors-kernel (2 tests) PASS selftest-vectors-user (2 tests) PASS selftest-smp (65 tests) PASS pci-test (1 tests) PASS pmu (3 tests) PASS gicv2-ipi (3 tests) PASS gicv3-ipi (3 tests) PASS gicv2-active (1 tests) PASS gicv3-active (1 tests) PASS psci (4 tests) FAIL timer (8 tests, 1 unexpected failures) Post: PASS selftest-setup (2 tests) PASS selftest-vectors-kernel (2 tests) PASS selftest-vectors-user (2 tests) PASS selftest-smp (65 tests) PASS pci-test (1 tests) PASS pmu (3 tests) PASS gicv2-ipi (3 tests) PASS gicv3-ipi (3 tests) PASS gicv2-active (1 tests) PASS gicv3-active (1 tests) PASS psci (4 tests) PASS timer (8 tests) Signed-off-by: Mike Galbraith <efault@gmx.de> --- virt/kvm/arm/arch_timer.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) --- a/virt/kvm/arm/arch_timer.c +++ b/virt/kvm/arm/arch_timer.c @@ -634,7 +634,7 @@ void kvm_timer_vcpu_init(struct kvm_vcpu hrtimer_init(&timer->bg_timer, CLOCK_MONOTONIC, HRTIMER_MODE_ABS); timer->bg_timer.function = kvm_bg_timer_expire; - hrtimer_init(&timer->phys_timer, CLOCK_MONOTONIC, HRTIMER_MODE_ABS); + hrtimer_init(&timer->phys_timer, CLOCK_MONOTONIC, HRTIMER_MODE_ABS_HARD); timer->phys_timer.function = kvm_phys_timer_expire; vtimer->irq.irq = default_vtimer_irq.irq; ^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [rt-patch 4/3] arm,KVM: Move phys_timer handling to hard irq context 2018-08-02 6:56 ` [rt-patch 4/3] arm,KVM: Move phys_timer handling to hard irq context Mike Galbraith @ 2018-08-02 16:31 ` Steven Rostedt 2018-08-02 16:50 ` Mike Galbraith 0 siblings, 1 reply; 28+ messages in thread From: Steven Rostedt @ 2018-08-02 16:31 UTC (permalink / raw) To: Mike Galbraith Cc: Sebastian Andrzej Siewior, Thomas Gleixner, LKML, linux-rt-users, Peter Zijlstra On Thu, 02 Aug 2018 08:56:20 +0200 Mike Galbraith <efault@gmx.de> wrote: > (arm-land adventures 1/3 take2 will have to wait, my cup runeth over) > > v4.14..v4.15 timer handling changes including calling kvm_timer_vcpu_load() I take it that this should be added to v4.16-rt and marked stable-rt? -- Steve > during kvm_preempt_ops.sched_in and taking vgic_dist.lpi_list_lock in the > timer interrupt handler required locks for which locking rules/context had > been changed be converted to raw_spinlock_t... > > Quoting virt/kvm/arm/vgic/vgic.c: > * Locking order is always: > * kvm->lock (mutex) > * its->cmd_lock (mutex) > * its->its_lock (mutex) > * vgic_cpu->ap_list_lock must be taken with IRQs disabled > * kvm->lpi_list_lock must be taken with IRQs disabled > * vgic_irq->irq_lock must be taken with IRQs disabled > * > * As the ap_list_lock might be taken from the timer interrupt handler, > * we have to disable IRQs before taking this lock and everything lower > * than it. > > ...and fixed the obvious bricking consequence of those changes for RT, > but left an RT specific kvm unit test timer failure in its wake. Handling > phys_timer in hard interrupt context as expected cures that failure. > > Pre: > PASS selftest-setup (2 tests) > PASS selftest-vectors-kernel (2 tests) > PASS selftest-vectors-user (2 tests) > PASS selftest-smp (65 tests) > PASS pci-test (1 tests) > PASS pmu (3 tests) > PASS gicv2-ipi (3 tests) > PASS gicv3-ipi (3 tests) > PASS gicv2-active (1 tests) > PASS gicv3-active (1 tests) > PASS psci (4 tests) > FAIL timer (8 tests, 1 unexpected failures) > > Post: > PASS selftest-setup (2 tests) > PASS selftest-vectors-kernel (2 tests) > PASS selftest-vectors-user (2 tests) > PASS selftest-smp (65 tests) > PASS pci-test (1 tests) > PASS pmu (3 tests) > PASS gicv2-ipi (3 tests) > PASS gicv3-ipi (3 tests) > PASS gicv2-active (1 tests) > PASS gicv3-active (1 tests) > PASS psci (4 tests) > PASS timer (8 tests) > > Signed-off-by: Mike Galbraith <efault@gmx.de> > --- > virt/kvm/arm/arch_timer.c | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > --- a/virt/kvm/arm/arch_timer.c > +++ b/virt/kvm/arm/arch_timer.c > @@ -634,7 +634,7 @@ void kvm_timer_vcpu_init(struct kvm_vcpu > hrtimer_init(&timer->bg_timer, CLOCK_MONOTONIC, HRTIMER_MODE_ABS); > timer->bg_timer.function = kvm_bg_timer_expire; > > - hrtimer_init(&timer->phys_timer, CLOCK_MONOTONIC, HRTIMER_MODE_ABS); > + hrtimer_init(&timer->phys_timer, CLOCK_MONOTONIC, HRTIMER_MODE_ABS_HARD); > timer->phys_timer.function = kvm_phys_timer_expire; > > vtimer->irq.irq = default_vtimer_irq.irq; ^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [rt-patch 4/3] arm,KVM: Move phys_timer handling to hard irq context 2018-08-02 16:31 ` Steven Rostedt @ 2018-08-02 16:50 ` Mike Galbraith 2018-08-02 17:43 ` Mike Galbraith 0 siblings, 1 reply; 28+ messages in thread From: Mike Galbraith @ 2018-08-02 16:50 UTC (permalink / raw) To: Steven Rostedt Cc: Sebastian Andrzej Siewior, Thomas Gleixner, LKML, linux-rt-users, Peter Zijlstra On Thu, 2018-08-02 at 12:31 -0400, Steven Rostedt wrote: > On Thu, 02 Aug 2018 08:56:20 +0200 > Mike Galbraith <efault@gmx.de> wrote: > > > (arm-land adventures 1/3 take2 will have to wait, my cup runeth over) > > > > v4.14..v4.15 timer handling changes including calling kvm_timer_vcpu_load() > > I take it that this should be added to v4.16-rt and marked stable-rt? Yeah, barring way sexier ideas of course. -Mike ^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [rt-patch 4/3] arm,KVM: Move phys_timer handling to hard irq context 2018-08-02 16:50 ` Mike Galbraith @ 2018-08-02 17:43 ` Mike Galbraith 2018-08-04 12:25 ` Mike Galbraith 0 siblings, 1 reply; 28+ messages in thread From: Mike Galbraith @ 2018-08-02 17:43 UTC (permalink / raw) To: Steven Rostedt Cc: Sebastian Andrzej Siewior, Thomas Gleixner, LKML, linux-rt-users, Peter Zijlstra On Thu, 2018-08-02 at 18:50 +0200, Mike Galbraith wrote: > On Thu, 2018-08-02 at 12:31 -0400, Steven Rostedt wrote: > > On Thu, 02 Aug 2018 08:56:20 +0200 > > Mike Galbraith <efault@gmx.de> wrote: > > > > > (arm-land adventures 1/3 take2 will have to wait, my cup runeth over) > > > > > > v4.14..v4.15 timer handling changes including calling kvm_timer_vcpu_load() > > > > I take it that this should be added to v4.16-rt and marked stable-rt? > > Yeah, barring way sexier ideas of course. Gah, wait. WRT marking for stable-rt, only the pmu allocation splat fix is applicable to @stable-rt, kvm locking woes start at 4.15. -Mike ^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [rt-patch 4/3] arm,KVM: Move phys_timer handling to hard irq context 2018-08-02 17:43 ` Mike Galbraith @ 2018-08-04 12:25 ` Mike Galbraith 2018-08-05 4:52 ` Mike Galbraith 0 siblings, 1 reply; 28+ messages in thread From: Mike Galbraith @ 2018-08-04 12:25 UTC (permalink / raw) To: Steven Rostedt Cc: Sebastian Andrzej Siewior, Thomas Gleixner, LKML, linux-rt-users, Peter Zijlstra On Thu, 2018-08-02 at 19:43 +0200, Mike Galbraith wrote: > On Thu, 2018-08-02 at 18:50 +0200, Mike Galbraith wrote: > > On Thu, 2018-08-02 at 12:31 -0400, Steven Rostedt wrote: > > > On Thu, 02 Aug 2018 08:56:20 +0200 > > > Mike Galbraith <efault@gmx.de> wrote: > > > > > > > (arm-land adventures 1/3 take2 will have to wait, my cup runeth over) > > > > > > > > v4.14..v4.15 timer handling changes including calling kvm_timer_vcpu_load() > > > > > > I take it that this should be added to v4.16-rt and marked stable-rt? > > > > Yeah, barring way sexier ideas of course. > > Gah, wait. WRT marking for stable-rt, only the pmu allocation splat > fix is applicable to @stable-rt, kvm locking woes start at 4.15. None of that series got picked up, so the pmu patch is moot as well. The warning is harmless anyway. Besides, there are more interesting fish in the arm64 sea than kvm. virgin 4.16.18-rt12-rt [ 537.236131] ITS queue timeout (65440 65504 4640) [ 537.236150] ITS cmd its_build_inv_cmd failed [ 537.236160] sched: RT throttling activated [ 1229.180124] ITS queue timeout (65440 65504 3744) [ 1229.180134] ITS cmd its_build_inv_cmd failed (converts those to trace_printk [invalidating virgin source warantee]) # tracer: nop # # _-----=> irqs-off # / _----=> need-resched # | / _----=> need-resched_lazy # || / _---=> hardirq/softirq # ||| / _--=> preempt-depth # |||| / delay # TASK-PID CPU# ||||| TIMESTAMP FUNCTION # | | | ||||| | | <idle>-0 [000] d..h2.. 84.967153: its_wait_for_range_completion+0xa8/0xe0: ITS queue timeout (65440 65504 96) <idle>-0 [000] d..h2.. 84.967155: its_send_single_command+0xd4/0x130: ITS cmd its_build_inv_cmd failed (starts cyclictest+kbuild) [ 9609.602489] NOHZ: local_softirq_pending 10 [10089.311713] NOHZ: local_softirq_pending 80 [10089.311735] NOHZ: local_softirq_pending 80 [10097.315809] NOHZ: local_softirq_pending 80 [10112.262436] NOHZ: local_softirq_pending 10 [10118.966066] NOHZ: local_softirq_pending 80 [10121.211106] NOHZ: local_softirq_pending 80 [10121.287093] NOHZ: local_softirq_pending 80 [10329.501202] NOHZ: local_softirq_pending 80 [10329.501231] NOHZ: local_softirq_pending 80 T:60 (17291) P:99 I:31000 C: 18128 Min: 5 Act: 12 Avg: 107 Max: 1581163 <== Thar she blows! irq/464-eth0-Tx-3955 [060] d...111 11015.059743: its_wait_for_range_completion+0xa8/0xe0: ITS queue timeout (65440 65504 5280) irq/464-eth0-Tx-3955 [060] d...111 11015.059749: <stack trace> => irq_chip_unmask_parent+0x2c/0x38 => its_unmask_msi_irq+0x28/0x38 => unmask_irq.part.4+0x30/0x50 => unmask_threaded_irq+0x4c/0x58 => irq_finalize_oneshot.part.2+0x9c/0x100 => irq_forced_thread_fn+0x60/0x98 => irq_thread+0x148/0x1e0 => kthread+0x140/0x148 => ret_from_fork+0x10/0x18 irq/464-eth0-Tx-3955 [060] d...111 11015.059751: its_send_single_command+0xd4/0x130: ITS cmd its_build_inv_cmd failed irq/464-eth0-Tx-3955 [060] d...111 11015.059752: <stack trace> => its_unmask_msi_irq+0x28/0x38 => unmask_irq.part.4+0x30/0x50 => unmask_threaded_irq+0x4c/0x58 => irq_finalize_oneshot.part.2+0x9c/0x100 => irq_forced_thread_fn+0x60/0x98 => irq_thread+0x148/0x1e0 => kthread+0x140/0x148 => ret_from_fork+0x10/0x18 irq/464-eth0-Tx-3955 [060] d..h311 11015.059764: update_curr_rt+0x280/0x320: sched: RT throttling activated irq/464-eth0-Tx-3955 [060] d..h311 11015.059765: <stack trace> => update_process_times+0x48/0x60 => tick_sched_handle.isra.5+0x34/0x70 => tick_sched_timer+0x54/0xa8 => __hrtimer_run_queues+0xf8/0x388 => hrtimer_interrupt+0xe4/0x258 => arch_timer_handler_phys+0x38/0x58 => handle_percpu_devid_irq+0xa8/0x2c8 => generic_handle_irq+0x34/0x50 => __handle_domain_irq+0x8c/0x100 => gic_handle_irq+0x80/0x18c => el1_irq+0xb4/0x13c => _raw_spin_unlock_irq+0x2c/0x78 => irq_finalize_oneshot.part.2+0x6c/0x100 => irq_forced_thread_fn+0x60/0x98 => irq_thread+0x148/0x1e0 => kthread+0x140/0x148 => ret_from_fork+0x10/0x18 <idle>-0 [008] d..h3.. 11015.374565: sched_rt_period_timer+0x2f0/0x420: sched: RT throttling deactivated <idle>-0 [008] d..h3.. 11015.374570: <stack trace> => arch_timer_handler_phys+0x38/0x58 => handle_percpu_devid_irq+0xa8/0x2c8 => generic_handle_irq+0x34/0x50 => __handle_domain_irq+0x8c/0x100 => gic_handle_irq+0x80/0x18c => el1_irq+0xb4/0x13c => arch_cpu_idle+0x2c/0x1d0 => default_idle_call+0x30/0x48 => do_idle+0x1c4/0x218 => cpu_startup_entry+0x2c/0x30 => secondary_start_kernel+0x184/0x1d0 ^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [rt-patch 4/3] arm,KVM: Move phys_timer handling to hard irq context 2018-08-04 12:25 ` Mike Galbraith @ 2018-08-05 4:52 ` Mike Galbraith 2018-08-07 14:46 ` Sebastian Andrzej Siewior 0 siblings, 1 reply; 28+ messages in thread From: Mike Galbraith @ 2018-08-05 4:52 UTC (permalink / raw) To: Steven Rostedt Cc: Sebastian Andrzej Siewior, Thomas Gleixner, LKML, linux-rt-users, Peter Zijlstra On Sat, 2018-08-04 at 14:25 +0200, Mike Galbraith wrote: > > Besides, there are more interesting fish in the arm64 sea than kvm. > > virgin 4.16.18-rt12-rt > > [ 537.236131] ITS queue timeout (65440 65504 4640) > [ 537.236150] ITS cmd its_build_inv_cmd failed FWIW, I had thought that something 4.16..master had made that business go entirely away, but it only became much more infrequent. So much for plan to slowly sneak up on whatever than was via ~bisect. ^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [rt-patch 4/3] arm,KVM: Move phys_timer handling to hard irq context 2018-08-05 4:52 ` Mike Galbraith @ 2018-08-07 14:46 ` Sebastian Andrzej Siewior 0 siblings, 0 replies; 28+ messages in thread From: Sebastian Andrzej Siewior @ 2018-08-07 14:46 UTC (permalink / raw) To: Mike Galbraith Cc: Steven Rostedt, Thomas Gleixner, LKML, linux-rt-users, Peter Zijlstra On 2018-08-05 06:52:57 [+0200], Mike Galbraith wrote: > > FWIW, I had thought that something 4.16..master had made that business > go entirely away, but it only became much more infrequent. So much for > plan to slowly sneak up on whatever than was via ~bisect. I just released v4.18 based RT. Please repost the kvm/RT patches once things have settled. Sebastian ^ permalink raw reply [flat|nested] 28+ messages in thread
end of thread, other threads:[~2018-08-22 13:57 UTC | newest] Thread overview: 28+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2018-07-27 21:57 [ANNOUNCE] v4.16.18-rt11 Sebastian Andrzej Siewior 2018-07-28 9:07 ` candidates for @devel-rt localversion-rt++ Mike Galbraith 2018-07-28 9:13 ` [rt-patch 1/3] arm64/acpi/perf: move pmu allocation to an early CPU up hook Mike Galbraith 2018-08-02 13:45 ` [rt-patch 1/3 v2] " Mike Galbraith 2018-07-29 6:55 ` candidates for @devel-rt localversion-rt++ Mike Galbraith 2018-07-29 11:47 ` Mike Galbraith 2018-07-30 16:24 ` Mike Galbraith 2018-07-31 18:28 ` bisected - arm64 kvm unit test failures Mike Galbraith 2018-08-01 5:35 ` Marc Zyngier 2018-08-01 6:02 ` Mike Galbraith 2018-08-01 7:22 ` Marc Zyngier 2018-08-01 7:48 ` Mike Galbraith 2018-08-01 9:20 ` Mike Galbraith 2018-08-21 15:34 ` Marc Zyngier 2018-08-22 13:38 ` Mike Galbraith 2018-08-22 13:50 ` Marc Zyngier 2018-08-22 13:57 ` Mike Galbraith [not found] ` <1532764179.9882.14.camel@gmx.de> 2018-07-28 9:07 ` [rt-patch 2/3] sched: Introduce raw_cond_resched_lock() Mike Galbraith 2018-07-28 9:07 ` [rt-patch 3/3] arm, KVM: convert vgic_irq.irq_lock to raw_spinlock_t Mike Galbraith 2018-07-30 9:27 ` Peter Zijlstra 2018-07-30 13:34 ` Mike Galbraith 2018-08-02 6:56 ` [rt-patch 4/3] arm,KVM: Move phys_timer handling to hard irq context Mike Galbraith 2018-08-02 16:31 ` Steven Rostedt 2018-08-02 16:50 ` Mike Galbraith 2018-08-02 17:43 ` Mike Galbraith 2018-08-04 12:25 ` Mike Galbraith 2018-08-05 4:52 ` Mike Galbraith 2018-08-07 14:46 ` Sebastian Andrzej Siewior
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).