Linux-rt-users archive on lore.kernel.org
 help / color / Atom feed
From: Rasmus Villemoes <rasmus.villemoes@prevas.dk>
To: Steven Rostedt <rostedt@goodmis.org>,
	linux-kernel@vger.kernel.org,
	linux-rt-users <linux-rt-users@vger.kernel.org>
Cc: Thomas Gleixner <tglx@linutronix.de>,
	Carsten Emde <C.Emde@osadl.org>,
	Sebastian Andrzej Siewior <bigeasy@linutronix.de>,
	John Kacur <jkacur@redhat.com>, Julia Cartwright <julia@ni.com>,
	Daniel Wagner <wagi@monom.org>, Tom Zanussi <zanussi@kernel.org>,
	Julien Grall <julien.grall@arm.com>
Subject: Re: [PATCH RT 10/30] hrtimer: Prevent using hrtimer_grab_expiry_lock() on migration_base
Date: Mon, 27 Apr 2020 15:10:00 +0200
Message-ID: <ac4168c7-68d6-e558-644e-e0daceea1f61@prevas.dk> (raw)
In-Reply-To: <20200123203943.749508731@goodmis.org>

On 23/01/2020 21.39, Steven Rostedt wrote:
> 4.19.94-rt39-rc2 stable review patch.
> If anyone has any objections, please let me know.
> 
> ------------------
> 
> From: Julien Grall <julien.grall@arm.com>
> 
> [ Upstream commit cef1b87f98823af923a386f3f69149acb212d4a1 ]
> 
> As tglx puts it:
> |If base == migration_base then there is no point to lock soft_expiry_lock
> |simply because the timer is not executing the callback in soft irq context
> |and the whole lock/unlock dance can be avoided.
> 
> Furthermore, all the path leading to hrtimer_grab_expiry_lock() assumes
> timer->base and timer->base->cpu_base are always non-NULL. So it is safe
> to remove the NULL checks here.
> 
> Signed-off-by: Julien Grall <julien.grall@arm.com>
> Link: https://lkml.kernel.org/r/alpine.DEB.2.21.1908211557420.2223@nanos.tec.linutronix.de
> Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
> [bigeasy: rewrite changelog]
> Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>

Pretty late to the party, but I think I've bisected a problem to this
patch (and its required fixup for !SMP, "hrtimer: Add a missing bracket
and hide `migration_base on !SMP").

Originally, a customer reported that upgrading from 4.19.82-rt30 to
v4.19.106-rt45 failed to boot, stalling around the time the network gets
initialized (this is a board with an embedded Marvell switch). Perhaps 1
in 10 times, the board would come up successfully. I haven't been able
to reproduce that particular problem (or, perhaps I've seen it once or
twice, but not nearly often enough to use that as a basis for bisection).

However, building with their rescue initrd and booting that, the board
would consistently hang during reboot. Sometimes I would get lines like

[   72.956630] sched: RT throttling activated
[   72.973769] lanx: port 1(lan1) entered disabled state
[   73.000401] lanx: port 2(lan2) entered disabled state
[   73.974951] lanx: port 3(lan3) entered disabled state
[   73.997473] lanx: port 4(lan4) entered disabled state
[   74.968006] lanx: port 5(lan5) entered disabled state

other times there would be no output, but the board was still hanging.
Reverting

b1a471ec4df1 - hrtimer: Prevent using hrtimer_grab_expiry_lock() on
migration_base
40aae5708e7a - hrtimer: Add a missing bracket and hide `migration_base'
on !SMP

on top of v4.19.94-rt39 makes that problem go away, i.e. the board
reboots as expected.

The board is a 32 bit powerpc (mpc8309) !SMP. Any ideas what I can do to
debug this further?

Thanks,
Rasmus

  reply index

Thread overview: 38+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-01-23 20:39 [PATCH RT 00/30] Linux 4.19.94-rt39-rc2 Steven Rostedt
2020-01-23 20:39 ` [PATCH RT 01/30] i2c: exynos5: Remove IRQF_ONESHOT Steven Rostedt
2020-01-23 20:39 ` [PATCH RT 02/30] i2c: hix5hd2: " Steven Rostedt
2020-01-23 20:39 ` [PATCH RT 03/30] sched/deadline: Ensure inactive_timer runs in hardirq context Steven Rostedt
2020-01-23 20:39 ` [PATCH RT 04/30] thermal/x86_pkg_temp: make pkg_temp_lock a raw spinlock Steven Rostedt
2020-01-23 20:39 ` [PATCH RT 05/30] dma-buf: Use seqlock_t instread disabling preemption Steven Rostedt
2020-01-23 20:39 ` [PATCH RT 06/30] KVM: arm/arm64: Let the timer expire in hardirq context on RT Steven Rostedt
2020-01-23 20:39 ` [PATCH RT 07/30] x86: preempt: Check preemption level before looking at lazy-preempt Steven Rostedt
2020-01-23 20:39 ` [PATCH RT 08/30] hrtimer: Use READ_ONCE to access timer->base in hrimer_grab_expiry_lock() Steven Rostedt
2020-01-23 20:39 ` [PATCH RT 09/30] hrtimer: Dont grab the expiry lock for non-soft hrtimer Steven Rostedt
2020-01-23 20:39 ` [PATCH RT 10/30] hrtimer: Prevent using hrtimer_grab_expiry_lock() on migration_base Steven Rostedt
2020-04-27 13:10   ` Rasmus Villemoes [this message]
2020-04-27 19:06     ` Steven Rostedt
2020-04-27 19:26       ` Tom Zanussi
2020-04-28  6:51         ` Rasmus Villemoes
2020-04-28  7:03   ` Rasmus Villemoes
2020-04-28 12:59     ` Tom Zanussi
2020-04-28 13:07       ` Rasmus Villemoes
2020-04-28 13:43         ` Tom Zanussi
2020-01-23 20:39 ` [PATCH RT 11/30] hrtimer: Add a missing bracket and hide `migration_base on !SMP Steven Rostedt
2020-01-23 20:39 ` [PATCH RT 12/30] posix-timers: Unlock expiry lock in the early return Steven Rostedt
2020-01-23 20:39 ` [PATCH RT 14/30] sched: __set_cpus_allowed_ptr: Check cpus_mask, not cpus_ptr Steven Rostedt
2020-01-23 20:39 ` [PATCH RT 15/30] sched: Remove dead __migrate_disabled() check Steven Rostedt
2020-01-23 20:39 ` [PATCH RT 16/30] sched: migrate disable: Protect cpus_ptr with lock Steven Rostedt
2020-01-23 20:39 ` [PATCH RT 17/30] lib/smp_processor_id: Dont use cpumask_equal() Steven Rostedt
2020-01-23 20:39 ` [PATCH RT 18/30] futex: Make the futex_hash_bucket spinlock_t again and bring back its old state Steven Rostedt
2020-01-23 20:39 ` [PATCH RT 19/30] locking/rtmutex: Clean ->pi_blocked_on in the error case Steven Rostedt
2020-01-23 20:39 ` [PATCH RT 20/30] lib/ubsan: Dont seralize UBSAN report Steven Rostedt
2020-01-23 20:39 ` [PATCH RT 21/30] kmemleak: Change the lock of kmemleak_object to raw_spinlock_t Steven Rostedt
2020-01-23 20:39 ` [PATCH RT 22/30] sched: migrate_enable: Use select_fallback_rq() Steven Rostedt
2020-01-23 20:39 ` [PATCH RT 23/30] sched: Lazy migrate_disable processing Steven Rostedt
2020-01-23 20:39 ` [PATCH RT 24/30] sched: migrate_enable: Use stop_one_cpu_nowait() Steven Rostedt
2020-01-23 20:39 ` [PATCH RT 25/30] Revert "ARM: Initialize split page table locks for vector page" Steven Rostedt
2020-01-23 20:39 ` [PATCH RT 26/30] locking: Make spinlock_t and rwlock_t a RCU section on RT Steven Rostedt
2020-01-23 20:39 ` [PATCH RT 27/30] sched/core: migrate_enable() must access takedown_cpu_task on !HOTPLUG_CPU Steven Rostedt
2020-01-23 20:39 ` [PATCH RT 28/30] lib/smp_processor_id: Adjust check_preemption_disabled() Steven Rostedt
2020-01-23 20:39 ` [PATCH RT 29/30] sched: migrate_enable: Busy loop until the migration request is completed Steven Rostedt
2020-01-23 20:40 ` [PATCH RT 30/30] Linux 4.19.94-rt39-rc2 Steven Rostedt

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=ac4168c7-68d6-e558-644e-e0daceea1f61@prevas.dk \
    --to=rasmus.villemoes@prevas.dk \
    --cc=C.Emde@osadl.org \
    --cc=bigeasy@linutronix.de \
    --cc=jkacur@redhat.com \
    --cc=julia@ni.com \
    --cc=julien.grall@arm.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-rt-users@vger.kernel.org \
    --cc=rostedt@goodmis.org \
    --cc=tglx@linutronix.de \
    --cc=wagi@monom.org \
    --cc=zanussi@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Linux-rt-users archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-rt-users/0 linux-rt-users/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-rt-users linux-rt-users/ https://lore.kernel.org/linux-rt-users \
		linux-rt-users@vger.kernel.org
	public-inbox-index linux-rt-users

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-rt-users


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git