All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH 00/36] hrtimer: Provide softirq context hrtimers
@ 2017-11-29 15:30 Anna-Maria Gleixner
  2017-11-29 15:30 ` [PATCH v3 01/36] timers: Use static keys for migrate_enable/nohz_active Anna-Maria Gleixner
                   ` (35 more replies)
  0 siblings, 36 replies; 45+ messages in thread
From: Anna-Maria Gleixner @ 2017-11-29 15:30 UTC (permalink / raw)
  To: LKML
  Cc: Thomas Gleixner, Peter Zijlstra, Ingo Molnar, keescook,
	Christoph Hellwig, John Stultz

There are quite some places in the kernel which use a combination of
hrtimers and tasklets to make use of the precise expiry of hrtimers, which
schedule a tasklet to bring the actual function into softirq context.

This was introduced when the previous hrtimer softirq code was
removed. That code was implemented by expiring the timer in hard irq
context and then deferring the execution of the callback into softirq
context. That caused a lot of pointless shuffling between the rbtree and a
linked list.

In recent discussions it turned out that more potential users of hrtimers
in softirq context might come up. Aside of that the RT patches need this
functionality as well to defer hrtimers into softirq context if their
callbacks are not interrupt safe on RT.

This series implements a new approach by adding SOFT hrtimer mode and
instead of doing the list shuffle, timers started with this mode are put
into separate soft expiry hrtimer queues. These queues are evaluated only
when the hardirq context detects that the first expiring timer in the
softirq queues has expired. That makes the overhead in the hardirq context
minimal.

The series reworks the code to reuse as much as possible from the existing
facilities for the new softirq hrtimers and integrates them with all
flavours of hrtimers (HIGH_RES=y/n - NOHZ=y/n).

To achieve this quite some of the conditionals in the existing code are
removed for the price of adding some pointless data and state tracking to
the HIGH_RES=n case. That's minimal, but well worth it as it increases the
readability and maintainability of the code.

The first part of the series implements the new functionality and the
second part converts the hrtimer/tasklet users to make use of it and
removes struct hrtimer_tasklet and the surrounding helper functions.

This series is available from git as well:

	git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git WIP.timers

Thanks,

        Anna-Maria

---
v2..v3:

  - Use static keys for migrate_enable and nohz_active
  - fix missing struct documentation
  - integrate Sebastian Siewiors suggestions and fix bug he mentioned:
    https://lkml.kernel.org/r/20171110124224.ykr6n4z7zgypojnb@breakpoint.cc


v1..v2:

  - integration of Peter Zijlstras patch:
    https://lkml.kernel.org/r/20170927164025.GI17526@worktop.programming.kicks-ass.net
  - using hrtimer_mode instead of additional hrtimer clock bases
  - folding the fix for updating the base offsets:
    https://lkml.kernel.org/r/20171006102820.ou4wpm56ed6m3ewr@linutronix.de
  - rework of 08/25 - 10/25 (all of those patches are facing reduction of
    conditional code) and make hrtimer_force_reprogram() unconditional as well
  - integration of new versions of "ALSA/dummy: Replace tasklet with
    softirq hrtimer" and "net/cdc_ncm: Replace tasklet with softirq
    hrtimer"
  - additional hrtimer/tasklet user conversion: "net/mvpp2: Replace tasklet
    with softirq hrtimer"
  - additional fixes of several wrong comments
  - update hrtimer tracing (mode and clock bases)


 drivers/net/ethernet/marvell/mvpp2.c  |   62 +--
 drivers/net/wireless/mac80211_hwsim.c |   44 +-
 drivers/usb/gadget/function/f_ncm.c   |   30 -
 include/linux/hrtimer.h               |  113 +++--
 include/linux/interrupt.h             |   25 -
 include/net/xfrm.h                    |    2 
 include/trace/events/timer.h          |   37 +
 kernel/softirq.c                      |   51 --
 kernel/time/hrtimer.c                 |  643 +++++++++++++++++++++-------------
 kernel/time/tick-internal.h           |   13 
 kernel/time/tick-sched.c              |    2 
 kernel/time/timer.c                   |   98 ++---
 net/can/bcm.c                         |  156 ++------
 net/xfrm/xfrm_state.c                 |   30 -
 sound/drivers/dummy.c                 |   27 -
 15 files changed, 702 insertions(+), 631 deletions(-)

^ permalink raw reply	[flat|nested] 45+ messages in thread

* [PATCH v3 01/36] timers: Use static keys for migrate_enable/nohz_active
  2017-11-29 15:30 [PATCH 00/36] hrtimer: Provide softirq context hrtimers Anna-Maria Gleixner
@ 2017-11-29 15:30 ` Anna-Maria Gleixner
  2017-11-29 15:30 ` [PATCH v3 02/36] hrtimer: Correct blantanly wrong comment Anna-Maria Gleixner
                   ` (34 subsequent siblings)
  35 siblings, 0 replies; 45+ messages in thread
From: Anna-Maria Gleixner @ 2017-11-29 15:30 UTC (permalink / raw)
  To: LKML
  Cc: Thomas Gleixner, Peter Zijlstra, Ingo Molnar, keescook,
	Christoph Hellwig, John Stultz, Anna-Maria Gleixner

From: Thomas Gleixner <tglx@linutronix.de>

The members migrate_enable and nohz_active in the timer/hrtimer per CPU
bases have been introduced to avoid accessing global variables for these
decisions.

Still that results in a (cache hot) load and conditional branch, which can
be avoided by using static keys.

Implement it with static keys and optimize for the most critical case of
high performance networking which tends to disable the timer migration
functionality.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Anna-Maria Gleixner <anna-maria@linutronix.de>
---
 include/linux/hrtimer.h     |  4 --
 kernel/time/hrtimer.c       | 17 +++------
 kernel/time/tick-internal.h | 19 ++++++----
 kernel/time/tick-sched.c    |  2 +-
 kernel/time/timer.c         | 91 +++++++++++++++++++++++----------------------
 5 files changed, 65 insertions(+), 68 deletions(-)

diff --git a/include/linux/hrtimer.h b/include/linux/hrtimer.h
index 012c37fdb688..79b2a8d29d8c 100644
--- a/include/linux/hrtimer.h
+++ b/include/linux/hrtimer.h
@@ -153,8 +153,6 @@ enum  hrtimer_base_type {
  * @cpu:		cpu number
  * @active_bases:	Bitfield to mark bases with active timers
  * @clock_was_set_seq:	Sequence counter of clock was set events
- * @migration_enabled:	The migration of hrtimers to other cpus is enabled
- * @nohz_active:	The nohz functionality is enabled
  * @expires_next:	absolute time of the next event which was scheduled
  *			via clock_set_next_event()
  * @next_timer:		Pointer to the first expiring timer
@@ -178,8 +176,6 @@ struct hrtimer_cpu_base {
 	unsigned int			cpu;
 	unsigned int			active_bases;
 	unsigned int			clock_was_set_seq;
-	bool				migration_enabled;
-	bool				nohz_active;
 #ifdef CONFIG_HIGH_RES_TIMERS
 	unsigned int			in_hrtirq	: 1,
 					hres_active	: 1,
diff --git a/kernel/time/hrtimer.c b/kernel/time/hrtimer.c
index d32520840fde..69d203d8b12d 100644
--- a/kernel/time/hrtimer.c
+++ b/kernel/time/hrtimer.c
@@ -178,23 +178,16 @@ hrtimer_check_target(struct hrtimer *timer, struct hrtimer_clock_base *new_base)
 #endif
 }
 
-#ifdef CONFIG_NO_HZ_COMMON
-static inline
-struct hrtimer_cpu_base *get_target_base(struct hrtimer_cpu_base *base,
-					 int pinned)
-{
-	if (pinned || !base->migration_enabled)
-		return base;
-	return &per_cpu(hrtimer_bases, get_nohz_timer_target());
-}
-#else
 static inline
 struct hrtimer_cpu_base *get_target_base(struct hrtimer_cpu_base *base,
 					 int pinned)
 {
+#if defined(CONFIG_SMP) && defined(CONFIG_NO_HZ_COMMON)
+	if (static_branch_unlikely(&timers_migration_enabled) && !pinned)
+		return &per_cpu(hrtimer_bases, get_nohz_timer_target());
+#endif
 	return base;
 }
-#endif
 
 /*
  * We switch the timer base to a power-optimized selected CPU target,
@@ -969,7 +962,7 @@ void hrtimer_start_range_ns(struct hrtimer *timer, ktime_t tim,
 		 * Kick to reschedule the next tick to handle the new timer
 		 * on dynticks target.
 		 */
-		if (new_base->cpu_base->nohz_active)
+		if (is_timers_nohz_active())
 			wake_up_nohz_cpu(new_base->cpu_base->cpu);
 	} else {
 		hrtimer_reprogram(timer, new_base);
diff --git a/kernel/time/tick-internal.h b/kernel/time/tick-internal.h
index f8e1845aa464..4ac74dff59f0 100644
--- a/kernel/time/tick-internal.h
+++ b/kernel/time/tick-internal.h
@@ -150,14 +150,19 @@ static inline void tick_nohz_init(void) { }
 
 #ifdef CONFIG_NO_HZ_COMMON
 extern unsigned long tick_nohz_active;
-#else
+extern void timers_update_nohz(void);
+extern struct static_key_false timers_nohz_active;
+static inline bool is_timers_nohz_active(void)
+{
+	return static_branch_unlikely(&timers_nohz_active);
+}
+# ifdef CONFIG_SMP
+extern struct static_key_false timers_migration_enabled;
+# endif
+#else /* CONFIG_NO_HZ_COMMON */
+static inline void timers_update_nohz(void) { }
 #define tick_nohz_active (0)
-#endif
-
-#if defined(CONFIG_SMP) && defined(CONFIG_NO_HZ_COMMON)
-extern void timers_update_migration(bool update_nohz);
-#else
-static inline void timers_update_migration(bool update_nohz) { }
+static inline bool is_timers_nohz_active(void) { return false; }
 #endif
 
 DECLARE_PER_CPU(struct hrtimer_cpu_base, hrtimer_bases);
diff --git a/kernel/time/tick-sched.c b/kernel/time/tick-sched.c
index 99578f06c8d4..f371f25f3510 100644
--- a/kernel/time/tick-sched.c
+++ b/kernel/time/tick-sched.c
@@ -1079,7 +1079,7 @@ static inline void tick_nohz_activate(struct tick_sched *ts, int mode)
 	ts->nohz_mode = mode;
 	/* One update is enough */
 	if (!test_and_set_bit(0, &tick_nohz_active))
-		timers_update_migration(true);
+		timers_update_nohz();
 }
 
 /**
diff --git a/kernel/time/timer.c b/kernel/time/timer.c
index ffebcf878fba..1e2140a23044 100644
--- a/kernel/time/timer.c
+++ b/kernel/time/timer.c
@@ -200,8 +200,6 @@ struct timer_base {
 	unsigned long		clk;
 	unsigned long		next_expiry;
 	unsigned int		cpu;
-	bool			migration_enabled;
-	bool			nohz_active;
 	bool			is_idle;
 	bool			must_forward_clk;
 	DECLARE_BITMAP(pending_map, WHEEL_SIZE);
@@ -210,45 +208,59 @@ struct timer_base {
 
 static DEFINE_PER_CPU(struct timer_base, timer_bases[NR_BASES]);
 
-#if defined(CONFIG_SMP) && defined(CONFIG_NO_HZ_COMMON)
+#ifdef CONFIG_NO_HZ_COMMON
+
+DEFINE_STATIC_KEY_FALSE(timers_nohz_active);
+static DEFINE_MUTEX(timer_keys_mutex);
+
+static void timer_update_keys(struct work_struct *work);
+static DECLARE_WORK(timer_update_work, timer_update_keys);
+
+#ifdef CONFIG_SMP
 unsigned int sysctl_timer_migration = 1;
 
-void timers_update_migration(bool update_nohz)
+DEFINE_STATIC_KEY_FALSE(timers_migration_enabled);
+
+static void timers_update_migration(void)
 {
 	bool on = sysctl_timer_migration && tick_nohz_active;
-	unsigned int cpu;
 
-	/* Avoid the loop, if nothing to update */
-	if (this_cpu_read(timer_bases[BASE_STD].migration_enabled) == on)
-		return;
+	if (on)
+		static_branch_enable(&timers_migration_enabled);
+	else
+		static_branch_disable(&timers_migration_enabled);
+}
+#else
+static inline void timers_update_migration(void) { }
+#endif /* !CONFIG_SMP */
 
-	for_each_possible_cpu(cpu) {
-		per_cpu(timer_bases[BASE_STD].migration_enabled, cpu) = on;
-		per_cpu(timer_bases[BASE_DEF].migration_enabled, cpu) = on;
-		per_cpu(hrtimer_bases.migration_enabled, cpu) = on;
-		if (!update_nohz)
-			continue;
-		per_cpu(timer_bases[BASE_STD].nohz_active, cpu) = true;
-		per_cpu(timer_bases[BASE_DEF].nohz_active, cpu) = true;
-		per_cpu(hrtimer_bases.nohz_active, cpu) = true;
-	}
+static void timer_update_keys(struct work_struct *work)
+{
+	mutex_lock(&timer_keys_mutex);
+	timers_update_migration();
+	static_branch_enable(&timers_nohz_active);
+	mutex_unlock(&timer_keys_mutex);
+}
+
+void timers_update_nohz(void)
+{
+	schedule_work(&timer_update_work);
 }
 
 int timer_migration_handler(struct ctl_table *table, int write,
 			    void __user *buffer, size_t *lenp,
 			    loff_t *ppos)
 {
-	static DEFINE_MUTEX(mutex);
 	int ret;
 
-	mutex_lock(&mutex);
+	mutex_lock(&timer_keys_mutex);
 	ret = proc_dointvec_minmax(table, write, buffer, lenp, ppos);
 	if (!ret && write)
-		timers_update_migration(false);
-	mutex_unlock(&mutex);
+		timers_update_migration();
+	mutex_unlock(&timer_keys_mutex);
 	return ret;
 }
-#endif
+#endif /* NO_HZ_COMMON */
 
 static unsigned long round_jiffies_common(unsigned long j, int cpu,
 		bool force_up)
@@ -534,7 +546,7 @@ __internal_add_timer(struct timer_base *base, struct timer_list *timer)
 static void
 trigger_dyntick_cpu(struct timer_base *base, struct timer_list *timer)
 {
-	if (!IS_ENABLED(CONFIG_NO_HZ_COMMON) || !base->nohz_active)
+	if (!is_timers_nohz_active())
 		return;
 
 	/*
@@ -826,7 +838,7 @@ static inline struct timer_base *get_timer_cpu_base(u32 tflags, u32 cpu)
 	 * If the timer is deferrable and nohz is active then we need to use
 	 * the deferrable base.
 	 */
-	if (IS_ENABLED(CONFIG_NO_HZ_COMMON) && base->nohz_active &&
+	if (is_timers_nohz_active() &&
 	    (tflags & TIMER_DEFERRABLE))
 		base = per_cpu_ptr(&timer_bases[BASE_DEF], cpu);
 	return base;
@@ -840,7 +852,7 @@ static inline struct timer_base *get_timer_this_cpu_base(u32 tflags)
 	 * If the timer is deferrable and nohz is active then we need to use
 	 * the deferrable base.
 	 */
-	if (IS_ENABLED(CONFIG_NO_HZ_COMMON) && base->nohz_active &&
+	if (is_timers_nohz_active() &&
 	    (tflags & TIMER_DEFERRABLE))
 		base = this_cpu_ptr(&timer_bases[BASE_DEF]);
 	return base;
@@ -851,21 +863,20 @@ static inline struct timer_base *get_timer_base(u32 tflags)
 	return get_timer_cpu_base(tflags, tflags & TIMER_CPUMASK);
 }
 
-#ifdef CONFIG_NO_HZ_COMMON
 static inline struct timer_base *
 get_target_base(struct timer_base *base, unsigned tflags)
 {
-#ifdef CONFIG_SMP
-	if ((tflags & TIMER_PINNED) || !base->migration_enabled)
-		return get_timer_this_cpu_base(tflags);
-	return get_timer_cpu_base(tflags, get_nohz_timer_target());
-#else
-	return get_timer_this_cpu_base(tflags);
+#if defined(CONFIG_SMP) && defined(CONFIG_NO_HZ_COMMON)
+	if (static_branch_unlikely(&timers_migration_enabled) &&
+	    !(tflags & TIMER_PINNED))
+		return get_timer_cpu_base(tflags, get_nohz_timer_target());
 #endif
+	return get_timer_this_cpu_base(tflags);
 }
 
 static inline void forward_timer_base(struct timer_base *base)
 {
+#ifdef CONFIG_NO_HZ_COMMON
 	unsigned long jnow;
 
 	/*
@@ -889,16 +900,8 @@ static inline void forward_timer_base(struct timer_base *base)
 		base->clk = jnow;
 	else
 		base->clk = base->next_expiry;
-}
-#else
-static inline struct timer_base *
-get_target_base(struct timer_base *base, unsigned tflags)
-{
-	return get_timer_this_cpu_base(tflags);
-}
-
-static inline void forward_timer_base(struct timer_base *base) { }
 #endif
+}
 
 
 /*
@@ -1684,7 +1687,7 @@ static __latent_entropy void run_timer_softirq(struct softirq_action *h)
 	base->must_forward_clk = false;
 
 	__run_timers(base);
-	if (IS_ENABLED(CONFIG_NO_HZ_COMMON) && base->nohz_active)
+	if (is_timers_nohz_active())
 		__run_timers(this_cpu_ptr(&timer_bases[BASE_DEF]));
 }
 
@@ -1698,7 +1701,7 @@ void run_local_timers(void)
 	hrtimer_run_queues();
 	/* Raise the softirq only if required. */
 	if (time_before(jiffies, base->clk)) {
-		if (!IS_ENABLED(CONFIG_NO_HZ_COMMON) || !base->nohz_active)
+		if (!is_timers_nohz_active())
 			return;
 		/* CPU is awake, so check the deferrable base. */
 		base++;
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 45+ messages in thread

* [PATCH v3 02/36] hrtimer: Correct blantanly wrong comment
  2017-11-29 15:30 [PATCH 00/36] hrtimer: Provide softirq context hrtimers Anna-Maria Gleixner
  2017-11-29 15:30 ` [PATCH v3 01/36] timers: Use static keys for migrate_enable/nohz_active Anna-Maria Gleixner
@ 2017-11-29 15:30 ` Anna-Maria Gleixner
  2017-11-29 15:30 ` [PATCH v3 03/36] hrtimer: Fix kerneldoc for struct hrtimer_cpu_base Anna-Maria Gleixner
                   ` (33 subsequent siblings)
  35 siblings, 0 replies; 45+ messages in thread
From: Anna-Maria Gleixner @ 2017-11-29 15:30 UTC (permalink / raw)
  To: LKML
  Cc: Thomas Gleixner, Peter Zijlstra, Ingo Molnar, keescook,
	Christoph Hellwig, John Stultz, Anna-Maria Gleixner

From: Thomas Gleixner <tglx@linutronix.de>

The protection of a hrtimer which runs its callback against migration to a
different CPU has nothing to do with hard interrupt context.

The protection against migration of a hrtimer running the expiry callback
is the pointer in the cpu_base which holds a pointer to the currently
running timer. This pointer is evaluated in the code which potentially
switches the timer base and makes sure it's kept on the CPU on which the
callback is running.

Reported-by: Anna-Maria Gleixner <anna-maria@linutronix.de>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Anna-Maria Gleixner <anna-maria@linutronix.de>
---
 kernel/time/hrtimer.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/kernel/time/hrtimer.c b/kernel/time/hrtimer.c
index 69d203d8b12d..aee49c0c58b9 100644
--- a/kernel/time/hrtimer.c
+++ b/kernel/time/hrtimer.c
@@ -1195,9 +1195,9 @@ static void __run_hrtimer(struct hrtimer_cpu_base *cpu_base,
 		timer->is_rel = false;
 
 	/*
-	 * Because we run timers from hardirq context, there is no chance
-	 * they get migrated to another cpu, therefore its safe to unlock
-	 * the timer base.
+	 * The timer is marked as running in the cpu base, so it is
+	 * protected against migration to a different CPU even if the lock
+	 * is dropped.
 	 */
 	raw_spin_unlock(&cpu_base->lock);
 	trace_hrtimer_expire_entry(timer, now);
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 45+ messages in thread

* [PATCH v3 03/36] hrtimer: Fix kerneldoc for struct hrtimer_cpu_base
  2017-11-29 15:30 [PATCH 00/36] hrtimer: Provide softirq context hrtimers Anna-Maria Gleixner
  2017-11-29 15:30 ` [PATCH v3 01/36] timers: Use static keys for migrate_enable/nohz_active Anna-Maria Gleixner
  2017-11-29 15:30 ` [PATCH v3 02/36] hrtimer: Correct blantanly wrong comment Anna-Maria Gleixner
@ 2017-11-29 15:30 ` Anna-Maria Gleixner
  2017-11-29 15:30 ` [PATCH v3 04/36] hrtimer: Cleanup clock argument in schedule_hrtimeout_range_clock() Anna-Maria Gleixner
                   ` (32 subsequent siblings)
  35 siblings, 0 replies; 45+ messages in thread
From: Anna-Maria Gleixner @ 2017-11-29 15:30 UTC (permalink / raw)
  To: LKML
  Cc: Thomas Gleixner, Peter Zijlstra, Ingo Molnar, keescook,
	Christoph Hellwig, John Stultz, Anna-Maria Gleixner

The sequence '/**' marks the start of a struct description. Add the
missing second asterisk. While at it adapt the ordering of the struct
members to the struct definition and document the purpose of
expires_next more precisely.

Signed-off-by: Anna-Maria Gleixner <anna-maria@linutronix.de>
---
 include/linux/hrtimer.h | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/include/linux/hrtimer.h b/include/linux/hrtimer.h
index 79b2a8d29d8c..b3a382be8db0 100644
--- a/include/linux/hrtimer.h
+++ b/include/linux/hrtimer.h
@@ -144,7 +144,7 @@ enum  hrtimer_base_type {
 	HRTIMER_MAX_CLOCK_BASES,
 };
 
-/*
+/**
  * struct hrtimer_cpu_base - the per cpu clock bases
  * @lock:		lock protecting the base and associated clock bases
  *			and timers
@@ -153,12 +153,12 @@ enum  hrtimer_base_type {
  * @cpu:		cpu number
  * @active_bases:	Bitfield to mark bases with active timers
  * @clock_was_set_seq:	Sequence counter of clock was set events
- * @expires_next:	absolute time of the next event which was scheduled
- *			via clock_set_next_event()
- * @next_timer:		Pointer to the first expiring timer
  * @in_hrtirq:		hrtimer_interrupt() is currently executing
  * @hres_active:	State of high resolution mode
  * @hang_detected:	The last hrtimer interrupt detected a hang
+ * @expires_next:	absolute time of the next event, is required for remote
+ *			hrtimer enqueue
+ * @next_timer:		Pointer to the first expiring timer
  * @nr_events:		Total number of hrtimer interrupt events
  * @nr_retries:		Total number of hrtimer interrupt retries
  * @nr_hangs:		Total number of hrtimer interrupt hangs
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 45+ messages in thread

* [PATCH v3 04/36] hrtimer: Cleanup clock argument in schedule_hrtimeout_range_clock()
  2017-11-29 15:30 [PATCH 00/36] hrtimer: Provide softirq context hrtimers Anna-Maria Gleixner
                   ` (2 preceding siblings ...)
  2017-11-29 15:30 ` [PATCH v3 03/36] hrtimer: Fix kerneldoc for struct hrtimer_cpu_base Anna-Maria Gleixner
@ 2017-11-29 15:30 ` Anna-Maria Gleixner
  2017-11-29 15:30 ` [PATCH v3 05/36] hrtimer: Fix hrtimer function description Anna-Maria Gleixner
                   ` (31 subsequent siblings)
  35 siblings, 0 replies; 45+ messages in thread
From: Anna-Maria Gleixner @ 2017-11-29 15:30 UTC (permalink / raw)
  To: LKML
  Cc: Thomas Gleixner, Peter Zijlstra, Ingo Molnar, keescook,
	Christoph Hellwig, John Stultz, Anna-Maria Gleixner

schedule_hrtimeout_range_clock() uses an integer for the clock id
instead of the predefined type "clockid_t". The ID of the clock is
indicated in hrtimer code as clock_id. Therefore change the name of
the variable as well to make it consistent.

While at it, clean up the description for the function parameters clock_id
and mode. The clock modes and the clock ids are not restricted as the
comment suggests. Fix the mode description as well for the callers of
schedule_hrtimeout_range_clock().

No functional change.

Signed-off-by: Anna-Maria Gleixner <anna-maria@linutronix.de>
---
 include/linux/hrtimer.h |  2 +-
 kernel/time/hrtimer.c   | 12 ++++++------
 2 files changed, 7 insertions(+), 7 deletions(-)

diff --git a/include/linux/hrtimer.h b/include/linux/hrtimer.h
index b3a382be8db0..931ce9c89c93 100644
--- a/include/linux/hrtimer.h
+++ b/include/linux/hrtimer.h
@@ -462,7 +462,7 @@ extern int schedule_hrtimeout_range(ktime_t *expires, u64 delta,
 extern int schedule_hrtimeout_range_clock(ktime_t *expires,
 					  u64 delta,
 					  const enum hrtimer_mode mode,
-					  int clock);
+					  clockid_t clock_id);
 extern int schedule_hrtimeout(ktime_t *expires, const enum hrtimer_mode mode);
 
 /* Soft interrupt function to run the hrtimer queues: */
diff --git a/kernel/time/hrtimer.c b/kernel/time/hrtimer.c
index aee49c0c58b9..74a64db375d1 100644
--- a/kernel/time/hrtimer.c
+++ b/kernel/time/hrtimer.c
@@ -1662,12 +1662,12 @@ void __init hrtimers_init(void)
  * schedule_hrtimeout_range_clock - sleep until timeout
  * @expires:	timeout value (ktime_t)
  * @delta:	slack in expires timeout (ktime_t)
- * @mode:	timer mode, HRTIMER_MODE_ABS or HRTIMER_MODE_REL
- * @clock:	timer clock, CLOCK_MONOTONIC or CLOCK_REALTIME
+ * @mode:	timer mode
+ * @clock_id:	timer clock to be used
  */
 int __sched
 schedule_hrtimeout_range_clock(ktime_t *expires, u64 delta,
-			       const enum hrtimer_mode mode, int clock)
+			       const enum hrtimer_mode mode, clockid_t clock_id)
 {
 	struct hrtimer_sleeper t;
 
@@ -1688,7 +1688,7 @@ schedule_hrtimeout_range_clock(ktime_t *expires, u64 delta,
 		return -EINTR;
 	}
 
-	hrtimer_init_on_stack(&t.timer, clock, mode);
+	hrtimer_init_on_stack(&t.timer, clock_id, mode);
 	hrtimer_set_expires_range_ns(&t.timer, *expires, delta);
 
 	hrtimer_init_sleeper(&t, current);
@@ -1710,7 +1710,7 @@ schedule_hrtimeout_range_clock(ktime_t *expires, u64 delta,
  * schedule_hrtimeout_range - sleep until timeout
  * @expires:	timeout value (ktime_t)
  * @delta:	slack in expires timeout (ktime_t)
- * @mode:	timer mode, HRTIMER_MODE_ABS or HRTIMER_MODE_REL
+ * @mode:	timer mode
  *
  * Make the current task sleep until the given expiry time has
  * elapsed. The routine will return immediately unless
@@ -1749,7 +1749,7 @@ EXPORT_SYMBOL_GPL(schedule_hrtimeout_range);
 /**
  * schedule_hrtimeout - sleep until timeout
  * @expires:	timeout value (ktime_t)
- * @mode:	timer mode, HRTIMER_MODE_ABS or HRTIMER_MODE_REL
+ * @mode:	timer mode
  *
  * Make the current task sleep until the given expiry time has
  * elapsed. The routine will return immediately unless
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 45+ messages in thread

* [PATCH v3 05/36] hrtimer: Fix hrtimer function description
  2017-11-29 15:30 [PATCH 00/36] hrtimer: Provide softirq context hrtimers Anna-Maria Gleixner
                   ` (3 preceding siblings ...)
  2017-11-29 15:30 ` [PATCH v3 04/36] hrtimer: Cleanup clock argument in schedule_hrtimeout_range_clock() Anna-Maria Gleixner
@ 2017-11-29 15:30 ` Anna-Maria Gleixner
  2017-11-29 15:30 ` [PATCH v3 06/36] hrtimer: Ensure POSIX compliance (relative CLOCK_REALTIME hrtimers) Anna-Maria Gleixner
                   ` (30 subsequent siblings)
  35 siblings, 0 replies; 45+ messages in thread
From: Anna-Maria Gleixner @ 2017-11-29 15:30 UTC (permalink / raw)
  To: LKML
  Cc: Thomas Gleixner, Peter Zijlstra, Ingo Molnar, keescook,
	Christoph Hellwig, John Stultz, Anna-Maria Gleixner

The hrtimer_start[_range_ns]() starts a timer reliable on this CPU only
when HRTIMER_MODE_PINNED is set. Furthermore the HRTIMER_MODE_PINNED mode
is not considered, when a hrtimer is initialized.

Signed-off-by: Anna-Maria Gleixner <anna-maria@linutronix.de>
---
 include/linux/hrtimer.h | 6 +++---
 kernel/time/hrtimer.c   | 9 +++++----
 2 files changed, 8 insertions(+), 7 deletions(-)

diff --git a/include/linux/hrtimer.h b/include/linux/hrtimer.h
index 931ce9c89c93..4e6a8841dcbe 100644
--- a/include/linux/hrtimer.h
+++ b/include/linux/hrtimer.h
@@ -361,11 +361,11 @@ extern void hrtimer_start_range_ns(struct hrtimer *timer, ktime_t tim,
 				   u64 range_ns, const enum hrtimer_mode mode);
 
 /**
- * hrtimer_start - (re)start an hrtimer on the current CPU
+ * hrtimer_start - (re)start an hrtimer
  * @timer:	the timer to be added
  * @tim:	expiry time
- * @mode:	expiry mode: absolute (HRTIMER_MODE_ABS) or
- *		relative (HRTIMER_MODE_REL)
+ * @mode:	timer mode: absolute (HRTIMER_MODE_ABS) or
+ *		relative (HRTIMER_MODE_REL), and pinned (HRTIMER_MODE_PINNED)
  */
 static inline void hrtimer_start(struct hrtimer *timer, ktime_t tim,
 				 const enum hrtimer_mode mode)
diff --git a/kernel/time/hrtimer.c b/kernel/time/hrtimer.c
index 74a64db375d1..ae7b29c4e03e 100644
--- a/kernel/time/hrtimer.c
+++ b/kernel/time/hrtimer.c
@@ -924,12 +924,12 @@ static inline ktime_t hrtimer_update_lowres(struct hrtimer *timer, ktime_t tim,
 }
 
 /**
- * hrtimer_start_range_ns - (re)start an hrtimer on the current CPU
+ * hrtimer_start_range_ns - (re)start an hrtimer
  * @timer:	the timer to be added
  * @tim:	expiry time
  * @delta_ns:	"slack" range for the timer
- * @mode:	expiry mode: absolute (HRTIMER_MODE_ABS) or
- *		relative (HRTIMER_MODE_REL)
+ * @mode:	timer mode: absolute (HRTIMER_MODE_ABS) or
+ *		relative (HRTIMER_MODE_REL), and pinned (HRTIMER_MODE_PINNED)
  */
 void hrtimer_start_range_ns(struct hrtimer *timer, ktime_t tim,
 			    u64 delta_ns, const enum hrtimer_mode mode)
@@ -1107,7 +1107,8 @@ static void __hrtimer_init(struct hrtimer *timer, clockid_t clock_id,
  * hrtimer_init - initialize a timer to the given clock
  * @timer:	the timer to be initialized
  * @clock_id:	the clock to be used
- * @mode:	timer mode abs/rel
+ * @mode:	timer mode: absolute (HRTIMER_MODE_ABS) or
+ *		relative (HRTIMER_MODE_REL); pinned is not considered here!
  */
 void hrtimer_init(struct hrtimer *timer, clockid_t clock_id,
 		  enum hrtimer_mode mode)
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 45+ messages in thread

* [PATCH v3 06/36] hrtimer: Ensure POSIX compliance (relative CLOCK_REALTIME hrtimers)
  2017-11-29 15:30 [PATCH 00/36] hrtimer: Provide softirq context hrtimers Anna-Maria Gleixner
                   ` (4 preceding siblings ...)
  2017-11-29 15:30 ` [PATCH v3 05/36] hrtimer: Fix hrtimer function description Anna-Maria Gleixner
@ 2017-11-29 15:30 ` Anna-Maria Gleixner
  2017-11-29 15:30 ` [PATCH v3 07/36] hrtimer: Cleanup hrtimer_mode enum Anna-Maria Gleixner
                   ` (29 subsequent siblings)
  35 siblings, 0 replies; 45+ messages in thread
From: Anna-Maria Gleixner @ 2017-11-29 15:30 UTC (permalink / raw)
  To: LKML
  Cc: Thomas Gleixner, Peter Zijlstra, Ingo Molnar, keescook,
	Christoph Hellwig, John Stultz, Anna-Maria Gleixner

POSIX specification defines, that relative CLOCK_REALTIME timers are not
affected by clock modifications. Those timers have to use CLOCK_MONOTONIC
to ensure POSIX compliance.

The introduction of the additional mode HRTIMER_MODE_PINNED broke this
requirement for pinned timers. There is no user space visible impact
because user space timers are not using the pinned mode, but for
consistency reasons this needs to be fixed.

Check whether the mode has the HRTIMER_MODE_REL bit set instead of
comparing with HRTIMER_MODE_ABS.


Fixes: 597d0275736d ("timers: Framework for identifying pinned timers")
Signed-off-by: Anna-Maria Gleixner <anna-maria@linutronix.de>
---
 kernel/time/hrtimer.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/kernel/time/hrtimer.c b/kernel/time/hrtimer.c
index ae7b29c4e03e..9945ea6b0e5c 100644
--- a/kernel/time/hrtimer.c
+++ b/kernel/time/hrtimer.c
@@ -1095,7 +1095,12 @@ static void __hrtimer_init(struct hrtimer *timer, clockid_t clock_id,
 
 	cpu_base = raw_cpu_ptr(&hrtimer_bases);
 
-	if (clock_id == CLOCK_REALTIME && mode != HRTIMER_MODE_ABS)
+	/*
+	 * Posix magic: Relative CLOCK_REALTIME timers are not affected by
+	 * clock modifications, so they needs to become CLOCK_MONOTONIC to
+	 * ensure Posix compliance.
+	 */
+	if (clock_id == CLOCK_REALTIME && mode & HRTIMER_MODE_REL)
 		clock_id = CLOCK_MONOTONIC;
 
 	base = hrtimer_clockid_to_base(clock_id);
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 45+ messages in thread

* [PATCH v3 07/36] hrtimer: Cleanup hrtimer_mode enum
  2017-11-29 15:30 [PATCH 00/36] hrtimer: Provide softirq context hrtimers Anna-Maria Gleixner
                   ` (5 preceding siblings ...)
  2017-11-29 15:30 ` [PATCH v3 06/36] hrtimer: Ensure POSIX compliance (relative CLOCK_REALTIME hrtimers) Anna-Maria Gleixner
@ 2017-11-29 15:30 ` Anna-Maria Gleixner
  2017-11-29 15:30 ` [PATCH v3 08/36] tracing/hrtimer: Take all clock bases and modes into account Anna-Maria Gleixner
                   ` (28 subsequent siblings)
  35 siblings, 0 replies; 45+ messages in thread
From: Anna-Maria Gleixner @ 2017-11-29 15:30 UTC (permalink / raw)
  To: LKML
  Cc: Thomas Gleixner, Peter Zijlstra, Ingo Molnar, keescook,
	Christoph Hellwig, John Stultz, Anna-Maria Gleixner

It's not obvious that the HRTIMER_MODE variants are bit combinations
because all modes are hard coded constants.

Change it so the bit meanings are clear and use the symbols for creating
modes which combine bits.

While at it get rid of the ugly tail comments.

Signed-off-by: Anna-Maria Gleixner <anna-maria@linutronix.de>
---
 include/linux/hrtimer.h | 16 +++++++++++-----
 1 file changed, 11 insertions(+), 5 deletions(-)

diff --git a/include/linux/hrtimer.h b/include/linux/hrtimer.h
index 4e6a8841dcbe..28f267cf2851 100644
--- a/include/linux/hrtimer.h
+++ b/include/linux/hrtimer.h
@@ -28,13 +28,19 @@ struct hrtimer_cpu_base;
 
 /*
  * Mode arguments of xxx_hrtimer functions:
+ *
+ * HRTIMER_MODE_ABS		- Time value is absolute
+ * HRTIMER_MODE_REL		- Time value is relative to now
+ * HRTIMER_MODE_PINNED		- Timer is bound to CPU (is only considered
+ *				  when starting the timer)
  */
 enum hrtimer_mode {
-	HRTIMER_MODE_ABS = 0x0,		/* Time value is absolute */
-	HRTIMER_MODE_REL = 0x1,		/* Time value is relative to now */
-	HRTIMER_MODE_PINNED = 0x02,	/* Timer is bound to CPU */
-	HRTIMER_MODE_ABS_PINNED = 0x02,
-	HRTIMER_MODE_REL_PINNED = 0x03,
+	HRTIMER_MODE_ABS	= 0x00,
+	HRTIMER_MODE_REL	= 0x01,
+	HRTIMER_MODE_PINNED	= 0x02,
+
+	HRTIMER_MODE_ABS_PINNED = HRTIMER_MODE_ABS | HRTIMER_MODE_PINNED,
+	HRTIMER_MODE_REL_PINNED = HRTIMER_MODE_REL | HRTIMER_MODE_PINNED,
 };
 
 /*
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 45+ messages in thread

* [PATCH v3 08/36] tracing/hrtimer: Take all clock bases and modes into account
  2017-11-29 15:30 [PATCH 00/36] hrtimer: Provide softirq context hrtimers Anna-Maria Gleixner
                   ` (6 preceding siblings ...)
  2017-11-29 15:30 ` [PATCH v3 07/36] hrtimer: Cleanup hrtimer_mode enum Anna-Maria Gleixner
@ 2017-11-29 15:30 ` Anna-Maria Gleixner
  2017-11-29 15:30 ` [PATCH v3 09/36] tracing/hrtimer: Print hrtimer mode in hrtimer_start tracepoint Anna-Maria Gleixner
                   ` (27 subsequent siblings)
  35 siblings, 0 replies; 45+ messages in thread
From: Anna-Maria Gleixner @ 2017-11-29 15:30 UTC (permalink / raw)
  To: LKML
  Cc: Thomas Gleixner, Peter Zijlstra, Ingo Molnar, keescook,
	Christoph Hellwig, John Stultz, Anna-Maria Gleixner

So far only CLOCK_MONOTONIC and CLOCK_REALTIME were taken into account as
well as HRTIMER_MODE_ABS/REL in hrtimer_init tracepoint. The query for
detecting timer mode ABS or REL is not valid, since the introduction of
HRTIMER_MODE_PINNED.

HRTIMER_MODE_PINNED is not evaluated in hrtimer_init() call. But for the
sake of completeness print all given modes.

Signed-off-by: Anna-Maria Gleixner <anna-maria@linutronix.de>
---
 include/trace/events/timer.h | 20 ++++++++++++++++----
 1 file changed, 16 insertions(+), 4 deletions(-)

diff --git a/include/trace/events/timer.h b/include/trace/events/timer.h
index 16e305e69f34..c6f728037c53 100644
--- a/include/trace/events/timer.h
+++ b/include/trace/events/timer.h
@@ -136,6 +136,20 @@ DEFINE_EVENT(timer_class, timer_cancel,
 	TP_ARGS(timer)
 );
 
+#define decode_clockid(type)						\
+	__print_symbolic(type,						\
+		{ CLOCK_REALTIME,	"CLOCK_REALTIME"	},	\
+		{ CLOCK_MONOTONIC,	"CLOCK_MONOTONIC"	},	\
+		{ CLOCK_BOOTTIME,	"CLOCK_BOOTTIME"	},	\
+		{ CLOCK_TAI,		"CLOCK_TAI"		})
+
+#define decode_hrtimer_mode(mode)					\
+	__print_symbolic(mode,						\
+		{ HRTIMER_MODE_ABS,		"ABS"		},	\
+		{ HRTIMER_MODE_REL,		"REL"		},	\
+		{ HRTIMER_MODE_ABS_PINNED,	"ABS|PINNED"	},	\
+		{ HRTIMER_MODE_REL_PINNED,	"REL|PINNED"	})
+
 /**
  * hrtimer_init - called when the hrtimer is initialized
  * @hrtimer:	pointer to struct hrtimer
@@ -162,10 +176,8 @@ TRACE_EVENT(hrtimer_init,
 	),
 
 	TP_printk("hrtimer=%p clockid=%s mode=%s", __entry->hrtimer,
-		  __entry->clockid == CLOCK_REALTIME ?
-			"CLOCK_REALTIME" : "CLOCK_MONOTONIC",
-		  __entry->mode == HRTIMER_MODE_ABS ?
-			"HRTIMER_MODE_ABS" : "HRTIMER_MODE_REL")
+		  decode_clockid(__entry->clockid),
+		  decode_hrtimer_mode(__entry->mode))
 );
 
 /**
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 45+ messages in thread

* [PATCH v3 09/36] tracing/hrtimer: Print hrtimer mode in hrtimer_start tracepoint
  2017-11-29 15:30 [PATCH 00/36] hrtimer: Provide softirq context hrtimers Anna-Maria Gleixner
                   ` (7 preceding siblings ...)
  2017-11-29 15:30 ` [PATCH v3 08/36] tracing/hrtimer: Take all clock bases and modes into account Anna-Maria Gleixner
@ 2017-11-29 15:30 ` Anna-Maria Gleixner
  2017-11-29 15:30 ` [PATCH v3 10/36] hrtimer: Switch for loop to _ffs() evaluation Anna-Maria Gleixner
                   ` (26 subsequent siblings)
  35 siblings, 0 replies; 45+ messages in thread
From: Anna-Maria Gleixner @ 2017-11-29 15:30 UTC (permalink / raw)
  To: LKML
  Cc: Thomas Gleixner, Peter Zijlstra, Ingo Molnar, keescook,
	Christoph Hellwig, John Stultz, Anna-Maria Gleixner

The hrtimer_start tracepoint lacks the mode information. The mode is
important because consecutive starts can switch from ABS to REL or from
PINNED to non PINNED.

Add the mode information.

Signed-off-by: Anna-Maria Gleixner <anna-maria@linutronix.de>
---
 include/trace/events/timer.h | 13 ++++++++-----
 kernel/time/hrtimer.c        | 16 +++++++++-------
 2 files changed, 17 insertions(+), 12 deletions(-)

diff --git a/include/trace/events/timer.h b/include/trace/events/timer.h
index c6f728037c53..744b4310b24b 100644
--- a/include/trace/events/timer.h
+++ b/include/trace/events/timer.h
@@ -186,15 +186,16 @@ TRACE_EVENT(hrtimer_init,
  */
 TRACE_EVENT(hrtimer_start,
 
-	TP_PROTO(struct hrtimer *hrtimer),
+	TP_PROTO(struct hrtimer *hrtimer, enum hrtimer_mode mode),
 
-	TP_ARGS(hrtimer),
+	TP_ARGS(hrtimer, mode),
 
 	TP_STRUCT__entry(
 		__field( void *,	hrtimer		)
 		__field( void *,	function	)
 		__field( s64,		expires		)
 		__field( s64,		softexpires	)
+		__field( enum hrtimer_mode,	mode	)
 	),
 
 	TP_fast_assign(
@@ -202,12 +203,14 @@ TRACE_EVENT(hrtimer_start,
 		__entry->function	= hrtimer->function;
 		__entry->expires	= hrtimer_get_expires(hrtimer);
 		__entry->softexpires	= hrtimer_get_softexpires(hrtimer);
+		__entry->mode		= mode;
 	),
 
-	TP_printk("hrtimer=%p function=%pf expires=%llu softexpires=%llu",
-		  __entry->hrtimer, __entry->function,
+	TP_printk("hrtimer=%p function=%pf expires=%llu softexpires=%llu "
+		  "mode=%s", __entry->hrtimer, __entry->function,
 		  (unsigned long long) __entry->expires,
-		  (unsigned long long) __entry->softexpires)
+		  (unsigned long long) __entry->softexpires,
+		  decode_hrtimer_mode(__entry->mode))
 );
 
 /**
diff --git a/kernel/time/hrtimer.c b/kernel/time/hrtimer.c
index 9945ea6b0e5c..0e000b58f0d3 100644
--- a/kernel/time/hrtimer.c
+++ b/kernel/time/hrtimer.c
@@ -435,10 +435,11 @@ debug_init(struct hrtimer *timer, clockid_t clockid,
 	trace_hrtimer_init(timer, clockid, mode);
 }
 
-static inline void debug_activate(struct hrtimer *timer)
+static inline void debug_activate(struct hrtimer *timer,
+				  enum hrtimer_mode mode)
 {
 	debug_hrtimer_activate(timer);
-	trace_hrtimer_start(timer);
+	trace_hrtimer_start(timer, mode);
 }
 
 static inline void debug_deactivate(struct hrtimer *timer)
@@ -828,9 +829,10 @@ EXPORT_SYMBOL_GPL(hrtimer_forward);
  * Returns 1 when the new timer is the leftmost timer in the tree.
  */
 static int enqueue_hrtimer(struct hrtimer *timer,
-			   struct hrtimer_clock_base *base)
+			   struct hrtimer_clock_base *base,
+			   enum hrtimer_mode mode)
 {
-	debug_activate(timer);
+	debug_activate(timer, mode);
 
 	base->cpu_base->active_bases |= 1 << base->index;
 
@@ -953,7 +955,7 @@ void hrtimer_start_range_ns(struct hrtimer *timer, ktime_t tim,
 	/* Switch the timer base, if necessary: */
 	new_base = switch_hrtimer_base(timer, base, mode & HRTIMER_MODE_PINNED);
 
-	leftmost = enqueue_hrtimer(timer, new_base);
+	leftmost = enqueue_hrtimer(timer, new_base, mode);
 	if (!leftmost)
 		goto unlock;
 
@@ -1222,7 +1224,7 @@ static void __run_hrtimer(struct hrtimer_cpu_base *cpu_base,
 	 */
 	if (restart != HRTIMER_NORESTART &&
 	    !(timer->state & HRTIMER_STATE_ENQUEUED))
-		enqueue_hrtimer(timer, base);
+		enqueue_hrtimer(timer, base, HRTIMER_MODE_ABS);
 
 	/*
 	 * Separate the ->running assignment from the ->state assignment.
@@ -1621,7 +1623,7 @@ static void migrate_hrtimer_list(struct hrtimer_clock_base *old_base,
 		 * sort out already expired timers and reprogram the
 		 * event device.
 		 */
-		enqueue_hrtimer(timer, new_base);
+		enqueue_hrtimer(timer, new_base, HRTIMER_MODE_ABS);
 	}
 }
 
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 45+ messages in thread

* [PATCH v3 10/36] hrtimer: Switch for loop to _ffs() evaluation
  2017-11-29 15:30 [PATCH 00/36] hrtimer: Provide softirq context hrtimers Anna-Maria Gleixner
                   ` (8 preceding siblings ...)
  2017-11-29 15:30 ` [PATCH v3 09/36] tracing/hrtimer: Print hrtimer mode in hrtimer_start tracepoint Anna-Maria Gleixner
@ 2017-11-29 15:30 ` Anna-Maria Gleixner
  2017-11-29 16:06   ` Joe Perches
  2017-11-29 15:30 ` [PATCH v3 11/36] hrtimer: Store running timer in hrtimer_clock_base Anna-Maria Gleixner
                   ` (25 subsequent siblings)
  35 siblings, 1 reply; 45+ messages in thread
From: Anna-Maria Gleixner @ 2017-11-29 15:30 UTC (permalink / raw)
  To: LKML
  Cc: Thomas Gleixner, Peter Zijlstra, Ingo Molnar, keescook,
	Christoph Hellwig, John Stultz, Anna-Maria Gleixner

Looping over all clock bases to find active bits is suboptimal if not all
bases are active.

Avoid this by converting it to a __ffs() evaluation. The functionallity is
outsourced into an own function and is called via a macro as suggested by
Peter Zijlstra.

Suggested-by: Peter Zijlstra <peterz@infradead.org>
Signed-off-by: Anna-Maria Gleixner <anna-maria@linutronix.de>
---
 kernel/time/hrtimer.c | 31 +++++++++++++++++++++----------
 1 file changed, 21 insertions(+), 10 deletions(-)

diff --git a/kernel/time/hrtimer.c b/kernel/time/hrtimer.c
index 0e000b58f0d3..326faa5719ff 100644
--- a/kernel/time/hrtimer.c
+++ b/kernel/time/hrtimer.c
@@ -448,6 +448,23 @@ static inline void debug_deactivate(struct hrtimer *timer)
 	trace_hrtimer_cancel(timer);
 }
 
+static struct hrtimer_clock_base *
+__next_base(struct hrtimer_cpu_base *cpu_base, unsigned int *active)
+{
+	struct hrtimer_clock_base *base = NULL;
+
+	if (*active) {
+		unsigned int idx = __ffs(*active);
+		*active &= ~(1U << idx);
+		base = &cpu_base->clock_base[idx];
+	}
+
+	return base;
+}
+
+#define for_each_active_base(base, cpu_base, active)	\
+	while ((base = __next_base((cpu_base), &(active))))
+
 #if defined(CONFIG_NO_HZ_COMMON) || defined(CONFIG_HIGH_RES_TIMERS)
 static inline void hrtimer_update_next_timer(struct hrtimer_cpu_base *cpu_base,
 					     struct hrtimer *timer)
@@ -459,18 +476,15 @@ static inline void hrtimer_update_next_timer(struct hrtimer_cpu_base *cpu_base,
 
 static ktime_t __hrtimer_get_next_event(struct hrtimer_cpu_base *cpu_base)
 {
-	struct hrtimer_clock_base *base = cpu_base->clock_base;
+	struct hrtimer_clock_base *base;
 	unsigned int active = cpu_base->active_bases;
 	ktime_t expires, expires_next = KTIME_MAX;
 
 	hrtimer_update_next_timer(cpu_base, NULL);
-	for (; active; base++, active >>= 1) {
+	for_each_active_base(base, cpu_base, active) {
 		struct timerqueue_node *next;
 		struct hrtimer *timer;
 
-		if (!(active & 0x01))
-			continue;
-
 		next = timerqueue_getnext(&base->active);
 		timer = container_of(next, struct hrtimer, node);
 		expires = ktime_sub(hrtimer_get_expires(timer), base->offset);
@@ -1241,16 +1255,13 @@ static void __run_hrtimer(struct hrtimer_cpu_base *cpu_base,
 
 static void __hrtimer_run_queues(struct hrtimer_cpu_base *cpu_base, ktime_t now)
 {
-	struct hrtimer_clock_base *base = cpu_base->clock_base;
+	struct hrtimer_clock_base *base;
 	unsigned int active = cpu_base->active_bases;
 
-	for (; active; base++, active >>= 1) {
+	for_each_active_base(base, cpu_base, active) {
 		struct timerqueue_node *node;
 		ktime_t basenow;
 
-		if (!(active & 0x01))
-			continue;
-
 		basenow = ktime_add(now, base->offset);
 
 		while ((node = timerqueue_getnext(&base->active))) {
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 45+ messages in thread

* [PATCH v3 11/36] hrtimer: Store running timer in hrtimer_clock_base
  2017-11-29 15:30 [PATCH 00/36] hrtimer: Provide softirq context hrtimers Anna-Maria Gleixner
                   ` (9 preceding siblings ...)
  2017-11-29 15:30 ` [PATCH v3 10/36] hrtimer: Switch for loop to _ffs() evaluation Anna-Maria Gleixner
@ 2017-11-29 15:30 ` Anna-Maria Gleixner
  2017-11-29 15:30 ` [PATCH v3 12/36] hrtimer: Make room in struct hrtimer_cpu_base Anna-Maria Gleixner
                   ` (24 subsequent siblings)
  35 siblings, 0 replies; 45+ messages in thread
From: Anna-Maria Gleixner @ 2017-11-29 15:30 UTC (permalink / raw)
  To: LKML
  Cc: Thomas Gleixner, Peter Zijlstra, Ingo Molnar, keescook,
	Christoph Hellwig, John Stultz, Anna-Maria Gleixner

The pointer to the currently running timer is stored in hrtimer_cpu_base
before the base lock is dropped and the callback is invoked.

This results in two levels of indirections and the upcoming support for
softirq based hrtimer requires splitting the "running" storage into soft
and hard irq context expiry.

Storing both in the cpu base would require conditionals in all code paths
accessing that information.

It's possible to have a per clock base sequence count and running pointer
without changing the semantics of the related mechanisms because the timer
base pointer cannot be changed while a timer is running the callback.

Unfortunately this makes cpu_clock base larger than 32 bytes on 32bit
kernels. Instead of having huge gaps due to alignment, remove the alignment
and let the compiler pack cpu base for 32bit. The resulting cache access
patterns are fortunately not really different from the current
behaviour. On 64bit kernels the 64byte alignment stays and the behaviour is
unchanged. This was determined by analyzing the resulting layout and
looking at the number of cache lines involved for the frequently used
clocks.

Signed-off-by: Anna-Maria Gleixner <anna-maria@linutronix.de>
---
 include/linux/hrtimer.h | 20 +++++++++-----------
 kernel/time/hrtimer.c   | 28 +++++++++++++---------------
 2 files changed, 22 insertions(+), 26 deletions(-)

diff --git a/include/linux/hrtimer.h b/include/linux/hrtimer.h
index 28f267cf2851..1bae7b9f071d 100644
--- a/include/linux/hrtimer.h
+++ b/include/linux/hrtimer.h
@@ -118,9 +118,9 @@ struct hrtimer_sleeper {
 };
 
 #ifdef CONFIG_64BIT
-# define HRTIMER_CLOCK_BASE_ALIGN	64
+# define __hrtimer_clock_base_align	____cacheline_aligned
 #else
-# define HRTIMER_CLOCK_BASE_ALIGN	32
+# define __hrtimer_clock_base_align
 #endif
 
 /**
@@ -129,18 +129,22 @@ struct hrtimer_sleeper {
  * @index:		clock type index for per_cpu support when moving a
  *			timer to a base on another cpu.
  * @clockid:		clock id for per_cpu support
+ * @seq:		seqcount around __run_hrtimer
+ * @running:		pointer to the currently running hrtimer
  * @active:		red black tree root node for the active timers
  * @get_time:		function to retrieve the current time of the clock
  * @offset:		offset of this clock to the monotonic base
  */
 struct hrtimer_clock_base {
 	struct hrtimer_cpu_base	*cpu_base;
-	int			index;
+	unsigned int		index;
 	clockid_t		clockid;
+	seqcount_t		seq;
+	struct hrtimer		*running;
 	struct timerqueue_head	active;
 	ktime_t			(*get_time)(void);
 	ktime_t			offset;
-} __attribute__((__aligned__(HRTIMER_CLOCK_BASE_ALIGN)));
+} __hrtimer_clock_base_align;
 
 enum  hrtimer_base_type {
 	HRTIMER_BASE_MONOTONIC,
@@ -154,8 +158,6 @@ enum  hrtimer_base_type {
  * struct hrtimer_cpu_base - the per cpu clock bases
  * @lock:		lock protecting the base and associated clock bases
  *			and timers
- * @seq:		seqcount around __run_hrtimer
- * @running:		pointer to the currently running hrtimer
  * @cpu:		cpu number
  * @active_bases:	Bitfield to mark bases with active timers
  * @clock_was_set_seq:	Sequence counter of clock was set events
@@ -177,8 +179,6 @@ enum  hrtimer_base_type {
  */
 struct hrtimer_cpu_base {
 	raw_spinlock_t			lock;
-	seqcount_t			seq;
-	struct hrtimer			*running;
 	unsigned int			cpu;
 	unsigned int			active_bases;
 	unsigned int			clock_was_set_seq;
@@ -198,8 +198,6 @@ struct hrtimer_cpu_base {
 
 static inline void hrtimer_set_expires(struct hrtimer *timer, ktime_t time)
 {
-	BUILD_BUG_ON(sizeof(struct hrtimer_clock_base) > HRTIMER_CLOCK_BASE_ALIGN);
-
 	timer->node.expires = time;
 	timer->_softexpires = time;
 }
@@ -424,7 +422,7 @@ static inline int hrtimer_is_queued(struct hrtimer *timer)
  */
 static inline int hrtimer_callback_running(struct hrtimer *timer)
 {
-	return timer->base->cpu_base->running == timer;
+	return timer->base->running == timer;
 }
 
 /* Forward a hrtimer so it expires after now: */
diff --git a/kernel/time/hrtimer.c b/kernel/time/hrtimer.c
index 326faa5719ff..046b509beed6 100644
--- a/kernel/time/hrtimer.c
+++ b/kernel/time/hrtimer.c
@@ -70,7 +70,6 @@
 DEFINE_PER_CPU(struct hrtimer_cpu_base, hrtimer_bases) =
 {
 	.lock = __RAW_SPIN_LOCK_UNLOCKED(hrtimer_bases.lock),
-	.seq = SEQCNT_ZERO(hrtimer_bases.seq),
 	.clock_base =
 	{
 		{
@@ -118,7 +117,6 @@ static const int hrtimer_clock_to_base_table[MAX_CLOCKS] = {
  * timer->base->cpu_base
  */
 static struct hrtimer_cpu_base migration_cpu_base = {
-	.seq = SEQCNT_ZERO(migration_cpu_base),
 	.clock_base = { { .cpu_base = &migration_cpu_base, }, },
 };
 
@@ -1148,19 +1146,19 @@ EXPORT_SYMBOL_GPL(hrtimer_init);
  */
 bool hrtimer_active(const struct hrtimer *timer)
 {
-	struct hrtimer_cpu_base *cpu_base;
+	struct hrtimer_clock_base *base;
 	unsigned int seq;
 
 	do {
-		cpu_base = READ_ONCE(timer->base->cpu_base);
-		seq = raw_read_seqcount_begin(&cpu_base->seq);
+		base = READ_ONCE(timer->base);
+		seq = raw_read_seqcount_begin(&base->seq);
 
 		if (timer->state != HRTIMER_STATE_INACTIVE ||
-		    cpu_base->running == timer)
+		    base->running == timer)
 			return true;
 
-	} while (read_seqcount_retry(&cpu_base->seq, seq) ||
-		 cpu_base != READ_ONCE(timer->base->cpu_base));
+	} while (read_seqcount_retry(&base->seq, seq) ||
+		 base != READ_ONCE(timer->base));
 
 	return false;
 }
@@ -1194,16 +1192,16 @@ static void __run_hrtimer(struct hrtimer_cpu_base *cpu_base,
 	lockdep_assert_held(&cpu_base->lock);
 
 	debug_deactivate(timer);
-	cpu_base->running = timer;
+	base->running = timer;
 
 	/*
 	 * Separate the ->running assignment from the ->state assignment.
 	 *
 	 * As with a regular write barrier, this ensures the read side in
-	 * hrtimer_active() cannot observe cpu_base->running == NULL &&
+	 * hrtimer_active() cannot observe base->running == NULL &&
 	 * timer->state == INACTIVE.
 	 */
-	raw_write_seqcount_barrier(&cpu_base->seq);
+	raw_write_seqcount_barrier(&base->seq);
 
 	__remove_hrtimer(timer, base, HRTIMER_STATE_INACTIVE, 0);
 	fn = timer->function;
@@ -1244,13 +1242,13 @@ static void __run_hrtimer(struct hrtimer_cpu_base *cpu_base,
 	 * Separate the ->running assignment from the ->state assignment.
 	 *
 	 * As with a regular write barrier, this ensures the read side in
-	 * hrtimer_active() cannot observe cpu_base->running == NULL &&
+	 * hrtimer_active() cannot observe base->running.timer == NULL &&
 	 * timer->state == INACTIVE.
 	 */
-	raw_write_seqcount_barrier(&cpu_base->seq);
+	raw_write_seqcount_barrier(&base->seq);
 
-	WARN_ON_ONCE(cpu_base->running != timer);
-	cpu_base->running = NULL;
+	WARN_ON_ONCE(base->running != timer);
+	base->running = NULL;
 }
 
 static void __hrtimer_run_queues(struct hrtimer_cpu_base *cpu_base, ktime_t now)
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 45+ messages in thread

* [PATCH v3 12/36] hrtimer: Make room in struct hrtimer_cpu_base
  2017-11-29 15:30 [PATCH 00/36] hrtimer: Provide softirq context hrtimers Anna-Maria Gleixner
                   ` (10 preceding siblings ...)
  2017-11-29 15:30 ` [PATCH v3 11/36] hrtimer: Store running timer in hrtimer_clock_base Anna-Maria Gleixner
@ 2017-11-29 15:30 ` Anna-Maria Gleixner
  2017-11-29 15:30 ` [PATCH v3 13/36] hrtimer: Reduce conditional code (hres_active) Anna-Maria Gleixner
                   ` (23 subsequent siblings)
  35 siblings, 0 replies; 45+ messages in thread
From: Anna-Maria Gleixner @ 2017-11-29 15:30 UTC (permalink / raw)
  To: LKML
  Cc: Thomas Gleixner, Peter Zijlstra, Ingo Molnar, keescook,
	Christoph Hellwig, John Stultz, Anna-Maria Gleixner

The upcoming softirq based hrtimers support requires an additional field in
the hrtimer_cpu_base struct, which would grow the struct size beyond a
cache line.

The struct members nr_retries and nr_hangs of hrtimer_cpu_base are solely
used for diagnostic output and have no requirement to be unsigned int.

Make them unsigned short to create room for the new struct member. No
functional change.

Signed-off-by: Anna-Maria Gleixner <anna-maria@linutronix.de>
---
 include/linux/hrtimer.h | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/include/linux/hrtimer.h b/include/linux/hrtimer.h
index 1bae7b9f071d..56e56bcb6f0f 100644
--- a/include/linux/hrtimer.h
+++ b/include/linux/hrtimer.h
@@ -189,8 +189,8 @@ struct hrtimer_cpu_base {
 	ktime_t				expires_next;
 	struct hrtimer			*next_timer;
 	unsigned int			nr_events;
-	unsigned int			nr_retries;
-	unsigned int			nr_hangs;
+	unsigned short			nr_retries;
+	unsigned short			nr_hangs;
 	unsigned int			max_hang_time;
 #endif
 	struct hrtimer_clock_base	clock_base[HRTIMER_MAX_CLOCK_BASES];
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 45+ messages in thread

* [PATCH v3 13/36] hrtimer: Reduce conditional code (hres_active)
  2017-11-29 15:30 [PATCH 00/36] hrtimer: Provide softirq context hrtimers Anna-Maria Gleixner
                   ` (11 preceding siblings ...)
  2017-11-29 15:30 ` [PATCH v3 12/36] hrtimer: Make room in struct hrtimer_cpu_base Anna-Maria Gleixner
@ 2017-11-29 15:30 ` Anna-Maria Gleixner
  2017-11-29 15:30 ` [PATCH v3 14/36] hrtimer: Use accesor functions instead of direct access Anna-Maria Gleixner
                   ` (22 subsequent siblings)
  35 siblings, 0 replies; 45+ messages in thread
From: Anna-Maria Gleixner @ 2017-11-29 15:30 UTC (permalink / raw)
  To: LKML
  Cc: Thomas Gleixner, Peter Zijlstra, Ingo Molnar, keescook,
	Christoph Hellwig, John Stultz, Anna-Maria Gleixner

The hrtimer_cpu_base struct has the CONFIG_HIGH_RES_TIMERS conditional
struct member hres_active. All related functions to this member are
conditional as well.

There is no functional change, when the hres_active member is
unconditional with all related functions and is set to zero during
initialization.

The conditional code sections can be avoided by adding IS_ENABLED(HIGHRES)
conditionals into common functions, which ensures dead code elimination.

Suggested-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Anna-Maria Gleixner <anna-maria@linutronix.de>
---
 include/linux/hrtimer.h | 20 ++++++++------------
 kernel/time/hrtimer.c   | 31 +++++++++++++++----------------
 2 files changed, 23 insertions(+), 28 deletions(-)

diff --git a/include/linux/hrtimer.h b/include/linux/hrtimer.h
index 56e56bcb6f0f..22627b3a33fe 100644
--- a/include/linux/hrtimer.h
+++ b/include/linux/hrtimer.h
@@ -161,8 +161,8 @@ enum  hrtimer_base_type {
  * @cpu:		cpu number
  * @active_bases:	Bitfield to mark bases with active timers
  * @clock_was_set_seq:	Sequence counter of clock was set events
- * @in_hrtirq:		hrtimer_interrupt() is currently executing
  * @hres_active:	State of high resolution mode
+ * @in_hrtirq:		hrtimer_interrupt() is currently executing
  * @hang_detected:	The last hrtimer interrupt detected a hang
  * @expires_next:	absolute time of the next event, is required for remote
  *			hrtimer enqueue
@@ -182,9 +182,9 @@ struct hrtimer_cpu_base {
 	unsigned int			cpu;
 	unsigned int			active_bases;
 	unsigned int			clock_was_set_seq;
+	unsigned int			hres_active	: 1;
 #ifdef CONFIG_HIGH_RES_TIMERS
 	unsigned int			in_hrtirq	: 1,
-					hres_active	: 1,
 					hang_detected	: 1;
 	ktime_t				expires_next;
 	struct hrtimer			*next_timer;
@@ -266,16 +266,17 @@ static inline ktime_t hrtimer_cb_get_time(struct hrtimer *timer)
 	return timer->base->get_time();
 }
 
+static inline int hrtimer_is_hres_active(struct hrtimer *timer)
+{
+	return IS_ENABLED(CONFIG_HIGH_RES_TIMERS) ?
+		timer->base->cpu_base->hres_active : 0;
+}
+
 #ifdef CONFIG_HIGH_RES_TIMERS
 struct clock_event_device;
 
 extern void hrtimer_interrupt(struct clock_event_device *dev);
 
-static inline int hrtimer_is_hres_active(struct hrtimer *timer)
-{
-	return timer->base->cpu_base->hres_active;
-}
-
 /*
  * The resolution of the clocks. The resolution value is returned in
  * the clock_getres() system call to give application programmers an
@@ -298,11 +299,6 @@ extern unsigned int hrtimer_resolution;
 
 #define hrtimer_resolution	(unsigned int)LOW_RES_NSEC
 
-static inline int hrtimer_is_hres_active(struct hrtimer *timer)
-{
-	return 0;
-}
-
 static inline void clock_was_set_delayed(void) { }
 
 #endif
diff --git a/kernel/time/hrtimer.c b/kernel/time/hrtimer.c
index 046b509beed6..8453dca6e0c5 100644
--- a/kernel/time/hrtimer.c
+++ b/kernel/time/hrtimer.c
@@ -512,6 +512,20 @@ static inline ktime_t hrtimer_update_base(struct hrtimer_cpu_base *base)
 					    offs_real, offs_boot, offs_tai);
 }
 
+/*
+ * Is the high resolution mode active ?
+ */
+static inline int __hrtimer_hres_active(struct hrtimer_cpu_base *cpu_base)
+{
+	return IS_ENABLED(CONFIG_HIGH_RES_TIMERS) ?
+		cpu_base->hres_active : 0;
+}
+
+static inline int hrtimer_hres_active(void)
+{
+	return __hrtimer_hres_active(this_cpu_ptr(&hrtimer_bases));
+}
+
 /* High resolution timer related functions */
 #ifdef CONFIG_HIGH_RES_TIMERS
 
@@ -541,19 +555,6 @@ static inline int hrtimer_is_hres_enabled(void)
 }
 
 /*
- * Is the high resolution mode active ?
- */
-static inline int __hrtimer_hres_active(struct hrtimer_cpu_base *cpu_base)
-{
-	return cpu_base->hres_active;
-}
-
-static inline int hrtimer_hres_active(void)
-{
-	return __hrtimer_hres_active(this_cpu_ptr(&hrtimer_bases));
-}
-
-/*
  * Reprogram the event source with checking both queues for the
  * next event
  * Called with interrupts disabled and base->lock held
@@ -661,7 +662,6 @@ static void hrtimer_reprogram(struct hrtimer *timer,
 static inline void hrtimer_init_hres(struct hrtimer_cpu_base *base)
 {
 	base->expires_next = KTIME_MAX;
-	base->hres_active = 0;
 }
 
 /*
@@ -720,8 +720,6 @@ void clock_was_set_delayed(void)
 
 #else
 
-static inline int __hrtimer_hres_active(struct hrtimer_cpu_base *b) { return 0; }
-static inline int hrtimer_hres_active(void) { return 0; }
 static inline int hrtimer_is_hres_enabled(void) { return 0; }
 static inline void hrtimer_switch_to_hres(void) { }
 static inline void
@@ -1600,6 +1598,7 @@ int hrtimers_prepare_cpu(unsigned int cpu)
 	}
 
 	cpu_base->cpu = cpu;
+	cpu_base->hres_active = 0;
 	hrtimer_init_hres(cpu_base);
 	return 0;
 }
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 45+ messages in thread

* [PATCH v3 14/36] hrtimer: Use accesor functions instead of direct access
  2017-11-29 15:30 [PATCH 00/36] hrtimer: Provide softirq context hrtimers Anna-Maria Gleixner
                   ` (12 preceding siblings ...)
  2017-11-29 15:30 ` [PATCH v3 13/36] hrtimer: Reduce conditional code (hres_active) Anna-Maria Gleixner
@ 2017-11-29 15:30 ` Anna-Maria Gleixner
  2017-11-29 15:30 ` [PATCH v3 15/36] hrtimer: Make the remote enqueue check unconditional Anna-Maria Gleixner
                   ` (21 subsequent siblings)
  35 siblings, 0 replies; 45+ messages in thread
From: Anna-Maria Gleixner @ 2017-11-29 15:30 UTC (permalink / raw)
  To: LKML
  Cc: Thomas Gleixner, Peter Zijlstra, Ingo Molnar, keescook,
	Christoph Hellwig, John Stultz, Anna-Maria Gleixner

__hrtimer_hres_active() is now available unconditionally. Replace the
direct access to hrtimer_cpu_base.hres_active.

No functional change.

Signed-off-by: Anna-Maria Gleixner <anna-maria@linutronix.de>
---
 kernel/time/hrtimer.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/kernel/time/hrtimer.c b/kernel/time/hrtimer.c
index 8453dca6e0c5..0a1804c65a15 100644
--- a/kernel/time/hrtimer.c
+++ b/kernel/time/hrtimer.c
@@ -564,7 +564,7 @@ hrtimer_force_reprogram(struct hrtimer_cpu_base *cpu_base, int skip_equal)
 {
 	ktime_t expires_next;
 
-	if (!cpu_base->hres_active)
+	if (!__hrtimer_hres_active(cpu_base))
 		return;
 
 	expires_next = __hrtimer_get_next_event(cpu_base);
@@ -673,7 +673,7 @@ static void retrigger_next_event(void *arg)
 {
 	struct hrtimer_cpu_base *base = this_cpu_ptr(&hrtimer_bases);
 
-	if (!base->hres_active)
+	if (!__hrtimer_hres_active(base))
 		return;
 
 	raw_spin_lock(&base->lock);
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 45+ messages in thread

* [PATCH v3 15/36] hrtimer: Make the remote enqueue check unconditional
  2017-11-29 15:30 [PATCH 00/36] hrtimer: Provide softirq context hrtimers Anna-Maria Gleixner
                   ` (13 preceding siblings ...)
  2017-11-29 15:30 ` [PATCH v3 14/36] hrtimer: Use accesor functions instead of direct access Anna-Maria Gleixner
@ 2017-11-29 15:30 ` Anna-Maria Gleixner
  2017-11-29 15:30 ` [PATCH v3 16/36] hrtimer: Make hrtimer_cpu_base.next_timer handling unconditional Anna-Maria Gleixner
                   ` (20 subsequent siblings)
  35 siblings, 0 replies; 45+ messages in thread
From: Anna-Maria Gleixner @ 2017-11-29 15:30 UTC (permalink / raw)
  To: LKML
  Cc: Thomas Gleixner, Peter Zijlstra, Ingo Molnar, keescook,
	Christoph Hellwig, John Stultz, Anna-Maria Gleixner

hrtimer_cpu_base.expires_next is used to cache the next event armed in the
timer hardware. The value is used to check whether an hrtimer can be
enqueued remotely. If the new hrtimer is expiring before expires_next, then
remote enqueue is not possible as the remote hrtimer hardware cannot be
accessed for reprogramming to an earlier expiry time.

The remote enqueue check is currently conditional on
CONFIG_HIGH_RES_TIMERS=y and hrtimer_cpu_base.hres_active. There is no
compelling reason to make this conditional.

Move hrtimer_cpu_base.expires_next out of the CONFIG_HIGH_RES_TIMERS=y
guarded area and remove the conditionals in hrtimer_check_target().

The check is currently a NOOP for the CONFIG_HIGH_RES_TIMERS=n and the
!hrtimer_cpu_base.hres_active case because in these cases nothing updates
hrtimer_cpu_base.expires_next yet. This will be changed with later patches
which further reduce the #ifdef zoo in this code.

Signed-off-by: Anna-Maria Gleixner <anna-maria@linutronix.de>
---
 include/linux/hrtimer.h |  6 +++---
 kernel/time/hrtimer.c   | 26 ++++++--------------------
 2 files changed, 9 insertions(+), 23 deletions(-)

diff --git a/include/linux/hrtimer.h b/include/linux/hrtimer.h
index 22627b3a33fe..bb7270e8bc37 100644
--- a/include/linux/hrtimer.h
+++ b/include/linux/hrtimer.h
@@ -164,13 +164,13 @@ enum  hrtimer_base_type {
  * @hres_active:	State of high resolution mode
  * @in_hrtirq:		hrtimer_interrupt() is currently executing
  * @hang_detected:	The last hrtimer interrupt detected a hang
- * @expires_next:	absolute time of the next event, is required for remote
- *			hrtimer enqueue
  * @next_timer:		Pointer to the first expiring timer
  * @nr_events:		Total number of hrtimer interrupt events
  * @nr_retries:		Total number of hrtimer interrupt retries
  * @nr_hangs:		Total number of hrtimer interrupt hangs
  * @max_hang_time:	Maximum time spent in hrtimer_interrupt
+ * @expires_next:	absolute time of the next event, is required for remote
+ *			hrtimer enqueue
  * @clock_base:		array of clock bases for this cpu
  *
  * Note: next_timer is just an optimization for __remove_hrtimer().
@@ -186,13 +186,13 @@ struct hrtimer_cpu_base {
 #ifdef CONFIG_HIGH_RES_TIMERS
 	unsigned int			in_hrtirq	: 1,
 					hang_detected	: 1;
-	ktime_t				expires_next;
 	struct hrtimer			*next_timer;
 	unsigned int			nr_events;
 	unsigned short			nr_retries;
 	unsigned short			nr_hangs;
 	unsigned int			max_hang_time;
 #endif
+	ktime_t				expires_next;
 	struct hrtimer_clock_base	clock_base[HRTIMER_MAX_CLOCK_BASES];
 } ____cacheline_aligned;
 
diff --git a/kernel/time/hrtimer.c b/kernel/time/hrtimer.c
index 0a1804c65a15..069e3dfd0e3c 100644
--- a/kernel/time/hrtimer.c
+++ b/kernel/time/hrtimer.c
@@ -154,26 +154,21 @@ struct hrtimer_clock_base *lock_hrtimer_base(const struct hrtimer *timer,
 }
 
 /*
- * With HIGHRES=y we do not migrate the timer when it is expiring
- * before the next event on the target cpu because we cannot reprogram
- * the target cpu hardware and we would cause it to fire late.
+ * We do not migrate the timer when it is expiring before the next
+ * event on the target cpu. When high resolution is enabled, we cannot
+ * reprogram the target cpu hardware and we would cause it to fire
+ * late. To keep it simple, we handle the high resolution enabled and
+ * disabled case similar.
  *
  * Called with cpu_base->lock of target cpu held.
  */
 static int
 hrtimer_check_target(struct hrtimer *timer, struct hrtimer_clock_base *new_base)
 {
-#ifdef CONFIG_HIGH_RES_TIMERS
 	ktime_t expires;
 
-	if (!new_base->cpu_base->hres_active)
-		return 0;
-
 	expires = ktime_sub(hrtimer_get_expires(timer), new_base->offset);
 	return expires <= new_base->cpu_base->expires_next;
-#else
-	return 0;
-#endif
 }
 
 static inline
@@ -657,14 +652,6 @@ static void hrtimer_reprogram(struct hrtimer *timer,
 }
 
 /*
- * Initialize the high resolution related parts of cpu_base
- */
-static inline void hrtimer_init_hres(struct hrtimer_cpu_base *base)
-{
-	base->expires_next = KTIME_MAX;
-}
-
-/*
  * Retrigger next event is called after clock was set
  *
  * Called with interrupts disabled via on_each_cpu()
@@ -729,7 +716,6 @@ static inline int hrtimer_reprogram(struct hrtimer *timer,
 {
 	return 0;
 }
-static inline void hrtimer_init_hres(struct hrtimer_cpu_base *base) { }
 static inline void retrigger_next_event(void *arg) { }
 
 #endif /* CONFIG_HIGH_RES_TIMERS */
@@ -1599,7 +1585,7 @@ int hrtimers_prepare_cpu(unsigned int cpu)
 
 	cpu_base->cpu = cpu;
 	cpu_base->hres_active = 0;
-	hrtimer_init_hres(cpu_base);
+	cpu_base->expires_next = KTIME_MAX;
 	return 0;
 }
 
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 45+ messages in thread

* [PATCH v3 16/36] hrtimer: Make hrtimer_cpu_base.next_timer handling unconditional
  2017-11-29 15:30 [PATCH 00/36] hrtimer: Provide softirq context hrtimers Anna-Maria Gleixner
                   ` (14 preceding siblings ...)
  2017-11-29 15:30 ` [PATCH v3 15/36] hrtimer: Make the remote enqueue check unconditional Anna-Maria Gleixner
@ 2017-11-29 15:30 ` Anna-Maria Gleixner
  2017-11-29 15:30 ` [PATCH v3 17/36] hrtimer: Make hrtimer_reprogramm() unconditional Anna-Maria Gleixner
                   ` (19 subsequent siblings)
  35 siblings, 0 replies; 45+ messages in thread
From: Anna-Maria Gleixner @ 2017-11-29 15:30 UTC (permalink / raw)
  To: LKML
  Cc: Thomas Gleixner, Peter Zijlstra, Ingo Molnar, keescook,
	Christoph Hellwig, John Stultz, Anna-Maria Gleixner

hrtimer_cpu_base.next_timer stores the pointer to the next expiring timer
in a cpu base.

This pointer cannot be dereferenced and is solely used to check whether a
hrtimer which is removed is the hrtimer which is the first to expire in the
CPU base. If this is the case, then the timer hardware needs to be
reprogrammed to avoid an extra interrupt for nothing.

Again, this is conditional functionality, but there is no compelling reason
to make this conditional. As a preparation, hrtimer_cpu_base.next_timer
needs to be available unconditonal. Aside of that the upcoming support for
softirq based hrtimers requires access to this pointer unconditionally.

Make the update of hrtimer_cpu_base.next_timer unconditional and remove the
ifdef cruft. The impact on CONFIG_HIGH_RES_TIMERS=n && CONFIG_NOHZ=n is
marginal as it's just a store on an already dirtied cacheline.

No functional change.

Signed-off-by: Anna-Maria Gleixner <anna-maria@linutronix.de>
---
 include/linux/hrtimer.h |  4 ++--
 kernel/time/hrtimer.c   | 12 ++----------
 2 files changed, 4 insertions(+), 12 deletions(-)

diff --git a/include/linux/hrtimer.h b/include/linux/hrtimer.h
index bb7270e8bc37..2d3e1d678a4d 100644
--- a/include/linux/hrtimer.h
+++ b/include/linux/hrtimer.h
@@ -164,13 +164,13 @@ enum  hrtimer_base_type {
  * @hres_active:	State of high resolution mode
  * @in_hrtirq:		hrtimer_interrupt() is currently executing
  * @hang_detected:	The last hrtimer interrupt detected a hang
- * @next_timer:		Pointer to the first expiring timer
  * @nr_events:		Total number of hrtimer interrupt events
  * @nr_retries:		Total number of hrtimer interrupt retries
  * @nr_hangs:		Total number of hrtimer interrupt hangs
  * @max_hang_time:	Maximum time spent in hrtimer_interrupt
  * @expires_next:	absolute time of the next event, is required for remote
  *			hrtimer enqueue
+ * @next_timer:		Pointer to the first expiring timer
  * @clock_base:		array of clock bases for this cpu
  *
  * Note: next_timer is just an optimization for __remove_hrtimer().
@@ -186,13 +186,13 @@ struct hrtimer_cpu_base {
 #ifdef CONFIG_HIGH_RES_TIMERS
 	unsigned int			in_hrtirq	: 1,
 					hang_detected	: 1;
-	struct hrtimer			*next_timer;
 	unsigned int			nr_events;
 	unsigned short			nr_retries;
 	unsigned short			nr_hangs;
 	unsigned int			max_hang_time;
 #endif
 	ktime_t				expires_next;
+	struct hrtimer			*next_timer;
 	struct hrtimer_clock_base	clock_base[HRTIMER_MAX_CLOCK_BASES];
 } ____cacheline_aligned;
 
diff --git a/kernel/time/hrtimer.c b/kernel/time/hrtimer.c
index 069e3dfd0e3c..b424331dffae 100644
--- a/kernel/time/hrtimer.c
+++ b/kernel/time/hrtimer.c
@@ -459,21 +459,13 @@ __next_base(struct hrtimer_cpu_base *cpu_base, unsigned int *active)
 	while ((base = __next_base((cpu_base), &(active))))
 
 #if defined(CONFIG_NO_HZ_COMMON) || defined(CONFIG_HIGH_RES_TIMERS)
-static inline void hrtimer_update_next_timer(struct hrtimer_cpu_base *cpu_base,
-					     struct hrtimer *timer)
-{
-#ifdef CONFIG_HIGH_RES_TIMERS
-	cpu_base->next_timer = timer;
-#endif
-}
-
 static ktime_t __hrtimer_get_next_event(struct hrtimer_cpu_base *cpu_base)
 {
 	struct hrtimer_clock_base *base;
 	unsigned int active = cpu_base->active_bases;
 	ktime_t expires, expires_next = KTIME_MAX;
 
-	hrtimer_update_next_timer(cpu_base, NULL);
+	cpu_base->next_timer = NULL;
 	for_each_active_base(base, cpu_base, active) {
 		struct timerqueue_node *next;
 		struct hrtimer *timer;
@@ -483,7 +475,7 @@ static ktime_t __hrtimer_get_next_event(struct hrtimer_cpu_base *cpu_base)
 		expires = ktime_sub(hrtimer_get_expires(timer), base->offset);
 		if (expires < expires_next) {
 			expires_next = expires;
-			hrtimer_update_next_timer(cpu_base, timer);
+			cpu_base->next_timer = timer;
 		}
 	}
 	/*
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 45+ messages in thread

* [PATCH v3 17/36] hrtimer: Make hrtimer_reprogramm() unconditional
  2017-11-29 15:30 [PATCH 00/36] hrtimer: Provide softirq context hrtimers Anna-Maria Gleixner
                   ` (15 preceding siblings ...)
  2017-11-29 15:30 ` [PATCH v3 16/36] hrtimer: Make hrtimer_cpu_base.next_timer handling unconditional Anna-Maria Gleixner
@ 2017-11-29 15:30 ` Anna-Maria Gleixner
  2017-11-29 15:30 ` [PATCH v3 18/36] hrtimer: Make hrtimer_force_reprogramm() unconditionally available Anna-Maria Gleixner
                   ` (18 subsequent siblings)
  35 siblings, 0 replies; 45+ messages in thread
From: Anna-Maria Gleixner @ 2017-11-29 15:30 UTC (permalink / raw)
  To: LKML
  Cc: Thomas Gleixner, Peter Zijlstra, Ingo Molnar, keescook,
	Christoph Hellwig, John Stultz, Anna-Maria Gleixner

hrtimer_reprogram() needs to be available unconditionally for softirq based
hrtimers. Move the function and all required struct members out of the
CONFIG_HIGH_RES_TIMERS #ifdef.

There is no functional change because hrtimer_reprogram() is only invoked
when hrtimer_cpu_base.hres_active is true. Making it unconditional
increases the text size for the CONFIG_HIGH_RES_TIMERS=n case, but avoids
replication of that code for the upcoming softirq based hrtimers support.

Signed-off-by: Anna-Maria Gleixner <anna-maria@linutronix.de>
---
 include/linux/hrtimer.h |   6 +--
 kernel/time/hrtimer.c   | 129 +++++++++++++++++++++++-------------------------
 2 files changed, 65 insertions(+), 70 deletions(-)

diff --git a/include/linux/hrtimer.h b/include/linux/hrtimer.h
index 2d3e1d678a4d..98ed35767ac5 100644
--- a/include/linux/hrtimer.h
+++ b/include/linux/hrtimer.h
@@ -182,10 +182,10 @@ struct hrtimer_cpu_base {
 	unsigned int			cpu;
 	unsigned int			active_bases;
 	unsigned int			clock_was_set_seq;
-	unsigned int			hres_active	: 1;
-#ifdef CONFIG_HIGH_RES_TIMERS
-	unsigned int			in_hrtirq	: 1,
+	unsigned int			hres_active	: 1,
+					in_hrtirq	: 1,
 					hang_detected	: 1;
+#ifdef CONFIG_HIGH_RES_TIMERS
 	unsigned int			nr_events;
 	unsigned short			nr_retries;
 	unsigned short			nr_hangs;
diff --git a/kernel/time/hrtimer.c b/kernel/time/hrtimer.c
index b424331dffae..c9173699b549 100644
--- a/kernel/time/hrtimer.c
+++ b/kernel/time/hrtimer.c
@@ -582,68 +582,6 @@ hrtimer_force_reprogram(struct hrtimer_cpu_base *cpu_base, int skip_equal)
 }
 
 /*
- * When a timer is enqueued and expires earlier than the already enqueued
- * timers, we have to check, whether it expires earlier than the timer for
- * which the clock event device was armed.
- *
- * Called with interrupts disabled and base->cpu_base.lock held
- */
-static void hrtimer_reprogram(struct hrtimer *timer,
-			      struct hrtimer_clock_base *base)
-{
-	struct hrtimer_cpu_base *cpu_base = this_cpu_ptr(&hrtimer_bases);
-	ktime_t expires = ktime_sub(hrtimer_get_expires(timer), base->offset);
-
-	WARN_ON_ONCE(hrtimer_get_expires_tv64(timer) < 0);
-
-	/*
-	 * If the timer is not on the current cpu, we cannot reprogram
-	 * the other cpus clock event device.
-	 */
-	if (base->cpu_base != cpu_base)
-		return;
-
-	/*
-	 * If the hrtimer interrupt is running, then it will
-	 * reevaluate the clock bases and reprogram the clock event
-	 * device. The callbacks are always executed in hard interrupt
-	 * context so we don't need an extra check for a running
-	 * callback.
-	 */
-	if (cpu_base->in_hrtirq)
-		return;
-
-	/*
-	 * CLOCK_REALTIME timer might be requested with an absolute
-	 * expiry time which is less than base->offset. Set it to 0.
-	 */
-	if (expires < 0)
-		expires = 0;
-
-	if (expires >= cpu_base->expires_next)
-		return;
-
-	/* Update the pointer to the next expiring timer */
-	cpu_base->next_timer = timer;
-
-	/*
-	 * If a hang was detected in the last timer interrupt then we
-	 * do not schedule a timer which is earlier than the expiry
-	 * which we enforced in the hang detection. We want the system
-	 * to make progress.
-	 */
-	if (cpu_base->hang_detected)
-		return;
-
-	/*
-	 * Program the timer hardware. We enforce the expiry for
-	 * events which are already in the past.
-	 */
-	cpu_base->expires_next = expires;
-	tick_program_event(expires, 1);
-}
-
-/*
  * Retrigger next event is called after clock was set
  *
  * Called with interrupts disabled via on_each_cpu()
@@ -703,16 +641,73 @@ static inline int hrtimer_is_hres_enabled(void) { return 0; }
 static inline void hrtimer_switch_to_hres(void) { }
 static inline void
 hrtimer_force_reprogram(struct hrtimer_cpu_base *base, int skip_equal) { }
-static inline int hrtimer_reprogram(struct hrtimer *timer,
-				    struct hrtimer_clock_base *base)
-{
-	return 0;
-}
 static inline void retrigger_next_event(void *arg) { }
 
 #endif /* CONFIG_HIGH_RES_TIMERS */
 
 /*
+ * When a timer is enqueued and expires earlier than the already enqueued
+ * timers, we have to check, whether it expires earlier than the timer for
+ * which the clock event device was armed.
+ *
+ * Called with interrupts disabled and base->cpu_base.lock held
+ */
+static void hrtimer_reprogram(struct hrtimer *timer,
+			      struct hrtimer_clock_base *base)
+{
+	struct hrtimer_cpu_base *cpu_base = this_cpu_ptr(&hrtimer_bases);
+	ktime_t expires = ktime_sub(hrtimer_get_expires(timer), base->offset);
+
+	WARN_ON_ONCE(hrtimer_get_expires_tv64(timer) < 0);
+
+	/*
+	 * If the timer is not on the current cpu, we cannot reprogram
+	 * the other cpus clock event device.
+	 */
+	if (base->cpu_base != cpu_base)
+		return;
+
+	/*
+	 * If the hrtimer interrupt is running, then it will
+	 * reevaluate the clock bases and reprogram the clock event
+	 * device. The callbacks are always executed in hard interrupt
+	 * context so we don't need an extra check for a running
+	 * callback.
+	 */
+	if (cpu_base->in_hrtirq)
+		return;
+
+	/*
+	 * CLOCK_REALTIME timer might be requested with an absolute
+	 * expiry time which is less than base->offset. Set it to 0.
+	 */
+	if (expires < 0)
+		expires = 0;
+
+	if (expires >= cpu_base->expires_next)
+		return;
+
+	/* Update the pointer to the next expiring timer */
+	cpu_base->next_timer = timer;
+
+	/*
+	 * If a hang was detected in the last timer interrupt then we
+	 * do not schedule a timer which is earlier than the expiry
+	 * which we enforced in the hang detection. We want the system
+	 * to make progress.
+	 */
+	if (cpu_base->hang_detected)
+		return;
+
+	/*
+	 * Program the timer hardware. We enforce the expiry for
+	 * events which are already in the past.
+	 */
+	cpu_base->expires_next = expires;
+	tick_program_event(expires, 1);
+}
+
+/*
  * Clock realtime was set
  *
  * Change the offset of the realtime clock vs. the monotonic
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 45+ messages in thread

* [PATCH v3 18/36] hrtimer: Make hrtimer_force_reprogramm() unconditionally available
  2017-11-29 15:30 [PATCH 00/36] hrtimer: Provide softirq context hrtimers Anna-Maria Gleixner
                   ` (16 preceding siblings ...)
  2017-11-29 15:30 ` [PATCH v3 17/36] hrtimer: Make hrtimer_reprogramm() unconditional Anna-Maria Gleixner
@ 2017-11-29 15:30 ` Anna-Maria Gleixner
  2017-11-29 15:30 ` [PATCH v3 19/36] hrtimer: Unify handling of hrtimer remove Anna-Maria Gleixner
                   ` (17 subsequent siblings)
  35 siblings, 0 replies; 45+ messages in thread
From: Anna-Maria Gleixner @ 2017-11-29 15:30 UTC (permalink / raw)
  To: LKML
  Cc: Thomas Gleixner, Peter Zijlstra, Ingo Molnar, keescook,
	Christoph Hellwig, John Stultz, Anna-Maria Gleixner

hrtimer_force_reprogram() needs to be available unconditionally for softirq
based hrtimers. Move the function and all required struct members out of
the CONFIG_HIGH_RES_TIMERS #ifdef.

There is no functional change because hrtimer_force_reprogram() is only
invoked when hrtimer_cpu_base.hres_active is true and
CONFIG_HIGH_RES_TIMERS=y.

Making it unconditional increases the text size for the
CONFIG_HIGH_RES_TIMERS=n case slightly, but avoids replication of that code
for the upcoming softirq based hrtimers support. Most of the code gets
eliminated in the CONFIG_HIGH_RES_TIMERS=n case by the compiler.

Signed-off-by: Anna-Maria Gleixner <anna-maria@linutronix.de>
---
 kernel/time/hrtimer.c | 58 +++++++++++++++++++++++++--------------------------
 1 file changed, 28 insertions(+), 30 deletions(-)

diff --git a/kernel/time/hrtimer.c b/kernel/time/hrtimer.c
index c9173699b549..0502b8b6e0bc 100644
--- a/kernel/time/hrtimer.c
+++ b/kernel/time/hrtimer.c
@@ -513,34 +513,6 @@ static inline int hrtimer_hres_active(void)
 	return __hrtimer_hres_active(this_cpu_ptr(&hrtimer_bases));
 }
 
-/* High resolution timer related functions */
-#ifdef CONFIG_HIGH_RES_TIMERS
-
-/*
- * High resolution timer enabled ?
- */
-static bool hrtimer_hres_enabled __read_mostly  = true;
-unsigned int hrtimer_resolution __read_mostly = LOW_RES_NSEC;
-EXPORT_SYMBOL_GPL(hrtimer_resolution);
-
-/*
- * Enable / Disable high resolution mode
- */
-static int __init setup_hrtimer_hres(char *str)
-{
-	return (kstrtobool(str, &hrtimer_hres_enabled) == 0);
-}
-
-__setup("highres=", setup_hrtimer_hres);
-
-/*
- * hrtimer_high_res_enabled - query, if the highres mode is enabled
- */
-static inline int hrtimer_is_hres_enabled(void)
-{
-	return hrtimer_hres_enabled;
-}
-
 /*
  * Reprogram the event source with checking both queues for the
  * next event
@@ -581,6 +553,34 @@ hrtimer_force_reprogram(struct hrtimer_cpu_base *cpu_base, int skip_equal)
 	tick_program_event(cpu_base->expires_next, 1);
 }
 
+/* High resolution timer related functions */
+#ifdef CONFIG_HIGH_RES_TIMERS
+
+/*
+ * High resolution timer enabled ?
+ */
+static bool hrtimer_hres_enabled __read_mostly  = true;
+unsigned int hrtimer_resolution __read_mostly = LOW_RES_NSEC;
+EXPORT_SYMBOL_GPL(hrtimer_resolution);
+
+/*
+ * Enable / Disable high resolution mode
+ */
+static int __init setup_hrtimer_hres(char *str)
+{
+	return (kstrtobool(str, &hrtimer_hres_enabled) == 0);
+}
+
+__setup("highres=", setup_hrtimer_hres);
+
+/*
+ * hrtimer_high_res_enabled - query, if the highres mode is enabled
+ */
+static inline int hrtimer_is_hres_enabled(void)
+{
+	return hrtimer_hres_enabled;
+}
+
 /*
  * Retrigger next event is called after clock was set
  *
@@ -639,8 +639,6 @@ void clock_was_set_delayed(void)
 
 static inline int hrtimer_is_hres_enabled(void) { return 0; }
 static inline void hrtimer_switch_to_hres(void) { }
-static inline void
-hrtimer_force_reprogram(struct hrtimer_cpu_base *base, int skip_equal) { }
 static inline void retrigger_next_event(void *arg) { }
 
 #endif /* CONFIG_HIGH_RES_TIMERS */
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 45+ messages in thread

* [PATCH v3 19/36] hrtimer: Unify handling of hrtimer remove
  2017-11-29 15:30 [PATCH 00/36] hrtimer: Provide softirq context hrtimers Anna-Maria Gleixner
                   ` (17 preceding siblings ...)
  2017-11-29 15:30 ` [PATCH v3 18/36] hrtimer: Make hrtimer_force_reprogramm() unconditionally available Anna-Maria Gleixner
@ 2017-11-29 15:30 ` Anna-Maria Gleixner
  2017-11-29 15:30 ` [PATCH v3 20/36] hrtimer: Unify handling of remote enqueue Anna-Maria Gleixner
                   ` (16 subsequent siblings)
  35 siblings, 0 replies; 45+ messages in thread
From: Anna-Maria Gleixner @ 2017-11-29 15:30 UTC (permalink / raw)
  To: LKML
  Cc: Thomas Gleixner, Peter Zijlstra, Ingo Molnar, keescook,
	Christoph Hellwig, John Stultz, Anna-Maria Gleixner

When the first hrtimer on the current CPU is removed,
hrtimer_force_reprogram() is invoked but only when
CONFIG_HIGH_RES_TIMERS=y and hrtimer_cpu_base.hres_active is set.

hrtimer_force_reprogram() updates hrtimer_cpu_base.expires_next and
reprograms the clock event device. When CONFIG_HIGH_RES_TIMERS=y and
hrtimer_cpu_base.hres_active is set, a pointless hrtimer interrupt can be
prevented.

hrtimer_check_target() makes the 'can remote enqueue' decision. As soon as
hrtimer_check_target() is unconditionally available and
hrtimer_cpu_base.expires_next is updated by hrtimer_reprogram(),
hrtimer_force_reprogram() needs to be available unconditionally as well to
prevent the following scenario with CONFIG_HIGH_RES_TIMERS=n:

- the first hrtimer on this CPU is removed and hrtimer_force_reprogram() is
  not executed

- CPU goes idle (next timer is calculated and hrtimers are taken into
  account)

- a hrtimer is enqueued remote on the idle CPU: hrtimer_check_target()
  compares expiry value and hrtimer_cpu_base.expires_next. The expiry value
  is after expires_next, so the hrtimer is enqueued. This timer will fire
  late, if it expires before the effective first hrtimer on this CPU and
  the comparison was with an outdated expires_next value.

To prevent this scenario, make hrtimer_force_reprogram() unconditional
except the effective reprogramming part, which gets eliminated by the
compiler in the CONFIG_HIGH_RES_TIMERS=n case.

Signed-off-by: Anna-Maria Gleixner <anna-maria@linutronix.de>
---
 kernel/time/hrtimer.c | 10 ++++------
 1 file changed, 4 insertions(+), 6 deletions(-)

diff --git a/kernel/time/hrtimer.c b/kernel/time/hrtimer.c
index 0502b8b6e0bc..617573109398 100644
--- a/kernel/time/hrtimer.c
+++ b/kernel/time/hrtimer.c
@@ -523,9 +523,6 @@ hrtimer_force_reprogram(struct hrtimer_cpu_base *cpu_base, int skip_equal)
 {
 	ktime_t expires_next;
 
-	if (!__hrtimer_hres_active(cpu_base))
-		return;
-
 	expires_next = __hrtimer_get_next_event(cpu_base);
 
 	if (skip_equal && expires_next == cpu_base->expires_next)
@@ -534,6 +531,9 @@ hrtimer_force_reprogram(struct hrtimer_cpu_base *cpu_base, int skip_equal)
 	cpu_base->expires_next = expires_next;
 
 	/*
+	 * If hres is not active, hardware does not have to be
+	 * reprogrammed yet.
+	 *
 	 * If a hang was detected in the last timer interrupt then we
 	 * leave the hang delay active in the hardware. We want the
 	 * system to make progress. That also prevents the following
@@ -547,7 +547,7 @@ hrtimer_force_reprogram(struct hrtimer_cpu_base *cpu_base, int skip_equal)
 	 * set. So we'd effectivly block all timers until the T2 event
 	 * fires.
 	 */
-	if (cpu_base->hang_detected)
+	if (!__hrtimer_hres_active(cpu_base) || cpu_base->hang_detected)
 		return;
 
 	tick_program_event(cpu_base->expires_next, 1);
@@ -846,7 +846,6 @@ static void __remove_hrtimer(struct hrtimer *timer,
 	if (!timerqueue_del(&base->active, &timer->node))
 		cpu_base->active_bases &= ~(1 << base->index);
 
-#ifdef CONFIG_HIGH_RES_TIMERS
 	/*
 	 * Note: If reprogram is false we do not update
 	 * cpu_base->next_timer. This happens when we remove the first
@@ -857,7 +856,6 @@ static void __remove_hrtimer(struct hrtimer *timer,
 	 */
 	if (reprogram && timer == cpu_base->next_timer)
 		hrtimer_force_reprogram(cpu_base, 1);
-#endif
 }
 
 /*
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 45+ messages in thread

* [PATCH v3 20/36] hrtimer: Unify handling of remote enqueue
  2017-11-29 15:30 [PATCH 00/36] hrtimer: Provide softirq context hrtimers Anna-Maria Gleixner
                   ` (18 preceding siblings ...)
  2017-11-29 15:30 ` [PATCH v3 19/36] hrtimer: Unify handling of hrtimer remove Anna-Maria Gleixner
@ 2017-11-29 15:30 ` Anna-Maria Gleixner
  2017-11-29 15:30 ` [PATCH v3 21/36] hrtimer: Make remote enqueue decision less restrictive Anna-Maria Gleixner
                   ` (15 subsequent siblings)
  35 siblings, 0 replies; 45+ messages in thread
From: Anna-Maria Gleixner @ 2017-11-29 15:30 UTC (permalink / raw)
  To: LKML
  Cc: Thomas Gleixner, Peter Zijlstra, Ingo Molnar, keescook,
	Christoph Hellwig, John Stultz, Anna-Maria Gleixner

hrtimer_reprogram() is conditionally invoked from hrtimer_start_range_ns()
when hrtimer_cpu_base.hres_active is true.

In the !hres_active case there is a special condition for the nohz_active
case:

  If the newly enqueued timer expires before the first expiring timer on a
  remote CPU then the remote CPU needs to be notified and woken up from a
  NOHZ idle sleep to take the new first expiring timer into account.

Previous changes have already established the prerequisites to make the
remote enqueue behaviour the same whether high resolution mode is active or
not:

  If the to be enqueued timer expires before the first expiring timer on a
  remote CPU, then it cannot be enqueued there.

This was done for the high resolution mode because there is no way to
access the remote CPU timer hardware. The same is true for NOHZ, but was
handled differently by unconditionally enqueuing the timer and waking up
the remote CPU so it can reprogram its timer. Again there is no compelling
reason for this difference.

hrtimer_check_target(), which makes the 'can remote enqueue' decision is
already unconditional, but not yet functional because nothing updates
hrtimer_cpu_base.expires_next in the !hres_active case.

To unify this the following changes are required:

 1) Make the store of the new first expiry time unconditonal in
    hrtimer_reprogram() and check __hrtimer_hres_active() before proceeding
    to the actual hardware access. This check also lets the compiler
    eliminate the rest of the function in case of CONFIG_HIGH_RES_TIMERS=n.

 2) Invoke hrtimer_reprogram() unconditionally from
    hrtimer_start_range_ns()

 3) Remove the remote wakeup special case for the !high_res && nohz_active
    case.

Confine the timers_nohz_active static key to timer.c which is the only user
now.

Signed-off-by: Anna-Maria Gleixner <anna-maria@linutronix.de>
---
 kernel/time/hrtimer.c       | 18 ++++++------------
 kernel/time/tick-internal.h |  6 ------
 kernel/time/timer.c         |  9 ++++++++-
 3 files changed, 14 insertions(+), 19 deletions(-)

diff --git a/kernel/time/hrtimer.c b/kernel/time/hrtimer.c
index 617573109398..90a8aa9a7fa2 100644
--- a/kernel/time/hrtimer.c
+++ b/kernel/time/hrtimer.c
@@ -687,21 +687,24 @@ static void hrtimer_reprogram(struct hrtimer *timer,
 
 	/* Update the pointer to the next expiring timer */
 	cpu_base->next_timer = timer;
+	cpu_base->expires_next = expires;
 
 	/*
+	 * If hres is not active, hardware does not have to be
+	 * programmed yet.
+	 *
 	 * If a hang was detected in the last timer interrupt then we
 	 * do not schedule a timer which is earlier than the expiry
 	 * which we enforced in the hang detection. We want the system
 	 * to make progress.
 	 */
-	if (cpu_base->hang_detected)
+	if (!__hrtimer_hres_active(cpu_base) || cpu_base->hang_detected)
 		return;
 
 	/*
 	 * Program the timer hardware. We enforce the expiry for
 	 * events which are already in the past.
 	 */
-	cpu_base->expires_next = expires;
 	tick_program_event(expires, 1);
 }
 
@@ -938,16 +941,7 @@ void hrtimer_start_range_ns(struct hrtimer *timer, ktime_t tim,
 	if (!leftmost)
 		goto unlock;
 
-	if (!hrtimer_is_hres_active(timer)) {
-		/*
-		 * Kick to reschedule the next tick to handle the new timer
-		 * on dynticks target.
-		 */
-		if (is_timers_nohz_active())
-			wake_up_nohz_cpu(new_base->cpu_base->cpu);
-	} else {
-		hrtimer_reprogram(timer, new_base);
-	}
+	hrtimer_reprogram(timer, new_base);
 unlock:
 	unlock_hrtimer_base(timer, &flags);
 }
diff --git a/kernel/time/tick-internal.h b/kernel/time/tick-internal.h
index 4ac74dff59f0..e277284c2831 100644
--- a/kernel/time/tick-internal.h
+++ b/kernel/time/tick-internal.h
@@ -151,18 +151,12 @@ static inline void tick_nohz_init(void) { }
 #ifdef CONFIG_NO_HZ_COMMON
 extern unsigned long tick_nohz_active;
 extern void timers_update_nohz(void);
-extern struct static_key_false timers_nohz_active;
-static inline bool is_timers_nohz_active(void)
-{
-	return static_branch_unlikely(&timers_nohz_active);
-}
 # ifdef CONFIG_SMP
 extern struct static_key_false timers_migration_enabled;
 # endif
 #else /* CONFIG_NO_HZ_COMMON */
 static inline void timers_update_nohz(void) { }
 #define tick_nohz_active (0)
-static inline bool is_timers_nohz_active(void) { return false; }
 #endif
 
 DECLARE_PER_CPU(struct hrtimer_cpu_base, hrtimer_bases);
diff --git a/kernel/time/timer.c b/kernel/time/timer.c
index 1e2140a23044..4b39029fbe99 100644
--- a/kernel/time/timer.c
+++ b/kernel/time/timer.c
@@ -210,7 +210,7 @@ static DEFINE_PER_CPU(struct timer_base, timer_bases[NR_BASES]);
 
 #ifdef CONFIG_NO_HZ_COMMON
 
-DEFINE_STATIC_KEY_FALSE(timers_nohz_active);
+static DEFINE_STATIC_KEY_FALSE(timers_nohz_active);
 static DEFINE_MUTEX(timer_keys_mutex);
 
 static void timer_update_keys(struct work_struct *work);
@@ -260,6 +260,13 @@ int timer_migration_handler(struct ctl_table *table, int write,
 	mutex_unlock(&timer_keys_mutex);
 	return ret;
 }
+
+static inline bool is_timers_nohz_active(void)
+{
+	return static_branch_unlikely(&timers_nohz_active);
+}
+#else
+static inline bool is_timers_nohz_active(void) { return false; }
 #endif /* NO_HZ_COMMON */
 
 static unsigned long round_jiffies_common(unsigned long j, int cpu,
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 45+ messages in thread

* [PATCH v3 21/36] hrtimer: Make remote enqueue decision less restrictive
  2017-11-29 15:30 [PATCH 00/36] hrtimer: Provide softirq context hrtimers Anna-Maria Gleixner
                   ` (19 preceding siblings ...)
  2017-11-29 15:30 ` [PATCH v3 20/36] hrtimer: Unify handling of remote enqueue Anna-Maria Gleixner
@ 2017-11-29 15:30 ` Anna-Maria Gleixner
  2017-11-29 15:30 ` [PATCH v3 22/36] hrtimer: Remove base argument from hrtimer_reprogram() Anna-Maria Gleixner
                   ` (14 subsequent siblings)
  35 siblings, 0 replies; 45+ messages in thread
From: Anna-Maria Gleixner @ 2017-11-29 15:30 UTC (permalink / raw)
  To: LKML
  Cc: Thomas Gleixner, Peter Zijlstra, Ingo Molnar, keescook,
	Christoph Hellwig, John Stultz, Anna-Maria Gleixner

The current decision whether a timer can be queued on a remote CPU checks
for timer->expiry <= remote_cpu_base.expires_next.

This is too restrictive because a timer with the same expiry time as an
existing timer will be enqueued on right-hand size of the existing timer
inside the rbtree, i.e. behind the first expiring timer.

So its safe to allow enqueuing timers with the same expiry time as the
first expiring timer on a remote CPU base.

Signed-off-by: Anna-Maria Gleixner <anna-maria@linutronix.de>
---
 kernel/time/hrtimer.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/kernel/time/hrtimer.c b/kernel/time/hrtimer.c
index 90a8aa9a7fa2..e0521d8f247b 100644
--- a/kernel/time/hrtimer.c
+++ b/kernel/time/hrtimer.c
@@ -168,7 +168,7 @@ hrtimer_check_target(struct hrtimer *timer, struct hrtimer_clock_base *new_base)
 	ktime_t expires;
 
 	expires = ktime_sub(hrtimer_get_expires(timer), new_base->offset);
-	return expires <= new_base->cpu_base->expires_next;
+	return expires < new_base->cpu_base->expires_next;
 }
 
 static inline
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 45+ messages in thread

* [PATCH v3 22/36] hrtimer: Remove base argument from hrtimer_reprogram()
  2017-11-29 15:30 [PATCH 00/36] hrtimer: Provide softirq context hrtimers Anna-Maria Gleixner
                   ` (20 preceding siblings ...)
  2017-11-29 15:30 ` [PATCH v3 21/36] hrtimer: Make remote enqueue decision less restrictive Anna-Maria Gleixner
@ 2017-11-29 15:30 ` Anna-Maria Gleixner
  2017-11-29 15:30 ` [PATCH v3 23/36] hrtimer: Split hrtimer_start_range_ns() Anna-Maria Gleixner
                   ` (13 subsequent siblings)
  35 siblings, 0 replies; 45+ messages in thread
From: Anna-Maria Gleixner @ 2017-11-29 15:30 UTC (permalink / raw)
  To: LKML
  Cc: Thomas Gleixner, Peter Zijlstra, Ingo Molnar, keescook,
	Christoph Hellwig, John Stultz, Anna-Maria Gleixner

hrtimer_reprogram() must have access to the hrtimer_clock_base of the new
first expiring timer to access hrtimer_clock_base.offset for adjusting the
expiry time to CLOCK_MONOTONIC. This is required to evaluate whether the
new left most timer in the hrtimer_clock_base is the first expiring timer
of all clock bases in a hrtimer_cpu_base.

The only user of hrtimer_reprogram() is hrtimer_start_range_ns(), which has
a pointer to hrtimer_clock_base already and hands it in as an argument. But
hrtimer_start_range_ns() will be split for the upcoming support for softirq
based hrtimers to avoid code duplication and will lose the direct access to
the clock base pointer.

Instead of handing in timer and timer->base as an argument remove the base
argument from hrtimer_reprogram() and retrieve the clock base internally.

Signed-off-by: Anna-Maria Gleixner <anna-maria@linutronix.de>
---
 kernel/time/hrtimer.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/kernel/time/hrtimer.c b/kernel/time/hrtimer.c
index e0521d8f247b..771f3463502e 100644
--- a/kernel/time/hrtimer.c
+++ b/kernel/time/hrtimer.c
@@ -650,10 +650,10 @@ static inline void retrigger_next_event(void *arg) { }
  *
  * Called with interrupts disabled and base->cpu_base.lock held
  */
-static void hrtimer_reprogram(struct hrtimer *timer,
-			      struct hrtimer_clock_base *base)
+static void hrtimer_reprogram(struct hrtimer *timer)
 {
 	struct hrtimer_cpu_base *cpu_base = this_cpu_ptr(&hrtimer_bases);
+	struct hrtimer_clock_base *base = timer->base;
 	ktime_t expires = ktime_sub(hrtimer_get_expires(timer), base->offset);
 
 	WARN_ON_ONCE(hrtimer_get_expires_tv64(timer) < 0);
@@ -941,7 +941,7 @@ void hrtimer_start_range_ns(struct hrtimer *timer, ktime_t tim,
 	if (!leftmost)
 		goto unlock;
 
-	hrtimer_reprogram(timer, new_base);
+	hrtimer_reprogram(timer);
 unlock:
 	unlock_hrtimer_base(timer, &flags);
 }
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 45+ messages in thread

* [PATCH v3 23/36] hrtimer: Split hrtimer_start_range_ns()
  2017-11-29 15:30 [PATCH 00/36] hrtimer: Provide softirq context hrtimers Anna-Maria Gleixner
                   ` (21 preceding siblings ...)
  2017-11-29 15:30 ` [PATCH v3 22/36] hrtimer: Remove base argument from hrtimer_reprogram() Anna-Maria Gleixner
@ 2017-11-29 15:30 ` Anna-Maria Gleixner
  2017-11-29 15:30 ` [PATCH v3 24/36] hrtimer: Split __hrtimer_get_next_event() Anna-Maria Gleixner
                   ` (12 subsequent siblings)
  35 siblings, 0 replies; 45+ messages in thread
From: Anna-Maria Gleixner @ 2017-11-29 15:30 UTC (permalink / raw)
  To: LKML
  Cc: Thomas Gleixner, Peter Zijlstra, Ingo Molnar, keescook,
	Christoph Hellwig, John Stultz, Anna-Maria Gleixner

Preparatory patch for softirq based hrtimers to avoid code duplication. No
functional change.

Signed-off-by: Anna-Maria Gleixner <anna-maria@linutronix.de>
---
 kernel/time/hrtimer.c | 44 ++++++++++++++++++++++++--------------------
 1 file changed, 24 insertions(+), 20 deletions(-)

diff --git a/kernel/time/hrtimer.c b/kernel/time/hrtimer.c
index 771f3463502e..c1412e9dcfea 100644
--- a/kernel/time/hrtimer.c
+++ b/kernel/time/hrtimer.c
@@ -907,22 +907,11 @@ static inline ktime_t hrtimer_update_lowres(struct hrtimer *timer, ktime_t tim,
 	return tim;
 }
 
-/**
- * hrtimer_start_range_ns - (re)start an hrtimer
- * @timer:	the timer to be added
- * @tim:	expiry time
- * @delta_ns:	"slack" range for the timer
- * @mode:	timer mode: absolute (HRTIMER_MODE_ABS) or
- *		relative (HRTIMER_MODE_REL), and pinned (HRTIMER_MODE_PINNED)
- */
-void hrtimer_start_range_ns(struct hrtimer *timer, ktime_t tim,
-			    u64 delta_ns, const enum hrtimer_mode mode)
+static int __hrtimer_start_range_ns(struct hrtimer *timer, ktime_t tim,
+				    u64 delta_ns, const enum hrtimer_mode mode,
+				    struct hrtimer_clock_base *base)
 {
-	struct hrtimer_clock_base *base, *new_base;
-	unsigned long flags;
-	int leftmost;
-
-	base = lock_hrtimer_base(timer, &flags);
+	struct hrtimer_clock_base *new_base;
 
 	/* Remove an active timer from the queue: */
 	remove_hrtimer(timer, base, true);
@@ -937,12 +926,27 @@ void hrtimer_start_range_ns(struct hrtimer *timer, ktime_t tim,
 	/* Switch the timer base, if necessary: */
 	new_base = switch_hrtimer_base(timer, base, mode & HRTIMER_MODE_PINNED);
 
-	leftmost = enqueue_hrtimer(timer, new_base, mode);
-	if (!leftmost)
-		goto unlock;
+	return enqueue_hrtimer(timer, new_base, mode);
+}
+/**
+ * hrtimer_start_range_ns - (re)start an hrtimer
+ * @timer:	the timer to be added
+ * @tim:	expiry time
+ * @delta_ns:	"slack" range for the timer
+ * @mode:	timer mode: absolute (HRTIMER_MODE_ABS) or
+ *		relative (HRTIMER_MODE_REL), and pinned (HRTIMER_MODE_PINNED)
+ */
+void hrtimer_start_range_ns(struct hrtimer *timer, ktime_t tim,
+			    u64 delta_ns, const enum hrtimer_mode mode)
+{
+	struct hrtimer_clock_base *base;
+	unsigned long flags;
+
+	base = lock_hrtimer_base(timer, &flags);
+
+	if (__hrtimer_start_range_ns(timer, tim, delta_ns, mode, base))
+		hrtimer_reprogram(timer);
 
-	hrtimer_reprogram(timer);
-unlock:
 	unlock_hrtimer_base(timer, &flags);
 }
 EXPORT_SYMBOL_GPL(hrtimer_start_range_ns);
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 45+ messages in thread

* [PATCH v3 24/36] hrtimer: Split __hrtimer_get_next_event()
  2017-11-29 15:30 [PATCH 00/36] hrtimer: Provide softirq context hrtimers Anna-Maria Gleixner
                   ` (22 preceding siblings ...)
  2017-11-29 15:30 ` [PATCH v3 23/36] hrtimer: Split hrtimer_start_range_ns() Anna-Maria Gleixner
@ 2017-11-29 15:30 ` Anna-Maria Gleixner
  2017-11-29 15:30 ` [PATCH v3 25/36] hrtimer: Use irqsave/irqrestore around __run_hrtimer() Anna-Maria Gleixner
                   ` (11 subsequent siblings)
  35 siblings, 0 replies; 45+ messages in thread
From: Anna-Maria Gleixner @ 2017-11-29 15:30 UTC (permalink / raw)
  To: LKML
  Cc: Thomas Gleixner, Peter Zijlstra, Ingo Molnar, keescook,
	Christoph Hellwig, John Stultz, Anna-Maria Gleixner

Preparatory patch for softirq based hrtimers to avoid code duplication. No
functional change.

Signed-off-by: Anna-Maria Gleixner <anna-maria@linutronix.de>
---
 kernel/time/hrtimer.c | 20 ++++++++++++++++----
 1 file changed, 16 insertions(+), 4 deletions(-)

diff --git a/kernel/time/hrtimer.c b/kernel/time/hrtimer.c
index c1412e9dcfea..1bb369b1accf 100644
--- a/kernel/time/hrtimer.c
+++ b/kernel/time/hrtimer.c
@@ -459,13 +459,13 @@ __next_base(struct hrtimer_cpu_base *cpu_base, unsigned int *active)
 	while ((base = __next_base((cpu_base), &(active))))
 
 #if defined(CONFIG_NO_HZ_COMMON) || defined(CONFIG_HIGH_RES_TIMERS)
-static ktime_t __hrtimer_get_next_event(struct hrtimer_cpu_base *cpu_base)
+static ktime_t __hrtimer_next_event_base(struct hrtimer_cpu_base *cpu_base,
+					 unsigned int active,
+					 ktime_t expires_next)
 {
 	struct hrtimer_clock_base *base;
-	unsigned int active = cpu_base->active_bases;
-	ktime_t expires, expires_next = KTIME_MAX;
+	ktime_t expires;
 
-	cpu_base->next_timer = NULL;
 	for_each_active_base(base, cpu_base, active) {
 		struct timerqueue_node *next;
 		struct hrtimer *timer;
@@ -487,6 +487,18 @@ static ktime_t __hrtimer_get_next_event(struct hrtimer_cpu_base *cpu_base)
 		expires_next = 0;
 	return expires_next;
 }
+
+static ktime_t __hrtimer_get_next_event(struct hrtimer_cpu_base *cpu_base)
+{
+	unsigned int active = cpu_base->active_bases;
+	ktime_t expires_next = KTIME_MAX;
+
+	cpu_base->next_timer = NULL;
+
+	expires_next = __hrtimer_next_event_base(cpu_base, active, expires_next);
+
+	return expires_next;
+}
 #endif
 
 static inline ktime_t hrtimer_update_base(struct hrtimer_cpu_base *base)
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 45+ messages in thread

* [PATCH v3 25/36] hrtimer: Use irqsave/irqrestore around __run_hrtimer()
  2017-11-29 15:30 [PATCH 00/36] hrtimer: Provide softirq context hrtimers Anna-Maria Gleixner
                   ` (23 preceding siblings ...)
  2017-11-29 15:30 ` [PATCH v3 24/36] hrtimer: Split __hrtimer_get_next_event() Anna-Maria Gleixner
@ 2017-11-29 15:30 ` Anna-Maria Gleixner
  2017-11-29 15:30 ` [PATCH v3 26/36] hrtimer: Add clock bases and hrtimer mode for soft irq context Anna-Maria Gleixner
                   ` (10 subsequent siblings)
  35 siblings, 0 replies; 45+ messages in thread
From: Anna-Maria Gleixner @ 2017-11-29 15:30 UTC (permalink / raw)
  To: LKML
  Cc: Thomas Gleixner, Peter Zijlstra, Ingo Molnar, keescook,
	Christoph Hellwig, John Stultz, Anna-Maria Gleixner

__run_hrtimer() is called with the hrtimer_cpu_base.lock held and
interrupts disabled. Before invoking the timer callback the base lock is
dropped, but interrupts stay disabled.

The upcoming support for softirq based hrtimers requires that interrupts
are enabled before the timer callback is invoked.

To avoid code duplication, take hrtimer_cpu_base.lock with
raw_spin_lock_irqsave(flags) at the call site and hand in the flags as
argument. So raw_spin_unlock_irqrestore() before the callback invocation
will either keep interrupts disabled in interrupt context or restore to
interrupt enabled state when called from softirq context.

Suggested-by: Peter Zijlstra <peterz@infradead.org>
Signed-off-by: Anna-Maria Gleixner <anna-maria@linutronix.de>
---
 kernel/time/hrtimer.c | 31 ++++++++++++++++++-------------
 1 file changed, 18 insertions(+), 13 deletions(-)

diff --git a/kernel/time/hrtimer.c b/kernel/time/hrtimer.c
index 1bb369b1accf..afb54c47e81c 100644
--- a/kernel/time/hrtimer.c
+++ b/kernel/time/hrtimer.c
@@ -1161,7 +1161,8 @@ EXPORT_SYMBOL_GPL(hrtimer_active);
 
 static void __run_hrtimer(struct hrtimer_cpu_base *cpu_base,
 			  struct hrtimer_clock_base *base,
-			  struct hrtimer *timer, ktime_t *now)
+			  struct hrtimer *timer, ktime_t *now,
+			  unsigned long flags)
 {
 	enum hrtimer_restart (*fn)(struct hrtimer *);
 	int restart;
@@ -1196,11 +1197,11 @@ static void __run_hrtimer(struct hrtimer_cpu_base *cpu_base,
 	 * protected against migration to a different CPU even if the lock
 	 * is dropped.
 	 */
-	raw_spin_unlock(&cpu_base->lock);
+	raw_spin_unlock_irqrestore(&cpu_base->lock, flags);
 	trace_hrtimer_expire_entry(timer, now);
 	restart = fn(timer);
 	trace_hrtimer_expire_exit(timer);
-	raw_spin_lock(&cpu_base->lock);
+	raw_spin_lock_irq(&cpu_base->lock);
 
 	/*
 	 * Note: We clear the running state after enqueue_hrtimer and
@@ -1228,7 +1229,8 @@ static void __run_hrtimer(struct hrtimer_cpu_base *cpu_base,
 	base->running = NULL;
 }
 
-static void __hrtimer_run_queues(struct hrtimer_cpu_base *cpu_base, ktime_t now)
+static void __hrtimer_run_queues(struct hrtimer_cpu_base *cpu_base, ktime_t now,
+				 unsigned long flags)
 {
 	struct hrtimer_clock_base *base;
 	unsigned int active = cpu_base->active_bases;
@@ -1259,7 +1261,7 @@ static void __hrtimer_run_queues(struct hrtimer_cpu_base *cpu_base, ktime_t now)
 			if (basenow < hrtimer_get_softexpires_tv64(timer))
 				break;
 
-			__run_hrtimer(cpu_base, base, timer, &basenow);
+			__run_hrtimer(cpu_base, base, timer, &basenow, flags);
 		}
 	}
 }
@@ -1274,13 +1276,14 @@ void hrtimer_interrupt(struct clock_event_device *dev)
 {
 	struct hrtimer_cpu_base *cpu_base = this_cpu_ptr(&hrtimer_bases);
 	ktime_t expires_next, now, entry_time, delta;
+	unsigned long flags;
 	int retries = 0;
 
 	BUG_ON(!cpu_base->hres_active);
 	cpu_base->nr_events++;
 	dev->next_event = KTIME_MAX;
 
-	raw_spin_lock(&cpu_base->lock);
+	raw_spin_lock_irqsave(&cpu_base->lock, flags);
 	entry_time = now = hrtimer_update_base(cpu_base);
 retry:
 	cpu_base->in_hrtirq = 1;
@@ -1293,7 +1296,7 @@ void hrtimer_interrupt(struct clock_event_device *dev)
 	 */
 	cpu_base->expires_next = KTIME_MAX;
 
-	__hrtimer_run_queues(cpu_base, now);
+	__hrtimer_run_queues(cpu_base, now, flags);
 
 	/* Reevaluate the clock bases for the next expiry */
 	expires_next = __hrtimer_get_next_event(cpu_base);
@@ -1303,7 +1306,7 @@ void hrtimer_interrupt(struct clock_event_device *dev)
 	 */
 	cpu_base->expires_next = expires_next;
 	cpu_base->in_hrtirq = 0;
-	raw_spin_unlock(&cpu_base->lock);
+	raw_spin_unlock_irqrestore(&cpu_base->lock, flags);
 
 	/* Reprogramming necessary ? */
 	if (!tick_program_event(expires_next, 0)) {
@@ -1324,7 +1327,7 @@ void hrtimer_interrupt(struct clock_event_device *dev)
 	 * Acquire base lock for updating the offsets and retrieving
 	 * the current time.
 	 */
-	raw_spin_lock(&cpu_base->lock);
+	raw_spin_lock_irqsave(&cpu_base->lock, flags);
 	now = hrtimer_update_base(cpu_base);
 	cpu_base->nr_retries++;
 	if (++retries < 3)
@@ -1337,7 +1340,8 @@ void hrtimer_interrupt(struct clock_event_device *dev)
 	 */
 	cpu_base->nr_hangs++;
 	cpu_base->hang_detected = 1;
-	raw_spin_unlock(&cpu_base->lock);
+	raw_spin_unlock_irqrestore(&cpu_base->lock, flags);
+
 	delta = ktime_sub(now, entry_time);
 	if ((unsigned int)delta > cpu_base->max_hang_time)
 		cpu_base->max_hang_time = (unsigned int) delta;
@@ -1379,6 +1383,7 @@ static inline void __hrtimer_peek_ahead_timers(void) { }
 void hrtimer_run_queues(void)
 {
 	struct hrtimer_cpu_base *cpu_base = this_cpu_ptr(&hrtimer_bases);
+	unsigned long flags;
 	ktime_t now;
 
 	if (__hrtimer_hres_active(cpu_base))
@@ -1396,10 +1401,10 @@ void hrtimer_run_queues(void)
 		return;
 	}
 
-	raw_spin_lock(&cpu_base->lock);
+	raw_spin_lock_irqsave(&cpu_base->lock, flags);
 	now = hrtimer_update_base(cpu_base);
-	__hrtimer_run_queues(cpu_base, now);
-	raw_spin_unlock(&cpu_base->lock);
+	__hrtimer_run_queues(cpu_base, now, flags);
+	raw_spin_unlock_irqrestore(&cpu_base->lock, flags);
 }
 
 /*
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 45+ messages in thread

* [PATCH v3 26/36] hrtimer: Add clock bases and hrtimer mode for soft irq context
  2017-11-29 15:30 [PATCH 00/36] hrtimer: Provide softirq context hrtimers Anna-Maria Gleixner
                   ` (24 preceding siblings ...)
  2017-11-29 15:30 ` [PATCH v3 25/36] hrtimer: Use irqsave/irqrestore around __run_hrtimer() Anna-Maria Gleixner
@ 2017-11-29 15:30 ` Anna-Maria Gleixner
  2017-11-29 15:30 ` [PATCH v3 27/36] hrtimer: Prepare handling of hard and softirq based hrtimers Anna-Maria Gleixner
                   ` (9 subsequent siblings)
  35 siblings, 0 replies; 45+ messages in thread
From: Anna-Maria Gleixner @ 2017-11-29 15:30 UTC (permalink / raw)
  To: LKML
  Cc: Thomas Gleixner, Peter Zijlstra, Ingo Molnar, keescook,
	Christoph Hellwig, John Stultz, Anna-Maria Gleixner

hrtimer callback functions are always executed in hard interrupt
context. Users of hrtimer which need their timer function to be executed
in soft interrupt context, make use of tasklets to get the proper context.

Add additional hrtimer clock bases for timers which must expire in softirq
context, so the detour via the tasklet can be avoided. This is also
required for RT, where the majority of hrtimer is moved into softirq
hrtimer context.

The selection of the expiry mode happens via a mode bit. Introduce
HRTIMER_MODE_SOFT and the matching combinations with the ABS/REL/PINNED
bits and update the decoding of hrtimer_mode in tracepoints.

Signed-off-by: Anna-Maria Gleixner <anna-maria@linutronix.de>
---
 include/linux/hrtimer.h      | 14 ++++++++++++++
 include/trace/events/timer.h |  6 +++++-
 kernel/time/hrtimer.c        | 20 ++++++++++++++++++++
 3 files changed, 39 insertions(+), 1 deletion(-)

diff --git a/include/linux/hrtimer.h b/include/linux/hrtimer.h
index 98ed35767ac5..26ae8a868ea8 100644
--- a/include/linux/hrtimer.h
+++ b/include/linux/hrtimer.h
@@ -33,14 +33,24 @@ struct hrtimer_cpu_base;
  * HRTIMER_MODE_REL		- Time value is relative to now
  * HRTIMER_MODE_PINNED		- Timer is bound to CPU (is only considered
  *				  when starting the timer)
+ * HRTIMER_MODE_SOFT		- Timer callback function will be executed in
+ *				  soft irq context
  */
 enum hrtimer_mode {
 	HRTIMER_MODE_ABS	= 0x00,
 	HRTIMER_MODE_REL	= 0x01,
 	HRTIMER_MODE_PINNED	= 0x02,
+	HRTIMER_MODE_SOFT	= 0x04,
 
 	HRTIMER_MODE_ABS_PINNED = HRTIMER_MODE_ABS | HRTIMER_MODE_PINNED,
 	HRTIMER_MODE_REL_PINNED = HRTIMER_MODE_REL | HRTIMER_MODE_PINNED,
+
+	HRTIMER_MODE_ABS_SOFT	= HRTIMER_MODE_ABS | HRTIMER_MODE_SOFT,
+	HRTIMER_MODE_REL_SOFT	= HRTIMER_MODE_REL | HRTIMER_MODE_SOFT,
+
+	HRTIMER_MODE_ABS_PINNED_SOFT = HRTIMER_MODE_ABS_PINNED | HRTIMER_MODE_SOFT,
+	HRTIMER_MODE_REL_PINNED_SOFT = HRTIMER_MODE_REL_PINNED | HRTIMER_MODE_SOFT,
+
 };
 
 /*
@@ -151,6 +161,10 @@ enum  hrtimer_base_type {
 	HRTIMER_BASE_REALTIME,
 	HRTIMER_BASE_BOOTTIME,
 	HRTIMER_BASE_TAI,
+	HRTIMER_BASE_MONOTONIC_SOFT,
+	HRTIMER_BASE_REALTIME_SOFT,
+	HRTIMER_BASE_BOOTTIME_SOFT,
+	HRTIMER_BASE_TAI_SOFT,
 	HRTIMER_MAX_CLOCK_BASES,
 };
 
diff --git a/include/trace/events/timer.h b/include/trace/events/timer.h
index 744b4310b24b..a57e4ee989d6 100644
--- a/include/trace/events/timer.h
+++ b/include/trace/events/timer.h
@@ -148,7 +148,11 @@ DEFINE_EVENT(timer_class, timer_cancel,
 		{ HRTIMER_MODE_ABS,		"ABS"		},	\
 		{ HRTIMER_MODE_REL,		"REL"		},	\
 		{ HRTIMER_MODE_ABS_PINNED,	"ABS|PINNED"	},	\
-		{ HRTIMER_MODE_REL_PINNED,	"REL|PINNED"	})
+		{ HRTIMER_MODE_REL_PINNED,	"REL|PINNED"	},	\
+		{ HRTIMER_MODE_ABS_SOFT,	"ABS|SOFT"	},	\
+		{ HRTIMER_MODE_REL_SOFT,	"REL|SOFT"	},	\
+		{ HRTIMER_MODE_ABS_PINNED_SOFT,	"ABS|PINNED|SOFT" },	\
+		{ HRTIMER_MODE_REL_PINNED_SOFT,	"REL|PINNED|SOFT" })
 
 /**
  * hrtimer_init - called when the hrtimer is initialized
diff --git a/kernel/time/hrtimer.c b/kernel/time/hrtimer.c
index afb54c47e81c..e2724f8a2340 100644
--- a/kernel/time/hrtimer.c
+++ b/kernel/time/hrtimer.c
@@ -92,6 +92,26 @@ DEFINE_PER_CPU(struct hrtimer_cpu_base, hrtimer_bases) =
 			.clockid = CLOCK_TAI,
 			.get_time = &ktime_get_clocktai,
 		},
+		{
+			.index = HRTIMER_BASE_MONOTONIC_SOFT,
+			.clockid = CLOCK_MONOTONIC,
+			.get_time = &ktime_get,
+		},
+		{
+			.index = HRTIMER_BASE_REALTIME_SOFT,
+			.clockid = CLOCK_REALTIME,
+			.get_time = &ktime_get_real,
+		},
+		{
+			.index = HRTIMER_BASE_BOOTTIME_SOFT,
+			.clockid = CLOCK_BOOTTIME,
+			.get_time = &ktime_get_boottime,
+		},
+		{
+			.index = HRTIMER_BASE_TAI_SOFT,
+			.clockid = CLOCK_TAI,
+			.get_time = &ktime_get_clocktai,
+		},
 	}
 };
 
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 45+ messages in thread

* [PATCH v3 27/36] hrtimer: Prepare handling of hard and softirq based hrtimers
  2017-11-29 15:30 [PATCH 00/36] hrtimer: Provide softirq context hrtimers Anna-Maria Gleixner
                   ` (25 preceding siblings ...)
  2017-11-29 15:30 ` [PATCH v3 26/36] hrtimer: Add clock bases and hrtimer mode for soft irq context Anna-Maria Gleixner
@ 2017-11-29 15:30 ` Anna-Maria Gleixner
  2017-11-29 15:30 ` [PATCH v3 28/36] hrtimer: Implement support for " Anna-Maria Gleixner
                   ` (8 subsequent siblings)
  35 siblings, 0 replies; 45+ messages in thread
From: Anna-Maria Gleixner @ 2017-11-29 15:30 UTC (permalink / raw)
  To: LKML
  Cc: Thomas Gleixner, Peter Zijlstra, Ingo Molnar, keescook,
	Christoph Hellwig, John Stultz, Anna-Maria Gleixner

The softirq based hrtimer can utilize most of the existing hrtimers
functions, but need to operate on a different data set.

Add an active_mask argument to various functions so the hard and soft bases
can be selected. Fixup the existing callers and hand in the ACTIVE_HARD
mask.

Signed-off-by: Anna-Maria Gleixner <anna-maria@linutronix.de>
---
 kernel/time/hrtimer.c | 38 +++++++++++++++++++++++++++++---------
 1 file changed, 29 insertions(+), 9 deletions(-)

diff --git a/kernel/time/hrtimer.c b/kernel/time/hrtimer.c
index e2724f8a2340..65211fd7288d 100644
--- a/kernel/time/hrtimer.c
+++ b/kernel/time/hrtimer.c
@@ -60,6 +60,15 @@
 #include "tick-internal.h"
 
 /*
+ * Masks for selecting the soft and hard context timers from
+ * cpu_base->active
+ */
+#define MASK_SHIFT		(HRTIMER_BASE_MONOTONIC_SOFT)
+#define HRTIMER_ACTIVE_HARD	((1U << MASK_SHIFT) - 1)
+#define HRTIMER_ACTIVE_SOFT	(HRTIMER_ACTIVE_HARD << MASK_SHIFT)
+#define HRTIMER_ACTIVE_ALL	(HRTIMER_ACTIVE_SOFT | HRTIMER_ACTIVE_HARD)
+
+/*
  * The timer bases:
  *
  * There are more clockids than hrtimer bases. Thus, we index
@@ -508,13 +517,24 @@ static ktime_t __hrtimer_next_event_base(struct hrtimer_cpu_base *cpu_base,
 	return expires_next;
 }
 
-static ktime_t __hrtimer_get_next_event(struct hrtimer_cpu_base *cpu_base)
+/*
+ * Recomputes cpu_base::*next_timer and returns the earliest expires_next but
+ * does not set cpu_base::*expires_next, that is done by hrtimer_reprogram.
+ *
+ * @active_mask must be one of:
+ *  - HRTIMER_ACTIVE,
+ *  - HRTIMER_ACTIVE_SOFT, or
+ *  - HRTIMER_ACTIVE_HARD.
+ */
+static ktime_t __hrtimer_get_next_event(struct hrtimer_cpu_base *cpu_base,
+					unsigned int active_mask)
 {
-	unsigned int active = cpu_base->active_bases;
+	unsigned int active;
 	ktime_t expires_next = KTIME_MAX;
 
 	cpu_base->next_timer = NULL;
 
+	active = cpu_base->active_bases & active_mask;
 	expires_next = __hrtimer_next_event_base(cpu_base, active, expires_next);
 
 	return expires_next;
@@ -555,7 +575,7 @@ hrtimer_force_reprogram(struct hrtimer_cpu_base *cpu_base, int skip_equal)
 {
 	ktime_t expires_next;
 
-	expires_next = __hrtimer_get_next_event(cpu_base);
+	expires_next = __hrtimer_get_next_event(cpu_base, HRTIMER_ACTIVE_HARD);
 
 	if (skip_equal && expires_next == cpu_base->expires_next)
 		return;
@@ -1076,7 +1096,7 @@ u64 hrtimer_get_next_event(void)
 	raw_spin_lock_irqsave(&cpu_base->lock, flags);
 
 	if (!__hrtimer_hres_active(cpu_base))
-		expires = __hrtimer_get_next_event(cpu_base);
+		expires = __hrtimer_get_next_event(cpu_base, HRTIMER_ACTIVE_HARD);
 
 	raw_spin_unlock_irqrestore(&cpu_base->lock, flags);
 
@@ -1250,10 +1270,10 @@ static void __run_hrtimer(struct hrtimer_cpu_base *cpu_base,
 }
 
 static void __hrtimer_run_queues(struct hrtimer_cpu_base *cpu_base, ktime_t now,
-				 unsigned long flags)
+				 unsigned long flags, unsigned int active_mask)
 {
 	struct hrtimer_clock_base *base;
-	unsigned int active = cpu_base->active_bases;
+	unsigned int active = cpu_base->active_bases & active_mask;
 
 	for_each_active_base(base, cpu_base, active) {
 		struct timerqueue_node *node;
@@ -1316,10 +1336,10 @@ void hrtimer_interrupt(struct clock_event_device *dev)
 	 */
 	cpu_base->expires_next = KTIME_MAX;
 
-	__hrtimer_run_queues(cpu_base, now, flags);
+	__hrtimer_run_queues(cpu_base, now, flags, HRTIMER_ACTIVE_HARD);
 
 	/* Reevaluate the clock bases for the next expiry */
-	expires_next = __hrtimer_get_next_event(cpu_base);
+	expires_next = __hrtimer_get_next_event(cpu_base, HRTIMER_ACTIVE_HARD);
 	/*
 	 * Store the new expiry value so the migration code can verify
 	 * against it.
@@ -1423,7 +1443,7 @@ void hrtimer_run_queues(void)
 
 	raw_spin_lock_irqsave(&cpu_base->lock, flags);
 	now = hrtimer_update_base(cpu_base);
-	__hrtimer_run_queues(cpu_base, now, flags);
+	__hrtimer_run_queues(cpu_base, now, flags, HRTIMER_ACTIVE_HARD);
 	raw_spin_unlock_irqrestore(&cpu_base->lock, flags);
 }
 
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 45+ messages in thread

* [PATCH v3 28/36] hrtimer: Implement support for softirq based hrtimers
  2017-11-29 15:30 [PATCH 00/36] hrtimer: Provide softirq context hrtimers Anna-Maria Gleixner
                   ` (26 preceding siblings ...)
  2017-11-29 15:30 ` [PATCH v3 27/36] hrtimer: Prepare handling of hard and softirq based hrtimers Anna-Maria Gleixner
@ 2017-11-29 15:30 ` Anna-Maria Gleixner
  2017-12-14 10:47   ` Sebastian Andrzej Siewior
  2017-11-29 15:30 ` [PATCH v3 29/36] hrtimer: Implement SOFT/HARD clock base selection Anna-Maria Gleixner
                   ` (7 subsequent siblings)
  35 siblings, 1 reply; 45+ messages in thread
From: Anna-Maria Gleixner @ 2017-11-29 15:30 UTC (permalink / raw)
  To: LKML
  Cc: Thomas Gleixner, Peter Zijlstra, Ingo Molnar, keescook,
	Christoph Hellwig, John Stultz, Anna-Maria Gleixner

hrtimer callbacks are always invoked in hard interrupt context. Several
users in tree require soft interrupt context for their callbacks and
achieve this by combining a hrtimer with a tasklet. The hrtimer schedules
the tasklet in hard interrupt context and the tasklet callback gets invoked
in softirq context later.

That's suboptimal and aside of that the real-time patch moves most of the
hrtimers into softirq context. So adding native support for hrtimers
expiring in softirq context is a valuable extension for both mainline and
the RT patch set.

Each valid hrtimer clock id has two associated hrtimer clock bases: one for
timers expiring in hardirq context and one for timers expiring in softirq
context.

Implement the functionality to associate a hrtimer with the hard or softirq
related clock bases and update the relevant functions to take them into
account when the next expiry time needs to be evaluated.

Add a check into the hard interrupt context handler functions to check
whether the first expiring softirq based timer has expired. If it's expired
the softirq is raised and the accounting of softirq based timers to
evaluate the next expiry time for programming the timer hardware is skipped
until the softirq processing has finished. At the end of the softirq
processing the regular processing is resumed.

Suggested-by: Thomas Gleixner <tglx@linutronix.de>
Suggested-by: Peter Zijlstra <peterz@infradead.org>
Signed-off-by: Anna-Maria Gleixner <anna-maria@linutronix.de>
---
 include/linux/hrtimer.h |  21 +++--
 kernel/time/hrtimer.c   | 201 ++++++++++++++++++++++++++++++++++++++++--------
 2 files changed, 186 insertions(+), 36 deletions(-)

diff --git a/include/linux/hrtimer.h b/include/linux/hrtimer.h
index 26ae8a868ea8..c7902ca7c9f4 100644
--- a/include/linux/hrtimer.h
+++ b/include/linux/hrtimer.h
@@ -103,6 +103,7 @@ enum hrtimer_restart {
  * @base:	pointer to the timer base (per cpu and per clock)
  * @state:	state information (See bit values above)
  * @is_rel:	Set if the timer was armed relative
+ * @is_soft:	Set if hrtimer will be expired in soft interrupt context.
  *
  * The hrtimer structure must be initialized by hrtimer_init()
  */
@@ -113,6 +114,7 @@ struct hrtimer {
 	struct hrtimer_clock_base	*base;
 	u8				state;
 	u8				is_rel;
+	u8				is_soft;
 };
 
 /**
@@ -178,13 +180,18 @@ enum  hrtimer_base_type {
  * @hres_active:	State of high resolution mode
  * @in_hrtirq:		hrtimer_interrupt() is currently executing
  * @hang_detected:	The last hrtimer interrupt detected a hang
+ * @softirq_activated:	displays, if the softirq is raised - update of softirq
+ *			related settings is not required then.
  * @nr_events:		Total number of hrtimer interrupt events
  * @nr_retries:		Total number of hrtimer interrupt retries
  * @nr_hangs:		Total number of hrtimer interrupt hangs
  * @max_hang_time:	Maximum time spent in hrtimer_interrupt
  * @expires_next:	absolute time of the next event, is required for remote
- *			hrtimer enqueue
+ *			hrtimer enqueue; it is the total first expiry time (hard
+ *			and soft hrtimer are taken into account)
  * @next_timer:		Pointer to the first expiring timer
+ * @softirq_expires_next: Time to check, if soft queues needs also to be expired
+ * @softirq_next_timer: Pointer to the first expiring softirq based timer
  * @clock_base:		array of clock bases for this cpu
  *
  * Note: next_timer is just an optimization for __remove_hrtimer().
@@ -196,9 +203,10 @@ struct hrtimer_cpu_base {
 	unsigned int			cpu;
 	unsigned int			active_bases;
 	unsigned int			clock_was_set_seq;
-	unsigned int			hres_active	: 1,
-					in_hrtirq	: 1,
-					hang_detected	: 1;
+	unsigned int			hres_active		: 1,
+					in_hrtirq		: 1,
+					hang_detected		: 1,
+					softirq_activated       : 1;
 #ifdef CONFIG_HIGH_RES_TIMERS
 	unsigned int			nr_events;
 	unsigned short			nr_retries;
@@ -207,6 +215,8 @@ struct hrtimer_cpu_base {
 #endif
 	ktime_t				expires_next;
 	struct hrtimer			*next_timer;
+	ktime_t				softirq_expires_next;
+	struct hrtimer			*softirq_next_timer;
 	struct hrtimer_clock_base	clock_base[HRTIMER_MAX_CLOCK_BASES];
 } ____cacheline_aligned;
 
@@ -379,7 +389,8 @@ extern void hrtimer_start_range_ns(struct hrtimer *timer, ktime_t tim,
  * @timer:	the timer to be added
  * @tim:	expiry time
  * @mode:	timer mode: absolute (HRTIMER_MODE_ABS) or
- *		relative (HRTIMER_MODE_REL), and pinned (HRTIMER_MODE_PINNED)
+ *		relative (HRTIMER_MODE_REL), and pinned (HRTIMER_MODE_PINNED);
+ *		softirq based mode is considered for debug purpose only!
  */
 static inline void hrtimer_start(struct hrtimer *timer, ktime_t tim,
 				 const enum hrtimer_mode mode)
diff --git a/kernel/time/hrtimer.c b/kernel/time/hrtimer.c
index 65211fd7288d..40b14f025829 100644
--- a/kernel/time/hrtimer.c
+++ b/kernel/time/hrtimer.c
@@ -68,6 +68,9 @@
 #define HRTIMER_ACTIVE_SOFT	(HRTIMER_ACTIVE_HARD << MASK_SHIFT)
 #define HRTIMER_ACTIVE_ALL	(HRTIMER_ACTIVE_SOFT | HRTIMER_ACTIVE_HARD)
 
+/* Define for debug mode check */
+#define HRTIMER_MODECHECK	true
+
 /*
  * The timer bases:
  *
@@ -411,8 +414,17 @@ static inline void debug_hrtimer_init(struct hrtimer *timer)
 	debug_object_init(timer, &hrtimer_debug_descr);
 }
 
-static inline void debug_hrtimer_activate(struct hrtimer *timer)
+static inline void debug_hrtimer_activate(struct hrtimer *timer,
+					  enum hrtimer_mode mode,
+					  bool modecheck)
 {
+	/*
+	 * Check whether the HRTIMER_MODE_SOFT bit and hrtimer.is_soft
+	 * match, when a timer is started via__hrtimer_start_range_ns().
+	 */
+	if (modecheck)
+		WARN_ON_ONCE(!(mode & HRTIMER_MODE_SOFT) ^ !timer->is_soft);
+
 	debug_object_activate(timer, &hrtimer_debug_descr);
 }
 
@@ -444,8 +456,11 @@ void destroy_hrtimer_on_stack(struct hrtimer *timer)
 EXPORT_SYMBOL_GPL(destroy_hrtimer_on_stack);
 
 #else
+
 static inline void debug_hrtimer_init(struct hrtimer *timer) { }
-static inline void debug_hrtimer_activate(struct hrtimer *timer) { }
+static inline void debug_hrtimer_activate(struct hrtimer *timer,
+					  enum hrtimer_mode mode,
+					  bool modecheck) { }
 static inline void debug_hrtimer_deactivate(struct hrtimer *timer) { }
 #endif
 
@@ -458,9 +473,10 @@ debug_init(struct hrtimer *timer, clockid_t clockid,
 }
 
 static inline void debug_activate(struct hrtimer *timer,
-				  enum hrtimer_mode mode)
+				  enum hrtimer_mode mode,
+				  bool modecheck)
 {
-	debug_hrtimer_activate(timer);
+	debug_hrtimer_activate(timer, mode, modecheck);
 	trace_hrtimer_start(timer, mode);
 }
 
@@ -487,7 +503,6 @@ __next_base(struct hrtimer_cpu_base *cpu_base, unsigned int *active)
 #define for_each_active_base(base, cpu_base, active)	\
 	while ((base = __next_base((cpu_base), &(active))))
 
-#if defined(CONFIG_NO_HZ_COMMON) || defined(CONFIG_HIGH_RES_TIMERS)
 static ktime_t __hrtimer_next_event_base(struct hrtimer_cpu_base *cpu_base,
 					 unsigned int active,
 					 ktime_t expires_next)
@@ -504,7 +519,10 @@ static ktime_t __hrtimer_next_event_base(struct hrtimer_cpu_base *cpu_base,
 		expires = ktime_sub(hrtimer_get_expires(timer), base->offset);
 		if (expires < expires_next) {
 			expires_next = expires;
-			cpu_base->next_timer = timer;
+			if (timer->is_soft)
+				cpu_base->softirq_next_timer = timer;
+			else
+				cpu_base->next_timer = timer;
 		}
 	}
 	/*
@@ -521,25 +539,42 @@ static ktime_t __hrtimer_next_event_base(struct hrtimer_cpu_base *cpu_base,
  * Recomputes cpu_base::*next_timer and returns the earliest expires_next but
  * does not set cpu_base::*expires_next, that is done by hrtimer_reprogram.
  *
+ * When a softirq is pending, we can ignore the HRTIMER_ACTIVE_SOFT bases,
+ * those timers will get run whenever the softirq gets handled, at the end of
+ * hrtimer_run_softirq(), hrtimer_update_softirq_timer() will re-add these bases.
+ *
+ * Therefore softirq values are those from the HRTIMER_ACTIVE_SOFT clock bases.
+ * The !softirq values are the minima across HRTIMER_ACTIVE, unless an actual
+ * softirq is pending, in which case they're the minima of HRTIMER_ACTIVE_HARD.
+ *
  * @active_mask must be one of:
  *  - HRTIMER_ACTIVE,
  *  - HRTIMER_ACTIVE_SOFT, or
  *  - HRTIMER_ACTIVE_HARD.
  */
-static ktime_t __hrtimer_get_next_event(struct hrtimer_cpu_base *cpu_base,
-					unsigned int active_mask)
+static ktime_t
+__hrtimer_get_next_event(struct hrtimer_cpu_base *cpu_base, unsigned int active_mask)
 {
 	unsigned int active;
+	struct hrtimer *next_timer = NULL;
 	ktime_t expires_next = KTIME_MAX;
 
-	cpu_base->next_timer = NULL;
+	if (!cpu_base->softirq_activated && (active_mask & HRTIMER_ACTIVE_SOFT)) {
+		active = cpu_base->active_bases & HRTIMER_ACTIVE_SOFT;
+		cpu_base->softirq_next_timer = NULL;
+		expires_next = __hrtimer_next_event_base(cpu_base, active, KTIME_MAX);
 
-	active = cpu_base->active_bases & active_mask;
-	expires_next = __hrtimer_next_event_base(cpu_base, active, expires_next);
+		next_timer = cpu_base->softirq_next_timer;
+	}
+
+	if (active_mask & HRTIMER_ACTIVE_HARD) {
+		active = cpu_base->active_bases & HRTIMER_ACTIVE_HARD;
+		cpu_base->next_timer = next_timer;
+		expires_next = __hrtimer_next_event_base(cpu_base, active, expires_next);
+	}
 
 	return expires_next;
 }
-#endif
 
 static inline ktime_t hrtimer_update_base(struct hrtimer_cpu_base *base)
 {
@@ -547,8 +582,14 @@ static inline ktime_t hrtimer_update_base(struct hrtimer_cpu_base *base)
 	ktime_t *offs_boot = &base->clock_base[HRTIMER_BASE_BOOTTIME].offset;
 	ktime_t *offs_tai = &base->clock_base[HRTIMER_BASE_TAI].offset;
 
-	return ktime_get_update_offsets_now(&base->clock_was_set_seq,
+	ktime_t now = ktime_get_update_offsets_now(&base->clock_was_set_seq,
 					    offs_real, offs_boot, offs_tai);
+
+	base->clock_base[HRTIMER_BASE_REALTIME_SOFT].offset = *offs_real;
+	base->clock_base[HRTIMER_BASE_BOOTTIME_SOFT].offset = *offs_boot;
+	base->clock_base[HRTIMER_BASE_TAI_SOFT].offset = *offs_tai;
+
+	return now;
 }
 
 /*
@@ -575,7 +616,23 @@ hrtimer_force_reprogram(struct hrtimer_cpu_base *cpu_base, int skip_equal)
 {
 	ktime_t expires_next;
 
-	expires_next = __hrtimer_get_next_event(cpu_base, HRTIMER_ACTIVE_HARD);
+	/*
+	 * Find the current next expiration time.
+	 */
+	expires_next = __hrtimer_get_next_event(cpu_base, HRTIMER_ACTIVE_ALL);
+
+	if (cpu_base->next_timer && cpu_base->next_timer->is_soft) {
+		/*
+		 * When the softirq is activated, hrtimer has to be
+		 * programmed with the first hard hrtimer because soft
+		 * timer interrupt could occur too late.
+		 */
+		if (cpu_base->softirq_activated)
+			expires_next = __hrtimer_get_next_event(cpu_base,
+								HRTIMER_ACTIVE_HARD);
+		else
+			cpu_base->softirq_expires_next = expires_next;
+	}
 
 	if (skip_equal && expires_next == cpu_base->expires_next)
 		return;
@@ -702,7 +759,7 @@ static inline void retrigger_next_event(void *arg) { }
  *
  * Called with interrupts disabled and base->cpu_base.lock held
  */
-static void hrtimer_reprogram(struct hrtimer *timer)
+static void hrtimer_reprogram(struct hrtimer *timer, bool reprogram)
 {
 	struct hrtimer_cpu_base *cpu_base = this_cpu_ptr(&hrtimer_bases);
 	struct hrtimer_clock_base *base = timer->base;
@@ -711,6 +768,28 @@ static void hrtimer_reprogram(struct hrtimer *timer)
 	WARN_ON_ONCE(hrtimer_get_expires_tv64(timer) < 0);
 
 	/*
+	 * CLOCK_REALTIME timer might be requested with an absolute
+	 * expiry time which is less than base->offset. Set it to 0.
+	 */
+	if (expires < 0)
+		expires = 0;
+
+	if (timer->is_soft) {
+		if (cpu_base->softirq_activated)
+			return;
+
+		if (!ktime_before(expires, cpu_base->softirq_expires_next))
+			return;
+
+		cpu_base->softirq_next_timer = timer;
+		cpu_base->softirq_expires_next = expires;
+
+		if (!ktime_before(expires, cpu_base->expires_next) ||
+		    !reprogram)
+			return;
+	}
+
+	/*
 	 * If the timer is not on the current cpu, we cannot reprogram
 	 * the other cpus clock event device.
 	 */
@@ -727,13 +806,6 @@ static void hrtimer_reprogram(struct hrtimer *timer)
 	if (cpu_base->in_hrtirq)
 		return;
 
-	/*
-	 * CLOCK_REALTIME timer might be requested with an absolute
-	 * expiry time which is less than base->offset. Set it to 0.
-	 */
-	if (expires < 0)
-		expires = 0;
-
 	if (expires >= cpu_base->expires_next)
 		return;
 
@@ -866,9 +938,10 @@ EXPORT_SYMBOL_GPL(hrtimer_forward);
  */
 static int enqueue_hrtimer(struct hrtimer *timer,
 			   struct hrtimer_clock_base *base,
-			   enum hrtimer_mode mode)
+			   enum hrtimer_mode mode,
+			   bool modecheck)
 {
-	debug_activate(timer, mode);
+	debug_activate(timer, mode, modecheck);
 
 	base->cpu_base->active_bases |= 1 << base->index;
 
@@ -959,6 +1032,31 @@ static inline ktime_t hrtimer_update_lowres(struct hrtimer *timer, ktime_t tim,
 	return tim;
 }
 
+static void
+hrtimer_update_softirq_timer(struct hrtimer_cpu_base *cpu_base, bool reprogram)
+{
+	ktime_t expires;
+
+	/*
+	 * Find the next SOFT expiration.
+	 */
+	expires = __hrtimer_get_next_event(cpu_base, HRTIMER_ACTIVE_SOFT);
+
+	/*
+	 * reprogramming needs to be triggered, even if the next soft
+	 * hrtimer expires at the same time than the next hard
+	 * hrtimer. cpu_base->softirq_expires_next needs to be updated!
+	 */
+	if (!reprogram || expires == KTIME_MAX)
+		return;
+
+	/*
+	 * cpu_base->*next_timer is recomputed by __hrtimer_get_next_event()
+	 * cpu_base->*expires_next is only set by hrtimer_reprogram()
+	 */
+	hrtimer_reprogram(cpu_base->softirq_next_timer, reprogram);
+}
+
 static int __hrtimer_start_range_ns(struct hrtimer *timer, ktime_t tim,
 				    u64 delta_ns, const enum hrtimer_mode mode,
 				    struct hrtimer_clock_base *base)
@@ -978,7 +1076,7 @@ static int __hrtimer_start_range_ns(struct hrtimer *timer, ktime_t tim,
 	/* Switch the timer base, if necessary: */
 	new_base = switch_hrtimer_base(timer, base, mode & HRTIMER_MODE_PINNED);
 
-	return enqueue_hrtimer(timer, new_base, mode);
+	return enqueue_hrtimer(timer, new_base, mode, HRTIMER_MODECHECK);
 }
 /**
  * hrtimer_start_range_ns - (re)start an hrtimer
@@ -986,7 +1084,8 @@ static int __hrtimer_start_range_ns(struct hrtimer *timer, ktime_t tim,
  * @tim:	expiry time
  * @delta_ns:	"slack" range for the timer
  * @mode:	timer mode: absolute (HRTIMER_MODE_ABS) or
- *		relative (HRTIMER_MODE_REL), and pinned (HRTIMER_MODE_PINNED)
+ *		relative (HRTIMER_MODE_REL), and pinned (HRTIMER_MODE_PINNED);
+ *		softirq based mode is considered for debug purpose only!
  */
 void hrtimer_start_range_ns(struct hrtimer *timer, ktime_t tim,
 			    u64 delta_ns, const enum hrtimer_mode mode)
@@ -997,7 +1096,7 @@ void hrtimer_start_range_ns(struct hrtimer *timer, ktime_t tim,
 	base = lock_hrtimer_base(timer, &flags);
 
 	if (__hrtimer_start_range_ns(timer, tim, delta_ns, mode, base))
-		hrtimer_reprogram(timer);
+		hrtimer_reprogram(timer, true);
 
 	unlock_hrtimer_base(timer, &flags);
 }
@@ -1096,7 +1195,7 @@ u64 hrtimer_get_next_event(void)
 	raw_spin_lock_irqsave(&cpu_base->lock, flags);
 
 	if (!__hrtimer_hres_active(cpu_base))
-		expires = __hrtimer_get_next_event(cpu_base, HRTIMER_ACTIVE_HARD);
+		expires = __hrtimer_get_next_event(cpu_base, HRTIMER_ACTIVE_ALL);
 
 	raw_spin_unlock_irqrestore(&cpu_base->lock, flags);
 
@@ -1254,7 +1353,8 @@ static void __run_hrtimer(struct hrtimer_cpu_base *cpu_base,
 	 */
 	if (restart != HRTIMER_NORESTART &&
 	    !(timer->state & HRTIMER_STATE_ENQUEUED))
-		enqueue_hrtimer(timer, base, HRTIMER_MODE_ABS);
+		enqueue_hrtimer(timer, base, HRTIMER_MODE_ABS,
+				!HRTIMER_MODECHECK);
 
 	/*
 	 * Separate the ->running assignment from the ->state assignment.
@@ -1306,6 +1406,23 @@ static void __hrtimer_run_queues(struct hrtimer_cpu_base *cpu_base, ktime_t now,
 	}
 }
 
+static __latent_entropy void hrtimer_run_softirq(struct softirq_action *h)
+{
+	struct hrtimer_cpu_base *cpu_base = this_cpu_ptr(&hrtimer_bases);
+	unsigned long flags;
+	ktime_t now;
+
+	raw_spin_lock_irqsave(&cpu_base->lock, flags);
+
+	now = hrtimer_update_base(cpu_base);
+	__hrtimer_run_queues(cpu_base, now, flags, HRTIMER_ACTIVE_SOFT);
+
+	cpu_base->softirq_activated = 0;
+	hrtimer_update_softirq_timer(cpu_base, true);
+
+	raw_spin_unlock_irqrestore(&cpu_base->lock, flags);
+}
+
 #ifdef CONFIG_HIGH_RES_TIMERS
 
 /*
@@ -1336,10 +1453,16 @@ void hrtimer_interrupt(struct clock_event_device *dev)
 	 */
 	cpu_base->expires_next = KTIME_MAX;
 
+	if (!ktime_before(now, cpu_base->softirq_expires_next)) {
+		cpu_base->softirq_expires_next = KTIME_MAX;
+		cpu_base->softirq_activated = 1;
+		raise_softirq_irqoff(HRTIMER_SOFTIRQ);
+	}
+
 	__hrtimer_run_queues(cpu_base, now, flags, HRTIMER_ACTIVE_HARD);
 
 	/* Reevaluate the clock bases for the next expiry */
-	expires_next = __hrtimer_get_next_event(cpu_base, HRTIMER_ACTIVE_HARD);
+	expires_next = __hrtimer_get_next_event(cpu_base, HRTIMER_ACTIVE_ALL);
 	/*
 	 * Store the new expiry value so the migration code can verify
 	 * against it.
@@ -1443,6 +1566,13 @@ void hrtimer_run_queues(void)
 
 	raw_spin_lock_irqsave(&cpu_base->lock, flags);
 	now = hrtimer_update_base(cpu_base);
+
+	if (!ktime_before(now, cpu_base->softirq_expires_next)) {
+		cpu_base->softirq_expires_next = KTIME_MAX;
+		cpu_base->softirq_activated = 1;
+		raise_softirq_irqoff(HRTIMER_SOFTIRQ);
+	}
+
 	__hrtimer_run_queues(cpu_base, now, flags, HRTIMER_ACTIVE_HARD);
 	raw_spin_unlock_irqrestore(&cpu_base->lock, flags);
 }
@@ -1624,6 +1754,7 @@ int hrtimers_prepare_cpu(unsigned int cpu)
 	cpu_base->cpu = cpu;
 	cpu_base->hres_active = 0;
 	cpu_base->expires_next = KTIME_MAX;
+	cpu_base->softirq_expires_next = KTIME_MAX;
 	return 0;
 }
 
@@ -1655,7 +1786,8 @@ static void migrate_hrtimer_list(struct hrtimer_clock_base *old_base,
 		 * sort out already expired timers and reprogram the
 		 * event device.
 		 */
-		enqueue_hrtimer(timer, new_base, HRTIMER_MODE_ABS);
+		enqueue_hrtimer(timer, new_base, HRTIMER_MODE_ABS,
+				!HRTIMER_MODECHECK);
 	}
 }
 
@@ -1682,6 +1814,12 @@ int hrtimers_dead_cpu(unsigned int scpu)
 				     &new_base->clock_base[i]);
 	}
 
+	/*
+	 * The migration might have changed the first expiring softirq
+	 * timer on this CPU. Update it.
+	 */
+	hrtimer_update_softirq_timer(new_base, false);
+
 	raw_spin_unlock(&old_base->lock);
 	raw_spin_unlock(&new_base->lock);
 
@@ -1696,6 +1834,7 @@ int hrtimers_dead_cpu(unsigned int scpu)
 void __init hrtimers_init(void)
 {
 	hrtimers_prepare_cpu(smp_processor_id());
+	open_softirq(HRTIMER_SOFTIRQ, hrtimer_run_softirq);
 }
 
 /**
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 45+ messages in thread

* [PATCH v3 29/36] hrtimer: Implement SOFT/HARD clock base selection
  2017-11-29 15:30 [PATCH 00/36] hrtimer: Provide softirq context hrtimers Anna-Maria Gleixner
                   ` (27 preceding siblings ...)
  2017-11-29 15:30 ` [PATCH v3 28/36] hrtimer: Implement support for " Anna-Maria Gleixner
@ 2017-11-29 15:30 ` Anna-Maria Gleixner
  2017-11-29 15:30 ` [PATCH v3 30/36] can/bcm: Replace hrtimer_tasklet with softirq based hrtimer Anna-Maria Gleixner
                   ` (6 subsequent siblings)
  35 siblings, 0 replies; 45+ messages in thread
From: Anna-Maria Gleixner @ 2017-11-29 15:30 UTC (permalink / raw)
  To: LKML
  Cc: Thomas Gleixner, Peter Zijlstra, Ingo Molnar, keescook,
	Christoph Hellwig, John Stultz, Anna-Maria Gleixner

All prerequisites to handle hrtimers for expiry in either hard or soft
interrupt context are in place.

Add the missing bit in hrtimer_init() which associates the timer to the
hard or the soft irq clock base.

Signed-off-by: Anna-Maria Gleixner <anna-maria@linutronix.de>
---
 kernel/time/hrtimer.c | 15 +++++++++++----
 1 file changed, 11 insertions(+), 4 deletions(-)

diff --git a/kernel/time/hrtimer.c b/kernel/time/hrtimer.c
index 40b14f025829..68328ceefc7f 100644
--- a/kernel/time/hrtimer.c
+++ b/kernel/time/hrtimer.c
@@ -1218,8 +1218,9 @@ static inline int hrtimer_clockid_to_base(clockid_t clock_id)
 static void __hrtimer_init(struct hrtimer *timer, clockid_t clock_id,
 			   enum hrtimer_mode mode)
 {
+	bool softtimer = !!(mode & HRTIMER_MODE_SOFT);
+	int base = softtimer ? HRTIMER_MAX_CLOCK_BASES / 2 : 0;
 	struct hrtimer_cpu_base *cpu_base;
-	int base;
 
 	memset(timer, 0, sizeof(struct hrtimer));
 
@@ -1233,7 +1234,8 @@ static void __hrtimer_init(struct hrtimer *timer, clockid_t clock_id,
 	if (clock_id == CLOCK_REALTIME && mode & HRTIMER_MODE_REL)
 		clock_id = CLOCK_MONOTONIC;
 
-	base = hrtimer_clockid_to_base(clock_id);
+	base += hrtimer_clockid_to_base(clock_id);
+	timer->is_soft = softtimer;
 	timer->base = &cpu_base->clock_base[base];
 	timerqueue_init(&timer->node);
 }
@@ -1242,8 +1244,13 @@ static void __hrtimer_init(struct hrtimer *timer, clockid_t clock_id,
  * hrtimer_init - initialize a timer to the given clock
  * @timer:	the timer to be initialized
  * @clock_id:	the clock to be used
- * @mode:	timer mode: absolute (HRTIMER_MODE_ABS) or
- *		relative (HRTIMER_MODE_REL); pinned is not considered here!
+ * @mode:       The modes which are relevant for intitialization:
+ *              HRTIMER_MODE_ABS, HRTIMER_MODE_REL, HRTIMER_MODE_ABS_SOFT,
+ *              HRTIMER_MODE_REL_SOFT
+ *
+ *              The PINNED variants of the above can be handed in,
+ *              but the PINNED bit is ignored as pinning happens
+ *              when the hrtimer is started
  */
 void hrtimer_init(struct hrtimer *timer, clockid_t clock_id,
 		  enum hrtimer_mode mode)
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 45+ messages in thread

* [PATCH v3 30/36] can/bcm: Replace hrtimer_tasklet with softirq based hrtimer
  2017-11-29 15:30 [PATCH 00/36] hrtimer: Provide softirq context hrtimers Anna-Maria Gleixner
                   ` (28 preceding siblings ...)
  2017-11-29 15:30 ` [PATCH v3 29/36] hrtimer: Implement SOFT/HARD clock base selection Anna-Maria Gleixner
@ 2017-11-29 15:30 ` Anna-Maria Gleixner
  2017-11-29 15:30 ` [PATCH v3 31/36] mac80211_hwsim: Replace hrtimer tasklet with softirq hrtimer Anna-Maria Gleixner
                   ` (5 subsequent siblings)
  35 siblings, 0 replies; 45+ messages in thread
From: Anna-Maria Gleixner @ 2017-11-29 15:30 UTC (permalink / raw)
  To: LKML
  Cc: Thomas Gleixner, Peter Zijlstra, Ingo Molnar, keescook,
	Christoph Hellwig, John Stultz, Anna-Maria Gleixner,
	Oliver Hartkopp, Marc Kleine-Budde, linux-can

From: Thomas Gleixner <tglx@linutronix.de>

Switch the timer to HRTIMER_MODE_SOFT, which executed the timer
callback in softirq context and remove the hrtimer_tasklet.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Anna-Maria Gleixner <anna-maria@linutronix.de>
Acked-by: Oliver Hartkopp <socketcan@hartkopp.net>
Cc: Oliver Hartkopp <socketcan@hartkopp.net>
Cc: Marc Kleine-Budde <mkl@pengutronix.de>
Cc: linux-can@vger.kernel.org
---
 net/can/bcm.c | 156 ++++++++++++++++++++--------------------------------------
 1 file changed, 52 insertions(+), 104 deletions(-)

diff --git a/net/can/bcm.c b/net/can/bcm.c
index 13690334efa3..9cc67ac257f1 100644
--- a/net/can/bcm.c
+++ b/net/can/bcm.c
@@ -102,7 +102,6 @@ struct bcm_op {
 	unsigned long frames_abs, frames_filtered;
 	struct bcm_timeval ival1, ival2;
 	struct hrtimer timer, thrtimer;
-	struct tasklet_struct tsklet, thrtsklet;
 	ktime_t rx_stamp, kt_ival1, kt_ival2, kt_lastmsg;
 	int rx_ifindex;
 	int cfsiz;
@@ -364,25 +363,34 @@ static void bcm_send_to_user(struct bcm_op *op, struct bcm_msg_head *head,
 	}
 }
 
-static void bcm_tx_start_timer(struct bcm_op *op)
+static bool bcm_tx_set_expiry(struct bcm_op *op, struct hrtimer *hrt)
 {
+	ktime_t ival;
+
 	if (op->kt_ival1 && op->count)
-		hrtimer_start(&op->timer,
-			      ktime_add(ktime_get(), op->kt_ival1),
-			      HRTIMER_MODE_ABS);
+		ival = op->kt_ival1;
 	else if (op->kt_ival2)
-		hrtimer_start(&op->timer,
-			      ktime_add(ktime_get(), op->kt_ival2),
-			      HRTIMER_MODE_ABS);
+		ival = op->kt_ival2;
+	else
+		return false;
+
+	hrtimer_set_expires(hrt, ktime_add(ktime_get(), ival));
+	return true;
 }
 
-static void bcm_tx_timeout_tsklet(unsigned long data)
+static void bcm_tx_start_timer(struct bcm_op *op)
 {
-	struct bcm_op *op = (struct bcm_op *)data;
+	if (bcm_tx_set_expiry(op, &op->timer))
+		hrtimer_start_expires(&op->timer, HRTIMER_MODE_ABS_SOFT);
+}
+
+/* bcm_tx_timeout_handler - performs cyclic CAN frame transmissions */
+static enum hrtimer_restart bcm_tx_timeout_handler(struct hrtimer *hrtimer)
+{
+	struct bcm_op *op = container_of(hrtimer, struct bcm_op, timer);
 	struct bcm_msg_head msg_head;
 
 	if (op->kt_ival1 && (op->count > 0)) {
-
 		op->count--;
 		if (!op->count && (op->flags & TX_COUNTEVT)) {
 
@@ -399,22 +407,12 @@ static void bcm_tx_timeout_tsklet(unsigned long data)
 		}
 		bcm_can_tx(op);
 
-	} else if (op->kt_ival2)
+	} else if (op->kt_ival2) {
 		bcm_can_tx(op);
+	}
 
-	bcm_tx_start_timer(op);
-}
-
-/*
- * bcm_tx_timeout_handler - performs cyclic CAN frame transmissions
- */
-static enum hrtimer_restart bcm_tx_timeout_handler(struct hrtimer *hrtimer)
-{
-	struct bcm_op *op = container_of(hrtimer, struct bcm_op, timer);
-
-	tasklet_schedule(&op->tsklet);
-
-	return HRTIMER_NORESTART;
+	return bcm_tx_set_expiry(op, &op->timer) ?
+		HRTIMER_RESTART : HRTIMER_NORESTART;
 }
 
 /*
@@ -480,7 +478,7 @@ static void bcm_rx_update_and_send(struct bcm_op *op,
 		/* do not send the saved data - only start throttle timer */
 		hrtimer_start(&op->thrtimer,
 			      ktime_add(op->kt_lastmsg, op->kt_ival2),
-			      HRTIMER_MODE_ABS);
+			      HRTIMER_MODE_ABS_SOFT);
 		return;
 	}
 
@@ -539,14 +537,21 @@ static void bcm_rx_starttimer(struct bcm_op *op)
 		return;
 
 	if (op->kt_ival1)
-		hrtimer_start(&op->timer, op->kt_ival1, HRTIMER_MODE_REL);
+		hrtimer_start(&op->timer, op->kt_ival1, HRTIMER_MODE_REL_SOFT);
 }
 
-static void bcm_rx_timeout_tsklet(unsigned long data)
+/* bcm_rx_timeout_handler - when the (cyclic) CAN frame reception timed out */
+static enum hrtimer_restart bcm_rx_timeout_handler(struct hrtimer *hrtimer)
 {
-	struct bcm_op *op = (struct bcm_op *)data;
+	struct bcm_op *op = container_of(hrtimer, struct bcm_op, timer);
 	struct bcm_msg_head msg_head;
 
+	/* if user wants to be informed, when cyclic CAN-Messages come back */
+	if ((op->flags & RX_ANNOUNCE_RESUME) && op->last_frames) {
+		/* clear received CAN frames to indicate 'nothing received' */
+		memset(op->last_frames, 0, op->nframes * op->cfsiz);
+	}
+
 	/* create notification to user */
 	msg_head.opcode  = RX_TIMEOUT;
 	msg_head.flags   = op->flags;
@@ -557,25 +562,6 @@ static void bcm_rx_timeout_tsklet(unsigned long data)
 	msg_head.nframes = 0;
 
 	bcm_send_to_user(op, &msg_head, NULL, 0);
-}
-
-/*
- * bcm_rx_timeout_handler - when the (cyclic) CAN frame reception timed out
- */
-static enum hrtimer_restart bcm_rx_timeout_handler(struct hrtimer *hrtimer)
-{
-	struct bcm_op *op = container_of(hrtimer, struct bcm_op, timer);
-
-	/* schedule before NET_RX_SOFTIRQ */
-	tasklet_hi_schedule(&op->tsklet);
-
-	/* no restart of the timer is done here! */
-
-	/* if user wants to be informed, when cyclic CAN-Messages come back */
-	if ((op->flags & RX_ANNOUNCE_RESUME) && op->last_frames) {
-		/* clear received CAN frames to indicate 'nothing received' */
-		memset(op->last_frames, 0, op->nframes * op->cfsiz);
-	}
 
 	return HRTIMER_NORESTART;
 }
@@ -583,14 +569,12 @@ static enum hrtimer_restart bcm_rx_timeout_handler(struct hrtimer *hrtimer)
 /*
  * bcm_rx_do_flush - helper for bcm_rx_thr_flush
  */
-static inline int bcm_rx_do_flush(struct bcm_op *op, int update,
-				  unsigned int index)
+static inline int bcm_rx_do_flush(struct bcm_op *op, unsigned int index)
 {
 	struct canfd_frame *lcf = op->last_frames + op->cfsiz * index;
 
 	if ((op->last_frames) && (lcf->flags & RX_THR)) {
-		if (update)
-			bcm_rx_changed(op, lcf);
+		bcm_rx_changed(op, lcf);
 		return 1;
 	}
 	return 0;
@@ -598,11 +582,8 @@ static inline int bcm_rx_do_flush(struct bcm_op *op, int update,
 
 /*
  * bcm_rx_thr_flush - Check for throttled data and send it to the userspace
- *
- * update == 0 : just check if throttled data is available  (any irq context)
- * update == 1 : check and send throttled data to userspace (soft_irq context)
  */
-static int bcm_rx_thr_flush(struct bcm_op *op, int update)
+static int bcm_rx_thr_flush(struct bcm_op *op)
 {
 	int updated = 0;
 
@@ -611,24 +592,16 @@ static int bcm_rx_thr_flush(struct bcm_op *op, int update)
 
 		/* for MUX filter we start at index 1 */
 		for (i = 1; i < op->nframes; i++)
-			updated += bcm_rx_do_flush(op, update, i);
+			updated += bcm_rx_do_flush(op, i);
 
 	} else {
 		/* for RX_FILTER_ID and simple filter */
-		updated += bcm_rx_do_flush(op, update, 0);
+		updated += bcm_rx_do_flush(op, 0);
 	}
 
 	return updated;
 }
 
-static void bcm_rx_thr_tsklet(unsigned long data)
-{
-	struct bcm_op *op = (struct bcm_op *)data;
-
-	/* push the changed data to the userspace */
-	bcm_rx_thr_flush(op, 1);
-}
-
 /*
  * bcm_rx_thr_handler - the time for blocked content updates is over now:
  *                      Check for throttled data and send it to the userspace
@@ -637,9 +610,7 @@ static enum hrtimer_restart bcm_rx_thr_handler(struct hrtimer *hrtimer)
 {
 	struct bcm_op *op = container_of(hrtimer, struct bcm_op, thrtimer);
 
-	tasklet_schedule(&op->thrtsklet);
-
-	if (bcm_rx_thr_flush(op, 0)) {
+	if (bcm_rx_thr_flush(op)) {
 		hrtimer_forward(hrtimer, ktime_get(), op->kt_ival2);
 		return HRTIMER_RESTART;
 	} else {
@@ -735,23 +706,8 @@ static struct bcm_op *bcm_find_op(struct list_head *ops,
 
 static void bcm_remove_op(struct bcm_op *op)
 {
-	if (op->tsklet.func) {
-		while (test_bit(TASKLET_STATE_SCHED, &op->tsklet.state) ||
-		       test_bit(TASKLET_STATE_RUN, &op->tsklet.state) ||
-		       hrtimer_active(&op->timer)) {
-			hrtimer_cancel(&op->timer);
-			tasklet_kill(&op->tsklet);
-		}
-	}
-
-	if (op->thrtsklet.func) {
-		while (test_bit(TASKLET_STATE_SCHED, &op->thrtsklet.state) ||
-		       test_bit(TASKLET_STATE_RUN, &op->thrtsklet.state) ||
-		       hrtimer_active(&op->thrtimer)) {
-			hrtimer_cancel(&op->thrtimer);
-			tasklet_kill(&op->thrtsklet);
-		}
-	}
+	hrtimer_cancel(&op->timer);
+	hrtimer_cancel(&op->thrtimer);
 
 	if ((op->frames) && (op->frames != &op->sframe))
 		kfree(op->frames);
@@ -979,15 +935,13 @@ static int bcm_tx_setup(struct bcm_msg_head *msg_head, struct msghdr *msg,
 		op->ifindex = ifindex;
 
 		/* initialize uninitialized (kzalloc) structure */
-		hrtimer_init(&op->timer, CLOCK_MONOTONIC, HRTIMER_MODE_REL);
+		hrtimer_init(&op->timer, CLOCK_MONOTONIC,
+			     HRTIMER_MODE_REL_SOFT);
 		op->timer.function = bcm_tx_timeout_handler;
 
-		/* initialize tasklet for tx countevent notification */
-		tasklet_init(&op->tsklet, bcm_tx_timeout_tsklet,
-			     (unsigned long) op);
-
 		/* currently unused in tx_ops */
-		hrtimer_init(&op->thrtimer, CLOCK_MONOTONIC, HRTIMER_MODE_REL);
+		hrtimer_init(&op->thrtimer, CLOCK_MONOTONIC,
+			     HRTIMER_MODE_REL_SOFT);
 
 		/* add this bcm_op to the list of the tx_ops */
 		list_add(&op->list, &bo->tx_ops);
@@ -1150,20 +1104,14 @@ static int bcm_rx_setup(struct bcm_msg_head *msg_head, struct msghdr *msg,
 		op->rx_ifindex = ifindex;
 
 		/* initialize uninitialized (kzalloc) structure */
-		hrtimer_init(&op->timer, CLOCK_MONOTONIC, HRTIMER_MODE_REL);
+		hrtimer_init(&op->timer, CLOCK_MONOTONIC,
+			     HRTIMER_MODE_REL_SOFT);
 		op->timer.function = bcm_rx_timeout_handler;
 
-		/* initialize tasklet for rx timeout notification */
-		tasklet_init(&op->tsklet, bcm_rx_timeout_tsklet,
-			     (unsigned long) op);
-
-		hrtimer_init(&op->thrtimer, CLOCK_MONOTONIC, HRTIMER_MODE_REL);
+		hrtimer_init(&op->thrtimer, CLOCK_MONOTONIC,
+			     HRTIMER_MODE_REL_SOFT);
 		op->thrtimer.function = bcm_rx_thr_handler;
 
-		/* initialize tasklet for rx throttle handling */
-		tasklet_init(&op->thrtsklet, bcm_rx_thr_tsklet,
-			     (unsigned long) op);
-
 		/* add this bcm_op to the list of the rx_ops */
 		list_add(&op->list, &bo->rx_ops);
 
@@ -1209,12 +1157,12 @@ static int bcm_rx_setup(struct bcm_msg_head *msg_head, struct msghdr *msg,
 			 */
 			op->kt_lastmsg = 0;
 			hrtimer_cancel(&op->thrtimer);
-			bcm_rx_thr_flush(op, 1);
+			bcm_rx_thr_flush(op);
 		}
 
 		if ((op->flags & STARTTIMER) && op->kt_ival1)
 			hrtimer_start(&op->timer, op->kt_ival1,
-				      HRTIMER_MODE_REL);
+				      HRTIMER_MODE_REL_SOFT);
 	}
 
 	/* now we can register for can_ids, if we added a new bcm_op */
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 45+ messages in thread

* [PATCH v3 31/36] mac80211_hwsim: Replace hrtimer tasklet with softirq hrtimer
  2017-11-29 15:30 [PATCH 00/36] hrtimer: Provide softirq context hrtimers Anna-Maria Gleixner
                   ` (29 preceding siblings ...)
  2017-11-29 15:30 ` [PATCH v3 30/36] can/bcm: Replace hrtimer_tasklet with softirq based hrtimer Anna-Maria Gleixner
@ 2017-11-29 15:30 ` Anna-Maria Gleixner
  2017-12-11 11:27   ` Johannes Berg
  2017-11-29 15:30 ` [PATCH v3 32/36] xfrm: " Anna-Maria Gleixner
                   ` (4 subsequent siblings)
  35 siblings, 1 reply; 45+ messages in thread
From: Anna-Maria Gleixner @ 2017-11-29 15:30 UTC (permalink / raw)
  To: LKML
  Cc: Thomas Gleixner, Peter Zijlstra, Ingo Molnar, keescook,
	Christoph Hellwig, John Stultz, Anna-Maria Gleixner,
	linux-wireless, Johannes Berg, Kalle Valo

From: Thomas Gleixner <tglx@linutronix.de>

Switch the timer to HRTIMER_MODE_SOFT, which executed the timer
callback in softirq context and remove the hrtimer_tasklet.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Anna-Maria Gleixner <anna-maria@linutronix.de>
Cc: linux-wireless@vger.kernel.org
Cc: Johannes Berg <johannes@sipsolutions.net>
Cc: Kalle Valo <kvalo@codeaurora.org>
---
 drivers/net/wireless/mac80211_hwsim.c | 44 ++++++++++++++++-------------------
 1 file changed, 20 insertions(+), 24 deletions(-)

diff --git a/drivers/net/wireless/mac80211_hwsim.c b/drivers/net/wireless/mac80211_hwsim.c
index 10b075a46b26..d3f1861398a7 100644
--- a/drivers/net/wireless/mac80211_hwsim.c
+++ b/drivers/net/wireless/mac80211_hwsim.c
@@ -537,7 +537,7 @@ struct mac80211_hwsim_data {
 	unsigned int rx_filter;
 	bool started, idle, scanning;
 	struct mutex mutex;
-	struct tasklet_hrtimer beacon_timer;
+	struct hrtimer beacon_timer;
 	enum ps_mode {
 		PS_DISABLED, PS_ENABLED, PS_AUTO_POLL, PS_MANUAL_POLL
 	} ps;
@@ -1418,7 +1418,7 @@ static void mac80211_hwsim_stop(struct ieee80211_hw *hw)
 {
 	struct mac80211_hwsim_data *data = hw->priv;
 	data->started = false;
-	tasklet_hrtimer_cancel(&data->beacon_timer);
+	hrtimer_cancel(&data->beacon_timer);
 	wiphy_dbg(hw->wiphy, "%s\n", __func__);
 }
 
@@ -1541,14 +1541,12 @@ static enum hrtimer_restart
 mac80211_hwsim_beacon(struct hrtimer *timer)
 {
 	struct mac80211_hwsim_data *data =
-		container_of(timer, struct mac80211_hwsim_data,
-			     beacon_timer.timer);
+		container_of(timer, struct mac80211_hwsim_data, beacon_timer);
 	struct ieee80211_hw *hw = data->hw;
 	u64 bcn_int = data->beacon_int;
-	ktime_t next_bcn;
 
 	if (!data->started)
-		goto out;
+		return HRTIMER_NORESTART;
 
 	ieee80211_iterate_active_interfaces_atomic(
 		hw, IEEE80211_IFACE_ITER_NORMAL,
@@ -1560,11 +1558,9 @@ mac80211_hwsim_beacon(struct hrtimer *timer)
 		data->bcn_delta = 0;
 	}
 
-	next_bcn = ktime_add(hrtimer_get_expires(timer),
-			     ns_to_ktime(bcn_int * 1000));
-	tasklet_hrtimer_start(&data->beacon_timer, next_bcn, HRTIMER_MODE_ABS);
-out:
-	return HRTIMER_NORESTART;
+	hrtimer_forward(&data->beacon_timer, hrtimer_get_expires(timer),
+			ns_to_ktime(bcn_int * NSEC_PER_USEC));
+	return HRTIMER_RESTART;
 }
 
 static const char * const hwsim_chanwidths[] = {
@@ -1638,15 +1634,15 @@ static int mac80211_hwsim_config(struct ieee80211_hw *hw, u32 changed)
 	mutex_unlock(&data->mutex);
 
 	if (!data->started || !data->beacon_int)
-		tasklet_hrtimer_cancel(&data->beacon_timer);
-	else if (!hrtimer_is_queued(&data->beacon_timer.timer)) {
+		hrtimer_cancel(&data->beacon_timer);
+	else if (!hrtimer_is_queued(&data->beacon_timer)) {
 		u64 tsf = mac80211_hwsim_get_tsf(hw, NULL);
 		u32 bcn_int = data->beacon_int;
 		u64 until_tbtt = bcn_int - do_div(tsf, bcn_int);
 
-		tasklet_hrtimer_start(&data->beacon_timer,
-				      ns_to_ktime(until_tbtt * 1000),
-				      HRTIMER_MODE_REL);
+		hrtimer_start(&data->beacon_timer,
+			      ns_to_ktime(until_tbtt * 1000),
+			      HRTIMER_MODE_REL_SOFT);
 	}
 
 	return 0;
@@ -1709,7 +1705,7 @@ static void mac80211_hwsim_bss_info_changed(struct ieee80211_hw *hw,
 			  info->enable_beacon, info->beacon_int);
 		vp->bcn_en = info->enable_beacon;
 		if (data->started &&
-		    !hrtimer_is_queued(&data->beacon_timer.timer) &&
+		    !hrtimer_is_queued(&data->beacon_timer) &&
 		    info->enable_beacon) {
 			u64 tsf, until_tbtt;
 			u32 bcn_int;
@@ -1717,9 +1713,9 @@ static void mac80211_hwsim_bss_info_changed(struct ieee80211_hw *hw,
 			tsf = mac80211_hwsim_get_tsf(hw, vif);
 			bcn_int = data->beacon_int;
 			until_tbtt = bcn_int - do_div(tsf, bcn_int);
-			tasklet_hrtimer_start(&data->beacon_timer,
-					      ns_to_ktime(until_tbtt * 1000),
-					      HRTIMER_MODE_REL);
+			hrtimer_start(&data->beacon_timer,
+				      ns_to_ktime(until_tbtt * 1000),
+				      HRTIMER_MODE_REL_SOFT);
 		} else if (!info->enable_beacon) {
 			unsigned int count = 0;
 			ieee80211_iterate_active_interfaces_atomic(
@@ -1728,7 +1724,7 @@ static void mac80211_hwsim_bss_info_changed(struct ieee80211_hw *hw,
 			wiphy_dbg(hw->wiphy, "  beaconing vifs remaining: %u",
 				  count);
 			if (count == 0) {
-				tasklet_hrtimer_cancel(&data->beacon_timer);
+				hrtimer_cancel(&data->beacon_timer);
 				data->beacon_int = 0;
 			}
 		}
@@ -2720,9 +2716,9 @@ static int mac80211_hwsim_new_radio(struct genl_info *info,
 				    data->debugfs,
 				    data, &hwsim_simulate_radar);
 
-	tasklet_hrtimer_init(&data->beacon_timer,
-			     mac80211_hwsim_beacon,
-			     CLOCK_MONOTONIC, HRTIMER_MODE_ABS);
+	hrtimer_init(&data->beacon_timer, CLOCK_MONOTONIC,
+		     HRTIMER_MODE_REL_SOFT);
+	data->beacon_timer.function = mac80211_hwsim_beacon;
 
 	spin_lock_bh(&hwsim_radio_lock);
 	list_add_tail(&data->list, &hwsim_radios);
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 45+ messages in thread

* [PATCH v3 32/36] xfrm: Replace hrtimer tasklet with softirq hrtimer
  2017-11-29 15:30 [PATCH 00/36] hrtimer: Provide softirq context hrtimers Anna-Maria Gleixner
                   ` (30 preceding siblings ...)
  2017-11-29 15:30 ` [PATCH v3 31/36] mac80211_hwsim: Replace hrtimer tasklet with softirq hrtimer Anna-Maria Gleixner
@ 2017-11-29 15:30 ` Anna-Maria Gleixner
  2017-11-29 15:30 ` [PATCH v3 33/36] softirq: Remove tasklet_hrtimer Anna-Maria Gleixner
                   ` (3 subsequent siblings)
  35 siblings, 0 replies; 45+ messages in thread
From: Anna-Maria Gleixner @ 2017-11-29 15:30 UTC (permalink / raw)
  To: LKML
  Cc: Thomas Gleixner, Peter Zijlstra, Ingo Molnar, keescook,
	Christoph Hellwig, John Stultz, Anna-Maria Gleixner,
	Steffen Klassert, netdev, Herbert Xu, David S. Miller

From: Thomas Gleixner <tglx@linutronix.de>

Switch the timer to HRTIMER_MODE_SOFT, which executed the timer
callback in softirq context and remove the hrtimer_tasklet.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Anna-Maria Gleixner <anna-maria@linutronix.de>
Cc: Steffen Klassert <steffen.klassert@secunet.com>
Cc: netdev@vger.kernel.org
Cc: Herbert Xu <herbert@gondor.apana.org.au>
Cc: "David S. Miller" <davem@davemloft.net>
---
 include/net/xfrm.h    |  2 +-
 net/xfrm/xfrm_state.c | 30 ++++++++++++++++++------------
 2 files changed, 19 insertions(+), 13 deletions(-)

diff --git a/include/net/xfrm.h b/include/net/xfrm.h
index dc28a98ce97c..e706ec81bd14 100644
--- a/include/net/xfrm.h
+++ b/include/net/xfrm.h
@@ -217,7 +217,7 @@ struct xfrm_state {
 	struct xfrm_stats	stats;
 
 	struct xfrm_lifetime_cur curlft;
-	struct tasklet_hrtimer	mtimer;
+	struct hrtimer		mtimer;
 
 	struct xfrm_state_offload xso;
 
diff --git a/net/xfrm/xfrm_state.c b/net/xfrm/xfrm_state.c
index 065d89606888..4be5fc7038af 100644
--- a/net/xfrm/xfrm_state.c
+++ b/net/xfrm/xfrm_state.c
@@ -426,7 +426,7 @@ static void xfrm_put_mode(struct xfrm_mode *mode)
 
 static void xfrm_state_gc_destroy(struct xfrm_state *x)
 {
-	tasklet_hrtimer_cancel(&x->mtimer);
+	hrtimer_cancel(&x->mtimer);
 	del_timer_sync(&x->rtimer);
 	kfree(x->aead);
 	kfree(x->aalg);
@@ -471,8 +471,8 @@ static void xfrm_state_gc_task(struct work_struct *work)
 
 static enum hrtimer_restart xfrm_timer_handler(struct hrtimer *me)
 {
-	struct tasklet_hrtimer *thr = container_of(me, struct tasklet_hrtimer, timer);
-	struct xfrm_state *x = container_of(thr, struct xfrm_state, mtimer);
+	struct xfrm_state *x = container_of(me, struct xfrm_state, mtimer);
+	enum hrtimer_restart ret = HRTIMER_NORESTART;
 	unsigned long now = get_seconds();
 	long next = LONG_MAX;
 	int warn = 0;
@@ -536,7 +536,8 @@ static enum hrtimer_restart xfrm_timer_handler(struct hrtimer *me)
 		km_state_expired(x, 0, 0);
 resched:
 	if (next != LONG_MAX) {
-		tasklet_hrtimer_start(&x->mtimer, ktime_set(next, 0), HRTIMER_MODE_REL);
+		hrtimer_forward_now(&x->mtimer, ktime_set(next, 0));
+		ret = HRTIMER_RESTART;
 	}
 
 	goto out;
@@ -553,7 +554,7 @@ static enum hrtimer_restart xfrm_timer_handler(struct hrtimer *me)
 
 out:
 	spin_unlock(&x->lock);
-	return HRTIMER_NORESTART;
+	return ret;
 }
 
 static void xfrm_replay_timer_handler(struct timer_list *t);
@@ -572,8 +573,8 @@ struct xfrm_state *xfrm_state_alloc(struct net *net)
 		INIT_HLIST_NODE(&x->bydst);
 		INIT_HLIST_NODE(&x->bysrc);
 		INIT_HLIST_NODE(&x->byspi);
-		tasklet_hrtimer_init(&x->mtimer, xfrm_timer_handler,
-					CLOCK_BOOTTIME, HRTIMER_MODE_ABS);
+		hrtimer_init(&x->mtimer, CLOCK_BOOTTIME, HRTIMER_MODE_ABS_SOFT);
+		x->mtimer.function = xfrm_timer_handler;
 		timer_setup(&x->rtimer, xfrm_replay_timer_handler, 0);
 		x->curlft.add_time = get_seconds();
 		x->lft.soft_byte_limit = XFRM_INF;
@@ -1029,7 +1030,9 @@ xfrm_state_find(const xfrm_address_t *daddr, const xfrm_address_t *saddr,
 				hlist_add_head_rcu(&x->byspi, net->xfrm.state_byspi + h);
 			}
 			x->lft.hard_add_expires_seconds = net->xfrm.sysctl_acq_expires;
-			tasklet_hrtimer_start(&x->mtimer, ktime_set(net->xfrm.sysctl_acq_expires, 0), HRTIMER_MODE_REL);
+			hrtimer_start(&x->mtimer,
+				      ktime_set(net->xfrm.sysctl_acq_expires, 0),
+				      HRTIMER_MODE_REL_SOFT);
 			net->xfrm.state_num++;
 			xfrm_hash_grow_check(net, x->bydst.next != NULL);
 			spin_unlock_bh(&net->xfrm.xfrm_state_lock);
@@ -1140,7 +1143,7 @@ static void __xfrm_state_insert(struct xfrm_state *x)
 		hlist_add_head_rcu(&x->byspi, net->xfrm.state_byspi + h);
 	}
 
-	tasklet_hrtimer_start(&x->mtimer, ktime_set(1, 0), HRTIMER_MODE_REL);
+	hrtimer_start(&x->mtimer, ktime_set(1, 0), HRTIMER_MODE_REL_SOFT);
 	if (x->replay_maxage)
 		mod_timer(&x->rtimer, jiffies + x->replay_maxage);
 
@@ -1244,7 +1247,9 @@ static struct xfrm_state *__find_acq_core(struct net *net,
 		x->mark.m = m->m;
 		x->lft.hard_add_expires_seconds = net->xfrm.sysctl_acq_expires;
 		xfrm_state_hold(x);
-		tasklet_hrtimer_start(&x->mtimer, ktime_set(net->xfrm.sysctl_acq_expires, 0), HRTIMER_MODE_REL);
+		hrtimer_start(&x->mtimer,
+			      ktime_set(net->xfrm.sysctl_acq_expires, 0),
+			      HRTIMER_MODE_REL_SOFT);
 		list_add(&x->km.all, &net->xfrm.state_all);
 		hlist_add_head_rcu(&x->bydst, net->xfrm.state_bydst + h);
 		h = xfrm_src_hash(net, daddr, saddr, family);
@@ -1543,7 +1548,8 @@ int xfrm_state_update(struct xfrm_state *x)
 		memcpy(&x1->lft, &x->lft, sizeof(x1->lft));
 		x1->km.dying = 0;
 
-		tasklet_hrtimer_start(&x1->mtimer, ktime_set(1, 0), HRTIMER_MODE_REL);
+		hrtimer_start(&x1->mtimer, ktime_set(1, 0),
+			      HRTIMER_MODE_REL_SOFT);
 		if (x1->curlft.use_time)
 			xfrm_state_check_expire(x1);
 
@@ -1567,7 +1573,7 @@ int xfrm_state_check_expire(struct xfrm_state *x)
 	if (x->curlft.bytes >= x->lft.hard_byte_limit ||
 	    x->curlft.packets >= x->lft.hard_packet_limit) {
 		x->km.state = XFRM_STATE_EXPIRED;
-		tasklet_hrtimer_start(&x->mtimer, 0, HRTIMER_MODE_REL);
+		hrtimer_start(&x->mtimer, 0, HRTIMER_MODE_REL_SOFT);
 		return -EINVAL;
 	}
 
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 45+ messages in thread

* [PATCH v3 33/36] softirq: Remove tasklet_hrtimer
  2017-11-29 15:30 [PATCH 00/36] hrtimer: Provide softirq context hrtimers Anna-Maria Gleixner
                   ` (31 preceding siblings ...)
  2017-11-29 15:30 ` [PATCH v3 32/36] xfrm: " Anna-Maria Gleixner
@ 2017-11-29 15:30 ` Anna-Maria Gleixner
  2017-11-29 15:30   ` Anna-Maria Gleixner
                   ` (2 subsequent siblings)
  35 siblings, 0 replies; 45+ messages in thread
From: Anna-Maria Gleixner @ 2017-11-29 15:30 UTC (permalink / raw)
  To: LKML
  Cc: Thomas Gleixner, Peter Zijlstra, Ingo Molnar, keescook,
	Christoph Hellwig, John Stultz, Anna-Maria Gleixner

From: Thomas Gleixner <tglx@linutronix.de>

There are no more tasklet_hrtimer users of this interface.
Remove it.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Anna-Maria Gleixner <anna-maria@linutronix.de>
---
 include/linux/interrupt.h | 25 -----------------------
 kernel/softirq.c          | 51 -----------------------------------------------
 2 files changed, 76 deletions(-)

diff --git a/include/linux/interrupt.h b/include/linux/interrupt.h
index 69c238210325..af15bfd89598 100644
--- a/include/linux/interrupt.h
+++ b/include/linux/interrupt.h
@@ -618,31 +618,6 @@ extern void tasklet_kill_immediate(struct tasklet_struct *t, unsigned int cpu);
 extern void tasklet_init(struct tasklet_struct *t,
 			 void (*func)(unsigned long), unsigned long data);
 
-struct tasklet_hrtimer {
-	struct hrtimer		timer;
-	struct tasklet_struct	tasklet;
-	enum hrtimer_restart	(*function)(struct hrtimer *);
-};
-
-extern void
-tasklet_hrtimer_init(struct tasklet_hrtimer *ttimer,
-		     enum hrtimer_restart (*function)(struct hrtimer *),
-		     clockid_t which_clock, enum hrtimer_mode mode);
-
-static inline
-void tasklet_hrtimer_start(struct tasklet_hrtimer *ttimer, ktime_t time,
-			   const enum hrtimer_mode mode)
-{
-	hrtimer_start(&ttimer->timer, time, mode);
-}
-
-static inline
-void tasklet_hrtimer_cancel(struct tasklet_hrtimer *ttimer)
-{
-	hrtimer_cancel(&ttimer->timer);
-	tasklet_kill(&ttimer->tasklet);
-}
-
 /*
  * Autoprobing for irqs:
  *
diff --git a/kernel/softirq.c b/kernel/softirq.c
index 2f5e87f1bae2..d0dd7795409f 100644
--- a/kernel/softirq.c
+++ b/kernel/softirq.c
@@ -584,57 +584,6 @@ void tasklet_kill(struct tasklet_struct *t)
 }
 EXPORT_SYMBOL(tasklet_kill);
 
-/*
- * tasklet_hrtimer
- */
-
-/*
- * The trampoline is called when the hrtimer expires. It schedules a tasklet
- * to run __tasklet_hrtimer_trampoline() which in turn will call the intended
- * hrtimer callback, but from softirq context.
- */
-static enum hrtimer_restart __hrtimer_tasklet_trampoline(struct hrtimer *timer)
-{
-	struct tasklet_hrtimer *ttimer =
-		container_of(timer, struct tasklet_hrtimer, timer);
-
-	tasklet_hi_schedule(&ttimer->tasklet);
-	return HRTIMER_NORESTART;
-}
-
-/*
- * Helper function which calls the hrtimer callback from
- * tasklet/softirq context
- */
-static void __tasklet_hrtimer_trampoline(unsigned long data)
-{
-	struct tasklet_hrtimer *ttimer = (void *)data;
-	enum hrtimer_restart restart;
-
-	restart = ttimer->function(&ttimer->timer);
-	if (restart != HRTIMER_NORESTART)
-		hrtimer_restart(&ttimer->timer);
-}
-
-/**
- * tasklet_hrtimer_init - Init a tasklet/hrtimer combo for softirq callbacks
- * @ttimer:	 tasklet_hrtimer which is initialized
- * @function:	 hrtimer callback function which gets called from softirq context
- * @which_clock: clock id (CLOCK_MONOTONIC/CLOCK_REALTIME)
- * @mode:	 hrtimer mode (HRTIMER_MODE_ABS/HRTIMER_MODE_REL)
- */
-void tasklet_hrtimer_init(struct tasklet_hrtimer *ttimer,
-			  enum hrtimer_restart (*function)(struct hrtimer *),
-			  clockid_t which_clock, enum hrtimer_mode mode)
-{
-	hrtimer_init(&ttimer->timer, which_clock, mode);
-	ttimer->timer.function = __hrtimer_tasklet_trampoline;
-	tasklet_init(&ttimer->tasklet, __tasklet_hrtimer_trampoline,
-		     (unsigned long)ttimer);
-	ttimer->function = function;
-}
-EXPORT_SYMBOL_GPL(tasklet_hrtimer_init);
-
 void __init softirq_init(void)
 {
 	int cpu;
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 45+ messages in thread

* [PATCH v3 34/36] ALSA/dummy: Replace tasklet with softirq hrtimer
  2017-11-29 15:30 [PATCH 00/36] hrtimer: Provide softirq context hrtimers Anna-Maria Gleixner
@ 2017-11-29 15:30   ` Anna-Maria Gleixner
  2017-11-29 15:30 ` [PATCH v3 02/36] hrtimer: Correct blantanly wrong comment Anna-Maria Gleixner
                     ` (34 subsequent siblings)
  35 siblings, 0 replies; 45+ messages in thread
From: Anna-Maria Gleixner @ 2017-11-29 15:30 UTC (permalink / raw)
  To: LKML
  Cc: Thomas Gleixner, Peter Zijlstra, Ingo Molnar, keescook,
	Christoph Hellwig, John Stultz, Anna-Maria Gleixner, alsa-devel,
	Takashi Sakamoto, Takashi Iwai, Jaroslav Kysela

From: Thomas Gleixner <tglx@linutronix.de>

The tasklet is used to defer the execution of snd_pcm_period_elapsed() to
the softirq context. Using the HRTIMER_MODE_SOFT mode invokes the timer
callback in softirq context as well which renders the tasklet useless.

[o-takashi: avoid stall due to a call of hrtimer_cancel() on a callback
            of hrtimer]

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Anna-Maria Gleixner <anna-maria@linutronix.de>
Cc: alsa-devel@alsa-project.org
Cc: Takashi Sakamoto <o-takashi@sakamocchi.jp>
Cc: Takashi Iwai <tiwai@suse.com>
Cc: Jaroslav Kysela <perex@perex.cz>
Link: http://lkml.kernel.org/r/20170905161820.jtysvxtfleunbbmf@breakpoint.cc
---
 sound/drivers/dummy.c | 27 ++++++++++++---------------
 1 file changed, 12 insertions(+), 15 deletions(-)

diff --git a/sound/drivers/dummy.c b/sound/drivers/dummy.c
index 7b2b1f766b00..6ad2ff57833d 100644
--- a/sound/drivers/dummy.c
+++ b/sound/drivers/dummy.c
@@ -375,17 +375,9 @@ struct dummy_hrtimer_pcm {
 	ktime_t period_time;
 	atomic_t running;
 	struct hrtimer timer;
-	struct tasklet_struct tasklet;
 	struct snd_pcm_substream *substream;
 };
 
-static void dummy_hrtimer_pcm_elapsed(unsigned long priv)
-{
-	struct dummy_hrtimer_pcm *dpcm = (struct dummy_hrtimer_pcm *)priv;
-	if (atomic_read(&dpcm->running))
-		snd_pcm_period_elapsed(dpcm->substream);
-}
-
 static enum hrtimer_restart dummy_hrtimer_callback(struct hrtimer *timer)
 {
 	struct dummy_hrtimer_pcm *dpcm;
@@ -393,7 +385,14 @@ static enum hrtimer_restart dummy_hrtimer_callback(struct hrtimer *timer)
 	dpcm = container_of(timer, struct dummy_hrtimer_pcm, timer);
 	if (!atomic_read(&dpcm->running))
 		return HRTIMER_NORESTART;
-	tasklet_schedule(&dpcm->tasklet);
+	/*
+	 * In cases of XRUN and draining, this calls .trigger to stop PCM
+	 * substream.
+	 */
+	snd_pcm_period_elapsed(dpcm->substream);
+	if (!atomic_read(&dpcm->running))
+		return HRTIMER_NORESTART;
+
 	hrtimer_forward_now(timer, dpcm->period_time);
 	return HRTIMER_RESTART;
 }
@@ -403,7 +402,7 @@ static int dummy_hrtimer_start(struct snd_pcm_substream *substream)
 	struct dummy_hrtimer_pcm *dpcm = substream->runtime->private_data;
 
 	dpcm->base_time = hrtimer_cb_get_time(&dpcm->timer);
-	hrtimer_start(&dpcm->timer, dpcm->period_time, HRTIMER_MODE_REL);
+	hrtimer_start(&dpcm->timer, dpcm->period_time, HRTIMER_MODE_REL_SOFT);
 	atomic_set(&dpcm->running, 1);
 	return 0;
 }
@@ -413,14 +412,14 @@ static int dummy_hrtimer_stop(struct snd_pcm_substream *substream)
 	struct dummy_hrtimer_pcm *dpcm = substream->runtime->private_data;
 
 	atomic_set(&dpcm->running, 0);
-	hrtimer_cancel(&dpcm->timer);
+	if (!hrtimer_callback_running(&dpcm->timer))
+		hrtimer_cancel(&dpcm->timer);
 	return 0;
 }
 
 static inline void dummy_hrtimer_sync(struct dummy_hrtimer_pcm *dpcm)
 {
 	hrtimer_cancel(&dpcm->timer);
-	tasklet_kill(&dpcm->tasklet);
 }
 
 static snd_pcm_uframes_t
@@ -465,12 +464,10 @@ static int dummy_hrtimer_create(struct snd_pcm_substream *substream)
 	if (!dpcm)
 		return -ENOMEM;
 	substream->runtime->private_data = dpcm;
-	hrtimer_init(&dpcm->timer, CLOCK_MONOTONIC, HRTIMER_MODE_REL);
+	hrtimer_init(&dpcm->timer, CLOCK_MONOTONIC, HRTIMER_MODE_REL_SOFT);
 	dpcm->timer.function = dummy_hrtimer_callback;
 	dpcm->substream = substream;
 	atomic_set(&dpcm->running, 0);
-	tasklet_init(&dpcm->tasklet, dummy_hrtimer_pcm_elapsed,
-		     (unsigned long)dpcm);
 	return 0;
 }
 
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 45+ messages in thread

* [PATCH v3 34/36] ALSA/dummy: Replace tasklet with softirq hrtimer
@ 2017-11-29 15:30   ` Anna-Maria Gleixner
  0 siblings, 0 replies; 45+ messages in thread
From: Anna-Maria Gleixner @ 2017-11-29 15:30 UTC (permalink / raw)
  To: LKML
  Cc: alsa-devel, keescook, Peter Zijlstra, Takashi Iwai,
	Takashi Sakamoto, Ingo Molnar, John Stultz, Thomas Gleixner,
	Anna-Maria Gleixner, Christoph Hellwig

From: Thomas Gleixner <tglx@linutronix.de>

The tasklet is used to defer the execution of snd_pcm_period_elapsed() to
the softirq context. Using the HRTIMER_MODE_SOFT mode invokes the timer
callback in softirq context as well which renders the tasklet useless.

[o-takashi: avoid stall due to a call of hrtimer_cancel() on a callback
            of hrtimer]

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Anna-Maria Gleixner <anna-maria@linutronix.de>
Cc: alsa-devel@alsa-project.org
Cc: Takashi Sakamoto <o-takashi@sakamocchi.jp>
Cc: Takashi Iwai <tiwai@suse.com>
Cc: Jaroslav Kysela <perex@perex.cz>
Link: http://lkml.kernel.org/r/20170905161820.jtysvxtfleunbbmf@breakpoint.cc
---
 sound/drivers/dummy.c | 27 ++++++++++++---------------
 1 file changed, 12 insertions(+), 15 deletions(-)

diff --git a/sound/drivers/dummy.c b/sound/drivers/dummy.c
index 7b2b1f766b00..6ad2ff57833d 100644
--- a/sound/drivers/dummy.c
+++ b/sound/drivers/dummy.c
@@ -375,17 +375,9 @@ struct dummy_hrtimer_pcm {
 	ktime_t period_time;
 	atomic_t running;
 	struct hrtimer timer;
-	struct tasklet_struct tasklet;
 	struct snd_pcm_substream *substream;
 };
 
-static void dummy_hrtimer_pcm_elapsed(unsigned long priv)
-{
-	struct dummy_hrtimer_pcm *dpcm = (struct dummy_hrtimer_pcm *)priv;
-	if (atomic_read(&dpcm->running))
-		snd_pcm_period_elapsed(dpcm->substream);
-}
-
 static enum hrtimer_restart dummy_hrtimer_callback(struct hrtimer *timer)
 {
 	struct dummy_hrtimer_pcm *dpcm;
@@ -393,7 +385,14 @@ static enum hrtimer_restart dummy_hrtimer_callback(struct hrtimer *timer)
 	dpcm = container_of(timer, struct dummy_hrtimer_pcm, timer);
 	if (!atomic_read(&dpcm->running))
 		return HRTIMER_NORESTART;
-	tasklet_schedule(&dpcm->tasklet);
+	/*
+	 * In cases of XRUN and draining, this calls .trigger to stop PCM
+	 * substream.
+	 */
+	snd_pcm_period_elapsed(dpcm->substream);
+	if (!atomic_read(&dpcm->running))
+		return HRTIMER_NORESTART;
+
 	hrtimer_forward_now(timer, dpcm->period_time);
 	return HRTIMER_RESTART;
 }
@@ -403,7 +402,7 @@ static int dummy_hrtimer_start(struct snd_pcm_substream *substream)
 	struct dummy_hrtimer_pcm *dpcm = substream->runtime->private_data;
 
 	dpcm->base_time = hrtimer_cb_get_time(&dpcm->timer);
-	hrtimer_start(&dpcm->timer, dpcm->period_time, HRTIMER_MODE_REL);
+	hrtimer_start(&dpcm->timer, dpcm->period_time, HRTIMER_MODE_REL_SOFT);
 	atomic_set(&dpcm->running, 1);
 	return 0;
 }
@@ -413,14 +412,14 @@ static int dummy_hrtimer_stop(struct snd_pcm_substream *substream)
 	struct dummy_hrtimer_pcm *dpcm = substream->runtime->private_data;
 
 	atomic_set(&dpcm->running, 0);
-	hrtimer_cancel(&dpcm->timer);
+	if (!hrtimer_callback_running(&dpcm->timer))
+		hrtimer_cancel(&dpcm->timer);
 	return 0;
 }
 
 static inline void dummy_hrtimer_sync(struct dummy_hrtimer_pcm *dpcm)
 {
 	hrtimer_cancel(&dpcm->timer);
-	tasklet_kill(&dpcm->tasklet);
 }
 
 static snd_pcm_uframes_t
@@ -465,12 +464,10 @@ static int dummy_hrtimer_create(struct snd_pcm_substream *substream)
 	if (!dpcm)
 		return -ENOMEM;
 	substream->runtime->private_data = dpcm;
-	hrtimer_init(&dpcm->timer, CLOCK_MONOTONIC, HRTIMER_MODE_REL);
+	hrtimer_init(&dpcm->timer, CLOCK_MONOTONIC, HRTIMER_MODE_REL_SOFT);
 	dpcm->timer.function = dummy_hrtimer_callback;
 	dpcm->substream = substream;
 	atomic_set(&dpcm->running, 0);
-	tasklet_init(&dpcm->tasklet, dummy_hrtimer_pcm_elapsed,
-		     (unsigned long)dpcm);
 	return 0;
 }
 
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 45+ messages in thread

* [PATCH v3 35/36] usb/gadget/NCM: Replace tasklet with softirq hrtimer
  2017-11-29 15:30 [PATCH 00/36] hrtimer: Provide softirq context hrtimers Anna-Maria Gleixner
                   ` (33 preceding siblings ...)
  2017-11-29 15:30   ` Anna-Maria Gleixner
@ 2017-11-29 15:31 ` Anna-Maria Gleixner
  2017-12-04 13:14   ` Felipe Balbi
  2017-11-29 15:31 ` [PATCH v3 36/36] net/mvpp2: " Anna-Maria Gleixner
  35 siblings, 1 reply; 45+ messages in thread
From: Anna-Maria Gleixner @ 2017-11-29 15:31 UTC (permalink / raw)
  To: LKML
  Cc: Thomas Gleixner, Peter Zijlstra, Ingo Molnar, keescook,
	Christoph Hellwig, John Stultz, Anna-Maria Gleixner,
	Felipe Balbi, linux-usb

From: Thomas Gleixner <tglx@linutronix.de>

The tx_tasklet tasklet is used in invoke the hrtimer (task_timer) in
softirq context. This can be also achieved without the tasklet but
with HRTIMER_MODE_SOFT as hrtimer mode.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Anna-Maria Gleixner <anna-maria@linutronix.de>
Cc: Felipe Balbi <balbi@kernel.org>
Cc: linux-usb@vger.kernel.org
---
 drivers/usb/gadget/function/f_ncm.c | 30 +++++++-----------------------
 1 file changed, 7 insertions(+), 23 deletions(-)

diff --git a/drivers/usb/gadget/function/f_ncm.c b/drivers/usb/gadget/function/f_ncm.c
index c5bce8e22983..5780fba620ab 100644
--- a/drivers/usb/gadget/function/f_ncm.c
+++ b/drivers/usb/gadget/function/f_ncm.c
@@ -73,9 +73,7 @@ struct f_ncm {
 	struct sk_buff			*skb_tx_ndp;
 	u16				ndp_dgram_count;
 	bool				timer_force_tx;
-	struct tasklet_struct		tx_tasklet;
 	struct hrtimer			task_timer;
-
 	bool				timer_stopping;
 };
 
@@ -1104,7 +1102,7 @@ static struct sk_buff *ncm_wrap_ntb(struct gether *port,
 
 		/* Delay the timer. */
 		hrtimer_start(&ncm->task_timer, TX_TIMEOUT_NSECS,
-			      HRTIMER_MODE_REL);
+			      HRTIMER_MODE_REL_SOFT);
 
 		/* Add the datagram position entries */
 		ntb_ndp = skb_put_zero(ncm->skb_tx_ndp, dgram_idx_len);
@@ -1148,17 +1146,15 @@ static struct sk_buff *ncm_wrap_ntb(struct gether *port,
 }
 
 /*
- * This transmits the NTB if there are frames waiting.
+ * The transmit should only be run if no skb data has been sent
+ * for a certain duration.
  */
-static void ncm_tx_tasklet(unsigned long data)
+static enum hrtimer_restart ncm_tx_timeout(struct hrtimer *data)
 {
-	struct f_ncm	*ncm = (void *)data;
-
-	if (ncm->timer_stopping)
-		return;
+	struct f_ncm *ncm = container_of(data, struct f_ncm, task_timer);
 
 	/* Only send if data is available. */
-	if (ncm->skb_tx_data) {
+	if (!ncm->timer_stopping && ncm->skb_tx_data) {
 		ncm->timer_force_tx = true;
 
 		/* XXX This allowance of a NULL skb argument to ndo_start_xmit
@@ -1171,16 +1167,6 @@ static void ncm_tx_tasklet(unsigned long data)
 
 		ncm->timer_force_tx = false;
 	}
-}
-
-/*
- * The transmit should only be run if no skb data has been sent
- * for a certain duration.
- */
-static enum hrtimer_restart ncm_tx_timeout(struct hrtimer *data)
-{
-	struct f_ncm *ncm = container_of(data, struct f_ncm, task_timer);
-	tasklet_schedule(&ncm->tx_tasklet);
 	return HRTIMER_NORESTART;
 }
 
@@ -1513,8 +1499,7 @@ static int ncm_bind(struct usb_configuration *c, struct usb_function *f)
 	ncm->port.open = ncm_open;
 	ncm->port.close = ncm_close;
 
-	tasklet_init(&ncm->tx_tasklet, ncm_tx_tasklet, (unsigned long) ncm);
-	hrtimer_init(&ncm->task_timer, CLOCK_MONOTONIC, HRTIMER_MODE_REL);
+	hrtimer_init(&ncm->task_timer, CLOCK_MONOTONIC, HRTIMER_MODE_REL_SOFT);
 	ncm->task_timer.function = ncm_tx_timeout;
 
 	DBG(cdev, "CDC Network: %s speed IN/%s OUT/%s NOTIFY/%s\n",
@@ -1623,7 +1608,6 @@ static void ncm_unbind(struct usb_configuration *c, struct usb_function *f)
 	DBG(c->cdev, "ncm unbind\n");
 
 	hrtimer_cancel(&ncm->task_timer);
-	tasklet_kill(&ncm->tx_tasklet);
 
 	ncm_string_defs[0].id = 0;
 	usb_free_all_descriptors(f);
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 45+ messages in thread

* [PATCH v3 36/36] net/mvpp2: Replace tasklet with softirq hrtimer
  2017-11-29 15:30 [PATCH 00/36] hrtimer: Provide softirq context hrtimers Anna-Maria Gleixner
                   ` (34 preceding siblings ...)
  2017-11-29 15:31 ` [PATCH v3 35/36] usb/gadget/NCM: " Anna-Maria Gleixner
@ 2017-11-29 15:31 ` Anna-Maria Gleixner
  35 siblings, 0 replies; 45+ messages in thread
From: Anna-Maria Gleixner @ 2017-11-29 15:31 UTC (permalink / raw)
  To: LKML
  Cc: Thomas Gleixner, Peter Zijlstra, Ingo Molnar, keescook,
	Christoph Hellwig, John Stultz, Anna-Maria Gleixner,
	Thomas Petazzoni, netdev, David S. Miller

From: Thomas Gleixner <tglx@linutronix.de>

The tx_done_tasklet tasklet is used in invoke the hrtimer
(mvpp2_hr_timer_cb) in softirq context. This can be also achieved without
the tasklet but with HRTIMER_MODE_SOFT as hrtimer mode.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Anna-Maria Gleixner <anna-maria@linutronix.de>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: netdev@vger.kernel.org
Cc: "David S. Miller" <davem@davemloft.net>
---
 drivers/net/ethernet/marvell/mvpp2.c | 62 +++++++++++++++---------------------
 1 file changed, 25 insertions(+), 37 deletions(-)

diff --git a/drivers/net/ethernet/marvell/mvpp2.c b/drivers/net/ethernet/marvell/mvpp2.c
index 6c20e811f973..ab77f68c7fba 100644
--- a/drivers/net/ethernet/marvell/mvpp2.c
+++ b/drivers/net/ethernet/marvell/mvpp2.c
@@ -901,9 +901,8 @@ struct mvpp2_pcpu_stats {
 /* Per-CPU port control */
 struct mvpp2_port_pcpu {
 	struct hrtimer tx_done_timer;
+	struct net_device *dev;
 	bool timer_scheduled;
-	/* Tasklet for egress finalization */
-	struct tasklet_struct tx_done_tasklet;
 };
 
 struct mvpp2_queue_vector {
@@ -6181,46 +6180,34 @@ static void mvpp2_link_event(struct net_device *dev)
 	}
 }
 
-static void mvpp2_timer_set(struct mvpp2_port_pcpu *port_pcpu)
-{
-	ktime_t interval;
-
-	if (!port_pcpu->timer_scheduled) {
-		port_pcpu->timer_scheduled = true;
-		interval = MVPP2_TXDONE_HRTIMER_PERIOD_NS;
-		hrtimer_start(&port_pcpu->tx_done_timer, interval,
-			      HRTIMER_MODE_REL_PINNED);
-	}
-}
-
-static void mvpp2_tx_proc_cb(unsigned long data)
+static enum hrtimer_restart mvpp2_hr_timer_cb(struct hrtimer *timer)
 {
-	struct net_device *dev = (struct net_device *)data;
-	struct mvpp2_port *port = netdev_priv(dev);
-	struct mvpp2_port_pcpu *port_pcpu = this_cpu_ptr(port->pcpu);
+	struct net_device *dev;
+	struct mvpp2_port *port;
+	struct mvpp2_port_pcpu *port_pcpu;
 	unsigned int tx_todo, cause;
 
+	port_pcpu = container_of(timer, struct mvpp2_port_pcpu, tx_done_timer);
+	dev = port_pcpu->dev;
+
 	if (!netif_running(dev))
-		return;
+		return HRTIMER_NORESTART;
+
 	port_pcpu->timer_scheduled = false;
+	port = netdev_priv(dev);
 
 	/* Process all the Tx queues */
 	cause = (1 << port->ntxqs) - 1;
 	tx_todo = mvpp2_tx_done(port, cause, smp_processor_id());
 
 	/* Set the timer in case not all the packets were processed */
-	if (tx_todo)
-		mvpp2_timer_set(port_pcpu);
-}
-
-static enum hrtimer_restart mvpp2_hr_timer_cb(struct hrtimer *timer)
-{
-	struct mvpp2_port_pcpu *port_pcpu = container_of(timer,
-							 struct mvpp2_port_pcpu,
-							 tx_done_timer);
-
-	tasklet_schedule(&port_pcpu->tx_done_tasklet);
+	if (tx_todo && !port_pcpu->timer_scheduled) {
+		port_pcpu->timer_scheduled = true;
+		hrtimer_forward_now(&port_pcpu->tx_done_timer,
+				    MVPP2_TXDONE_HRTIMER_PERIOD_NS);
 
+		return HRTIMER_RESTART;
+	}
 	return HRTIMER_NORESTART;
 }
 
@@ -6698,7 +6685,12 @@ static int mvpp2_tx(struct sk_buff *skb, struct net_device *dev)
 	    txq_pcpu->count > 0) {
 		struct mvpp2_port_pcpu *port_pcpu = this_cpu_ptr(port->pcpu);
 
-		mvpp2_timer_set(port_pcpu);
+		if (!port_pcpu->timer_scheduled) {
+			port_pcpu->timer_scheduled = true;
+			hrtimer_start(&port_pcpu->tx_done_timer,
+				      MVPP2_TXDONE_HRTIMER_PERIOD_NS,
+				      HRTIMER_MODE_REL_PINNED_SOFT);
+		}
 	}
 
 	return NETDEV_TX_OK;
@@ -7127,7 +7119,6 @@ static int mvpp2_stop(struct net_device *dev)
 
 			hrtimer_cancel(&port_pcpu->tx_done_timer);
 			port_pcpu->timer_scheduled = false;
-			tasklet_kill(&port_pcpu->tx_done_tasklet);
 		}
 	}
 	mvpp2_cleanup_rxqs(port);
@@ -7918,13 +7909,10 @@ static int mvpp2_port_probe(struct platform_device *pdev,
 			port_pcpu = per_cpu_ptr(port->pcpu, cpu);
 
 			hrtimer_init(&port_pcpu->tx_done_timer, CLOCK_MONOTONIC,
-				     HRTIMER_MODE_REL_PINNED);
+				     HRTIMER_MODE_REL_PINNED_SOFT);
 			port_pcpu->tx_done_timer.function = mvpp2_hr_timer_cb;
 			port_pcpu->timer_scheduled = false;
-
-			tasklet_init(&port_pcpu->tx_done_tasklet,
-				     mvpp2_tx_proc_cb,
-				     (unsigned long)dev);
+			port_pcpu->dev = dev;
 		}
 	}
 
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 45+ messages in thread

* Re: [PATCH v3 10/36] hrtimer: Switch for loop to _ffs() evaluation
  2017-11-29 15:30 ` [PATCH v3 10/36] hrtimer: Switch for loop to _ffs() evaluation Anna-Maria Gleixner
@ 2017-11-29 16:06   ` Joe Perches
  0 siblings, 0 replies; 45+ messages in thread
From: Joe Perches @ 2017-11-29 16:06 UTC (permalink / raw)
  To: Anna-Maria Gleixner, LKML
  Cc: Thomas Gleixner, Peter Zijlstra, Ingo Molnar, keescook,
	Christoph Hellwig, John Stultz

On Wed, 2017-11-29 at 16:30 +0100, Anna-Maria Gleixner wrote:
> Looping over all clock bases to find active bits is suboptimal if not all
> bases are active.
> 
> Avoid this by converting it to a __ffs() evaluation. The functionallity is
> outsourced into an own function and is called via a macro as suggested by
> Peter Zijlstra.

style trivia: ignore at your pleasure...

> diff --git a/kernel/time/hrtimer.c b/kernel/time/hrtimer.c
[]
> @@ -448,6 +448,23 @@ static inline void debug_deactivate(struct hrtimer *timer)
>  	trace_hrtimer_cancel(timer);
>  }
>  
> +static struct hrtimer_clock_base *
> +__next_base(struct hrtimer_cpu_base *cpu_base, unsigned int *active)
> +{
> +	struct hrtimer_clock_base *base = NULL;
> +
> +	if (*active) {
> +		unsigned int idx = __ffs(*active);
> +		*active &= ~(1U << idx);
> +		base = &cpu_base->clock_base[idx];
> +	}
> +
> +	return base;
> +}

This might be nicer using an initial test and return like:

{
	unsigned int idx;

	if (!*active)
		return NULL;

	idx = __ffs(*active);
	*active &= ~(1U << idx);

	return &cpu_base->clock_base[idx];
}

^ permalink raw reply	[flat|nested] 45+ messages in thread

* Re: [PATCH v3 35/36] usb/gadget/NCM: Replace tasklet with softirq hrtimer
  2017-11-29 15:31 ` [PATCH v3 35/36] usb/gadget/NCM: " Anna-Maria Gleixner
@ 2017-12-04 13:14   ` Felipe Balbi
  2017-12-07  7:35     ` Anna-Maria Gleixner
  0 siblings, 1 reply; 45+ messages in thread
From: Felipe Balbi @ 2017-12-04 13:14 UTC (permalink / raw)
  To: Anna-Maria Gleixner, LKML
  Cc: Thomas Gleixner, Peter Zijlstra, Ingo Molnar, keescook,
	Christoph Hellwig, John Stultz, Anna-Maria Gleixner, linux-usb

[-- Attachment #1: Type: text/plain, Size: 664 bytes --]

Anna-Maria Gleixner <anna-maria@linutronix.de> writes:

> From: Thomas Gleixner <tglx@linutronix.de>
>
> The tx_tasklet tasklet is used in invoke the hrtimer (task_timer) in
> softirq context. This can be also achieved without the tasklet but
> with HRTIMER_MODE_SOFT as hrtimer mode.
>
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> Signed-off-by: Anna-Maria Gleixner <anna-maria@linutronix.de>
> Cc: Felipe Balbi <balbi@kernel.org>
> Cc: linux-usb@vger.kernel.org

This doesn't compile, so I'm assuming it depends on previous patches on
this series?

In that case:

Acked-by: Felipe Balbi <felipe.balbi@linux.intel.com>

-- 
balbi

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

^ permalink raw reply	[flat|nested] 45+ messages in thread

* Re: [PATCH v3 34/36] ALSA/dummy: Replace tasklet with softirq hrtimer
  2017-11-29 15:30   ` Anna-Maria Gleixner
@ 2017-12-06 16:27     ` Takashi Iwai
  -1 siblings, 0 replies; 45+ messages in thread
From: Takashi Iwai @ 2017-12-06 16:27 UTC (permalink / raw)
  To: Anna-Maria Gleixner
  Cc: LKML, alsa-devel, keescook, Peter Zijlstra, John Stultz,
	Thomas Gleixner, Christoph Hellwig, Jaroslav Kysela, Ingo Molnar,
	Takashi Sakamoto

On Wed, 29 Nov 2017 16:30:59 +0100,
Anna-Maria Gleixner wrote:
> 
> From: Thomas Gleixner <tglx@linutronix.de>
> 
> The tasklet is used to defer the execution of snd_pcm_period_elapsed() to
> the softirq context. Using the HRTIMER_MODE_SOFT mode invokes the timer
> callback in softirq context as well which renders the tasklet useless.
> 
> [o-takashi: avoid stall due to a call of hrtimer_cancel() on a callback
>             of hrtimer]
> 
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> Signed-off-by: Anna-Maria Gleixner <anna-maria@linutronix.de>
> Cc: alsa-devel@alsa-project.org
> Cc: Takashi Sakamoto <o-takashi@sakamocchi.jp>
> Cc: Takashi Iwai <tiwai@suse.com>
> Cc: Jaroslav Kysela <perex@perex.cz>
> Link: http://lkml.kernel.org/r/20170905161820.jtysvxtfleunbbmf@breakpoint.cc

Feel free to take my ack:
  Reviewed-by: Takashi Iwai <tiwai@suse.de>


thanks,

Takashi

^ permalink raw reply	[flat|nested] 45+ messages in thread

* Re: [PATCH v3 34/36] ALSA/dummy: Replace tasklet with softirq hrtimer
@ 2017-12-06 16:27     ` Takashi Iwai
  0 siblings, 0 replies; 45+ messages in thread
From: Takashi Iwai @ 2017-12-06 16:27 UTC (permalink / raw)
  To: Anna-Maria Gleixner
  Cc: alsa-devel, keescook, Peter Zijlstra, LKML, Ingo Molnar,
	John Stultz, Thomas Gleixner, Takashi Sakamoto,
	Christoph Hellwig

On Wed, 29 Nov 2017 16:30:59 +0100,
Anna-Maria Gleixner wrote:
> 
> From: Thomas Gleixner <tglx@linutronix.de>
> 
> The tasklet is used to defer the execution of snd_pcm_period_elapsed() to
> the softirq context. Using the HRTIMER_MODE_SOFT mode invokes the timer
> callback in softirq context as well which renders the tasklet useless.
> 
> [o-takashi: avoid stall due to a call of hrtimer_cancel() on a callback
>             of hrtimer]
> 
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> Signed-off-by: Anna-Maria Gleixner <anna-maria@linutronix.de>
> Cc: alsa-devel@alsa-project.org
> Cc: Takashi Sakamoto <o-takashi@sakamocchi.jp>
> Cc: Takashi Iwai <tiwai@suse.com>
> Cc: Jaroslav Kysela <perex@perex.cz>
> Link: http://lkml.kernel.org/r/20170905161820.jtysvxtfleunbbmf@breakpoint.cc

Feel free to take my ack:
  Reviewed-by: Takashi Iwai <tiwai@suse.de>


thanks,

Takashi

^ permalink raw reply	[flat|nested] 45+ messages in thread

* Re: [PATCH v3 35/36] usb/gadget/NCM: Replace tasklet with softirq hrtimer
  2017-12-04 13:14   ` Felipe Balbi
@ 2017-12-07  7:35     ` Anna-Maria Gleixner
  0 siblings, 0 replies; 45+ messages in thread
From: Anna-Maria Gleixner @ 2017-12-07  7:35 UTC (permalink / raw)
  To: Felipe Balbi
  Cc: LKML, Thomas Gleixner, Peter Zijlstra, Ingo Molnar, keescook,
	Christoph Hellwig, John Stultz, linux-usb

On Mon, 4 Dec 2017, Felipe Balbi wrote:

> Anna-Maria Gleixner <anna-maria@linutronix.de> writes:
> 
> > From: Thomas Gleixner <tglx@linutronix.de>
> >
> > The tx_tasklet tasklet is used in invoke the hrtimer (task_timer) in
> > softirq context. This can be also achieved without the tasklet but
> > with HRTIMER_MODE_SOFT as hrtimer mode.
> >
> > Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> > Signed-off-by: Anna-Maria Gleixner <anna-maria@linutronix.de>
> > Cc: Felipe Balbi <balbi@kernel.org>
> > Cc: linux-usb@vger.kernel.org
> 
> This doesn't compile, so I'm assuming it depends on previous patches on
> this series?

Yes, you are right. It depends on the patches for the softirq context
hrtimer rework.

> 
> In that case:
> 
> Acked-by: Felipe Balbi <felipe.balbi@linux.intel.com>
> 
> -- 
> balbi
> 

^ permalink raw reply	[flat|nested] 45+ messages in thread

* Re: [PATCH v3 31/36] mac80211_hwsim: Replace hrtimer tasklet with softirq hrtimer
  2017-11-29 15:30 ` [PATCH v3 31/36] mac80211_hwsim: Replace hrtimer tasklet with softirq hrtimer Anna-Maria Gleixner
@ 2017-12-11 11:27   ` Johannes Berg
  0 siblings, 0 replies; 45+ messages in thread
From: Johannes Berg @ 2017-12-11 11:27 UTC (permalink / raw)
  To: Anna-Maria Gleixner, LKML
  Cc: Thomas Gleixner, Peter Zijlstra, Ingo Molnar, keescook,
	Christoph Hellwig, John Stultz, linux-wireless, Kalle Valo

On Wed, 2017-11-29 at 16:30 +0100, Anna-Maria Gleixner wrote:
> From: Thomas Gleixner <tglx@linutronix.de>
> 
> Switch the timer to HRTIMER_MODE_SOFT, which executed the timer
> callback in softirq context and remove the hrtimer_tasklet.

You didn't change the commit log to HRTIMER_MODE_REL_SOFT, but
otherwise looks good to me.

Reviewed-by: Johannes Berg <johannes@sipsolutions.net

johannes

^ permalink raw reply	[flat|nested] 45+ messages in thread

* Re: [PATCH v3 28/36] hrtimer: Implement support for softirq based hrtimers
  2017-11-29 15:30 ` [PATCH v3 28/36] hrtimer: Implement support for " Anna-Maria Gleixner
@ 2017-12-14 10:47   ` Sebastian Andrzej Siewior
  0 siblings, 0 replies; 45+ messages in thread
From: Sebastian Andrzej Siewior @ 2017-12-14 10:47 UTC (permalink / raw)
  To: Anna-Maria Gleixner
  Cc: LKML, Thomas Gleixner, Peter Zijlstra, Ingo Molnar, keescook,
	Christoph Hellwig, John Stultz, bert schulze

On 2017-11-29 16:30:53 [+0100], Anna-Maria Gleixner wrote:
> diff --git a/kernel/time/hrtimer.c b/kernel/time/hrtimer.c
> index 65211fd7288d..40b14f025829 100644
> --- a/kernel/time/hrtimer.c
> +++ b/kernel/time/hrtimer.c
> @@ -711,6 +768,28 @@ static void hrtimer_reprogram(struct hrtimer *timer)
>  	WARN_ON_ONCE(hrtimer_get_expires_tv64(timer) < 0);
>  
>  	/*
> +	 * CLOCK_REALTIME timer might be requested with an absolute
> +	 * expiry time which is less than base->offset. Set it to 0.
> +	 */
> +	if (expires < 0)
> +		expires = 0;
> +
> +	if (timer->is_soft) {
> +		if (cpu_base->softirq_activated)
> +			return;
> +
> +		if (!ktime_before(expires, cpu_base->softirq_expires_next))
> +			return;
> +
> +		cpu_base->softirq_next_timer = timer;
> +		cpu_base->softirq_expires_next = expires;
> +
> +		if (!ktime_before(expires, cpu_base->expires_next) ||
> +		    !reprogram)
> +			return;
> +	}
> +
> +	/*
>  	 * If the timer is not on the current cpu, we cannot reprogram
>  	 * the other cpus clock event device.
>  	 */


if the timer is enqueud on CPUX and we run on CPUY then we have to
update the cpu_base bits of the correct CPU. Not sure if this accounts
for all the pieces but it might be okay with the check we have in
hrtimer_check_target() (that we have nothing to do but just wait).

Without this, all "sleep 1" which are invoked on CPU1 but migrated to/
programmed on CPU0 (due to timer migration wth NOHZ_FULL) won't expire
as expected.

Reported-by: bert schulze <spambemyguest@googlemail.com>
Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
---
 kernel/time/hrtimer.c | 12 +++++++-----
 1 file changed, 7 insertions(+), 5 deletions(-)

diff --git a/kernel/time/hrtimer.c b/kernel/time/hrtimer.c
index af16a5af2269..c2c344fda487 100644
--- a/kernel/time/hrtimer.c
+++ b/kernel/time/hrtimer.c
@@ -801,16 +801,18 @@ static void hrtimer_reprogram(struct hrtimer *timer, bool reprogram)
 		expires = 0;
 
 	if (timer->is_soft) {
-		if (cpu_base->softirq_activated)
+		struct hrtimer_cpu_base *timer_cpu_base = base->cpu_base;
+
+		if (timer_cpu_base->softirq_activated)
 			return;
 
-		if (!ktime_before(expires, cpu_base->softirq_expires_next))
+		if (!ktime_before(expires, timer_cpu_base->softirq_expires_next))
 			return;
 
-		cpu_base->softirq_next_timer = timer;
-		cpu_base->softirq_expires_next = expires;
+		timer_cpu_base->softirq_next_timer = timer;
+		timer_cpu_base->softirq_expires_next = expires;
 
-		if (!ktime_before(expires, cpu_base->expires_next) ||
+		if (!ktime_before(expires, timer_cpu_base->expires_next) ||
 		    !reprogram)
 			return;
 	}
-- 
2.15.1

^ permalink raw reply related	[flat|nested] 45+ messages in thread

end of thread, other threads:[~2017-12-14 10:47 UTC | newest]

Thread overview: 45+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-11-29 15:30 [PATCH 00/36] hrtimer: Provide softirq context hrtimers Anna-Maria Gleixner
2017-11-29 15:30 ` [PATCH v3 01/36] timers: Use static keys for migrate_enable/nohz_active Anna-Maria Gleixner
2017-11-29 15:30 ` [PATCH v3 02/36] hrtimer: Correct blantanly wrong comment Anna-Maria Gleixner
2017-11-29 15:30 ` [PATCH v3 03/36] hrtimer: Fix kerneldoc for struct hrtimer_cpu_base Anna-Maria Gleixner
2017-11-29 15:30 ` [PATCH v3 04/36] hrtimer: Cleanup clock argument in schedule_hrtimeout_range_clock() Anna-Maria Gleixner
2017-11-29 15:30 ` [PATCH v3 05/36] hrtimer: Fix hrtimer function description Anna-Maria Gleixner
2017-11-29 15:30 ` [PATCH v3 06/36] hrtimer: Ensure POSIX compliance (relative CLOCK_REALTIME hrtimers) Anna-Maria Gleixner
2017-11-29 15:30 ` [PATCH v3 07/36] hrtimer: Cleanup hrtimer_mode enum Anna-Maria Gleixner
2017-11-29 15:30 ` [PATCH v3 08/36] tracing/hrtimer: Take all clock bases and modes into account Anna-Maria Gleixner
2017-11-29 15:30 ` [PATCH v3 09/36] tracing/hrtimer: Print hrtimer mode in hrtimer_start tracepoint Anna-Maria Gleixner
2017-11-29 15:30 ` [PATCH v3 10/36] hrtimer: Switch for loop to _ffs() evaluation Anna-Maria Gleixner
2017-11-29 16:06   ` Joe Perches
2017-11-29 15:30 ` [PATCH v3 11/36] hrtimer: Store running timer in hrtimer_clock_base Anna-Maria Gleixner
2017-11-29 15:30 ` [PATCH v3 12/36] hrtimer: Make room in struct hrtimer_cpu_base Anna-Maria Gleixner
2017-11-29 15:30 ` [PATCH v3 13/36] hrtimer: Reduce conditional code (hres_active) Anna-Maria Gleixner
2017-11-29 15:30 ` [PATCH v3 14/36] hrtimer: Use accesor functions instead of direct access Anna-Maria Gleixner
2017-11-29 15:30 ` [PATCH v3 15/36] hrtimer: Make the remote enqueue check unconditional Anna-Maria Gleixner
2017-11-29 15:30 ` [PATCH v3 16/36] hrtimer: Make hrtimer_cpu_base.next_timer handling unconditional Anna-Maria Gleixner
2017-11-29 15:30 ` [PATCH v3 17/36] hrtimer: Make hrtimer_reprogramm() unconditional Anna-Maria Gleixner
2017-11-29 15:30 ` [PATCH v3 18/36] hrtimer: Make hrtimer_force_reprogramm() unconditionally available Anna-Maria Gleixner
2017-11-29 15:30 ` [PATCH v3 19/36] hrtimer: Unify handling of hrtimer remove Anna-Maria Gleixner
2017-11-29 15:30 ` [PATCH v3 20/36] hrtimer: Unify handling of remote enqueue Anna-Maria Gleixner
2017-11-29 15:30 ` [PATCH v3 21/36] hrtimer: Make remote enqueue decision less restrictive Anna-Maria Gleixner
2017-11-29 15:30 ` [PATCH v3 22/36] hrtimer: Remove base argument from hrtimer_reprogram() Anna-Maria Gleixner
2017-11-29 15:30 ` [PATCH v3 23/36] hrtimer: Split hrtimer_start_range_ns() Anna-Maria Gleixner
2017-11-29 15:30 ` [PATCH v3 24/36] hrtimer: Split __hrtimer_get_next_event() Anna-Maria Gleixner
2017-11-29 15:30 ` [PATCH v3 25/36] hrtimer: Use irqsave/irqrestore around __run_hrtimer() Anna-Maria Gleixner
2017-11-29 15:30 ` [PATCH v3 26/36] hrtimer: Add clock bases and hrtimer mode for soft irq context Anna-Maria Gleixner
2017-11-29 15:30 ` [PATCH v3 27/36] hrtimer: Prepare handling of hard and softirq based hrtimers Anna-Maria Gleixner
2017-11-29 15:30 ` [PATCH v3 28/36] hrtimer: Implement support for " Anna-Maria Gleixner
2017-12-14 10:47   ` Sebastian Andrzej Siewior
2017-11-29 15:30 ` [PATCH v3 29/36] hrtimer: Implement SOFT/HARD clock base selection Anna-Maria Gleixner
2017-11-29 15:30 ` [PATCH v3 30/36] can/bcm: Replace hrtimer_tasklet with softirq based hrtimer Anna-Maria Gleixner
2017-11-29 15:30 ` [PATCH v3 31/36] mac80211_hwsim: Replace hrtimer tasklet with softirq hrtimer Anna-Maria Gleixner
2017-12-11 11:27   ` Johannes Berg
2017-11-29 15:30 ` [PATCH v3 32/36] xfrm: " Anna-Maria Gleixner
2017-11-29 15:30 ` [PATCH v3 33/36] softirq: Remove tasklet_hrtimer Anna-Maria Gleixner
2017-11-29 15:30 ` [PATCH v3 34/36] ALSA/dummy: Replace tasklet with softirq hrtimer Anna-Maria Gleixner
2017-11-29 15:30   ` Anna-Maria Gleixner
2017-12-06 16:27   ` Takashi Iwai
2017-12-06 16:27     ` Takashi Iwai
2017-11-29 15:31 ` [PATCH v3 35/36] usb/gadget/NCM: " Anna-Maria Gleixner
2017-12-04 13:14   ` Felipe Balbi
2017-12-07  7:35     ` Anna-Maria Gleixner
2017-11-29 15:31 ` [PATCH v3 36/36] net/mvpp2: " Anna-Maria Gleixner

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.