[PATCH 00/25] hrtimer: Provide softirq context hrtimers

[PATCH 00/25] hrtimer: Provide softirq context hrtimers

[Anna-Maria Gleixner]

There are quite some places in the kernel which use a combination of
hrtimers and tasklets to make use of the precise expiry of hrtimers, which
schedule a tasklet to bring the actual function into softirq context.

This was introduced when the previous hrtimer softirq code was
removed. That code was implemented by expiring the timer in hard irq
context and then deferring the execution of the callback into softirq
context. That caused a lot of pointless shuffling between the rbtree and a
linked list.

In recent discussions it turned out that more potential users of hrtimers
in softirq context might come up. Aside of that the RT patches need this
functionality as well to defer hrtimers into softirq context if their
callbacks are not interrupt safe on RT.

This series implements a new approach by adding SOFT_* clock ids and
instead of doing the list shuffle, timers started with these clock ids are
put into separate soft expiry hrtimer queues. These queues are evaluated
only when the hardirq context detects that the first expiring timer in the
softirq queues has expired. That makes the overhead in the hardirq context
minimal.

The series reworks the code to reuse as much as possible from the existing
facilities for the new softirq hrtimers and integrates them with all
flavours of hrtimers (HIGH_RES=y/n - NOHZ=y/n).

To achieve this quite some of the conditionals in the existing code are
removed for the price of adding some pointless data and state tracking to
the HIGH_RES=n case. That's minimal, but well worth it as it increases the
readability and maintainability of the code.

The first part of the series implements the new functionality and the
second part converts the hrtimer/tasklet users to make use of it and
removes struct hrtimer_tasklet and the surrounding helper functions.

This series is available from git as well:

    git://git.kernel.org/pub/scm/linux/kernel/git/bigeasy/linux-hrtimer.git soft_hrtimer
    https://git.kernel.org/bigeasy/linux-hrtimer/h/soft_hrtimer


Thanks,

	Anna-Maria

---
 drivers/net/usb/cdc_ncm.c             |   37 +-
 drivers/net/wireless/mac80211_hwsim.c |   44 +-
 drivers/usb/gadget/function/f_ncm.c   |   28 -
 include/linux/hrtimer.h               |   76 ++---
 include/linux/interrupt.h             |   25 -
 include/linux/usb/cdc_ncm.h           |    2 
 include/net/xfrm.h                    |    2 
 kernel/softirq.c                      |   51 ---
 kernel/time/hrtimer.c                 |  513 +++++++++++++++++++++-------------
 net/can/bcm.c                         |  150 +++------
 net/xfrm/xfrm_state.c                 |   29 +
 sound/drivers/dummy.c                 |   16 -
 12 files changed, 484 insertions(+), 489 deletions(-)

[PATCH 01/25] hrtimer: Use predefined function for updating next_timer

[Anna-Maria Gleixner]

[-- Attachment #1: hrtimer_Use_predefined_function_for_updating_next_timer.patch --]
[-- Type: text/plain, Size: 623 bytes --]

From: Anna-Maria Gleixner <anna-maria@linutronix.de>

There already exist a function for updating the next_timer
hrtimer_update_next_timer().

Signed-off-by: Anna-Maria Gleixner <anna-maria@linutronix.de>
---
 kernel/time/hrtimer.c |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

--- a/kernel/time/hrtimer.c
+++ b/kernel/time/hrtimer.c
@@ -630,7 +630,7 @@ static void hrtimer_reprogram(struct hrt
 		return;
 
 	/* Update the pointer to the next expiring timer */
-	cpu_base->next_timer = timer;
+	hrtimer_update_next_timer(cpu_base, timer);
 
 	/*
 	 * If a hang was detected in the last timer interrupt then we

[PATCH 02/25] hrtimer: Correct blantanly wrong comment

[Anna-Maria Gleixner]

[-- Attachment #1: hrtimer_Correct_blantanly_wrong_comment.patch --]
[-- Type: text/plain, Size: 1297 bytes --]

From: Thomas Gleixner <tglx@linutronix.de>

The protection of a hrtimer which runs its callback against migration to a
different CPU has nothing to do with hard interrupt context.

The protection against migration of a hrtimer running the expiry callback
is the pointer in the cpu_base which holds a pointer to the currently
running timer. This pointer is evaluated in the code which potentially
switches the timer base and makes sure it's kept on the CPU on which the
callback is running.

Reported-by: Anna-Maria Gleixner <anna-maria@linutronix.de>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Anna-Maria Gleixner <anna-maria@linutronix.de>
---
 kernel/time/hrtimer.c |    6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

--- a/kernel/time/hrtimer.c
+++ b/kernel/time/hrtimer.c
@@ -1204,9 +1204,9 @@ static void __run_hrtimer(struct hrtimer
 		timer->is_rel = false;
 
 	/*
-	 * Because we run timers from hardirq context, there is no chance
-	 * they get migrated to another cpu, therefore its safe to unlock
-	 * the timer base.
+	 * The timer is marked as running in the cpu base, so it is
+	 * protected against migration to a different CPU even if the lock
+	 * is dropped.
 	 */
 	raw_spin_unlock(&cpu_base->lock);
 	trace_hrtimer_expire_entry(timer, now);

[PATCH 03/25] hrtimer: Fix kerneldoc for struct hrtimer_cpu_base

[Anna-Maria Gleixner]

[-- Attachment #1: hrtimer_Fix_kerneldoc_for_struct_hrtimer_cpu_base.patch --]
[-- Type: text/plain, Size: 1617 bytes --]

From: Anna-Maria Gleixner <anna-maria@linutronix.de>

The sequence '/**' marks the start of a struct description. Add the
missing second asterisk. While at it adapt the ordering of the struct
members to the struct definition and document the purpose of
expires_next more precisely.

Signed-off-by: Anna-Maria Gleixner <anna-maria@linutronix.de>
---
 include/linux/hrtimer.h |    8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

--- a/include/linux/hrtimer.h
+++ b/include/linux/hrtimer.h
@@ -144,7 +144,7 @@ enum  hrtimer_base_type {
 	HRTIMER_MAX_CLOCK_BASES,
 };
 
-/*
+/**
  * struct hrtimer_cpu_base - the per cpu clock bases
  * @lock:		lock protecting the base and associated clock bases
  *			and timers
@@ -155,12 +155,12 @@ enum  hrtimer_base_type {
  * @clock_was_set_seq:	Sequence counter of clock was set events
  * @migration_enabled:	The migration of hrtimers to other cpus is enabled
  * @nohz_active:	The nohz functionality is enabled
- * @expires_next:	absolute time of the next event which was scheduled
- *			via clock_set_next_event()
- * @next_timer:		Pointer to the first expiring timer
  * @in_hrtirq:		hrtimer_interrupt() is currently executing
  * @hres_active:	State of high resolution mode
  * @hang_detected:	The last hrtimer interrupt detected a hang
+ * @expires_next:	absolute time of the next event, is required for remote
+ *			hrtimer enqueue
+ * @next_timer:		Pointer to the first expiring timer
  * @nr_events:		Total number of hrtimer interrupt events
  * @nr_retries:		Total number of hrtimer interrupt retries
  * @nr_hangs:		Total number of hrtimer interrupt hangs

[PATCH 04/25] hrtimer: Cleanup clock argument in schedule_hrtimeout_range_clock()

[Anna-Maria Gleixner]

[-- Attachment #1: hrtimer_Cleanup_clock_argument_in_schedule_hrtimeout_range_clock.patch --]
[-- Type: text/plain, Size: 2050 bytes --]

From: Anna-Maria Gleixner <anna-maria@linutronix.de>

schedule_hrtimeout_range_clock() uses an integer for the clock id
instead of the predefined type "clockid_t". The ID of the clock is
indicated in hrtimer code as clock_id. Therefore change the name of
the variable as well to make it consistent.

While at it, clean up the description for the function parameters clock_id
and mode. The clock modes and the clock ids are not restricted as the
comment suggests.

No functional change.

Signed-off-by: Anna-Maria Gleixner <anna-maria@linutronix.de>
---
 include/linux/hrtimer.h |    2 +-
 kernel/time/hrtimer.c   |    8 ++++----
 2 files changed, 5 insertions(+), 5 deletions(-)

--- a/include/linux/hrtimer.h
+++ b/include/linux/hrtimer.h
@@ -466,7 +466,7 @@ extern int schedule_hrtimeout_range(ktim
 extern int schedule_hrtimeout_range_clock(ktime_t *expires,
 					  u64 delta,
 					  const enum hrtimer_mode mode,
-					  int clock);
+					  clockid_t clock_id);
 extern int schedule_hrtimeout(ktime_t *expires, const enum hrtimer_mode mode);
 
 /* Soft interrupt function to run the hrtimer queues: */
--- a/kernel/time/hrtimer.c
+++ b/kernel/time/hrtimer.c
@@ -1671,12 +1671,12 @@ void __init hrtimers_init(void)
  * schedule_hrtimeout_range_clock - sleep until timeout
  * @expires:	timeout value (ktime_t)
  * @delta:	slack in expires timeout (ktime_t)
- * @mode:	timer mode, HRTIMER_MODE_ABS or HRTIMER_MODE_REL
- * @clock:	timer clock, CLOCK_MONOTONIC or CLOCK_REALTIME
+ * @mode:	timer mode
+ * @clock_id:	timer clock to be used
  */
 int __sched
 schedule_hrtimeout_range_clock(ktime_t *expires, u64 delta,
-			       const enum hrtimer_mode mode, int clock)
+			       const enum hrtimer_mode mode, clockid_t clock_id)
 {
 	struct hrtimer_sleeper t;
 
@@ -1697,7 +1697,7 @@ schedule_hrtimeout_range_clock(ktime_t *
 		return -EINTR;
 	}
 
-	hrtimer_init_on_stack(&t.timer, clock, mode);
+	hrtimer_init_on_stack(&t.timer, clock_id, mode);
 	hrtimer_set_expires_range_ns(&t.timer, *expires, delta);
 
 	hrtimer_init_sleeper(&t, current);

[PATCH 05/25] hrtimer: Switch for loop to _ffs() evaluation

[Anna-Maria Gleixner]

[-- Attachment #1: hrtimer_Switch_for_loop_to__ffs_evaluation.patch --]
[-- Type: text/plain, Size: 1756 bytes --]

From: Anna-Maria Gleixner <anna-maria@linutronix.de>

Looping over all clock bases to find active bits is suboptimal if not all
bases are active.

Avoid this by converting it to a __ffs() evaluation.

Signed-off-by: Anna-Maria Gleixner <anna-maria@linutronix.de>
---
 kernel/time/hrtimer.c |   18 ++++++++++--------
 1 file changed, 10 insertions(+), 8 deletions(-)

--- a/kernel/time/hrtimer.c
+++ b/kernel/time/hrtimer.c
@@ -465,17 +465,18 @@ static inline void hrtimer_update_next_t
 
 static ktime_t __hrtimer_get_next_event(struct hrtimer_cpu_base *cpu_base)
 {
-	struct hrtimer_clock_base *base = cpu_base->clock_base;
 	unsigned int active = cpu_base->active_bases;
 	ktime_t expires, expires_next = KTIME_MAX;
 
 	hrtimer_update_next_timer(cpu_base, NULL);
-	for (; active; base++, active >>= 1) {
+	while (active) {
+		unsigned int id = __ffs(active);
+		struct hrtimer_clock_base *base;
 		struct timerqueue_node *next;
 		struct hrtimer *timer;
 
-		if (!(active & 0x01))
-			continue;
+		active &= ~(1U << id);
+		base = cpu_base->clock_base + id;
 
 		next = timerqueue_getnext(&base->active);
 		timer = container_of(next, struct hrtimer, node);
@@ -1242,15 +1243,16 @@ static void __run_hrtimer(struct hrtimer
 
 static void __hrtimer_run_queues(struct hrtimer_cpu_base *cpu_base, ktime_t now)
 {
-	struct hrtimer_clock_base *base = cpu_base->clock_base;
 	unsigned int active = cpu_base->active_bases;
 
-	for (; active; base++, active >>= 1) {
+	while (active) {
+		unsigned int id = __ffs(active);
+		struct hrtimer_clock_base *base;
 		struct timerqueue_node *node;
 		ktime_t basenow;
 
-		if (!(active & 0x01))
-			continue;
+		active &= ~(1U << id);
+		base = cpu_base->clock_base + id;
 
 		basenow = ktime_add(now, base->offset);
 

[PATCH 07/25] hrtimer: Reduce conditional code (hres_active)

[Anna-Maria Gleixner]

[-- Attachment #1: hrtimer--make-hrtimer_hres_active---available-in-all-cases.patch --]
[-- Type: text/plain, Size: 3583 bytes --]

The hrtimer_cpu_base struct has the CONFIG_HIGH_RES_TIMERS conditional
struct member hres_active. All related functions to this member are
conditional as well.

There is no functional change, when the hres_active member is unconditional
with all related functions and is set to zero during initialization. This
makes the code easier to read.

Signed-off-by: Anna-Maria Gleixner <anna-maria@linutronix.de>
---
 include/linux/hrtimer.h |   17 ++++++-----------
 kernel/time/hrtimer.c   |   30 ++++++++++++++----------------
 2 files changed, 20 insertions(+), 27 deletions(-)

--- a/include/linux/hrtimer.h
+++ b/include/linux/hrtimer.h
@@ -180,9 +180,9 @@ struct hrtimer_cpu_base {
 	unsigned int			clock_was_set_seq;
 	bool				migration_enabled;
 	bool				nohz_active;
+	unsigned int			hres_active	: 1;
 #ifdef CONFIG_HIGH_RES_TIMERS
 	unsigned int			in_hrtirq	: 1,
-					hres_active	: 1,
 					hang_detected	: 1;
 	ktime_t				expires_next;
 	struct hrtimer			*next_timer;
@@ -264,16 +264,16 @@ static inline ktime_t hrtimer_cb_get_tim
 	return timer->base->get_time();
 }
 
-#ifdef CONFIG_HIGH_RES_TIMERS
-struct clock_event_device;
-
-extern void hrtimer_interrupt(struct clock_event_device *dev);
-
 static inline int hrtimer_is_hres_active(struct hrtimer *timer)
 {
 	return timer->base->cpu_base->hres_active;
 }
 
+#ifdef CONFIG_HIGH_RES_TIMERS
+struct clock_event_device;
+
+extern void hrtimer_interrupt(struct clock_event_device *dev);
+
 /*
  * The resolution of the clocks. The resolution value is returned in
  * the clock_getres() system call to give application programmers an
@@ -296,11 +296,6 @@ extern unsigned int hrtimer_resolution;
 
 #define hrtimer_resolution	(unsigned int)LOW_RES_NSEC
 
-static inline int hrtimer_is_hres_active(struct hrtimer *timer)
-{
-	return 0;
-}
-
 static inline void clock_was_set_delayed(void) { }
 
 #endif
--- a/kernel/time/hrtimer.c
+++ b/kernel/time/hrtimer.c
@@ -505,6 +505,19 @@ static inline ktime_t hrtimer_update_bas
 					    offs_real, offs_boot, offs_tai);
 }
 
+/*
+ * Is the high resolution mode active ?
+ */
+static inline int __hrtimer_hres_active(struct hrtimer_cpu_base *cpu_base)
+{
+	return cpu_base->hres_active;
+}
+
+static inline int hrtimer_hres_active(void)
+{
+	return __hrtimer_hres_active(this_cpu_ptr(&hrtimer_bases));
+}
+
 /* High resolution timer related functions */
 #ifdef CONFIG_HIGH_RES_TIMERS
 
@@ -534,19 +547,6 @@ static inline int hrtimer_is_hres_enable
 }
 
 /*
- * Is the high resolution mode active ?
- */
-static inline int __hrtimer_hres_active(struct hrtimer_cpu_base *cpu_base)
-{
-	return cpu_base->hres_active;
-}
-
-static inline int hrtimer_hres_active(void)
-{
-	return __hrtimer_hres_active(this_cpu_ptr(&hrtimer_bases));
-}
-
-/*
  * Reprogram the event source with checking both queues for the
  * next event
  * Called with interrupts disabled and base->lock held
@@ -654,7 +654,6 @@ static void hrtimer_reprogram(struct hrt
 static inline void hrtimer_init_hres(struct hrtimer_cpu_base *base)
 {
 	base->expires_next = KTIME_MAX;
-	base->hres_active = 0;
 }
 
 /*
@@ -713,8 +712,6 @@ void clock_was_set_delayed(void)
 
 #else
 
-static inline int __hrtimer_hres_active(struct hrtimer_cpu_base *b) { return 0; }
-static inline int hrtimer_hres_active(void) { return 0; }
 static inline int hrtimer_is_hres_enabled(void) { return 0; }
 static inline void hrtimer_switch_to_hres(void) { }
 static inline void
@@ -1592,6 +1589,7 @@ int hrtimers_prepare_cpu(unsigned int cp
 	}
 
 	cpu_base->cpu = cpu;
+	cpu_base->hres_active = 0;
 	hrtimer_init_hres(cpu_base);
 	return 0;
 }

Re: [PATCH 07/25] hrtimer: Reduce conditional code (hres_active)

[Peter Zijlstra]

On Thu, Aug 31, 2017 at 12:23:36PM -0000, Anna-Maria Gleixner wrote:
> The hrtimer_cpu_base struct has the CONFIG_HIGH_RES_TIMERS conditional
> struct member hres_active. All related functions to this member are
> conditional as well.
> 
> There is no functional change, when the hres_active member is unconditional
> with all related functions and is set to zero during initialization. This
> makes the code easier to read.

But you make what was a compile time dead-code-elimination into a
runtime load-and-branch.

Unless the compiler is overly clever and is able to deduce from the lack
of assignments that it must always be 0, is it?

Re: [PATCH 07/25] hrtimer: Reduce conditional code (hres_active)

[Thomas Gleixner]

On Mon, 25 Sep 2017, Peter Zijlstra wrote:
> On Thu, Aug 31, 2017 at 12:23:36PM -0000, Anna-Maria Gleixner wrote:
> > The hrtimer_cpu_base struct has the CONFIG_HIGH_RES_TIMERS conditional
> > struct member hres_active. All related functions to this member are
> > conditional as well.
> > 
> > There is no functional change, when the hres_active member is unconditional
> > with all related functions and is set to zero during initialization. This
> > makes the code easier to read.
> 
> But you make what was a compile time dead-code-elimination into a
> runtime load-and-branch.
> 
> Unless the compiler is overly clever and is able to deduce from the lack
> of assignments that it must always be 0, is it?

Probably not. So the simple way to solve this is to make the new common

is_highres_active()
{
	return IS_ENABLED(CONFIG_HIGH_RES) ? base->hres_active : 0;
}

Thanks,

	tglx

[PATCH 06/25] hrtimer: Store running timer in hrtimer_clock_base

[Anna-Maria Gleixner]

[-- Attachment #1: hrtimer_Store_running_timer_in_hrtimer_clock_base.patch --]
[-- Type: text/plain, Size: 6338 bytes --]

From: Anna-Maria Gleixner <anna-maria@linutronix.de>

The pointer to the currently running timer is stored in hrtimer_cpu_base
before the base lock is dropped and the callback is invoked.

This results in two levels of indirections and the upcoming support for
softirq based hrtimer requires splitting the "running" storage into soft
and hard irq context expiry.

Storing both in the cpu base would require conditionals in all code paths
accessing that information.

It's possible to have a per clock base sequence count and running pointer
without changing the semantics of the related mechanisms because the timer
base pointer cannot be changed while a timer is running the callback.

Unfortunately this makes cpu_clock base larger than 32 bytes on 32bit
kernels. Instead of having huge gaps due to alignment, remove the alignment
and let the compiler pack cpu base for 32bit. The resulting cache access
patterns are fortunately not really different from the current
behaviour. On 64bit kernels the 64byte alignment stays and the behaviour is
unchanged.

Signed-off-by: Anna-Maria Gleixner <anna-maria@linutronix.de>
---
 include/linux/hrtimer.h |   20 +++++++++-----------
 kernel/time/hrtimer.c   |   28 +++++++++++++---------------
 2 files changed, 22 insertions(+), 26 deletions(-)

--- a/include/linux/hrtimer.h
+++ b/include/linux/hrtimer.h
@@ -112,9 +112,9 @@ struct hrtimer_sleeper {
 };
 
 #ifdef CONFIG_64BIT
-# define HRTIMER_CLOCK_BASE_ALIGN	64
+# define __hrtimer_clock_base_align	____cacheline_aligned
 #else
-# define HRTIMER_CLOCK_BASE_ALIGN	32
+# define __hrtimer_clock_base_align
 #endif
 
 /**
@@ -123,18 +123,22 @@ struct hrtimer_sleeper {
  * @index:		clock type index for per_cpu support when moving a
  *			timer to a base on another cpu.
  * @clockid:		clock id for per_cpu support
+ * @seq:		seqcount around __run_hrtimer
+ * @running:		pointer to the currently running hrtimer
  * @active:		red black tree root node for the active timers
  * @get_time:		function to retrieve the current time of the clock
  * @offset:		offset of this clock to the monotonic base
  */
 struct hrtimer_clock_base {
 	struct hrtimer_cpu_base	*cpu_base;
-	int			index;
+	unsigned int		index;
 	clockid_t		clockid;
+	seqcount_t		seq;
+	struct hrtimer		*running;
 	struct timerqueue_head	active;
 	ktime_t			(*get_time)(void);
 	ktime_t			offset;
-} __attribute__((__aligned__(HRTIMER_CLOCK_BASE_ALIGN)));
+} __hrtimer_clock_base_align;
 
 enum  hrtimer_base_type {
 	HRTIMER_BASE_MONOTONIC,
@@ -148,8 +152,6 @@ enum  hrtimer_base_type {
  * struct hrtimer_cpu_base - the per cpu clock bases
  * @lock:		lock protecting the base and associated clock bases
  *			and timers
- * @seq:		seqcount around __run_hrtimer
- * @running:		pointer to the currently running hrtimer
  * @cpu:		cpu number
  * @active_bases:	Bitfield to mark bases with active timers
  * @clock_was_set_seq:	Sequence counter of clock was set events
@@ -173,8 +175,6 @@ enum  hrtimer_base_type {
  */
 struct hrtimer_cpu_base {
 	raw_spinlock_t			lock;
-	seqcount_t			seq;
-	struct hrtimer			*running;
 	unsigned int			cpu;
 	unsigned int			active_bases;
 	unsigned int			clock_was_set_seq;
@@ -196,8 +196,6 @@ struct hrtimer_cpu_base {
 
 static inline void hrtimer_set_expires(struct hrtimer *timer, ktime_t time)
 {
-	BUILD_BUG_ON(sizeof(struct hrtimer_clock_base) > HRTIMER_CLOCK_BASE_ALIGN);
-
 	timer->node.expires = time;
 	timer->_softexpires = time;
 }
@@ -422,7 +420,7 @@ static inline int hrtimer_is_queued(stru
  */
 static inline int hrtimer_callback_running(struct hrtimer *timer)
 {
-	return timer->base->cpu_base->running == timer;
+	return timer->base->running == timer;
 }
 
 /* Forward a hrtimer so it expires after now: */
--- a/kernel/time/hrtimer.c
+++ b/kernel/time/hrtimer.c
@@ -70,7 +70,6 @@
 DEFINE_PER_CPU(struct hrtimer_cpu_base, hrtimer_bases) =
 {
 	.lock = __RAW_SPIN_LOCK_UNLOCKED(hrtimer_bases.lock),
-	.seq = SEQCNT_ZERO(hrtimer_bases.seq),
 	.clock_base =
 	{
 		{
@@ -118,7 +117,6 @@ static const int hrtimer_clock_to_base_t
  * timer->base->cpu_base
  */
 static struct hrtimer_cpu_base migration_cpu_base = {
-	.seq = SEQCNT_ZERO(migration_cpu_base),
 	.clock_base = { { .cpu_base = &migration_cpu_base, }, },
 };
 
@@ -1136,19 +1134,19 @@ EXPORT_SYMBOL_GPL(hrtimer_init);
  */
 bool hrtimer_active(const struct hrtimer *timer)
 {
-	struct hrtimer_cpu_base *cpu_base;
+	struct hrtimer_clock_base *base;
 	unsigned int seq;
 
 	do {
-		cpu_base = READ_ONCE(timer->base->cpu_base);
-		seq = raw_read_seqcount_begin(&cpu_base->seq);
+		base = READ_ONCE(timer->base);
+		seq = raw_read_seqcount_begin(&base->seq);
 
 		if (timer->state != HRTIMER_STATE_INACTIVE ||
-		    cpu_base->running == timer)
+		    base->running == timer)
 			return true;
 
-	} while (read_seqcount_retry(&cpu_base->seq, seq) ||
-		 cpu_base != READ_ONCE(timer->base->cpu_base));
+	} while (read_seqcount_retry(&base->seq, seq) ||
+		 base != READ_ONCE(timer->base));
 
 	return false;
 }
@@ -1182,16 +1180,16 @@ static void __run_hrtimer(struct hrtimer
 	lockdep_assert_held(&cpu_base->lock);
 
 	debug_deactivate(timer);
-	cpu_base->running = timer;
+	base->running = timer;
 
 	/*
 	 * Separate the ->running assignment from the ->state assignment.
 	 *
 	 * As with a regular write barrier, this ensures the read side in
-	 * hrtimer_active() cannot observe cpu_base->running == NULL &&
+	 * hrtimer_active() cannot observe base->running == NULL &&
 	 * timer->state == INACTIVE.
 	 */
-	raw_write_seqcount_barrier(&cpu_base->seq);
+	raw_write_seqcount_barrier(&base->seq);
 
 	__remove_hrtimer(timer, base, HRTIMER_STATE_INACTIVE, 0);
 	fn = timer->function;
@@ -1232,13 +1230,13 @@ static void __run_hrtimer(struct hrtimer
 	 * Separate the ->running assignment from the ->state assignment.
 	 *
 	 * As with a regular write barrier, this ensures the read side in
-	 * hrtimer_active() cannot observe cpu_base->running == NULL &&
+	 * hrtimer_active() cannot observe base->running.timer == NULL &&
 	 * timer->state == INACTIVE.
 	 */
-	raw_write_seqcount_barrier(&cpu_base->seq);
+	raw_write_seqcount_barrier(&base->seq);
 
-	WARN_ON_ONCE(cpu_base->running != timer);
-	cpu_base->running = NULL;
+	WARN_ON_ONCE(base->running != timer);
+	base->running = NULL;
 }
 
 static void __hrtimer_run_queues(struct hrtimer_cpu_base *cpu_base, ktime_t now)

Re: [PATCH 06/25] hrtimer: Store running timer in hrtimer_clock_base

[Peter Zijlstra]

On Thu, Aug 31, 2017 at 12:23:36PM -0000, Anna-Maria Gleixner wrote:
> Unfortunately this makes cpu_clock base larger than 32 bytes on 32bit
> kernels. Instead of having huge gaps due to alignment, remove the alignment
> and let the compiler pack cpu base for 32bit. The resulting cache access
> patterns are fortunately not really different from the current
> behaviour.

How did you determine this? Perf lacks much useful tooling right about
there :/

Re: [PATCH 06/25] hrtimer: Store running timer in hrtimer_clock_base

[Thomas Gleixner]

On Mon, 25 Sep 2017, Peter Zijlstra wrote:

> On Thu, Aug 31, 2017 at 12:23:36PM -0000, Anna-Maria Gleixner wrote:
> > Unfortunately this makes cpu_clock base larger than 32 bytes on 32bit
> > kernels. Instead of having huge gaps due to alignment, remove the alignment
> > and let the compiler pack cpu base for 32bit. The resulting cache access
> > patterns are fortunately not really different from the current
> > behaviour.
> 
> How did you determine this? Perf lacks much useful tooling right about
> there :/

Just by analyzing the resulting layout and looking at the number of cache
lines involved for the frequently used clocks (mono/real).

Thanks,

	tglx

[PATCH 08/25] hrtimer: Reduce conditional code (expires_next, next_timer)

[Anna-Maria Gleixner]

[-- Attachment #1: hrtimer--Make-expires_next-available.patch --]
[-- Type: text/plain, Size: 4093 bytes --]

The hrtimer_cpu_base struct member expires_next and next_timer are
conditional members (CONFIG_HIGH_RES_TIMERS). This makes the hrtimer code
more complex and harder to understand than it actually is.

Reduce the conditionals related to those two struct members.

Signed-off-by: Anna-Maria Gleixner <anna-maria@linutronix.de>
---
 include/linux/hrtimer.h |   10 +++++-----
 kernel/time/hrtimer.c   |   24 +++++-------------------
 2 files changed, 10 insertions(+), 24 deletions(-)

--- a/include/linux/hrtimer.h
+++ b/include/linux/hrtimer.h
@@ -160,13 +160,13 @@ enum  hrtimer_base_type {
  * @in_hrtirq:		hrtimer_interrupt() is currently executing
  * @hres_active:	State of high resolution mode
  * @hang_detected:	The last hrtimer interrupt detected a hang
- * @expires_next:	absolute time of the next event, is required for remote
- *			hrtimer enqueue
- * @next_timer:		Pointer to the first expiring timer
  * @nr_events:		Total number of hrtimer interrupt events
  * @nr_retries:		Total number of hrtimer interrupt retries
  * @nr_hangs:		Total number of hrtimer interrupt hangs
  * @max_hang_time:	Maximum time spent in hrtimer_interrupt
+ * @expires_next:	absolute time of the next event, is required for remote
+ *			hrtimer enqueue
+ * @next_timer:		Pointer to the first expiring timer
  * @clock_base:		array of clock bases for this cpu
  *
  * Note: next_timer is just an optimization for __remove_hrtimer().
@@ -184,13 +184,13 @@ struct hrtimer_cpu_base {
 #ifdef CONFIG_HIGH_RES_TIMERS
 	unsigned int			in_hrtirq	: 1,
 					hang_detected	: 1;
-	ktime_t				expires_next;
-	struct hrtimer			*next_timer;
 	unsigned int			nr_events;
 	unsigned int			nr_retries;
 	unsigned int			nr_hangs;
 	unsigned int			max_hang_time;
 #endif
+	ktime_t				expires_next;
+	struct hrtimer			*next_timer;
 	struct hrtimer_clock_base	clock_base[HRTIMER_MAX_CLOCK_BASES];
 } ____cacheline_aligned;
 
--- a/kernel/time/hrtimer.c
+++ b/kernel/time/hrtimer.c
@@ -154,16 +154,16 @@ struct hrtimer_clock_base *lock_hrtimer_
 }
 
 /*
- * With HIGHRES=y we do not migrate the timer when it is expiring
- * before the next event on the target cpu because we cannot reprogram
- * the target cpu hardware and we would cause it to fire late.
+ * With high resolution timers enabled we do not migrate the timer
+ * when it is expiring before the next event on the target cpu because
+ * we cannot reprogram the target cpu hardware and we would cause it
+ * to fire late.
  *
  * Called with cpu_base->lock of target cpu held.
  */
 static int
 hrtimer_check_target(struct hrtimer *timer, struct hrtimer_clock_base *new_base)
 {
-#ifdef CONFIG_HIGH_RES_TIMERS
 	ktime_t expires;
 
 	if (!new_base->cpu_base->hres_active)
@@ -171,9 +171,6 @@ hrtimer_check_target(struct hrtimer *tim
 
 	expires = ktime_sub(hrtimer_get_expires(timer), new_base->offset);
 	return expires <= new_base->cpu_base->expires_next;
-#else
-	return 0;
-#endif
 }
 
 #ifdef CONFIG_NO_HZ_COMMON
@@ -456,9 +453,7 @@ static inline void debug_deactivate(stru
 static inline void hrtimer_update_next_timer(struct hrtimer_cpu_base *cpu_base,
 					     struct hrtimer *timer)
 {
-#ifdef CONFIG_HIGH_RES_TIMERS
 	cpu_base->next_timer = timer;
-#endif
 }
 
 static ktime_t __hrtimer_get_next_event(struct hrtimer_cpu_base *cpu_base)
@@ -649,14 +644,6 @@ static void hrtimer_reprogram(struct hrt
 }
 
 /*
- * Initialize the high resolution related parts of cpu_base
- */
-static inline void hrtimer_init_hres(struct hrtimer_cpu_base *base)
-{
-	base->expires_next = KTIME_MAX;
-}
-
-/*
  * Retrigger next event is called after clock was set
  *
  * Called with interrupts disabled via on_each_cpu()
@@ -721,7 +708,6 @@ static inline int hrtimer_reprogram(stru
 {
 	return 0;
 }
-static inline void hrtimer_init_hres(struct hrtimer_cpu_base *base) { }
 static inline void retrigger_next_event(void *arg) { }
 
 #endif /* CONFIG_HIGH_RES_TIMERS */
@@ -1590,7 +1576,7 @@ int hrtimers_prepare_cpu(unsigned int cp
 
 	cpu_base->cpu = cpu;
 	cpu_base->hres_active = 0;
-	hrtimer_init_hres(cpu_base);
+	cpu_base->expires_next = KTIME_MAX;
 	return 0;
 }
 

Re: [PATCH 08/25] hrtimer: Reduce conditional code (expires_next, next_timer)

[Peter Zijlstra]

On Thu, Aug 31, 2017 at 12:23:37PM -0000, Anna-Maria Gleixner wrote:
> The hrtimer_cpu_base struct member expires_next and next_timer are
> conditional members (CONFIG_HIGH_RES_TIMERS). This makes the hrtimer code
> more complex and harder to understand than it actually is.
> 
> Reduce the conditionals related to those two struct members.

So this grows the data structure for !HIGH_RES for easier code? We no
longer care about archs that don't support HIGH_RES?

Re: [PATCH 08/25] hrtimer: Reduce conditional code (expires_next, next_timer)

[Thomas Gleixner]

On Tue, 26 Sep 2017, Peter Zijlstra wrote:

> On Thu, Aug 31, 2017 at 12:23:37PM -0000, Anna-Maria Gleixner wrote:
> > The hrtimer_cpu_base struct member expires_next and next_timer are
> > conditional members (CONFIG_HIGH_RES_TIMERS). This makes the hrtimer code
> > more complex and harder to understand than it actually is.
> > 
> > Reduce the conditionals related to those two struct members.
> 
> So this grows the data structure for !HIGH_RES for easier code? We no
> longer care about archs that don't support HIGH_RES?

We care, but there is a tradeoff between the required conditional code and
the marginal storage overhead. Easier to follow code which does a few
superflous things for the !highres case is certainly preferred.

Thanks,

	tglx

[PATCH 09/25] hrtimer: Reduce conditional code (hrtimer_reprogram())

[Anna-Maria Gleixner]

[-- Attachment #1: hrtimer--reprogramming.patch --]
[-- Type: text/plain, Size: 6103 bytes --]

The hrtimer_reprogram() is currently required only when
CONFIG_HIGH_RES_TIMERS is set. Additional bitfields of hrtimer_cpu_base
struct are high resolution timer specific as well.

To simplify the hrtimer code, the behaviour of CONFIG_HIGH_RES_TIMERS and
!CONFIG_HIGH_RES_TIMERS should be similar. As preparation for this, the
function hrtimer_reprogram() and required hrtimer_cpu_base struct members
are moved outside the conditional area.

Signed-off-by: Anna-Maria Gleixner <anna-maria@linutronix.de>
---
 include/linux/hrtimer.h |    6 +-
 kernel/time/hrtimer.c   |  131 +++++++++++++++++++++++-------------------------
 2 files changed, 66 insertions(+), 71 deletions(-)

--- a/include/linux/hrtimer.h
+++ b/include/linux/hrtimer.h
@@ -180,10 +180,10 @@ struct hrtimer_cpu_base {
 	unsigned int			clock_was_set_seq;
 	bool				migration_enabled;
 	bool				nohz_active;
-	unsigned int			hres_active	: 1;
-#ifdef CONFIG_HIGH_RES_TIMERS
-	unsigned int			in_hrtirq	: 1,
+	unsigned int			hres_active	: 1,
+					in_hrtirq	: 1,
 					hang_detected	: 1;
+#ifdef CONFIG_HIGH_RES_TIMERS
 	unsigned int			nr_events;
 	unsigned int			nr_retries;
 	unsigned int			nr_hangs;
--- a/kernel/time/hrtimer.c
+++ b/kernel/time/hrtimer.c
@@ -449,13 +449,13 @@ static inline void debug_deactivate(stru
 	trace_hrtimer_cancel(timer);
 }
 
-#if defined(CONFIG_NO_HZ_COMMON) || defined(CONFIG_HIGH_RES_TIMERS)
 static inline void hrtimer_update_next_timer(struct hrtimer_cpu_base *cpu_base,
 					     struct hrtimer *timer)
 {
 	cpu_base->next_timer = timer;
 }
 
+#if defined(CONFIG_NO_HZ_COMMON) || defined(CONFIG_HIGH_RES_TIMERS)
 static ktime_t __hrtimer_get_next_event(struct hrtimer_cpu_base *cpu_base)
 {
 	unsigned int active = cpu_base->active_bases;
@@ -582,68 +582,6 @@ hrtimer_force_reprogram(struct hrtimer_c
 }
 
 /*
- * When a timer is enqueued and expires earlier than the already enqueued
- * timers, we have to check, whether it expires earlier than the timer for
- * which the clock event device was armed.
- *
- * Called with interrupts disabled and base->cpu_base.lock held
- */
-static void hrtimer_reprogram(struct hrtimer *timer,
-			      struct hrtimer_clock_base *base)
-{
-	struct hrtimer_cpu_base *cpu_base = this_cpu_ptr(&hrtimer_bases);
-	ktime_t expires = ktime_sub(hrtimer_get_expires(timer), base->offset);
-
-	WARN_ON_ONCE(hrtimer_get_expires_tv64(timer) < 0);
-
-	/*
-	 * If the timer is not on the current cpu, we cannot reprogram
-	 * the other cpus clock event device.
-	 */
-	if (base->cpu_base != cpu_base)
-		return;
-
-	/*
-	 * If the hrtimer interrupt is running, then it will
-	 * reevaluate the clock bases and reprogram the clock event
-	 * device. The callbacks are always executed in hard interrupt
-	 * context so we don't need an extra check for a running
-	 * callback.
-	 */
-	if (cpu_base->in_hrtirq)
-		return;
-
-	/*
-	 * CLOCK_REALTIME timer might be requested with an absolute
-	 * expiry time which is less than base->offset. Set it to 0.
-	 */
-	if (expires < 0)
-		expires = 0;
-
-	if (expires >= cpu_base->expires_next)
-		return;
-
-	/* Update the pointer to the next expiring timer */
-	hrtimer_update_next_timer(cpu_base, timer);
-
-	/*
-	 * If a hang was detected in the last timer interrupt then we
-	 * do not schedule a timer which is earlier than the expiry
-	 * which we enforced in the hang detection. We want the system
-	 * to make progress.
-	 */
-	if (cpu_base->hang_detected)
-		return;
-
-	/*
-	 * Program the timer hardware. We enforce the expiry for
-	 * events which are already in the past.
-	 */
-	cpu_base->expires_next = expires;
-	tick_program_event(expires, 1);
-}
-
-/*
  * Retrigger next event is called after clock was set
  *
  * Called with interrupts disabled via on_each_cpu()
@@ -703,16 +641,73 @@ static inline int hrtimer_is_hres_enable
 static inline void hrtimer_switch_to_hres(void) { }
 static inline void
 hrtimer_force_reprogram(struct hrtimer_cpu_base *base, int skip_equal) { }
-static inline int hrtimer_reprogram(struct hrtimer *timer,
-				    struct hrtimer_clock_base *base)
-{
-	return 0;
-}
 static inline void retrigger_next_event(void *arg) { }
 
 #endif /* CONFIG_HIGH_RES_TIMERS */
 
 /*
+ * When a timer is enqueued and expires earlier than the already enqueued
+ * timers, we have to check, whether it expires earlier than the timer for
+ * which the clock event device was armed.
+ *
+ * Called with interrupts disabled and base->cpu_base.lock held
+ */
+static void hrtimer_reprogram(struct hrtimer *timer,
+			      struct hrtimer_clock_base *base)
+{
+	struct hrtimer_cpu_base *cpu_base = this_cpu_ptr(&hrtimer_bases);
+	ktime_t expires = ktime_sub(hrtimer_get_expires(timer), base->offset);
+
+	WARN_ON_ONCE(hrtimer_get_expires_tv64(timer) < 0);
+
+	/*
+	 * If the timer is not on the current cpu, we cannot reprogram
+	 * the other cpus clock event device.
+	 */
+	if (base->cpu_base != cpu_base)
+		return;
+
+	/*
+	 * If the hrtimer interrupt is running, then it will
+	 * reevaluate the clock bases and reprogram the clock event
+	 * device. The callbacks are always executed in hard interrupt
+	 * context so we don't need an extra check for a running
+	 * callback.
+	 */
+	if (cpu_base->in_hrtirq)
+		return;
+
+	/*
+	 * CLOCK_REALTIME timer might be requested with an absolute
+	 * expiry time which is less than base->offset. Set it to 0.
+	 */
+	if (expires < 0)
+		expires = 0;
+
+	if (expires >= cpu_base->expires_next)
+		return;
+
+	/* Update the pointer to the next expiring timer */
+	hrtimer_update_next_timer(cpu_base, timer);
+
+	/*
+	 * If a hang was detected in the last timer interrupt then we
+	 * do not schedule a timer which is earlier than the expiry
+	 * which we enforced in the hang detection. We want the system
+	 * to make progress.
+	 */
+	if (cpu_base->hang_detected)
+		return;
+
+	/*
+	 * Program the timer hardware. We enforce the expiry for
+	 * events which are already in the past.
+	 */
+	cpu_base->expires_next = expires;
+	tick_program_event(expires, 1);
+}
+
+/*
  * Clock realtime was set
  *
  * Change the offset of the realtime clock vs. the monotonic

Re: [PATCH 09/25] hrtimer: Reduce conditional code (hrtimer_reprogram())

[Peter Zijlstra]

On Thu, Aug 31, 2017 at 12:23:37PM -0000, Anna-Maria Gleixner wrote:
> The hrtimer_reprogram() is currently required only when
> CONFIG_HIGH_RES_TIMERS is set. Additional bitfields of hrtimer_cpu_base
> struct are high resolution timer specific as well.
> 
> To simplify the hrtimer code, the behaviour of CONFIG_HIGH_RES_TIMERS and
> !CONFIG_HIGH_RES_TIMERS should be similar. As preparation for this, the
> function hrtimer_reprogram() and required hrtimer_cpu_base struct members
> are moved outside the conditional area.

This Changelog is impenetrable. That is, I've no idea what you're
trying to say.

As to the patch, that makes hrtimer_reprogram exist unconditionally, but
the only callsite (so far) is condition on hrtimer_is_hres_active(),
which can only be true if HIGH_RES.

So what, except grow the image size, does this patch achieve?

[PATCH 10/25] hrtimer: Make handling of hrtimer reprogramming and enqueuing not conditional

[Anna-Maria Gleixner]

[-- Attachment #1: hrtimer--handle-gleich.patch --]
[-- Type: text/plain, Size: 3023 bytes --]

The hrtimer_reprogramming, remote timer enqueuing and handling of the
hrtimer_cpu_base struct member expires_next depend on the active high
resolution timers. This makes the code harder to understand.

To simplify the code, the hrtimer reprogramming is now executed
independently except for the real reprogramming part. The expires_next
stores now the first enqueued timer. Due to the adaption of the
check_target function, remote enqueuing is now only possible when the
expiry time is after the currently first expiry time independent of the
active high resolution timers.

Signed-off-by: Anna-Maria Gleixner <anna-maria@linutronix.de>
---
 kernel/time/hrtimer.c |   31 ++++++++++++-------------------
 1 file changed, 12 insertions(+), 19 deletions(-)

--- a/kernel/time/hrtimer.c
+++ b/kernel/time/hrtimer.c
@@ -154,10 +154,11 @@ struct hrtimer_clock_base *lock_hrtimer_
 }
 
 /*
- * With high resolution timers enabled we do not migrate the timer
- * when it is expiring before the next event on the target cpu because
- * we cannot reprogram the target cpu hardware and we would cause it
- * to fire late.
+ * We do not migrate the timer when it is expiring before the next
+ * event on the target cpu. When high resolution is enabled, we cannot
+ * reprogram the target cpu hardware and we would cause it to fire
+ * late. To keep it simple, we handle the high resolution enabled and
+ * disabled case similar.
  *
  * Called with cpu_base->lock of target cpu held.
  */
@@ -166,9 +167,6 @@ hrtimer_check_target(struct hrtimer *tim
 {
 	ktime_t expires;
 
-	if (!new_base->cpu_base->hres_active)
-		return 0;
-
 	expires = ktime_sub(hrtimer_get_expires(timer), new_base->offset);
 	return expires <= new_base->cpu_base->expires_next;
 }
@@ -689,21 +687,24 @@ static void hrtimer_reprogram(struct hrt
 
 	/* Update the pointer to the next expiring timer */
 	hrtimer_update_next_timer(cpu_base, timer);
+	cpu_base->expires_next = expires;
 
 	/*
+	 * If hres is not active, hardware does not have to be
+	 * programmed yet.
+	 *
 	 * If a hang was detected in the last timer interrupt then we
 	 * do not schedule a timer which is earlier than the expiry
 	 * which we enforced in the hang detection. We want the system
 	 * to make progress.
 	 */
-	if (cpu_base->hang_detected)
+	if (!__hrtimer_hres_active(cpu_base) || cpu_base->hang_detected)
 		return;
 
 	/*
 	 * Program the timer hardware. We enforce the expiry for
 	 * events which are already in the past.
 	 */
-	cpu_base->expires_next = expires;
 	tick_program_event(expires, 1);
 }
 
@@ -943,16 +944,8 @@ void hrtimer_start_range_ns(struct hrtim
 	if (!leftmost)
 		goto unlock;
 
-	if (!hrtimer_is_hres_active(timer)) {
-		/*
-		 * Kick to reschedule the next tick to handle the new timer
-		 * on dynticks target.
-		 */
-		if (new_base->cpu_base->nohz_active)
-			wake_up_nohz_cpu(new_base->cpu_base->cpu);
-	} else {
-		hrtimer_reprogram(timer, new_base);
-	}
+	hrtimer_reprogram(timer, new_base);
+
 unlock:
 	unlock_hrtimer_base(timer, &flags);
 }

Re: [PATCH 10/25] hrtimer: Make handling of hrtimer reprogramming and enqueuing not conditional

[Peter Zijlstra]

On Thu, Aug 31, 2017 at 12:23:38PM -0000, Anna-Maria Gleixner wrote:
> The hrtimer_reprogramming, remote timer enqueuing and handling of the
> hrtimer_cpu_base struct member expires_next depend on the active high
> resolution timers. This makes the code harder to understand.
> 
> To simplify the code, the hrtimer reprogramming is now executed
> independently except for the real reprogramming part. The expires_next
> stores now the first enqueued timer. Due to the adaption of the
> check_target function, remote enqueuing is now only possible when the
> expiry time is after the currently first expiry time independent of the
> active high resolution timers.

Sorry, very hard to follow. What?

So we do this to unconditionally track expire_next, such that we can
(later) use hrtimer_check_target()?

Re: [PATCH 10/25] hrtimer: Make handling of hrtimer reprogramming and enqueuing not conditional

[Anna-Maria Gleixner]

On Tue, 26 Sep 2017, Peter Zijlstra wrote:

> On Thu, Aug 31, 2017 at 12:23:38PM -0000, Anna-Maria Gleixner wrote:
> > The hrtimer_reprogramming, remote timer enqueuing and handling of the
> > hrtimer_cpu_base struct member expires_next depend on the active high
> > resolution timers. This makes the code harder to understand.
> > 
> > To simplify the code, the hrtimer reprogramming is now executed
> > independently except for the real reprogramming part. The expires_next
> > stores now the first enqueued timer. Due to the adaption of the
> > check_target function, remote enqueuing is now only possible when the
> > expiry time is after the currently first expiry time independent of the
> > active high resolution timers.
> 
> Sorry, very hard to follow. What?

I'm sorry, I have to rework the commit messages...

> 
> So we do this to unconditionally track expire_next, such that we can
> (later) use hrtimer_check_target()?
> 

The main goal of this patch and the three patches before in the series
is to reduce the conditional code. I tried to split it into pieces...

next_timer stores the pointer to the first expiring timer. In
__remove_hrtimer() next_timer is compared to the removed timer; if the
pointers match then the hardware needs to be reprogrammed. This is
done to avoid the extra interrupt which was armed for the removed
timer. So its a HIGH_RES only functionality.

For the softirq mode we need that pointer to have access to the first
expiring timer unconditionally.

expires_next stores the next event armed in hardware. That's used to
check whether a timer can be enqueued remotely. If the new timer is
expiring before expires_next, then remote enqueue is not possible as
we cannot access the remote timer hardware to reprogram it. This is
currently conditional for the HIGH_RES case, but there is no reason to
make this conditional. So I made HIGH_RES and !HIGH_RES behave the
same way just to further reduce the ifdef and conditional zoo.

Anna-Maria

[PATCH 11/25] hrtimer: Allow remote hrtimer enqueue with "expires_next" as expiry time

[Anna-Maria Gleixner]

[-- Attachment #1: hrtimer--check-target-fix.patch --]
[-- Type: text/plain, Size: 910 bytes --]

When enqueuing a timer with expiry X into a timer queue, where already
a timer with expriy X is queued, the new timer is queued on the
right-hand side of the already queued timer.

Therefore it is no problem, to enqueue a hrtimer on a remote CPU with the
same expiry time than the programmed expiry time (expires_next) on this
CPU, because the reprogramming path is not executed - it is not the
"leftmost" hrtimer.

Signed-off-by: Anna-Maria Gleixner <anna-maria@linutronix.de>
---
 kernel/time/hrtimer.c |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

--- a/kernel/time/hrtimer.c
+++ b/kernel/time/hrtimer.c
@@ -168,7 +168,7 @@ hrtimer_check_target(struct hrtimer *tim
 	ktime_t expires;
 
 	expires = ktime_sub(hrtimer_get_expires(timer), new_base->offset);
-	return expires <= new_base->cpu_base->expires_next;
+	return expires < new_base->cpu_base->expires_next;
 }
 
 #ifdef CONFIG_NO_HZ_COMMON

[PATCH 13/25] hrtimer: Split out code from hrtimer_start_range_ns() for reuse

[Anna-Maria Gleixner]

[-- Attachment #1: hrtimer_Split_out_code_from_hrtimer_start_range_ns_for_reuse.patch --]
[-- Type: text/plain, Size: 2286 bytes --]

From: Anna-Maria Gleixner <anna-maria@linutronix.de>

Preparatory patch for softirq based hrtimers. No functional change.

Signed-off-by: Anna-Maria Gleixner <anna-maria@linutronix.de>
---
 kernel/time/hrtimer.c |   44 ++++++++++++++++++++++++--------------------
 1 file changed, 24 insertions(+), 20 deletions(-)

--- a/kernel/time/hrtimer.c
+++ b/kernel/time/hrtimer.c
@@ -910,22 +910,11 @@ static inline ktime_t hrtimer_update_low
 	return tim;
 }
 
-/**
- * hrtimer_start_range_ns - (re)start an hrtimer on the current CPU
- * @timer:	the timer to be added
- * @tim:	expiry time
- * @delta_ns:	"slack" range for the timer
- * @mode:	expiry mode: absolute (HRTIMER_MODE_ABS) or
- *		relative (HRTIMER_MODE_REL)
- */
-void hrtimer_start_range_ns(struct hrtimer *timer, ktime_t tim,
-			    u64 delta_ns, const enum hrtimer_mode mode)
+static int __hrtimer_start_range_ns(struct hrtimer *timer, ktime_t tim,
+				    u64 delta_ns, const enum hrtimer_mode mode,
+				    struct hrtimer_clock_base *base)
 {
-	struct hrtimer_clock_base *base, *new_base;
-	unsigned long flags;
-	int leftmost;
-
-	base = lock_hrtimer_base(timer, &flags);
+	struct hrtimer_clock_base *new_base;
 
 	/* Remove an active timer from the queue: */
 	remove_hrtimer(timer, base, true);
@@ -940,13 +929,28 @@ void hrtimer_start_range_ns(struct hrtim
 	/* Switch the timer base, if necessary: */
 	new_base = switch_hrtimer_base(timer, base, mode & HRTIMER_MODE_PINNED);
 
-	leftmost = enqueue_hrtimer(timer, new_base);
-	if (!leftmost)
-		goto unlock;
+	return enqueue_hrtimer(timer, new_base);
+}
+
+/**
+ * hrtimer_start_range_ns - (re)start an hrtimer on the current CPU
+ * @timer:	the timer to be added
+ * @tim:	expiry time
+ * @delta_ns:	"slack" range for the timer
+ * @mode:	expiry mode: absolute (HRTIMER_MODE_ABS) or
+ *		relative (HRTIMER_MODE_REL)
+ */
+void hrtimer_start_range_ns(struct hrtimer *timer, ktime_t tim,
+			    u64 delta_ns, const enum hrtimer_mode mode)
+{
+	struct hrtimer_clock_base *base;
+	unsigned long flags;
+
+	base = lock_hrtimer_base(timer, &flags);
 
-	hrtimer_reprogram(timer);
+	if (__hrtimer_start_range_ns(timer, tim, delta_ns, mode, base))
+		hrtimer_reprogram(timer);
 
-unlock:
 	unlock_hrtimer_base(timer, &flags);
 }
 EXPORT_SYMBOL_GPL(hrtimer_start_range_ns);

[PATCH 12/25] hrtimer: Simplify hrtimer_reprogram() call

[Anna-Maria Gleixner]

[-- Attachment #1: hrtimer--Simplify-hrtimer_reprogram---call.patch --]
[-- Type: text/plain, Size: 1121 bytes --]

The hrtimer_reprogramm() call can be simplified by dereferencing the
hrtimer clock base inside the function. It is a preparatory change for
softirq based hrtimers.

Signed-off-by: Anna-Maria Gleixner <anna-maria@linutronix.de>
---
 kernel/time/hrtimer.c |    6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

--- a/kernel/time/hrtimer.c
+++ b/kernel/time/hrtimer.c
@@ -650,10 +650,10 @@ static inline void retrigger_next_event(
  *
  * Called with interrupts disabled and base->cpu_base.lock held
  */
-static void hrtimer_reprogram(struct hrtimer *timer,
-			      struct hrtimer_clock_base *base)
+static void hrtimer_reprogram(struct hrtimer *timer)
 {
 	struct hrtimer_cpu_base *cpu_base = this_cpu_ptr(&hrtimer_bases);
+	struct hrtimer_clock_base *base = timer->base;
 	ktime_t expires = ktime_sub(hrtimer_get_expires(timer), base->offset);
 
 	WARN_ON_ONCE(hrtimer_get_expires_tv64(timer) < 0);
@@ -944,7 +944,7 @@ void hrtimer_start_range_ns(struct hrtim
 	if (!leftmost)
 		goto unlock;
 
-	hrtimer_reprogram(timer, new_base);
+	hrtimer_reprogram(timer);
 
 unlock:
 	unlock_hrtimer_base(timer, &flags);

[PATCH 14/25] hrtimer: Split out code from __hrtimer_get_next_event() for reuse

[Anna-Maria Gleixner]

[-- Attachment #1: hrtimer_Split_out_code_from___hrtimer_get_next_event_for_reuse.patch --]
[-- Type: text/plain, Size: 1458 bytes --]

From: Anna-Maria Gleixner <anna-maria@linutronix.de>

Preparatory patch for softirq based hrtimers. No functional change.

Signed-off-by: Anna-Maria Gleixner <anna-maria@linutronix.de>
---
 kernel/time/hrtimer.c |   20 ++++++++++++++++----
 1 file changed, 16 insertions(+), 4 deletions(-)

--- a/kernel/time/hrtimer.c
+++ b/kernel/time/hrtimer.c
@@ -454,12 +454,12 @@ static inline void hrtimer_update_next_t
 }
 
 #if defined(CONFIG_NO_HZ_COMMON) || defined(CONFIG_HIGH_RES_TIMERS)
-static ktime_t __hrtimer_get_next_event(struct hrtimer_cpu_base *cpu_base)
+static ktime_t __hrtimer_next_event_base(struct hrtimer_cpu_base *cpu_base,
+					 unsigned int active,
+					 ktime_t expires_next)
 {
-	unsigned int active = cpu_base->active_bases;
-	ktime_t expires, expires_next = KTIME_MAX;
+	ktime_t expires;
 
-	hrtimer_update_next_timer(cpu_base, NULL);
 	while (active) {
 		unsigned int id = __ffs(active);
 		struct hrtimer_clock_base *base;
@@ -486,6 +486,18 @@ static ktime_t __hrtimer_get_next_event(
 		expires_next = 0;
 	return expires_next;
 }
+
+static ktime_t __hrtimer_get_next_event(struct hrtimer_cpu_base *cpu_base)
+{
+	unsigned int active = cpu_base->active_bases;
+	ktime_t expires_next = KTIME_MAX;
+
+	hrtimer_update_next_timer(cpu_base, NULL);
+
+	expires_next = __hrtimer_next_event_base(cpu_base, active, expires_next);
+
+	return expires_next;
+}
 #endif
 
 static inline ktime_t hrtimer_update_base(struct hrtimer_cpu_base *base)

[PATCH 15/25] hrtimer: Add clock bases for soft irq context

[Anna-Maria Gleixner]

[-- Attachment #1: hrtimer_Add_clock_bases_for_soft_irq_context.patch --]
[-- Type: text/plain, Size: 3712 bytes --]

From: Anna-Maria Gleixner <anna-maria@linutronix.de>

hrtimer callback functions are always executed in hard interrupt
context. Users of hrtimer which need their timer function to be executed
in soft interrupt context, make use of tasklets to get the proper context.

Add additional clock bases for timers which must expire in softirq context,
so the detour via the tasklet can be avoided. This is also required for RT,
where the majority of hrtimer is moved into softirq context.

Keep the new clockids internal to hrtimer for now, so they can't be
accessed from other code until the rest of the changes is in place.

Signed-off-by: Anna-Maria Gleixner <anna-maria@linutronix.de>
---
 include/linux/hrtimer.h |    4 +++
 kernel/time/hrtimer.c   |   56 ++++++++++++++++++++++++++++++++++++++++++------
 2 files changed, 54 insertions(+), 6 deletions(-)

--- a/include/linux/hrtimer.h
+++ b/include/linux/hrtimer.h
@@ -145,6 +145,10 @@ enum  hrtimer_base_type {
 	HRTIMER_BASE_REALTIME,
 	HRTIMER_BASE_BOOTTIME,
 	HRTIMER_BASE_TAI,
+	HRTIMER_BASE_MONOTONIC_SOFT,
+	HRTIMER_BASE_REALTIME_SOFT,
+	HRTIMER_BASE_BOOTTIME_SOFT,
+	HRTIMER_BASE_TAI_SOFT,
 	HRTIMER_MAX_CLOCK_BASES,
 };
 
--- a/kernel/time/hrtimer.c
+++ b/kernel/time/hrtimer.c
@@ -60,6 +60,18 @@
 #include "tick-internal.h"
 
 /*
+ * Clock ids for timers which expire in softirq context. These clock ids
+ * are kernel internal and never exported to user space. Kept internal
+ * until the rest of the functionality is in place.
+ */
+#define HRTIMER_BASE_SOFT_MASK	MAX_CLOCKS
+
+#define CLOCK_REALTIME_SOFT	(CLOCK_REALTIME	 | HRTIMER_BASE_SOFT_MASK)
+#define CLOCK_MONOTONIC_SOFT	(CLOCK_MONOTONIC | HRTIMER_BASE_SOFT_MASK)
+#define CLOCK_BOOTTIME_SOFT	(CLOCK_BOOTTIME	 | HRTIMER_BASE_SOFT_MASK)
+#define CLOCK_TAI_SOFT		(CLOCK_TAI	 | HRTIMER_BASE_SOFT_MASK)
+
+/*
  * The timer bases:
  *
  * There are more clockids than hrtimer bases. Thus, we index
@@ -92,17 +104,43 @@ DEFINE_PER_CPU(struct hrtimer_cpu_base,
 			.clockid = CLOCK_TAI,
 			.get_time = &ktime_get_clocktai,
 		},
+		{
+			.index = HRTIMER_BASE_MONOTONIC_SOFT,
+			.clockid = CLOCK_MONOTONIC_SOFT,
+			.get_time = &ktime_get,
+		},
+		{
+			.index = HRTIMER_BASE_REALTIME_SOFT,
+			.clockid = CLOCK_REALTIME_SOFT,
+			.get_time = &ktime_get_real,
+		},
+		{
+			.index = HRTIMER_BASE_BOOTTIME_SOFT,
+			.clockid = CLOCK_BOOTTIME_SOFT,
+			.get_time = &ktime_get_boottime,
+		},
+		{
+			.index = HRTIMER_BASE_TAI_SOFT,
+			.clockid = CLOCK_TAI_SOFT,
+			.get_time = &ktime_get_clocktai,
+		},
 	}
 };
 
-static const int hrtimer_clock_to_base_table[MAX_CLOCKS] = {
+#define MAX_CLOCKS_HRT		(MAX_CLOCKS * 2)
+
+static const int hrtimer_clock_to_base_table[MAX_CLOCKS_HRT] = {
 	/* Make sure we catch unsupported clockids */
-	[0 ... MAX_CLOCKS - 1]	= HRTIMER_MAX_CLOCK_BASES,
+	[0 ... MAX_CLOCKS_HRT - 1]	= HRTIMER_MAX_CLOCK_BASES,
 
-	[CLOCK_REALTIME]	= HRTIMER_BASE_REALTIME,
-	[CLOCK_MONOTONIC]	= HRTIMER_BASE_MONOTONIC,
-	[CLOCK_BOOTTIME]	= HRTIMER_BASE_BOOTTIME,
-	[CLOCK_TAI]		= HRTIMER_BASE_TAI,
+	[CLOCK_REALTIME]		= HRTIMER_BASE_REALTIME,
+	[CLOCK_MONOTONIC]		= HRTIMER_BASE_MONOTONIC,
+	[CLOCK_BOOTTIME]		= HRTIMER_BASE_BOOTTIME,
+	[CLOCK_TAI]			= HRTIMER_BASE_TAI,
+	[CLOCK_REALTIME_SOFT]		= HRTIMER_BASE_REALTIME_SOFT,
+	[CLOCK_MONOTONIC_SOFT]		= HRTIMER_BASE_MONOTONIC_SOFT,
+	[CLOCK_BOOTTIME_SOFT]		= HRTIMER_BASE_BOOTTIME_SOFT,
+	[CLOCK_TAI_SOFT]		= HRTIMER_BASE_TAI_SOFT,
 };
 
 /*
@@ -1652,6 +1690,12 @@ int hrtimers_dead_cpu(unsigned int scpu)
 
 void __init hrtimers_init(void)
 {
+	/*
+	 * It is necessary, that the soft base mask is a single
+	 * bit.
+	 */
+	BUILD_BUG_ON_NOT_POWER_OF_2(HRTIMER_BASE_SOFT_MASK);
+
 	hrtimers_prepare_cpu(smp_processor_id());
 }
 

[PATCH 16/25] hrtimer: Allow function reuse for softirq based hrtimer

[Anna-Maria Gleixner]

[-- Attachment #1: hrtimer_Allow_function_reuse_for_softirq_based_hrtimer.patch --]
[-- Type: text/plain, Size: 2498 bytes --]

From: Anna-Maria Gleixner <anna-maria@linutronix.de>

The softirq based hrtimer can utilize most of the existing hrtimer
functions, but need to operate on a different data set. Add an active_mask
argument to various functions so the hard and soft bases can be
selected. Fixup the existing callers and hand in the ACTIVE_HARD mask.

Signed-off-by: Anna-Maria Gleixner <anna-maria@linutronix.de>
---
 kernel/time/hrtimer.c |   20 +++++++++++++++-----
 1 file changed, 15 insertions(+), 5 deletions(-)

--- a/kernel/time/hrtimer.c
+++ b/kernel/time/hrtimer.c
@@ -72,6 +72,14 @@
 #define CLOCK_TAI_SOFT		(CLOCK_TAI	 | HRTIMER_BASE_SOFT_MASK)
 
 /*
+ * Masks for selecting the soft and hard context timers from
+ * cpu_base->active
+ */
+#define MASK_SHIFT		(HRTIMER_BASE_MONOTONIC_SOFT)
+#define HRTIMER_ACTIVE_HARD	((1U << MASK_SHIFT) - 1)
+#define HRTIMER_ACTIVE_SOFT	(HRTIMER_ACTIVE_HARD << MASK_SHIFT)
+
+/*
  * The timer bases:
  *
  * There are more clockids than hrtimer bases. Thus, we index
@@ -527,11 +535,12 @@ static ktime_t __hrtimer_next_event_base
 
 static ktime_t __hrtimer_get_next_event(struct hrtimer_cpu_base *cpu_base)
 {
-	unsigned int active = cpu_base->active_bases;
+	unsigned int active;
 	ktime_t expires_next = KTIME_MAX;
 
 	hrtimer_update_next_timer(cpu_base, NULL);
 
+	active = cpu_base->active_bases & HRTIMER_ACTIVE_HARD;
 	expires_next = __hrtimer_next_event_base(cpu_base, active, expires_next);
 
 	return expires_next;
@@ -1264,9 +1273,10 @@ static void __run_hrtimer(struct hrtimer
 	base->running = NULL;
 }
 
-static void __hrtimer_run_queues(struct hrtimer_cpu_base *cpu_base, ktime_t now)
+static void __hrtimer_run_queues(struct hrtimer_cpu_base *cpu_base, ktime_t now,
+				 unsigned int active_mask)
 {
-	unsigned int active = cpu_base->active_bases;
+	unsigned int active = cpu_base->active_bases & active_mask;
 
 	while (active) {
 		unsigned int id = __ffs(active);
@@ -1333,7 +1343,7 @@ void hrtimer_interrupt(struct clock_even
 	 */
 	cpu_base->expires_next = KTIME_MAX;
 
-	__hrtimer_run_queues(cpu_base, now);
+	__hrtimer_run_queues(cpu_base, now, HRTIMER_ACTIVE_HARD);
 
 	/* Reevaluate the clock bases for the next expiry */
 	expires_next = __hrtimer_get_next_event(cpu_base);
@@ -1438,7 +1448,7 @@ void hrtimer_run_queues(void)
 
 	raw_spin_lock(&cpu_base->lock);
 	now = hrtimer_update_base(cpu_base);
-	__hrtimer_run_queues(cpu_base, now);
+	__hrtimer_run_queues(cpu_base, now, HRTIMER_ACTIVE_HARD);
 	raw_spin_unlock(&cpu_base->lock);
 }
 

[PATCH 18/25] hrtimer: Enable soft and hard hrtimer

[Anna-Maria Gleixner]

[-- Attachment #1: hrtimer_Enable_soft_and_hard_hrtimer.patch --]
[-- Type: text/plain, Size: 2742 bytes --]

From: Anna-Maria Gleixner <anna-maria@linutronix.de>

Move the definition of the clock ids, to be available not only
internal. The transition between clock id and hrtimer base is now
expanded by the soft hrtimer bases and the corresponding clock
ids. Update all hard hrtimer restricted queries to handle soft and
hard hrtimers similarly.

Signed-off-by: Anna-Maria Gleixner <anna-maria@linutronix.de>
---
 include/linux/hrtimer.h |   11 +++++++++++
 kernel/time/hrtimer.c   |   22 +++++++---------------
 2 files changed, 18 insertions(+), 15 deletions(-)

--- a/include/linux/hrtimer.h
+++ b/include/linux/hrtimer.h
@@ -23,6 +23,17 @@
 #include <linux/timer.h>
 #include <linux/timerqueue.h>
 
+/*
+ * Clock ids for hrtimers which expire in softirq context. These clock ids
+ * are kernel internal and never exported to user space.
+ */
+#define HRTIMER_BASE_SOFT_MASK	MAX_CLOCKS
+
+#define CLOCK_REALTIME_SOFT	(CLOCK_REALTIME	 | HRTIMER_BASE_SOFT_MASK)
+#define CLOCK_MONOTONIC_SOFT	(CLOCK_MONOTONIC | HRTIMER_BASE_SOFT_MASK)
+#define CLOCK_BOOTTIME_SOFT	(CLOCK_BOOTTIME	 | HRTIMER_BASE_SOFT_MASK)
+#define CLOCK_TAI_SOFT		(CLOCK_TAI	 | HRTIMER_BASE_SOFT_MASK)
+
 struct hrtimer_clock_base;
 struct hrtimer_cpu_base;
 
--- a/kernel/time/hrtimer.c
+++ b/kernel/time/hrtimer.c
@@ -60,18 +60,6 @@
 #include "tick-internal.h"
 
 /*
- * Clock ids for timers which expire in softirq context. These clock ids
- * are kernel internal and never exported to user space. Kept internal
- * until the rest of the functionality is in place.
- */
-#define HRTIMER_BASE_SOFT_MASK	MAX_CLOCKS
-
-#define CLOCK_REALTIME_SOFT	(CLOCK_REALTIME	 | HRTIMER_BASE_SOFT_MASK)
-#define CLOCK_MONOTONIC_SOFT	(CLOCK_MONOTONIC | HRTIMER_BASE_SOFT_MASK)
-#define CLOCK_BOOTTIME_SOFT	(CLOCK_BOOTTIME	 | HRTIMER_BASE_SOFT_MASK)
-#define CLOCK_TAI_SOFT		(CLOCK_TAI	 | HRTIMER_BASE_SOFT_MASK)
-
-/*
  * Masks for selecting the soft and hard context timers from
  * cpu_base->active
  */
@@ -1173,7 +1161,7 @@ u64 hrtimer_get_next_event(void)
 
 static inline int hrtimer_clockid_to_base(clockid_t clock_id)
 {
-	if (likely(clock_id < MAX_CLOCKS)) {
+	if (likely(clock_id < MAX_CLOCKS_HRT)) {
 		int base = hrtimer_clock_to_base_table[clock_id];
 
 		if (likely(base != HRTIMER_MAX_CLOCK_BASES))
@@ -1193,8 +1181,12 @@ static void __hrtimer_init(struct hrtime
 
 	cpu_base = raw_cpu_ptr(&hrtimer_bases);
 
-	if (clock_id == CLOCK_REALTIME && mode != HRTIMER_MODE_ABS)
-		clock_id = CLOCK_MONOTONIC;
+	if (mode != HRTIMER_MODE_ABS) {
+		if (clock_id == CLOCK_REALTIME)
+			clock_id = CLOCK_MONOTONIC;
+		else if (clock_id == CLOCK_REALTIME_SOFT)
+			clock_id = CLOCK_MONOTONIC_SOFT;
+	}
 
 	base = hrtimer_clockid_to_base(clock_id);
 	timer->base = &cpu_base->clock_base[base];

Re: [PATCH 18/25] hrtimer: Enable soft and hard hrtimer

[Peter Zijlstra]

On Thu, Aug 31, 2017 at 12:23:42PM -0000, Anna-Maria Gleixner wrote:
> --- a/include/linux/hrtimer.h
> +++ b/include/linux/hrtimer.h
> @@ -23,6 +23,17 @@
>  #include <linux/timer.h>
>  #include <linux/timerqueue.h>
>  
> +/*
> + * Clock ids for hrtimers which expire in softirq context. These clock ids
> + * are kernel internal and never exported to user space.
> + */
> +#define HRTIMER_BASE_SOFT_MASK	MAX_CLOCKS
> +
> +#define CLOCK_REALTIME_SOFT	(CLOCK_REALTIME	 | HRTIMER_BASE_SOFT_MASK)
> +#define CLOCK_MONOTONIC_SOFT	(CLOCK_MONOTONIC | HRTIMER_BASE_SOFT_MASK)
> +#define CLOCK_BOOTTIME_SOFT	(CLOCK_BOOTTIME	 | HRTIMER_BASE_SOFT_MASK)
> +#define CLOCK_TAI_SOFT		(CLOCK_TAI	 | HRTIMER_BASE_SOFT_MASK)
> +
>  struct hrtimer_clock_base;
>  struct hrtimer_cpu_base;
>  

> --- a/kernel/time/hrtimer.c
> +++ b/kernel/time/hrtimer.c
> @@ -1173,7 +1161,7 @@ u64 hrtimer_get_next_event(void)
>  
>  static inline int hrtimer_clockid_to_base(clockid_t clock_id)
>  {
> +	if (likely(clock_id < MAX_CLOCKS_HRT)) {
>  		int base = hrtimer_clock_to_base_table[clock_id];
>  
>  		if (likely(base != HRTIMER_MAX_CLOCK_BASES))
> @@ -1193,8 +1181,12 @@ static void __hrtimer_init(struct hrtime
>  
>  	cpu_base = raw_cpu_ptr(&hrtimer_bases);
>  
> +	if (mode != HRTIMER_MODE_ABS) {
> +		if (clock_id == CLOCK_REALTIME)
> +			clock_id = CLOCK_MONOTONIC;
> +		else if (clock_id == CLOCK_REALTIME_SOFT)
> +			clock_id = CLOCK_MONOTONIC_SOFT;
> +	}
>  
>  	base = hrtimer_clockid_to_base(clock_id);
>  	timer->base = &cpu_base->clock_base[base];

So why expose these extra bases at all, why not stick another flag in
MODE? These extra bases is a pure implementation detail imo; you could
equally implement the functionality without (albeit at extra cost).

Re: [PATCH 18/25] hrtimer: Enable soft and hard hrtimer

[Anna-Maria Gleixner]

On Tue, 26 Sep 2017, Peter Zijlstra wrote:

> On Thu, Aug 31, 2017 at 12:23:42PM -0000, Anna-Maria Gleixner wrote:
> > --- a/include/linux/hrtimer.h
> > +++ b/include/linux/hrtimer.h
> > @@ -23,6 +23,17 @@
> >  #include <linux/timer.h>
> >  #include <linux/timerqueue.h>
> >  
> > +/*
> > + * Clock ids for hrtimers which expire in softirq context. These clock ids
> > + * are kernel internal and never exported to user space.
> > + */
> > +#define HRTIMER_BASE_SOFT_MASK	MAX_CLOCKS
> > +
> > +#define CLOCK_REALTIME_SOFT	(CLOCK_REALTIME	 | HRTIMER_BASE_SOFT_MASK)
> > +#define CLOCK_MONOTONIC_SOFT	(CLOCK_MONOTONIC | HRTIMER_BASE_SOFT_MASK)
> > +#define CLOCK_BOOTTIME_SOFT	(CLOCK_BOOTTIME	 | HRTIMER_BASE_SOFT_MASK)
> > +#define CLOCK_TAI_SOFT		(CLOCK_TAI	 | HRTIMER_BASE_SOFT_MASK)
> > +
> >  struct hrtimer_clock_base;
> >  struct hrtimer_cpu_base;
> >  
> 
> > --- a/kernel/time/hrtimer.c
> > +++ b/kernel/time/hrtimer.c
> > @@ -1173,7 +1161,7 @@ u64 hrtimer_get_next_event(void)
> >  
> >  static inline int hrtimer_clockid_to_base(clockid_t clock_id)
> >  {
> > +	if (likely(clock_id < MAX_CLOCKS_HRT)) {
> >  		int base = hrtimer_clock_to_base_table[clock_id];
> >  
> >  		if (likely(base != HRTIMER_MAX_CLOCK_BASES))
> > @@ -1193,8 +1181,12 @@ static void __hrtimer_init(struct hrtime
> >  
> >  	cpu_base = raw_cpu_ptr(&hrtimer_bases);
> >  
> > +	if (mode != HRTIMER_MODE_ABS) {
> > +		if (clock_id == CLOCK_REALTIME)
> > +			clock_id = CLOCK_MONOTONIC;
> > +		else if (clock_id == CLOCK_REALTIME_SOFT)
> > +			clock_id = CLOCK_MONOTONIC_SOFT;
> > +	}
> >  
> >  	base = hrtimer_clockid_to_base(clock_id);
> >  	timer->base = &cpu_base->clock_base[base];
> 
> So why expose these extra bases at all, why not stick another flag in
> MODE? These extra bases is a pure implementation detail imo; you could
> equally implement the functionality without (albeit at extra cost).
> 

It was Thomas' request not to use a flag for this.

Anna-Maria

Re: [PATCH 18/25] hrtimer: Enable soft and hard hrtimer

[Thomas Gleixner]

On Wed, 27 Sep 2017, Anna-Maria Gleixner wrote:
> On Tue, 26 Sep 2017, Peter Zijlstra wrote:
> 
> > On Thu, Aug 31, 2017 at 12:23:42PM -0000, Anna-Maria Gleixner wrote:
> > > --- a/include/linux/hrtimer.h
> > > +++ b/include/linux/hrtimer.h
> > > @@ -23,6 +23,17 @@
> > >  #include <linux/timer.h>
> > >  #include <linux/timerqueue.h>
> > >  
> > > +/*
> > > + * Clock ids for hrtimers which expire in softirq context. These clock ids
> > > + * are kernel internal and never exported to user space.
> > > + */
> > > +#define HRTIMER_BASE_SOFT_MASK	MAX_CLOCKS
> > > +
> > > +#define CLOCK_REALTIME_SOFT	(CLOCK_REALTIME	 | HRTIMER_BASE_SOFT_MASK)
> > > +#define CLOCK_MONOTONIC_SOFT	(CLOCK_MONOTONIC | HRTIMER_BASE_SOFT_MASK)
> > > +#define CLOCK_BOOTTIME_SOFT	(CLOCK_BOOTTIME	 | HRTIMER_BASE_SOFT_MASK)
> > > +#define CLOCK_TAI_SOFT		(CLOCK_TAI	 | HRTIMER_BASE_SOFT_MASK)
> > > +
> > >  struct hrtimer_clock_base;
> > >  struct hrtimer_cpu_base;
> > >  
> > 
> > > --- a/kernel/time/hrtimer.c
> > > +++ b/kernel/time/hrtimer.c
> > > @@ -1173,7 +1161,7 @@ u64 hrtimer_get_next_event(void)
> > >  
> > >  static inline int hrtimer_clockid_to_base(clockid_t clock_id)
> > >  {
> > > +	if (likely(clock_id < MAX_CLOCKS_HRT)) {
> > >  		int base = hrtimer_clock_to_base_table[clock_id];
> > >  
> > >  		if (likely(base != HRTIMER_MAX_CLOCK_BASES))
> > > @@ -1193,8 +1181,12 @@ static void __hrtimer_init(struct hrtime
> > >  
> > >  	cpu_base = raw_cpu_ptr(&hrtimer_bases);
> > >  
> > > +	if (mode != HRTIMER_MODE_ABS) {
> > > +		if (clock_id == CLOCK_REALTIME)
> > > +			clock_id = CLOCK_MONOTONIC;
> > > +		else if (clock_id == CLOCK_REALTIME_SOFT)
> > > +			clock_id = CLOCK_MONOTONIC_SOFT;
> > > +	}
> > >  
> > >  	base = hrtimer_clockid_to_base(clock_id);
> > >  	timer->base = &cpu_base->clock_base[base];
> > 
> > So why expose these extra bases at all, why not stick another flag in
> > MODE? These extra bases is a pure implementation detail imo; you could
> > equally implement the functionality without (albeit at extra cost).

Right and that cost too high.

> > 
> 
> It was Thomas' request not to use a flag for this.

We can make that a flag as well. There is no hard requirement for making it
a CLOCK. Peter is right that it's a pure internal implementation detail and
we can hide it from the outside world.

Thanks,

	tglx

Re: [PATCH 18/25] hrtimer: Enable soft and hard hrtimer

[Peter Zijlstra]

On Wed, Sep 27, 2017 at 05:54:02PM +0200, Thomas Gleixner wrote:

> > > So why expose these extra bases at all, why not stick another flag in
> > > MODE? These extra bases is a pure implementation detail imo; you could
> > > equally implement the functionality without (albeit at extra cost).
> 
> Right and that cost too high.

Yes, no question about that.

> > It was Thomas' request not to use a flag for this.
> 
> We can make that a flag as well. There is no hard requirement for making it
> a CLOCK. Peter is right that it's a pure internal implementation detail and
> we can hide it from the outside world.

Yeah, that was my point.

[PATCH 17/25] hrtimer: Implementation of softirq hrtimer handling

[Anna-Maria Gleixner]

[-- Attachment #1: hrtimer_Implementation_of_softirq_hrtimer_handling.patch --]
[-- Type: text/plain, Size: 9669 bytes --]

From: Anna-Maria Gleixner <anna-maria@linutronix.de>

hrtimers are executed always in hard irq context. If a hrtimer callback
function needs to be exectued in softirq context, the detour using tasklets
is required. To facilitate this, also in regards to real time specific
handling of hrtimers, new clock ids ease the use of hrtimers in softirq
context.

Every clock ID is available for soft and hard hrtimers. The hrtimers are
handled the same way when they are enqueued. When the hrtimer_interrupt
raises, a check is implemented, if the HRTIMER_SOFTIRQ has to be raised as
well. If it is raised, the soft hrtimers are not taken into account when
for example _hrtimer_get_next_event() is called. At the end of the softirq,
all hrtimer_cpu_base struct members are updated, so that the soft hrtimers
are also taken into account.

Suggested-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Anna-Maria Gleixner <anna-maria@linutronix.de>
---
 include/linux/hrtimer.h |    8 ++-
 kernel/time/hrtimer.c   |  125 ++++++++++++++++++++++++++++++++++++++++++++----
 2 files changed, 122 insertions(+), 11 deletions(-)

--- a/include/linux/hrtimer.h
+++ b/include/linux/hrtimer.h
@@ -161,6 +161,8 @@ enum  hrtimer_base_type {
  * @clock_was_set_seq:	Sequence counter of clock was set events
  * @migration_enabled:	The migration of hrtimers to other cpus is enabled
  * @nohz_active:	The nohz functionality is enabled
+ * @softirq_activated:	displays, if the softirq is raised - update of softirq
+ *			related settings is not required then.
  * @in_hrtirq:		hrtimer_interrupt() is currently executing
  * @hres_active:	State of high resolution mode
  * @hang_detected:	The last hrtimer interrupt detected a hang
@@ -169,8 +171,10 @@ enum  hrtimer_base_type {
  * @nr_hangs:		Total number of hrtimer interrupt hangs
  * @max_hang_time:	Maximum time spent in hrtimer_interrupt
  * @expires_next:	absolute time of the next event, is required for remote
- *			hrtimer enqueue
+ *			hrtimer enqueue; it is the total first expiry time (hard
+ *			and soft hrtimer are taken into account)
  * @next_timer:		Pointer to the first expiring timer
+ * @softirq_expires_next: Time to check, if soft queues needs also to be expired
  * @clock_base:		array of clock bases for this cpu
  *
  * Note: next_timer is just an optimization for __remove_hrtimer().
@@ -184,6 +188,7 @@ struct hrtimer_cpu_base {
 	unsigned int			clock_was_set_seq;
 	bool				migration_enabled;
 	bool				nohz_active;
+	bool				softirq_activated;
 	unsigned int			hres_active	: 1,
 					in_hrtirq	: 1,
 					hang_detected	: 1;
@@ -195,6 +200,7 @@ struct hrtimer_cpu_base {
 #endif
 	ktime_t				expires_next;
 	struct hrtimer			*next_timer;
+	ktime_t				softirq_expires_next;
 	struct hrtimer_clock_base	clock_base[HRTIMER_MAX_CLOCK_BASES];
 } ____cacheline_aligned;
 
--- a/kernel/time/hrtimer.c
+++ b/kernel/time/hrtimer.c
@@ -499,7 +499,6 @@ static inline void hrtimer_update_next_t
 	cpu_base->next_timer = timer;
 }
 
-#if defined(CONFIG_NO_HZ_COMMON) || defined(CONFIG_HIGH_RES_TIMERS)
 static ktime_t __hrtimer_next_event_base(struct hrtimer_cpu_base *cpu_base,
 					 unsigned int active,
 					 ktime_t expires_next)
@@ -540,12 +539,23 @@ static ktime_t __hrtimer_get_next_event(
 
 	hrtimer_update_next_timer(cpu_base, NULL);
 
+	if (!cpu_base->softirq_activated) {
+		active = cpu_base->active_bases & HRTIMER_ACTIVE_SOFT;
+		expires_next = __hrtimer_next_event_base(cpu_base, active,
+							 expires_next);
+		cpu_base->softirq_expires_next = expires_next;
+	}
+
 	active = cpu_base->active_bases & HRTIMER_ACTIVE_HARD;
 	expires_next = __hrtimer_next_event_base(cpu_base, active, expires_next);
 
+	/*
+	 * cpu_base->expires_next is not updated here. It is set only
+	 * in hrtimer_reprogramming path!
+	 */
+
 	return expires_next;
 }
-#endif
 
 static inline ktime_t hrtimer_update_base(struct hrtimer_cpu_base *base)
 {
@@ -969,6 +979,49 @@ static inline ktime_t hrtimer_update_low
 	return tim;
 }
 
+static void hrtimer_reprogram_softirq(struct hrtimer *timer)
+{
+	struct hrtimer_clock_base *base = timer->base;
+	struct hrtimer_cpu_base *cpu_base = base->cpu_base;
+	ktime_t expires;
+
+	/*
+	 * The softirq timer is not rearmed, when the softirq was raised
+	 * and has not yet run to completion.
+	 */
+	if (cpu_base->softirq_activated)
+		return;
+
+	expires = ktime_sub(hrtimer_get_expires(timer), base->offset);
+
+	if (!ktime_before(expires, cpu_base->softirq_expires_next))
+		return;
+
+	cpu_base->softirq_expires_next = expires;
+
+	if (!ktime_before(expires, cpu_base->expires_next))
+		return;
+	hrtimer_reprogram(timer);
+}
+
+static void hrtimer_update_softirq_timer(struct hrtimer_cpu_base *cpu_base,
+					 bool reprogram)
+{
+	ktime_t expires;
+
+	expires = __hrtimer_get_next_event(cpu_base);
+
+	if (!reprogram || !ktime_before(expires, cpu_base->expires_next))
+		return;
+	/*
+	 * next_timer can be used here, because
+	 * hrtimer_get_next_event() updated the next
+	 * timer. expires_next is only set when reprogramming function
+	 * is called.
+	 */
+	hrtimer_reprogram(cpu_base->next_timer);
+}
+
 static int __hrtimer_start_range_ns(struct hrtimer *timer, ktime_t tim,
 				    u64 delta_ns, const enum hrtimer_mode mode,
 				    struct hrtimer_clock_base *base)
@@ -1007,9 +1060,12 @@ void hrtimer_start_range_ns(struct hrtim
 
 	base = lock_hrtimer_base(timer, &flags);
 
-	if (__hrtimer_start_range_ns(timer, tim, delta_ns, mode, base))
-		hrtimer_reprogram(timer);
-
+	if (__hrtimer_start_range_ns(timer, tim, delta_ns, mode, base)) {
+		if (timer->base->index < HRTIMER_BASE_MONOTONIC_SOFT)
+			hrtimer_reprogram(timer);
+		else
+			hrtimer_reprogram_softirq(timer);
+	}
 	unlock_hrtimer_base(timer, &flags);
 }
 EXPORT_SYMBOL_GPL(hrtimer_start_range_ns);
@@ -1206,7 +1262,8 @@ EXPORT_SYMBOL_GPL(hrtimer_active);
 
 static void __run_hrtimer(struct hrtimer_cpu_base *cpu_base,
 			  struct hrtimer_clock_base *base,
-			  struct hrtimer *timer, ktime_t *now)
+			  struct hrtimer *timer, ktime_t *now,
+			  bool hardirq)
 {
 	enum hrtimer_restart (*fn)(struct hrtimer *);
 	int restart;
@@ -1241,11 +1298,19 @@ static void __run_hrtimer(struct hrtimer
 	 * protected against migration to a different CPU even if the lock
 	 * is dropped.
 	 */
-	raw_spin_unlock(&cpu_base->lock);
+	if (hardirq)
+		raw_spin_unlock(&cpu_base->lock);
+	else
+		raw_spin_unlock_irq(&cpu_base->lock);
+
 	trace_hrtimer_expire_entry(timer, now);
 	restart = fn(timer);
 	trace_hrtimer_expire_exit(timer);
-	raw_spin_lock(&cpu_base->lock);
+
+	if (hardirq)
+		raw_spin_lock(&cpu_base->lock);
+	else
+		raw_spin_lock_irq(&cpu_base->lock);
 
 	/*
 	 * Note: We clear the running state after enqueue_hrtimer and
@@ -1309,11 +1374,28 @@ static void __hrtimer_run_queues(struct
 			if (basenow < hrtimer_get_softexpires_tv64(timer))
 				break;
 
-			__run_hrtimer(cpu_base, base, timer, &basenow);
+			__run_hrtimer(cpu_base, base, timer, &basenow,
+				      active_mask == HRTIMER_ACTIVE_HARD);
 		}
 	}
 }
 
+static __latent_entropy void hrtimer_run_softirq(struct softirq_action *h)
+{
+	struct hrtimer_cpu_base *cpu_base = this_cpu_ptr(&hrtimer_bases);
+	ktime_t now;
+
+	raw_spin_lock_irq(&cpu_base->lock);
+
+	now = hrtimer_update_base(cpu_base);
+	__hrtimer_run_queues(cpu_base, now, HRTIMER_ACTIVE_SOFT);
+
+	cpu_base->softirq_activated = 0;
+	hrtimer_update_softirq_timer(cpu_base, true);
+
+	raw_spin_unlock_irq(&cpu_base->lock);
+}
+
 #ifdef CONFIG_HIGH_RES_TIMERS
 
 /*
@@ -1343,9 +1425,15 @@ void hrtimer_interrupt(struct clock_even
 	 */
 	cpu_base->expires_next = KTIME_MAX;
 
+	if (!ktime_before(now, cpu_base->softirq_expires_next)) {
+		cpu_base->softirq_expires_next = KTIME_MAX;
+		cpu_base->softirq_activated = 1;
+		raise_softirq_irqoff(HRTIMER_SOFTIRQ);
+	}
+
 	__hrtimer_run_queues(cpu_base, now, HRTIMER_ACTIVE_HARD);
 
-	/* Reevaluate the clock bases for the next expiry */
+	/* Reevaluate the hard interrupt clock bases for the next expiry */
 	expires_next = __hrtimer_get_next_event(cpu_base);
 	/*
 	 * Store the new expiry value so the migration code can verify
@@ -1448,6 +1536,13 @@ void hrtimer_run_queues(void)
 
 	raw_spin_lock(&cpu_base->lock);
 	now = hrtimer_update_base(cpu_base);
+
+	if (!ktime_before(now, cpu_base->softirq_expires_next)) {
+		cpu_base->softirq_expires_next = KTIME_MAX;
+		cpu_base->softirq_activated = 1;
+		raise_softirq_irqoff(HRTIMER_SOFTIRQ);
+	}
+
 	__hrtimer_run_queues(cpu_base, now, HRTIMER_ACTIVE_HARD);
 	raw_spin_unlock(&cpu_base->lock);
 }
@@ -1629,6 +1724,7 @@ int hrtimers_prepare_cpu(unsigned int cp
 	cpu_base->cpu = cpu;
 	cpu_base->hres_active = 0;
 	cpu_base->expires_next = KTIME_MAX;
+	cpu_base->softirq_expires_next = KTIME_MAX;
 	return 0;
 }
 
@@ -1672,6 +1768,7 @@ int hrtimers_dead_cpu(unsigned int scpu)
 	BUG_ON(cpu_online(scpu));
 	tick_cancel_sched_timer(scpu);
 
+	local_bh_disable();
 	local_irq_disable();
 	old_base = &per_cpu(hrtimer_bases, scpu);
 	new_base = this_cpu_ptr(&hrtimer_bases);
@@ -1687,12 +1784,19 @@ int hrtimers_dead_cpu(unsigned int scpu)
 				     &new_base->clock_base[i]);
 	}
 
+	/*
+	 * The migration might have changed the first expiring softirq
+	 * timer on this CPU. Update it.
+	 */
+	hrtimer_update_softirq_timer(new_base, false);
+
 	raw_spin_unlock(&old_base->lock);
 	raw_spin_unlock(&new_base->lock);
 
 	/* Check, if we got expired work to do */
 	__hrtimer_peek_ahead_timers();
 	local_irq_enable();
+	local_bh_enable();
 	return 0;
 }
 
@@ -1707,6 +1811,7 @@ void __init hrtimers_init(void)
 	BUILD_BUG_ON_NOT_POWER_OF_2(HRTIMER_BASE_SOFT_MASK);
 
 	hrtimers_prepare_cpu(smp_processor_id());
+	open_softirq(HRTIMER_SOFTIRQ, hrtimer_run_softirq);
 }
 
 /**

Re: [PATCH 17/25] hrtimer: Implementation of softirq hrtimer handling

[Peter Zijlstra]

On Thu, Aug 31, 2017 at 12:23:42PM -0000, Anna-Maria Gleixner wrote:
> @@ -184,6 +188,7 @@ struct hrtimer_cpu_base {
>  	unsigned int			clock_was_set_seq;
>  	bool				migration_enabled;
>  	bool				nohz_active;
> +	bool				softirq_activated;
>  	unsigned int			hres_active	: 1,
>  					in_hrtirq	: 1,
>  					hang_detected	: 1;

Why do we have bools and a bitfield? Should that not all be part of the
bitfield?

Re: [PATCH 17/25] hrtimer: Implementation of softirq hrtimer handling

[Peter Zijlstra]

On Thu, Aug 31, 2017 at 12:23:42PM -0000, Anna-Maria Gleixner wrote:
>  static void __run_hrtimer(struct hrtimer_cpu_base *cpu_base,
>  			  struct hrtimer_clock_base *base,
> -			  struct hrtimer *timer, ktime_t *now)
> +			  struct hrtimer *timer, ktime_t *now,
> +			  bool hardirq)
>  {
>  	enum hrtimer_restart (*fn)(struct hrtimer *);
>  	int restart;
> @@ -1241,11 +1298,19 @@ static void __run_hrtimer(struct hrtimer
>  	 * protected against migration to a different CPU even if the lock
>  	 * is dropped.
>  	 */
> -	raw_spin_unlock(&cpu_base->lock);
> +	if (hardirq)
> +		raw_spin_unlock(&cpu_base->lock);
> +	else
> +		raw_spin_unlock_irq(&cpu_base->lock);
> +
>  	trace_hrtimer_expire_entry(timer, now);
>  	restart = fn(timer);
>  	trace_hrtimer_expire_exit(timer);
> -	raw_spin_lock(&cpu_base->lock);
> +
> +	if (hardirq)
> +		raw_spin_lock(&cpu_base->lock);
> +	else
> +		raw_spin_lock_irq(&cpu_base->lock);
>  

That's just nasty...

Re: [PATCH 17/25] hrtimer: Implementation of softirq hrtimer handling

[Anna-Maria Gleixner]

On Tue, 26 Sep 2017, Peter Zijlstra wrote:

> On Thu, Aug 31, 2017 at 12:23:42PM -0000, Anna-Maria Gleixner wrote:
> >  static void __run_hrtimer(struct hrtimer_cpu_base *cpu_base,
> >  			  struct hrtimer_clock_base *base,
> > -			  struct hrtimer *timer, ktime_t *now)
> > +			  struct hrtimer *timer, ktime_t *now,
> > +			  bool hardirq)
> >  {
> >  	enum hrtimer_restart (*fn)(struct hrtimer *);
> >  	int restart;
> > @@ -1241,11 +1298,19 @@ static void __run_hrtimer(struct hrtimer
> >  	 * protected against migration to a different CPU even if the lock
> >  	 * is dropped.
> >  	 */
> > -	raw_spin_unlock(&cpu_base->lock);
> > +	if (hardirq)
> > +		raw_spin_unlock(&cpu_base->lock);
> > +	else
> > +		raw_spin_unlock_irq(&cpu_base->lock);
> > +
> >  	trace_hrtimer_expire_entry(timer, now);
> >  	restart = fn(timer);
> >  	trace_hrtimer_expire_exit(timer);
> > -	raw_spin_lock(&cpu_base->lock);
> > +
> > +	if (hardirq)
> > +		raw_spin_lock(&cpu_base->lock);
> > +	else
> > +		raw_spin_lock_irq(&cpu_base->lock);
> >  
> 
> That's just nasty...
> 

I know and Thomas was unhappy about that as well, but we did not come
up with a better solution.

The nasty alternative is:

static void __run_hrtimer(struct hrtimer_cpu_base *cpu_base,
			  struct hrtimer_clock_base *base,
-			  struct hrtimer *timer, ktime_t *now)
+			  struct hrtimer *timer, ktime_t *now,
+                   	  unsigned long flags)
...
-	raw_spin_unlock(&cpu_base->lock);
+	raw_spin_unlock_irqrestore(&cpu_base->lock, flags);

...
-	raw_spin_lock(&cpu_base->lock);
+	raw_spin_lock_irq(&cpu_base->lock, flags);

and hand in flags from the callsites via local_save_flags().

We wanted to avoid the pointless lock_irq for the interrupt context,
but yes the conditional is equally bad.


Anna-Maria

Re: [PATCH 17/25] hrtimer: Implementation of softirq hrtimer handling

[Peter Zijlstra]

On Wed, Sep 27, 2017 at 04:22:39PM +0200, Anna-Maria Gleixner wrote:
> I know and Thomas was unhappy about that as well, but we did not come
> up with a better solution.
> 
> The nasty alternative is:
> 
> static void __run_hrtimer(struct hrtimer_cpu_base *cpu_base,
> 			  struct hrtimer_clock_base *base,
> -			  struct hrtimer *timer, ktime_t *now)
> +			  struct hrtimer *timer, ktime_t *now,
> +                   	  unsigned long flags)
> ...
> -	raw_spin_unlock(&cpu_base->lock);
> +	raw_spin_unlock_irqrestore(&cpu_base->lock, flags);
> 
> ...
> -	raw_spin_lock(&cpu_base->lock);
> +	raw_spin_lock_irq(&cpu_base->lock, flags);
> 
> and hand in flags from the callsites via local_save_flags().

Just push it up one more level and use raw_spin_lock_irqsave() in the
(soft)irq handlers and then pass the flags around.

Thta's what I did in that patch I just send.

Re: [PATCH 17/25] hrtimer: Implementation of softirq hrtimer handling

[Peter Zijlstra]

On Thu, Aug 31, 2017 at 12:23:42PM -0000, Anna-Maria Gleixner wrote:
> @@ -540,12 +539,23 @@ static ktime_t __hrtimer_get_next_event(
>  
>  	hrtimer_update_next_timer(cpu_base, NULL);
>  
> +	if (!cpu_base->softirq_activated) {
> +		active = cpu_base->active_bases & HRTIMER_ACTIVE_SOFT;
> +		expires_next = __hrtimer_next_event_base(cpu_base, active,
> +							 expires_next);
> +		cpu_base->softirq_expires_next = expires_next;
> +	}
> +

So this bugged the living daylight out of me. That's a very asymmetric
thing to do. The normal rule is that get_next_event() updates
cpu_base::next_timer and hrtimer_reprogram() updates
cpu_base::expires_next. And here you make a giant mess of things.

The below is a fairly large diff on top of this patch which is
_completely_ untested, but should hopefully restore sanity.

 - fixes the mixing of bool and bitfields in cpu_base
 - restores the whole get_next_timer() / reprogram() symmetry
   by adding cpu_base::softirq_next_timer.
 - adds a comment hopefully explaining the why of that
 - adds for_each_active_base() helper, we had two copies of that, and
   for a brief moment, I had another few, luckily those didn't survive
   :-)
 - uses irqsave/irqrestore for the cpu_base->lock fiddling around
   __run_hrtimer().
 - folded hrtimer_reprogram() and hrtimer_reprogram_softirq()
 - removed superfluous local_bh_disable(), since local_irq_disable()
   already implies much the same.

Please consider...

---
--- a/include/linux/hrtimer.h
+++ b/include/linux/hrtimer.h
@@ -186,21 +186,26 @@ struct hrtimer_cpu_base {
 	unsigned int			cpu;
 	unsigned int			active_bases;
 	unsigned int			clock_was_set_seq;
-	bool				migration_enabled;
-	bool				nohz_active;
-	bool				softirq_activated;
-	unsigned int			hres_active	: 1,
-					in_hrtirq	: 1,
-					hang_detected	: 1;
+
+	unsigned int			migration_enabled : 1;
+	unsigned int			nohz_active	  : 1;
+	unsigned int			softirq_activated : 1;
+	unsigned int			hres_active	  : 1;
+	unsigned int			in_hrtirq	  : 1;
+	unsigned int			hang_detected	  : 1;
+
 #ifdef CONFIG_HIGH_RES_TIMERS
 	unsigned int			nr_events;
 	unsigned int			nr_retries;
 	unsigned int			nr_hangs;
 	unsigned int			max_hang_time;
 #endif
+
 	ktime_t				expires_next;
 	struct hrtimer			*next_timer;
 	ktime_t				softirq_expires_next;
+	struct hrtimer			*softirq_next_timer;
+
 	struct hrtimer_clock_base	clock_base[HRTIMER_MAX_CLOCK_BASES];
 } ____cacheline_aligned;
 
--- a/kernel/time/hrtimer.c
+++ b/kernel/time/hrtimer.c
@@ -78,6 +78,7 @@
 #define MASK_SHIFT		(HRTIMER_BASE_MONOTONIC_SOFT)
 #define HRTIMER_ACTIVE_HARD	((1U << MASK_SHIFT) - 1)
 #define HRTIMER_ACTIVE_SOFT	(HRTIMER_ACTIVE_HARD << MASK_SHIFT)
+#define HRTIMER_ACTIVE		(HRTIMER_ACTIVE_SOFT | HRTIMER_ACTIVE_HARD)
 
 /*
  * The timer bases:
@@ -493,33 +494,49 @@ static inline void debug_deactivate(stru
 	trace_hrtimer_cancel(timer);
 }
 
-static inline void hrtimer_update_next_timer(struct hrtimer_cpu_base *cpu_base,
-					     struct hrtimer *timer)
+static inline bool hrtimer_is_softirq(struct hrtimer *timer)
 {
-	cpu_base->next_timer = timer;
+	return timer->base->index >= HRTIMER_BASE_MONOTONIC_SOFT;
+}
+
+
+static struct hrtimer_block_base *
+__next_base(struct hrtimer_cpu_base *cpu_base, unsigned int *active)
+{
+	struct hrtimer_clock_base *base = NULL;
+
+	if (*active) {
+		int idx = __ffs(*active);
+		*active &= (1U << idx);
+		base = cpu_base->clock_base[idx];
+	}
+
+	return base;
 }
 
+#define for_each_active_base(base, cpu_base, active)	\
+	while ((base = __next_base((cpu_base), &(active))))
+
 static ktime_t __hrtimer_next_event_base(struct hrtimer_cpu_base *cpu_base,
 					 unsigned int active,
 					 ktime_t expires_next)
 {
+	struct hrtimer_clock_base *base;
 	ktime_t expires;
 
-	while (active) {
-		unsigned int id = __ffs(active);
-		struct hrtimer_clock_base *base;
+	for_each_active_base(base, cpu_base, active) {
 		struct timerqueue_node *next;
 		struct hrtimer *timer;
 
-		active &= ~(1U << id);
-		base = cpu_base->clock_base + id;
-
 		next = timerqueue_getnext(&base->active);
 		timer = container_of(next, struct hrtimer, node);
 		expires = ktime_sub(hrtimer_get_expires(timer), base->offset);
 		if (expires < expires_next) {
 			expires_next = expires;
-			hrtimer_update_next_timer(cpu_base, timer);
+			if (hrtimer_is_softirq(timer))
+				cpu_base->softirq_next_timer = timer;
+			else
+				cpu_base->next_timer = timer;
 		}
 	}
 	/*
@@ -529,30 +546,47 @@ static ktime_t __hrtimer_next_event_base
 	 */
 	if (expires_next < 0)
 		expires_next = 0;
+
 	return expires_next;
 }
 
-static ktime_t __hrtimer_get_next_event(struct hrtimer_cpu_base *cpu_base)
+/*
+ * Recomputes cpu_base::*next_timer and returns the earliest expires_next but
+ * does not set cpu_base::*expires_next, that is done by hrtimer_reprogram.
+ *
+ * When a softirq is pending, we can ignore the HRTIMER_ACTIVE_SOFT bases,
+ * those timers will get run whenever the softirq gets handled, at the end of
+ * hrtimer_run_softirq(), hrtimer_update_softirq_timer() will re-add these bases.
+ *
+ * Therefore softirq values are those from the HRTIMER_ACTIVE_SOFT clock bases.
+ * The !softirq values are the minima across HRTIMER_ACTIVE, unless an actual
+ * softirq is pending, in which case they're the minima of HRTIMER_ACTIVE_HARD.
+ *
+ * @active_mask must be one of:
+ *  - HRTIMER_ACTIVE,
+ *  - HRTIMER_ACTIVE_SOFT, or
+ *  - HRTIMER_ACTIVE_HARD.
+ */
+static ktime_t
+__hrtimer_get_next_event(struct hrtimer_cpu_base *cpu_base, unsigned int active_mask)
 {
 	unsigned int active;
+	struct hrtimer *next_timer = NULL;
 	ktime_t expires_next = KTIME_MAX;
 
-	hrtimer_update_next_timer(cpu_base, NULL);
-
-	if (!cpu_base->softirq_activated) {
+	if (!cpu_base->softirq_activated && (active_mask & HRTIMER_ACTIVE_SOFT)) {
 		active = cpu_base->active_bases & HRTIMER_ACTIVE_SOFT;
-		expires_next = __hrtimer_next_event_base(cpu_base, active,
-							 expires_next);
-		cpu_base->softirq_expires_next = expires_next;
-	}
+		cpu_base->softirq_next_timer = next_timer;
+		expires_next = __hrtimer_next_event_base(cpu_base, active, expires_next);
 
-	active = cpu_base->active_bases & HRTIMER_ACTIVE_HARD;
-	expires_next = __hrtimer_next_event_base(cpu_base, active, expires_next);
+		next_timer = cpu_base->softirq_next_timer;
+	}
 
-	/*
-	 * cpu_base->expires_next is not updated here. It is set only
-	 * in hrtimer_reprogramming path!
-	 */
+	if (active_mask & HRTIMER_ACTIVE_HARD) {
+		active = cpu_base->active_bases & HRTIMER_ACTIVE_HARD;
+		cpu_base->next_timer = next_timer;
+		expires_next = __hrtimer_next_event_base(cpu_base, active, expires_next);
+	}
 
 	return expires_next;
 }
@@ -621,7 +655,10 @@ hrtimer_force_reprogram(struct hrtimer_c
 	if (!cpu_base->hres_active)
 		return;
 
-	expires_next = __hrtimer_get_next_event(cpu_base);
+	/*
+	 * Find the current next expiration time.
+	 */
+	expires_next = __hrtimer_get_next_event(cpu_base, HRTIMER_ACTIVE);
 
 	if (skip_equal && expires_next == cpu_base->expires_next)
 		return;
@@ -727,6 +764,20 @@ static void hrtimer_reprogram(struct hrt
 
 	WARN_ON_ONCE(hrtimer_get_expires_tv64(timer) < 0);
 
+	if (hrtimer_is_softirq(timer)) {
+		if (cpu_base->softirq_activated)
+			return;
+
+		if (!ktime_before(expires, cpu_base->softirq_expires_next))
+			return;
+
+		cpu_base->softirq_next_timer = timer;
+		cpu_base->softirq_expires_next = expires;
+
+		if (!ktime_before(expires, cpu_base->expires_next))
+			return;
+	}
+
 	/*
 	 * If the timer is not on the current cpu, we cannot reprogram
 	 * the other cpus clock event device.
@@ -755,7 +806,7 @@ static void hrtimer_reprogram(struct hrt
 		return;
 
 	/* Update the pointer to the next expiring timer */
-	hrtimer_update_next_timer(cpu_base, timer);
+	cpu_base->next_timer = timer;
 	cpu_base->expires_next = expires;
 
 	/*
@@ -979,47 +1030,24 @@ static inline ktime_t hrtimer_update_low
 	return tim;
 }
 
-static void hrtimer_reprogram_softirq(struct hrtimer *timer)
+static void
+hrtimer_update_softirq_timer(struct hrtimer_cpu_base *cpu_base, bool reprogram)
 {
-	struct hrtimer_clock_base *base = timer->base;
-	struct hrtimer_cpu_base *cpu_base = base->cpu_base;
 	ktime_t expires;
 
 	/*
-	 * The softirq timer is not rearmed, when the softirq was raised
-	 * and has not yet run to completion.
+	 * Find the next SOFT expiration.
 	 */
-	if (cpu_base->softirq_activated)
-		return;
-
-	expires = ktime_sub(hrtimer_get_expires(timer), base->offset);
-
-	if (!ktime_before(expires, cpu_base->softirq_expires_next))
-		return;
-
-	cpu_base->softirq_expires_next = expires;
-
-	if (!ktime_before(expires, cpu_base->expires_next))
-		return;
-	hrtimer_reprogram(timer);
-}
-
-static void hrtimer_update_softirq_timer(struct hrtimer_cpu_base *cpu_base,
-					 bool reprogram)
-{
-	ktime_t expires;
-
-	expires = __hrtimer_get_next_event(cpu_base);
+	expires = __hrtimer_get_next_event(cpu_base, HRTIMER_ACTIVE_SOFT);
 
 	if (!reprogram || !ktime_before(expires, cpu_base->expires_next))
 		return;
+
 	/*
-	 * next_timer can be used here, because
-	 * hrtimer_get_next_event() updated the next
-	 * timer. expires_next is only set when reprogramming function
-	 * is called.
+	 * cpu_base->*next_timer is recomputed by __hrtimer_get_next_event()
+	 * cpu_base->*expires_next is only set by hrtimer_reprogram()
 	 */
-	hrtimer_reprogram(cpu_base->next_timer);
+	hrtimer_reprogram(cpu_base->softirq_next_timer);
 }
 
 static int __hrtimer_start_range_ns(struct hrtimer *timer, ktime_t tim,
@@ -1060,12 +1088,9 @@ void hrtimer_start_range_ns(struct hrtim
 
 	base = lock_hrtimer_base(timer, &flags);
 
-	if (__hrtimer_start_range_ns(timer, tim, delta_ns, mode, base)) {
-		if (timer->base->index < HRTIMER_BASE_MONOTONIC_SOFT)
-			hrtimer_reprogram(timer);
-		else
-			hrtimer_reprogram_softirq(timer);
-	}
+	if (__hrtimer_start_range_ns(timer, tim, delta_ns, mode, base))
+		hrtimer_reprogram(timer);
+
 	unlock_hrtimer_base(timer, &flags);
 }
 EXPORT_SYMBOL_GPL(hrtimer_start_range_ns);
@@ -1163,7 +1188,7 @@ u64 hrtimer_get_next_event(void)
 	raw_spin_lock_irqsave(&cpu_base->lock, flags);
 
 	if (!__hrtimer_hres_active(cpu_base))
-		expires = __hrtimer_get_next_event(cpu_base);
+		expires = __hrtimer_get_next_event(cpu_base, HRTIMER_ACTIVE);
 
 	raw_spin_unlock_irqrestore(&cpu_base->lock, flags);
 
@@ -1263,7 +1288,7 @@ EXPORT_SYMBOL_GPL(hrtimer_active);
 static void __run_hrtimer(struct hrtimer_cpu_base *cpu_base,
 			  struct hrtimer_clock_base *base,
 			  struct hrtimer *timer, ktime_t *now,
-			  bool hardirq)
+			  unsigned long flags)
 {
 	enum hrtimer_restart (*fn)(struct hrtimer *);
 	int restart;
@@ -1298,19 +1323,13 @@ static void __run_hrtimer(struct hrtimer
 	 * protected against migration to a different CPU even if the lock
 	 * is dropped.
 	 */
-	if (hardirq)
-		raw_spin_unlock(&cpu_base->lock);
-	else
-		raw_spin_unlock_irq(&cpu_base->lock);
+	raw_spin_unlock_irqrestore(&cpu_base->lock, flags);
 
 	trace_hrtimer_expire_entry(timer, now);
 	restart = fn(timer);
 	trace_hrtimer_expire_exit(timer);
 
-	if (hardirq)
-		raw_spin_lock(&cpu_base->lock);
-	else
-		raw_spin_lock_irq(&cpu_base->lock);
+	raw_spin_lock_irqsave(&cpu_base->lock, flags);
 
 	/*
 	 * Note: We clear the running state after enqueue_hrtimer and
@@ -1339,19 +1358,15 @@ static void __run_hrtimer(struct hrtimer
 }
 
 static void __hrtimer_run_queues(struct hrtimer_cpu_base *cpu_base, ktime_t now,
-				 unsigned int active_mask)
+				 unsigned int active_mask, unsigned long flags)
 {
 	unsigned int active = cpu_base->active_bases & active_mask;
+	struct hrtimer_clock_base *base;
 
-	while (active) {
-		unsigned int id = __ffs(active);
-		struct hrtimer_clock_base *base;
+	for_each_active_base(base, cpu_base, active) {
 		struct timerqueue_node *node;
 		ktime_t basenow;
 
-		active &= ~(1U << id);
-		base = cpu_base->clock_base + id;
-
 		basenow = ktime_add(now, base->offset);
 
 		while ((node = timerqueue_getnext(&base->active))) {
@@ -1374,8 +1389,7 @@ static void __hrtimer_run_queues(struct
 			if (basenow < hrtimer_get_softexpires_tv64(timer))
 				break;
 
-			__run_hrtimer(cpu_base, base, timer, &basenow,
-				      active_mask == HRTIMER_ACTIVE_HARD);
+			__run_hrtimer(cpu_base, base, timer, &basenow, flags);
 		}
 	}
 }
@@ -1383,17 +1397,18 @@ static void __hrtimer_run_queues(struct
 static __latent_entropy void hrtimer_run_softirq(struct softirq_action *h)
 {
 	struct hrtimer_cpu_base *cpu_base = this_cpu_ptr(&hrtimer_bases);
+	unsigned long flags;
 	ktime_t now;
 
-	raw_spin_lock_irq(&cpu_base->lock);
+	raw_spin_lock_irqsave(&cpu_base->lock, flags);
 
 	now = hrtimer_update_base(cpu_base);
-	__hrtimer_run_queues(cpu_base, now, HRTIMER_ACTIVE_SOFT);
+	__hrtimer_run_queues(cpu_base, now, HRTIMER_ACTIVE_SOFT, flags);
 
 	cpu_base->softirq_activated = 0;
 	hrtimer_update_softirq_timer(cpu_base, true);
 
-	raw_spin_unlock_irq(&cpu_base->lock);
+	raw_spin_unlock_irqrestore(&cpu_base->lock, flags);
 }
 
 #ifdef CONFIG_HIGH_RES_TIMERS
@@ -1406,13 +1421,14 @@ void hrtimer_interrupt(struct clock_even
 {
 	struct hrtimer_cpu_base *cpu_base = this_cpu_ptr(&hrtimer_bases);
 	ktime_t expires_next, now, entry_time, delta;
+	unsigned long flags;
 	int retries = 0;
 
 	BUG_ON(!cpu_base->hres_active);
 	cpu_base->nr_events++;
 	dev->next_event = KTIME_MAX;
 
-	raw_spin_lock(&cpu_base->lock);
+	raw_spin_lock_irqsave(&cpu_base->lock, flags);
 	entry_time = now = hrtimer_update_base(cpu_base);
 retry:
 	cpu_base->in_hrtirq = 1;
@@ -1431,17 +1447,17 @@ void hrtimer_interrupt(struct clock_even
 		raise_softirq_irqoff(HRTIMER_SOFTIRQ);
 	}
 
-	__hrtimer_run_queues(cpu_base, now, HRTIMER_ACTIVE_HARD);
+	__hrtimer_run_queues(cpu_base, now, HRTIMER_ACTIVE_HARD, flags);
 
-	/* Reevaluate the hard interrupt clock bases for the next expiry */
-	expires_next = __hrtimer_get_next_event(cpu_base);
+	/* Reevaluate the clock bases for the next expiry */
+	expires_next = __hrtimer_get_next_event(cpu_base, HRTIMER_ACTIVE);
 	/*
 	 * Store the new expiry value so the migration code can verify
 	 * against it.
 	 */
 	cpu_base->expires_next = expires_next;
 	cpu_base->in_hrtirq = 0;
-	raw_spin_unlock(&cpu_base->lock);
+	raw_spin_unlock_irqrestore(&cpu_base->lock, flags);
 
 	/* Reprogramming necessary ? */
 	if (!tick_program_event(expires_next, 0)) {
@@ -1462,7 +1478,7 @@ void hrtimer_interrupt(struct clock_even
 	 * Acquire base lock for updating the offsets and retrieving
 	 * the current time.
 	 */
-	raw_spin_lock(&cpu_base->lock);
+	raw_spin_lock_irqsave(&cpu_base->lock, flags);
 	now = hrtimer_update_base(cpu_base);
 	cpu_base->nr_retries++;
 	if (++retries < 3)
@@ -1475,7 +1491,8 @@ void hrtimer_interrupt(struct clock_even
 	 */
 	cpu_base->nr_hangs++;
 	cpu_base->hang_detected = 1;
-	raw_spin_unlock(&cpu_base->lock);
+	raw_spin_unlock_irqrestore(&cpu_base->lock, flags);
+
 	delta = ktime_sub(now, entry_time);
 	if ((unsigned int)delta > cpu_base->max_hang_time)
 		cpu_base->max_hang_time = (unsigned int) delta;
@@ -1517,6 +1534,7 @@ static inline void __hrtimer_peek_ahead_
 void hrtimer_run_queues(void)
 {
 	struct hrtimer_cpu_base *cpu_base = this_cpu_ptr(&hrtimer_bases);
+	unsigned long flags;
 	ktime_t now;
 
 	if (__hrtimer_hres_active(cpu_base))
@@ -1534,7 +1552,7 @@ void hrtimer_run_queues(void)
 		return;
 	}
 
-	raw_spin_lock(&cpu_base->lock);
+	raw_spin_lock_irqsave(&cpu_base->lock, flags);
 	now = hrtimer_update_base(cpu_base);
 
 	if (!ktime_before(now, cpu_base->softirq_expires_next)) {
@@ -1543,8 +1561,8 @@ void hrtimer_run_queues(void)
 		raise_softirq_irqoff(HRTIMER_SOFTIRQ);
 	}
 
-	__hrtimer_run_queues(cpu_base, now, HRTIMER_ACTIVE_HARD);
-	raw_spin_unlock(&cpu_base->lock);
+	__hrtimer_run_queues(cpu_base, now, HRTIMER_ACTIVE_HARD, flags);
+	raw_spin_unlock_irqrestore(&cpu_base->lock, flags);
 }
 
 /*
@@ -1768,7 +1786,6 @@ int hrtimers_dead_cpu(unsigned int scpu)
 	BUG_ON(cpu_online(scpu));
 	tick_cancel_sched_timer(scpu);
 
-	local_bh_disable();
 	local_irq_disable();
 	old_base = &per_cpu(hrtimer_bases, scpu);
 	new_base = this_cpu_ptr(&hrtimer_bases);
@@ -1796,7 +1813,6 @@ int hrtimers_dead_cpu(unsigned int scpu)
 	/* Check, if we got expired work to do */
 	__hrtimer_peek_ahead_timers();
 	local_irq_enable();
-	local_bh_enable();
 	return 0;
 }
 

Re: [PATCH 17/25] hrtimer: Implementation of softirq hrtimer handling

[Thomas Gleixner]

On Wed, 27 Sep 2017, Peter Zijlstra wrote:
> On Thu, Aug 31, 2017 at 12:23:42PM -0000, Anna-Maria Gleixner wrote:
>  static void __run_hrtimer(struct hrtimer_cpu_base *cpu_base,
>  			  struct hrtimer_clock_base *base,
>  			  struct hrtimer *timer, ktime_t *now,
> -			  bool hardirq)
> +			  unsigned long flags)
>  {
>  	enum hrtimer_restart (*fn)(struct hrtimer *);
>  	int restart;
> @@ -1298,19 +1323,13 @@ static void __run_hrtimer(struct hrtimer
>  	 * protected against migration to a different CPU even if the lock
>  	 * is dropped.
>  	 */
> -	if (hardirq)
> -		raw_spin_unlock(&cpu_base->lock);
> -	else
> -		raw_spin_unlock_irq(&cpu_base->lock);
> +	raw_spin_unlock_irqrestore(&cpu_base->lock, flags);
>  
>  	trace_hrtimer_expire_entry(timer, now);
>  	restart = fn(timer);
>  	trace_hrtimer_expire_exit(timer);
>  
> -	if (hardirq)
> -		raw_spin_lock(&cpu_base->lock);
> -	else
> -		raw_spin_lock_irq(&cpu_base->lock);
> +	raw_spin_lock_irqsave(&cpu_base->lock, flags);

No point in irqsave() here. _irq() is sufficient as we leave the function
right after that.

Thanks,

	tglx

Re: [PATCH 17/25] hrtimer: Implementation of softirq hrtimer handling

[Peter Zijlstra]

On Thu, Sep 28, 2017 at 09:59:50AM +0200, Thomas Gleixner wrote:
> On Wed, 27 Sep 2017, Peter Zijlstra wrote:
> > On Thu, Aug 31, 2017 at 12:23:42PM -0000, Anna-Maria Gleixner wrote:
> >  static void __run_hrtimer(struct hrtimer_cpu_base *cpu_base,
> >  			  struct hrtimer_clock_base *base,
> >  			  struct hrtimer *timer, ktime_t *now,
> > -			  bool hardirq)
> > +			  unsigned long flags)
> >  {
> >  	enum hrtimer_restart (*fn)(struct hrtimer *);
> >  	int restart;
> > @@ -1298,19 +1323,13 @@ static void __run_hrtimer(struct hrtimer
> >  	 * protected against migration to a different CPU even if the lock
> >  	 * is dropped.
> >  	 */
> > -	if (hardirq)
> > -		raw_spin_unlock(&cpu_base->lock);
> > -	else
> > -		raw_spin_unlock_irq(&cpu_base->lock);
> > +	raw_spin_unlock_irqrestore(&cpu_base->lock, flags);
> >  
> >  	trace_hrtimer_expire_entry(timer, now);
> >  	restart = fn(timer);
> >  	trace_hrtimer_expire_exit(timer);
> >  
> > -	if (hardirq)
> > -		raw_spin_lock(&cpu_base->lock);
> > -	else
> > -		raw_spin_lock_irq(&cpu_base->lock);
> > +	raw_spin_lock_irqsave(&cpu_base->lock, flags);
> 
> No point in irqsave() here. _irq() is sufficient as we leave the function
> right after that.

True, I wasn't entirely sure on the irq tracing assumptions (if any) and
was too lazy to double check.

Re: [PATCH 17/25] hrtimer: Implementation of softirq hrtimer handling

[Sebastian Andrzej Siewior]

this is late I know…

On 2017-09-27 18:40:26 [+0200], Peter Zijlstra wrote:
>  - removed superfluous local_bh_disable(), since local_irq_disable()
>    already implies much the same.

it is not superfluous.

> Please consider...
> 
> @@ -1768,7 +1786,6 @@ int hrtimers_dead_cpu(unsigned int scpu)
>  	BUG_ON(cpu_online(scpu));
>  	tick_cancel_sched_timer(scpu);
>  
> -	local_bh_disable();
>  	local_irq_disable();
>  	old_base = &per_cpu(hrtimer_bases, scpu);
>  	new_base = this_cpu_ptr(&hrtimer_bases);
> @@ -1796,7 +1813,6 @@ int hrtimers_dead_cpu(unsigned int scpu)
>  	/* Check, if we got expired work to do */
>  	__hrtimer_peek_ahead_timers();
>  	local_irq_enable();
> -	local_bh_enable();
>  	return 0;
>  }

we need in there. That local_bh_disable() is required in order to let
raise_softirq_irqoff() not do anything stupid in particular we need
!in_interrupt() defer wakeup_softirqd() until local_bh_enable().
Otherwise wakeup_softirqd() might actually try to wakeup the process and
go after the pi_lock which can't happen while holding cpu_base->lock.

Sebastian

Re: [PATCH 17/25] hrtimer: Implementation of softirq hrtimer handling

[Peter Zijlstra]

On Tue, Dec 19, 2017 at 09:58:44AM +0100, Sebastian Andrzej Siewior wrote:
> this is late I know…
> 
> On 2017-09-27 18:40:26 [+0200], Peter Zijlstra wrote:
> >  - removed superfluous local_bh_disable(), since local_irq_disable()
> >    already implies much the same.
> 
> it is not superfluous.
> 
> > Please consider...
> > 
> > @@ -1768,7 +1786,6 @@ int hrtimers_dead_cpu(unsigned int scpu)
> >  	BUG_ON(cpu_online(scpu));
> >  	tick_cancel_sched_timer(scpu);
> >  
> > -	local_bh_disable();
> >  	local_irq_disable();
> >  	old_base = &per_cpu(hrtimer_bases, scpu);
> >  	new_base = this_cpu_ptr(&hrtimer_bases);
> > @@ -1796,7 +1813,6 @@ int hrtimers_dead_cpu(unsigned int scpu)
> >  	/* Check, if we got expired work to do */
> >  	__hrtimer_peek_ahead_timers();
> >  	local_irq_enable();
> > -	local_bh_enable();
> >  	return 0;
> >  }
> 
> we need in there. That local_bh_disable() is required in order to let
> raise_softirq_irqoff() not do anything stupid in particular we need
> !in_interrupt() defer wakeup_softirqd() until local_bh_enable().
> Otherwise wakeup_softirqd() might actually try to wakeup the process and
> go after the pi_lock which can't happen while holding cpu_base->lock.

Argh, that's horrible and definitely needs a comment.

Re: [PATCH 17/25] hrtimer: Implementation of softirq hrtimer handling

[Anna-Maria Gleixner]

[-- Attachment #1: Type: text/plain, Size: 1420 bytes --]

On Tue, 19 Dec 2017, Peter Zijlstra wrote:

> On Tue, Dec 19, 2017 at 09:58:44AM +0100, Sebastian Andrzej Siewior wrote:
> > this is late I know…
> > 
> > On 2017-09-27 18:40:26 [+0200], Peter Zijlstra wrote:
> > >  - removed superfluous local_bh_disable(), since local_irq_disable()
> > >    already implies much the same.
> > 
> > it is not superfluous.
> > 
> > > Please consider...
> > > 
> > > @@ -1768,7 +1786,6 @@ int hrtimers_dead_cpu(unsigned int scpu)
> > >  	BUG_ON(cpu_online(scpu));
> > >  	tick_cancel_sched_timer(scpu);
> > >  
> > > -	local_bh_disable();
> > >  	local_irq_disable();
> > >  	old_base = &per_cpu(hrtimer_bases, scpu);
> > >  	new_base = this_cpu_ptr(&hrtimer_bases);
> > > @@ -1796,7 +1813,6 @@ int hrtimers_dead_cpu(unsigned int scpu)
> > >  	/* Check, if we got expired work to do */
> > >  	__hrtimer_peek_ahead_timers();
> > >  	local_irq_enable();
> > > -	local_bh_enable();
> > >  	return 0;
> > >  }
> > 
> > we need in there. That local_bh_disable() is required in order to let
> > raise_softirq_irqoff() not do anything stupid in particular we need
> > !in_interrupt() defer wakeup_softirqd() until local_bh_enable().
> > Otherwise wakeup_softirqd() might actually try to wakeup the process and
> > go after the pi_lock which can't happen while holding cpu_base->lock.
> 
> Argh, that's horrible and definitely needs a comment.
> 

will add it in v4 with a comment.

Anna-Maria

[PATCH 20/25] mac80211_hwsim: Replace hrtimer tasklet with softirq hrtimer

[Anna-Maria Gleixner]

From: Thomas Gleixner <tglx@linutronix.de>

Switch the timer to CLOCK_MONOTONIC_SOFT, which executed the timer
callback in softirq context and remove the hrtimer_tasklet.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Anna-Maria Gleixner <anna-maria@linutronix.de>
Cc: Johannes Berg <johannes@sipsolutions.net>
Cc: Kalle Valo <kvalo@codeaurora.org>
Cc: linux-wireless@vger.kernel.org

---
 drivers/net/wireless/mac80211_hwsim.c |   44 +++++++++++++++-------------------
 1 file changed, 20 insertions(+), 24 deletions(-)

--- a/drivers/net/wireless/mac80211_hwsim.c
+++ b/drivers/net/wireless/mac80211_hwsim.c
@@ -537,7 +537,7 @@ struct mac80211_hwsim_data {
 	unsigned int rx_filter;
 	bool started, idle, scanning;
 	struct mutex mutex;
-	struct tasklet_hrtimer beacon_timer;
+	struct hrtimer beacon_timer;
 	enum ps_mode {
 		PS_DISABLED, PS_ENABLED, PS_AUTO_POLL, PS_MANUAL_POLL
 	} ps;
@@ -1420,7 +1420,7 @@ static void mac80211_hwsim_stop(struct i
 {
 	struct mac80211_hwsim_data *data = hw->priv;
 	data->started = false;
-	tasklet_hrtimer_cancel(&data->beacon_timer);
+	hrtimer_cancel(&data->beacon_timer);
 	wiphy_debug(hw->wiphy, "%s\n", __func__);
 }
 
@@ -1543,14 +1543,12 @@ static enum hrtimer_restart
 mac80211_hwsim_beacon(struct hrtimer *timer)
 {
 	struct mac80211_hwsim_data *data =
-		container_of(timer, struct mac80211_hwsim_data,
-			     beacon_timer.timer);
+		container_of(timer, struct mac80211_hwsim_data, beacon_timer);
 	struct ieee80211_hw *hw = data->hw;
 	u64 bcn_int = data->beacon_int;
-	ktime_t next_bcn;
 
 	if (!data->started)
-		goto out;
+		return HRTIMER_NORESTART;
 
 	ieee80211_iterate_active_interfaces_atomic(
 		hw, IEEE80211_IFACE_ITER_NORMAL,
@@ -1562,11 +1560,9 @@ mac80211_hwsim_beacon(struct hrtimer *ti
 		data->bcn_delta = 0;
 	}
 
-	next_bcn = ktime_add(hrtimer_get_expires(timer),
-			     ns_to_ktime(bcn_int * 1000));
-	tasklet_hrtimer_start(&data->beacon_timer, next_bcn, HRTIMER_MODE_ABS);
-out:
-	return HRTIMER_NORESTART;
+	hrtimer_forward(&data->beacon_timer, hrtimer_get_expires(timer),
+			ns_to_ktime(bcn_int * NSEC_PER_USEC));
+	return HRTIMER_RESTART;
 }
 
 static const char * const hwsim_chanwidths[] = {
@@ -1640,15 +1636,15 @@ static int mac80211_hwsim_config(struct
 	mutex_unlock(&data->mutex);
 
 	if (!data->started || !data->beacon_int)
-		tasklet_hrtimer_cancel(&data->beacon_timer);
-	else if (!hrtimer_is_queued(&data->beacon_timer.timer)) {
+		hrtimer_cancel(&data->beacon_timer);
+	else if (!hrtimer_is_queued(&data->beacon_timer)) {
 		u64 tsf = mac80211_hwsim_get_tsf(hw, NULL);
 		u32 bcn_int = data->beacon_int;
 		u64 until_tbtt = bcn_int - do_div(tsf, bcn_int);
 
-		tasklet_hrtimer_start(&data->beacon_timer,
-				      ns_to_ktime(until_tbtt * 1000),
-				      HRTIMER_MODE_REL);
+		hrtimer_start(&data->beacon_timer,
+			      ns_to_ktime(until_tbtt * 1000),
+			      HRTIMER_MODE_REL);
 	}
 
 	return 0;
@@ -1711,7 +1707,7 @@ static void mac80211_hwsim_bss_info_chan
 			    info->enable_beacon, info->beacon_int);
 		vp->bcn_en = info->enable_beacon;
 		if (data->started &&
-		    !hrtimer_is_queued(&data->beacon_timer.timer) &&
+		    !hrtimer_is_queued(&data->beacon_timer) &&
 		    info->enable_beacon) {
 			u64 tsf, until_tbtt;
 			u32 bcn_int;
@@ -1719,9 +1715,9 @@ static void mac80211_hwsim_bss_info_chan
 			tsf = mac80211_hwsim_get_tsf(hw, vif);
 			bcn_int = data->beacon_int;
 			until_tbtt = bcn_int - do_div(tsf, bcn_int);
-			tasklet_hrtimer_start(&data->beacon_timer,
-					      ns_to_ktime(until_tbtt * 1000),
-					      HRTIMER_MODE_REL);
+			hrtimer_start(&data->beacon_timer,
+				      ns_to_ktime(until_tbtt * 1000),
+				      HRTIMER_MODE_REL);
 		} else if (!info->enable_beacon) {
 			unsigned int count = 0;
 			ieee80211_iterate_active_interfaces_atomic(
@@ -1730,7 +1726,7 @@ static void mac80211_hwsim_bss_info_chan
 			wiphy_debug(hw->wiphy, "  beaconing vifs remaining: %u",
 				    count);
 			if (count == 0) {
-				tasklet_hrtimer_cancel(&data->beacon_timer);
+				hrtimer_cancel(&data->beacon_timer);
 				data->beacon_int = 0;
 			}
 		}
@@ -2722,9 +2718,9 @@ static int mac80211_hwsim_new_radio(stru
 				    data->debugfs,
 				    data, &hwsim_simulate_radar);
 
-	tasklet_hrtimer_init(&data->beacon_timer,
-			     mac80211_hwsim_beacon,
-			     CLOCK_MONOTONIC, HRTIMER_MODE_ABS);
+	hrtimer_init(&data->beacon_timer, CLOCK_MONOTONIC_SOFT,
+		     HRTIMER_MODE_ABS);
+	data->beacon_timer.function = mac80211_hwsim_beacon;
 
 	spin_lock_bh(&hwsim_radio_lock);
 	list_add_tail(&data->list, &hwsim_radios);

[PATCH 20/25] mac80211_hwsim: Replace hrtimer tasklet with softirq hrtimer

[Anna-Maria Gleixner]

[-- Attachment #1: mac80211_hwsim_Replace_hrtimer_tasklet_with_softirq_hrtimer.patch --]
[-- Type: text/plain, Size: 4567 bytes --]

From: Thomas Gleixner <tglx@linutronix.de>

Switch the timer to CLOCK_MONOTONIC_SOFT, which executed the timer
callback in softirq context and remove the hrtimer_tasklet.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Anna-Maria Gleixner <anna-maria@linutronix.de>
Cc: Johannes Berg <johannes@sipsolutions.net>
Cc: Kalle Valo <kvalo@codeaurora.org>
Cc: linux-wireless@vger.kernel.org

---
 drivers/net/wireless/mac80211_hwsim.c |   44 +++++++++++++++-------------------
 1 file changed, 20 insertions(+), 24 deletions(-)

--- a/drivers/net/wireless/mac80211_hwsim.c
+++ b/drivers/net/wireless/mac80211_hwsim.c
@@ -537,7 +537,7 @@ struct mac80211_hwsim_data {
 	unsigned int rx_filter;
 	bool started, idle, scanning;
 	struct mutex mutex;
-	struct tasklet_hrtimer beacon_timer;
+	struct hrtimer beacon_timer;
 	enum ps_mode {
 		PS_DISABLED, PS_ENABLED, PS_AUTO_POLL, PS_MANUAL_POLL
 	} ps;
@@ -1420,7 +1420,7 @@ static void mac80211_hwsim_stop(struct i
 {
 	struct mac80211_hwsim_data *data = hw->priv;
 	data->started = false;
-	tasklet_hrtimer_cancel(&data->beacon_timer);
+	hrtimer_cancel(&data->beacon_timer);
 	wiphy_debug(hw->wiphy, "%s\n", __func__);
 }
 
@@ -1543,14 +1543,12 @@ static enum hrtimer_restart
 mac80211_hwsim_beacon(struct hrtimer *timer)
 {
 	struct mac80211_hwsim_data *data =
-		container_of(timer, struct mac80211_hwsim_data,
-			     beacon_timer.timer);
+		container_of(timer, struct mac80211_hwsim_data, beacon_timer);
 	struct ieee80211_hw *hw = data->hw;
 	u64 bcn_int = data->beacon_int;
-	ktime_t next_bcn;
 
 	if (!data->started)
-		goto out;
+		return HRTIMER_NORESTART;
 
 	ieee80211_iterate_active_interfaces_atomic(
 		hw, IEEE80211_IFACE_ITER_NORMAL,
@@ -1562,11 +1560,9 @@ mac80211_hwsim_beacon(struct hrtimer *ti
 		data->bcn_delta = 0;
 	}
 
-	next_bcn = ktime_add(hrtimer_get_expires(timer),
-			     ns_to_ktime(bcn_int * 1000));
-	tasklet_hrtimer_start(&data->beacon_timer, next_bcn, HRTIMER_MODE_ABS);
-out:
-	return HRTIMER_NORESTART;
+	hrtimer_forward(&data->beacon_timer, hrtimer_get_expires(timer),
+			ns_to_ktime(bcn_int * NSEC_PER_USEC));
+	return HRTIMER_RESTART;
 }
 
 static const char * const hwsim_chanwidths[] = {
@@ -1640,15 +1636,15 @@ static int mac80211_hwsim_config(struct
 	mutex_unlock(&data->mutex);
 
 	if (!data->started || !data->beacon_int)
-		tasklet_hrtimer_cancel(&data->beacon_timer);
-	else if (!hrtimer_is_queued(&data->beacon_timer.timer)) {
+		hrtimer_cancel(&data->beacon_timer);
+	else if (!hrtimer_is_queued(&data->beacon_timer)) {
 		u64 tsf = mac80211_hwsim_get_tsf(hw, NULL);
 		u32 bcn_int = data->beacon_int;
 		u64 until_tbtt = bcn_int - do_div(tsf, bcn_int);
 
-		tasklet_hrtimer_start(&data->beacon_timer,
-				      ns_to_ktime(until_tbtt * 1000),
-				      HRTIMER_MODE_REL);
+		hrtimer_start(&data->beacon_timer,
+			      ns_to_ktime(until_tbtt * 1000),
+			      HRTIMER_MODE_REL);
 	}
 
 	return 0;
@@ -1711,7 +1707,7 @@ static void mac80211_hwsim_bss_info_chan
 			    info->enable_beacon, info->beacon_int);
 		vp->bcn_en = info->enable_beacon;
 		if (data->started &&
-		    !hrtimer_is_queued(&data->beacon_timer.timer) &&
+		    !hrtimer_is_queued(&data->beacon_timer) &&
 		    info->enable_beacon) {
 			u64 tsf, until_tbtt;
 			u32 bcn_int;
@@ -1719,9 +1715,9 @@ static void mac80211_hwsim_bss_info_chan
 			tsf = mac80211_hwsim_get_tsf(hw, vif);
 			bcn_int = data->beacon_int;
 			until_tbtt = bcn_int - do_div(tsf, bcn_int);
-			tasklet_hrtimer_start(&data->beacon_timer,
-					      ns_to_ktime(until_tbtt * 1000),
-					      HRTIMER_MODE_REL);
+			hrtimer_start(&data->beacon_timer,
+				      ns_to_ktime(until_tbtt * 1000),
+				      HRTIMER_MODE_REL);
 		} else if (!info->enable_beacon) {
 			unsigned int count = 0;
 			ieee80211_iterate_active_interfaces_atomic(
@@ -1730,7 +1726,7 @@ static void mac80211_hwsim_bss_info_chan
 			wiphy_debug(hw->wiphy, "  beaconing vifs remaining: %u",
 				    count);
 			if (count == 0) {
-				tasklet_hrtimer_cancel(&data->beacon_timer);
+				hrtimer_cancel(&data->beacon_timer);
 				data->beacon_int = 0;
 			}
 		}
@@ -2722,9 +2718,9 @@ static int mac80211_hwsim_new_radio(stru
 				    data->debugfs,
 				    data, &hwsim_simulate_radar);
 
-	tasklet_hrtimer_init(&data->beacon_timer,
-			     mac80211_hwsim_beacon,
-			     CLOCK_MONOTONIC, HRTIMER_MODE_ABS);
+	hrtimer_init(&data->beacon_timer, CLOCK_MONOTONIC_SOFT,
+		     HRTIMER_MODE_ABS);
+	data->beacon_timer.function = mac80211_hwsim_beacon;
 
 	spin_lock_bh(&hwsim_radio_lock);
 	list_add_tail(&data->list, &hwsim_radios);

Re: [PATCH 20/25] mac80211_hwsim: Replace hrtimer tasklet with softirq hrtimer

[Johannes Berg]

On Thu, 2017-08-31 at 12:23 +0000, Anna-Maria Gleixner wrote:
> From: Thomas Gleixner <tglx@linutronix.de>
> 
> Switch the timer to CLOCK_MONOTONIC_SOFT, which executed the timer
> callback in softirq context and remove the hrtimer_tasklet.
> 
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> Signed-off-by: Anna-Maria Gleixner <anna-maria@linutronix.de>
> Cc: Johannes Berg <johannes@sipsolutions.net>
> Cc: Kalle Valo <kvalo@codeaurora.org>
> Cc: linux-wireless@vger.kernel.org
> 
This looks fine to me,

Reviewed-by: Johannes Berg <johannes@sipsolutions.net>

Are you planning to integrate all patches in the series through some
other tree, perhaps to be able to get rid of the tasklet_hrtimer API,
or should I apply this?

johannes

Re: [PATCH 20/25] mac80211_hwsim: Replace hrtimer tasklet with softirq hrtimer

[Thomas Gleixner]

On Tue, 5 Sep 2017, Johannes Berg wrote:
> On Thu, 2017-08-31 at 12:23 +0000, Anna-Maria Gleixner wrote:
> > From: Thomas Gleixner <tglx@linutronix.de>
> > 
> > Switch the timer to CLOCK_MONOTONIC_SOFT, which executed the timer
> > callback in softirq context and remove the hrtimer_tasklet.
> > 
> > Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> > Signed-off-by: Anna-Maria Gleixner <anna-maria@linutronix.de>
> > Cc: Johannes Berg <johannes@sipsolutions.net>
> > Cc: Kalle Valo <kvalo@codeaurora.org>
> > Cc: linux-wireless@vger.kernel.org
> > 
> This looks fine to me,
> 
> Reviewed-by: Johannes Berg <johannes@sipsolutions.net>
> 
> Are you planning to integrate all patches in the series through some
> other tree, perhaps to be able to get rid of the tasklet_hrtimer API,
> or should I apply this?

The patch depends on the hrtimer core changes, so we either delay the
removal for a release cycle or just take the whole lot through
tip:timers/core and get rid of it in one go.

Thanks,

	tglx

Re: [PATCH 20/25] mac80211_hwsim: Replace hrtimer tasklet with softirq hrtimer

[Johannes Berg]

On Tue, 2017-09-05 at 10:49 +0200, Thomas Gleixner wrote:
> 
> > Are you planning to integrate all patches in the series through
> > some other tree, perhaps to be able to get rid of the
> > tasklet_hrtimer API, or should I apply this?
> 
> The patch depends on the hrtimer core changes, so we either delay the
> removal for a release cycle or just take the whole lot through
> tip:timers/core and get rid of it in one go.

Ok. For now then I'll assume that it'll go through tip; if not please
resend when it's appropriate.

Thanks,
johannes

[PATCH 19/25] can/bcm: Replace hrtimer_tasklet with softirq based hrtimer

[Anna-Maria Gleixner]

[-- Attachment #1: canbcm_Replace_hrtimer_tasklet_with_softirq_based_hrtimer.patch --]
[-- Type: text/plain, Size: 9061 bytes --]

From: Thomas Gleixner <tglx@linutronix.de>

Switch the timer to CLOCK_MONOTONIC_SOFT, which executed the timer
callback in softirq context and remove the hrtimer_tasklet.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Anna-Maria Gleixner <anna-maria@linutronix.de>
Cc: Oliver Hartkopp <socketcan@hartkopp.net>
Cc: Marc Kleine-Budde <mkl@pengutronix.de>
Cc: linux-can@vger.kernel.org
---
 net/can/bcm.c |  150 ++++++++++++++++++----------------------------------------
 1 file changed, 49 insertions(+), 101 deletions(-)

--- a/net/can/bcm.c
+++ b/net/can/bcm.c
@@ -102,7 +102,6 @@ struct bcm_op {
 	unsigned long frames_abs, frames_filtered;
 	struct bcm_timeval ival1, ival2;
 	struct hrtimer timer, thrtimer;
-	struct tasklet_struct tsklet, thrtsklet;
 	ktime_t rx_stamp, kt_ival1, kt_ival2, kt_lastmsg;
 	int rx_ifindex;
 	int cfsiz;
@@ -364,25 +363,34 @@ static void bcm_send_to_user(struct bcm_
 	}
 }
 
-static void bcm_tx_start_timer(struct bcm_op *op)
+static bool bcm_tx_set_expiry(struct bcm_op *op, struct hrtimer *hrt)
 {
+	ktime_t ival;
+
 	if (op->kt_ival1 && op->count)
-		hrtimer_start(&op->timer,
-			      ktime_add(ktime_get(), op->kt_ival1),
-			      HRTIMER_MODE_ABS);
+		ival = op->kt_ival1;
 	else if (op->kt_ival2)
-		hrtimer_start(&op->timer,
-			      ktime_add(ktime_get(), op->kt_ival2),
-			      HRTIMER_MODE_ABS);
+		ival = op->kt_ival2;
+	else
+		return false;
+
+	hrtimer_set_expires(hrt, ktime_add(ktime_get(), ival));
+	return true;
 }
 
-static void bcm_tx_timeout_tsklet(unsigned long data)
+static void bcm_tx_start_timer(struct bcm_op *op)
 {
-	struct bcm_op *op = (struct bcm_op *)data;
+	if (bcm_tx_set_expiry(op, &op->timer))
+		hrtimer_start_expires(&op->timer, HRTIMER_MODE_ABS);
+}
+
+/* bcm_tx_timeout_handler - performs cyclic CAN frame transmissions */
+static enum hrtimer_restart bcm_tx_timeout_handler(struct hrtimer *hrtimer)
+{
+	struct bcm_op *op = container_of(hrtimer, struct bcm_op, timer);
 	struct bcm_msg_head msg_head;
 
 	if (op->kt_ival1 && (op->count > 0)) {
-
 		op->count--;
 		if (!op->count && (op->flags & TX_COUNTEVT)) {
 
@@ -399,22 +407,12 @@ static void bcm_tx_timeout_tsklet(unsign
 		}
 		bcm_can_tx(op);
 
-	} else if (op->kt_ival2)
+	} else if (op->kt_ival2) {
 		bcm_can_tx(op);
+	}
 
-	bcm_tx_start_timer(op);
-}
-
-/*
- * bcm_tx_timeout_handler - performs cyclic CAN frame transmissions
- */
-static enum hrtimer_restart bcm_tx_timeout_handler(struct hrtimer *hrtimer)
-{
-	struct bcm_op *op = container_of(hrtimer, struct bcm_op, timer);
-
-	tasklet_schedule(&op->tsklet);
-
-	return HRTIMER_NORESTART;
+	return bcm_tx_set_expiry(op, &op->timer) ?
+		HRTIMER_RESTART : HRTIMER_NORESTART;
 }
 
 /*
@@ -542,11 +540,18 @@ static void bcm_rx_starttimer(struct bcm
 		hrtimer_start(&op->timer, op->kt_ival1, HRTIMER_MODE_REL);
 }
 
-static void bcm_rx_timeout_tsklet(unsigned long data)
+/* bcm_rx_timeout_handler - when the (cyclic) CAN frame reception timed out */
+static enum hrtimer_restart bcm_rx_timeout_handler(struct hrtimer *hrtimer)
 {
-	struct bcm_op *op = (struct bcm_op *)data;
+	struct bcm_op *op = container_of(hrtimer, struct bcm_op, timer);
 	struct bcm_msg_head msg_head;
 
+	/* if user wants to be informed, when cyclic CAN-Messages come back */
+	if ((op->flags & RX_ANNOUNCE_RESUME) && op->last_frames) {
+		/* clear received CAN frames to indicate 'nothing received' */
+		memset(op->last_frames, 0, op->nframes * op->cfsiz);
+	}
+
 	/* create notification to user */
 	msg_head.opcode  = RX_TIMEOUT;
 	msg_head.flags   = op->flags;
@@ -557,25 +562,6 @@ static void bcm_rx_timeout_tsklet(unsign
 	msg_head.nframes = 0;
 
 	bcm_send_to_user(op, &msg_head, NULL, 0);
-}
-
-/*
- * bcm_rx_timeout_handler - when the (cyclic) CAN frame reception timed out
- */
-static enum hrtimer_restart bcm_rx_timeout_handler(struct hrtimer *hrtimer)
-{
-	struct bcm_op *op = container_of(hrtimer, struct bcm_op, timer);
-
-	/* schedule before NET_RX_SOFTIRQ */
-	tasklet_hi_schedule(&op->tsklet);
-
-	/* no restart of the timer is done here! */
-
-	/* if user wants to be informed, when cyclic CAN-Messages come back */
-	if ((op->flags & RX_ANNOUNCE_RESUME) && op->last_frames) {
-		/* clear received CAN frames to indicate 'nothing received' */
-		memset(op->last_frames, 0, op->nframes * op->cfsiz);
-	}
 
 	return HRTIMER_NORESTART;
 }
@@ -583,14 +569,12 @@ static enum hrtimer_restart bcm_rx_timeo
 /*
  * bcm_rx_do_flush - helper for bcm_rx_thr_flush
  */
-static inline int bcm_rx_do_flush(struct bcm_op *op, int update,
-				  unsigned int index)
+static inline int bcm_rx_do_flush(struct bcm_op *op, unsigned int index)
 {
 	struct canfd_frame *lcf = op->last_frames + op->cfsiz * index;
 
 	if ((op->last_frames) && (lcf->flags & RX_THR)) {
-		if (update)
-			bcm_rx_changed(op, lcf);
+		bcm_rx_changed(op, lcf);
 		return 1;
 	}
 	return 0;
@@ -598,11 +582,8 @@ static inline int bcm_rx_do_flush(struct
 
 /*
  * bcm_rx_thr_flush - Check for throttled data and send it to the userspace
- *
- * update == 0 : just check if throttled data is available  (any irq context)
- * update == 1 : check and send throttled data to userspace (soft_irq context)
  */
-static int bcm_rx_thr_flush(struct bcm_op *op, int update)
+static int bcm_rx_thr_flush(struct bcm_op *op)
 {
 	int updated = 0;
 
@@ -611,24 +592,16 @@ static int bcm_rx_thr_flush(struct bcm_o
 
 		/* for MUX filter we start at index 1 */
 		for (i = 1; i < op->nframes; i++)
-			updated += bcm_rx_do_flush(op, update, i);
+			updated += bcm_rx_do_flush(op, i);
 
 	} else {
 		/* for RX_FILTER_ID and simple filter */
-		updated += bcm_rx_do_flush(op, update, 0);
+		updated += bcm_rx_do_flush(op, 0);
 	}
 
 	return updated;
 }
 
-static void bcm_rx_thr_tsklet(unsigned long data)
-{
-	struct bcm_op *op = (struct bcm_op *)data;
-
-	/* push the changed data to the userspace */
-	bcm_rx_thr_flush(op, 1);
-}
-
 /*
  * bcm_rx_thr_handler - the time for blocked content updates is over now:
  *                      Check for throttled data and send it to the userspace
@@ -637,9 +610,7 @@ static enum hrtimer_restart bcm_rx_thr_h
 {
 	struct bcm_op *op = container_of(hrtimer, struct bcm_op, thrtimer);
 
-	tasklet_schedule(&op->thrtsklet);
-
-	if (bcm_rx_thr_flush(op, 0)) {
+	if (bcm_rx_thr_flush(op)) {
 		hrtimer_forward(hrtimer, ktime_get(), op->kt_ival2);
 		return HRTIMER_RESTART;
 	} else {
@@ -735,23 +706,8 @@ static struct bcm_op *bcm_find_op(struct
 
 static void bcm_remove_op(struct bcm_op *op)
 {
-	if (op->tsklet.func) {
-		while (test_bit(TASKLET_STATE_SCHED, &op->tsklet.state) ||
-		       test_bit(TASKLET_STATE_RUN, &op->tsklet.state) ||
-		       hrtimer_active(&op->timer)) {
-			hrtimer_cancel(&op->timer);
-			tasklet_kill(&op->tsklet);
-		}
-	}
-
-	if (op->thrtsklet.func) {
-		while (test_bit(TASKLET_STATE_SCHED, &op->thrtsklet.state) ||
-		       test_bit(TASKLET_STATE_RUN, &op->thrtsklet.state) ||
-		       hrtimer_active(&op->thrtimer)) {
-			hrtimer_cancel(&op->thrtimer);
-			tasklet_kill(&op->thrtsklet);
-		}
-	}
+	hrtimer_cancel(&op->timer);
+	hrtimer_cancel(&op->thrtimer);
 
 	if ((op->frames) && (op->frames != &op->sframe))
 		kfree(op->frames);
@@ -979,15 +935,13 @@ static int bcm_tx_setup(struct bcm_msg_h
 		op->ifindex = ifindex;
 
 		/* initialize uninitialized (kzalloc) structure */
-		hrtimer_init(&op->timer, CLOCK_MONOTONIC, HRTIMER_MODE_REL);
+		hrtimer_init(&op->timer, CLOCK_MONOTONIC_SOFT,
+			     HRTIMER_MODE_REL);
 		op->timer.function = bcm_tx_timeout_handler;
 
-		/* initialize tasklet for tx countevent notification */
-		tasklet_init(&op->tsklet, bcm_tx_timeout_tsklet,
-			     (unsigned long) op);
-
 		/* currently unused in tx_ops */
-		hrtimer_init(&op->thrtimer, CLOCK_MONOTONIC, HRTIMER_MODE_REL);
+		hrtimer_init(&op->thrtimer, CLOCK_MONOTONIC_SOFT,
+			     HRTIMER_MODE_REL);
 
 		/* add this bcm_op to the list of the tx_ops */
 		list_add(&op->list, &bo->tx_ops);
@@ -1150,20 +1104,14 @@ static int bcm_rx_setup(struct bcm_msg_h
 		op->rx_ifindex = ifindex;
 
 		/* initialize uninitialized (kzalloc) structure */
-		hrtimer_init(&op->timer, CLOCK_MONOTONIC, HRTIMER_MODE_REL);
+		hrtimer_init(&op->timer, CLOCK_MONOTONIC_SOFT,
+			     HRTIMER_MODE_REL);
 		op->timer.function = bcm_rx_timeout_handler;
 
-		/* initialize tasklet for rx timeout notification */
-		tasklet_init(&op->tsklet, bcm_rx_timeout_tsklet,
-			     (unsigned long) op);
-
-		hrtimer_init(&op->thrtimer, CLOCK_MONOTONIC, HRTIMER_MODE_REL);
+		hrtimer_init(&op->thrtimer, CLOCK_MONOTONIC_SOFT,
+			     HRTIMER_MODE_REL);
 		op->thrtimer.function = bcm_rx_thr_handler;
 
-		/* initialize tasklet for rx throttle handling */
-		tasklet_init(&op->thrtsklet, bcm_rx_thr_tsklet,
-			     (unsigned long) op);
-
 		/* add this bcm_op to the list of the rx_ops */
 		list_add(&op->list, &bo->rx_ops);
 
@@ -1209,7 +1157,7 @@ static int bcm_rx_setup(struct bcm_msg_h
 			 */
 			op->kt_lastmsg = 0;
 			hrtimer_cancel(&op->thrtimer);
-			bcm_rx_thr_flush(op, 1);
+			bcm_rx_thr_flush(op);
 		}
 
 		if ((op->flags & STARTTIMER) && op->kt_ival1)

Re: [PATCH 19/25] can/bcm: Replace hrtimer_tasklet with softirq based hrtimer

[Oliver Hartkopp]

Hi Anna-Maria & Thomas,

thanks for the effort!

I'm really happy that the hrtimer becomes softirq capable as it 
basically reverts this ugly commit from January 2009:

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit?id=6e5c172cf7ca1ab878cc6a6a4c1d52fef60f3ee0

 From the first look the changes seem to be correct.

I'll pick the entire patchset for a test and give a Acked|Tested-by then.

Best regards,
Oliver

ps.

On 08/31/2017 02:23 PM, Anna-Maria Gleixner wrote:

(nothing here!)

Please send patches inline and not as attachments. Tnx!

Re: [PATCH 19/25] can/bcm: Replace hrtimer_tasklet with softirq based hrtimer

[Thomas Gleixner]

On Fri, 1 Sep 2017, Oliver Hartkopp wrote:

> Hi Anna-Maria & Thomas,
> 
> thanks for the effort!
> 
> I'm really happy that the hrtimer becomes softirq capable as it basically
> reverts this ugly commit from January 2009:
> 
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit?id=6e5c172cf7ca1ab878cc6a6a4c1d52fef60f3ee0
> 
> From the first look the changes seem to be correct.
> 
> I'll pick the entire patchset for a test and give a Acked|Tested-by then.
> 
> Best regards,
> Oliver
> 
> ps.
> 
> On 08/31/2017 02:23 PM, Anna-Maria Gleixner wrote:
> 
> (nothing here!)
> 
> Please send patches inline and not as attachments. Tnx!

PS:

Content-Type: text/plain; charset=UTF-8
Content-Disposition: inline;
 filename=canbcm_Replace_hrtimer_tasklet_with_softirq_based_hrtimer.patch

Please get a proper mail client, which actually parses the
Content-Disposition tag correcly

Thanks,

	tglx

Re: [PATCH 19/25] can/bcm: Replace hrtimer_tasklet with softirq based hrtimer

[Oliver Hartkopp]

On 09/01/2017 05:56 PM, Thomas Gleixner wrote:
> On Fri, 1 Sep 2017, Oliver Hartkopp wrote:

>>
>> On 08/31/2017 02:23 PM, Anna-Maria Gleixner wrote:
>>
>> (nothing here!)
>>
>> Please send patches inline and not as attachments. Tnx!
> 
> PS:
> 
> Content-Type: text/plain; charset=UTF-8
> Content-Disposition: inline;
>   filename=canbcm_Replace_hrtimer_tasklet_with_softirq_based_hrtimer.patch
> 
> Please get a proper mail client, which actually parses the
> Content-Disposition tag correcly
> 

Aah!

Setting

	mail.reply_quote_inline = true

in the config editor fixes that issue when replying a quilt post with 
Thunderbird.

Tnx!

Best,
Oliver

Re: [PATCH 19/25] can/bcm: Replace hrtimer_tasklet with softirq based hrtimer

[Oliver Hartkopp]

On 08/31/2017 02:23 PM, Anna-Maria Gleixner wrote:
> From: Thomas Gleixner <tglx@linutronix.de>
> 
> Switch the timer to CLOCK_MONOTONIC_SOFT, which executed the timer
> callback in softirq context and remove the hrtimer_tasklet.
> 
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> Signed-off-by: Anna-Maria Gleixner <anna-maria@linutronix.de>
> Cc: Oliver Hartkopp <socketcan@hartkopp.net>

Acked-by: Oliver Hartkopp <socketcan@hartkopp.net>

Thanks,
Oliver

> Cc: Marc Kleine-Budde <mkl@pengutronix.de>
> Cc: linux-can@vger.kernel.org
> ---
>   net/can/bcm.c |  150 ++++++++++++++++++----------------------------------------
>   1 file changed, 49 insertions(+), 101 deletions(-)
> 
> --- a/net/can/bcm.c
> +++ b/net/can/bcm.c
> @@ -102,7 +102,6 @@ struct bcm_op {
>   	unsigned long frames_abs, frames_filtered;
>   	struct bcm_timeval ival1, ival2;
>   	struct hrtimer timer, thrtimer;
> -	struct tasklet_struct tsklet, thrtsklet;
>   	ktime_t rx_stamp, kt_ival1, kt_ival2, kt_lastmsg;
>   	int rx_ifindex;
>   	int cfsiz;
> @@ -364,25 +363,34 @@ static void bcm_send_to_user(struct bcm_
>   	}
>   }
>   
> -static void bcm_tx_start_timer(struct bcm_op *op)
> +static bool bcm_tx_set_expiry(struct bcm_op *op, struct hrtimer *hrt)
>   {
> +	ktime_t ival;
> +
>   	if (op->kt_ival1 && op->count)
> -		hrtimer_start(&op->timer,
> -			      ktime_add(ktime_get(), op->kt_ival1),
> -			      HRTIMER_MODE_ABS);
> +		ival = op->kt_ival1;
>   	else if (op->kt_ival2)
> -		hrtimer_start(&op->timer,
> -			      ktime_add(ktime_get(), op->kt_ival2),
> -			      HRTIMER_MODE_ABS);
> +		ival = op->kt_ival2;
> +	else
> +		return false;
> +
> +	hrtimer_set_expires(hrt, ktime_add(ktime_get(), ival));
> +	return true;
>   }
>   
> -static void bcm_tx_timeout_tsklet(unsigned long data)
> +static void bcm_tx_start_timer(struct bcm_op *op)
>   {
> -	struct bcm_op *op = (struct bcm_op *)data;
> +	if (bcm_tx_set_expiry(op, &op->timer))
> +		hrtimer_start_expires(&op->timer, HRTIMER_MODE_ABS);
> +}
> +
> +/* bcm_tx_timeout_handler - performs cyclic CAN frame transmissions */
> +static enum hrtimer_restart bcm_tx_timeout_handler(struct hrtimer *hrtimer)
> +{
> +	struct bcm_op *op = container_of(hrtimer, struct bcm_op, timer);
>   	struct bcm_msg_head msg_head;
>   
>   	if (op->kt_ival1 && (op->count > 0)) {
> -
>   		op->count--;
>   		if (!op->count && (op->flags & TX_COUNTEVT)) {
>   
> @@ -399,22 +407,12 @@ static void bcm_tx_timeout_tsklet(unsign
>   		}
>   		bcm_can_tx(op);
>   
> -	} else if (op->kt_ival2)
> +	} else if (op->kt_ival2) {
>   		bcm_can_tx(op);
> +	}
>   
> -	bcm_tx_start_timer(op);
> -}
> -
> -/*
> - * bcm_tx_timeout_handler - performs cyclic CAN frame transmissions
> - */
> -static enum hrtimer_restart bcm_tx_timeout_handler(struct hrtimer *hrtimer)
> -{
> -	struct bcm_op *op = container_of(hrtimer, struct bcm_op, timer);
> -
> -	tasklet_schedule(&op->tsklet);
> -
> -	return HRTIMER_NORESTART;
> +	return bcm_tx_set_expiry(op, &op->timer) ?
> +		HRTIMER_RESTART : HRTIMER_NORESTART;
>   }
>   
>   /*
> @@ -542,11 +540,18 @@ static void bcm_rx_starttimer(struct bcm
>   		hrtimer_start(&op->timer, op->kt_ival1, HRTIMER_MODE_REL);
>   }
>   
> -static void bcm_rx_timeout_tsklet(unsigned long data)
> +/* bcm_rx_timeout_handler - when the (cyclic) CAN frame reception timed out */
> +static enum hrtimer_restart bcm_rx_timeout_handler(struct hrtimer *hrtimer)
>   {
> -	struct bcm_op *op = (struct bcm_op *)data;
> +	struct bcm_op *op = container_of(hrtimer, struct bcm_op, timer);
>   	struct bcm_msg_head msg_head;
>   
> +	/* if user wants to be informed, when cyclic CAN-Messages come back */
> +	if ((op->flags & RX_ANNOUNCE_RESUME) && op->last_frames) {
> +		/* clear received CAN frames to indicate 'nothing received' */
> +		memset(op->last_frames, 0, op->nframes * op->cfsiz);
> +	}
> +
>   	/* create notification to user */
>   	msg_head.opcode  = RX_TIMEOUT;
>   	msg_head.flags   = op->flags;
> @@ -557,25 +562,6 @@ static void bcm_rx_timeout_tsklet(unsign
>   	msg_head.nframes = 0;
>   
>   	bcm_send_to_user(op, &msg_head, NULL, 0);
> -}
> -
> -/*
> - * bcm_rx_timeout_handler - when the (cyclic) CAN frame reception timed out
> - */
> -static enum hrtimer_restart bcm_rx_timeout_handler(struct hrtimer *hrtimer)
> -{
> -	struct bcm_op *op = container_of(hrtimer, struct bcm_op, timer);
> -
> -	/* schedule before NET_RX_SOFTIRQ */
> -	tasklet_hi_schedule(&op->tsklet);
> -
> -	/* no restart of the timer is done here! */
> -
> -	/* if user wants to be informed, when cyclic CAN-Messages come back */
> -	if ((op->flags & RX_ANNOUNCE_RESUME) && op->last_frames) {
> -		/* clear received CAN frames to indicate 'nothing received' */
> -		memset(op->last_frames, 0, op->nframes * op->cfsiz);
> -	}
>   
>   	return HRTIMER_NORESTART;
>   }
> @@ -583,14 +569,12 @@ static enum hrtimer_restart bcm_rx_timeo
>   /*
>    * bcm_rx_do_flush - helper for bcm_rx_thr_flush
>    */
> -static inline int bcm_rx_do_flush(struct bcm_op *op, int update,
> -				  unsigned int index)
> +static inline int bcm_rx_do_flush(struct bcm_op *op, unsigned int index)
>   {
>   	struct canfd_frame *lcf = op->last_frames + op->cfsiz * index;
>   
>   	if ((op->last_frames) && (lcf->flags & RX_THR)) {
> -		if (update)
> -			bcm_rx_changed(op, lcf);
> +		bcm_rx_changed(op, lcf);
>   		return 1;
>   	}
>   	return 0;
> @@ -598,11 +582,8 @@ static inline int bcm_rx_do_flush(struct
>   
>   /*
>    * bcm_rx_thr_flush - Check for throttled data and send it to the userspace
> - *
> - * update == 0 : just check if throttled data is available  (any irq context)
> - * update == 1 : check and send throttled data to userspace (soft_irq context)
>    */
> -static int bcm_rx_thr_flush(struct bcm_op *op, int update)
> +static int bcm_rx_thr_flush(struct bcm_op *op)
>   {
>   	int updated = 0;
>   
> @@ -611,24 +592,16 @@ static int bcm_rx_thr_flush(struct bcm_o
>   
>   		/* for MUX filter we start at index 1 */
>   		for (i = 1; i < op->nframes; i++)
> -			updated += bcm_rx_do_flush(op, update, i);
> +			updated += bcm_rx_do_flush(op, i);
>   
>   	} else {
>   		/* for RX_FILTER_ID and simple filter */
> -		updated += bcm_rx_do_flush(op, update, 0);
> +		updated += bcm_rx_do_flush(op, 0);
>   	}
>   
>   	return updated;
>   }
>   
> -static void bcm_rx_thr_tsklet(unsigned long data)
> -{
> -	struct bcm_op *op = (struct bcm_op *)data;
> -
> -	/* push the changed data to the userspace */
> -	bcm_rx_thr_flush(op, 1);
> -}
> -
>   /*
>    * bcm_rx_thr_handler - the time for blocked content updates is over now:
>    *                      Check for throttled data and send it to the userspace
> @@ -637,9 +610,7 @@ static enum hrtimer_restart bcm_rx_thr_h
>   {
>   	struct bcm_op *op = container_of(hrtimer, struct bcm_op, thrtimer);
>   
> -	tasklet_schedule(&op->thrtsklet);
> -
> -	if (bcm_rx_thr_flush(op, 0)) {
> +	if (bcm_rx_thr_flush(op)) {
>   		hrtimer_forward(hrtimer, ktime_get(), op->kt_ival2);
>   		return HRTIMER_RESTART;
>   	} else {
> @@ -735,23 +706,8 @@ static struct bcm_op *bcm_find_op(struct
>   
>   static void bcm_remove_op(struct bcm_op *op)
>   {
> -	if (op->tsklet.func) {
> -		while (test_bit(TASKLET_STATE_SCHED, &op->tsklet.state) ||
> -		       test_bit(TASKLET_STATE_RUN, &op->tsklet.state) ||
> -		       hrtimer_active(&op->timer)) {
> -			hrtimer_cancel(&op->timer);
> -			tasklet_kill(&op->tsklet);
> -		}
> -	}
> -
> -	if (op->thrtsklet.func) {
> -		while (test_bit(TASKLET_STATE_SCHED, &op->thrtsklet.state) ||
> -		       test_bit(TASKLET_STATE_RUN, &op->thrtsklet.state) ||
> -		       hrtimer_active(&op->thrtimer)) {
> -			hrtimer_cancel(&op->thrtimer);
> -			tasklet_kill(&op->thrtsklet);
> -		}
> -	}
> +	hrtimer_cancel(&op->timer);
> +	hrtimer_cancel(&op->thrtimer);
>   
>   	if ((op->frames) && (op->frames != &op->sframe))
>   		kfree(op->frames);
> @@ -979,15 +935,13 @@ static int bcm_tx_setup(struct bcm_msg_h
>   		op->ifindex = ifindex;
>   
>   		/* initialize uninitialized (kzalloc) structure */
> -		hrtimer_init(&op->timer, CLOCK_MONOTONIC, HRTIMER_MODE_REL);
> +		hrtimer_init(&op->timer, CLOCK_MONOTONIC_SOFT,
> +			     HRTIMER_MODE_REL);
>   		op->timer.function = bcm_tx_timeout_handler;
>   
> -		/* initialize tasklet for tx countevent notification */
> -		tasklet_init(&op->tsklet, bcm_tx_timeout_tsklet,
> -			     (unsigned long) op);
> -
>   		/* currently unused in tx_ops */
> -		hrtimer_init(&op->thrtimer, CLOCK_MONOTONIC, HRTIMER_MODE_REL);
> +		hrtimer_init(&op->thrtimer, CLOCK_MONOTONIC_SOFT,
> +			     HRTIMER_MODE_REL);
>   
>   		/* add this bcm_op to the list of the tx_ops */
>   		list_add(&op->list, &bo->tx_ops);
> @@ -1150,20 +1104,14 @@ static int bcm_rx_setup(struct bcm_msg_h
>   		op->rx_ifindex = ifindex;
>   
>   		/* initialize uninitialized (kzalloc) structure */
> -		hrtimer_init(&op->timer, CLOCK_MONOTONIC, HRTIMER_MODE_REL);
> +		hrtimer_init(&op->timer, CLOCK_MONOTONIC_SOFT,
> +			     HRTIMER_MODE_REL);
>   		op->timer.function = bcm_rx_timeout_handler;
>   
> -		/* initialize tasklet for rx timeout notification */
> -		tasklet_init(&op->tsklet, bcm_rx_timeout_tsklet,
> -			     (unsigned long) op);
> -
> -		hrtimer_init(&op->thrtimer, CLOCK_MONOTONIC, HRTIMER_MODE_REL);
> +		hrtimer_init(&op->thrtimer, CLOCK_MONOTONIC_SOFT,
> +			     HRTIMER_MODE_REL);
>   		op->thrtimer.function = bcm_rx_thr_handler;
>   
> -		/* initialize tasklet for rx throttle handling */
> -		tasklet_init(&op->thrtsklet, bcm_rx_thr_tsklet,
> -			     (unsigned long) op);
> -
>   		/* add this bcm_op to the list of the rx_ops */
>   		list_add(&op->list, &bo->rx_ops);
>   
> @@ -1209,7 +1157,7 @@ static int bcm_rx_setup(struct bcm_msg_h
>   			 */
>   			op->kt_lastmsg = 0;
>   			hrtimer_cancel(&op->thrtimer);
> -			bcm_rx_thr_flush(op, 1);
> +			bcm_rx_thr_flush(op);
>   		}
>   
>   		if ((op->flags & STARTTIMER) && op->kt_ival1)
> 
> 

[PATCH 21/25] xfrm: Replace hrtimer tasklet with softirq hrtimer

[Anna-Maria Gleixner]

[-- Attachment #1: xfrm_Replace_hrtimer_tasklet_with_softirq_hrtimer.patch --]
[-- Type: text/plain, Size: 4743 bytes --]

From: Thomas Gleixner <tglx@linutronix.de>

Switch the timer to CLOCK_MONOTONIC_SOFT, which executed the timer
callback in softirq context and remove the hrtimer_tasklet.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Anna-Maria Gleixner <anna-maria@linutronix.de>
Cc: Steffen Klassert <steffen.klassert@secunet.com>
Cc: Herbert Xu <herbert@gondor.apana.org.au>
Cc: "David S. Miller" <davem@davemloft.net>
Cc: netdev@vger.kernel.org

---
 include/net/xfrm.h    |    2 +-
 net/xfrm/xfrm_state.c |   29 +++++++++++++++++------------
 2 files changed, 18 insertions(+), 13 deletions(-)

--- a/include/net/xfrm.h
+++ b/include/net/xfrm.h
@@ -213,7 +213,7 @@ struct xfrm_state {
 	struct xfrm_stats	stats;
 
 	struct xfrm_lifetime_cur curlft;
-	struct tasklet_hrtimer	mtimer;
+	struct hrtimer		mtimer;
 
 	struct xfrm_state_offload xso;
 
--- a/net/xfrm/xfrm_state.c
+++ b/net/xfrm/xfrm_state.c
@@ -418,7 +418,7 @@ static void xfrm_put_mode(struct xfrm_mo
 
 static void xfrm_state_gc_destroy(struct xfrm_state *x)
 {
-	tasklet_hrtimer_cancel(&x->mtimer);
+	hrtimer_cancel(&x->mtimer);
 	del_timer_sync(&x->rtimer);
 	kfree(x->aead);
 	kfree(x->aalg);
@@ -463,8 +463,8 @@ static void xfrm_state_gc_task(struct wo
 
 static enum hrtimer_restart xfrm_timer_handler(struct hrtimer *me)
 {
-	struct tasklet_hrtimer *thr = container_of(me, struct tasklet_hrtimer, timer);
-	struct xfrm_state *x = container_of(thr, struct xfrm_state, mtimer);
+	struct xfrm_state *x = container_of(me, struct xfrm_state, mtimer);
+	enum hrtimer_restart ret = HRTIMER_NORESTART;
 	unsigned long now = get_seconds();
 	long next = LONG_MAX;
 	int warn = 0;
@@ -528,7 +528,8 @@ static enum hrtimer_restart xfrm_timer_h
 		km_state_expired(x, 0, 0);
 resched:
 	if (next != LONG_MAX) {
-		tasklet_hrtimer_start(&x->mtimer, ktime_set(next, 0), HRTIMER_MODE_REL);
+		hrtimer_forward_now(&x->mtimer, ktime_set(next, 0));
+		ret = HRTIMER_RESTART;
 	}
 
 	goto out;
@@ -545,7 +546,7 @@ static enum hrtimer_restart xfrm_timer_h
 
 out:
 	spin_unlock(&x->lock);
-	return HRTIMER_NORESTART;
+	return ret;
 }
 
 static void xfrm_replay_timer_handler(unsigned long data);
@@ -564,8 +565,8 @@ struct xfrm_state *xfrm_state_alloc(stru
 		INIT_HLIST_NODE(&x->bydst);
 		INIT_HLIST_NODE(&x->bysrc);
 		INIT_HLIST_NODE(&x->byspi);
-		tasklet_hrtimer_init(&x->mtimer, xfrm_timer_handler,
-					CLOCK_BOOTTIME, HRTIMER_MODE_ABS);
+		hrtimer_init(&x->mtimer, CLOCK_BOOTTIME_SOFT, HRTIMER_MODE_ABS);
+		x->mtimer.function = xfrm_timer_handler;
 		setup_timer(&x->rtimer, xfrm_replay_timer_handler,
 				(unsigned long)x);
 		x->curlft.add_time = get_seconds();
@@ -1021,7 +1022,9 @@ xfrm_state_find(const xfrm_address_t *da
 				hlist_add_head_rcu(&x->byspi, net->xfrm.state_byspi + h);
 			}
 			x->lft.hard_add_expires_seconds = net->xfrm.sysctl_acq_expires;
-			tasklet_hrtimer_start(&x->mtimer, ktime_set(net->xfrm.sysctl_acq_expires, 0), HRTIMER_MODE_REL);
+			hrtimer_start(&x->mtimer,
+				      ktime_set(net->xfrm.sysctl_acq_expires, 0),
+				      HRTIMER_MODE_REL);
 			net->xfrm.state_num++;
 			xfrm_hash_grow_check(net, x->bydst.next != NULL);
 			spin_unlock_bh(&net->xfrm.xfrm_state_lock);
@@ -1132,7 +1135,7 @@ static void __xfrm_state_insert(struct x
 		hlist_add_head_rcu(&x->byspi, net->xfrm.state_byspi + h);
 	}
 
-	tasklet_hrtimer_start(&x->mtimer, ktime_set(1, 0), HRTIMER_MODE_REL);
+	hrtimer_start(&x->mtimer, ktime_set(1, 0), HRTIMER_MODE_REL);
 	if (x->replay_maxage)
 		mod_timer(&x->rtimer, jiffies + x->replay_maxage);
 
@@ -1236,7 +1239,9 @@ static struct xfrm_state *__find_acq_cor
 		x->mark.m = m->m;
 		x->lft.hard_add_expires_seconds = net->xfrm.sysctl_acq_expires;
 		xfrm_state_hold(x);
-		tasklet_hrtimer_start(&x->mtimer, ktime_set(net->xfrm.sysctl_acq_expires, 0), HRTIMER_MODE_REL);
+		hrtimer_start(&x->mtimer,
+			      ktime_set(net->xfrm.sysctl_acq_expires, 0),
+			      HRTIMER_MODE_REL);
 		list_add(&x->km.all, &net->xfrm.state_all);
 		hlist_add_head_rcu(&x->bydst, net->xfrm.state_bydst + h);
 		h = xfrm_src_hash(net, daddr, saddr, family);
@@ -1535,7 +1540,7 @@ int xfrm_state_update(struct xfrm_state
 		memcpy(&x1->lft, &x->lft, sizeof(x1->lft));
 		x1->km.dying = 0;
 
-		tasklet_hrtimer_start(&x1->mtimer, ktime_set(1, 0), HRTIMER_MODE_REL);
+		hrtimer_start(&x1->mtimer, ktime_set(1, 0), HRTIMER_MODE_REL);
 		if (x1->curlft.use_time)
 			xfrm_state_check_expire(x1);
 
@@ -1559,7 +1564,7 @@ int xfrm_state_check_expire(struct xfrm_
 	if (x->curlft.bytes >= x->lft.hard_byte_limit ||
 	    x->curlft.packets >= x->lft.hard_packet_limit) {
 		x->km.state = XFRM_STATE_EXPIRED;
-		tasklet_hrtimer_start(&x->mtimer, 0, HRTIMER_MODE_REL);
+		hrtimer_start(&x->mtimer, 0, HRTIMER_MODE_REL);
 		return -EINVAL;
 	}
 

[PATCH 23/25] ALSA/dummy: Replace tasklet with softirq hrtimer

[Anna-Maria Gleixner]

[-- Attachment #1: ALSAdummy_Replace_tasklet_with_softirq_hrtimer.patch --]
[-- Type: text/plain, Size: 2276 bytes --]

From: Thomas Gleixner <tglx@linutronix.de>

The tasklet is used to defer the execution of snd_pcm_period_elapsed() to
the softirq context. Using the CLOCK_MONOTONIC_SOFT base invokes the timer
callback in softirq context as well which renders the tasklet useless.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Anna-Maria Gleixner <anna-maria@linutronix.de>
Cc: Jaroslav Kysela <perex@perex.cz>
Cc: Takashi Iwai <tiwai@suse.com>
Cc: Takashi Sakamoto <o-takashi@sakamocchi.jp>
Cc: alsa-devel@alsa-project.org
---
 sound/drivers/dummy.c |   16 +++-------------
 1 file changed, 3 insertions(+), 13 deletions(-)

--- a/sound/drivers/dummy.c
+++ b/sound/drivers/dummy.c
@@ -376,17 +376,9 @@ struct dummy_hrtimer_pcm {
 	ktime_t period_time;
 	atomic_t running;
 	struct hrtimer timer;
-	struct tasklet_struct tasklet;
 	struct snd_pcm_substream *substream;
 };
 
-static void dummy_hrtimer_pcm_elapsed(unsigned long priv)
-{
-	struct dummy_hrtimer_pcm *dpcm = (struct dummy_hrtimer_pcm *)priv;
-	if (atomic_read(&dpcm->running))
-		snd_pcm_period_elapsed(dpcm->substream);
-}
-
 static enum hrtimer_restart dummy_hrtimer_callback(struct hrtimer *timer)
 {
 	struct dummy_hrtimer_pcm *dpcm;
@@ -394,7 +386,8 @@ static enum hrtimer_restart dummy_hrtime
 	dpcm = container_of(timer, struct dummy_hrtimer_pcm, timer);
 	if (!atomic_read(&dpcm->running))
 		return HRTIMER_NORESTART;
-	tasklet_schedule(&dpcm->tasklet);
+
+	snd_pcm_period_elapsed(dpcm->substream);
 	hrtimer_forward_now(timer, dpcm->period_time);
 	return HRTIMER_RESTART;
 }
@@ -421,7 +414,6 @@ static int dummy_hrtimer_stop(struct snd
 static inline void dummy_hrtimer_sync(struct dummy_hrtimer_pcm *dpcm)
 {
 	hrtimer_cancel(&dpcm->timer);
-	tasklet_kill(&dpcm->tasklet);
 }
 
 static snd_pcm_uframes_t
@@ -466,12 +458,10 @@ static int dummy_hrtimer_create(struct s
 	if (!dpcm)
 		return -ENOMEM;
 	substream->runtime->private_data = dpcm;
-	hrtimer_init(&dpcm->timer, CLOCK_MONOTONIC, HRTIMER_MODE_REL);
+	hrtimer_init(&dpcm->timer, CLOCK_MONOTONIC_SOFT, HRTIMER_MODE_REL);
 	dpcm->timer.function = dummy_hrtimer_callback;
 	dpcm->substream = substream;
 	atomic_set(&dpcm->running, 0);
-	tasklet_init(&dpcm->tasklet, dummy_hrtimer_pcm_elapsed,
-		     (unsigned long)dpcm);
 	return 0;
 }
 

Re: [PATCH 23/25] ALSA/dummy: Replace tasklet with softirq hrtimer

[Takashi Sakamoto]

Hi,

On Aug 31 2017 21:23, Anna-Maria Gleixner wrote:
> From: Thomas Gleixner <tglx@linutronix.de>
> 
> The tasklet is used to defer the execution of snd_pcm_period_elapsed() to
> the softirq context. Using the CLOCK_MONOTONIC_SOFT base invokes the timer
> callback in softirq context as well which renders the tasklet useless.
> 
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> Signed-off-by: Anna-Maria Gleixner <anna-maria@linutronix.de>
> Cc: Jaroslav Kysela <perex@perex.cz>
> Cc: Takashi Iwai <tiwai@suse.com>
> Cc: Takashi Sakamoto <o-takashi@sakamocchi.jp>
> Cc: alsa-devel@alsa-project.org
> ---
>  sound/drivers/dummy.c |   16 +++-------------
>  1 file changed, 3 insertions(+), 13 deletions(-)

I prefer this patch as long as this driver can still receive callbacks
from hrtimer subsystem.

Reviewed-by: Takashi Sakamoto <o-takashi@sakamocchi.jp>

Unfortunately, I have too poor machine to compile whole kernel now, thus
didn't do any tests, sorry.

I note that ALSA pcsp driver uses a combination of hrtimer/tasklet for the
same purpose. I think we can simplify it, too. Please refer to a patch in
the end of this message. (But not tested yet for the above reason...)

> --- a/sound/drivers/dummy.c
> +++ b/sound/drivers/dummy.c
> @@ -376,17 +376,9 @@ struct dummy_hrtimer_pcm {
>  	ktime_t period_time;
>  	atomic_t running;
>  	struct hrtimer timer;
> -	struct tasklet_struct tasklet;
>  	struct snd_pcm_substream *substream;
>  };
>  
> -static void dummy_hrtimer_pcm_elapsed(unsigned long priv)
> -{
> -	struct dummy_hrtimer_pcm *dpcm = (struct dummy_hrtimer_pcm *)priv;
> -	if (atomic_read(&dpcm->running))
> -		snd_pcm_period_elapsed(dpcm->substream);
> -}
> -
>  static enum hrtimer_restart dummy_hrtimer_callback(struct hrtimer *timer)
>  {
>  	struct dummy_hrtimer_pcm *dpcm;
> @@ -394,7 +386,8 @@ static enum hrtimer_restart dummy_hrtime
>  	dpcm = container_of(timer, struct dummy_hrtimer_pcm, timer);
>  	if (!atomic_read(&dpcm->running))
>  		return HRTIMER_NORESTART;
> -	tasklet_schedule(&dpcm->tasklet);
> +
> +	snd_pcm_period_elapsed(dpcm->substream);
>  	hrtimer_forward_now(timer, dpcm->period_time);
>  	return HRTIMER_RESTART;
>  }
> @@ -421,7 +414,6 @@ static int dummy_hrtimer_stop(struct snd
>  static inline void dummy_hrtimer_sync(struct dummy_hrtimer_pcm *dpcm)
>  {
>  	hrtimer_cancel(&dpcm->timer);
> -	tasklet_kill(&dpcm->tasklet);
>  }
>  
>  static snd_pcm_uframes_t
> @@ -466,12 +458,10 @@ static int dummy_hrtimer_create(struct s
>  	if (!dpcm)
>  		return -ENOMEM;
>  	substream->runtime->private_data = dpcm;
> -	hrtimer_init(&dpcm->timer, CLOCK_MONOTONIC, HRTIMER_MODE_REL);
> +	hrtimer_init(&dpcm->timer, CLOCK_MONOTONIC_SOFT, HRTIMER_MODE_REL);
>  	dpcm->timer.function = dummy_hrtimer_callback;
>  	dpcm->substream = substream;
>  	atomic_set(&dpcm->running, 0);
> -	tasklet_init(&dpcm->tasklet, dummy_hrtimer_pcm_elapsed,
> -		     (unsigned long)dpcm);
>  	return 0;
>  }

>From b2417f83e0ccbdfe1fc6870817ff0a1bd9243c77 Mon Sep 17 00:00:00 2001
From: Takashi Sakamoto <o-takashi@sakamocchi.jp>
Date: Thu, 31 Aug 2017 22:52:33 +0900
Subject: [PATCH] ALSA: pcsp: code optimization for hrtimer in software IRQ
 context

In a development period for v4.14, code refactoring was done for hrtimer
subsystem. This enables to receive callbacks from hrtimer in software IRQ
context.

Currently, ALSA pcsp driver uses hrtimer callback to handle period elapse
of PCM buffer and actual calculation of pointer position is done in
software IRQ context of tasklet. This can be simplified to the introduced
hrtimer in software IRQ context.

This commit applies an optimization for the above reason.

Signed-off-by: Takashi Sakamoto <o-takashi@sakamocchi.jp>
---
 sound/drivers/pcsp/pcsp.c     |  2 +-
 sound/drivers/pcsp/pcsp_lib.c | 24 +++++-------------------
 2 files changed, 6 insertions(+), 20 deletions(-)

diff --git a/sound/drivers/pcsp/pcsp.c b/sound/drivers/pcsp/pcsp.c
index 0dd3f46eb03e..8fac38b81c4f 100644
--- a/sound/drivers/pcsp/pcsp.c
+++ b/sound/drivers/pcsp/pcsp.c
@@ -100,7 +100,7 @@ static int snd_card_pcsp_probe(int devnum, struct device *dev)
 	if (devnum != 0)
 		return -EINVAL;
 
-	hrtimer_init(&pcsp_chip.timer, CLOCK_MONOTONIC, HRTIMER_MODE_REL);
+	hrtimer_init(&pcsp_chip.timer, CLOCK_MONOTONIC_SOFT, HRTIMER_MODE_REL);
 	pcsp_chip.timer.function = pcsp_do_timer;
 
 	err = snd_card_new(dev, index, id, THIS_MODULE, 0, &card);
diff --git a/sound/drivers/pcsp/pcsp_lib.c b/sound/drivers/pcsp/pcsp_lib.c
index 2f5a35f38ce1..d2b67463ddd3 100644
--- a/sound/drivers/pcsp/pcsp_lib.c
+++ b/sound/drivers/pcsp/pcsp_lib.c
@@ -21,22 +21,6 @@ MODULE_PARM_DESC(nforce_wa, "Apply NForce chipset workaround "
 
 #define DMIX_WANTS_S16	1
 
-/*
- * Call snd_pcm_period_elapsed in a tasklet
- * This avoids spinlock messes and long-running irq contexts
- */
-static void pcsp_call_pcm_elapsed(unsigned long priv)
-{
-	if (atomic_read(&pcsp_chip.timer_active)) {
-		struct snd_pcm_substream *substream;
-		substream = pcsp_chip.playback_substream;
-		if (substream)
-			snd_pcm_period_elapsed(substream);
-	}
-}
-
-static DECLARE_TASKLET(pcsp_pcm_tasklet, pcsp_call_pcm_elapsed, 0);
-
 /* write the port and returns the next expire time in ns;
  * called at the trigger-start and in hrtimer callback
  */
@@ -121,8 +105,11 @@ static void pcsp_pointer_update(struct snd_pcsp *chip)
 	}
 	spin_unlock_irqrestore(&chip->substream_lock, flags);
 
-	if (periods_elapsed)
-		tasklet_schedule(&pcsp_pcm_tasklet);
+	if (!periods_elapsed)
+		return;
+
+	if (atomic_read(&pcsp_chip.timer_active))
+		snd_pcm_period_elapsed(substream);
 }
 
 enum hrtimer_restart pcsp_do_timer(struct hrtimer *handle)
@@ -195,7 +182,6 @@ void pcsp_sync_stop(struct snd_pcsp *chip)
 	pcsp_stop_playing(chip);
 	local_irq_enable();
 	hrtimer_cancel(&chip->timer);
-	tasklet_kill(&pcsp_pcm_tasklet);
 }
 
 static int snd_pcsp_playback_close(struct snd_pcm_substream *substream)
-- 
2.11.0

Re: [PATCH 23/25] ALSA/dummy: Replace tasklet with softirq hrtimer

[Takashi Sakamoto]

Hi,

On Aug 31 2017 21:23, Anna-Maria Gleixner wrote:
> From: Thomas Gleixner <tglx@linutronix.de>
> 
> The tasklet is used to defer the execution of snd_pcm_period_elapsed() to
> the softirq context. Using the CLOCK_MONOTONIC_SOFT base invokes the timer
> callback in softirq context as well which renders the tasklet useless.
> 
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> Signed-off-by: Anna-Maria Gleixner <anna-maria@linutronix.de>
> Cc: Jaroslav Kysela <perex@perex.cz>
> Cc: Takashi Iwai <tiwai@suse.com>
> Cc: Takashi Sakamoto <o-takashi@sakamocchi.jp>
> Cc: alsa-devel@alsa-project.org
> ---
>  sound/drivers/dummy.c |   16 +++-------------
>  1 file changed, 3 insertions(+), 13 deletions(-)

I prefer this patch as long as this driver can still receive callbacks
from hrtimer subsystem.

Reviewed-by: Takashi Sakamoto <o-takashi@sakamocchi.jp>

Unfortunately, I have too poor machine to compile whole kernel now, thus
didn't do any tests, sorry.

I note that ALSA pcsp driver uses a combination of hrtimer/tasklet for the
same purpose. I think we can simplify it, too. Please refer to a patch in
the end of this message. (But not tested yet for the above reason...)

> --- a/sound/drivers/dummy.c
> +++ b/sound/drivers/dummy.c
> @@ -376,17 +376,9 @@ struct dummy_hrtimer_pcm {
>  	ktime_t period_time;
>  	atomic_t running;
>  	struct hrtimer timer;
> -	struct tasklet_struct tasklet;
>  	struct snd_pcm_substream *substream;
>  };
>  
> -static void dummy_hrtimer_pcm_elapsed(unsigned long priv)
> -{
> -	struct dummy_hrtimer_pcm *dpcm = (struct dummy_hrtimer_pcm *)priv;
> -	if (atomic_read(&dpcm->running))
> -		snd_pcm_period_elapsed(dpcm->substream);
> -}
> -
>  static enum hrtimer_restart dummy_hrtimer_callback(struct hrtimer *timer)
>  {
>  	struct dummy_hrtimer_pcm *dpcm;
> @@ -394,7 +386,8 @@ static enum hrtimer_restart dummy_hrtime
>  	dpcm = container_of(timer, struct dummy_hrtimer_pcm, timer);
>  	if (!atomic_read(&dpcm->running))
>  		return HRTIMER_NORESTART;
> -	tasklet_schedule(&dpcm->tasklet);
> +
> +	snd_pcm_period_elapsed(dpcm->substream);
>  	hrtimer_forward_now(timer, dpcm->period_time);
>  	return HRTIMER_RESTART;
>  }
> @@ -421,7 +414,6 @@ static int dummy_hrtimer_stop(struct snd
>  static inline void dummy_hrtimer_sync(struct dummy_hrtimer_pcm *dpcm)
>  {
>  	hrtimer_cancel(&dpcm->timer);
> -	tasklet_kill(&dpcm->tasklet);
>  }
>  
>  static snd_pcm_uframes_t
> @@ -466,12 +458,10 @@ static int dummy_hrtimer_create(struct s
>  	if (!dpcm)
>  		return -ENOMEM;
>  	substream->runtime->private_data = dpcm;
> -	hrtimer_init(&dpcm->timer, CLOCK_MONOTONIC, HRTIMER_MODE_REL);
> +	hrtimer_init(&dpcm->timer, CLOCK_MONOTONIC_SOFT, HRTIMER_MODE_REL);
>  	dpcm->timer.function = dummy_hrtimer_callback;
>  	dpcm->substream = substream;
>  	atomic_set(&dpcm->running, 0);
> -	tasklet_init(&dpcm->tasklet, dummy_hrtimer_pcm_elapsed,
> -		     (unsigned long)dpcm);
>  	return 0;
>  }

>From b2417f83e0ccbdfe1fc6870817ff0a1bd9243c77 Mon Sep 17 00:00:00 2001
From: Takashi Sakamoto <o-takashi@sakamocchi.jp>
Date: Thu, 31 Aug 2017 22:52:33 +0900
Subject: [PATCH] ALSA: pcsp: code optimization for hrtimer in software IRQ
 context

In a development period for v4.14, code refactoring was done for hrtimer
subsystem. This enables to receive callbacks from hrtimer in software IRQ
context.

Currently, ALSA pcsp driver uses hrtimer callback to handle period elapse
of PCM buffer and actual calculation of pointer position is done in
software IRQ context of tasklet. This can be simplified to the introduced
hrtimer in software IRQ context.

This commit applies an optimization for the above reason.

Signed-off-by: Takashi Sakamoto <o-takashi@sakamocchi.jp>
---
 sound/drivers/pcsp/pcsp.c     |  2 +-
 sound/drivers/pcsp/pcsp_lib.c | 24 +++++-------------------
 2 files changed, 6 insertions(+), 20 deletions(-)

diff --git a/sound/drivers/pcsp/pcsp.c b/sound/drivers/pcsp/pcsp.c
index 0dd3f46eb03e..8fac38b81c4f 100644
--- a/sound/drivers/pcsp/pcsp.c
+++ b/sound/drivers/pcsp/pcsp.c
@@ -100,7 +100,7 @@ static int snd_card_pcsp_probe(int devnum, struct device *dev)
 	if (devnum != 0)
 		return -EINVAL;
 
-	hrtimer_init(&pcsp_chip.timer, CLOCK_MONOTONIC, HRTIMER_MODE_REL);
+	hrtimer_init(&pcsp_chip.timer, CLOCK_MONOTONIC_SOFT, HRTIMER_MODE_REL);
 	pcsp_chip.timer.function = pcsp_do_timer;
 
 	err = snd_card_new(dev, index, id, THIS_MODULE, 0, &card);
diff --git a/sound/drivers/pcsp/pcsp_lib.c b/sound/drivers/pcsp/pcsp_lib.c
index 2f5a35f38ce1..d2b67463ddd3 100644
--- a/sound/drivers/pcsp/pcsp_lib.c
+++ b/sound/drivers/pcsp/pcsp_lib.c
@@ -21,22 +21,6 @@ MODULE_PARM_DESC(nforce_wa, "Apply NForce chipset workaround "
 
 #define DMIX_WANTS_S16	1
 
-/*
- * Call snd_pcm_period_elapsed in a tasklet
- * This avoids spinlock messes and long-running irq contexts
- */
-static void pcsp_call_pcm_elapsed(unsigned long priv)
-{
-	if (atomic_read(&pcsp_chip.timer_active)) {
-		struct snd_pcm_substream *substream;
-		substream = pcsp_chip.playback_substream;
-		if (substream)
-			snd_pcm_period_elapsed(substream);
-	}
-}
-
-static DECLARE_TASKLET(pcsp_pcm_tasklet, pcsp_call_pcm_elapsed, 0);
-
 /* write the port and returns the next expire time in ns;
  * called at the trigger-start and in hrtimer callback
  */
@@ -121,8 +105,11 @@ static void pcsp_pointer_update(struct snd_pcsp *chip)
 	}
 	spin_unlock_irqrestore(&chip->substream_lock, flags);
 
-	if (periods_elapsed)
-		tasklet_schedule(&pcsp_pcm_tasklet);
+	if (!periods_elapsed)
+		return;
+
+	if (atomic_read(&pcsp_chip.timer_active))
+		snd_pcm_period_elapsed(substream);
 }
 
 enum hrtimer_restart pcsp_do_timer(struct hrtimer *handle)
@@ -195,7 +182,6 @@ void pcsp_sync_stop(struct snd_pcsp *chip)
 	pcsp_stop_playing(chip);
 	local_irq_enable();
 	hrtimer_cancel(&chip->timer);
-	tasklet_kill(&pcsp_pcm_tasklet);
 }
 
 static int snd_pcsp_playback_close(struct snd_pcm_substream *substream)
-- 
2.11.0

Re: [PATCH 23/25] ALSA/dummy: Replace tasklet with softirq hrtimer

[Takashi Iwai]

On Thu, 31 Aug 2017 16:21:17 +0200,
Takashi Sakamoto wrote:
> 
> Hi,
> 
> On Aug 31 2017 21:23, Anna-Maria Gleixner wrote:
> > From: Thomas Gleixner <tglx@linutronix.de>
> > 
> > The tasklet is used to defer the execution of snd_pcm_period_elapsed() to
> > the softirq context. Using the CLOCK_MONOTONIC_SOFT base invokes the timer
> > callback in softirq context as well which renders the tasklet useless.
> > 
> > Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> > Signed-off-by: Anna-Maria Gleixner <anna-maria@linutronix.de>
> > Cc: Jaroslav Kysela <perex@perex.cz>
> > Cc: Takashi Iwai <tiwai@suse.com>
> > Cc: Takashi Sakamoto <o-takashi@sakamocchi.jp>
> > Cc: alsa-devel@alsa-project.org
> > ---
> >  sound/drivers/dummy.c |   16 +++-------------
> >  1 file changed, 3 insertions(+), 13 deletions(-)
> 
> I prefer this patch as long as this driver can still receive callbacks
> from hrtimer subsystem.
> 
> Reviewed-by: Takashi Sakamoto <o-takashi@sakamocchi.jp>
> 
> Unfortunately, I have too poor machine to compile whole kernel now, thus
> didn't do any tests, sorry.
> 
> I note that ALSA pcsp driver uses a combination of hrtimer/tasklet for the
> same purpose. I think we can simplify it, too. Please refer to a patch in
> the end of this message. (But not tested yet for the above reason...)

The pcsp is a bit special.  It's really high frequent irq calls for
controlling the beep on/off, thus offloading the whole isn't good.


thanks,

Takashi

Re: [PATCH 23/25] ALSA/dummy: Replace tasklet with softirq hrtimer

[Takashi Iwai]

On Thu, 31 Aug 2017 16:21:17 +0200,
Takashi Sakamoto wrote:
> 
> Hi,
> 
> On Aug 31 2017 21:23, Anna-Maria Gleixner wrote:
> > From: Thomas Gleixner <tglx@linutronix.de>
> > 
> > The tasklet is used to defer the execution of snd_pcm_period_elapsed() to
> > the softirq context. Using the CLOCK_MONOTONIC_SOFT base invokes the timer
> > callback in softirq context as well which renders the tasklet useless.
> > 
> > Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> > Signed-off-by: Anna-Maria Gleixner <anna-maria@linutronix.de>
> > Cc: Jaroslav Kysela <perex@perex.cz>
> > Cc: Takashi Iwai <tiwai@suse.com>
> > Cc: Takashi Sakamoto <o-takashi@sakamocchi.jp>
> > Cc: alsa-devel@alsa-project.org
> > ---
> >  sound/drivers/dummy.c |   16 +++-------------
> >  1 file changed, 3 insertions(+), 13 deletions(-)
> 
> I prefer this patch as long as this driver can still receive callbacks
> from hrtimer subsystem.
> 
> Reviewed-by: Takashi Sakamoto <o-takashi@sakamocchi.jp>
> 
> Unfortunately, I have too poor machine to compile whole kernel now, thus
> didn't do any tests, sorry.
> 
> I note that ALSA pcsp driver uses a combination of hrtimer/tasklet for the
> same purpose. I think we can simplify it, too. Please refer to a patch in
> the end of this message. (But not tested yet for the above reason...)

The pcsp is a bit special.  It's really high frequent irq calls for
controlling the beep on/off, thus offloading the whole isn't good.


thanks,

Takashi

Re: [PATCH 23/25] ALSA/dummy: Replace tasklet with softirq hrtimer

[Takashi Iwai]

On Thu, 31 Aug 2017 14:23:45 +0200,
Anna-Maria Gleixner wrote:
> 
> From: Thomas Gleixner <tglx@linutronix.de>
> 
> The tasklet is used to defer the execution of snd_pcm_period_elapsed() to
> the softirq context. Using the CLOCK_MONOTONIC_SOFT base invokes the timer
> callback in softirq context as well which renders the tasklet useless.
> 
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> Signed-off-by: Anna-Maria Gleixner <anna-maria@linutronix.de>
> Cc: Jaroslav Kysela <perex@perex.cz>
> Cc: Takashi Iwai <tiwai@suse.com>
> Cc: Takashi Sakamoto <o-takashi@sakamocchi.jp>
> Cc: alsa-devel@alsa-project.org

I gave it at try, but it caused a kernel hang, unfortunately.

The reason is that snd_pcm_period_elapased() may stop the stream
(e.g. when reaching at the end).  With this patchset, it'll lead to
the call of hrtimer_cancel() from the hrtimer callback itself, thus it
stalls.

Below is the additional fix over your patch for working around it.
I believe it should cover most corner cases, and seems working fine
through quick tests, so far.


thanks,

Takashi

---
diff --git a/sound/drivers/dummy.c b/sound/drivers/dummy.c
index 273d60c42125..b5dd64e3dab1 100644
--- a/sound/drivers/dummy.c
+++ b/sound/drivers/dummy.c
@@ -375,6 +375,7 @@ struct dummy_hrtimer_pcm {
 	ktime_t base_time;
 	ktime_t period_time;
 	atomic_t running;
+	atomic_t callback_running;
 	struct hrtimer timer;
 	struct snd_pcm_substream *substream;
 };
@@ -387,8 +388,15 @@ static enum hrtimer_restart dummy_hrtimer_callback(struct hrtimer *timer)
 	if (!atomic_read(&dpcm->running))
 		return HRTIMER_NORESTART;
 
+	atomic_inc(&dpcm->callback_running);
 	snd_pcm_period_elapsed(dpcm->substream);
+	atomic_dec(&dpcm->callback_running);
+	/* may be flipped during snd_pcm_period_elapsed() */
+	if (!atomic_read(&dpcm->running))
+		return HRTIMER_NORESTART;
+
 	hrtimer_forward_now(timer, dpcm->period_time);
+	atomic_dec(&dpcm->callback_running);
 	return HRTIMER_RESTART;
 }
 
@@ -407,7 +415,9 @@ static int dummy_hrtimer_stop(struct snd_pcm_substream *substream)
 	struct dummy_hrtimer_pcm *dpcm = substream->runtime->private_data;
 
 	atomic_set(&dpcm->running, 0);
-	hrtimer_cancel(&dpcm->timer);
+	/* issue hrtimer_cancel() only when called outside the callback */
+	if (!atomic_read(&dpcm->callback_running))
+		hrtimer_cancel(&dpcm->timer);
 	return 0;
 }
 
@@ -462,6 +472,7 @@ static int dummy_hrtimer_create(struct snd_pcm_substream *substream)
 	dpcm->timer.function = dummy_hrtimer_callback;
 	dpcm->substream = substream;
 	atomic_set(&dpcm->running, 0);
+	atomic_set(&dpcm->callback_running, 0);
 	return 0;
 }
 

Re: [PATCH 23/25] ALSA/dummy: Replace tasklet with softirq hrtimer

[Takashi Iwai]

On Thu, 31 Aug 2017 14:23:45 +0200,
Anna-Maria Gleixner wrote:
> 
> From: Thomas Gleixner <tglx@linutronix.de>
> 
> The tasklet is used to defer the execution of snd_pcm_period_elapsed() to
> the softirq context. Using the CLOCK_MONOTONIC_SOFT base invokes the timer
> callback in softirq context as well which renders the tasklet useless.
> 
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> Signed-off-by: Anna-Maria Gleixner <anna-maria@linutronix.de>
> Cc: Jaroslav Kysela <perex@perex.cz>
> Cc: Takashi Iwai <tiwai@suse.com>
> Cc: Takashi Sakamoto <o-takashi@sakamocchi.jp>
> Cc: alsa-devel@alsa-project.org

I gave it at try, but it caused a kernel hang, unfortunately.

The reason is that snd_pcm_period_elapased() may stop the stream
(e.g. when reaching at the end).  With this patchset, it'll lead to
the call of hrtimer_cancel() from the hrtimer callback itself, thus it
stalls.

Below is the additional fix over your patch for working around it.
I believe it should cover most corner cases, and seems working fine
through quick tests, so far.


thanks,

Takashi

---
diff --git a/sound/drivers/dummy.c b/sound/drivers/dummy.c
index 273d60c42125..b5dd64e3dab1 100644
--- a/sound/drivers/dummy.c
+++ b/sound/drivers/dummy.c
@@ -375,6 +375,7 @@ struct dummy_hrtimer_pcm {
 	ktime_t base_time;
 	ktime_t period_time;
 	atomic_t running;
+	atomic_t callback_running;
 	struct hrtimer timer;
 	struct snd_pcm_substream *substream;
 };
@@ -387,8 +388,15 @@ static enum hrtimer_restart dummy_hrtimer_callback(struct hrtimer *timer)
 	if (!atomic_read(&dpcm->running))
 		return HRTIMER_NORESTART;
 
+	atomic_inc(&dpcm->callback_running);
 	snd_pcm_period_elapsed(dpcm->substream);
+	atomic_dec(&dpcm->callback_running);
+	/* may be flipped during snd_pcm_period_elapsed() */
+	if (!atomic_read(&dpcm->running))
+		return HRTIMER_NORESTART;
+
 	hrtimer_forward_now(timer, dpcm->period_time);
+	atomic_dec(&dpcm->callback_running);
 	return HRTIMER_RESTART;
 }
 
@@ -407,7 +415,9 @@ static int dummy_hrtimer_stop(struct snd_pcm_substream *substream)
 	struct dummy_hrtimer_pcm *dpcm = substream->runtime->private_data;
 
 	atomic_set(&dpcm->running, 0);
-	hrtimer_cancel(&dpcm->timer);
+	/* issue hrtimer_cancel() only when called outside the callback */
+	if (!atomic_read(&dpcm->callback_running))
+		hrtimer_cancel(&dpcm->timer);
 	return 0;
 }
 
@@ -462,6 +472,7 @@ static int dummy_hrtimer_create(struct snd_pcm_substream *substream)
 	dpcm->timer.function = dummy_hrtimer_callback;
 	dpcm->substream = substream;
 	atomic_set(&dpcm->running, 0);
+	atomic_set(&dpcm->callback_running, 0);
 	return 0;
 }
 

Re: [PATCH 23/25] ALSA/dummy: Replace tasklet with softirq hrtimer

[Takashi Sakamoto]

Hi,

On Sep 1 2017 00:36, Takashi Iwai wrote:
> I gave it at try, but it caused a kernel hang, unfortunately.
> 
> The reason is that snd_pcm_period_elapased() may stop the stream
> (e.g. when reaching at the end).  With this patchset, it'll lead to
> the call of hrtimer_cancel() from the hrtimer callback itself, thus it
> stalls.
 
I can reproduce this bug.

> Below is the additional fix over your patch for working around it.
> I believe it should cover most corner cases, and seems working fine
> through quick tests, so far.

This patch looks good to me, too. But I have an alternative.

We can use 'hrtimer_callback_running()' to detect whether to be on hrtimer
callback or not (please read '__run_hrtimer()' in 'kernel/time/hrtimer.c').
Usage of this helper function on .stop callback to skip cancellation can
avoid the stall. In this case, after stopping PCM substream, the hrtimer
callback should return HRTIMER_NORESTART to avoid restarting, as well as
your patch.  Please test a patch in this message.

> ---
> diff --git a/sound/drivers/dummy.c b/sound/drivers/dummy.c
> index 273d60c42125..b5dd64e3dab1 100644
> --- a/sound/drivers/dummy.c
> +++ b/sound/drivers/dummy.c
> @@ -375,6 +375,7 @@ struct dummy_hrtimer_pcm {
>   	ktime_t base_time;
>   	ktime_t period_time;
>   	atomic_t running;
> +	atomic_t callback_running;
>   	struct hrtimer timer;
>   	struct snd_pcm_substream *substream;
>   };
> @@ -387,8 +388,15 @@ static enum hrtimer_restart dummy_hrtimer_callback(struct hrtimer *timer)
>   	if (!atomic_read(&dpcm->running))
>   		return HRTIMER_NORESTART;
>   
> +	atomic_inc(&dpcm->callback_running);
>   	snd_pcm_period_elapsed(dpcm->substream);
> +	atomic_dec(&dpcm->callback_running);
> +	/* may be flipped during snd_pcm_period_elapsed() */
> +	if (!atomic_read(&dpcm->running))
> +		return HRTIMER_NORESTART;
> +
>   	hrtimer_forward_now(timer, dpcm->period_time);
> +	atomic_dec(&dpcm->callback_running);
>   	return HRTIMER_RESTART;
>   }
>   
> @@ -407,7 +415,9 @@ static int dummy_hrtimer_stop(struct snd_pcm_substream *substream)
>   	struct dummy_hrtimer_pcm *dpcm = substream->runtime->private_data;
>   
>   	atomic_set(&dpcm->running, 0);
> -	hrtimer_cancel(&dpcm->timer);
> +	/* issue hrtimer_cancel() only when called outside the callback */
> +	if (!atomic_read(&dpcm->callback_running))
> +		hrtimer_cancel(&dpcm->timer);
>   	return 0;
>   }
>   
> @@ -462,6 +472,7 @@ static int dummy_hrtimer_create(struct snd_pcm_substream *substream)
>   	dpcm->timer.function = dummy_hrtimer_callback;
>   	dpcm->substream = substream;
>   	atomic_set(&dpcm->running, 0);
> +	atomic_set(&dpcm->callback_running, 0);
>   	return 0;
>   }

>From 07d61ba2a1c0e06e914443225e194d99f2d8c58d Mon Sep 17 00:00:00 2001
From: Takashi Sakamoto <o-takashi@sakamocchi.jp>
Date: Fri, 1 Sep 2017 19:10:18 +0900
Subject: [PATCH] ALSA: dummy: avoid stall due to a call of hrtimer_cancel() on
 a callback of hrtimer

A call of 'htrimer_cancel()' on a callback of hrtimer brings endless loop
because 'struct hrtimer_clock_base.running' is not NULL on the callback.
In hrtimer subsystem, this member is used to indicate the instance of
hrtimer gets callbacks and there's a helper function,
'hrtimer_callback_running()' to check it.

ALSA dummy driver uses hrtimer to emulate hardware interrupt per period
of PCM buffer. When XRUN occurs on PCM substream, in a call of
'snd_pcm_period_elapsed()', 'struct snd_pcm_ops.stop()' is called to
stop the substream. In current implementation, 'hrtimer_cancel()' is
used to wait for cancellation of hrtimer. However, as described, this
brings endless loop.

For this problem, this commit uses 'hrtimer_callback_running()' to
detect whether to be on a callback of hrtimer or not, then skip
cancellation of hrtimer in hrtimer callbacks. Furthermore, at a case of
XRUN, hrtimer callback returns HRTIMER_NORESTART after a call of
'snd_pcm_period_elapsed()' to discontinue hrtimr because cancellation is
skipped.

Signed-off-by: Takashi Sakamoto <o-takashi@sakamocchi.jp>
---
 sound/drivers/dummy.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/sound/drivers/dummy.c b/sound/drivers/dummy.c
index 273d60c42125..9caf754c6135 100644
--- a/sound/drivers/dummy.c
+++ b/sound/drivers/dummy.c
@@ -387,7 +387,11 @@ static enum hrtimer_restart dummy_hrtimer_callback(struct hrtimer *timer)
 	if (!atomic_read(&dpcm->running))
 		return HRTIMER_NORESTART;
 
+	/* In a case of XRUN, this calls .trigger to stop PCM substream. */
 	snd_pcm_period_elapsed(dpcm->substream);
+	if (!atomic_read(&dpcm->running))
+		return HRTIMER_NORESTART;
+
 	hrtimer_forward_now(timer, dpcm->period_time);
 	return HRTIMER_RESTART;
 }
@@ -407,7 +411,8 @@ static int dummy_hrtimer_stop(struct snd_pcm_substream *substream)
 	struct dummy_hrtimer_pcm *dpcm = substream->runtime->private_data;
 
 	atomic_set(&dpcm->running, 0);
-	hrtimer_cancel(&dpcm->timer);
+	if (!hrtimer_callback_running(&dpcm->timer))
+		hrtimer_cancel(&dpcm->timer);
 	return 0;
 }
 
-- 
2.11.0


Regards

Takashi Sakamoto

Re: [PATCH 23/25] ALSA/dummy: Replace tasklet with softirq hrtimer

[Takashi Sakamoto]

Hi,

On Sep 1 2017 00:36, Takashi Iwai wrote:
> I gave it at try, but it caused a kernel hang, unfortunately.
> 
> The reason is that snd_pcm_period_elapased() may stop the stream
> (e.g. when reaching at the end).  With this patchset, it'll lead to
> the call of hrtimer_cancel() from the hrtimer callback itself, thus it
> stalls.
 
I can reproduce this bug.

> Below is the additional fix over your patch for working around it.
> I believe it should cover most corner cases, and seems working fine
> through quick tests, so far.

This patch looks good to me, too. But I have an alternative.

We can use 'hrtimer_callback_running()' to detect whether to be on hrtimer
callback or not (please read '__run_hrtimer()' in 'kernel/time/hrtimer.c').
Usage of this helper function on .stop callback to skip cancellation can
avoid the stall. In this case, after stopping PCM substream, the hrtimer
callback should return HRTIMER_NORESTART to avoid restarting, as well as
your patch.  Please test a patch in this message.

> ---
> diff --git a/sound/drivers/dummy.c b/sound/drivers/dummy.c
> index 273d60c42125..b5dd64e3dab1 100644
> --- a/sound/drivers/dummy.c
> +++ b/sound/drivers/dummy.c
> @@ -375,6 +375,7 @@ struct dummy_hrtimer_pcm {
>   	ktime_t base_time;
>   	ktime_t period_time;
>   	atomic_t running;
> +	atomic_t callback_running;
>   	struct hrtimer timer;
>   	struct snd_pcm_substream *substream;
>   };
> @@ -387,8 +388,15 @@ static enum hrtimer_restart dummy_hrtimer_callback(struct hrtimer *timer)
>   	if (!atomic_read(&dpcm->running))
>   		return HRTIMER_NORESTART;
>   
> +	atomic_inc(&dpcm->callback_running);
>   	snd_pcm_period_elapsed(dpcm->substream);
> +	atomic_dec(&dpcm->callback_running);
> +	/* may be flipped during snd_pcm_period_elapsed() */
> +	if (!atomic_read(&dpcm->running))
> +		return HRTIMER_NORESTART;
> +
>   	hrtimer_forward_now(timer, dpcm->period_time);
> +	atomic_dec(&dpcm->callback_running);
>   	return HRTIMER_RESTART;
>   }
>   
> @@ -407,7 +415,9 @@ static int dummy_hrtimer_stop(struct snd_pcm_substream *substream)
>   	struct dummy_hrtimer_pcm *dpcm = substream->runtime->private_data;
>   
>   	atomic_set(&dpcm->running, 0);
> -	hrtimer_cancel(&dpcm->timer);
> +	/* issue hrtimer_cancel() only when called outside the callback */
> +	if (!atomic_read(&dpcm->callback_running))
> +		hrtimer_cancel(&dpcm->timer);
>   	return 0;
>   }
>   
> @@ -462,6 +472,7 @@ static int dummy_hrtimer_create(struct snd_pcm_substream *substream)
>   	dpcm->timer.function = dummy_hrtimer_callback;
>   	dpcm->substream = substream;
>   	atomic_set(&dpcm->running, 0);
> +	atomic_set(&dpcm->callback_running, 0);
>   	return 0;
>   }

>From 07d61ba2a1c0e06e914443225e194d99f2d8c58d Mon Sep 17 00:00:00 2001
From: Takashi Sakamoto <o-takashi@sakamocchi.jp>
Date: Fri, 1 Sep 2017 19:10:18 +0900
Subject: [PATCH] ALSA: dummy: avoid stall due to a call of hrtimer_cancel() on
 a callback of hrtimer

A call of 'htrimer_cancel()' on a callback of hrtimer brings endless loop
because 'struct hrtimer_clock_base.running' is not NULL on the callback.
In hrtimer subsystem, this member is used to indicate the instance of
hrtimer gets callbacks and there's a helper function,
'hrtimer_callback_running()' to check it.

ALSA dummy driver uses hrtimer to emulate hardware interrupt per period
of PCM buffer. When XRUN occurs on PCM substream, in a call of
'snd_pcm_period_elapsed()', 'struct snd_pcm_ops.stop()' is called to
stop the substream. In current implementation, 'hrtimer_cancel()' is
used to wait for cancellation of hrtimer. However, as described, this
brings endless loop.

For this problem, this commit uses 'hrtimer_callback_running()' to
detect whether to be on a callback of hrtimer or not, then skip
cancellation of hrtimer in hrtimer callbacks. Furthermore, at a case of
XRUN, hrtimer callback returns HRTIMER_NORESTART after a call of
'snd_pcm_period_elapsed()' to discontinue hrtimr because cancellation is
skipped.

Signed-off-by: Takashi Sakamoto <o-takashi@sakamocchi.jp>
---
 sound/drivers/dummy.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/sound/drivers/dummy.c b/sound/drivers/dummy.c
index 273d60c42125..9caf754c6135 100644
--- a/sound/drivers/dummy.c
+++ b/sound/drivers/dummy.c
@@ -387,7 +387,11 @@ static enum hrtimer_restart dummy_hrtimer_callback(struct hrtimer *timer)
 	if (!atomic_read(&dpcm->running))
 		return HRTIMER_NORESTART;
 
+	/* In a case of XRUN, this calls .trigger to stop PCM substream. */
 	snd_pcm_period_elapsed(dpcm->substream);
+	if (!atomic_read(&dpcm->running))
+		return HRTIMER_NORESTART;
+
 	hrtimer_forward_now(timer, dpcm->period_time);
 	return HRTIMER_RESTART;
 }
@@ -407,7 +411,8 @@ static int dummy_hrtimer_stop(struct snd_pcm_substream *substream)
 	struct dummy_hrtimer_pcm *dpcm = substream->runtime->private_data;
 
 	atomic_set(&dpcm->running, 0);
-	hrtimer_cancel(&dpcm->timer);
+	if (!hrtimer_callback_running(&dpcm->timer))
+		hrtimer_cancel(&dpcm->timer);
 	return 0;
 }
 
-- 
2.11.0


Regards

Takashi Sakamoto

Re: [PATCH 23/25] ALSA/dummy: Replace tasklet with softirq hrtimer

[Takashi Iwai]

On Fri, 01 Sep 2017 12:25:37 +0200,
Takashi Sakamoto wrote:
> 
> Hi,
> 
> On Sep 1 2017 00:36, Takashi Iwai wrote:
> > I gave it at try, but it caused a kernel hang, unfortunately.
> > 
> > The reason is that snd_pcm_period_elapased() may stop the stream
> > (e.g. when reaching at the end).  With this patchset, it'll lead to
> > the call of hrtimer_cancel() from the hrtimer callback itself, thus it
> > stalls.
>  
> I can reproduce this bug.
> 
> > Below is the additional fix over your patch for working around it.
> > I believe it should cover most corner cases, and seems working fine
> > through quick tests, so far.
> 
> This patch looks good to me, too. But I have an alternative.
> 
> We can use 'hrtimer_callback_running()' to detect whether to be on hrtimer
> callback or not (please read '__run_hrtimer()' in 'kernel/time/hrtimer.c').

A good point, this is a better choice.

> Usage of this helper function on .stop callback to skip cancellation can
> avoid the stall. In this case, after stopping PCM substream, the hrtimer
> callback should return HRTIMER_NORESTART to avoid restarting, as well as
> your patch.  Please test a patch in this message.
> 
> > ---
> > diff --git a/sound/drivers/dummy.c b/sound/drivers/dummy.c
> > index 273d60c42125..b5dd64e3dab1 100644
> > --- a/sound/drivers/dummy.c
> > +++ b/sound/drivers/dummy.c
> > @@ -375,6 +375,7 @@ struct dummy_hrtimer_pcm {
> >   	ktime_t base_time;
> >   	ktime_t period_time;
> >   	atomic_t running;
> > +	atomic_t callback_running;
> >   	struct hrtimer timer;
> >   	struct snd_pcm_substream *substream;
> >   };
> > @@ -387,8 +388,15 @@ static enum hrtimer_restart dummy_hrtimer_callback(struct hrtimer *timer)
> >   	if (!atomic_read(&dpcm->running))
> >   		return HRTIMER_NORESTART;
> >   
> > +	atomic_inc(&dpcm->callback_running);
> >   	snd_pcm_period_elapsed(dpcm->substream);
> > +	atomic_dec(&dpcm->callback_running);
> > +	/* may be flipped during snd_pcm_period_elapsed() */
> > +	if (!atomic_read(&dpcm->running))
> > +		return HRTIMER_NORESTART;
> > +
> >   	hrtimer_forward_now(timer, dpcm->period_time);
> > +	atomic_dec(&dpcm->callback_running);
> >   	return HRTIMER_RESTART;
> >   }
> >   
> > @@ -407,7 +415,9 @@ static int dummy_hrtimer_stop(struct snd_pcm_substream *substream)
> >   	struct dummy_hrtimer_pcm *dpcm = substream->runtime->private_data;
> >   
> >   	atomic_set(&dpcm->running, 0);
> > -	hrtimer_cancel(&dpcm->timer);
> > +	/* issue hrtimer_cancel() only when called outside the callback */
> > +	if (!atomic_read(&dpcm->callback_running))
> > +		hrtimer_cancel(&dpcm->timer);
> >   	return 0;
> >   }
> >   
> > @@ -462,6 +472,7 @@ static int dummy_hrtimer_create(struct snd_pcm_substream *substream)
> >   	dpcm->timer.function = dummy_hrtimer_callback;
> >   	dpcm->substream = substream;
> >   	atomic_set(&dpcm->running, 0);
> > +	atomic_set(&dpcm->callback_running, 0);
> >   	return 0;
> >   }
> 
> >From 07d61ba2a1c0e06e914443225e194d99f2d8c58d Mon Sep 17 00:00:00 2001
> From: Takashi Sakamoto <o-takashi@sakamocchi.jp>
> Date: Fri, 1 Sep 2017 19:10:18 +0900
> Subject: [PATCH] ALSA: dummy: avoid stall due to a call of hrtimer_cancel() on
>  a callback of hrtimer
> 
> A call of 'htrimer_cancel()' on a callback of hrtimer brings endless loop
> because 'struct hrtimer_clock_base.running' is not NULL on the callback.
> In hrtimer subsystem, this member is used to indicate the instance of
> hrtimer gets callbacks and there's a helper function,
> 'hrtimer_callback_running()' to check it.
> 
> ALSA dummy driver uses hrtimer to emulate hardware interrupt per period
> of PCM buffer. When XRUN occurs on PCM substream, in a call of
> 'snd_pcm_period_elapsed()', 'struct snd_pcm_ops.stop()' is called to
> stop the substream. In current implementation, 'hrtimer_cancel()' is
> used to wait for cancellation of hrtimer. However, as described, this
> brings endless loop.

It's not only about XRUN.  When the stream finishes the draining, it
stops the stream gracefully -- that is the very normal operation.

> For this problem, this commit uses 'hrtimer_callback_running()' to
> detect whether to be on a callback of hrtimer or not, then skip
> cancellation of hrtimer in hrtimer callbacks. Furthermore, at a case of
> XRUN, hrtimer callback returns HRTIMER_NORESTART after a call of
> 'snd_pcm_period_elapsed()' to discontinue hrtimr because cancellation is
> skipped.
> 
> Signed-off-by: Takashi Sakamoto <o-takashi@sakamocchi.jp>

It's better to fold the fix into the original patch instead of
introducing a bug and fixing it.


Takashi

Re: [PATCH 23/25] ALSA/dummy: Replace tasklet with softirq hrtimer

[Takashi Iwai]

On Fri, 01 Sep 2017 12:25:37 +0200,
Takashi Sakamoto wrote:
> 
> Hi,
> 
> On Sep 1 2017 00:36, Takashi Iwai wrote:
> > I gave it at try, but it caused a kernel hang, unfortunately.
> > 
> > The reason is that snd_pcm_period_elapased() may stop the stream
> > (e.g. when reaching at the end).  With this patchset, it'll lead to
> > the call of hrtimer_cancel() from the hrtimer callback itself, thus it
> > stalls.
>  
> I can reproduce this bug.
> 
> > Below is the additional fix over your patch for working around it.
> > I believe it should cover most corner cases, and seems working fine
> > through quick tests, so far.
> 
> This patch looks good to me, too. But I have an alternative.
> 
> We can use 'hrtimer_callback_running()' to detect whether to be on hrtimer
> callback or not (please read '__run_hrtimer()' in 'kernel/time/hrtimer.c').

A good point, this is a better choice.

> Usage of this helper function on .stop callback to skip cancellation can
> avoid the stall. In this case, after stopping PCM substream, the hrtimer
> callback should return HRTIMER_NORESTART to avoid restarting, as well as
> your patch.  Please test a patch in this message.
> 
> > ---
> > diff --git a/sound/drivers/dummy.c b/sound/drivers/dummy.c
> > index 273d60c42125..b5dd64e3dab1 100644
> > --- a/sound/drivers/dummy.c
> > +++ b/sound/drivers/dummy.c
> > @@ -375,6 +375,7 @@ struct dummy_hrtimer_pcm {
> >   	ktime_t base_time;
> >   	ktime_t period_time;
> >   	atomic_t running;
> > +	atomic_t callback_running;
> >   	struct hrtimer timer;
> >   	struct snd_pcm_substream *substream;
> >   };
> > @@ -387,8 +388,15 @@ static enum hrtimer_restart dummy_hrtimer_callback(struct hrtimer *timer)
> >   	if (!atomic_read(&dpcm->running))
> >   		return HRTIMER_NORESTART;
> >   
> > +	atomic_inc(&dpcm->callback_running);
> >   	snd_pcm_period_elapsed(dpcm->substream);
> > +	atomic_dec(&dpcm->callback_running);
> > +	/* may be flipped during snd_pcm_period_elapsed() */
> > +	if (!atomic_read(&dpcm->running))
> > +		return HRTIMER_NORESTART;
> > +
> >   	hrtimer_forward_now(timer, dpcm->period_time);
> > +	atomic_dec(&dpcm->callback_running);
> >   	return HRTIMER_RESTART;
> >   }
> >   
> > @@ -407,7 +415,9 @@ static int dummy_hrtimer_stop(struct snd_pcm_substream *substream)
> >   	struct dummy_hrtimer_pcm *dpcm = substream->runtime->private_data;
> >   
> >   	atomic_set(&dpcm->running, 0);
> > -	hrtimer_cancel(&dpcm->timer);
> > +	/* issue hrtimer_cancel() only when called outside the callback */
> > +	if (!atomic_read(&dpcm->callback_running))
> > +		hrtimer_cancel(&dpcm->timer);
> >   	return 0;
> >   }
> >   
> > @@ -462,6 +472,7 @@ static int dummy_hrtimer_create(struct snd_pcm_substream *substream)
> >   	dpcm->timer.function = dummy_hrtimer_callback;
> >   	dpcm->substream = substream;
> >   	atomic_set(&dpcm->running, 0);
> > +	atomic_set(&dpcm->callback_running, 0);
> >   	return 0;
> >   }
> 
> >From 07d61ba2a1c0e06e914443225e194d99f2d8c58d Mon Sep 17 00:00:00 2001
> From: Takashi Sakamoto <o-takashi@sakamocchi.jp>
> Date: Fri, 1 Sep 2017 19:10:18 +0900
> Subject: [PATCH] ALSA: dummy: avoid stall due to a call of hrtimer_cancel() on
>  a callback of hrtimer
> 
> A call of 'htrimer_cancel()' on a callback of hrtimer brings endless loop
> because 'struct hrtimer_clock_base.running' is not NULL on the callback.
> In hrtimer subsystem, this member is used to indicate the instance of
> hrtimer gets callbacks and there's a helper function,
> 'hrtimer_callback_running()' to check it.
> 
> ALSA dummy driver uses hrtimer to emulate hardware interrupt per period
> of PCM buffer. When XRUN occurs on PCM substream, in a call of
> 'snd_pcm_period_elapsed()', 'struct snd_pcm_ops.stop()' is called to
> stop the substream. In current implementation, 'hrtimer_cancel()' is
> used to wait for cancellation of hrtimer. However, as described, this
> brings endless loop.

It's not only about XRUN.  When the stream finishes the draining, it
stops the stream gracefully -- that is the very normal operation.

> For this problem, this commit uses 'hrtimer_callback_running()' to
> detect whether to be on a callback of hrtimer or not, then skip
> cancellation of hrtimer in hrtimer callbacks. Furthermore, at a case of
> XRUN, hrtimer callback returns HRTIMER_NORESTART after a call of
> 'snd_pcm_period_elapsed()' to discontinue hrtimr because cancellation is
> skipped.
> 
> Signed-off-by: Takashi Sakamoto <o-takashi@sakamocchi.jp>

It's better to fold the fix into the original patch instead of
introducing a bug and fixing it.


Takashi

Re: [PATCH 23/25] ALSA/dummy: Replace tasklet with softirq hrtimer

[Takashi Sakamoto]

On p 1 2017 20:58, Takashi Iwai wrote:
>> >From 07d61ba2a1c0e06e914443225e194d99f2d8c58d Mon Sep 17 00:00:00 2001
>> From: Takashi Sakamoto <o-takashi@sakamocchi.jp>
>> Date: Fri, 1 Sep 2017 19:10:18 +0900
>> Subject: [PATCH] ALSA: dummy: avoid stall due to a call of hrtimer_cancel() on
>>   a callback of hrtimer
>>
>> A call of 'htrimer_cancel()' on a callback of hrtimer brings endless loop
>> because 'struct hrtimer_clock_base.running' is not NULL on the callback.
>> In hrtimer subsystem, this member is used to indicate the instance of
>> hrtimer gets callbacks and there's a helper function,
>> 'hrtimer_callback_running()' to check it.
>>
>> ALSA dummy driver uses hrtimer to emulate hardware interrupt per period
>> of PCM buffer. When XRUN occurs on PCM substream, in a call of
>> 'snd_pcm_period_elapsed()', 'struct snd_pcm_ops.stop()' is called to
>> stop the substream. In current implementation, 'hrtimer_cancel()' is
>> used to wait for cancellation of hrtimer. However, as described, this
>> brings endless loop.
> 
> It's not only about XRUN.  When the stream finishes the draining, it
> stops the stream gracefully -- that is the very normal operation.

I overlooked it. Thanks for your indication.

>> For this problem, this commit uses 'hrtimer_callback_running()' to
>> detect whether to be on a callback of hrtimer or not, then skip
>> cancellation of hrtimer in hrtimer callbacks. Furthermore, at a case of
>> XRUN, hrtimer callback returns HRTIMER_NORESTART after a call of
>> 'snd_pcm_period_elapsed()' to discontinue hrtimr because cancellation is
>> skipped.
>>
>> Signed-off-by: Takashi Sakamoto <o-takashi@sakamocchi.jp>
> 
> It's better to fold the fix into the original patch instead of
> introducing a bug and fixing it.

Yep. I request the authors to include this fix.


Well, in sound subsystem, there're a few drivers which uses hrtimer:
  - snd-pcsp
  - snd-sh-dac-audio
  - snd-soc-imx-pcm-fiq

As a quick glance, 'snd-sh-dac-audio' includes the same bug, too. 
Additionally, 'snd-soc-imx-pcm-fiq' maintains hrtimer with loose manner 
in a point of state of PCM substream and it shall gain the same bug if 
improved. Later, I posted some patches for them.


Thanks

Takashi Sakamoto

Re: [PATCH 23/25] ALSA/dummy: Replace tasklet with softirq hrtimer

[Takashi Sakamoto]

On p 1 2017 20:58, Takashi Iwai wrote:
>> >From 07d61ba2a1c0e06e914443225e194d99f2d8c58d Mon Sep 17 00:00:00 2001
>> From: Takashi Sakamoto <o-takashi@sakamocchi.jp>
>> Date: Fri, 1 Sep 2017 19:10:18 +0900
>> Subject: [PATCH] ALSA: dummy: avoid stall due to a call of hrtimer_cancel() on
>>   a callback of hrtimer
>>
>> A call of 'htrimer_cancel()' on a callback of hrtimer brings endless loop
>> because 'struct hrtimer_clock_base.running' is not NULL on the callback.
>> In hrtimer subsystem, this member is used to indicate the instance of
>> hrtimer gets callbacks and there's a helper function,
>> 'hrtimer_callback_running()' to check it.
>>
>> ALSA dummy driver uses hrtimer to emulate hardware interrupt per period
>> of PCM buffer. When XRUN occurs on PCM substream, in a call of
>> 'snd_pcm_period_elapsed()', 'struct snd_pcm_ops.stop()' is called to
>> stop the substream. In current implementation, 'hrtimer_cancel()' is
>> used to wait for cancellation of hrtimer. However, as described, this
>> brings endless loop.
> 
> It's not only about XRUN.  When the stream finishes the draining, it
> stops the stream gracefully -- that is the very normal operation.

I overlooked it. Thanks for your indication.

>> For this problem, this commit uses 'hrtimer_callback_running()' to
>> detect whether to be on a callback of hrtimer or not, then skip
>> cancellation of hrtimer in hrtimer callbacks. Furthermore, at a case of
>> XRUN, hrtimer callback returns HRTIMER_NORESTART after a call of
>> 'snd_pcm_period_elapsed()' to discontinue hrtimr because cancellation is
>> skipped.
>>
>> Signed-off-by: Takashi Sakamoto <o-takashi@sakamocchi.jp>
> 
> It's better to fold the fix into the original patch instead of
> introducing a bug and fixing it.

Yep. I request the authors to include this fix.


Well, in sound subsystem, there're a few drivers which uses hrtimer:
  - snd-pcsp
  - snd-sh-dac-audio
  - snd-soc-imx-pcm-fiq

As a quick glance, 'snd-sh-dac-audio' includes the same bug, too. 
Additionally, 'snd-soc-imx-pcm-fiq' maintains hrtimer with loose manner 
in a point of state of PCM substream and it shall gain the same bug if 
improved. Later, I posted some patches for them.


Thanks

Takashi Sakamoto

Re: [PATCH 23/25] ALSA/dummy: Replace tasklet with softirq hrtimer

[Takashi Sakamoto]

Hi,

On Sep 2 2017 10:19, Takashi Sakamoto wrote:
> Well, in sound subsystem, there're a few drivers which uses hrtimer:
>   - snd-pcsp
>   - snd-sh-dac-audio
>   - snd-soc-imx-pcm-fiq
> 
> As a quick glance, 'snd-sh-dac-audio' includes the same bug, too. 
> Additionally, 'snd-soc-imx-pcm-fiq' maintains hrtimer with loose manner 
> in a point of state of PCM substream and it shall gain the same bug if 
> improved. Later, I posted some patches for them.

After reading code thoroughly, I conclude that no need to fix these two
drivers. They're programmed with own protections. The former
(snd-sh-dac-audio) has 'struct snd_sh_dac.empty' and the latter
(snd-soc-imx-pcm-fiq) has 'struct imx_pcm_runtime_data.playing' and
'.capturing', to avoid cancellation of hrtimer on hrtimer callback.

These ways are not necessarily efficient but actually have no trouble.
I leave them as is.


Regards

Takashi Sakamoto

[PATCH 23/25 v2] ALSA/dummy: Replace tasklet with softirq hrtimer

[Sebastian Andrzej Siewior]

From: Thomas Gleixner <tglx@linutronix.de>

The tasklet is used to defer the execution of snd_pcm_period_elapsed() to
the softirq context. Using the CLOCK_MONOTONIC_SOFT base invokes the timer
callback in softirq context as well which renders the tasklet useless.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Anna-Maria Gleixner <anna-maria@linutronix.de>
Cc: Jaroslav Kysela <perex@perex.cz>
Cc: Takashi Iwai <tiwai@suse.com>
Cc: Takashi Sakamoto <o-takashi@sakamocchi.jp>
Cc: alsa-devel@alsa-project.org
[o-takashi: avoid stall due to a call of hrtimer_cancel() on a callback
            of hrtimer]
Signed-off-by: Takashi Sakamoto <o-takashi@sakamocchi.jp>
Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
---
On 2017-09-02 10:19:45 [+0900], Takashi Sakamoto wrote:
> Yep. I request the authors to include this 
Thank you for providing a fix.

v1…v2: merged Takashi Sakamoto fixup of the original patch into v2.

So this patch now is okay?

 sound/drivers/dummy.c | 23 +++++++++--------------
 1 file changed, 9 insertions(+), 14 deletions(-)

diff --git a/sound/drivers/dummy.c b/sound/drivers/dummy.c
index dd5ed037adf2..3d01fe17ed36 100644
--- a/sound/drivers/dummy.c
+++ b/sound/drivers/dummy.c
@@ -376,17 +376,9 @@ struct dummy_hrtimer_pcm {
 	ktime_t period_time;
 	atomic_t running;
 	struct hrtimer timer;
-	struct tasklet_struct tasklet;
 	struct snd_pcm_substream *substream;
 };
 
-static void dummy_hrtimer_pcm_elapsed(unsigned long priv)
-{
-	struct dummy_hrtimer_pcm *dpcm = (struct dummy_hrtimer_pcm *)priv;
-	if (atomic_read(&dpcm->running))
-		snd_pcm_period_elapsed(dpcm->substream);
-}
-
 static enum hrtimer_restart dummy_hrtimer_callback(struct hrtimer *timer)
 {
 	struct dummy_hrtimer_pcm *dpcm;
@@ -394,7 +386,12 @@ static enum hrtimer_restart dummy_hrtimer_callback(struct hrtimer *timer)
 	dpcm = container_of(timer, struct dummy_hrtimer_pcm, timer);
 	if (!atomic_read(&dpcm->running))
 		return HRTIMER_NORESTART;
-	tasklet_schedule(&dpcm->tasklet);
+
+	/* In a case of XRUN, this calls .trigger to stop PCM substream. */
+	snd_pcm_period_elapsed(dpcm->substream);
+	if (!atomic_read(&dpcm->running))
+		return HRTIMER_NORESTART;
+
 	hrtimer_forward_now(timer, dpcm->period_time);
 	return HRTIMER_RESTART;
 }
@@ -414,14 +411,14 @@ static int dummy_hrtimer_stop(struct snd_pcm_substream *substream)
 	struct dummy_hrtimer_pcm *dpcm = substream->runtime->private_data;
 
 	atomic_set(&dpcm->running, 0);
-	hrtimer_cancel(&dpcm->timer);
+	if (!hrtimer_callback_running(&dpcm->timer))
+		hrtimer_cancel(&dpcm->timer);
 	return 0;
 }
 
 static inline void dummy_hrtimer_sync(struct dummy_hrtimer_pcm *dpcm)
 {
 	hrtimer_cancel(&dpcm->timer);
-	tasklet_kill(&dpcm->tasklet);
 }
 
 static snd_pcm_uframes_t
@@ -466,12 +463,10 @@ static int dummy_hrtimer_create(struct snd_pcm_substream *substream)
 	if (!dpcm)
 		return -ENOMEM;
 	substream->runtime->private_data = dpcm;
-	hrtimer_init(&dpcm->timer, CLOCK_MONOTONIC, HRTIMER_MODE_REL);
+	hrtimer_init(&dpcm->timer, CLOCK_MONOTONIC_SOFT, HRTIMER_MODE_REL);
 	dpcm->timer.function = dummy_hrtimer_callback;
 	dpcm->substream = substream;
 	atomic_set(&dpcm->running, 0);
-	tasklet_init(&dpcm->tasklet, dummy_hrtimer_pcm_elapsed,
-		     (unsigned long)dpcm);
 	return 0;
 }
 
-- 
2.14.1

[PATCH 23/25 v2] ALSA/dummy: Replace tasklet with softirq hrtimer

[Sebastian Andrzej Siewior]

From: Thomas Gleixner <tglx@linutronix.de>

The tasklet is used to defer the execution of snd_pcm_period_elapsed() to
the softirq context. Using the CLOCK_MONOTONIC_SOFT base invokes the timer
callback in softirq context as well which renders the tasklet useless.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Anna-Maria Gleixner <anna-maria@linutronix.de>
Cc: Jaroslav Kysela <perex@perex.cz>
Cc: Takashi Iwai <tiwai@suse.com>
Cc: Takashi Sakamoto <o-takashi@sakamocchi.jp>
Cc: alsa-devel@alsa-project.org
[o-takashi: avoid stall due to a call of hrtimer_cancel() on a callback
            of hrtimer]
Signed-off-by: Takashi Sakamoto <o-takashi@sakamocchi.jp>
Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
---
On 2017-09-02 10:19:45 [+0900], Takashi Sakamoto wrote:
> Yep. I request the authors to include this 
Thank you for providing a fix.

v1…v2: merged Takashi Sakamoto fixup of the original patch into v2.

So this patch now is okay?

 sound/drivers/dummy.c | 23 +++++++++--------------
 1 file changed, 9 insertions(+), 14 deletions(-)

diff --git a/sound/drivers/dummy.c b/sound/drivers/dummy.c
index dd5ed037adf2..3d01fe17ed36 100644
--- a/sound/drivers/dummy.c
+++ b/sound/drivers/dummy.c
@@ -376,17 +376,9 @@ struct dummy_hrtimer_pcm {
 	ktime_t period_time;
 	atomic_t running;
 	struct hrtimer timer;
-	struct tasklet_struct tasklet;
 	struct snd_pcm_substream *substream;
 };
 
-static void dummy_hrtimer_pcm_elapsed(unsigned long priv)
-{
-	struct dummy_hrtimer_pcm *dpcm = (struct dummy_hrtimer_pcm *)priv;
-	if (atomic_read(&dpcm->running))
-		snd_pcm_period_elapsed(dpcm->substream);
-}
-
 static enum hrtimer_restart dummy_hrtimer_callback(struct hrtimer *timer)
 {
 	struct dummy_hrtimer_pcm *dpcm;
@@ -394,7 +386,12 @@ static enum hrtimer_restart dummy_hrtimer_callback(struct hrtimer *timer)
 	dpcm = container_of(timer, struct dummy_hrtimer_pcm, timer);
 	if (!atomic_read(&dpcm->running))
 		return HRTIMER_NORESTART;
-	tasklet_schedule(&dpcm->tasklet);
+
+	/* In a case of XRUN, this calls .trigger to stop PCM substream. */
+	snd_pcm_period_elapsed(dpcm->substream);
+	if (!atomic_read(&dpcm->running))
+		return HRTIMER_NORESTART;
+
 	hrtimer_forward_now(timer, dpcm->period_time);
 	return HRTIMER_RESTART;
 }
@@ -414,14 +411,14 @@ static int dummy_hrtimer_stop(struct snd_pcm_substream *substream)
 	struct dummy_hrtimer_pcm *dpcm = substream->runtime->private_data;
 
 	atomic_set(&dpcm->running, 0);
-	hrtimer_cancel(&dpcm->timer);
+	if (!hrtimer_callback_running(&dpcm->timer))
+		hrtimer_cancel(&dpcm->timer);
 	return 0;
 }
 
 static inline void dummy_hrtimer_sync(struct dummy_hrtimer_pcm *dpcm)
 {
 	hrtimer_cancel(&dpcm->timer);
-	tasklet_kill(&dpcm->tasklet);
 }
 
 static snd_pcm_uframes_t
@@ -466,12 +463,10 @@ static int dummy_hrtimer_create(struct snd_pcm_substream *substream)
 	if (!dpcm)
 		return -ENOMEM;
 	substream->runtime->private_data = dpcm;
-	hrtimer_init(&dpcm->timer, CLOCK_MONOTONIC, HRTIMER_MODE_REL);
+	hrtimer_init(&dpcm->timer, CLOCK_MONOTONIC_SOFT, HRTIMER_MODE_REL);
 	dpcm->timer.function = dummy_hrtimer_callback;
 	dpcm->substream = substream;
 	atomic_set(&dpcm->running, 0);
-	tasklet_init(&dpcm->tasklet, dummy_hrtimer_pcm_elapsed,
-		     (unsigned long)dpcm);
 	return 0;
 }
 
-- 
2.14.1

_______________________________________________
Alsa-devel mailing list
Alsa-devel@alsa-project.org
http://mailman.alsa-project.org/mailman/listinfo/alsa-devel

Re: [PATCH 23/25 v2] ALSA/dummy: Replace tasklet with softirq hrtimer

[Takashi Iwai]

On Tue, 05 Sep 2017 17:53:51 +0200,
Sebastian Andrzej Siewior wrote:
> 
> From: Thomas Gleixner <tglx@linutronix.de>
> 
> The tasklet is used to defer the execution of snd_pcm_period_elapsed() to
> the softirq context. Using the CLOCK_MONOTONIC_SOFT base invokes the timer
> callback in softirq context as well which renders the tasklet useless.
> 
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> Signed-off-by: Anna-Maria Gleixner <anna-maria@linutronix.de>
> Cc: Jaroslav Kysela <perex@perex.cz>
> Cc: Takashi Iwai <tiwai@suse.com>
> Cc: Takashi Sakamoto <o-takashi@sakamocchi.jp>
> Cc: alsa-devel@alsa-project.org
> [o-takashi: avoid stall due to a call of hrtimer_cancel() on a callback
>             of hrtimer]
> Signed-off-by: Takashi Sakamoto <o-takashi@sakamocchi.jp>
> Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
> ---
> On 2017-09-02 10:19:45 [+0900], Takashi Sakamoto wrote:
> > Yep. I request the authors to include this 
> Thank you for providing a fix.
> 
> v1…v2: merged Takashi Sakamoto fixup of the original patch into v2.
> 
> So this patch now is okay?

Note that you can try it by yourself easily, as it's a dummy driver
that doesn't need anything special.  Just run aplay for that device
(e.g. aplay -Dplughw:2 for card#2) can reproduce the original
problem.

> @@ -394,7 +386,12 @@ static enum hrtimer_restart dummy_hrtimer_callback(struct hrtimer *timer)
>  	dpcm = container_of(timer, struct dummy_hrtimer_pcm, timer);
>  	if (!atomic_read(&dpcm->running))
>  		return HRTIMER_NORESTART;
> -	tasklet_schedule(&dpcm->tasklet);
> +
> +	/* In a case of XRUN, this calls .trigger to stop PCM substream. */

As mentioned, the stop happens not only with XRUN but also in a normal
situation by draining.

Other than that, looks OK to me (but not tested it).


thanks,

Takashi


> +	snd_pcm_period_elapsed(dpcm->substream);
> +	if (!atomic_read(&dpcm->running))
> +		return HRTIMER_NORESTART;
> +
>  	hrtimer_forward_now(timer, dpcm->period_time);
>  	return HRTIMER_RESTART;
>  }
> @@ -414,14 +411,14 @@ static int dummy_hrtimer_stop(struct snd_pcm_substream *substream)
>  	struct dummy_hrtimer_pcm *dpcm = substream->runtime->private_data;
>  
>  	atomic_set(&dpcm->running, 0);
> -	hrtimer_cancel(&dpcm->timer);
> +	if (!hrtimer_callback_running(&dpcm->timer))
> +		hrtimer_cancel(&dpcm->timer);
>  	return 0;
>  }
>  
>  static inline void dummy_hrtimer_sync(struct dummy_hrtimer_pcm *dpcm)
>  {
>  	hrtimer_cancel(&dpcm->timer);
> -	tasklet_kill(&dpcm->tasklet);
>  }
>  
>  static snd_pcm_uframes_t
> @@ -466,12 +463,10 @@ static int dummy_hrtimer_create(struct snd_pcm_substream *substream)
>  	if (!dpcm)
>  		return -ENOMEM;
>  	substream->runtime->private_data = dpcm;
> -	hrtimer_init(&dpcm->timer, CLOCK_MONOTONIC, HRTIMER_MODE_REL);
> +	hrtimer_init(&dpcm->timer, CLOCK_MONOTONIC_SOFT, HRTIMER_MODE_REL);
>  	dpcm->timer.function = dummy_hrtimer_callback;
>  	dpcm->substream = substream;
>  	atomic_set(&dpcm->running, 0);
> -	tasklet_init(&dpcm->tasklet, dummy_hrtimer_pcm_elapsed,
> -		     (unsigned long)dpcm);
>  	return 0;
>  }
>  
> -- 
> 2.14.1
> 

Re: [PATCH 23/25 v2] ALSA/dummy: Replace tasklet with softirq hrtimer

[Takashi Iwai]

On Tue, 05 Sep 2017 17:53:51 +0200,
Sebastian Andrzej Siewior wrote:
> 
> From: Thomas Gleixner <tglx@linutronix.de>
> 
> The tasklet is used to defer the execution of snd_pcm_period_elapsed() to
> the softirq context. Using the CLOCK_MONOTONIC_SOFT base invokes the timer
> callback in softirq context as well which renders the tasklet useless.
> 
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> Signed-off-by: Anna-Maria Gleixner <anna-maria@linutronix.de>
> Cc: Jaroslav Kysela <perex@perex.cz>
> Cc: Takashi Iwai <tiwai@suse.com>
> Cc: Takashi Sakamoto <o-takashi@sakamocchi.jp>
> Cc: alsa-devel@alsa-project.org
> [o-takashi: avoid stall due to a call of hrtimer_cancel() on a callback
>             of hrtimer]
> Signed-off-by: Takashi Sakamoto <o-takashi@sakamocchi.jp>
> Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
> ---
> On 2017-09-02 10:19:45 [+0900], Takashi Sakamoto wrote:
> > Yep. I request the authors to include this 
> Thank you for providing a fix.
> 
> v1…v2: merged Takashi Sakamoto fixup of the original patch into v2.
> 
> So this patch now is okay?

Note that you can try it by yourself easily, as it's a dummy driver
that doesn't need anything special.  Just run aplay for that device
(e.g. aplay -Dplughw:2 for card#2) can reproduce the original
problem.

> @@ -394,7 +386,12 @@ static enum hrtimer_restart dummy_hrtimer_callback(struct hrtimer *timer)
>  	dpcm = container_of(timer, struct dummy_hrtimer_pcm, timer);
>  	if (!atomic_read(&dpcm->running))
>  		return HRTIMER_NORESTART;
> -	tasklet_schedule(&dpcm->tasklet);
> +
> +	/* In a case of XRUN, this calls .trigger to stop PCM substream. */

As mentioned, the stop happens not only with XRUN but also in a normal
situation by draining.

Other than that, looks OK to me (but not tested it).


thanks,

Takashi


> +	snd_pcm_period_elapsed(dpcm->substream);
> +	if (!atomic_read(&dpcm->running))
> +		return HRTIMER_NORESTART;
> +
>  	hrtimer_forward_now(timer, dpcm->period_time);
>  	return HRTIMER_RESTART;
>  }
> @@ -414,14 +411,14 @@ static int dummy_hrtimer_stop(struct snd_pcm_substream *substream)
>  	struct dummy_hrtimer_pcm *dpcm = substream->runtime->private_data;
>  
>  	atomic_set(&dpcm->running, 0);
> -	hrtimer_cancel(&dpcm->timer);
> +	if (!hrtimer_callback_running(&dpcm->timer))
> +		hrtimer_cancel(&dpcm->timer);
>  	return 0;
>  }
>  
>  static inline void dummy_hrtimer_sync(struct dummy_hrtimer_pcm *dpcm)
>  {
>  	hrtimer_cancel(&dpcm->timer);
> -	tasklet_kill(&dpcm->tasklet);
>  }
>  
>  static snd_pcm_uframes_t
> @@ -466,12 +463,10 @@ static int dummy_hrtimer_create(struct snd_pcm_substream *substream)
>  	if (!dpcm)
>  		return -ENOMEM;
>  	substream->runtime->private_data = dpcm;
> -	hrtimer_init(&dpcm->timer, CLOCK_MONOTONIC, HRTIMER_MODE_REL);
> +	hrtimer_init(&dpcm->timer, CLOCK_MONOTONIC_SOFT, HRTIMER_MODE_REL);
>  	dpcm->timer.function = dummy_hrtimer_callback;
>  	dpcm->substream = substream;
>  	atomic_set(&dpcm->running, 0);
> -	tasklet_init(&dpcm->tasklet, dummy_hrtimer_pcm_elapsed,
> -		     (unsigned long)dpcm);
>  	return 0;
>  }
>  
> -- 
> 2.14.1
> 
_______________________________________________
Alsa-devel mailing list
Alsa-devel@alsa-project.org
http://mailman.alsa-project.org/mailman/listinfo/alsa-devel

Re: [PATCH 23/25 v2] ALSA/dummy: Replace tasklet with softirq hrtimer

[Takashi Sakamoto]

On Sep 6 2017 00:53, Sebastian Andrzej Siewior wrote:
> From: Thomas Gleixner <tglx@linutronix.de>
> 
> The tasklet is used to defer the execution of snd_pcm_period_elapsed() to
> the softirq context. Using the CLOCK_MONOTONIC_SOFT base invokes the timer
> callback in softirq context as well which renders the tasklet useless.
> 
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> Signed-off-by: Anna-Maria Gleixner <anna-maria@linutronix.de>
> Cc: Jaroslav Kysela <perex@perex.cz>
> Cc: Takashi Iwai <tiwai@suse.com>
> Cc: Takashi Sakamoto <o-takashi@sakamocchi.jp>
> Cc: alsa-devel@alsa-project.org
> [o-takashi: avoid stall due to a call of hrtimer_cancel() on a callback
>              of hrtimer]
> Signed-off-by: Takashi Sakamoto <o-takashi@sakamocchi.jp>
> Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
> ---
> On 2017-09-02 10:19:45 [+0900], Takashi Sakamoto wrote:
>> Yep. I request the authors to include this
> Thank you for providing a fix.
> 
> v1…v2: merged Takashi Sakamoto fixup of the original patch into v2.
> 
> So this patch now is okay?

I have a small nitpick.

>   sound/drivers/dummy.c | 23 +++++++++--------------
>   1 file changed, 9 insertions(+), 14 deletions(-)
> 
> diff --git a/sound/drivers/dummy.c b/sound/drivers/dummy.c
> index dd5ed037adf2..3d01fe17ed36 100644
> --- a/sound/drivers/dummy.c
> +++ b/sound/drivers/dummy.c
> @@ -376,17 +376,9 @@ struct dummy_hrtimer_pcm {
>   	ktime_t period_time;
>   	atomic_t running;
>   	struct hrtimer timer;
> -	struct tasklet_struct tasklet;
>   	struct snd_pcm_substream *substream;
>   };
>   
> -static void dummy_hrtimer_pcm_elapsed(unsigned long priv)
> -{
> -	struct dummy_hrtimer_pcm *dpcm = (struct dummy_hrtimer_pcm *)priv;
> -	if (atomic_read(&dpcm->running))
> -		snd_pcm_period_elapsed(dpcm->substream);
> -}
> -
>   static enum hrtimer_restart dummy_hrtimer_callback(struct hrtimer *timer)
>   {
>   	struct dummy_hrtimer_pcm *dpcm;
> @@ -394,7 +386,12 @@ static enum hrtimer_restart dummy_hrtimer_callback(struct hrtimer *timer)
>   	dpcm = container_of(timer, struct dummy_hrtimer_pcm, timer);
>   	if (!atomic_read(&dpcm->running))
>   		return HRTIMER_NORESTART;
> -	tasklet_schedule(&dpcm->tasklet);
> +
> +	/* In a case of XRUN, this calls .trigger to stop PCM substream. */

As Iwai-san mentioned, in this function, .trigger can be called in two 
cases; XRUN occurs and draining is done. Thus, let you change the 
comment as 'In cases of XRUN and draining, this calls .trigger to stop 
PCM substream.'. I'm sorry to trouble you.

snd_pcm_period_elapsed()
->snd_pcm_update_hw_ptr0()
   ->snd_pcm_update_state()
     ->snd_pcm_drain_done()
       ...
       ->.trigger(TRIGGER_STOP)
     ->xrun()
       ->snd_pcm_stop()
         ...
         ->.trigger(TRIGGER_STOP)

> +	snd_pcm_period_elapsed(dpcm->substream);
> +	if (!atomic_read(&dpcm->running))
> +		return HRTIMER_NORESTART;
> +
>   	hrtimer_forward_now(timer, dpcm->period_time);
>   	return HRTIMER_RESTART;
>   }
> @@ -414,14 +411,14 @@ static int dummy_hrtimer_stop(struct snd_pcm_substream *substream)
>   	struct dummy_hrtimer_pcm *dpcm = substream->runtime->private_data;
>   
>   	atomic_set(&dpcm->running, 0);
> -	hrtimer_cancel(&dpcm->timer);
> +	if (!hrtimer_callback_running(&dpcm->timer))
> +		hrtimer_cancel(&dpcm->timer);
>   	return 0;
>   }
>   
>   static inline void dummy_hrtimer_sync(struct dummy_hrtimer_pcm *dpcm)
>   {
>   	hrtimer_cancel(&dpcm->timer);
> -	tasklet_kill(&dpcm->tasklet);
>   }
>   
>   static snd_pcm_uframes_t
> @@ -466,12 +463,10 @@ static int dummy_hrtimer_create(struct snd_pcm_substream *substream)
>   	if (!dpcm)
>   		return -ENOMEM;
>   	substream->runtime->private_data = dpcm;
> -	hrtimer_init(&dpcm->timer, CLOCK_MONOTONIC, HRTIMER_MODE_REL);
> +	hrtimer_init(&dpcm->timer, CLOCK_MONOTONIC_SOFT, HRTIMER_MODE_REL);
>   	dpcm->timer.function = dummy_hrtimer_callback;
>   	dpcm->substream = substream;
>   	atomic_set(&dpcm->running, 0);
> -	tasklet_init(&dpcm->tasklet, dummy_hrtimer_pcm_elapsed,
> -		     (unsigned long)dpcm);
>   	return 0;
>   }


Regards

Takashi Sakamoto

[PATCH 23/25 v3] ALSA/dummy: Replace tasklet with softirq hrtimer

[Sebastian Andrzej Siewior]

From: Thomas Gleixner <tglx@linutronix.de>

The tasklet is used to defer the execution of snd_pcm_period_elapsed() to
the softirq context. Using the CLOCK_MONOTONIC_SOFT base invokes the timer
callback in softirq context as well which renders the tasklet useless.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Anna-Maria Gleixner <anna-maria@linutronix.de>
Cc: Jaroslav Kysela <perex@perex.cz>
Cc: Takashi Iwai <tiwai@suse.com>
Cc: Takashi Sakamoto <o-takashi@sakamocchi.jp>
Cc: alsa-devel@alsa-project.org
[o-takashi: avoid stall due to a call of hrtimer_cancel() on a callback
            of hrtimer]
Signed-off-by: Takashi Sakamoto <o-takashi@sakamocchi.jp>
Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
---
On 2017-09-06 01:05:43 [+0900], Takashi Sakamoto wrote:
> As Iwai-san mentioned, in this function, .trigger can be called in two
> cases; XRUN occurs and draining is done. Thus, let you change the comment as
> 'In cases of XRUN and draining, this calls .trigger to stop PCM substream.'.
> I'm sorry to trouble you.
> 
> snd_pcm_period_elapsed()
> ->snd_pcm_update_hw_ptr0()
>   ->snd_pcm_update_state()
>     ->snd_pcm_drain_done()
>       ...
>       ->.trigger(TRIGGER_STOP)
>     ->xrun()
>       ->snd_pcm_stop()
>         ...
>         ->.trigger(TRIGGER_STOP)
> 

I think you asked me just to update the comment. Did I do it right?

v2…v3: updated the comment as per Takashi Sakamoto's suggestion.
v1…v2: merged Takashi Sakamoto fixup of the original patch into v2.

 sound/drivers/dummy.c | 25 +++++++++++--------------
 1 file changed, 11 insertions(+), 14 deletions(-)

diff --git a/sound/drivers/dummy.c b/sound/drivers/dummy.c
index dd5ed037adf2..cdd851286f92 100644
--- a/sound/drivers/dummy.c
+++ b/sound/drivers/dummy.c
@@ -376,17 +376,9 @@ struct dummy_hrtimer_pcm {
 	ktime_t period_time;
 	atomic_t running;
 	struct hrtimer timer;
-	struct tasklet_struct tasklet;
 	struct snd_pcm_substream *substream;
 };
 
-static void dummy_hrtimer_pcm_elapsed(unsigned long priv)
-{
-	struct dummy_hrtimer_pcm *dpcm = (struct dummy_hrtimer_pcm *)priv;
-	if (atomic_read(&dpcm->running))
-		snd_pcm_period_elapsed(dpcm->substream);
-}
-
 static enum hrtimer_restart dummy_hrtimer_callback(struct hrtimer *timer)
 {
 	struct dummy_hrtimer_pcm *dpcm;
@@ -394,7 +386,14 @@ static enum hrtimer_restart dummy_hrtimer_callback(struct hrtimer *timer)
 	dpcm = container_of(timer, struct dummy_hrtimer_pcm, timer);
 	if (!atomic_read(&dpcm->running))
 		return HRTIMER_NORESTART;
-	tasklet_schedule(&dpcm->tasklet);
+	/*
+	 * In cases of XRUN and draining, this calls .trigger to stop PCM
+	 * substream.
+	 */
+	snd_pcm_period_elapsed(dpcm->substream);
+	if (!atomic_read(&dpcm->running))
+		return HRTIMER_NORESTART;
+
 	hrtimer_forward_now(timer, dpcm->period_time);
 	return HRTIMER_RESTART;
 }
@@ -414,14 +413,14 @@ static int dummy_hrtimer_stop(struct snd_pcm_substream *substream)
 	struct dummy_hrtimer_pcm *dpcm = substream->runtime->private_data;
 
 	atomic_set(&dpcm->running, 0);
-	hrtimer_cancel(&dpcm->timer);
+	if (!hrtimer_callback_running(&dpcm->timer))
+		hrtimer_cancel(&dpcm->timer);
 	return 0;
 }
 
 static inline void dummy_hrtimer_sync(struct dummy_hrtimer_pcm *dpcm)
 {
 	hrtimer_cancel(&dpcm->timer);
-	tasklet_kill(&dpcm->tasklet);
 }
 
 static snd_pcm_uframes_t
@@ -466,12 +465,10 @@ static int dummy_hrtimer_create(struct snd_pcm_substream *substream)
 	if (!dpcm)
 		return -ENOMEM;
 	substream->runtime->private_data = dpcm;
-	hrtimer_init(&dpcm->timer, CLOCK_MONOTONIC, HRTIMER_MODE_REL);
+	hrtimer_init(&dpcm->timer, CLOCK_MONOTONIC_SOFT, HRTIMER_MODE_REL);
 	dpcm->timer.function = dummy_hrtimer_callback;
 	dpcm->substream = substream;
 	atomic_set(&dpcm->running, 0);
-	tasklet_init(&dpcm->tasklet, dummy_hrtimer_pcm_elapsed,
-		     (unsigned long)dpcm);
 	return 0;
 }
 
-- 
2.14.1

[PATCH 23/25 v3] ALSA/dummy: Replace tasklet with softirq hrtimer

[Sebastian Andrzej Siewior]

From: Thomas Gleixner <tglx@linutronix.de>

The tasklet is used to defer the execution of snd_pcm_period_elapsed() to
the softirq context. Using the CLOCK_MONOTONIC_SOFT base invokes the timer
callback in softirq context as well which renders the tasklet useless.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Anna-Maria Gleixner <anna-maria@linutronix.de>
Cc: Jaroslav Kysela <perex@perex.cz>
Cc: Takashi Iwai <tiwai@suse.com>
Cc: Takashi Sakamoto <o-takashi@sakamocchi.jp>
Cc: alsa-devel@alsa-project.org
[o-takashi: avoid stall due to a call of hrtimer_cancel() on a callback
            of hrtimer]
Signed-off-by: Takashi Sakamoto <o-takashi@sakamocchi.jp>
Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
---
On 2017-09-06 01:05:43 [+0900], Takashi Sakamoto wrote:
> As Iwai-san mentioned, in this function, .trigger can be called in two
> cases; XRUN occurs and draining is done. Thus, let you change the comment as
> 'In cases of XRUN and draining, this calls .trigger to stop PCM substream.'.
> I'm sorry to trouble you.
> 
> snd_pcm_period_elapsed()
> ->snd_pcm_update_hw_ptr0()
>   ->snd_pcm_update_state()
>     ->snd_pcm_drain_done()
>       ...
>       ->.trigger(TRIGGER_STOP)
>     ->xrun()
>       ->snd_pcm_stop()
>         ...
>         ->.trigger(TRIGGER_STOP)
> 

I think you asked me just to update the comment. Did I do it right?

v2…v3: updated the comment as per Takashi Sakamoto's suggestion.
v1…v2: merged Takashi Sakamoto fixup of the original patch into v2.

 sound/drivers/dummy.c | 25 +++++++++++--------------
 1 file changed, 11 insertions(+), 14 deletions(-)

diff --git a/sound/drivers/dummy.c b/sound/drivers/dummy.c
index dd5ed037adf2..cdd851286f92 100644
--- a/sound/drivers/dummy.c
+++ b/sound/drivers/dummy.c
@@ -376,17 +376,9 @@ struct dummy_hrtimer_pcm {
 	ktime_t period_time;
 	atomic_t running;
 	struct hrtimer timer;
-	struct tasklet_struct tasklet;
 	struct snd_pcm_substream *substream;
 };
 
-static void dummy_hrtimer_pcm_elapsed(unsigned long priv)
-{
-	struct dummy_hrtimer_pcm *dpcm = (struct dummy_hrtimer_pcm *)priv;
-	if (atomic_read(&dpcm->running))
-		snd_pcm_period_elapsed(dpcm->substream);
-}
-
 static enum hrtimer_restart dummy_hrtimer_callback(struct hrtimer *timer)
 {
 	struct dummy_hrtimer_pcm *dpcm;
@@ -394,7 +386,14 @@ static enum hrtimer_restart dummy_hrtimer_callback(struct hrtimer *timer)
 	dpcm = container_of(timer, struct dummy_hrtimer_pcm, timer);
 	if (!atomic_read(&dpcm->running))
 		return HRTIMER_NORESTART;
-	tasklet_schedule(&dpcm->tasklet);
+	/*
+	 * In cases of XRUN and draining, this calls .trigger to stop PCM
+	 * substream.
+	 */
+	snd_pcm_period_elapsed(dpcm->substream);
+	if (!atomic_read(&dpcm->running))
+		return HRTIMER_NORESTART;
+
 	hrtimer_forward_now(timer, dpcm->period_time);
 	return HRTIMER_RESTART;
 }
@@ -414,14 +413,14 @@ static int dummy_hrtimer_stop(struct snd_pcm_substream *substream)
 	struct dummy_hrtimer_pcm *dpcm = substream->runtime->private_data;
 
 	atomic_set(&dpcm->running, 0);
-	hrtimer_cancel(&dpcm->timer);
+	if (!hrtimer_callback_running(&dpcm->timer))
+		hrtimer_cancel(&dpcm->timer);
 	return 0;
 }
 
 static inline void dummy_hrtimer_sync(struct dummy_hrtimer_pcm *dpcm)
 {
 	hrtimer_cancel(&dpcm->timer);
-	tasklet_kill(&dpcm->tasklet);
 }
 
 static snd_pcm_uframes_t
@@ -466,12 +465,10 @@ static int dummy_hrtimer_create(struct snd_pcm_substream *substream)
 	if (!dpcm)
 		return -ENOMEM;
 	substream->runtime->private_data = dpcm;
-	hrtimer_init(&dpcm->timer, CLOCK_MONOTONIC, HRTIMER_MODE_REL);
+	hrtimer_init(&dpcm->timer, CLOCK_MONOTONIC_SOFT, HRTIMER_MODE_REL);
 	dpcm->timer.function = dummy_hrtimer_callback;
 	dpcm->substream = substream;
 	atomic_set(&dpcm->running, 0);
-	tasklet_init(&dpcm->tasklet, dummy_hrtimer_pcm_elapsed,
-		     (unsigned long)dpcm);
 	return 0;
 }
 
-- 
2.14.1

_______________________________________________
Alsa-devel mailing list
Alsa-devel@alsa-project.org
http://mailman.alsa-project.org/mailman/listinfo/alsa-devel

Re: [PATCH 23/25 v3] ALSA/dummy: Replace tasklet with softirq hrtimer

[Takashi Sakamoto]

On Sep 6 2017 01:18, Sebastian Andrzej Siewior wrote:
> From: Thomas Gleixner <tglx@linutronix.de>
> 
> The tasklet is used to defer the execution of snd_pcm_period_elapsed() to
> the softirq context. Using the CLOCK_MONOTONIC_SOFT base invokes the timer
> callback in softirq context as well which renders the tasklet useless.
> 
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> Signed-off-by: Anna-Maria Gleixner <anna-maria@linutronix.de>
> Cc: Jaroslav Kysela <perex@perex.cz>
> Cc: Takashi Iwai <tiwai@suse.com>
> Cc: Takashi Sakamoto <o-takashi@sakamocchi.jp>
> Cc: alsa-devel@alsa-project.org
> [o-takashi: avoid stall due to a call of hrtimer_cancel() on a callback
>              of hrtimer]
> Signed-off-by: Takashi Sakamoto <o-takashi@sakamocchi.jp>
> Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
> ---
> On 2017-09-06 01:05:43 [+0900], Takashi Sakamoto wrote:
>> As Iwai-san mentioned, in this function, .trigger can be called in two
>> cases; XRUN occurs and draining is done. Thus, let you change the comment as
>> 'In cases of XRUN and draining, this calls .trigger to stop PCM substream.'.
>> I'm sorry to trouble you.
>>
>> snd_pcm_period_elapsed()
>> ->snd_pcm_update_hw_ptr0()
>>    ->snd_pcm_update_state()
>>      ->snd_pcm_drain_done()
>>        ...
>>        ->.trigger(TRIGGER_STOP)
>>      ->xrun()
>>        ->snd_pcm_stop()
>>          ...
>>          ->.trigger(TRIGGER_STOP)
>>
> 
> I think you asked me just to update the comment. Did I do it right?
> 
> v2…v3: updated the comment as per Takashi Sakamoto's suggestion.
> v1…v2: merged Takashi Sakamoto fixup of the original patch into v2.
> 
>   sound/drivers/dummy.c | 25 +++++++++++--------------
>   1 file changed, 11 insertions(+), 14 deletions(-)

This Looks good to me. I did quick test and confirmed that it brings no 
stalls.

Tested-by: Takashi Sakamoto <o-takashi@sakamocchi.jp>

> diff --git a/sound/drivers/dummy.c b/sound/drivers/dummy.c
> index dd5ed037adf2..cdd851286f92 100644
> --- a/sound/drivers/dummy.c
> +++ b/sound/drivers/dummy.c
> @@ -376,17 +376,9 @@ struct dummy_hrtimer_pcm {
>   	ktime_t period_time;
>   	atomic_t running;
>   	struct hrtimer timer;
> -	struct tasklet_struct tasklet;
>   	struct snd_pcm_substream *substream;
>   };
>   
> -static void dummy_hrtimer_pcm_elapsed(unsigned long priv)
> -{
> -	struct dummy_hrtimer_pcm *dpcm = (struct dummy_hrtimer_pcm *)priv;
> -	if (atomic_read(&dpcm->running))
> -		snd_pcm_period_elapsed(dpcm->substream);
> -}
> -
>   static enum hrtimer_restart dummy_hrtimer_callback(struct hrtimer *timer)
>   {
>   	struct dummy_hrtimer_pcm *dpcm;
> @@ -394,7 +386,14 @@ static enum hrtimer_restart dummy_hrtimer_callback(struct hrtimer *timer)
>   	dpcm = container_of(timer, struct dummy_hrtimer_pcm, timer);
>   	if (!atomic_read(&dpcm->running))
>   		return HRTIMER_NORESTART;
> -	tasklet_schedule(&dpcm->tasklet);
> +	/*
> +	 * In cases of XRUN and draining, this calls .trigger to stop PCM
> +	 * substream.
> +	 */
> +	snd_pcm_period_elapsed(dpcm->substream);
> +	if (!atomic_read(&dpcm->running))
> +		return HRTIMER_NORESTART;
> +
>   	hrtimer_forward_now(timer, dpcm->period_time);
>   	return HRTIMER_RESTART;
>   }
> @@ -414,14 +413,14 @@ static int dummy_hrtimer_stop(struct snd_pcm_substream *substream)
>   	struct dummy_hrtimer_pcm *dpcm = substream->runtime->private_data;
>   
>   	atomic_set(&dpcm->running, 0);
> -	hrtimer_cancel(&dpcm->timer);
> +	if (!hrtimer_callback_running(&dpcm->timer))
> +		hrtimer_cancel(&dpcm->timer);
>   	return 0;
>   }
>   
>   static inline void dummy_hrtimer_sync(struct dummy_hrtimer_pcm *dpcm)
>   {
>   	hrtimer_cancel(&dpcm->timer);
> -	tasklet_kill(&dpcm->tasklet);
>   }
>   
>   static snd_pcm_uframes_t
> @@ -466,12 +465,10 @@ static int dummy_hrtimer_create(struct snd_pcm_substream *substream)
>   	if (!dpcm)
>   		return -ENOMEM;
>   	substream->runtime->private_data = dpcm;
> -	hrtimer_init(&dpcm->timer, CLOCK_MONOTONIC, HRTIMER_MODE_REL);
> +	hrtimer_init(&dpcm->timer, CLOCK_MONOTONIC_SOFT, HRTIMER_MODE_REL);
>   	dpcm->timer.function = dummy_hrtimer_callback;
>   	dpcm->substream = substream;
>   	atomic_set(&dpcm->running, 0);
> -	tasklet_init(&dpcm->tasklet, dummy_hrtimer_pcm_elapsed,
> -		     (unsigned long)dpcm);
>   	return 0;
>   }


Thanks

Takashi Sakamoto

Re: [PATCH 23/25 v3] ALSA/dummy: Replace tasklet with softirq hrtimer

[Takashi Sakamoto]

On Sep 6 2017 01:18, Sebastian Andrzej Siewior wrote:
> From: Thomas Gleixner <tglx@linutronix.de>
> 
> The tasklet is used to defer the execution of snd_pcm_period_elapsed() to
> the softirq context. Using the CLOCK_MONOTONIC_SOFT base invokes the timer
> callback in softirq context as well which renders the tasklet useless.
> 
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> Signed-off-by: Anna-Maria Gleixner <anna-maria@linutronix.de>
> Cc: Jaroslav Kysela <perex@perex.cz>
> Cc: Takashi Iwai <tiwai@suse.com>
> Cc: Takashi Sakamoto <o-takashi@sakamocchi.jp>
> Cc: alsa-devel@alsa-project.org
> [o-takashi: avoid stall due to a call of hrtimer_cancel() on a callback
>              of hrtimer]
> Signed-off-by: Takashi Sakamoto <o-takashi@sakamocchi.jp>
> Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
> ---
> On 2017-09-06 01:05:43 [+0900], Takashi Sakamoto wrote:
>> As Iwai-san mentioned, in this function, .trigger can be called in two
>> cases; XRUN occurs and draining is done. Thus, let you change the comment as
>> 'In cases of XRUN and draining, this calls .trigger to stop PCM substream.'.
>> I'm sorry to trouble you.
>>
>> snd_pcm_period_elapsed()
>> ->snd_pcm_update_hw_ptr0()
>>    ->snd_pcm_update_state()
>>      ->snd_pcm_drain_done()
>>        ...
>>        ->.trigger(TRIGGER_STOP)
>>      ->xrun()
>>        ->snd_pcm_stop()
>>          ...
>>          ->.trigger(TRIGGER_STOP)
>>
> 
> I think you asked me just to update the comment. Did I do it right?
> 
> v2…v3: updated the comment as per Takashi Sakamoto's suggestion.
> v1…v2: merged Takashi Sakamoto fixup of the original patch into v2.
> 
>   sound/drivers/dummy.c | 25 +++++++++++--------------
>   1 file changed, 11 insertions(+), 14 deletions(-)

This Looks good to me. I did quick test and confirmed that it brings no 
stalls.

Tested-by: Takashi Sakamoto <o-takashi@sakamocchi.jp>

> diff --git a/sound/drivers/dummy.c b/sound/drivers/dummy.c
> index dd5ed037adf2..cdd851286f92 100644
> --- a/sound/drivers/dummy.c
> +++ b/sound/drivers/dummy.c
> @@ -376,17 +376,9 @@ struct dummy_hrtimer_pcm {
>   	ktime_t period_time;
>   	atomic_t running;
>   	struct hrtimer timer;
> -	struct tasklet_struct tasklet;
>   	struct snd_pcm_substream *substream;
>   };
>   
> -static void dummy_hrtimer_pcm_elapsed(unsigned long priv)
> -{
> -	struct dummy_hrtimer_pcm *dpcm = (struct dummy_hrtimer_pcm *)priv;
> -	if (atomic_read(&dpcm->running))
> -		snd_pcm_period_elapsed(dpcm->substream);
> -}
> -
>   static enum hrtimer_restart dummy_hrtimer_callback(struct hrtimer *timer)
>   {
>   	struct dummy_hrtimer_pcm *dpcm;
> @@ -394,7 +386,14 @@ static enum hrtimer_restart dummy_hrtimer_callback(struct hrtimer *timer)
>   	dpcm = container_of(timer, struct dummy_hrtimer_pcm, timer);
>   	if (!atomic_read(&dpcm->running))
>   		return HRTIMER_NORESTART;
> -	tasklet_schedule(&dpcm->tasklet);
> +	/*
> +	 * In cases of XRUN and draining, this calls .trigger to stop PCM
> +	 * substream.
> +	 */
> +	snd_pcm_period_elapsed(dpcm->substream);
> +	if (!atomic_read(&dpcm->running))
> +		return HRTIMER_NORESTART;
> +
>   	hrtimer_forward_now(timer, dpcm->period_time);
>   	return HRTIMER_RESTART;
>   }
> @@ -414,14 +413,14 @@ static int dummy_hrtimer_stop(struct snd_pcm_substream *substream)
>   	struct dummy_hrtimer_pcm *dpcm = substream->runtime->private_data;
>   
>   	atomic_set(&dpcm->running, 0);
> -	hrtimer_cancel(&dpcm->timer);
> +	if (!hrtimer_callback_running(&dpcm->timer))
> +		hrtimer_cancel(&dpcm->timer);
>   	return 0;
>   }
>   
>   static inline void dummy_hrtimer_sync(struct dummy_hrtimer_pcm *dpcm)
>   {
>   	hrtimer_cancel(&dpcm->timer);
> -	tasklet_kill(&dpcm->tasklet);
>   }
>   
>   static snd_pcm_uframes_t
> @@ -466,12 +465,10 @@ static int dummy_hrtimer_create(struct snd_pcm_substream *substream)
>   	if (!dpcm)
>   		return -ENOMEM;
>   	substream->runtime->private_data = dpcm;
> -	hrtimer_init(&dpcm->timer, CLOCK_MONOTONIC, HRTIMER_MODE_REL);
> +	hrtimer_init(&dpcm->timer, CLOCK_MONOTONIC_SOFT, HRTIMER_MODE_REL);
>   	dpcm->timer.function = dummy_hrtimer_callback;
>   	dpcm->substream = substream;
>   	atomic_set(&dpcm->running, 0);
> -	tasklet_init(&dpcm->tasklet, dummy_hrtimer_pcm_elapsed,
> -		     (unsigned long)dpcm);
>   	return 0;
>   }


Thanks

Takashi Sakamoto
_______________________________________________
Alsa-devel mailing list
Alsa-devel@alsa-project.org
http://mailman.alsa-project.org/mailman/listinfo/alsa-devel

Re: [alsa-devel] [PATCH 23/25 v3] ALSA/dummy: Replace tasklet with softirq hrtimer

[Takashi Iwai]

On Wed, 06 Sep 2017 06:30:03 +0200,
Takashi Sakamoto wrote:
> 
> On Sep 6 2017 01:18, Sebastian Andrzej Siewior wrote:
> > From: Thomas Gleixner <tglx@linutronix.de>
> >
> > The tasklet is used to defer the execution of snd_pcm_period_elapsed() to
> > the softirq context. Using the CLOCK_MONOTONIC_SOFT base invokes the timer
> > callback in softirq context as well which renders the tasklet useless.
> >
> > Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> > Signed-off-by: Anna-Maria Gleixner <anna-maria@linutronix.de>
> > Cc: Jaroslav Kysela <perex@perex.cz>
> > Cc: Takashi Iwai <tiwai@suse.com>
> > Cc: Takashi Sakamoto <o-takashi@sakamocchi.jp>
> > Cc: alsa-devel@alsa-project.org
> > [o-takashi: avoid stall due to a call of hrtimer_cancel() on a callback
> >              of hrtimer]
> > Signed-off-by: Takashi Sakamoto <o-takashi@sakamocchi.jp>
> > Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
> > ---
> > On 2017-09-06 01:05:43 [+0900], Takashi Sakamoto wrote:
> >> As Iwai-san mentioned, in this function, .trigger can be called in two
> >> cases; XRUN occurs and draining is done. Thus, let you change the comment as
> >> 'In cases of XRUN and draining, this calls .trigger to stop PCM substream.'.
> >> I'm sorry to trouble you.
> >>
> >> snd_pcm_period_elapsed()
> >> ->snd_pcm_update_hw_ptr0()
> >>    ->snd_pcm_update_state()
> >>      ->snd_pcm_drain_done()
> >>        ...
> >>        ->.trigger(TRIGGER_STOP)
> >>      ->xrun()
> >>        ->snd_pcm_stop()
> >>          ...
> >>          ->.trigger(TRIGGER_STOP)
> >>
> >
> > I think you asked me just to update the comment. Did I do it right?
> >
> > v2…v3: updated the comment as per Takashi Sakamoto's suggestion.
> > v1…v2: merged Takashi Sakamoto fixup of the original patch into v2.
> >
> >   sound/drivers/dummy.c | 25 +++++++++++--------------
> >   1 file changed, 11 insertions(+), 14 deletions(-)
> 
> This Looks good to me. I did quick test and confirmed that it brings
> no stalls.
> 
> Tested-by: Takashi Sakamoto <o-takashi@sakamocchi.jp>

I suppose you'd apply all together in some tree like tip or whatever?
In anyway, feel free to take my ack for this patch:
  Reviewed-by: Takashi Iwai <tiwai@suse.de>


Thanks!

Takashi

Re: [PATCH 23/25 v3] ALSA/dummy: Replace tasklet with softirq hrtimer

[Takashi Iwai]

On Wed, 06 Sep 2017 06:30:03 +0200,
Takashi Sakamoto wrote:
> 
> On Sep 6 2017 01:18, Sebastian Andrzej Siewior wrote:
> > From: Thomas Gleixner <tglx@linutronix.de>
> >
> > The tasklet is used to defer the execution of snd_pcm_period_elapsed() to
> > the softirq context. Using the CLOCK_MONOTONIC_SOFT base invokes the timer
> > callback in softirq context as well which renders the tasklet useless.
> >
> > Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> > Signed-off-by: Anna-Maria Gleixner <anna-maria@linutronix.de>
> > Cc: Jaroslav Kysela <perex@perex.cz>
> > Cc: Takashi Iwai <tiwai@suse.com>
> > Cc: Takashi Sakamoto <o-takashi@sakamocchi.jp>
> > Cc: alsa-devel@alsa-project.org
> > [o-takashi: avoid stall due to a call of hrtimer_cancel() on a callback
> >              of hrtimer]
> > Signed-off-by: Takashi Sakamoto <o-takashi@sakamocchi.jp>
> > Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
> > ---
> > On 2017-09-06 01:05:43 [+0900], Takashi Sakamoto wrote:
> >> As Iwai-san mentioned, in this function, .trigger can be called in two
> >> cases; XRUN occurs and draining is done. Thus, let you change the comment as
> >> 'In cases of XRUN and draining, this calls .trigger to stop PCM substream.'.
> >> I'm sorry to trouble you.
> >>
> >> snd_pcm_period_elapsed()
> >> ->snd_pcm_update_hw_ptr0()
> >>    ->snd_pcm_update_state()
> >>      ->snd_pcm_drain_done()
> >>        ...
> >>        ->.trigger(TRIGGER_STOP)
> >>      ->xrun()
> >>        ->snd_pcm_stop()
> >>          ...
> >>          ->.trigger(TRIGGER_STOP)
> >>
> >
> > I think you asked me just to update the comment. Did I do it right?
> >
> > v2…v3: updated the comment as per Takashi Sakamoto's suggestion.
> > v1…v2: merged Takashi Sakamoto fixup of the original patch into v2.
> >
> >   sound/drivers/dummy.c | 25 +++++++++++--------------
> >   1 file changed, 11 insertions(+), 14 deletions(-)
> 
> This Looks good to me. I did quick test and confirmed that it brings
> no stalls.
> 
> Tested-by: Takashi Sakamoto <o-takashi@sakamocchi.jp>

I suppose you'd apply all together in some tree like tip or whatever?
In anyway, feel free to take my ack for this patch:
  Reviewed-by: Takashi Iwai <tiwai@suse.de>


Thanks!

Takashi
_______________________________________________
Alsa-devel mailing list
Alsa-devel@alsa-project.org
http://mailman.alsa-project.org/mailman/listinfo/alsa-devel

[PATCH 22/25] softirq: Remove tasklet_hrtimer

[Anna-Maria Gleixner]

[-- Attachment #1: softirq_Remove_tasklet_hrtimer.patch --]
[-- Type: text/plain, Size: 3179 bytes --]

From: Thomas Gleixner <tglx@linutronix.de>

There are no more tasklet_hrtimer users of this interface.
Remove it.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Anna-Maria Gleixner <anna-maria@linutronix.de>
---
 include/linux/interrupt.h |   25 ----------------------
 kernel/softirq.c          |   51 ----------------------------------------------
 2 files changed, 76 deletions(-)

--- a/include/linux/interrupt.h
+++ b/include/linux/interrupt.h
@@ -631,31 +631,6 @@ extern void tasklet_kill_immediate(struc
 extern void tasklet_init(struct tasklet_struct *t,
 			 void (*func)(unsigned long), unsigned long data);
 
-struct tasklet_hrtimer {
-	struct hrtimer		timer;
-	struct tasklet_struct	tasklet;
-	enum hrtimer_restart	(*function)(struct hrtimer *);
-};
-
-extern void
-tasklet_hrtimer_init(struct tasklet_hrtimer *ttimer,
-		     enum hrtimer_restart (*function)(struct hrtimer *),
-		     clockid_t which_clock, enum hrtimer_mode mode);
-
-static inline
-void tasklet_hrtimer_start(struct tasklet_hrtimer *ttimer, ktime_t time,
-			   const enum hrtimer_mode mode)
-{
-	hrtimer_start(&ttimer->timer, time, mode);
-}
-
-static inline
-void tasklet_hrtimer_cancel(struct tasklet_hrtimer *ttimer)
-{
-	hrtimer_cancel(&ttimer->timer);
-	tasklet_kill(&ttimer->tasklet);
-}
-
 /*
  * Autoprobing for irqs:
  *
--- a/kernel/softirq.c
+++ b/kernel/softirq.c
@@ -594,57 +594,6 @@ void tasklet_kill(struct tasklet_struct
 }
 EXPORT_SYMBOL(tasklet_kill);
 
-/*
- * tasklet_hrtimer
- */
-
-/*
- * The trampoline is called when the hrtimer expires. It schedules a tasklet
- * to run __tasklet_hrtimer_trampoline() which in turn will call the intended
- * hrtimer callback, but from softirq context.
- */
-static enum hrtimer_restart __hrtimer_tasklet_trampoline(struct hrtimer *timer)
-{
-	struct tasklet_hrtimer *ttimer =
-		container_of(timer, struct tasklet_hrtimer, timer);
-
-	tasklet_hi_schedule(&ttimer->tasklet);
-	return HRTIMER_NORESTART;
-}
-
-/*
- * Helper function which calls the hrtimer callback from
- * tasklet/softirq context
- */
-static void __tasklet_hrtimer_trampoline(unsigned long data)
-{
-	struct tasklet_hrtimer *ttimer = (void *)data;
-	enum hrtimer_restart restart;
-
-	restart = ttimer->function(&ttimer->timer);
-	if (restart != HRTIMER_NORESTART)
-		hrtimer_restart(&ttimer->timer);
-}
-
-/**
- * tasklet_hrtimer_init - Init a tasklet/hrtimer combo for softirq callbacks
- * @ttimer:	 tasklet_hrtimer which is initialized
- * @function:	 hrtimer callback function which gets called from softirq context
- * @which_clock: clock id (CLOCK_MONOTONIC/CLOCK_REALTIME)
- * @mode:	 hrtimer mode (HRTIMER_MODE_ABS/HRTIMER_MODE_REL)
- */
-void tasklet_hrtimer_init(struct tasklet_hrtimer *ttimer,
-			  enum hrtimer_restart (*function)(struct hrtimer *),
-			  clockid_t which_clock, enum hrtimer_mode mode)
-{
-	hrtimer_init(&ttimer->timer, which_clock, mode);
-	ttimer->timer.function = __hrtimer_tasklet_trampoline;
-	tasklet_init(&ttimer->tasklet, __tasklet_hrtimer_trampoline,
-		     (unsigned long)ttimer);
-	ttimer->function = function;
-}
-EXPORT_SYMBOL_GPL(tasklet_hrtimer_init);
-
 void __init softirq_init(void)
 {
 	int cpu;

[PATCH 25/25] usb/gadget/NCM: Replace tasklet with softirq hrtimer

[Anna-Maria Gleixner]

[-- Attachment #1: usbgadgetNCM_Replace_tasklet_with_softirq_hrtimer.patch --]
[-- Type: text/plain, Size: 2635 bytes --]

From: Thomas Gleixner <tglx@linutronix.de>

The tx_tasklet tasklet is used in invoke the hrtimer (task_timer) in
softirq context. This can be also achieved without the tasklet but with
CLOCK_MONOTONIC_SOFT as hrtimer base.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Anna-Maria Gleixner <anna-maria@linutronix.de>
Cc: Felipe Balbi <balbi@kernel.org>
Cc: linux-usb@vger.kernel.org
---
 drivers/usb/gadget/function/f_ncm.c |   28 ++++++----------------------
 1 file changed, 6 insertions(+), 22 deletions(-)

--- a/drivers/usb/gadget/function/f_ncm.c
+++ b/drivers/usb/gadget/function/f_ncm.c
@@ -77,9 +77,7 @@ struct f_ncm {
 	struct sk_buff			*skb_tx_ndp;
 	u16				ndp_dgram_count;
 	bool				timer_force_tx;
-	struct tasklet_struct		tx_tasklet;
 	struct hrtimer			task_timer;
-
 	bool				timer_stopping;
 };
 
@@ -1154,17 +1152,15 @@ static struct sk_buff *ncm_wrap_ntb(stru
 }
 
 /*
- * This transmits the NTB if there are frames waiting.
+ * The transmit should only be run if no skb data has been sent
+ * for a certain duration.
  */
-static void ncm_tx_tasklet(unsigned long data)
+static enum hrtimer_restart ncm_tx_timeout(struct hrtimer *data)
 {
-	struct f_ncm	*ncm = (void *)data;
-
-	if (ncm->timer_stopping)
-		return;
+	struct f_ncm *ncm = container_of(data, struct f_ncm, task_timer);
 
 	/* Only send if data is available. */
-	if (ncm->skb_tx_data) {
+	if (!ncm->timer_stopping && ncm->skb_tx_data) {
 		ncm->timer_force_tx = true;
 
 		/* XXX This allowance of a NULL skb argument to ndo_start_xmit
@@ -1177,16 +1173,6 @@ static void ncm_tx_tasklet(unsigned long
 
 		ncm->timer_force_tx = false;
 	}
-}
-
-/*
- * The transmit should only be run if no skb data has been sent
- * for a certain duration.
- */
-static enum hrtimer_restart ncm_tx_timeout(struct hrtimer *data)
-{
-	struct f_ncm *ncm = container_of(data, struct f_ncm, task_timer);
-	tasklet_schedule(&ncm->tx_tasklet);
 	return HRTIMER_NORESTART;
 }
 
@@ -1519,8 +1505,7 @@ static int ncm_bind(struct usb_configura
 	ncm->port.open = ncm_open;
 	ncm->port.close = ncm_close;
 
-	tasklet_init(&ncm->tx_tasklet, ncm_tx_tasklet, (unsigned long) ncm);
-	hrtimer_init(&ncm->task_timer, CLOCK_MONOTONIC, HRTIMER_MODE_REL);
+	hrtimer_init(&ncm->task_timer, CLOCK_MONOTONIC_SOFT, HRTIMER_MODE_REL);
 	ncm->task_timer.function = ncm_tx_timeout;
 
 	DBG(cdev, "CDC Network: %s speed IN/%s OUT/%s NOTIFY/%s\n",
@@ -1629,7 +1614,6 @@ static void ncm_unbind(struct usb_config
 	DBG(c->cdev, "ncm unbind\n");
 
 	hrtimer_cancel(&ncm->task_timer);
-	tasklet_kill(&ncm->tx_tasklet);
 
 	ncm_string_defs[0].id = 0;
 	usb_free_all_descriptors(f);

Re: [PATCH 25/25] usb/gadget/NCM: Replace tasklet with softirq hrtimer

[Felipe Balbi]

[-- Attachment #1: Type: text/plain, Size: 561 bytes --]

Anna-Maria Gleixner <anna-maria@linutronix.de> writes:

> From: Thomas Gleixner <tglx@linutronix.de>
>
> The tx_tasklet tasklet is used in invoke the hrtimer (task_timer) in
> softirq context. This can be also achieved without the tasklet but with
> CLOCK_MONOTONIC_SOFT as hrtimer base.
>
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> Signed-off-by: Anna-Maria Gleixner <anna-maria@linutronix.de>
> Cc: Felipe Balbi <balbi@kernel.org>
> Cc: linux-usb@vger.kernel.org

Acked-by: Felipe Balbi <felipe.balbi@linus.intel.com>

-- 
balbi

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

[PATCH 24/25] net/cdc_ncm: Replace tasklet with softirq hrtimer

[Anna-Maria Gleixner]

[-- Attachment #1: netcdc_ncm_Replace_tasklet_with_softirq_hrtimer.patch --]
[-- Type: text/plain, Size: 3486 bytes --]

From: Thomas Gleixner <tglx@linutronix.de>

The bh tasklet is used in invoke the hrtimer (cdc_ncm_tx_timer_cb) in
softirq context. This can be also achieved without the tasklet but with
CLOCK_MONOTONIC_SOFT as hrtimer base.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Anna-Maria Gleixner <anna-maria@linutronix.de>
Cc: Oliver Neukum <oliver@neukum.org>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: linux-usb@vger.kernel.org
Cc: netdev@vger.kernel.org
---
 drivers/net/usb/cdc_ncm.c   |   37 ++++++++++++++++---------------------
 include/linux/usb/cdc_ncm.h |    2 +-
 2 files changed, 17 insertions(+), 22 deletions(-)

--- a/drivers/net/usb/cdc_ncm.c
+++ b/drivers/net/usb/cdc_ncm.c
@@ -61,7 +61,6 @@ static bool prefer_mbim;
 module_param(prefer_mbim, bool, S_IRUGO | S_IWUSR);
 MODULE_PARM_DESC(prefer_mbim, "Prefer MBIM setting on dual NCM/MBIM functions");
 
-static void cdc_ncm_txpath_bh(unsigned long param);
 static void cdc_ncm_tx_timeout_start(struct cdc_ncm_ctx *ctx);
 static enum hrtimer_restart cdc_ncm_tx_timer_cb(struct hrtimer *hr_timer);
 static struct usb_driver cdc_ncm_driver;
@@ -777,10 +776,9 @@ int cdc_ncm_bind_common(struct usbnet *d
 	if (!ctx)
 		return -ENOMEM;
 
-	hrtimer_init(&ctx->tx_timer, CLOCK_MONOTONIC, HRTIMER_MODE_REL);
+	hrtimer_init(&ctx->tx_timer, CLOCK_MONOTONIC_SOFT, HRTIMER_MODE_REL);
 	ctx->tx_timer.function = &cdc_ncm_tx_timer_cb;
-	ctx->bh.data = (unsigned long)dev;
-	ctx->bh.func = cdc_ncm_txpath_bh;
+	ctx->usbnet = dev;
 	atomic_set(&ctx->stop, 0);
 	spin_lock_init(&ctx->mtx);
 
@@ -967,10 +965,7 @@ void cdc_ncm_unbind(struct usbnet *dev,
 
 	atomic_set(&ctx->stop, 1);
 
-	if (hrtimer_active(&ctx->tx_timer))
-		hrtimer_cancel(&ctx->tx_timer);
-
-	tasklet_kill(&ctx->bh);
+	hrtimer_cancel(&ctx->tx_timer);
 
 	/* handle devices with combined control and data interface */
 	if (ctx->control == ctx->data)
@@ -1348,20 +1343,9 @@ static void cdc_ncm_tx_timeout_start(str
 				HRTIMER_MODE_REL);
 }
 
-static enum hrtimer_restart cdc_ncm_tx_timer_cb(struct hrtimer *timer)
+static void cdc_ncm_txpath_bh(struct cdc_ncm_ctx *ctx)
 {
-	struct cdc_ncm_ctx *ctx =
-			container_of(timer, struct cdc_ncm_ctx, tx_timer);
-
-	if (!atomic_read(&ctx->stop))
-		tasklet_schedule(&ctx->bh);
-	return HRTIMER_NORESTART;
-}
-
-static void cdc_ncm_txpath_bh(unsigned long param)
-{
-	struct usbnet *dev = (struct usbnet *)param;
-	struct cdc_ncm_ctx *ctx = (struct cdc_ncm_ctx *)dev->data[0];
+	struct usbnet *dev = ctx->usbnet;
 
 	spin_lock_bh(&ctx->mtx);
 	if (ctx->tx_timer_pending != 0) {
@@ -1379,6 +1363,17 @@ static void cdc_ncm_txpath_bh(unsigned l
 	}
 }
 
+static enum hrtimer_restart cdc_ncm_tx_timer_cb(struct hrtimer *timer)
+{
+	struct cdc_ncm_ctx *ctx =
+			container_of(timer, struct cdc_ncm_ctx, tx_timer);
+
+	if (!atomic_read(&ctx->stop))
+		cdc_ncm_txpath_bh(ctx);
+
+	return HRTIMER_NORESTART;
+}
+
 struct sk_buff *
 cdc_ncm_tx_fixup(struct usbnet *dev, struct sk_buff *skb, gfp_t flags)
 {
--- a/include/linux/usb/cdc_ncm.h
+++ b/include/linux/usb/cdc_ncm.h
@@ -92,7 +92,6 @@
 struct cdc_ncm_ctx {
 	struct usb_cdc_ncm_ntb_parameters ncm_parm;
 	struct hrtimer tx_timer;
-	struct tasklet_struct bh;
 
 	const struct usb_cdc_ncm_desc *func_desc;
 	const struct usb_cdc_mbim_desc *mbim_desc;
@@ -101,6 +100,7 @@ struct cdc_ncm_ctx {
 
 	struct usb_interface *control;
 	struct usb_interface *data;
+	struct usbnet *usbnet;
 
 	struct sk_buff *tx_curr_skb;
 	struct sk_buff *tx_rem_skb;

Re: [PATCH 24/25] net/cdc_ncm: Replace tasklet with softirq hrtimer

[Greg Kroah-Hartman]

On Thu, Aug 31, 2017 at 12:23:46PM -0000, Anna-Maria Gleixner wrote:
> From: Thomas Gleixner <tglx@linutronix.de>
> 
> The bh tasklet is used in invoke the hrtimer (cdc_ncm_tx_timer_cb) in
> softirq context. This can be also achieved without the tasklet but with
> CLOCK_MONOTONIC_SOFT as hrtimer base.
> 
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> Signed-off-by: Anna-Maria Gleixner <anna-maria@linutronix.de>
> Cc: Oliver Neukum <oliver@neukum.org>
> Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
> Cc: linux-usb@vger.kernel.org
> Cc: netdev@vger.kernel.org


Acked-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

Re: [PATCH 24/25] net/cdc_ncm: Replace tasklet with softirq hrtimer

[Bjørn Mork]

Anna-Maria Gleixner <anna-maria@linutronix.de> writes:

> From: Thomas Gleixner <tglx@linutronix.de>
>
> The bh tasklet is used in invoke the hrtimer (cdc_ncm_tx_timer_cb) in
> softirq context. This can be also achieved without the tasklet but with
> CLOCK_MONOTONIC_SOFT as hrtimer base.
>
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> Signed-off-by: Anna-Maria Gleixner <anna-maria@linutronix.de>
> Cc: Oliver Neukum <oliver@neukum.org>
> Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
> Cc: linux-usb@vger.kernel.org
> Cc: netdev@vger.kernel.org
> ---
>  drivers/net/usb/cdc_ncm.c   |   37 ++++++++++++++++---------------------
>  include/linux/usb/cdc_ncm.h |    2 +-
>  2 files changed, 17 insertions(+), 22 deletions(-)
>
> --- a/drivers/net/usb/cdc_ncm.c
> +++ b/drivers/net/usb/cdc_ncm.c
> @@ -61,7 +61,6 @@ static bool prefer_mbim;
>  module_param(prefer_mbim, bool, S_IRUGO | S_IWUSR);
>  MODULE_PARM_DESC(prefer_mbim, "Prefer MBIM setting on dual NCM/MBIM functions");
>  
> -static void cdc_ncm_txpath_bh(unsigned long param);
>  static void cdc_ncm_tx_timeout_start(struct cdc_ncm_ctx *ctx);
>  static enum hrtimer_restart cdc_ncm_tx_timer_cb(struct hrtimer *hr_timer);
>  static struct usb_driver cdc_ncm_driver;
> @@ -777,10 +776,9 @@ int cdc_ncm_bind_common(struct usbnet *d
>  	if (!ctx)
>  		return -ENOMEM;
>  
> -	hrtimer_init(&ctx->tx_timer, CLOCK_MONOTONIC, HRTIMER_MODE_REL);
> +	hrtimer_init(&ctx->tx_timer, CLOCK_MONOTONIC_SOFT, HRTIMER_MODE_REL);
>  	ctx->tx_timer.function = &cdc_ncm_tx_timer_cb;
> -	ctx->bh.data = (unsigned long)dev;
> -	ctx->bh.func = cdc_ncm_txpath_bh;
> +	ctx->usbnet = dev;
>  	atomic_set(&ctx->stop, 0);
>  	spin_lock_init(&ctx->mtx);
>  
> @@ -967,10 +965,7 @@ void cdc_ncm_unbind(struct usbnet *dev,
>  
>  	atomic_set(&ctx->stop, 1);
>  
> -	if (hrtimer_active(&ctx->tx_timer))
> -		hrtimer_cancel(&ctx->tx_timer);
> -
> -	tasklet_kill(&ctx->bh);
> +	hrtimer_cancel(&ctx->tx_timer);
>  
>  	/* handle devices with combined control and data interface */
>  	if (ctx->control == ctx->data)
> @@ -1348,20 +1343,9 @@ static void cdc_ncm_tx_timeout_start(str
>  				HRTIMER_MODE_REL);
>  }
>  
> -static enum hrtimer_restart cdc_ncm_tx_timer_cb(struct hrtimer *timer)
> +static void cdc_ncm_txpath_bh(struct cdc_ncm_ctx *ctx)
>  {
> -	struct cdc_ncm_ctx *ctx =
> -			container_of(timer, struct cdc_ncm_ctx, tx_timer);
> -
> -	if (!atomic_read(&ctx->stop))
> -		tasklet_schedule(&ctx->bh);
> -	return HRTIMER_NORESTART;
> -}
> -
> -static void cdc_ncm_txpath_bh(unsigned long param)
> -{
> -	struct usbnet *dev = (struct usbnet *)param;
> -	struct cdc_ncm_ctx *ctx = (struct cdc_ncm_ctx *)dev->data[0];
> +	struct usbnet *dev = ctx->usbnet;
>  
>  	spin_lock_bh(&ctx->mtx);
>  	if (ctx->tx_timer_pending != 0) {
> @@ -1379,6 +1363,17 @@ static void cdc_ncm_txpath_bh(unsigned l
>  	}
>  }
>  
> +static enum hrtimer_restart cdc_ncm_tx_timer_cb(struct hrtimer *timer)
> +{
> +	struct cdc_ncm_ctx *ctx =
> +			container_of(timer, struct cdc_ncm_ctx, tx_timer);
> +
> +	if (!atomic_read(&ctx->stop))
> +		cdc_ncm_txpath_bh(ctx);
> +
> +	return HRTIMER_NORESTART;
> +}
> +
>  struct sk_buff *
>  cdc_ncm_tx_fixup(struct usbnet *dev, struct sk_buff *skb, gfp_t flags)
>  {
> --- a/include/linux/usb/cdc_ncm.h
> +++ b/include/linux/usb/cdc_ncm.h
> @@ -92,7 +92,6 @@
>  struct cdc_ncm_ctx {
>  	struct usb_cdc_ncm_ntb_parameters ncm_parm;
>  	struct hrtimer tx_timer;
> -	struct tasklet_struct bh;
>  
>  	const struct usb_cdc_ncm_desc *func_desc;
>  	const struct usb_cdc_mbim_desc *mbim_desc;
> @@ -101,6 +100,7 @@ struct cdc_ncm_ctx {
>  
>  	struct usb_interface *control;
>  	struct usb_interface *data;
> +	struct usbnet *usbnet;
>  
>  	struct sk_buff *tx_curr_skb;
>  	struct sk_buff *tx_rem_skb;




I believe the struct usbnet pointer is redundant.  We already have lots
of pointers back and forth here.  This should work, but is not tested:

	struct usbnet *dev = usb_get_intfdata(ctx->control):




Bjørn

[PATCH 24/25 v2] net/cdc_ncm: Replace tasklet with softirq hrtimer

[Sebastian Andrzej Siewior]

From: Thomas Gleixner <tglx@linutronix.de>

The bh tasklet is used in invoke the hrtimer (cdc_ncm_tx_timer_cb) in
softirq context. This can be also achieved without the tasklet but with
CLOCK_MONOTONIC_SOFT as hrtimer base.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Anna-Maria Gleixner <anna-maria@linutronix.de>
Cc: Oliver Neukum <oliver@neukum.org>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: linux-usb@vger.kernel.org
Cc: netdev@vger.kernel.org
Acked-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
[bigeasy: using usb_get_intfdata() as suggested by Bjørn Mork]
Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
---
On 2017-08-31 15:57:04 [+0200], Bjørn Mork wrote:
> I believe the struct usbnet pointer is redundant.  We already have lots
> of pointers back and forth here.  This should work, but is not tested:
> 
> 	struct usbnet *dev = usb_get_intfdata(ctx->control):

I think so, too. Still untested as I don't have a working gadget around.

v1…v2: Updated as suggested by Bjørn and added Greg's Acked-by.

 drivers/net/usb/cdc_ncm.c   | 36 +++++++++++++++---------------------
 include/linux/usb/cdc_ncm.h |  1 -
 2 files changed, 15 insertions(+), 22 deletions(-)

diff --git a/drivers/net/usb/cdc_ncm.c b/drivers/net/usb/cdc_ncm.c
index 8f572b9f3625..42f7bd90e6a4 100644
--- a/drivers/net/usb/cdc_ncm.c
+++ b/drivers/net/usb/cdc_ncm.c
@@ -61,7 +61,6 @@ static bool prefer_mbim;
 module_param(prefer_mbim, bool, S_IRUGO | S_IWUSR);
 MODULE_PARM_DESC(prefer_mbim, "Prefer MBIM setting on dual NCM/MBIM functions");
 
-static void cdc_ncm_txpath_bh(unsigned long param);
 static void cdc_ncm_tx_timeout_start(struct cdc_ncm_ctx *ctx);
 static enum hrtimer_restart cdc_ncm_tx_timer_cb(struct hrtimer *hr_timer);
 static struct usb_driver cdc_ncm_driver;
@@ -777,10 +776,8 @@ int cdc_ncm_bind_common(struct usbnet *dev, struct usb_interface *intf, u8 data_
 	if (!ctx)
 		return -ENOMEM;
 
-	hrtimer_init(&ctx->tx_timer, CLOCK_MONOTONIC, HRTIMER_MODE_REL);
+	hrtimer_init(&ctx->tx_timer, CLOCK_MONOTONIC_SOFT, HRTIMER_MODE_REL);
 	ctx->tx_timer.function = &cdc_ncm_tx_timer_cb;
-	ctx->bh.data = (unsigned long)dev;
-	ctx->bh.func = cdc_ncm_txpath_bh;
 	atomic_set(&ctx->stop, 0);
 	spin_lock_init(&ctx->mtx);
 
@@ -967,10 +964,7 @@ void cdc_ncm_unbind(struct usbnet *dev, struct usb_interface *intf)
 
 	atomic_set(&ctx->stop, 1);
 
-	if (hrtimer_active(&ctx->tx_timer))
-		hrtimer_cancel(&ctx->tx_timer);
-
-	tasklet_kill(&ctx->bh);
+	hrtimer_cancel(&ctx->tx_timer);
 
 	/* handle devices with combined control and data interface */
 	if (ctx->control == ctx->data)
@@ -1348,20 +1342,9 @@ static void cdc_ncm_tx_timeout_start(struct cdc_ncm_ctx *ctx)
 				HRTIMER_MODE_REL);
 }
 
-static enum hrtimer_restart cdc_ncm_tx_timer_cb(struct hrtimer *timer)
+static void cdc_ncm_txpath_bh(struct cdc_ncm_ctx *ctx)
 {
-	struct cdc_ncm_ctx *ctx =
-			container_of(timer, struct cdc_ncm_ctx, tx_timer);
-
-	if (!atomic_read(&ctx->stop))
-		tasklet_schedule(&ctx->bh);
-	return HRTIMER_NORESTART;
-}
-
-static void cdc_ncm_txpath_bh(unsigned long param)
-{
-	struct usbnet *dev = (struct usbnet *)param;
-	struct cdc_ncm_ctx *ctx = (struct cdc_ncm_ctx *)dev->data[0];
+	struct usbnet *dev = usb_get_intfdata(ctx->control);
 
 	spin_lock_bh(&ctx->mtx);
 	if (ctx->tx_timer_pending != 0) {
@@ -1379,6 +1362,17 @@ static void cdc_ncm_txpath_bh(unsigned long param)
 	}
 }
 
+static enum hrtimer_restart cdc_ncm_tx_timer_cb(struct hrtimer *timer)
+{
+	struct cdc_ncm_ctx *ctx =
+			container_of(timer, struct cdc_ncm_ctx, tx_timer);
+
+	if (!atomic_read(&ctx->stop))
+		cdc_ncm_txpath_bh(ctx);
+
+	return HRTIMER_NORESTART;
+}
+
 struct sk_buff *
 cdc_ncm_tx_fixup(struct usbnet *dev, struct sk_buff *skb, gfp_t flags)
 {
diff --git a/include/linux/usb/cdc_ncm.h b/include/linux/usb/cdc_ncm.h
index 1a59699cf82a..62b506fddf8d 100644
--- a/include/linux/usb/cdc_ncm.h
+++ b/include/linux/usb/cdc_ncm.h
@@ -92,7 +92,6 @@
 struct cdc_ncm_ctx {
 	struct usb_cdc_ncm_ntb_parameters ncm_parm;
 	struct hrtimer tx_timer;
-	struct tasklet_struct bh;
 
 	const struct usb_cdc_ncm_desc *func_desc;
 	const struct usb_cdc_mbim_desc *mbim_desc;
-- 
2.14.1

Re: [PATCH 00/25] hrtimer: Provide softirq context hrtimers

[Anna-Maria Gleixner]

I'm sorry for the typo in the email address!

    Anna-Maria

On Thu, 31 Aug 2017, Anna-Maria Gleixner wrote:

> There are quite some places in the kernel which use a combination of
> hrtimers and tasklets to make use of the precise expiry of hrtimers, which
> schedule a tasklet to bring the actual function into softirq context.
> 
> This was introduced when the previous hrtimer softirq code was
> removed. That code was implemented by expiring the timer in hard irq
> context and then deferring the execution of the callback into softirq
> context. That caused a lot of pointless shuffling between the rbtree and a
> linked list.
> 
> In recent discussions it turned out that more potential users of hrtimers
> in softirq context might come up. Aside of that the RT patches need this
> functionality as well to defer hrtimers into softirq context if their
> callbacks are not interrupt safe on RT.
> 
> This series implements a new approach by adding SOFT_* clock ids and
> instead of doing the list shuffle, timers started with these clock ids are
> put into separate soft expiry hrtimer queues. These queues are evaluated
> only when the hardirq context detects that the first expiring timer in the
> softirq queues has expired. That makes the overhead in the hardirq context
> minimal.
> 
> The series reworks the code to reuse as much as possible from the existing
> facilities for the new softirq hrtimers and integrates them with all
> flavours of hrtimers (HIGH_RES=y/n - NOHZ=y/n).
> 
> To achieve this quite some of the conditionals in the existing code are
> removed for the price of adding some pointless data and state tracking to
> the HIGH_RES=n case. That's minimal, but well worth it as it increases the
> readability and maintainability of the code.
> 
> The first part of the series implements the new functionality and the
> second part converts the hrtimer/tasklet users to make use of it and
> removes struct hrtimer_tasklet and the surrounding helper functions.
> 
> This series is available from git as well:
> 
>     git://git.kernel.org/pub/scm/linux/kernel/git/bigeasy/linux-hrtimer.git soft_hrtimer
>     https://git.kernel.org/bigeasy/linux-hrtimer/h/soft_hrtimer
> 
> 
> Thanks,
> 
> 	Anna-Maria
> 
> ---
>  drivers/net/usb/cdc_ncm.c             |   37 +-
>  drivers/net/wireless/mac80211_hwsim.c |   44 +-
>  drivers/usb/gadget/function/f_ncm.c   |   28 -
>  include/linux/hrtimer.h               |   76 ++---
>  include/linux/interrupt.h             |   25 -
>  include/linux/usb/cdc_ncm.h           |    2 
>  include/net/xfrm.h                    |    2 
>  kernel/softirq.c                      |   51 ---
>  kernel/time/hrtimer.c                 |  513 +++++++++++++++++++++-------------
>  net/can/bcm.c                         |  150 +++------
>  net/xfrm/xfrm_state.c                 |   29 +
>  sound/drivers/dummy.c                 |   16 -
>  12 files changed, 484 insertions(+), 489 deletions(-)
> 
>