From: Marcelo Tosatti <mtosatti@redhat.com>
To: Anna-Maria Gleixner <anna-maria@linutronix.de>
Cc: linux-kernel@vger.kernel.org, linux-rt-users@vger.kernel.org,
Thomas Gleixner <tglx@linutronix.de>,
Daniel Bristot de Oliveira <bristot@redhat.com>,
Luiz Capitulino <lcapitulino@redhat.com>,
Haris Okanovic <haris.okanovic@ni.com>
Subject: Re: [patch 2/3] timers: do not raise softirq unconditionally (spinlockless version)
Date: Thu, 30 May 2019 17:14:58 -0300 [thread overview]
Message-ID: <20190530201455.GC23199@amt.cnet> (raw)
In-Reply-To: <alpine.DEB.2.21.1905291653120.1395@somnus>
On Wed, May 29, 2019 at 04:53:26PM +0200, Anna-Maria Gleixner wrote:
> On Mon, 15 Apr 2019, Marcelo Tosatti wrote:
>
> > Check base->pending_map locklessly and skip raising timer softirq
> > if empty.
> >
> > What allows the lockless (and potentially racy against mod_timer)
> > check is that mod_timer will raise another timer softirq after
> > modifying base->pending_map.
>
> The raise of the timer softirq after adding the timer is done
> unconditionally - so there are timer softirqs raised which are not required
> at all, as mentioned before.
Yes. However i can't see a way to avoid that: its not possible to know
if the race described earlier happened or not.
Do you have a suggestion on how to avoid this or a way to avoid
the IPI+raise softirq ?
> This check is for !CONFIG_PREEMPT_RT_FULL only implemented. The commit
> message totally igonres that you are implementing something
> CONFIG_PREEMPT_RT_FULL dependent as well.
>
> > Signed-off-by: Marcelo Tosatti <mtosatti@redhat.com>
> >
> > ---
> > kernel/time/timer.c | 18 ++++++++++++++++++
> > 1 file changed, 18 insertions(+)
> >
> > Index: linux-rt-devel/kernel/time/timer.c
> > ===================================================================
> > --- linux-rt-devel.orig/kernel/time/timer.c 2019-04-15 14:21:02.788704354 -0300
> > +++ linux-rt-devel/kernel/time/timer.c 2019-04-15 14:22:56.755047354 -0300
> > @@ -1776,6 +1776,24 @@
> > if (time_before(jiffies, base->clk))
> > return;
> > }
> > +
> > +#ifdef CONFIG_PREEMPT_RT_FULL
> > +/* On RT, irq work runs from softirq */
> > + if (irq_work_needs_cpu())
> > + goto raise;
>
> So with this patch and the change you made in the patch before, timers on
> RT are expired only when there is pending irq work or after modifying a
> timer on a non housekeeping cpu?
Well, run_timer_softirq execute only if pending_map contains a bit set.
> With your patches I could create the following problematic situation on RT
> (if I understood everything properly): I add a timer which should expire in
> 50 jiffies to the wheel of a non housekeeping cpu. So it ends up 50 buckets
> away form now in the first wheel. This timer is the only timer in the wheel
> and the next timer softirq raise is required in 50 jiffies. After adding
> the timer, the timer interrupt is raised, and no timer has to be expired,
> because there is no timer pending.
But the softirq will be raised, because pending_map will be set:
+ if (!bitmap_empty(base->pending_map, WHEEL_SIZE))
+ goto raise;
No?
> If there is no irq work required during
> the next 51 jiffies and also no timer changed, the timer I added, will not
> expire in time. The timer_base will come out of idle but will not forward
> the base clk.
> This makes it even worse: When then adding a timer, the timer
> base is forwarded - but without checking for the next pending timer, so the
> first added timer will be delayed even more.
>
> So your implementation lacks forwarding the timer_base->clk when timer_base
> comes out of idle with respect to the next pending timer.
> > +#endif
> > + base = this_cpu_ptr(&timer_bases[BASE_STD]);
> > + if (!housekeeping_cpu(base->cpu, HK_FLAG_TIMER)) {
> > + if (!bitmap_empty(base->pending_map, WHEEL_SIZE))
> > + goto raise;
> > + base++;
> > + if (!bitmap_empty(base->pending_map, WHEEL_SIZE))
> > + goto raise;
> > +
> > + return;
> > + }
> > +
> > +raise:
> > raise_softirq(TIMER_SOFTIRQ);
> > }
> >
> >
>
> Thanks,
>
> Anna-Maria
next prev parent reply other threads:[~2019-05-30 20:15 UTC|newest]
Thread overview: 18+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-04-15 20:12 [patch 0/3] do not raise timer softirq unconditionally (spinlockless version) Marcelo Tosatti
2019-04-15 20:12 ` [patch 1/3] timers: raise timer softirq on __mod_timer/add_timer_on Marcelo Tosatti
2019-05-29 14:53 ` Anna-Maria Gleixner
2019-05-30 19:23 ` Marcelo Tosatti
2019-04-15 20:12 ` [patch 2/3] timers: do not raise softirq unconditionally (spinlockless version) Marcelo Tosatti
2019-05-29 14:53 ` Anna-Maria Gleixner
2019-05-30 20:14 ` Marcelo Tosatti [this message]
2019-05-31 11:55 ` Anna-Maria Gleixner
2019-06-11 11:45 ` Anna-Maria Gleixner
2019-06-04 6:29 ` Peter Xu
2019-06-06 15:14 ` Marcelo Tosatti
2019-04-15 20:12 ` [patch 3/3] timers: condense pending bitmap information Marcelo Tosatti
2019-04-15 20:17 ` [patch 0/3] do not raise timer softirq unconditionally (spinlockless version) Marcelo Tosatti
2019-05-06 3:22 ` Marcelo Tosatti
2019-05-06 7:17 ` Daniel Bristot de Oliveira
2019-05-06 9:22 ` Thomas Gleixner
2019-05-29 14:52 ` Anna-Maria Gleixner
2019-05-30 19:38 ` Marcelo Tosatti
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20190530201455.GC23199@amt.cnet \
--to=mtosatti@redhat.com \
--cc=anna-maria@linutronix.de \
--cc=bristot@redhat.com \
--cc=haris.okanovic@ni.com \
--cc=lcapitulino@redhat.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-rt-users@vger.kernel.org \
--cc=tglx@linutronix.de \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).