From: Marcelo Tosatti <mtosatti@redhat.com>
To: Andrea Arcangeli <aarcange@redhat.com>
Cc: kvm-devel <kvm@vger.kernel.org>,
Paolo Bonzini <pbonzini@redhat.com>,
Radim Krcmar <rkrcmar@redhat.com>,
"Rafael J. Wysocki" <rafael.j.wysocki@intel.com>,
Peter Zijlstra <peterz@infradead.org>,
Wanpeng Li <kernellwp@gmail.com>,
Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>,
Raslan KarimAllah <karahmed@amazon.de>,
Boris Ostrovsky <boris.ostrovsky@oracle.com>,
Ankur Arora <ankur.a.arora@oracle.com>,
Christian Borntraeger <borntraeger@de.ibm.com>
Subject: Re: [patch 1/3] drivers/cpuidle: add cpuidle-haltpoll driver
Date: Fri, 7 Jun 2019 17:20:47 -0300 [thread overview]
Message-ID: <20190607202044.GA5542@amt.cnet> (raw)
In-Reply-To: <20190606175103.GD28785@redhat.com>
Hi Andrea,
On Thu, Jun 06, 2019 at 01:51:03PM -0400, Andrea Arcangeli wrote:
> Hello,
>
> On Mon, Jun 03, 2019 at 07:52:43PM -0300, Marcelo Tosatti wrote:
> > +unsigned int guest_halt_poll_ns = 200000;
> > +module_param(guest_halt_poll_ns, uint, 0644);
> > +
> > +/* division factor to shrink halt_poll_ns */
> > +unsigned int guest_halt_poll_shrink = 2;
> > +module_param(guest_halt_poll_shrink, uint, 0644);
> > +
> > +/* multiplication factor to grow per-cpu halt_poll_ns */
> > +unsigned int guest_halt_poll_grow = 2;
> > +module_param(guest_halt_poll_grow, uint, 0644);
> > +
> > +/* value in ns to start growing per-cpu halt_poll_ns */
> > +unsigned int guest_halt_poll_grow_start = 10000;
> > +module_param(guest_halt_poll_grow_start, uint, 0644);
> > +
> > +/* value in ns to start growing per-cpu halt_poll_ns */
> > +bool guest_halt_poll_allow_shrink = true;
> > +module_param(guest_halt_poll_allow_shrink, bool, 0644);
>
> These variables can all be static. They also should be __read_mostly
> to be sure not to unnecessarily hit false sharing while going idle.
Fixed.
>
> > + while (!need_resched()) {
> > + cpu_relax();
> > + now = ktime_get();
> > +
> > + if (!ktime_before(now, end_spin)) {
> > + do_halt = 1;
> > + break;
> > + }
> > + }
>
> On skylake pause takes ~75 cycles with ple_gap=0 (and Marcelo found it
> takes 6 cycles with pause loop exiting enabled but that shall be fixed
> in the CPU and we can ignore it).
Right, that is a generic problem.
> So we could call ktime_get() only once every 100 times or more and
> we'd be still accurate down to at least 1usec.
>
> Ideally we'd like a ktime_try_get() that will break the seqcount loop
> if read_seqcount_retry fails. Something like below pseudocode:
>
> #define KTIME_ERR ((ktime_t) { .tv64 = 0 })
>
> ktime_t ktime_try_get(void)
> {
> [..]
> seq = read_seqcount_begin(&timekeeper_seq);
> secs = tk->xtime_sec + tk->wall_to_monotonic.tv_sec;
> nsecs = timekeeping_get_ns(&tk->tkr_mono) +
> tk->wall_to_monotonic.tv_nsec;
> if (unlikely(read_seqcount_retry(&timekeeper_seq, seq)))
> return KTIME_ERR;
> [..]
> }
>
> If it ktime_try_get() fails we keep calling it at every iteration of
> the loop, when finally it succeeds we call it again only after 100
> pause instructions or more. So we continue polling need_resched()
> while we wait timerkeeper_seq to be released (and hopefully by looping
> 100 times or more we'll reduce the frequency when we find
> timekeeper_seq locked).
>
> All we care is to react to need_resched ASAP and to have a resolution
> of the order of 1usec for the spin time.
>
> If 100 is too wired a new module parameter in __read_mostly section
> configured to 100 or more by default, sounds preferable than hitting
> every 75nsec on the timekeeper_seq seqcount cacheline.
I'll switch to the cheaper sched_clock as suggested by Peter.
Thanks!
>
> I doubt it'd make any measurable difference with a few vcpus, but with
> hundred of host CPUs and vcpus perhaps it's worth it.
>
> This of course can be done later once the patch is merged and if it's
> confirmed the above makes sense in practice and not just in theory. I
> wouldn't want to delay the merging for a possible micro optimization.
>
> Reviewed-by: Andrea Arcangeli <aarcange@redhat.com>
>
> Thanks,
> Andrea
next prev parent reply other threads:[~2019-06-07 21:39 UTC|newest]
Thread overview: 29+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-06-03 22:52 [patch 0/3] cpuidle-haltpoll driver (v2) Marcelo Tosatti
2019-06-03 22:52 ` [patch 1/3] drivers/cpuidle: add cpuidle-haltpoll driver Marcelo Tosatti
2019-06-05 8:16 ` Ankur Arora
2019-06-06 17:51 ` Andrea Arcangeli
2019-06-07 20:20 ` Marcelo Tosatti [this message]
2019-06-06 19:03 ` Peter Zijlstra
2019-06-07 9:54 ` Rafael J. Wysocki
2019-06-03 22:52 ` [patch 2/3] kvm: x86: add host poll control msrs Marcelo Tosatti
2019-06-06 12:04 ` Paolo Bonzini
2019-06-03 22:52 ` [patch 3/3] cpuidle-haltpoll: disable host side polling when kvm virtualized Marcelo Tosatti
2019-06-04 1:26 ` kbuild test robot
2019-06-04 12:24 ` [patch v2 " Marcelo Tosatti
2019-06-06 18:25 ` Joao Martins
2019-06-06 18:36 ` Andrea Arcangeli
2019-06-06 18:51 ` Joao Martins
2019-06-06 19:22 ` Joao Martins
2019-06-06 21:01 ` Andrea Arcangeli
2019-06-07 20:38 ` Marcelo Tosatti
2019-06-07 20:25 ` Marcelo Tosatti
2019-06-07 9:49 ` [patch 0/3] cpuidle-haltpoll driver (v2) Rafael J. Wysocki
2019-06-07 17:16 ` Marcelo Tosatti
2019-06-07 18:22 ` Paolo Bonzini
2019-06-07 21:38 ` Marcelo Tosatti
2019-06-10 14:59 ` Marcelo Tosatti
2019-06-10 22:03 ` Rafael J. Wysocki
2019-06-11 14:26 ` Marcelo Tosatti
2019-06-11 21:24 ` Rafael J. Wysocki
2019-06-17 15:57 ` Peter Zijlstra
2019-06-11 19:40 [patch 0/3] cpuidle-haltpoll driver (v3) Marcelo Tosatti
2019-06-11 19:40 ` [patch 1/3] drivers/cpuidle: add cpuidle-haltpoll driver Marcelo Tosatti
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20190607202044.GA5542@amt.cnet \
--to=mtosatti@redhat.com \
--cc=aarcange@redhat.com \
--cc=ankur.a.arora@oracle.com \
--cc=boris.ostrovsky@oracle.com \
--cc=borntraeger@de.ibm.com \
--cc=karahmed@amazon.de \
--cc=kernellwp@gmail.com \
--cc=konrad.wilk@oracle.com \
--cc=kvm@vger.kernel.org \
--cc=pbonzini@redhat.com \
--cc=peterz@infradead.org \
--cc=rafael.j.wysocki@intel.com \
--cc=rkrcmar@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).