From: Jan Beulich <firstname.lastname@example.org> To: "email@example.com" <firstname.lastname@example.org> Cc: "Andrew Cooper" <email@example.com>, "Wei Liu" <firstname.lastname@example.org>, "Roger Pau Monné" <email@example.com> Subject: [PATCH v4 1/3] x86/time: latch to-be-written TSC value early in rendezvous loop Date: Thu, 1 Apr 2021 11:54:05 +0200 [thread overview] Message-ID: <firstname.lastname@example.org> (raw) In-Reply-To: <email@example.com> To reduce latency on time_calibration_tsc_rendezvous()'s last loop iteration, read the value to be written on the last iteration at the end of the loop body (i.e. in particular at the end of the second to last iteration). On my single-socket 18-core Skylake system this reduces the average loop exit time on CPU0 (from the TSC write on the last iteration to until after the main loop) from around 32k cycles to around 29k (albeit the values measured on separate runs vary quite significantly). Signed-off-by: Jan Beulich <firstname.lastname@example.org> --- v4: Different approach. v3: New. --- Of course it would also be nice to avoid the pretty likely branch misprediction on the last iteration. But with the static prediction hints having been rather short-lived in the architecture, I don't see any good means to do so. --- a/xen/arch/x86/time.c +++ b/xen/arch/x86/time.c @@ -1683,7 +1683,7 @@ static void time_calibration_tsc_rendezv int i; struct calibration_rendezvous *r = _r; unsigned int total_cpus = cpumask_weight(&r->cpu_calibration_map); - uint64_t tsc = 0; + uint64_t tsc = 0, master_tsc = 0; /* Loop to get rid of cache effects on TSC skew. */ for ( i = 4; i >= 0; i-- ) @@ -1708,7 +1708,7 @@ static void time_calibration_tsc_rendezv atomic_inc(&r->semaphore); if ( i == 0 ) - write_tsc(r->master_tsc_stamp); + write_tsc(master_tsc); while ( atomic_read(&r->semaphore) != (2*total_cpus - 1) ) cpu_relax(); @@ -1730,7 +1730,7 @@ static void time_calibration_tsc_rendezv } if ( i == 0 ) - write_tsc(r->master_tsc_stamp); + write_tsc(master_tsc); atomic_inc(&r->semaphore); while ( atomic_read(&r->semaphore) > total_cpus ) @@ -1739,9 +1739,17 @@ static void time_calibration_tsc_rendezv /* Just in case a read above ended up reading zero. */ tsc += !tsc; + + /* + * To reduce latency of the TSC write on the last iteration, + * fetch the value to be written into a local variable. To avoid + * introducing yet another conditional branch (which the CPU may + * have difficulty predicting well) do this on all iterations. + */ + master_tsc = r->master_tsc_stamp; } - time_calibration_rendezvous_tail(r, tsc, r->master_tsc_stamp); + time_calibration_rendezvous_tail(r, tsc, master_tsc); } /* Ordinary rendezvous function which does not modify TSC values. */
next prev parent reply other threads:[~2021-04-01 9:54 UTC|newest] Thread overview: 14+ messages / expand[flat|nested] mbox.gz Atom feed top 2021-04-01 9:53 [PATCH v4 0/3] x86/time: calibration rendezvous adjustments Jan Beulich 2021-04-01 9:54 ` Jan Beulich [this message] 2021-04-20 15:44 ` [PATCH v4 1/3] x86/time: latch to-be-written TSC value early in rendezvous loop Roger Pau Monné 2021-04-01 9:54 ` [PATCH v4 2/3] x86/time: yield to hyperthreads after updating TSC during rendezvous Jan Beulich 2021-04-20 15:59 ` Roger Pau Monné 2021-04-21 9:57 ` Jan Beulich 2021-04-01 9:55 ` [PATCH v4 3/3] x86/time: avoid reading the platform timer in rendezvous functions Jan Beulich 2021-04-20 16:12 ` Roger Pau Monné 2021-04-21 10:06 ` Jan Beulich 2021-04-29 9:32 ` Jan Beulich 2021-04-29 12:48 ` Roger Pau Monné 2021-04-29 12:53 ` Roger Pau Monné 2021-04-29 13:51 ` Jan Beulich 2021-04-15 9:54 ` Ping: [PATCH v4 0/3] x86/time: calibration rendezvous adjustments Jan Beulich
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --subject='Re: [PATCH v4 1/3] x86/time: latch to-be-written TSC value early in rendezvous loop' \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).