From: Igor Druzhinin <igor.druzhinin@citrix.com>
To: "Roger Pau Monné" <roger.pau@citrix.com>
Cc: xen-devel@lists.xenproject.org, wl@xen.org, jbeulich@suse.com,
andrew.cooper3@citrix.com
Subject: Re: [Xen-devel] [PATCH] x86/time: update TSC stamp on restore from deep C-state
Date: Wed, 15 Jan 2020 12:36:08 +0000 [thread overview]
Message-ID: <7d9e9668-3b21-46c6-5428-b6e9394bb8fe@citrix.com> (raw)
In-Reply-To: <20200115094703.GH11756@Air-de-Roger>
On 15/01/2020 09:47, Roger Pau Monné wrote:
> On Tue, Jan 14, 2020 at 07:36:21PM +0000, Igor Druzhinin wrote:
>> If ITSC is not available on CPU (e.g if running nested as PV shim)
>> then X86_FEATURE_NONSTOP_TSC is not advertised in certain cases, i.e.
>> all AMD and some old Intel processors. In which case TSC would need to
>> be restored on CPU from platform time by Xen upon exiting deep C-states.
>>
>> As platform time might be behind the last TSC stamp recorded for the
>> current CPU, invariant of TSC stamp being always behind local TSC counter
>> is violated. This has an effect of get_s_time() going negative resulting
>> in eventual system hang or crash.
>>
>> Fix this issue by updating local TSC stamp along with TSC counter write.
>
> Thanks! I haven't seen such issue because I've been running the shim
> with nomigrate in order to prevent the vTSC overhead.
>
>>
>> Signed-off-by: Igor Druzhinin <igor.druzhinin@citrix.com>
>> ---
>> This caused reliable hangs of shim domains with multiple vCPUs on all AMD
>> systems. The problem got also reproduced on bare-metal by artifically
>> masking ITSC feature bit. The proposed fix has been verified for both
>> cases.
>> ---
>> xen/arch/x86/time.c | 8 +++++++-
>> 1 file changed, 7 insertions(+), 1 deletion(-)
>>
>> diff --git a/xen/arch/x86/time.c b/xen/arch/x86/time.c
>> index e79cb4d..f6b26f8 100644
>> --- a/xen/arch/x86/time.c
>> +++ b/xen/arch/x86/time.c
>> @@ -955,10 +955,16 @@ u64 stime2tsc(s_time_t stime)
>>
>> void cstate_restore_tsc(void)
>> {
>> + struct cpu_time *t = &this_cpu(cpu_time);
>> +
>> if ( boot_cpu_has(X86_FEATURE_NONSTOP_TSC) )
>> return;
>>
>> - write_tsc(stime2tsc(read_platform_stime(NULL)));
>> + t->stamp.master_stime = read_platform_stime(NULL);
>> + t->stamp.local_tsc = stime2tsc(t->stamp.master_stime);
>> + t->stamp.local_stime = t->stamp.master_stime;
>> +
>> + write_tsc(t->stamp.local_tsc);
>
> In order to avoid the TSC write (and the likely associated vmexit),
> could you instead do:
>
> t->stamp.local_stime = t->stamp.master_stime = read_platform_stime(NULL);
> t->stamp.local_tsc = rdtsc_ordered();
I think in that case RDTSC might return something behind platform time
which is not right I guess.
Igor
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
next prev parent reply other threads:[~2020-01-15 12:36 UTC|newest]
Thread overview: 23+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-01-14 19:36 [Xen-devel] [PATCH] x86/time: update TSC stamp on restore from deep C-state Igor Druzhinin
2020-01-15 9:47 ` Roger Pau Monné
2020-01-15 11:40 ` Jan Beulich
2020-01-15 11:53 ` Roger Pau Monné
2020-01-15 12:49 ` Jan Beulich
2020-01-15 13:44 ` Roger Pau Monné
2020-01-15 16:21 ` Jan Beulich
2020-01-16 9:33 ` Roger Pau Monné
2020-01-16 9:38 ` Jan Beulich
2020-01-16 12:09 ` Igor Druzhinin
2020-01-16 12:25 ` Roger Pau Monné
2020-01-15 12:25 ` Igor Druzhinin
2020-01-15 12:31 ` Igor Druzhinin
2020-01-15 12:41 ` Jan Beulich
2020-01-15 12:36 ` Igor Druzhinin [this message]
2020-01-15 13:23 ` Roger Pau Monné
2020-01-15 14:45 ` Igor Druzhinin
2020-01-15 11:32 ` Jan Beulich
2020-01-15 12:28 ` Igor Druzhinin
2020-01-15 12:39 ` Jan Beulich
2020-01-15 12:47 ` Igor Druzhinin
2020-01-15 12:54 ` Jan Beulich
2020-01-15 14:55 ` Andrew Cooper
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=7d9e9668-3b21-46c6-5428-b6e9394bb8fe@citrix.com \
--to=igor.druzhinin@citrix.com \
--cc=andrew.cooper3@citrix.com \
--cc=jbeulich@suse.com \
--cc=roger.pau@citrix.com \
--cc=wl@xen.org \
--cc=xen-devel@lists.xenproject.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).