From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=0.3 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,HTML_MESSAGE,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A43B5C433ED for ; Fri, 23 Apr 2021 13:31:10 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 2455561452 for ; Fri, 23 Apr 2021 13:31:10 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2455561452 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.116351.222078 (Exim 4.92) (envelope-from ) id 1lZvtU-00067Q-S6; Fri, 23 Apr 2021 13:30:52 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 116351.222078; Fri, 23 Apr 2021 13:30:52 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1lZvtU-00067J-No; Fri, 23 Apr 2021 13:30:52 +0000 Received: by outflank-mailman (input) for mailman id 116351; Fri, 23 Apr 2021 13:30:50 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1lZvtS-00067E-Id for xen-devel@lists.xen.org; Fri, 23 Apr 2021 13:30:50 +0000 Received: from mail-lf1-x134.google.com (unknown [2a00:1450:4864:20::134]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 864be813-29c8-400d-8190-250e47f8ced6; Fri, 23 Apr 2021 13:30:49 +0000 (UTC) Received: by mail-lf1-x134.google.com with SMTP id g8so77613121lfv.12 for ; Fri, 23 Apr 2021 06:30:49 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 864be813-29c8-400d-8190-250e47f8ced6 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=2diCxYQLzVRB2YkMNKUTd4ZRrUjQDrcMGT48yjVpfqI=; b=a3urx/0Dv2zccl4T7MohpfgvHRcm6y+VhuKEOyAXgQR5T9+biDEa1NYQqeEkEUYQEg zV2+wg2J+2u7TEZDQFooKbhbEzfy7xZzioZMUtPp/i1qS3BB/VWNwqbOlT1DgMPBUwHD dic3yvDzL6PpPJso9v0MSEtp0NkJw+iUAaRHUTl0wWRX++CBDBckN+NbFDY81TbD9XOV MDmXtpXyzm0CbY6VeOXIXW/VHVwhYJafSUuP8jTN3MHmR8xNoHJVfxMBUaUjXe0BO7Z2 IODN+pT1jen8ezelxdwMYsakkxK+oc6lu8R3z8c6oClEhdHi7N8XKXscVmmb39hfwrQm HagQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=2diCxYQLzVRB2YkMNKUTd4ZRrUjQDrcMGT48yjVpfqI=; b=SAX6wrvJ159RxYRjA/WmFBtkB3As9G/HkcBhLNPLwH8nuoqRGMQryeEReFhr/jFLsJ PWH3q50k3a+tz3hi2HkcAeUilU07lHZYvct0hqx/FDvyn1EKBViFxgB5Glf8XR19dxNl dfjQNYgxraMray/zVllWVu3QsK8KCJYr2Y1ugPmL9Jdc8Mb01meIEb9awnWHHhiSOSo4 ben5/O6FSH1Q6bkrX7HBUr6IDQqGaI3ufd1lqL/G9lVOkJRNLN0dFOO8eLMShVnetWlO dRdu5gnMSnRBspcAtxixO43bt6BwLgUGRfsOiG9lGBN4z9XczhBaeNN/dv5H9uVtuK9K MbuA== X-Gm-Message-State: AOAM532eBbwVqL90/IrXU/Q4/j3u+UX7ihmzvwlpxd3d+WpUz4qpN80i /y/bhBUyfdVMN9bnjnmU+reIo9QeVODJ/qmCK6A= X-Google-Smtp-Source: ABdhPJz77iI9FT3aGot4giC2mhkCuMZqFsERH98A0heoO8SBeZLkgkXmP3kdWnJ7woEWq9Ut4nRqCl7sTdpsk/4z89Q= X-Received: by 2002:a05:6512:108a:: with SMTP id j10mr2801378lfg.559.1619184648102; Fri, 23 Apr 2021 06:30:48 -0700 (PDT) MIME-Version: 1.0 References: <6237e102-f2cf-a66e-09b6-954ebfe28f8c@list.ru> <46f8bf3e-cd6e-e2de-94c1-c8a55fb10648@list.ru> <57478641-ed64-32bd-a577-428a50c880e2@suse.com> <33f08c57-08f7-b6c3-02ed-2b66c35665f2@list.ru> <3fa1051f-705c-fdf9-272d-69dd9e7dc01d@suse.com> In-Reply-To: <3fa1051f-705c-fdf9-272d-69dd9e7dc01d@suse.com> From: =?UTF-8?B?0JPQtdC+0YDQs9C40Lkg0JfQsNC50YbQtdCy?= Date: Fri, 23 Apr 2021 16:30:37 +0300 Message-ID: Subject: Re: Fwd: [BUG] Windows is frozen after restore from snapshot To: Jan Beulich Cc: xen-devel@lists.xen.org, Sergey Kovalev Content-Type: multipart/alternative; boundary="000000000000c6a87d05c0a3cd75" --000000000000c6a87d05c0a3cd75 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Thanks, but now I'll need to understand what your quoted "frozen" and > "unfrozen" mean. Plus obviously comparators and main counter are also > different, and it's there where I suspect the issue is "frozen" - this is initial snapshot which takes about from 30 seconds to 1 minute after restore to start dispatching timer interrupts to windows guest "unfrozen" - this is state which taken after restoring "frozen" one and waiting 90 seconds when guest start receiving interrupts and starts working as expected we also made some another snapshots (again after restoring from initial "frozen" one) when system still in 'freezed" state (about ~20-30 seconds from start of restore process) and in this snapshots HPET state stays the same as in initial "frozen" state except mc64 field: capabiliy: f424008086a201 res0: 0 config: 3 res1: 0 isr: 0 res2: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] mc64: 9bafb6e4e res3: 0 timer0: config: f0000000002934 cmp: fd4aa84c fsb: 0 res4: 0 timer1: config: f0000000000130 cmp: ffffffff fsb: 0 res4: 0 timer2: config: f0000000000130 cmp: ffffffff fsb: 0 res4: 0 period[0] =3D ee6b2 period[1] =3D 0 period[2] =3D 0 =D0=BF=D1=82, 23 =D0=B0=D0=BF=D1=80. 2021 =D0=B3. =D0=B2 16:21, Jan Beulich= : > On 23.04.2021 15:10, =D0=93=D0=B5=D0=BE=D1=80=D0=B3=D0=B8=D0=B9 =D0=97=D0= =B0=D0=B9=D1=86=D0=B5=D0=B2 wrote: > >> > >> Since > >> you've taken apart saved state, could you supply the full set of > >> values (ideally multiple ones, if you happen to have them, plus > >> ones where the problem didn't occur, to allow someone perhaps > >> spot a pattern)? > >> > > > > Here is full HPET state from "frozen" snapshot according to hvm_hw_hpet > > structure: > > > > capabiliy: f424008086a201 > > res0: 0 > > config: 3 > > res1: 0 > > isr: 0 > > res2: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0= , > 0, > > 0, 0] > > mc64: 97b90bc74 > > res3: 0 > > timer0: > > config: f0000000002934 > > cmp: fd4aa84c > > fsb: 0 > > res4: 0 > > timer1: > > config: f0000000000130 > > cmp: ffffffff > > fsb: 0 > > res4: 0 > > timer2: > > config: f0000000000130 > > cmp: ffffffff > > fsb: 0 > > res4: 0 > > period[0] =3D ee6b2 > > period[1] =3D 0 > > period[2] =3D 0 > > > > This one taken from snapshot of "unfrozen" one: > > > > capabiliy: f424008086a201 > > res0: 0 > > config: 3 > > res1: 0 > > isr: 0 > > res2: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0= , > 0, > > 0, 0] > > mc64: acbd23c57 > > res3: 0 > > timer0: > > config: f000000000293c > > cmp: acbd3761b > > fsb: 0 > > res4: 0 > > timer1: > > config: f0000000000130 > > cmp: ffffffff > > fsb: 0 > > res4: 0 > > timer2: > > config: f0000000000130 > > cmp: ffffffff > > fsb: 0 > > res4: 0 > > period[0] =3D ee6b2 > > period[1] =3D 0 > > period[2] =3D 0 > > > > The only difference is HPET_TN_PERIODIC flag for timers[0].config value > > Thanks, but now I'll need to understand what your quoted "frozen" and > "unfrozen" mean. Plus obviously comparators and main counter are also > different, and it's there where I suspect the issue is. > > Jan > --000000000000c6a87d05c0a3cd75 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


Thanks, but now I'll need to understand what your quoted "frozen&q= uot; and
"unfrozen" mean. Plus obviously comparators and main counter are = also
different, and it's there where I suspect the issue is
"frozen" - this is initial snapshot which takes= about from 30 seconds to 1 minute after restore to start dispatching timer= interrupts to windows guest
"unfrozen" - this is state= which taken after restoring "frozen" one and waiting 90 seconds = when guest start receiving interrupts and starts working as expected

we also made some another snapshots (again after res= toring from initial "frozen" one) when system still in 'freez= ed" state (about ~20-30 seconds from start of restore process) and in = this snapshots HPET state stays the same as in initial "frozen" s= tate except mc64 field:
capabiliy: f42= 4008086a201
res0: 0
config: 3
res1: 0
isr: 0
res2: [0, 0, 0,= 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
mc64:= 9bafb6e4e
res3: 0
timer0:
=C2=A0 =C2=A0 =C2=A0 =C2=A0 config: f00= 00000002934
=C2=A0 =C2=A0 =C2=A0 =C2=A0 cmp: fd4aa84c
=C2=A0 =C2=A0 = =C2=A0 =C2=A0 fsb: 0
=C2=A0 =C2=A0 =C2=A0 =C2=A0 res4: 0
timer1:
= =C2=A0 =C2=A0 =C2=A0 =C2=A0 config: f0000000000130
=C2=A0 =C2=A0 =C2=A0 = =C2=A0 cmp: ffffffff
=C2=A0 =C2=A0 =C2=A0 =C2=A0 fsb: 0
=C2=A0 =C2=A0= =C2=A0 =C2=A0 res4: 0
timer2:
=C2=A0 =C2=A0 =C2=A0 =C2=A0 config: f0= 000000000130
=C2=A0 =C2=A0 =C2=A0 =C2=A0 cmp: ffffffff
=C2=A0 =C2=A0 = =C2=A0 =C2=A0 fsb: 0
=C2=A0 =C2=A0 =C2=A0 =C2=A0 res4: 0
period[0] = =3D ee6b2
period[1] =3D 0
period[2] =3D 0

=D0=BF=D1=82, 23 =D0= =B0=D0=BF=D1=80. 2021 =D0=B3. =D0=B2 16:21, Jan Beulich <jbeulich@suse.com>:
On 23.04.2021 15:10, =D0=93=D0=B5=D0=BE=D1= =80=D0=B3=D0=B8=D0=B9 =D0=97=D0=B0=D0=B9=D1=86=D0=B5=D0=B2 wrote:
>>
>> Since
>> you've taken apart saved state, could you supply the full set = of
>> values (ideally multiple ones, if you happen to have them, plus >> ones where the problem didn't occur, to allow someone perhaps<= br> >> spot a pattern)?
>>
>
> Here is full HPET state from "frozen" snapshot according to = hvm_hw_hpet
> structure:
>
> capabiliy: f424008086a201
> res0: 0
> config: 3
> res1: 0
> isr: 0
> res2: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, = 0, 0,
> 0, 0]
> mc64: 97b90bc74
> res3: 0
> timer0:
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0config: f0000000002934
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0cmp: fd4aa84c
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0fsb: 0
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0res4: 0
> timer1:
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0config: f0000000000130
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0cmp: ffffffff
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0fsb: 0
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0res4: 0
> timer2:
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0config: f0000000000130
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0cmp: ffffffff
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0fsb: 0
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0res4: 0
> period[0] =3D ee6b2
> period[1] =3D 0
> period[2] =3D 0
>
> This one taken from snapshot of "unfrozen" one:
>
> capabiliy: f424008086a201
> res0: 0
> config: 3
> res1: 0
> isr: 0
> res2: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, = 0, 0,
> 0, 0]
> mc64: acbd23c57
> res3: 0
> timer0:
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0config: f000000000293c
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0cmp: acbd3761b
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0fsb: 0
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0res4: 0
> timer1:
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0config: f0000000000130
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0cmp: ffffffff
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0fsb: 0
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0res4: 0
> timer2:
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0config: f0000000000130
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0cmp: ffffffff
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0fsb: 0
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0res4: 0
> period[0] =3D ee6b2
> period[1] =3D 0
> period[2] =3D 0
>
> The only difference is HPET_TN_PERIODIC flag for timers[0].config valu= e

Thanks, but now I'll need to understand what your quoted "frozen&q= uot; and
"unfrozen" mean. Plus obviously comparators and main counter are = also
different, and it's there where I suspect the issue is.

Jan
--000000000000c6a87d05c0a3cd75--