From mboxrd@z Thu Jan  1 00:00:00 1970
From: Pasi =?iso-8859-1?Q?K=E4rkk=E4inen?= <pasik@iki.fi>
Subject: Re: Re: Test results for save/restore with upstream
	pv_ops domU kernels, 2.6.32.7 works OK
Date: Mon, 1 Feb 2010 18:12:58 +0200
Message-ID: <20100201161258.GA2861@reaktio.net>
References: <20100128212559.GF2861@reaktio.net>
	<1264761332.2965.4889.camel@zakaz.uk.xensource.com>
	<20100129105338.GM2861@reaktio.net>
	<20100201151010.GZ2861@reaktio.net> <4B66F28D.4070109@redhat.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Transfer-Encoding: quoted-printable
Return-path: <xen-devel-bounces@lists.xensource.com>
Content-Disposition: inline
In-Reply-To: <4B66F28D.4070109@redhat.com>
List-Unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>,
	<mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xensource.com>
List-Help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-Subscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>,
	<mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
Sender: xen-devel-bounces@lists.xensource.com
Errors-To: xen-devel-bounces@lists.xensource.com
To: Andrew Jones <drjones@redhat.com>
Cc: Jeremy Fitzhardinge <jeremy@goop.org>, "xen-devel@lists.xensource.com" <xen-devel@lists.xensource.com>, Ian Campbell <Ian.Campbell@citrix.com>
List-Id: xen-devel@lists.xenproject.org

On Mon, Feb 01, 2010 at 04:26:05PM +0100, Andrew Jones wrote:
> On 02/01/2010 04:10 PM, Pasi K=E4rkk=E4inen wrote:
> > On Fri, Jan 29, 2010 at 12:53:38PM +0200, Pasi K=E4rkk=E4inen wrote:
> >> On Fri, Jan 29, 2010 at 10:35:32AM +0000, Ian Campbell wrote:
> >>> On Thu, 2010-01-28 at 21:25 +0000, Pasi K=E4rkk=E4inen wrote:
> >>>> Hello,
> >>>>
> >>>> I just tried some save/restore tests with Fedora 12 Linux 2.6.31.1=
2 kernels.
> >>>> The exact Fedora kernel versions are: 2.6.31.12-174.2.3.fc12.i686.=
PAE and 2.6.31.12-174.2.3.fc12.x86_64.
> >>>>
> >>>> Dom0 for these tests was CentOS 5.4 (Xen 3.1.2).
> >>>>
> >>>> - F12 32bit 1vcpu PV guest:=20
> >>>> 	save+restore OK, BUG() in guest dmesg after restore [1]
> >>>>
> >>>> - F12 64bit 1vcpu PV guest:
> >>>> 	save+restore OK, BUG() in guest dmesg after restore [2]
> >>>
> >>> I think those are the same underlying bug and are fixed by=20
> >>> http://git.kernel.org/?p=3Dlinux/kernel/git/jeremy/xen.git;a=3Dcomm=
it;h=3D777df2b017ef34b2d1a172bf769582158839a860
> >>>
> >>
> >> Ok.=20
> >>
> >> There was also this traceback in the beginning of boot, on all 32bit=
/64bit, 1vcpu2/vcpu guest combinations:
> >>
> >> Performance Counters: Core2 events, Intel PMU driver.
> >> ------------[ cut here ]------------
> >> WARNING: at arch/x86/kernel/apic/apic.c:247 native_apic_write_dummy+=
0x32/0x3e() (Not tainted)
> >> Modules linked in:
> >> Pid: 0, comm: swapper Not tainted 2.6.31.12-174.2.3.fc12.i686.PAE #1
> >> Call Trace:
> >>  [<c043db4b>] warn_slowpath_common+0x70/0x87
> >>  [<c041cfb2>] ? native_apic_write_dummy+0x32/0x3e
> >>  [<c043db74>] warn_slowpath_null+0x12/0x15
> >>  [<c041cfb2>] native_apic_write_dummy+0x32/0x3e
> >>  [<c0411e04>] perf_counters_lapic_init+0x30/0x32
> >>  [<c09b3b1b>] init_hw_perf_counters+0x2bc/0x355
> >>  [<c09b3628>] identify_boot_cpu+0x21/0x23
> >>  [<c09b378e>] check_bugs+0xb/0xdc
> >>  [<c047fd73>] ? delayacct_init+0x47/0x4c
> >>  [<c09ab8b4>] start_kernel+0x31c/0x330
> >>  [<c09ab081>] i386_start_kernel+0x70/0x77
> >>  [<c09ae2bb>] xen_start_kernel+0x4b9/0x4c1
> >>  [<c04090a1>] ? syscall_exit+0x1/0x16
> >> ---[ end trace a7919e7f17c0a725 ]---
> >>
> >> Full boot logs here:
> >> http://pasik.reaktio.net/xen/debug/fedora/
> >>
> >=20
> >=20
> > This boot-time traceback disappeared when I updated the guest to 2.6.=
32.7.
> >=20
> >=20
> >>
> >>>>
> >>>> - F12 32bit 2vcpu PV guest:
> >>>> 	save doesn't work, guest stays as "migrating-f12test32" in "xm li=
st" forever and has to be "xm destroy"ed.
> >>>>
> >>>> - F12 64bit 2vcpu PV guest:
> >>>> 	save doesn't work, guest stays as "migrating-f12test64" in "xm li=
st" forever and has to be "xm destroy"ed.
> >>>>
> >>>>
> >>>> What's the best way to debug failing "xm save" ? There was no erro=
rs in "xm log", or in "xm dmesg".
> >>>
> >>> I think you might see some stuff in /var/log/xen/something but I do=
n't
> >>> have any particular tips apart from "add printf/printk".
> >>>
> >>
> >> I'll check /var/log/xen/.
> >>
> >>>> Also the guest "xm console" doesn't show anything before it dies.
> >>>>
> >>>> Is it possible some of the save/restore related patches didn't mak=
e it to 2.6.31.x stable kernels?=20
> >>>
> >>> AFAIK they only went into the 2.6.32 stable branch. Unfortunately I
> >>> think the 2.6.31 stable series has come to an end now.
> >>>
> >>
> >> Ok. I'll test 2.6.32.latest aswell.
> >>
> >=20
> > I grabbed upstream kernel.org Linux 2.6.32.7, and tested the followin=
g combinations:
> >=20
> > - F12 32bit 1vcpu PV guest
> > - F12 32bit 2vcpu PV guest
> > - F12 64bit 1vcpu PV guest
> > - F12 64bit 2vcpu PV guest
> >=20
> > save+restore was successfull for all of the above guests running 2.6.=
32.7.=20
> > No BUGs or tracebacks anymore.
> >=20
> > Any tips for git magic to get all the recent save/restore fixes that =
went to 2.6.32.x,
> > so I could send them to Fedora people to apply to F12 kernel?=20
> >=20
>=20
> This is great news, but it might a good idea to try 2 or more
> save-restore rounds in row first, if you haven't already. In the past
> I've seen 1 save/restore work, but then the 2nd round fail. Although,
> usually there's some symptom of badness on the 1st round as well.
>=20

I forgot to mention that I tried twice with all of the above guests :)
It seems stable.

I just did one more test.. save+restore 5 times in a row, with 4 vcpu PV =
guest.=20
No problems found.

-- Pasi