From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:33417) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1WBkZV-00054P-9n for qemu-devel@nongnu.org; Fri, 07 Feb 2014 07:30:22 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1WBkZQ-0003fL-3t for qemu-devel@nongnu.org; Fri, 07 Feb 2014 07:30:17 -0500 Received: from mail-ph.de-nserver.de ([85.158.179.214]:39028) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1WBkZP-0003f9-Jy for qemu-devel@nongnu.org; Fri, 07 Feb 2014 07:30:12 -0500 Message-ID: <52F4D1CA.6030501@profihost.ag> Date: Fri, 07 Feb 2014 13:30:02 +0100 From: Stefan Priebe - Profihost AG MIME-Version: 1.0 References: <50562927-0498-460b-8f29-98722e332149@mailpro> <52F496B4.5030505@profihost.ag> <20140207091513.GC2374@work-vm> <52F4A554.1080400@profihost.ag> <20140207093159.GD2374@work-vm> <52F4A952.6080007@profihost.ag> <52F4CB3E.3030203@profihost.ag> <20140207122157.GF2374@work-vm> In-Reply-To: <20140207122157.GF2374@work-vm> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Subject: Re: [Qemu-devel] [pve-devel] QEMU LIve Migration - swap_free: Bad swap file entry List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: "Dr. David Alan Gilbert" Cc: m.gibula@beyond.pl, qemu-devel , Alexandre DERUMIER Hi, Am 07.02.2014 13:21, schrieb Dr. David Alan Gilbert: > * Stefan Priebe - Profihost AG (s.priebe@profihost.ag) wrote: >> Hi, >> >> i was able to reproduce with a longer running test VM running the google >> stress test. > > Hmm that's quite a fun set of differences; I think I'd like > to understand whether the pattern is related to the pattern of what > the test is doing. > > Can you just give an explanation of exactly how you ran that test? > What you installed, how exactly you ran it. While migrating i've still no reliable way to reproduce but i'll try to. I can force the problem without migration when start with: bin/stressapptest -s 3600 -m 20 -i 20 -C 20 --force_errors => inject false errors to test error handling Stefan > Then Marcin and I can try and replicate it. > > Dave > >> And it happens exacly when the migration finishes it does not happen >> while the migration is running. >> >> Google Stress Output displays Memory errors: >> ------------------------------------------------ >> Page Error: miscompare on CPU 5(0xffff) at 0x7f52431341c0(0x0:DIMM >> Unknown): read:0x00000040000000bf, reread:0x00000040000000bf >> expected:0x00000040ffffffbf >> Report Error: miscompare : DIMM Unknown : 1 : 571s >> Page Error: miscompare on CPU 5(0xffff) at 0x7f52431341c8(0x0:DIMM >> Unknown): read:0x00000020000000df, reread:0x00000020000000df >> expected:0x00000020ffffffdf >> Report Error: miscompare : DIMM Unknown : 1 : 571s >> Hardware Error: miscompare on CPU 3(0xffff) at 0x7f4fd5c34020(0x0:DIMM >> Unknown): read:0x0000000000000000, reread:0x0000000000000000 >> expected:0xffffffffffffffff >> Report Error: miscompare : DIMM Unknown : 1 : 571s >> Hardware Error: miscompare on CPU 3(0xffff) at 0x7f4fd5c34028(0x0:DIMM >> Unknown): read:0x0000000000000000, reread:0x0000000000000000 >> expected:0xffffffffffffffff >> Report Error: miscompare : DIMM Unknown : 1 : 571s >> Hardware Error: miscompare on CPU 3(0xffff) at 0x7f4fd5c34060(0x0:DIMM >> Unknown): read:0x0000000000000000, reread:0x0000000000000000 >> expected:0xffffffffffffffff >> Report Error: miscompare : DIMM Unknown : 1 : 571s >> Hardware Error: miscompare on CPU 3(0xffff) at 0x7f4fd5c34068(0x0:DIMM >> Unknown): read:0x0000000000000000, reread:0x0000000000000000 >> expected:0xffffffffffffffff >> Report Error: miscompare : DIMM Unknown : 1 : 571s >> Hardware Error: miscompare on CPU 3(0xffff) at 0x7f4fd5c340a0(0x0:DIMM >> Unknown): read:0x0000000000000000, reread:0x0000000000000000 >> expected:0xffffffffffffffff >> Report Error: miscompare : DIMM Unknown : 1 : 571s >> Hardware Error: miscompare on CPU 3(0xffff) at 0x7f4fd5c340a8(0x0:DIMM >> Unknown): read:0x0000000000000000, reread:0x0000000000000000 >> expected:0xffffffffffffffff >> Report Error: miscompare : DIMM Unknown : 1 : 571s >> Hardware Error: miscompare on CPU 3(0xffff) at 0x7f4fd5c340e0(0x0:DIMM >> Unknown): read:0x0000000000000000, reread:0x0000000000000000 >> expected:0xffffffffffffffff >> Report Error: miscompare : DIMM Unknown : 1 : 571s >> Hardware Error: miscompare on CPU 3(0xffff) at 0x7f4fd5c340e8(0x0:DIMM >> Unknown): read:0x0000000000000000, reread:0x0000000000000000 >> expected:0xffffffffffffffff >> Report Error: miscompare : DIMM Unknown : 1 : 571s >> Hardware Error: miscompare on CPU 3(0xffff) at 0x7f4fd5c34120(0x0:DIMM >> Unknown): read:0x0000000000000000, reread:0x0000000000000000 >> expected:0xffffffffffffffff >> Report Error: miscompare : DIMM Unknown : 1 : 571s >> Hardware Error: miscompare on CPU 3(0xffff) at 0x7f4fd5c34128(0x0:DIMM >> Unknown): read:0x0000000000000000, reread:0x0000000000000000 >> expected:0xffffffffffffffff >> Report Error: miscompare : DIMM Unknown : 1 : 571s >> Hardware Error: miscompare on CPU 3(0xffff) at 0x7f4fd5c34160(0x0:DIMM >> Unknown): read:0x0000000000000000, reread:0x0000000000000000 >> expected:0xffffffffffffffff >> Report Error: miscompare : DIMM Unknown : 1 : 571s >> Hardware Error: miscompare on CPU 3(0xffff) at 0x7f4fd5c34168(0x0:DIMM >> Unknown): read:0x0000000000000000, reread:0x0000000000000000 >> expected:0xffffffffffffffff >> Report Error: miscompare : DIMM Unknown : 1 : 571s >> Hardware Error: miscompare on CPU 3(0xffff) at 0x7f4fd5c341a0(0x0:DIMM >> Unknown): read:0x0000000000000000, reread:0x0000000000000000 >> expected:0xffffffffffffffff >> Report Error: miscompare : DIMM Unknown : 1 : 571s >> Hardware Error: miscompare on CPU 3(0xffff) at 0x7f4fd5c341a8(0x0:DIMM >> Unknown): read:0x0000000000000000, reread:0x0000000000000000 >> expected:0xffffffffffffffff >> Report Error: miscompare : DIMM Unknown : 1 : 571s >> Hardware Error: miscompare on CPU 3(0xffff) at 0x7f4fd5c341e0(0x0:DIMM >> Unknown): read:0x0000000000000000, reread:0x0000000000000000 >> expected:0xffffffffffffffff >> Report Error: miscompare : DIMM Unknown : 1 : 571s >> Hardware Error: miscompare on CPU 3(0xffff) at 0x7f4fd5c341e8(0x0:DIMM >> Unknown): read:0x0000000000000000, reread:0x0000000000000000 >> expected:0xffffffffffffffff >> Report Error: miscompare : DIMM Unknown : 1 : 571s >> Hardware Error: miscompare on CPU 3(0xffff) at 0x7f4fd5c34220(0x0:DIMM >> Unknown): read:0x0000000000000000, reread:0x0000000000000000 >> expected:0xffffffffffffffff >> Report Error: miscompare : DIMM Unknown : 1 : 571s >> Hardware Error: miscompare on CPU 3(0xffff) at 0x7f4fd5c34228(0x0:DIMM >> Unknown): read:0x0000000000000000, reread:0x0000000000000000 >> expected:0xffffffffffffffff >> Report Error: miscompare : DIMM Unknown : 1 : 571s >> Hardware Error: miscompare on CPU 3(0xffff) at 0x7f4fd5c34260(0x0:DIMM >> Unknown): read:0x0000000000000000, reread:0x0000000000000000 >> expected:0xffffffffffffffff >> Report Error: miscompare : DIMM Unknown : 1 : 571s >> Hardware Error: miscompare on CPU 3(0xffff) at 0x7f4fd5c34268(0x0:DIMM >> Unknown): read:0x0000000000000000, reread:0x0000000000000000 >> expected:0xffffffffffffffff >> Report Error: miscompare : DIMM Unknown : 1 : 571s >> Hardware Error: miscompare on CPU 3(0xffff) at 0x7f4fd5c342a0(0x0:DIMM >> Unknown): read:0x0000000000000000, reread:0x0000000000000000 >> expected:0xffffffffffffffff >> Report Error: miscompare : DIMM Unknown : 1 : 571s >> Hardware Error: miscompare on CPU 3(0xffff) at 0x7f4fd5c342a8(0x0:DIMM >> Unknown): read:0x0000000000000000, reread:0x0000000000000000 >> expected:0xffffffffffffffff >> Report Error: miscompare : DIMM Unknown : 1 : 571s >> Hardware Error: miscompare on CPU 3(0xffff) at 0x7f4fd5c342e0(0x0:DIMM >> Unknown): read:0x0000000000000000, reread:0x0000000000000000 >> expected:0xffffffffffffffff >> Report Error: miscompare : DIMM Unknown : 1 : 571s >> Hardware Error: miscompare on CPU 3(0xffff) at 0x7f4fd5c342e8(0x0:DIMM >> Unknown): read:0x0000000000000000, reread:0x0000000000000000 >> expected:0xffffffffffffffff >> Report Error: miscompare : DIMM Unknown : 1 : 571s >> Hardware Error: miscompare on CPU 3(0xffff) at 0x7f4fd5c34320(0x0:DIMM >> Unknown): read:0x0000000000000000, reread:0x0000000000000000 >> expected:0xffffffffffffffff >> Report Error: miscompare : DIMM Unknown : 1 : 571s >> Hardware Error: miscompare on CPU 3(0xffff) at 0x7f4fd5c34328(0x0:DIMM >> Unknown): read:0x0000000000000000, reread:0x0000000000000000 >> expected:0xffffffffffffffff >> Report Error: miscompare : DIMM Unknown : 1 : 571s >> Hardware Error: miscompare on CPU 3(0xffff) at 0x7f4fd5c34360(0x0:DIMM >> Unknown): read:0x0000000000000000, reread:0x0000000000000000 >> expected:0xffffffffffffffff >> Report Error: miscompare : DIMM Unknown : 1 : 571s >> Hardware Error: miscompare on CPU 3(0xffff) at 0x7f4fd5c34368(0x0:DIMM >> Unknown): read:0x0000000000000000, reread:0x0000000000000000 >> expected:0xffffffffffffffff >> Report Error: miscompare : DIMM Unknown : 1 : 571s >> Hardware Error: miscompare on CPU 3(0xffff) at 0x7f4fd5c343a0(0x0:DIMM >> Unknown): read:0x0000000000000000, reread:0x0000000000000000 >> expected:0xffffffffffffffff >> Report Error: miscompare : DIMM Unknown : 1 : 571s >> Hardware Error: miscompare on CPU 3(0xffff) at 0x7f4fd5c343a8(0x0:DIMM >> Unknown): read:0x0000000000000000, reread:0x0000000000000000 >> expected:0xffffffffffffffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Hardware Error: miscompare on CPU 2(0xffff) at 0x7f535c235030(0x0:DIMM >> Unknown): read:0xfffffffd000000fd, reread:0xfffffffd000000fd >> expected:0xfffffffdfffffffd >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Hardware Error: miscompare on CPU 2(0xffff) at 0x7f535c235070(0x0:DIMM >> Unknown): read:0xfffffff7000000f7, reread:0xfffffff7000000f7 >> expected:0xfffffff7fffffff7 >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Hardware Error: miscompare on CPU 2(0xffff) at 0x7f535c2350b0(0x0:DIMM >> Unknown): read:0xffffffdf000000df, reread:0xffffffdf000000df >> expected:0xffffffdfffffffdf >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Hardware Error: miscompare on CPU 2(0xffff) at 0x7f535c2350f0(0x0:DIMM >> Unknown): read:0xffffff7f0000007f, reread:0xffffff7f0000007f >> expected:0xffffff7fffffff7f >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Hardware Error: miscompare on CPU 2(0xffff) at 0x7f535c235130(0x0:DIMM >> Unknown): read:0xfffffdff0000fd00, reread:0xfffffdff0000fd00 >> expected:0xfffffdfffffffdff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Hardware Error: miscompare on CPU 2(0xffff) at 0x7f535c235170(0x0:DIMM >> Unknown): read:0xfffff7ff0000f700, reread:0xfffff7ff0000f700 >> expected:0xfffff7fffffff7ff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Hardware Error: miscompare on CPU 2(0xffff) at 0x7f535c2351b0(0x0:DIMM >> Unknown): read:0xffffdfff0000df00, reread:0xffffdfff0000df00 >> expected:0xffffdfffffffdfff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Hardware Error: miscompare on CPU 2(0xffff) at 0x7f535c2351f0(0x0:DIMM >> Unknown): read:0xffff7fff00007f00, reread:0xffff7fff00007f00 >> expected:0xffff7fffffff7fff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Hardware Error: miscompare on CPU 2(0xffff) at 0x7f535c235230(0x0:DIMM >> Unknown): read:0xfffdffff00fd0000, reread:0xfffdffff00fd0000 >> expected:0xfffdfffffffdffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Hardware Error: miscompare on CPU 2(0xffff) at 0x7f535c235270(0x0:DIMM >> Unknown): read:0xfff7ffff00f70000, reread:0xfff7ffff00f70000 >> expected:0xfff7fffffff7ffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Hardware Error: miscompare on CPU 2(0xffff) at 0x7f535c2352b0(0x0:DIMM >> Unknown): read:0xffdfffff00df0000, reread:0xffdfffff00df0000 >> expected:0xffdfffffffdfffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Hardware Error: miscompare on CPU 2(0xffff) at 0x7f535c2352f0(0x0:DIMM >> Unknown): read:0xff7fffff007f0000, reread:0xff7fffff007f0000 >> expected:0xff7fffffff7fffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Hardware Error: miscompare on CPU 2(0xffff) at 0x7f535c235330(0x0:DIMM >> Unknown): read:0xfdfffffffd000000, reread:0xfdfffffffd000000 >> expected:0xfdfffffffdffffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Hardware Error: miscompare on CPU 2(0xffff) at 0x7f535c235370(0x0:DIMM >> Unknown): read:0xf7fffffff7000000, reread:0xf7fffffff7000000 >> expected:0xf7fffffff7ffffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Hardware Error: miscompare on CPU 2(0xffff) at 0x7f535c2353b0(0x0:DIMM >> Unknown): read:0xdfffffffdf000000, reread:0xdfffffffdf000000 >> expected:0xdfffffffdfffffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Hardware Error: miscompare on CPU 2(0xffff) at 0x7f535c2353f0(0x0:DIMM >> Unknown): read:0x7fffffff7f000000, reread:0x7fffffff7f000000 >> expected:0x7fffffff7fffffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Hardware Error: miscompare on CPU 2(0xffff) at 0x7f535c235430(0x0:DIMM >> Unknown): read:0xdfffffffdf000000, reread:0xdfffffffdf000000 >> expected:0xdfffffffdfffffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Hardware Error: miscompare on CPU 2(0xffff) at 0x7f535c235470(0x0:DIMM >> Unknown): read:0xf7fffffff7000000, reread:0xf7fffffff7000000 >> expected:0xf7fffffff7ffffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Hardware Error: miscompare on CPU 2(0xffff) at 0x7f535c2354b0(0x0:DIMM >> Unknown): read:0xfdfffffffd000000, reread:0xfdfffffffd000000 >> expected:0xfdfffffffdffffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Hardware Error: miscompare on CPU 2(0xffff) at 0x7f535c2354f0(0x0:DIMM >> Unknown): read:0xff7fffff007f0000, reread:0xff7fffff007f0000 >> expected:0xff7fffffff7fffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Hardware Error: miscompare on CPU 2(0xffff) at 0x7f535c235530(0x0:DIMM >> Unknown): read:0xffdfffff00df0000, reread:0xffdfffff00df0000 >> expected:0xffdfffffffdfffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Hardware Error: miscompare on CPU 2(0xffff) at 0x7f535c235570(0x0:DIMM >> Unknown): read:0xfff7ffff00f70000, reread:0xfff7ffff00f70000 >> expected:0xfff7fffffff7ffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Hardware Error: miscompare on CPU 2(0xffff) at 0x7f535c2355b0(0x0:DIMM >> Unknown): read:0xfffdffff00fd0000, reread:0xfffdffff00fd0000 >> expected:0xfffdfffffffdffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Hardware Error: miscompare on CPU 2(0xffff) at 0x7f535c2355f0(0x0:DIMM >> Unknown): read:0xffff7fff00007f00, reread:0xffff7fff00007f00 >> expected:0xffff7fffffff7fff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Hardware Error: miscompare on CPU 2(0xffff) at 0x7f535c235630(0x0:DIMM >> Unknown): read:0xffffdfff0000df00, reread:0xffffdfff0000df00 >> expected:0xffffdfffffffdfff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Hardware Error: miscompare on CPU 2(0xffff) at 0x7f535c235670(0x0:DIMM >> Unknown): read:0xfffff7ff0000f700, reread:0xfffff7ff0000f700 >> expected:0xfffff7fffffff7ff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Hardware Error: miscompare on CPU 2(0xffff) at 0x7f535c2356b0(0x0:DIMM >> Unknown): read:0xfffffdff0000fd00, reread:0xfffffdff0000fd00 >> expected:0xfffffdfffffffdff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Hardware Error: miscompare on CPU 2(0xffff) at 0x7f535c2356f0(0x0:DIMM >> Unknown): read:0xffffff7f0000007f, reread:0xffffff7f0000007f >> expected:0xffffff7fffffff7f >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Hardware Error: miscompare on CPU 2(0xffff) at 0x7f535c235730(0x0:DIMM >> Unknown): read:0xffffffdf000000df, reread:0xffffffdf000000df >> expected:0xffffffdfffffffdf >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Hardware Error: miscompare on CPU 2(0xffff) at 0x7f535c235770(0x0:DIMM >> Unknown): read:0xfffffff7000000f7, reread:0xfffffff7000000f7 >> expected:0xfffffff7fffffff7 >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Page Error: miscompare on CPU 15(0xffff) at 0x7f5184834020(0x0:DIMM >> Unknown): read:0x00000010000000ef, reread:0x00000010000000ef >> expected:0x00000010ffffffef >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Page Error: miscompare on CPU 15(0xffff) at 0x7f5184834028(0x0:DIMM >> Unknown): read:0x00000020000000df, reread:0x00000020000000df >> expected:0x00000020ffffffdf >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Page Error: miscompare on CPU 15(0xffff) at 0x7f5184834030(0x0:DIMM >> Unknown): read:0x00000040000000bf, reread:0x00000040000000bf >> expected:0x00000040ffffffbf >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Page Error: miscompare on CPU 15(0xffff) at 0x7f5184834038(0x0:DIMM >> Unknown): read:0x000000800000007f, reread:0x000000800000007f >> expected:0x00000080ffffff7f >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Page Error: miscompare on CPU 15(0xffff) at 0x7f5184834060(0x0:DIMM >> Unknown): read:0x000010000000efff, reread:0x000010000000efff >> expected:0x00001000ffffefff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Page Error: miscompare on CPU 15(0xffff) at 0x7f5184834068(0x0:DIMM >> Unknown): read:0x000020000000dfff, reread:0x000020000000dfff >> expected:0x00002000ffffdfff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Page Error: miscompare on CPU 15(0xffff) at 0x7f5184834070(0x0:DIMM >> Unknown): read:0x000040000000bfff, reread:0x000040000000bfff >> expected:0x00004000ffffbfff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Page Error: miscompare on CPU 15(0xffff) at 0x7f5184834078(0x0:DIMM >> Unknown): read:0x0000800000007fff, reread:0x0000800000007fff >> expected:0x00008000ffff7fff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Page Error: miscompare on CPU 15(0xffff) at 0x7f51848340a0(0x0:DIMM >> Unknown): read:0x0010000000ef00ff, reread:0x0010000000ef00ff >> expected:0x00100000ffefffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Page Error: miscompare on CPU 15(0xffff) at 0x7f51848340a8(0x0:DIMM >> Unknown): read:0x0020000000df00ff, reread:0x0020000000df00ff >> expected:0x00200000ffdfffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Page Error: miscompare on CPU 15(0xffff) at 0x7f51848340b0(0x0:DIMM >> Unknown): read:0x0040000000bf00ff, reread:0x0040000000bf00ff >> expected:0x00400000ffbfffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Page Error: miscompare on CPU 15(0xffff) at 0x7f51848340b8(0x0:DIMM >> Unknown): read:0x00800000007f00ff, reread:0x00800000007f00ff >> expected:0x00800000ff7fffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Page Error: miscompare on CPU 15(0xffff) at 0x7f51848340e0(0x0:DIMM >> Unknown): read:0x10000000ef0000ff, reread:0x10000000ef0000ff >> expected:0x10000000efffffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Page Error: miscompare on CPU 15(0xffff) at 0x7f51848340e8(0x0:DIMM >> Unknown): read:0x20000000df0000ff, reread:0x20000000df0000ff >> expected:0x20000000dfffffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Page Error: miscompare on CPU 15(0xffff) at 0x7f51848340f0(0x0:DIMM >> Unknown): read:0x40000000bf0000ff, reread:0x40000000bf0000ff >> expected:0x40000000bfffffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Page Error: miscompare on CPU 15(0xffff) at 0x7f51848340f8(0x0:DIMM >> Unknown): read:0x800000007f0000ff, reread:0x800000007f0000ff >> expected:0x800000007fffffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Page Error: miscompare on CPU 15(0xffff) at 0x7f5184834120(0x0:DIMM >> Unknown): read:0x04000000fb00ff00, reread:0x04000000fb00ff00 >> expected:0x04000000fbffffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Page Error: miscompare on CPU 15(0xffff) at 0x7f5184834128(0x0:DIMM >> Unknown): read:0x02000000fd00ff00, reread:0x02000000fd00ff00 >> expected:0x02000000fdffffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Page Error: miscompare on CPU 15(0xffff) at 0x7f5184834130(0x0:DIMM >> Unknown): read:0x01000000fe00ff00, reread:0x01000000fe00ff00 >> expected:0x01000000feffffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Page Error: miscompare on CPU 15(0xffff) at 0x7f5184834138(0x0:DIMM >> Unknown): read:0x00800000007fff00, reread:0x00800000007fff00 >> expected:0x00800000ff7fffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Page Error: miscompare on CPU 15(0xffff) at 0x7f5184834160(0x0:DIMM >> Unknown): read:0x0004000000fbff00, reread:0x0004000000fbff00 >> expected:0x00040000fffbffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Page Error: miscompare on CPU 15(0xffff) at 0x7f5184834168(0x0:DIMM >> Unknown): read:0x0002000000fdff00, reread:0x0002000000fdff00 >> expected:0x00020000fffdffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Page Error: miscompare on CPU 15(0xffff) at 0x7f5184834170(0x0:DIMM >> Unknown): read:0x0001000000feff00, reread:0x0001000000feff00 >> expected:0x00010000fffeffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Page Error: miscompare on CPU 15(0xffff) at 0x7f5184834178(0x0:DIMM >> Unknown): read:0x0000800000007f00, reread:0x0000800000007f00 >> expected:0x00008000ffff7fff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Page Error: miscompare on CPU 15(0xffff) at 0x7f51848341a0(0x0:DIMM >> Unknown): read:0x000004000000fb00, reread:0x000004000000fb00 >> expected:0x00000400fffffbff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Page Error: miscompare on CPU 15(0xffff) at 0x7f51848341a8(0x0:DIMM >> Unknown): read:0x000002000000fd00, reread:0x000002000000fd00 >> expected:0x00000200fffffdff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Page Error: miscompare on CPU 15(0xffff) at 0x7f51848341b0(0x0:DIMM >> Unknown): read:0x000001000000fe00, reread:0x000001000000fe00 >> expected:0x00000100fffffeff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Page Error: miscompare on CPU 15(0xffff) at 0x7f51848341b8(0x0:DIMM >> Unknown): read:0x000000800000ff7f, reread:0x000000800000ff7f >> expected:0x00000080ffffff7f >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Page Error: miscompare on CPU 15(0xffff) at 0x7f51848341e0(0x0:DIMM >> Unknown): read:0x000000040000fffb, reread:0x000000040000fffb >> expected:0x00000004fffffffb >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Page Error: miscompare on CPU 15(0xffff) at 0x7f51848341e8(0x0:DIMM >> Unknown): read:0x000000020000fffd, reread:0x000000020000fffd >> expected:0x00000002fffffffd >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Page Error: miscompare on CPU 4(0xffff) at 0x7f50d1c00008(0x0:DIMM >> Unknown): read:0xfffffff7000000fb, reread:0xfffffff7000000fb >> expected:0xfffffff7fffffffb >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Page Error: miscompare on CPU 4(0xffff) at 0x7f50d1c00018(0x0:DIMM >> Unknown): read:0xffffff7f000000bf, reread:0xffffff7f000000bf >> expected:0xffffff7fffffffbf >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Page Error: miscompare on CPU 4(0xffff) at 0x7f50d1c00028(0x0:DIMM >> Unknown): read:0xfffff7ff0000fb00, reread:0xfffff7ff0000fb00 >> expected:0xfffff7fffffffbff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Page Error: miscompare on CPU 4(0xffff) at 0x7f50d1c00038(0x0:DIMM >> Unknown): read:0xffff7fff0000bf00, reread:0xffff7fff0000bf00 >> expected:0xffff7fffffffbfff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Page Error: miscompare on CPU 4(0xffff) at 0x7f50d1c00048(0x0:DIMM >> Unknown): read:0xfff7ffff00fb0000, reread:0xfff7ffff00fb0000 >> expected:0xfff7fffffffbffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Page Error: miscompare on CPU 4(0xffff) at 0x7f50d1c00058(0x0:DIMM >> Unknown): read:0xff7fffff00bf0000, reread:0xff7fffff00bf0000 >> expected:0xff7fffffffbfffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Page Error: miscompare on CPU 4(0xffff) at 0x7f50d1c00068(0x0:DIMM >> Unknown): read:0xf7fffffffb000000, reread:0xf7fffffffb000000 >> expected:0xf7fffffffbffffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Page Error: miscompare on CPU 4(0xffff) at 0x7f50d1c00078(0x0:DIMM >> Unknown): read:0x7fffffffbf000000, reread:0x7fffffffbf000000 >> expected:0x7fffffffbfffffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Page Error: miscompare on CPU 4(0xffff) at 0x7f50d1c00088(0x0:DIMM >> Unknown): read:0xf7ffffffef000000, reread:0xf7ffffffef000000 >> expected:0xf7ffffffefffffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Page Error: miscompare on CPU 4(0xffff) at 0x7f50d1c00098(0x0:DIMM >> Unknown): read:0xff7ffffffe000000, reread:0xff7ffffffe000000 >> expected:0xff7ffffffeffffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Page Error: miscompare on CPU 4(0xffff) at 0x7f50d1c000a8(0x0:DIMM >> Unknown): read:0xfff7ffff00ef0000, reread:0xfff7ffff00ef0000 >> expected:0xfff7ffffffefffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Page Error: miscompare on CPU 4(0xffff) at 0x7f50d1c000b8(0x0:DIMM >> Unknown): read:0xffff7fff00fe0000, reread:0xffff7fff00fe0000 >> expected:0xffff7ffffffeffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Page Error: miscompare on CPU 4(0xffff) at 0x7f50d1c000c8(0x0:DIMM >> Unknown): read:0xfffff7ff0000ef00, reread:0xfffff7ff0000ef00 >> expected:0xfffff7ffffffefff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Page Error: miscompare on CPU 4(0xffff) at 0x7f50d1c000d8(0x0:DIMM >> Unknown): read:0xffffff7f0000fe00, reread:0xffffff7f0000fe00 >> expected:0xffffff7ffffffeff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Page Error: miscompare on CPU 4(0xffff) at 0x7f50d1c000e8(0x0:DIMM >> Unknown): read:0xfffffff7000000ef, reread:0xfffffff7000000ef >> expected:0xfffffff7ffffffef >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Page Error: miscompare on CPU 4(0xffff) at 0x7f50d1c000f8(0x0:DIMM >> Unknown): read:0xffffffff000000fe, reread:0xffffffff000000fe >> expected:0xfffffffffffffffe >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Page Error: miscompare on CPU 4(0xffff) at 0x7f50d1c00108(0x0:DIMM >> Unknown): read:0xfffffff7000000fb, reread:0xfffffff7000000fb >> expected:0xfffffff7fffffffb >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Page Error: miscompare on CPU 4(0xffff) at 0x7f50d1c00118(0x0:DIMM >> Unknown): read:0xffffff7f000000bf, reread:0xffffff7f000000bf >> expected:0xffffff7fffffffbf >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Page Error: miscompare on CPU 4(0xffff) at 0x7f50d1c00128(0x0:DIMM >> Unknown): read:0xfffff7ff0000fb00, reread:0xfffff7ff0000fb00 >> expected:0xfffff7fffffffbff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Page Error: miscompare on CPU 4(0xffff) at 0x7f50d1c00138(0x0:DIMM >> Unknown): read:0xffff7fff0000bf00, reread:0xffff7fff0000bf00 >> expected:0xffff7fffffffbfff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Page Error: miscompare on CPU 4(0xffff) at 0x7f50d1c00148(0x0:DIMM >> Unknown): read:0xfff7ffff00fb0000, reread:0xfff7ffff00fb0000 >> expected:0xfff7fffffffbffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Page Error: miscompare on CPU 4(0xffff) at 0x7f50d1c00158(0x0:DIMM >> Unknown): read:0xff7fffff00bf0000, reread:0xff7fffff00bf0000 >> expected:0xff7fffffffbfffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Page Error: miscompare on CPU 4(0xffff) at 0x7f50d1c00168(0x0:DIMM >> Unknown): read:0xf7fffffffb000000, reread:0xf7fffffffb000000 >> expected:0xf7fffffffbffffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Page Error: miscompare on CPU 4(0xffff) at 0x7f50d1c00178(0x0:DIMM >> Unknown): read:0x7fffffffbf000000, reread:0x7fffffffbf000000 >> expected:0x7fffffffbfffffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Page Error: miscompare on CPU 4(0xffff) at 0x7f50d1c00188(0x0:DIMM >> Unknown): read:0xf7ffffffef000000, reread:0xf7ffffffef000000 >> expected:0xf7ffffffefffffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Page Error: miscompare on CPU 4(0xffff) at 0x7f50d1c00198(0x0:DIMM >> Unknown): read:0xff7ffffffe000000, reread:0xff7ffffffe000000 >> expected:0xff7ffffffeffffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Page Error: miscompare on CPU 4(0xffff) at 0x7f50d1c001a8(0x0:DIMM >> Unknown): read:0xfff7ffff00ef0000, reread:0xfff7ffff00ef0000 >> expected:0xfff7ffffffefffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Page Error: miscompare on CPU 4(0xffff) at 0x7f50d1c001b8(0x0:DIMM >> Unknown): read:0xffff7fff00fe0000, reread:0xffff7fff00fe0000 >> expected:0xffff7ffffffeffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Page Error: miscompare on CPU 4(0xffff) at 0x7f50d1c001c8(0x0:DIMM >> Unknown): read:0xfffff7ff0000ef00, reread:0xfffff7ff0000ef00 >> expected:0xfffff7ffffffefff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Page Error: miscompare on CPU 4(0xffff) at 0x7f50d1c001d8(0x0:DIMM >> Unknown): read:0xffffff7f0000fe00, reread:0xffffff7f0000fe00 >> expected:0xffffff7ffffffeff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Hardware Error: miscompare on CPU 8(0xffff) at 0x7f4df7200010(0x0:DIMM >> Unknown): read:0x0000000000000000, reread:0x0000000000000000 >> expected:0xffffffffffffffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Hardware Error: miscompare on CPU 8(0xffff) at 0x7f4df7200018(0x0:DIMM >> Unknown): read:0x0000000000000000, reread:0x0000000000000000 >> expected:0xffffffffffffffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Hardware Error: miscompare on CPU 8(0xffff) at 0x7f4df7200050(0x0:DIMM >> Unknown): read:0x0000000000000000, reread:0x0000000000000000 >> expected:0xffffffffffffffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Hardware Error: miscompare on CPU 8(0xffff) at 0x7f4df7200058(0x0:DIMM >> Unknown): read:0x0000000000000000, reread:0x0000000000000000 >> expected:0xffffffffffffffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Hardware Error: miscompare on CPU 8(0xffff) at 0x7f4df7200090(0x0:DIMM >> Unknown): read:0x0000000000000000, reread:0x0000000000000000 >> expected:0xffffffffffffffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Hardware Error: miscompare on CPU 8(0xffff) at 0x7f4df7200098(0x0:DIMM >> Unknown): read:0x0000000000000000, reread:0x0000000000000000 >> expected:0xffffffffffffffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Hardware Error: miscompare on CPU 8(0xffff) at 0x7f4df72000d0(0x0:DIMM >> Unknown): read:0x0000000000000000, reread:0x0000000000000000 >> expected:0xffffffffffffffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Hardware Error: miscompare on CPU 8(0xffff) at 0x7f4df72000d8(0x0:DIMM >> Unknown): read:0x0000000000000000, reread:0x0000000000000000 >> expected:0xffffffffffffffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Hardware Error: miscompare on CPU 8(0xffff) at 0x7f4df7200110(0x0:DIMM >> Unknown): read:0x0000000000000000, reread:0x0000000000000000 >> expected:0xffffffffffffffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Hardware Error: miscompare on CPU 8(0xffff) at 0x7f4df7200118(0x0:DIMM >> Unknown): read:0x0000000000000000, reread:0x0000000000000000 >> expected:0xffffffffffffffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Hardware Error: miscompare on CPU 8(0xffff) at 0x7f4df7200150(0x0:DIMM >> Unknown): read:0x0000000000000000, reread:0x0000000000000000 >> expected:0xffffffffffffffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Hardware Error: miscompare on CPU 8(0xffff) at 0x7f4df7200158(0x0:DIMM >> Unknown): read:0x0000000000000000, reread:0x0000000000000000 >> expected:0xffffffffffffffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Hardware Error: miscompare on CPU 8(0xffff) at 0x7f4df7200190(0x0:DIMM >> Unknown): read:0x0000000000000000, reread:0x0000000000000000 >> expected:0xffffffffffffffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Hardware Error: miscompare on CPU 8(0xffff) at 0x7f4df7200198(0x0:DIMM >> Unknown): read:0x0000000000000000, reread:0x0000000000000000 >> expected:0xffffffffffffffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Hardware Error: miscompare on CPU 8(0xffff) at 0x7f4df72001d0(0x0:DIMM >> Unknown): read:0x0000000000000000, reread:0x0000000000000000 >> expected:0xffffffffffffffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Hardware Error: miscompare on CPU 8(0xffff) at 0x7f4df72001d8(0x0:DIMM >> Unknown): read:0x0000000000000000, reread:0x0000000000000000 >> expected:0xffffffffffffffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Hardware Error: miscompare on CPU 8(0xffff) at 0x7f4df7200210(0x0:DIMM >> Unknown): read:0x0000000000000000, reread:0x0000000000000000 >> expected:0xffffffffffffffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Hardware Error: miscompare on CPU 8(0xffff) at 0x7f4df7200218(0x0:DIMM >> Unknown): read:0x0000000000000000, reread:0x0000000000000000 >> expected:0xffffffffffffffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Hardware Error: miscompare on CPU 8(0xffff) at 0x7f4df7200250(0x0:DIMM >> Unknown): read:0x0000000000000000, reread:0x0000000000000000 >> expected:0xffffffffffffffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Hardware Error: miscompare on CPU 8(0xffff) at 0x7f4df7200258(0x0:DIMM >> Unknown): read:0x0000000000000000, reread:0x0000000000000000 >> expected:0xffffffffffffffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Hardware Error: miscompare on CPU 8(0xffff) at 0x7f4df7200290(0x0:DIMM >> Unknown): read:0x0000000000000000, reread:0x0000000000000000 >> expected:0xffffffffffffffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Hardware Error: miscompare on CPU 8(0xffff) at 0x7f4df7200298(0x0:DIMM >> Unknown): read:0x0000000000000000, reread:0x0000000000000000 >> expected:0xffffffffffffffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Hardware Error: miscompare on CPU 8(0xffff) at 0x7f4df72002d0(0x0:DIMM >> Unknown): read:0x0000000000000000, reread:0x0000000000000000 >> expected:0xffffffffffffffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Hardware Error: miscompare on CPU 8(0xffff) at 0x7f4df72002d8(0x0:DIMM >> Unknown): read:0x0000000000000000, reread:0x0000000000000000 >> expected:0xffffffffffffffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Hardware Error: miscompare on CPU 8(0xffff) at 0x7f4df7200310(0x0:DIMM >> Unknown): read:0x0000000000000000, reread:0x0000000000000000 >> expected:0xffffffffffffffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Hardware Error: miscompare on CPU 8(0xffff) at 0x7f4df7200318(0x0:DIMM >> Unknown): read:0x0000000000000000, reread:0x0000000000000000 >> expected:0xffffffffffffffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Hardware Error: miscompare on CPU 8(0xffff) at 0x7f4df7200350(0x0:DIMM >> Unknown): read:0x0000000000000000, reread:0x0000000000000000 >> expected:0xffffffffffffffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Hardware Error: miscompare on CPU 8(0xffff) at 0x7f4df7200358(0x0:DIMM >> Unknown): read:0x0000000000000000, reread:0x0000000000000000 >> expected:0xffffffffffffffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Hardware Error: miscompare on CPU 8(0xffff) at 0x7f4df7200390(0x0:DIMM >> Unknown): read:0x0000000000000000, reread:0x0000000000000000 >> expected:0xffffffffffffffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Hardware Error: miscompare on CPU 8(0xffff) at 0x7f4df7200398(0x0:DIMM >> Unknown): read:0x0000000000000000, reread:0x0000000000000000 >> expected:0xffffffffffffffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Page Error: miscompare on CPU 7(0xffff) at 0x7f4d8aa00000(0x0:DIMM >> Unknown): read:0x0000000000000000, reread:0x0000000000000000 >> expected:0xffffffffffffffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Page Error: miscompare on CPU 7(0xffff) at 0x7f4d8aa00008(0x0:DIMM >> Unknown): read:0x0000000000000000, reread:0x0000000000000000 >> expected:0xffffffffffffffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Page Error: miscompare on CPU 7(0xffff) at 0x7f4d8aa00010(0x0:DIMM >> Unknown): read:0x0000000000000000, reread:0x0000000000000000 >> expected:0xffffffffffffffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Page Error: miscompare on CPU 7(0xffff) at 0x7f4d8aa00018(0x0:DIMM >> Unknown): read:0x0000000000000000, reread:0x0000000000000000 >> expected:0xffffffffffffffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Page Error: miscompare on CPU 7(0xffff) at 0x7f4d8aa00040(0x0:DIMM >> Unknown): read:0x0000000000000000, reread:0x0000000000000000 >> expected:0xffffffffffffffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Page Error: miscompare on CPU 7(0xffff) at 0x7f4d8aa00048(0x0:DIMM >> Unknown): read:0x0000000000000000, reread:0x0000000000000000 >> expected:0xffffffffffffffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Page Error: miscompare on CPU 7(0xffff) at 0x7f4d8aa00050(0x0:DIMM >> Unknown): read:0x0000000000000000, reread:0x0000000000000000 >> expected:0xffffffffffffffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Page Error: miscompare on CPU 7(0xffff) at 0x7f4d8aa00058(0x0:DIMM >> Unknown): read:0x0000000000000000, reread:0x0000000000000000 >> expected:0xffffffffffffffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Page Error: miscompare on CPU 7(0xffff) at 0x7f4d8aa00080(0x0:DIMM >> Unknown): read:0x0000000000000000, reread:0x0000000000000000 >> expected:0xffffffffffffffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Page Error: miscompare on CPU 7(0xffff) at 0x7f4d8aa00088(0x0:DIMM >> Unknown): read:0x0000000000000000, reread:0x0000000000000000 >> expected:0xffffffffffffffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Page Error: miscompare on CPU 7(0xffff) at 0x7f4d8aa00090(0x0:DIMM >> Unknown): read:0x0000000000000000, reread:0x0000000000000000 >> expected:0xffffffffffffffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Page Error: miscompare on CPU 7(0xffff) at 0x7f4d8aa00098(0x0:DIMM >> Unknown): read:0x0000000000000000, reread:0x0000000000000000 >> expected:0xffffffffffffffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Page Error: miscompare on CPU 7(0xffff) at 0x7f4d8aa000c0(0x0:DIMM >> Unknown): read:0x0000000000000000, reread:0x0000000000000000 >> expected:0xffffffffffffffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Page Error: miscompare on CPU 7(0xffff) at 0x7f4d8aa000c8(0x0:DIMM >> Unknown): read:0x0000000000000000, reread:0x0000000000000000 >> expected:0xffffffffffffffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Page Error: miscompare on CPU 7(0xffff) at 0x7f4d8aa000d0(0x0:DIMM >> Unknown): read:0x0000000000000000, reread:0x0000000000000000 >> expected:0xffffffffffffffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Page Error: miscompare on CPU 7(0xffff) at 0x7f4d8aa000d8(0x0:DIMM >> Unknown): read:0x0000000000000000, reread:0x0000000000000000 >> expected:0xffffffffffffffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Page Error: miscompare on CPU 7(0xffff) at 0x7f4d8aa00100(0x0:DIMM >> Unknown): read:0x0000000000000000, reread:0x0000000000000000 >> expected:0xffffffffffffffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Page Error: miscompare on CPU 7(0xffff) at 0x7f4d8aa00108(0x0:DIMM >> Unknown): read:0x0000000000000000, reread:0x0000000000000000 >> expected:0xffffffffffffffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Page Error: miscompare on CPU 7(0xffff) at 0x7f4d8aa00110(0x0:DIMM >> Unknown): read:0x0000000000000000, reread:0x0000000000000000 >> expected:0xffffffffffffffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Page Error: miscompare on CPU 7(0xffff) at 0x7f4d8aa00118(0x0:DIMM >> Unknown): read:0x0000000000000000, reread:0x0000000000000000 >> expected:0xffffffffffffffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Page Error: miscompare on CPU 7(0xffff) at 0x7f4d8aa00140(0x0:DIMM >> Unknown): read:0x0000000000000000, reread:0x0000000000000000 >> expected:0xffffffffffffffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Page Error: miscompare on CPU 7(0xffff) at 0x7f4d8aa00148(0x0:DIMM >> Unknown): read:0x0000000000000000, reread:0x0000000000000000 >> expected:0xffffffffffffffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Page Error: miscompare on CPU 7(0xffff) at 0x7f4d8aa00150(0x0:DIMM >> Unknown): read:0x0000000000000000, reread:0x0000000000000000 >> expected:0xffffffffffffffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Page Error: miscompare on CPU 7(0xffff) at 0x7f4d8aa00158(0x0:DIMM >> Unknown): read:0x0000000000000000, reread:0x0000000000000000 >> expected:0xffffffffffffffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Page Error: miscompare on CPU 7(0xffff) at 0x7f4d8aa00180(0x0:DIMM >> Unknown): read:0x0000000000000000, reread:0x0000000000000000 >> expected:0xffffffffffffffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Page Error: miscompare on CPU 7(0xffff) at 0x7f4d8aa00188(0x0:DIMM >> Unknown): read:0x0000000000000000, reread:0x0000000000000000 >> expected:0xffffffffffffffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Page Error: miscompare on CPU 7(0xffff) at 0x7f4d8aa00190(0x0:DIMM >> Unknown): read:0x0000000000000000, reread:0x0000000000000000 >> expected:0xffffffffffffffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Page Error: miscompare on CPU 7(0xffff) at 0x7f4d8aa00198(0x0:DIMM >> Unknown): read:0x0000000000000000, reread:0x0000000000000000 >> expected:0xffffffffffffffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Page Error: miscompare on CPU 7(0xffff) at 0x7f4d8aa001c0(0x0:DIMM >> Unknown): read:0x0000000000000000, reread:0x0000000000000000 >> expected:0xffffffffffffffff >> Report Error: miscompare : DIMM Unknown : 1 : 572s >> Page Error: miscompare on CPU 7(0xffff) at 0x7f4d8aa001c8(0x0:DIMM >> Unknown): read:0x0000000000000000, reread:0x0000000000000000 >> expected:0xffffffffffffffff >> Log: Thread 61 found 85568 hardware incidents >> Log: Thread 62 found 169344 hardware incidents >> Log: Thread 63 found 44544 hardware incidents >> Log: Thread 64 found 149504 hardware incidents >> Log: Thread 65 found 131968 hardware incidents >> Log: Thread 66 found 150528 hardware incidents >> Log: Thread 67 found 144384 hardware incidents >> Log: Thread 68 found 149888 hardware incidents >> Stats: Found 1025728 hardware incidents >> Stats: Completed: 9176812.00M in 524.63s 17491.92MB/s, with 1025728 >> hardware incidents, 0 errors >> Stats: Memory Copy: 4890244.00M at 9402.74MB/s >> Stats: File Copy: 0.00M at 0.00MB/s >> Stats: Net Copy: 0.00M at 0.00MB/s >> Stats: Data Check: 0.00M at 0.00MB/s >> Stats: Invert Data: 4286568.00M at 8242.44MB/s >> Stats: Disk: 0.00M at 0.00MB/s >> >> Status: FAIL - test discovered HW problems >> ----------------------------------------------------------------------------------- >> >> Stefan >> Am 07.02.2014 10:37, schrieb Stefan Priebe - Profihost AG: >>> >>> Am 07.02.2014 10:31, schrieb Dr. David Alan Gilbert: >>>> * Stefan Priebe - Profihost AG (s.priebe@profihost.ag) wrote: >>>>> >>>>> Am 07.02.2014 10:15, schrieb Dr. David Alan Gilbert: >>>>>> * Stefan Priebe - Profihost AG (s.priebe@profihost.ag) wrote: >>>>>>> Am 07.02.2014 09:15, schrieb Alexandre DERUMIER: >>>>>>>> >>>>>>>> do you use xbzrle for live migration ? >>>>>>> >>>>>>> no - i'm really stucked right now with this. Biggest problem i can't >>>>>>> reproduce with test machines ;-( >>>>>> >>>>>> Only being able to test on your production VMs isn't fun; >>>>>> is it possible or you to run an extra program on these VMs - e.g. >>>>>> if we came up with a simple (userland) memory test? >>>>> >>>>> You mean to reproduce? >>>> >>>> I'm more interested in seeing what type of corruption is happening; >>>> if you've got a test VM that corrupts memory and we can run a program >>>> in that vm that writes a known pattern into memory and checks it >>>> then see what changed after migration, it might give a clue. >>>> >>>> But obviously this would only be of any use if run on the VM that actually >>>> fails. >>> >>> Right that makes sense - sadly i still don't know how to reproduce? Any >>> app ideas i can try? >>> >>> >>>>> I already tried https://code.google.com/p/stressapptest/ while migrating >>>>> on a test VM but this works fine. >>>>> >>>>> I also tried running mysql bench while migrating on a test vm and this >>>>> works too ;-( >>>> >>>> >>>> Dave >>>> -- >>>> Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK >>>> > -- > Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK >