From mboxrd@z Thu Jan 1 00:00:00 1970 From: Ian Campbell Subject: Re: Crash with paravirt-ops 2.6.31.6 kernel Date: Tue, 24 Nov 2009 09:48:39 +0000 Message-ID: <1259056119.7590.127.camel@zakaz.uk.xensource.com> References: <28846609.721258484676784.JavaMail.root@ifrit.dereferenced.org> <20091122095403.GA1496@wavehammer.waldi.eu.org> <1258989935.7590.52.camel@zakaz.uk.xensource.com> <4B0B2B5A.5000005@goop.org> Mime-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <4B0B2B5A.5000005@goop.org> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xensource.com Errors-To: xen-devel-bounces@lists.xensource.com To: Jeremy Fitzhardinge Cc: "xen-devel@lists.xensource.com" , Bastian Blank , "544145@bugs.debian.org" <544145@bugs.debian.org> List-Id: xen-devel@lists.xenproject.org On Tue, 2009-11-24 at 00:39 +0000, Jeremy Fitzhardinge wrote: > On 11/23/09 07:25, Ian Campbell wrote: > > On Sun, 2009-11-22 at 09:54 +0000, Bastian Blank wrote: > > > >> On Tue, Nov 17, 2009 at 10:04:36PM +0300, William Pitcock wrote: > >> > >>> [ 1.254927] init[1] general protection ip:f779042f sp:ff9b0340 error:0 > >>> > >> Hmm, this looks like the old Debian bug 544145[1]. For some reason the > >> hypervisor jumps back into 64bit mode after a syscall instruction. > >> Workaround: vdso32=0 or deinstall libc6-i686, > >> > > I just noticed that one of my test boxes has a AMD processor so I took a > > bit of a look into this. > > > > The issue seems to be with this bit of code in the hypervisor > > (xen/arch/x86/x86_64/entry.S): > > > > restore_all_guest: > > ASSERT_INTERRUPTS_DISABLED > > RESTORE_ALL > > testw $TRAP_syscall,4(%rsp) > > jz iret_exit_to_guest > > > > addq $8,%rsp > > popq %rcx # RIP > > popq %r11 # CS > > cmpw $FLAT_USER_CS32,%r11 > > popq %r11 # RFLAGS > > popq %rsp # RSP > > je 1f > > sysretq > > 1: sysretl > > > > We are attempting to return to the Linux defined __USER_CS32 (0x23) > > which does not match the test for the Xen defined FLAT_USER_CS32 > > (0xe023) and therefore we hit the sysretq instead of the sysretl which > > causes us to return with CS 0xe033 (FLAT_USER_CS64) instead of CS > > 0xe023. > > > > Ah, good detective work. > > > This patch to the kernel fixes things but doesn't seem that > > satisfactory: > > > > It is a bit ugly. I guess you could just assert that FLAT_USER_CS32 is > part of the iret ABI so the guest has to use it, which appears to be the > defacto definition. Yes, I suppose that is reasonable. > > I'm not sure > > what can realistically do since doing the Right Thing would seem to > > involve looking up the descriptor in the GDT to determine if the > > selector is 32 or 64 bit and/or context switching IA32_STAR in some > > fashion to allow guests to specify their own userspace CS for sysret 32 > > and 64. > > > > That would be a bit awkward to do in the iret fast path. Agreed, hence "realistically". > > > Perhaps simply not returning guest userspace with sysret (as above) > > makes most sense, a syscall already takes a trap through the hypervisor > > on both entry and exit so I'm not sure the difference between sysret and > > iret is going to be noticeable. > > > > Another option might be to define VGCF_compat_mode as a new flag to > > HYPERVISOR_iret and select sysretq/sysretl based on that. This would > > still expose Xen descriptors to guests which didn't ask for one but at > > least it would (probably) be a compatible descriptor. > > > > I don't think that's much of an improvement over using the Xen selector > for cs. Of course, it requires that the Xen selectors are actually part > of the ABI and won't change at some later point. I think the guest accessible-but-Xen-defined descriptors are part of the ABI, so that's OK. Ian.