[RFC PATCH 0/3] Real mode interrupt injection

[RFC PATCH 0/3] Real mode interrupt injection

[Mohammed Gamal]

This patch introduces real mode interrupt injection for VMX.
It currently invokes the x86 emulator to emulate interrupts
instead of manually setting VMX controls.

Needless to say, this is not meant for merging in its current state.
The emulator still needs some more work to get this completely operational.

Mohammed Gamal (3):
  x86 emulator: Expose emulate_int_real()
  x86: Add inject_realmode_interrupt() wrapper
  VMX: Emulated real mode interrupt injection

 arch/x86/include/asm/kvm_emulate.h |    3 ++-
 arch/x86/kvm/vmx.c                 |   11 +----------
 arch/x86/kvm/x86.c                 |   14 ++++++++++++++
 arch/x86/kvm/x86.h                 |    1 +
 4 files changed, 18 insertions(+), 11 deletions(-)

[RFC PATCH 1/3] x86 emulator: Expose emulate_int_real()

[Mohammed Gamal]

Signed-off-by: Mohammed Gamal <m.gamal005@gmail.com>
---
 arch/x86/include/asm/kvm_emulate.h |    3 ++-
 1 files changed, 2 insertions(+), 1 deletions(-)

diff --git a/arch/x86/include/asm/kvm_emulate.h b/arch/x86/include/asm/kvm_emulate.h
index f22e5da..6a7cce0 100644
--- a/arch/x86/include/asm/kvm_emulate.h
+++ b/arch/x86/include/asm/kvm_emulate.h
@@ -255,5 +255,6 @@ int x86_emulate_insn(struct x86_emulate_ctxt *ctxt);
 int emulator_task_switch(struct x86_emulate_ctxt *ctxt,
 			 u16 tss_selector, int reason,
 			 bool has_error_code, u32 error_code);
-
+int emulate_int_real(struct x86_emulate_ctxt *ctxt,
+		     struct x86_emulate_ops *ops, int irq);
 #endif /* _ASM_X86_KVM_X86_EMULATE_H */
-- 
1.7.0.4

[RFC PATCH 2/3] x86: Add inject_realmode_interrupt() wrapper

[Mohammed Gamal]

This adds a wrapper function inject_realmode_interrupt() around the
emulator function emulate_int_real() to allow real mode interrupt injection.

Signed-off-by: Mohammed Gamal <m.gamal005@gmail.com>
---
 arch/x86/kvm/x86.c |   14 ++++++++++++++
 arch/x86/kvm/x86.h |    1 +
 2 files changed, 15 insertions(+), 0 deletions(-)

diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 1722d37..5915f2e 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -3936,6 +3936,20 @@ static void inject_emulated_exception(struct kvm_vcpu *vcpu)
 		kvm_queue_exception(vcpu, ctxt->exception);
 }
 
+int inject_realmode_interrupt(struct kvm_vcpu *vcpu, int irq)
+{
+	struct x86_emulate_ctxt *ctxt = &vcpu->arch.emulate_ctxt;
+	int rc;
+
+	rc = emulate_int_real(ctxt, &emulate_ops, irq);
+
+	if (rc != X86EMUL_CONTINUE)
+		return EMULATE_FAIL;
+
+	return EMULATE_DONE;
+}
+EXPORT_SYMBOL_GPL(inject_realmode_interrupt);
+
 static int handle_emulation_failure(struct kvm_vcpu *vcpu)
 {
 	++vcpu->stat.insn_emulation_fail;
diff --git a/arch/x86/kvm/x86.h b/arch/x86/kvm/x86.h
index b7a4047..c6e8a4d 100644
--- a/arch/x86/kvm/x86.h
+++ b/arch/x86/kvm/x86.h
@@ -67,5 +67,6 @@ static inline int is_paging(struct kvm_vcpu *vcpu)
 
 void kvm_before_handle_nmi(struct kvm_vcpu *vcpu);
 void kvm_after_handle_nmi(struct kvm_vcpu *vcpu);
+int inject_realmode_interrupt(struct kvm_vcpu *vcpu, int irq);
 
 #endif
-- 
1.7.0.4

[RFC PATCH 3/3] VMX: Emulated real mode interrupt injection

[Mohammed Gamal]

Signed-off-by: Mohammed Gamal <m.gamal005@gmail.com>
---
 arch/x86/kvm/vmx.c |   11 +----------
 1 files changed, 1 insertions(+), 10 deletions(-)

diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c
index 652d317..d6cb7eb 100644
--- a/arch/x86/kvm/vmx.c
+++ b/arch/x86/kvm/vmx.c
@@ -2838,16 +2838,7 @@ static void vmx_inject_irq(struct kvm_vcpu *vcpu)
 
 	++vcpu->stat.irq_injections;
 	if (vmx->rmode.vm86_active) {
-		vmx->rmode.irq.pending = true;
-		vmx->rmode.irq.vector = irq;
-		vmx->rmode.irq.rip = kvm_rip_read(vcpu);
-		if (vcpu->arch.interrupt.soft)
-			vmx->rmode.irq.rip +=
-				vmx->vcpu.arch.event_exit_inst_len;
-		vmcs_write32(VM_ENTRY_INTR_INFO_FIELD,
-			     irq | INTR_TYPE_SOFT_INTR | INTR_INFO_VALID_MASK);
-		vmcs_write32(VM_ENTRY_INSTRUCTION_LEN, 1);
-		kvm_rip_write(vcpu, vmx->rmode.irq.rip - 1);
+		inject_realmode_interrupt(vcpu, irq);
 		return;
 	}
 	intr = irq | INTR_INFO_VALID_MASK;
-- 
1.7.0.4

Re: [RFC PATCH 0/3] Real mode interrupt injection

[Avi Kivity]

  On 08/08/2010 03:24 PM, Mohammed Gamal wrote:
> This patch introduces real mode interrupt injection for VMX.
> It currently invokes the x86 emulator to emulate interrupts
> instead of manually setting VMX controls.
>
> Needless to say, this is not meant for merging in its current state.
> The emulator still needs some more work to get this completely operational.

Well, what happens when you run with it?

-- 
I have a truly marvellous patch that fixes the bug which this
signature is too narrow to contain.

Re: [RFC PATCH 2/3] x86: Add inject_realmode_interrupt() wrapper

[Avi Kivity]

  On 08/08/2010 03:24 PM, Mohammed Gamal wrote:
> This adds a wrapper function inject_realmode_interrupt() around the
> emulator function emulate_int_real() to allow real mode interrupt injection.
>
> Signed-off-by: Mohammed Gamal<m.gamal005@gmail.com>
> ---
>   arch/x86/kvm/x86.c |   14 ++++++++++++++
>   arch/x86/kvm/x86.h |    1 +
>   2 files changed, 15 insertions(+), 0 deletions(-)
>

Really has no business in x86.c, as it's there to work around a vmx 
specific problem.

>
> +int inject_realmode_interrupt(struct kvm_vcpu *vcpu, int irq)
> +{
> +	struct x86_emulate_ctxt *ctxt =&vcpu->arch.emulate_ctxt;
> +	int rc;
> +
> +	rc = emulate_int_real(ctxt,&emulate_ops, irq);
> +
> +	if (rc != X86EMUL_CONTINUE)
> +		return EMULATE_FAIL;
> +
> +	return EMULATE_DONE;
> +}
> +EXPORT_SYMBOL_GPL(inject_realmode_interrupt);

Doesn't seem to add much value.  I'd say just export emulate_int_real() 
and call it from vmx.c.

Hmm, you aren't initializing the context.  So I guess you do need this 
function (and I guess it's better to keep it in x86.c near the rest of 
the emulation stuff, just make sure the emulation context is initialized).

-- 
I have a truly marvellous patch that fixes the bug which this
signature is too narrow to contain.

Re: [RFC PATCH 3/3] VMX: Emulated real mode interrupt injection

[Avi Kivity]

  On 08/08/2010 03:24 PM, Mohammed Gamal wrote:
> Signed-off-by: Mohammed Gamal<m.gamal005@gmail.com>
> ---
>   arch/x86/kvm/vmx.c |   11 +----------
>   1 files changed, 1 insertions(+), 10 deletions(-)
>
> diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c
> index 652d317..d6cb7eb 100644
> --- a/arch/x86/kvm/vmx.c
> +++ b/arch/x86/kvm/vmx.c
> @@ -2838,16 +2838,7 @@ static void vmx_inject_irq(struct kvm_vcpu *vcpu)
>
>   	++vcpu->stat.irq_injections;
>   	if (vmx->rmode.vm86_active) {
> -		vmx->rmode.irq.pending = true;
> -		vmx->rmode.irq.vector = irq;
> -		vmx->rmode.irq.rip = kvm_rip_read(vcpu);
> -		if (vcpu->arch.interrupt.soft)
> -			vmx->rmode.irq.rip +=
> -				vmx->vcpu.arch.event_exit_inst_len;

This has to be covered somehow.  Not sure exactly - probably keep the 
same code.  Jan?

> -		vmcs_write32(VM_ENTRY_INTR_INFO_FIELD,
> -			     irq | INTR_TYPE_SOFT_INTR | INTR_INFO_VALID_MASK);
> -		vmcs_write32(VM_ENTRY_INSTRUCTION_LEN, 1);
> -		kvm_rip_write(vcpu, vmx->rmode.irq.rip - 1);
> +		inject_realmode_interrupt(vcpu, irq);
>   		return;
>   	}

Error checks?

Need to do same to vmx_inject_nmi().

fixup_rmode_irq() just became dead code, you can remove it.  Also remove 
the entire vmx->rmode.irq thing.

-- 
I have a truly marvellous patch that fixes the bug which this
signature is too narrow to contain.

Re: [RFC PATCH 0/3] Real mode interrupt injection

[Mohammed Gamal]

On 8/10/10, Avi Kivity <avi@redhat.com> wrote:
>   On 08/08/2010 03:24 PM, Mohammed Gamal wrote:
>> This patch introduces real mode interrupt injection for VMX.
>> It currently invokes the x86 emulator to emulate interrupts
>> instead of manually setting VMX controls.
>>
>> Needless to say, this is not meant for merging in its current state.
>> The emulator still needs some more work to get this completely
>> operational.
>
> Well, what happens when you run with it?
The guest fails at two instances. First it sometimes encounters a
group 7 instruction (0x0f 0x00), which the emulator doesn't emulate.
Here is the relevant part of the trace:

qemu-system-x86-4321  [001]   150.002191: kvm_entry: vcpu 0
 qemu-system-x86-4321  [001]   150.002196: kvm_exit: reason
IO_INSTRUCTION rip 0x3
 qemu-system-x86-4321  [001]   150.002197: kvm_pio: pio_read at 0x70
size 1 count 1
 qemu-system-x86-4321  [001]   150.002205: kvm_entry: vcpu 0
 qemu-system-x86-4321  [001]   150.002210: kvm_exit: reason
IO_INSTRUCTION rip 0x5
 qemu-system-x86-4321  [001]   150.002213: kvm_emulate_insn: f0000:5:
e4 71 (real)
 qemu-system-x86-4321  [001]   150.002215: kvm_pio: pio_write at 0x71
size 1 count 1
 qemu-system-x86-4321  [001]   150.002223: kvm_entry: vcpu 0
 qemu-system-x86-4321  [001]   150.002228: kvm_exit: reason
EXCEPTION_NMI rip 0x18
 qemu-system-x86-4321  [001]   150.002229: kvm_page_fault: address
ffff error_code f
 qemu-system-x86-4321  [001]   150.002270: kvm_entry: vcpu 0
 qemu-system-x86-4321  [001]   150.002276: kvm_exit: reason
EXCEPTION_NMI rip 0x1a
 qemu-system-x86-4321  [001]   150.002277: kvm_page_fault: address
d4dc error_code f
 qemu-system-x86-4321  [001]   150.002284: kvm_entry: vcpu 0
 qemu-system-x86-4321  [001]   150.002289: kvm_exit: reason
EXCEPTION_NMI rip 0x1d
 qemu-system-x86-4321  [001]   150.002292: kvm_emulate_insn: f0000:1d:
0f 00 (real)
 qemu-system-x86-4321  [001]   150.002294: kvm_inj_exception: #UD (0x0)
 qemu-system-x86-4321  [001]   150.002296: kvm_entry: vcpu 0
 qemu-system-x86-4321  [001]   150.002301: kvm_exit: reason
EXCEPTION_NMI rip 0x1d
 qemu-system-x86-4321  [001]   150.002302: kvm_page_fault: address 18
error_code 9
 qemu-system-x86-4321  [001]   150.002306: kvm_inj_virq: irq 6
 qemu-system-x86-4321  [001]   150.002311: kvm_entry: vcpu 0
 qemu-system-x86-4321  [001]   150.002315: kvm_exit: reason
EXCEPTION_NMI rip 0x1d
 qemu-system-x86-4321  [001]   150.002318: kvm_emulate_insn: f0000:1d:
0f 00 (real)

In the other instance the guest seems to jump to nowhere after
successfully running the BIOS, the emulator then seems to emulate
garbage. Here is the relevant part of the trace:

qemu-system-x86-4327  [001]   169.394467: kvm_exit: reason
EXCEPTION_NMI rip 0x7e1f
 qemu-system-x86-4327  [001]   169.394467: kvm_page_fault: address 4c
error_code 9
 qemu-system-x86-4327  [001]   169.394470: kvm_inj_virq: irq 19
 qemu-system-x86-4327  [001]   169.394475: kvm_entry: vcpu 0
 qemu-system-x86-4327  [001]   169.394477: kvm_exit: reason
EXCEPTION_NMI rip 0x7e1f
 qemu-system-x86-4327  [001]   169.394478: kvm_page_fault: address
f7e1f error_code 1d
 qemu-system-x86-4327  [001]   169.394480: kvm_entry: vcpu 0
 qemu-system-x86-4327  [001]   169.394482: kvm_exit: reason
EXCEPTION_NMI rip 0x7e4d
 qemu-system-x86-4327  [001]   169.394482: kvm_page_fault: address
38e0 error_code f
 qemu-system-x86-4327  [001]   169.394496: kvm_entry: vcpu 0
 qemu-system-x86-4327  [001]   169.394498: kvm_exit: reason
EXCEPTION_NMI rip 0x8028
 qemu-system-x86-4327  [001]   169.394499: kvm_page_fault: address
f8028 error_code 1d
 qemu-system-x86-4327  [001]   169.394500: kvm_entry: vcpu 0
 qemu-system-x86-4327  [001]   169.394502: kvm_exit: reason
EXCEPTION_NMI rip 0x8034
 qemu-system-x86-4327  [001]   169.394505: kvm_emulate_insn:
f0000:8034: 66 c3 (real)
 qemu-system-x86-4327  [001]   169.394509: kvm_entry: vcpu 0
 qemu-system-x86-4327  [001]   169.394511: kvm_exit: reason
EXCEPTION_NMI rip 0x44bf8
 qemu-system-x86-4327  [001]   169.394516: kvm_emulate_insn:
f0000:44bf8: 00 00 (real)
 qemu-system-x86-4327  [001]   169.394519: kvm_entry: vcpu 0
 qemu-system-x86-4327  [001]   169.394521: kvm_exit: reason
EXCEPTION_NMI rip 0x44bfa
 qemu-system-x86-4327  [001]   169.394522: kvm_emulate_insn:
f0000:44bfa: 00 00 (real)
 qemu-system-x86-4327  [001]   169.394523: kvm_entry: vcpu 0
 qemu-system-x86-4327  [001]   169.394525: kvm_exit: reason
EXCEPTION_NMI rip 0x44bfc
 qemu-system-x86-4327  [001]   169.394526: kvm_emulate_insn:
f0000:44bfc: 00 00 (real)
 qemu-system-x86-4327  [001]   169.394527: kvm_entry: vcpu 0
 qemu-system-x86-4327  [001]   169.394529: kvm_exit: reason
EXCEPTION_NMI rip 0x44bfe
 qemu-system-x86-4327  [001]   169.394530: kvm_emulate_insn:
f0000:44bfe: 00 00 (real)
 qemu-system-x86-4327  [001]   169.394531: kvm_entry: vcpu 0
 qemu-system-x86-4327  [001]   169.394533: kvm_exit: reason
EXCEPTION_NMI rip 0x44c00
 qemu-system-x86-4327  [001]   169.394534: kvm_emulate_insn:
f0000:44c00: 00 00 (real)

I am not sure if this is correct behaviour at all.

Regards,
Mohammed

Re: [RFC PATCH 3/3] VMX: Emulated real mode interrupt injection

[Mohammed Gamal]

On Tue, Aug 10, 2010 at 6:03 AM, Avi Kivity <avi@redhat.com> wrote:
>  On 08/08/2010 03:24 PM, Mohammed Gamal wrote:
>>
>> Signed-off-by: Mohammed Gamal<m.gamal005@gmail.com>
>> ---
>>  arch/x86/kvm/vmx.c |   11 +----------
>>  1 files changed, 1 insertions(+), 10 deletions(-)
>>
>> diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c
>> index 652d317..d6cb7eb 100644
>> --- a/arch/x86/kvm/vmx.c
>> +++ b/arch/x86/kvm/vmx.c
>> @@ -2838,16 +2838,7 @@ static void vmx_inject_irq(struct kvm_vcpu *vcpu)
>>
>>        ++vcpu->stat.irq_injections;
>>        if (vmx->rmode.vm86_active) {
>> -               vmx->rmode.irq.pending = true;
>> -               vmx->rmode.irq.vector = irq;
>> -               vmx->rmode.irq.rip = kvm_rip_read(vcpu);
>> -               if (vcpu->arch.interrupt.soft)
>> -                       vmx->rmode.irq.rip +=
>> -                               vmx->vcpu.arch.event_exit_inst_len;
>
> This has to be covered somehow.  Not sure exactly - probably keep the same
> code.  Jan?
>
>> -               vmcs_write32(VM_ENTRY_INTR_INFO_FIELD,
>> -                            irq | INTR_TYPE_SOFT_INTR |
>> INTR_INFO_VALID_MASK);
>> -               vmcs_write32(VM_ENTRY_INSTRUCTION_LEN, 1);
>> -               kvm_rip_write(vcpu, vmx->rmode.irq.rip - 1);
>> +               inject_realmode_interrupt(vcpu, irq);
>>                return;
>>        }
>
> Error checks?
We return anyway. Is there somwhere in the vmcs where an error is
indicated (e.g. setting some flag or variable)? Or should we probably
let vmx_inject_irq() return an integer return value?

>
> Need to do same to vmx_inject_nmi().
>
> fixup_rmode_irq() just became dead code, you can remove it.  Also remove the
> entire vmx->rmode.irq thing.
>
> --
> I have a truly marvellous patch that fixes the bug which this
> signature is too narrow to contain.
>
>
dic

Re: [RFC PATCH 3/3] VMX: Emulated real mode interrupt injection

[Avi Kivity]

  On 08/10/2010 01:13 PM, Mohammed Gamal wrote:
>
>>
>>> -               vmcs_write32(VM_ENTRY_INTR_INFO_FIELD,
>>> -                            irq | INTR_TYPE_SOFT_INTR |
>>> INTR_INFO_VALID_MASK);
>>> -               vmcs_write32(VM_ENTRY_INSTRUCTION_LEN, 1);
>>> -               kvm_rip_write(vcpu, vmx->rmode.irq.rip - 1);
>>> +               inject_realmode_interrupt(vcpu, irq);
>>>                 return;
>>>         }
>> Error checks?
> We return anyway. Is there somwhere in the vmcs where an error is
> indicated (e.g. setting some flag or variable)? Or should we probably
> let vmx_inject_irq() return an integer return value?

That's too complicated for now.  I guess you can KVM_REQ_TRIPLE_FAULT, 
and the guest will reset.

-- 
I have a truly marvellous patch that fixes the bug which this
signature is too narrow to contain.

Re: [RFC PATCH 0/3] Real mode interrupt injection

[Avi Kivity]

  On 08/10/2010 01:06 PM, Mohammed Gamal wrote:
> On 8/10/10, Avi Kivity<avi@redhat.com>  wrote:
>>    On 08/08/2010 03:24 PM, Mohammed Gamal wrote:
>>> This patch introduces real mode interrupt injection for VMX.
>>> It currently invokes the x86 emulator to emulate interrupts
>>> instead of manually setting VMX controls.
>>>
>>> Needless to say, this is not meant for merging in its current state.
>>> The emulator still needs some more work to get this completely
>>> operational.
>> Well, what happens when you run with it?
> The guest fails at two instances. First it sometimes encounters a
> group 7 instruction (0x0f 0x00), which the emulator doesn't emulate.
> Here is the relevant part of the trace:
>
> qemu-system-x86-4321  [001]   150.002191: kvm_entry: vcpu 0
>   qemu-system-x86-4321  [001]   150.002196: kvm_exit: reason
> IO_INSTRUCTION rip 0x3
>   qemu-system-x86-4321  [001]   150.002197: kvm_pio: pio_read at 0x70
> size 1 count 1
>   qemu-system-x86-4321  [001]   150.002205: kvm_entry: vcpu 0
>   qemu-system-x86-4321  [001]   150.002210: kvm_exit: reason
> IO_INSTRUCTION rip 0x5
>   qemu-system-x86-4321  [001]   150.002213: kvm_emulate_insn: f0000:5:
> e4 71 (real)
>   qemu-system-x86-4321  [001]   150.002215: kvm_pio: pio_write at 0x71
> size 1 count 1
>   qemu-system-x86-4321  [001]   150.002223: kvm_entry: vcpu 0
>   qemu-system-x86-4321  [001]   150.002228: kvm_exit: reason
> EXCEPTION_NMI rip 0x18
>   qemu-system-x86-4321  [001]   150.002229: kvm_page_fault: address
> ffff error_code f
>   qemu-system-x86-4321  [001]   150.002270: kvm_entry: vcpu 0
>   qemu-system-x86-4321  [001]   150.002276: kvm_exit: reason
> EXCEPTION_NMI rip 0x1a
>   qemu-system-x86-4321  [001]   150.002277: kvm_page_fault: address
> d4dc error_code f
>   qemu-system-x86-4321  [001]   150.002284: kvm_entry: vcpu 0
>   qemu-system-x86-4321  [001]   150.002289: kvm_exit: reason
> EXCEPTION_NMI rip 0x1d
>   qemu-system-x86-4321  [001]   150.002292: kvm_emulate_insn: f0000:1d:
> 0f 00 (real)

Could be a real instruction - we don't emulate all of group 7, and 
they're useful.

Can you put your bios.bin somewhere?  We can see what's there.

I'll look at the second case later.

-- 
I have a truly marvellous patch that fixes the bug which this
signature is too narrow to contain.

Re: [RFC PATCH 0/3] Real mode interrupt injection

[Mohammed Gamal]

On Wed, Aug 11, 2010 at 2:02 AM, Avi Kivity <avi@redhat.com> wrote:
>  On 08/10/2010 01:06 PM, Mohammed Gamal wrote:
>>
>> On 8/10/10, Avi Kivity<avi@redhat.com>  wrote:
>>>
>>>   On 08/08/2010 03:24 PM, Mohammed Gamal wrote:
>>>>
>>>> This patch introduces real mode interrupt injection for VMX.
>>>> It currently invokes the x86 emulator to emulate interrupts
>>>> instead of manually setting VMX controls.
>>>>
>>>> Needless to say, this is not meant for merging in its current state.
>>>> The emulator still needs some more work to get this completely
>>>> operational.
>>>
>>> Well, what happens when you run with it?
>>
>> The guest fails at two instances. First it sometimes encounters a
>> group 7 instruction (0x0f 0x00), which the emulator doesn't emulate.
>> Here is the relevant part of the trace:
>>
>> qemu-system-x86-4321  [001]   150.002191: kvm_entry: vcpu 0
>>  qemu-system-x86-4321  [001]   150.002196: kvm_exit: reason
>> IO_INSTRUCTION rip 0x3
>>  qemu-system-x86-4321  [001]   150.002197: kvm_pio: pio_read at 0x70
>> size 1 count 1
>>  qemu-system-x86-4321  [001]   150.002205: kvm_entry: vcpu 0
>>  qemu-system-x86-4321  [001]   150.002210: kvm_exit: reason
>> IO_INSTRUCTION rip 0x5
>>  qemu-system-x86-4321  [001]   150.002213: kvm_emulate_insn: f0000:5:
>> e4 71 (real)
>>  qemu-system-x86-4321  [001]   150.002215: kvm_pio: pio_write at 0x71
>> size 1 count 1
>>  qemu-system-x86-4321  [001]   150.002223: kvm_entry: vcpu 0
>>  qemu-system-x86-4321  [001]   150.002228: kvm_exit: reason
>> EXCEPTION_NMI rip 0x18
>>  qemu-system-x86-4321  [001]   150.002229: kvm_page_fault: address
>> ffff error_code f
>>  qemu-system-x86-4321  [001]   150.002270: kvm_entry: vcpu 0
>>  qemu-system-x86-4321  [001]   150.002276: kvm_exit: reason
>> EXCEPTION_NMI rip 0x1a
>>  qemu-system-x86-4321  [001]   150.002277: kvm_page_fault: address
>> d4dc error_code f
>>  qemu-system-x86-4321  [001]   150.002284: kvm_entry: vcpu 0
>>  qemu-system-x86-4321  [001]   150.002289: kvm_exit: reason
>> EXCEPTION_NMI rip 0x1d
>>  qemu-system-x86-4321  [001]   150.002292: kvm_emulate_insn: f0000:1d:
>> 0f 00 (real)
>
> Could be a real instruction - we don't emulate all of group 7, and they're
> useful.
In fact, we don't emulate group 7 at all.

>
> Can you put your bios.bin somewhere?  We can see what's there.
>
> I'll look at the second case later.
>
> --
> I have a truly marvellous patch that fixes the bug which this
> signature is too narrow to contain.
>
>

Re: [RFC PATCH 0/3] Real mode interrupt injection

[Avi Kivity]

  On 08/10/2010 10:01 PM, Mohammed Gamal wrote:
> On Wed, Aug 11, 2010 at 4:19 AM, Mohammed Gamal<m.gamal005@gmail.com>  wrote:
>> On Wed, Aug 11, 2010 at 2:02 AM, Avi Kivity<avi@redhat.com>  wrote:
>>>   On 08/10/2010 01:06 PM, Mohammed Gamal wrote:
>>>> On 8/10/10, Avi Kivity<avi@redhat.com>    wrote:
>>>>>    On 08/08/2010 03:24 PM, Mohammed Gamal wrote:
>>>>>> This patch introduces real mode interrupt injection for VMX.
>>>>>> It currently invokes the x86 emulator to emulate interrupts
>>>>>> instead of manually setting VMX controls.
>>>>>>
>>>>>> Needless to say, this is not meant for merging in its current state.
>>>>>> The emulator still needs some more work to get this completely
>>>>>> operational.
>>>>> Well, what happens when you run with it?
>>>> The guest fails at two instances. First it sometimes encounters a
>>>> group 7 instruction (0x0f 0x00), which the emulator doesn't emulate.
>>>> Here is the relevant part of the trace:
>>>>
>>>> qemu-system-x86-4321  [001]   150.002191: kvm_entry: vcpu 0
>>>>   qemu-system-x86-4321  [001]   150.002196: kvm_exit: reason
>>>> IO_INSTRUCTION rip 0x3
>>>>   qemu-system-x86-4321  [001]   150.002197: kvm_pio: pio_read at 0x70
>>>> size 1 count 1
>>>>   qemu-system-x86-4321  [001]   150.002205: kvm_entry: vcpu 0
>>>>   qemu-system-x86-4321  [001]   150.002210: kvm_exit: reason
>>>> IO_INSTRUCTION rip 0x5
>>>>   qemu-system-x86-4321  [001]   150.002213: kvm_emulate_insn: f0000:5:
>>>> e4 71 (real)
>>>>   qemu-system-x86-4321  [001]   150.002215: kvm_pio: pio_write at 0x71
>>>> size 1 count 1
>>>>   qemu-system-x86-4321  [001]   150.002223: kvm_entry: vcpu 0
>>>>   qemu-system-x86-4321  [001]   150.002228: kvm_exit: reason
>>>> EXCEPTION_NMI rip 0x18
>>>>   qemu-system-x86-4321  [001]   150.002229: kvm_page_fault: address
>>>> ffff error_code f
>>>>   qemu-system-x86-4321  [001]   150.002270: kvm_entry: vcpu 0
>>>>   qemu-system-x86-4321  [001]   150.002276: kvm_exit: reason
>>>> EXCEPTION_NMI rip 0x1a
>>>>   qemu-system-x86-4321  [001]   150.002277: kvm_page_fault: address
>>>> d4dc error_code f
>>>>   qemu-system-x86-4321  [001]   150.002284: kvm_entry: vcpu 0
>>>>   qemu-system-x86-4321  [001]   150.002289: kvm_exit: reason
>>>> EXCEPTION_NMI rip 0x1d
>>>>   qemu-system-x86-4321  [001]   150.002292: kvm_emulate_insn: f0000:1d:
>>>> 0f 00 (real)
>>> Could be a real instruction - we don't emulate all of group 7, and they're
>>> useful.
>> In fact, we don't emulate group 7 at all.
>>

Right.  Well, turns out it isn't a real instruction:

    efffd:       e4 71                   in     $0x71,%al
    effff:       88 c2                   mov    %al,%dl
    f0001:       b0 b1                   mov    $0xb1,%al
    f0003:       e6 70                   out    %al,$0x70
    f0005:       e4 71                   in     $0x71,%al
    f0007:       0f b6 c0                movzbl %al,%eax
    f000a:       c1 e0 12                shl    $0x12,%eax
    f000d:       0f b6 d2                movzbl %dl,%edx
    f0010:       c1 e2 0a                shl    $0xa,%edx
    f0013:       09 d0                   or     %edx,%eax
    f0015:       05 00 00 10 00          add    $0x100000,%eax
    f001a:       a3 dc d4 0f 00          mov    %eax,0xfd4dc

This is 32-bit code, yet from the trace:

  qemu-system-x86-4321  [001]   150.002276: kvm_exit: reason EXCEPTION_NMI rip 0x1a
  qemu-system-x86-4321  [001]   150.002277: kvm_page_fault: address d4dc error_code f

The address is trimmed, so kvm thinks we're in real mode!  The '0f 00' 
is just leftover bytes from the instruction.

We'll need earlier traces to find how the mixup happened.


    f001f:       6a 01                   push   $0x1
    f0021:       31 d2                   xor    %edx,%edx
    f0023:       52                      push   %edx
    f0024:       50                      push   %eax
    f0025:       31 c0                   xor    %eax,%eax
    f0027:       31 d2                   xor    %edx,%edx


-- 
I have a truly marvellous patch that fixes the bug which this
signature is too narrow to contain.

Re: [RFC PATCH 0/3] Real mode interrupt injection

[Avi Kivity]

  On 08/10/2010 01:06 PM, Mohammed Gamal wrote:
>
> In the other instance the guest seems to jump to nowhere after
> successfully running the BIOS, the emulator then seems to emulate
> garbage. Here is the relevant part of the trace:
>
> qemu-system-x86-4327  [001]   169.394467: kvm_exit: reason
> EXCEPTION_NMI rip 0x7e1f
>   qemu-system-x86-4327  [001]   169.394467: kvm_page_fault: address 4c
> error_code 9

Here, the guest tried to execute INT 13, but exited since the IDT was 
paged out.

>   qemu-system-x86-4327  [001]   169.394470: kvm_inj_virq: irq 19

vmx_complete_interrupts() recovered the interrupt (0x13 == 19) and is 
reinjecting it

>   qemu-system-x86-4327  [001]   169.394475: kvm_entry: vcpu 0
>   qemu-system-x86-4327  [001]   169.394477: kvm_exit: reason
> EXCEPTION_NMI rip 0x7e1f
>   qemu-system-x86-4327  [001]   169.394478: kvm_page_fault: address
> f7e1f error_code 1d

f7e1f seems to be in the middle of some instruction:

    f7e03:       26 67 8b 28             addr32 mov %es:(%eax),%bp
    f7e07:       66 0f b7 ed             movzwl %bp,%ebp
    f7e0b:       66 83 c1 0c             add    $0xc,%ecx
    f7e0f:       66 89 c8                mov    %ecx,%eax
    f7e12:       66 c1 e8 04             shr    $0x4,%eax
    f7e16:       8e c0                   mov    %ax,%es
    f7e18:       66 83 e1 0f             and    $0xf,%ecx
    f7e1c:       26 67 66 8b 01          addr32 mov %es:(%ecx),%eax
    f7e21:       67 66 89 44 24 14       addr32 mov %eax,0x14(%esp)
    f7e27:       66 89 ee                mov    %ebp,%esi
    f7e2a:       66 0f af f2             imul   %edx,%esi
    f7e2e:       66 01 c6                add    %eax,%esi
    f7e31:       8c d0                   mov    %ss,%ax
    f7e33:       8e c0                   mov    %ax,%es
    f7e35:       66 89 f2                mov    %esi,%edx
    f7e38:       66 c1 ea 04             shr    $0x4,%edx
    f7e3c:       66 83 e6 0f             and    $0xf,%esi
    f7e40:       66 89 e9                mov    %ebp,%ecx
    f7e43:       67 66 8b 7c 24 18       addr32 mov 0x18(%esp),%edi

So, looks like the reinjection failed.  Please add trace_printk()s so we 
can see what values the emulator read from the IDT (and from what 
address it read them).

-- 
I have a truly marvellous patch that fixes the bug which this
signature is too narrow to contain.

Re: [RFC PATCH 0/3] Real mode interrupt injection

[Avi Kivity]

  On 08/11/2010 07:04 AM, Avi Kivity wrote:
>
> This is 32-bit code, yet from the trace:
>
>  qemu-system-x86-4321  [001]   150.002276: kvm_exit: reason 
> EXCEPTION_NMI rip 0x1a
>  qemu-system-x86-4321  [001]   150.002277: kvm_page_fault: address 
> d4dc error_code f
>
> The address is trimmed, so kvm thinks we're in real mode!  The '0f 00' 
> is just leftover bytes from the instruction.
>
> We'll need earlier traces to find how the mixup happened.
>

I think I have it:

static int load_segment_descriptor(struct x86_emulate_ctxt *ctxt,
                    struct x86_emulate_ops *ops,
                    u16 selector, int seg)
{
     struct desc_struct seg_desc;
     u8 dpl, rpl, cpl;
     unsigned err_vec = GP_VECTOR;
     u32 err_code = 0;
     bool null_selector = !(selector & ~0x3); /* 0000-0003 are null */
     int ret;

     memset(&seg_desc, 0, sizeof seg_desc);

     if ((seg <= VCPU_SREG_GS && ctxt->mode == X86EMUL_MODE_VM86)
         || ctxt->mode == X86EMUL_MODE_REAL) {
         /* set real mode segment descriptor */
         set_desc_base(&seg_desc, selector << 4);
         set_desc_limit(&seg_desc, 0xffff);
         seg_desc.type = 3;
         seg_desc.p = 1;
         seg_desc.s = 1;
         goto load;
     }


seg_desc is not initialized, so seg_desc.d gets a random value.  This 
selects 32-bit or 16-bit mode, and explains the confusion.  Likely the 
other case as well.

-- 
I have a truly marvellous patch that fixes the bug which this
signature is too narrow to contain.

Re: [RFC PATCH 0/3] Real mode interrupt injection

[Avi Kivity]

  On 08/11/2010 07:20 AM, Avi Kivity wrote:
>  On 08/11/2010 07:04 AM, Avi Kivity wrote:
>>
>> This is 32-bit code, yet from the trace:
>>
>>  qemu-system-x86-4321  [001]   150.002276: kvm_exit: reason 
>> EXCEPTION_NMI rip 0x1a
>>  qemu-system-x86-4321  [001]   150.002277: kvm_page_fault: address 
>> d4dc error_code f
>>
>> The address is trimmed, so kvm thinks we're in real mode!  The '0f 
>> 00' is just leftover bytes from the instruction.
>>
>> We'll need earlier traces to find how the mixup happened.
>>
>
> I think I have it:
>
> static int load_segment_descriptor(struct x86_emulate_ctxt *ctxt,
>                    struct x86_emulate_ops *ops,
>                    u16 selector, int seg)
> {
>     struct desc_struct seg_desc;
>     u8 dpl, rpl, cpl;
>     unsigned err_vec = GP_VECTOR;
>     u32 err_code = 0;
>     bool null_selector = !(selector & ~0x3); /* 0000-0003 are null */
>     int ret;
>
>     memset(&seg_desc, 0, sizeof seg_desc);
>
>     if ((seg <= VCPU_SREG_GS && ctxt->mode == X86EMUL_MODE_VM86)
>         || ctxt->mode == X86EMUL_MODE_REAL) {
>         /* set real mode segment descriptor */
>         set_desc_base(&seg_desc, selector << 4);
>         set_desc_limit(&seg_desc, 0xffff);
>         seg_desc.type = 3;
>         seg_desc.p = 1;
>         seg_desc.s = 1;
>         goto load;
>     }
>
>
> seg_desc is not initialized, so seg_desc.d gets a random value.  This 
> selects 32-bit or 16-bit mode, and explains the confusion.  Likely the 
> other case as well.
>

It is initialized, I even quoted the memset().  So the problem is 
somewhere else, we'll need more traces to find out.

-- 
I have a truly marvellous patch that fixes the bug which this
signature is too narrow to contain.

Re: [RFC PATCH 0/3] Real mode interrupt injection

[Mohammed Gamal]

On Wed, Aug 11, 2010 at 3:08 PM, Avi Kivity <avi@redhat.com> wrote:
>  On 08/11/2010 07:20 AM, Avi Kivity wrote:
>>
>>  On 08/11/2010 07:04 AM, Avi Kivity wrote:
>>>
>>> This is 32-bit code, yet from the trace:
>>>
>>>  qemu-system-x86-4321  [001]   150.002276: kvm_exit: reason EXCEPTION_NMI
>>> rip 0x1a
>>>  qemu-system-x86-4321  [001]   150.002277: kvm_page_fault: address d4dc
>>> error_code f
>>>
>>> The address is trimmed, so kvm thinks we're in real mode!  The '0f 00' is
>>> just leftover bytes from the instruction.
>>>
>>> We'll need earlier traces to find how the mixup happened.
>>>
>>
>> I think I have it:
>>
>> static int load_segment_descriptor(struct x86_emulate_ctxt *ctxt,
>>                   struct x86_emulate_ops *ops,
>>                   u16 selector, int seg)
>> {
>>    struct desc_struct seg_desc;
>>    u8 dpl, rpl, cpl;
>>    unsigned err_vec = GP_VECTOR;
>>    u32 err_code = 0;
>>    bool null_selector = !(selector & ~0x3); /* 0000-0003 are null */
>>    int ret;
>>
>>    memset(&seg_desc, 0, sizeof seg_desc);
>>
>>    if ((seg <= VCPU_SREG_GS && ctxt->mode == X86EMUL_MODE_VM86)
>>        || ctxt->mode == X86EMUL_MODE_REAL) {
>>        /* set real mode segment descriptor */
>>        set_desc_base(&seg_desc, selector << 4);
>>        set_desc_limit(&seg_desc, 0xffff);
>>        seg_desc.type = 3;
>>        seg_desc.p = 1;
>>        seg_desc.s = 1;
>>        goto load;
>>    }
>>
>>
>> seg_desc is not initialized, so seg_desc.d gets a random value.  This
>> selects 32-bit or 16-bit mode, and explains the confusion.  Likely the other
>> case as well.
>>
>
> It is initialized, I even quoted the memset().  So the problem is somewhere
> else, we'll need more traces to find out.
>
I was playing around with the non-atomic-injection branch. I decided
to use e_i_g_s=1, and it's worth noting that I never experienced these
faults with the switch enabled.
> --
> I have a truly marvellous patch that fixes the bug which this
> signature is too narrow to contain.
>
>

Re: [RFC PATCH 0/3] Real mode interrupt injection

[Avi Kivity]

  On 08/11/2010 07:22 PM, Mohammed Gamal wrote:
> On Wed, Aug 11, 2010 at 3:08 PM, Avi Kivity<avi@redhat.com>  wrote:
>>   On 08/11/2010 07:20 AM, Avi Kivity wrote:
>>>   On 08/11/2010 07:04 AM, Avi Kivity wrote:
>>>> This is 32-bit code, yet from the trace:
>>>>
>>>>   qemu-system-x86-4321  [001]   150.002276: kvm_exit: reason EXCEPTION_NMI
>>>> rip 0x1a
>>>>   qemu-system-x86-4321  [001]   150.002277: kvm_page_fault: address d4dc
>>>> error_code f
>>>>
>>>> The address is trimmed, so kvm thinks we're in real mode!  The '0f 00' is
>>>> just leftover bytes from the instruction.
>>>>
>>>> We'll need earlier traces to find how the mixup happened.
>>>>
>>> I think I have it:
>>>
>>> static int load_segment_descriptor(struct x86_emulate_ctxt *ctxt,
>>>                    struct x86_emulate_ops *ops,
>>>                    u16 selector, int seg)
>>> {
>>>     struct desc_struct seg_desc;
>>>     u8 dpl, rpl, cpl;
>>>     unsigned err_vec = GP_VECTOR;
>>>     u32 err_code = 0;
>>>     bool null_selector = !(selector&  ~0x3); /* 0000-0003 are null */
>>>     int ret;
>>>
>>>     memset(&seg_desc, 0, sizeof seg_desc);
>>>
>>>     if ((seg<= VCPU_SREG_GS&&  ctxt->mode == X86EMUL_MODE_VM86)
>>>         || ctxt->mode == X86EMUL_MODE_REAL) {
>>>         /* set real mode segment descriptor */
>>>         set_desc_base(&seg_desc, selector<<  4);
>>>         set_desc_limit(&seg_desc, 0xffff);
>>>         seg_desc.type = 3;
>>>         seg_desc.p = 1;
>>>         seg_desc.s = 1;
>>>         goto load;
>>>     }
>>>
>>>
>>> seg_desc is not initialized, so seg_desc.d gets a random value.  This
>>> selects 32-bit or 16-bit mode, and explains the confusion.  Likely the other
>>> case as well.
>>>
>> It is initialized, I even quoted the memset().  So the problem is somewhere
>> else, we'll need more traces to find out.
>>
> I was playing around with the non-atomic-injection branch. I decided
> to use e_i_g_s=1, and it's worth noting that I never experienced these
> faults with the switch enabled.

Well, it will be interesting to see what happened.  Can you post a 
complete trace (from bootup) somewhere?


-- 
I have a truly marvellous patch that fixes the bug which this
signature is too narrow to contain.

Re: [RFC PATCH 0/3] Real mode interrupt injection

[Mohammed Gamal]

On 8/12/10, Avi Kivity <avi@redhat.com> wrote:
>   On 08/11/2010 07:22 PM, Mohammed Gamal wrote:
>> On Wed, Aug 11, 2010 at 3:08 PM, Avi Kivity<avi@redhat.com>  wrote:
>>>   On 08/11/2010 07:20 AM, Avi Kivity wrote:
>>>>   On 08/11/2010 07:04 AM, Avi Kivity wrote:
>>>>> This is 32-bit code, yet from the trace:
>>>>>
>>>>>   qemu-system-x86-4321  [001]   150.002276: kvm_exit: reason
>>>>> EXCEPTION_NMI
>>>>> rip 0x1a
>>>>>   qemu-system-x86-4321  [001]   150.002277: kvm_page_fault: address
>>>>> d4dc
>>>>> error_code f
>>>>>
>>>>> The address is trimmed, so kvm thinks we're in real mode!  The '0f 00'
>>>>> is
>>>>> just leftover bytes from the instruction.
>>>>>
>>>>> We'll need earlier traces to find how the mixup happened.
>>>>>
>>>> I think I have it:
>>>>
>>>> static int load_segment_descriptor(struct x86_emulate_ctxt *ctxt,
>>>>                    struct x86_emulate_ops *ops,
>>>>                    u16 selector, int seg)
>>>> {
>>>>     struct desc_struct seg_desc;
>>>>     u8 dpl, rpl, cpl;
>>>>     unsigned err_vec = GP_VECTOR;
>>>>     u32 err_code = 0;
>>>>     bool null_selector = !(selector&  ~0x3); /* 0000-0003 are null */
>>>>     int ret;
>>>>
>>>>     memset(&seg_desc, 0, sizeof seg_desc);
>>>>
>>>>     if ((seg<= VCPU_SREG_GS&&  ctxt->mode == X86EMUL_MODE_VM86)
>>>>         || ctxt->mode == X86EMUL_MODE_REAL) {
>>>>         /* set real mode segment descriptor */
>>>>         set_desc_base(&seg_desc, selector<<  4);
>>>>         set_desc_limit(&seg_desc, 0xffff);
>>>>         seg_desc.type = 3;
>>>>         seg_desc.p = 1;
>>>>         seg_desc.s = 1;
>>>>         goto load;
>>>>     }
>>>>
>>>>
>>>> seg_desc is not initialized, so seg_desc.d gets a random value.  This
>>>> selects 32-bit or 16-bit mode, and explains the confusion.  Likely the
>>>> other
>>>> case as well.
>>>>
>>> It is initialized, I even quoted the memset().  So the problem is
>>> somewhere
>>> else, we'll need more traces to find out.
>>>
>> I was playing around with the non-atomic-injection branch. I decided
>> to use e_i_g_s=1, and it's worth noting that I never experienced these
>> faults with the switch enabled.
Hate to spoil it. I did experience the faults again with e_i_g_s=1,
although much less frequently.

What is rather really strange, is that I could get a Linux guest to
boot up completely both with e_i_g_s=1 and without it with the real
mode interrupt patch enabled. It looks to me like the problem mainly
happens when the BIOS tranfers control to the boot loader. Other
guests usually fail.

Would you like me to attach a trace?
>
> Well, it will be interesting to see what happened.  Can you post a
> complete trace (from bootup) somewhere?
>
>
> --
> I have a truly marvellous patch that fixes the bug which this
> signature is too narrow to contain.
>
>

Re: [RFC PATCH 0/3] Real mode interrupt injection

[Avi Kivity]

  On 08/12/2010 04:07 AM, Mohammed Gamal wrote:
>>>
>>> I was playing around with the non-atomic-injection branch. I decided
>>> to use e_i_g_s=1, and it's worth noting that I never experienced these
>>> faults with the switch enabled.
> Hate to spoil it. I did experience the faults again with e_i_g_s=1,
> although much less frequently.
>
> What is rather really strange, is that I could get a Linux guest to
> boot up completely both with e_i_g_s=1 and without it with the real
> mode interrupt patch enabled. It looks to me like the problem mainly
> happens when the BIOS tranfers control to the boot loader. Other
> guests usually fail.
>
> Would you like me to attach a trace?


It will be much too big, upload it somewhere or send it to be privately.

But, use the code with the interrupt injection setup fixed (see my 
comment to patch 2).


-- 
error compiling committee.c: too many arguments to function

Re: [RFC PATCH 0/3] Real mode interrupt injection

[Mohammed Gamal]

On Sun, Aug 15, 2010 at 3:23 PM, Avi Kivity <avi@redhat.com> wrote:
>  On 08/12/2010 04:07 AM, Mohammed Gamal wrote:
>>>>
>>>> I was playing around with the non-atomic-injection branch. I decided
>>>> to use e_i_g_s=1, and it's worth noting that I never experienced these
>>>> faults with the switch enabled.
>>
>> Hate to spoil it. I did experience the faults again with e_i_g_s=1,
>> although much less frequently.
>>
>> What is rather really strange, is that I could get a Linux guest to
>> boot up completely both with e_i_g_s=1 and without it with the real
>> mode interrupt patch enabled. It looks to me like the problem mainly
>> happens when the BIOS tranfers control to the boot loader. Other
>> guests usually fail.
>>
>> Would you like me to attach a trace?
>
>
> It will be much too big, upload it somewhere or send it to be privately.
>
> But, use the code with the interrupt injection setup fixed (see my comment
> to patch 2).
>
Please take a look at my latest patch series. We can take the discussion there.
>
> --
> error compiling committee.c: too many arguments to function
>
>