RE: single-step issue

RE: single-step issue

[Liu Yu-B13201]

 

> -----Original Message-----
> From: Hollis Blanchard [mailto:hollisb@us.ibm.com] 
> Sent: Wednesday, May 06, 2009 3:21 AM
> To: Liu Yu-B13201
> Subject: single-step issue
> 
> The e500 user manual (section 5.11.1, e500 Exception Priorities) says
> that Debug interrupts have higher priority than Program interrupts.
> However, that only matters if both exceptions exist at the same time,
> which I don't think is true in this case.
> 
> Ordinarily, the debug interrupt should occur *after* the 
> effects of the
> instruction have occurred. So I would expect that, when 
> single-stepping
> through a privileged guest instruction, you would get the program
> interrupt simply because the instruction can't be executed (therefore
> the debug interrupt couldn't possibly occur). That matches 
> the behavior
> you've seen, correct?
> 
> Solution: after emulate_instruction() in the Program handler succeeds,
> check vcpu->your_debug_flags to see if control should return 
> to qemu/gdb
> instead of back to the guest.
> 

Sorry for the late reply, I got some urgent work to deal with.

Yes. I should use CSRR0 to get next pc in single-step mode.
And it works well in normal case.

The problem I met is that when guest meet a privilege instructions,
The CSRR0 give out a strange address which should not be used to enter
into guest.

And I find it out now.
When guest meet a privilege instructions,
first the program interrupt occurs, but immediately after it, a debug
interrupt occurs,
and the CSRR0 is filled with the value of "kvmppc_booke_handlers +
ivor[IVOR6]" (kvmppc program interrupt handler),
which is indeed the next instruction of guest privilege instruction....

I guess the reason is that program interrupt doesn't clear MSR[DE] bit,
right?
If so, then we cannot check vcpu->debug_flags to see if control should
return to qemu/gdb.
Should we emulate privilege instructions in debug interrupt handler?

One thing I still don't get clear is that: does MSR[DE] is set while
running guest?
I see nowhere set it, but all kinds of debug interrupt seem could work.

RE: single-step issue

[Hollis Blanchard]

On Sun, 2009-05-10 at 15:07 +0800, Liu Yu-B13201 wrote:
> 
> > -----Original Message-----
> > From: Hollis Blanchard [mailto:hollisb@us.ibm.com] 
> > Sent: Wednesday, May 06, 2009 3:21 AM
> > To: Liu Yu-B13201
> > Subject: single-step issue
> > 
> > The e500 user manual (section 5.11.1, e500 Exception Priorities) says
> > that Debug interrupts have higher priority than Program interrupts.
> > However, that only matters if both exceptions exist at the same time,
> > which I don't think is true in this case.
> > 
> > Ordinarily, the debug interrupt should occur *after* the 
> > effects of the
> > instruction have occurred. So I would expect that, when 
> > single-stepping
> > through a privileged guest instruction, you would get the program
> > interrupt simply because the instruction can't be executed (therefore
> > the debug interrupt couldn't possibly occur). That matches 
> > the behavior
> > you've seen, correct?
> > 
> > Solution: after emulate_instruction() in the Program handler succeeds,
> > check vcpu->your_debug_flags to see if control should return 
> > to qemu/gdb
> > instead of back to the guest.
> > 
> 
> Sorry for the late reply, I got some urgent work to deal with.
> 
> Yes. I should use CSRR0 to get next pc in single-step mode.
> And it works well in normal case.
> 
> The problem I met is that when guest meet a privilege instructions,
> The CSRR0 give out a strange address which should not be used to enter
> into guest.
> 
> And I find it out now.
> When guest meet a privilege instructions,
> first the program interrupt occurs, but immediately after it, a debug
> interrupt occurs,
> and the CSRR0 is filled with the value of "kvmppc_booke_handlers +
> ivor[IVOR6]" (kvmppc program interrupt handler),
> which is indeed the next instruction of guest privilege instruction....

Nasty. I would like to add that this does *not* match the documented
behavior of debug interrupts I mentioned before, but there's not much we
can do about that now.

> I guess the reason is that program interrupt doesn't clear MSR[DE] bit,
> right?

I guess so.

> If so, then we cannot check vcpu->debug_flags to see if control should
> return to qemu/gdb.
> Should we emulate privilege instructions in debug interrupt handler?

You can't, because the debug handler doesn't know the address of the
instruction to emulate.

As a hack, the debug handler could see if CSRR0 matches one of the KVM
interrupt vectors, and if so set a flag and return to the original
handler. The normal handlers would then test and clear the flag to see
if they should emulate a debug event.

> One thing I still don't get clear is that: does MSR[DE] is set while
> running guest?
> I see nowhere set it, but all kinds of debug interrupt seem could work.

On 440, yes it is, but it's not heavily tested. 440 may have the same
issue.

In general, debugging with KVM on 440/e500 is a hack. Imagine setting a
guest breakpoint that happens to match the last "rfi" instruction in a
host -> guest switch. Since MSR[DE] only controls the *delivery* of the
interrupt, the breakpoint will trigger (but not be delivered) on every
context switch. Then, when landing in the guest (rfi set MSR[DE]=1), the
interrupt will be delivered. Even if the host handles the interrupt
correctly, the guest won't make forward progress, even though it's never
hit the breakpoint.

I don't think this problem should be an issue for single-stepping, but
it suggests we can't reliably use hardware's IAC/DAC/DVC/etc. We could
replace IAC with software breakpoints, and maybe DAC with page faults
(with high overhead), but there's no software emulation for DVC or e.g.
"branch taken" events that I can see.

-- 
Hollis Blanchard
IBM Linux Technology Center

RE: single-step issue

[Liu Yu-B13201]

 

> -----Original Message-----
> From: kvm-ppc-owner@vger.kernel.org 
> [mailto:kvm-ppc-owner@vger.kernel.org] On Behalf Of Hollis Blanchard
> Sent: Tuesday, May 12, 2009 1:40 AM
> To: Liu Yu-B13201
> Cc: kvm-ppc@vger.kernel.org
> Subject: RE: single-step issue
> 
> 
> > If so, then we cannot check vcpu->debug_flags to see if 
> control should
> > return to qemu/gdb.
> > Should we emulate privilege instructions in debug interrupt handler?
> 
> You can't, because the debug handler doesn't know the address of the
> instruction to emulate.
> 
> As a hack, the debug handler could see if CSRR0 matches one of the KVM
> interrupt vectors, and if so set a flag and return to the original
> handler. The normal handlers would then test and clear the flag to see
> if they should emulate a debug event.

Maybe we can.
For example as program interrupt already happened. We can get the fault
addr from SRR0.
Maybe that why the hardware designers use another register to save
pc/msr in debug interrupt....

Acctually I think most interrupts have the same problem, dec interrupt,
externel interrupt, tlb miss, storage,
Seems all of them I list don't clear MSR[DE].

The problem is how can we recognize the debug after normal interrupt.
If we can, then we can consider it as a normal interrupt, but at last
exit to qemu/gdb

One simple but dirty way to recognize it is comparing the value from
CSRR0 to handlers' address.
This can also find out the real interrupt.

What do you think?

> 
> > One thing I still don't get clear is that: does MSR[DE] is set while
> > running guest?
> > I see nowhere set it, but all kinds of debug interrupt seem 
> could work.
> 
> On 440, yes it is, but it's not heavily tested. 440 may have the same
> issue.
> 
> In general, debugging with KVM on 440/e500 is a hack. Imagine 
> setting a
> guest breakpoint that happens to match the last "rfi" instruction in a
> host -> guest switch. Since MSR[DE] only controls the 
> *delivery* of the
> interrupt, the breakpoint will trigger (but not be delivered) on every
> context switch. Then, when landing in the guest (rfi set 
> MSR[DE]=1), the
> interrupt will be delivered. Even if the host handles the interrupt
> correctly, the guest won't make forward progress, even though 
> it's never
> hit the breakpoint.

Not very understand.
You said "rfi set MSR[DE]=1".
Not rfi push SRR1 to MSR?
But I haven't found anywhere that set MSR[DE] bit into SRR1.

> 
> I don't think this problem should be an issue for single-stepping, but
> it suggests we can't reliably use hardware's IAC/DAC/DVC/etc. We could
> replace IAC with software breakpoints, and maybe DAC with page faults
> (with high overhead), but there's no software emulation for 
> DVC or e.g.
> "branch taken" events that I can see.
> 

Seems gdbserver uses software breakpoint by default.
Not sure how to enable hardware breakpoint in gdbserver.
Haven't test IAC/DAC.

RE: single-step issue

[Hollis Blanchard]

On Tue, 2009-05-12 at 19:14 +0800, Liu Yu-B13201 wrote:
> 
> > -----Original Message-----
> > From: kvm-ppc-owner@vger.kernel.org 
> > [mailto:kvm-ppc-owner@vger.kernel.org] On Behalf Of Hollis Blanchard
> > Sent: Tuesday, May 12, 2009 1:40 AM
> > To: Liu Yu-B13201
> > Cc: kvm-ppc@vger.kernel.org
> > Subject: RE: single-step issue
> > 
> > 
> > > If so, then we cannot check vcpu->debug_flags to see if 
> > control should
> > > return to qemu/gdb.
> > > Should we emulate privilege instructions in debug interrupt handler?
> > 
> > You can't, because the debug handler doesn't know the address of the
> > instruction to emulate.
> > 
> > As a hack, the debug handler could see if CSRR0 matches one of the KVM
> > interrupt vectors, and if so set a flag and return to the original
> > handler. The normal handlers would then test and clear the flag to see
> > if they should emulate a debug event.
> 
> Maybe we can.
> For example as program interrupt already happened. We can get the fault
> addr from SRR0.
> Maybe that why the hardware designers use another register to save
> pc/msr in debug interrupt....
> 
> Acctually I think most interrupts have the same problem, dec interrupt,
> externel interrupt, tlb miss, storage,
> Seems all of them I list don't clear MSR[DE].
> 
> The problem is how can we recognize the debug after normal interrupt.
> If we can, then we can consider it as a normal interrupt, but at last
> exit to qemu/gdb
> 
> One simple but dirty way to recognize it is comparing the value from
> CSRR0 to handlers' address.
> This can also find out the real interrupt.
> 
> What do you think?

As I said, I think it's a hack.

I guess you still need to compare CSRR0 with interrupt vector addresses
just so you know if you should use CSRR0 or SRR0. That sucks. At least
it's not a performance-sensitive path.

> > > One thing I still don't get clear is that: does MSR[DE] is set while
> > > running guest?
> > > I see nowhere set it, but all kinds of debug interrupt seem 
> > could work.
> > 
> > On 440, yes it is, but it's not heavily tested. 440 may have the same
> > issue.
> > 
> > In general, debugging with KVM on 440/e500 is a hack. Imagine 
> > setting a
> > guest breakpoint that happens to match the last "rfi" instruction in a
> > host -> guest switch. Since MSR[DE] only controls the 
> > *delivery* of the
> > interrupt, the breakpoint will trigger (but not be delivered) on every
> > context switch. Then, when landing in the guest (rfi set 
> > MSR[DE]=1), the
> > interrupt will be delivered. Even if the host handles the interrupt
> > correctly, the guest won't make forward progress, even though 
> > it's never
> > hit the breakpoint.
> 
> Not very understand.
> You said "rfi set MSR[DE]=1".
> Not rfi push SRR1 to MSR?

That's what I meant.

> But I haven't found anywhere that set MSR[DE] bit into SRR1.

See KVMPPC_MSR_MASK in booke_interrupts.S.

> > I don't think this problem should be an issue for single-stepping, but
> > it suggests we can't reliably use hardware's IAC/DAC/DVC/etc. We could
> > replace IAC with software breakpoints, and maybe DAC with page faults
> > (with high overhead), but there's no software emulation for 
> > DVC or e.g.
> > "branch taken" events that I can see.
> > 
> 
> Seems gdbserver uses software breakpoint by default.
> Not sure how to enable hardware breakpoint in gdbserver.

I think historically it has not been possible, but I've heard there is
ongoing work to add hardware breakpoint support to gdb for 440, at
least.

My point though is that the hardware debug mechanisms really need "guest
mode" awareness to be useful to us, and I'm afraid they don't.

As a side note, you might want to inquire internally about how e500mc
will handle this situation...

-- 
Hollis Blanchard
IBM Linux Technology Center

RE: single-step issue

[Liu Yu-B13201]

 

> -----Original Message-----
> From: Hollis Blanchard [mailto:hollisb@us.ibm.com] 
> Sent: Wednesday, May 13, 2009 12:39 AM
> To: Liu Yu-B13201
> Cc: kvm-ppc@vger.kernel.org
> Subject: RE: single-step issue
> 
> > > I don't think this problem should be an issue for 
> single-stepping, but
> > > it suggests we can't reliably use hardware's 
> IAC/DAC/DVC/etc. We could
> > > replace IAC with software breakpoints, and maybe DAC with 
> page faults
> > > (with high overhead), but there's no software emulation for 
> > > DVC or e.g.
> > > "branch taken" events that I can see.
> > > 
> > 
> > Seems gdbserver uses software breakpoint by default.
> > Not sure how to enable hardware breakpoint in gdbserver.
> 
> I think historically it has not been possible, but I've heard there is
> ongoing work to add hardware breakpoint support to gdb for 440, at
> least.
> 
> My point though is that the hardware debug mechanisms really 
> need "guest
> mode" awareness to be useful to us, and I'm afraid they don't.
> 
> As a side note, you might want to inquire internally about how e500mc
> will handle this situation...
> 

Yes. For e500mc there is a configuration that make debug interrupt only
*occur* in guest.