* Re: [PATCH RESEND v15 08/10] target-arm: kvm64: inject synchronous External Abort
@ 2018-11-21 14:33 gengdongjiu
2018-11-23 18:45 ` Peter Maydell
0 siblings, 1 reply; 7+ messages in thread
From: gengdongjiu @ 2018-11-21 14:33 UTC (permalink / raw)
To: Peter Maydell
Cc: Eduardo Habkost, kvm-devel, Michael S. Tsirkin, Marc Zyngier,
Marcelo Tosatti, QEMU Developers, Shannon Zhao, zhengxiang (A),
qemu-arm, James Morse, Paolo Bonzini, Igor Mammedov,
Laszlo Ersek, Richard Henderson
Hi Peter,
Thanks for the review and comments.
>
> On 8 November 2018 at 10:29, Dongjiu Geng <gengdongjiu@huawei.com> wrote:
> > Add synchronous external abort injection logic, setup exception type
> > and syndrome value. When switch to guest, guest will jump to the
> > synchronous external abort vector table entry.
> >
> > The ESR_ELx.DFSC is set to synchronous external abort(0x10), and
> > ESR_ELx.FnV is set to not valid(0x1), which will tell guest that FAR
> > is not valid and holds an UNKNOWN value.
> > These value will be set to KVM register structures through
> > KVM_SET_ONE_REG IOCTL.
> >
> > Signed-off-by: Dongjiu Geng <gengdongjiu@huawei.com>
> > ---
> > Marc is against that KVM inject the synchronous external abort(SEA) in
> > [1], so user space how to inject it. The test result that injection
> > SEA to guest by Qemu is shown in [2].
> >
> > [1]: https://lkml.org/lkml/2017/3/2/110
> > [2]:
> > Taking exception 4 [Data Abort]
> > ...from EL0 to EL1
> > ...with ESR 0x24/0x92000410
> > ...with FAR 0x0
> > ...with ELR 0x40cf04
> > ...to EL1 PC 0xffffffc000084c00 PSTATE 0x3c5 after kvm_inject_arm_sea
> > Unhandled fault: synchronous external abort (0x92000410) at
> > 0x0000007fa234c12c
> > CPU: 0 PID: 536 Comm: devmem Not tainted 4.1.0+ #20 Hardware name:
> > linux,dummy-virt (DT)
> > task: ffffffc019ab2b00 ti: ffffffc008134000 task.ti: ffffffc008134000
> > PC is at 0x40cf04 LR is at 0x40cdec pc : [<000000000040cf04>] lr :
> > [<000000000040cdec>] pstate: 60000000 sp : 0000007ff7b24130
> > x29: 0000007ff7b24260 x28: 0000000000000000
> > x27: 00000000000000ad x26: 000000000049c000
> > x25: 000000000048904b x24: 000000000049c000
> > x23: 0000000040600000 x22: 0000007ff7b243a0
> > x21: 0000000000000002 x20: 0000000000000000
> > x19: 0000000000000020 x18: 0000000000000000
> > x17: 000000000049c6d0 x16: 0000007fa22c85c0
> > x15: 0000000000005798 x14: 0000007fa2205f1c
> > x13: 0000007fa241ccb0 x12: 0000000000000137
> > x11: 0000000000000000 x10: 0000000000000000
> > x9 : 0000000000000000 x8 : 00000000000000de
> > x7 : 0000000000000000 x6 : 0000000000002000
> > x5 : 0000000040600000 x4 : 0000000000000003
> > x3 : 0000000000000001 x2 : 0000000000000000
> > x1 : 0000000000000000 x0 : 0000007fa2418000
> > ---
> > target/arm/cpu.h | 2 ++
> > target/arm/helper.c | 23 +++++++++++++++++++++++
> > target/arm/internals.h | 5 +++--
> > target/arm/kvm64.c | 39 +++++++++++++++++++++++++++++++++++++++
> > target/arm/op_helper.c | 2 +-
> > 5 files changed, 68 insertions(+), 3 deletions(-)
> >
> > diff --git a/target/arm/cpu.h b/target/arm/cpu.h index
> > b5eff79..502507d 100644
> > --- a/target/arm/cpu.h
> > +++ b/target/arm/cpu.h
> > @@ -2331,6 +2331,8 @@ bool write_list_to_cpustate(ARMCPU *cpu);
> > */
> > bool write_cpustate_to_list(ARMCPU *cpu);
> >
> > +bool write_part_cpustate_to_list(ARMCPU *cpu, ptrdiff_t fieldoffset);
> > +
> > #define ARM_CPUID_TI915T 0x54029152
> > #define ARM_CPUID_TI925T 0x54029252
> >
> > diff --git a/target/arm/helper.c b/target/arm/helper.c index
> > 9630193..df078ff 100644
> > --- a/target/arm/helper.c
> > +++ b/target/arm/helper.c
> > @@ -263,6 +263,29 @@ static bool raw_accessors_invalid(const ARMCPRegInfo *ri)
> > return true;
> > }
> >
> > +bool write_part_cpustate_to_list(ARMCPU *cpu, ptrdiff_t fieldoffset)
> > +{
> > + const ARMCPRegInfo *ri;
> > + uint32_t regidx, i;
> > +
> > + for (i = 0; i < cpu->cpreg_array_len; i++) {
> > + regidx = kvm_to_cpreg_id(cpu->cpreg_indexes[i]);
> > + ri = get_arm_cp_reginfo(cpu->cp_regs, regidx);
> > + if (!ri) {
> > + continue;
> > + }
> > +
> > + if (ri->type & ARM_CP_NO_RAW) {
> > + continue;
> > + }
> > + if (ri->fieldoffset == fieldoffset) {
> > + cpu->cpreg_values[i] = read_raw_cp_reg(&cpu->env, ri);
> > + return true;
> > + }
> > + }
> > + return false;
> > +}
>
> What is this about? Nothing else in QEMU needs to mess with the cpustate synchronization. My first assumption is that you should not
> need to do so either.
We should change the guest CP15 ESR_EL1's value, the only method is to change the cpu->cpreg_values[] in QEMU, then QEMU call write_list_to_kvmstate()
to set the cpu->cpreg_values[] to KVM which include the specified ESR_EL1 value, KVM do world switch, and then set the specified ESR_EL1's value to guest kernel.
About the detailed explanation, as shown in [2].
kvm_arm_handle_debug() does not need to do this because QEMU does not need to change CP15 registers, such as ESR_EL1.
>
> > +
> > bool write_cpustate_to_list(ARMCPU *cpu) {
> > /* Write the coprocessor state from cpu->env to the (index,value)
> > list. */ diff --git a/target/arm/internals.h b/target/arm/internals.h
> > index 6c2bb2d..04ea074 100644
> > --- a/target/arm/internals.h
> > +++ b/target/arm/internals.h
> > @@ -415,13 +415,14 @@ static inline uint32_t syn_insn_abort(int same_el, int ea, int s1ptw, int fsc)
> > | ARM_EL_IL | (ea << 9) | (s1ptw << 7) | fsc; }
> >
> > -static inline uint32_t syn_data_abort_no_iss(int same_el,
> > +static inline uint32_t syn_data_abort_no_iss(int same_el, int fnv,
> > int ea, int cm, int s1ptw,
> > int wnr, int fsc) {
> > return (EC_DATAABORT << ARM_EL_EC_SHIFT) | (same_el << ARM_EL_EC_SHIFT)
> > | ARM_EL_IL
> > - | (ea << 9) | (cm << 8) | (s1ptw << 7) | (wnr << 6) | fsc;
> > + | (fnv << 10) | (ea << 9) | (cm << 8) | (s1ptw << 7)
> > + | (wnr << 6) | fsc;
> > }
> >
> > static inline uint32_t syn_data_abort_with_iss(int same_el, diff
> > --git a/target/arm/kvm64.c b/target/arm/kvm64.c index 5de8ff0..0ca2b29
> > 100644
> > --- a/target/arm/kvm64.c
> > +++ b/target/arm/kvm64.c
> > @@ -594,6 +594,45 @@ int kvm_arm_cpreg_level(uint64_t regidx)
> > return KVM_PUT_RUNTIME_STATE;
> > }
> >
> > +/* Inject synchronous external abort */ static void
> > +kvm_inject_arm_sea(CPUState *c) {
> > + ARMCPU *cpu = ARM_CPU(c);
> > + CPUARMState *env = &cpu->env;
> > + CPUClass *cc = CPU_GET_CLASS(c);
> > + uint32_t esr;
> > + int ret;
> > +
> > + /* This exception is synchronous data abort */
> > + c->exception_index = EXCP_DATA_ABORT;
> > + /* Inject the exception to guest EL1 */
> > + env->exception.target_el = 1;
>
> These comments don't tell us anything that the code does not.
Thanks, do you mean I need to remove it or add more detailed comments to it?
>
> > +
> > + /* Set the DFSC to synchronous external abort and set FnV to not valid,
> > + * this will tell guest the FAR_ELx is UNKNOWN for this abort.
> > + */
> > +
> > + /* This exception comes from lower or current exception level. */
> > + if (arm_current_el(env) == PSTATE_MODE_EL0t) {
>
> arm_current_el() returns an integer; you should be comparing it to 0,1,2,3, not to PSTATE_MODE_* constants.
Yes, you are right, I will change it, thanks a lot for pointing out.
>
> > + esr = syn_data_abort_no_iss(0, 1, 0, 0, 0, 0, 0x10);
> > + } else {
> > + esr = syn_data_abort_no_iss(1, 1, 0, 0, 0, 0, 0x10);
> > + }
>
> Better written as
> bool same_el;
> [...]
>
> same_el = arm_current_el(env) == env->exception.target_el;
> env->exception.syndrome = syn_data_abort_no_iss(same_el, 1, 0, 0, 0, 0, 0x10);
Ok, thanks for the good suggestion.
>
> > +
> > + env->exception.syndrome = esr;
> > +
> > + cc->do_interrupt(c);
>
> You need to take the iothread lock before calling do_interrupt.
> Compare the code in kvm_arm_handle_debug().
In the old code, it does not have iothread lock, this lock should be added recently as shown [1], so I forget it.
Sure I will add it.
[1]:
-----------------------------------------------------------
qemu_mutex_lock_iothread();
cc->do_interrupt(cs);
qemu_mutex_unlock_iothread();
-----------------------------------------------------------
>
> > + if (kvm_enabled()) {
>
> How can we get here if KVM is not enabled ?
Indeed, in my current patch, if it is non-KVM mode, it cannot got here,
I will remove this judgment.
>
> > + /* write ESR_EL1 from cpustate to list*/
> > + ret = write_part_cpustate_to_list(cpu,
> > + offsetof(CPUARMState, cp15.esr_el[1]));
> > + if (!ret) {
> > + fprintf(stderr, "<%s> failed to set esr_el1\n", __func__);
> > + abort();
> > + }
>
> kvm_arm_handle_debug() doesn't need to do this, I don't understand why this would be different.
[2]:
kvm_arm_handle_debug() does not need to change guest cp15 registers, in order to inject a synchronous external abort,
we needs to set guest syndrome error register ESR_EL1's value through KVM, when guest kernel jump to Synchronous EL1h
exception vector table,it will read the ESR_EL1's value and know this is synchronous external abort and do recovery, otherwise it does not the error type.
In the KVM mode, it use below call stack to set the CP15 ESR_EL1 register.
-->kvm_arch_put_registers()
------> write_list_to_kvmstate()
bool write_list_to_kvmstate(ARMCPU *cpu, int level)
{
...................
case KVM_REG_SIZE_U64:
r.addr = (uintptr_t)(cpu->cpreg_values + i);
break;
default:
abort();
}
ret = kvm_vcpu_ioctl(cs, KVM_SET_ONE_REG, &r);
..............
}
As we can see QEMU uses cpu->cpreg_values[] to set the CP15 registers to KVM, so I add a function write_part_cpustate_to_list()
to change the value in the cpu->cpreg_values[] which is corresponding to the ESR_EL1's value. In current QEMU code,
it should not have API to change some value in cpu->cpreg_values[], so I add a new function.
>
> > + }
> > +}
> > +
> > #define AARCH64_CORE_REG(x) (KVM_REG_ARM64 | KVM_REG_SIZE_U64 | \
> > KVM_REG_ARM_CORE | KVM_REG_ARM_CORE_REG(x))
> >
> > diff --git a/target/arm/op_helper.c b/target/arm/op_helper.c index
> > 90741f6..3f1d656 100644
> > --- a/target/arm/op_helper.c
> > +++ b/target/arm/op_helper.c
> > @@ -109,7 +109,7 @@ static inline uint32_t merge_syn_data_abort(uint32_t template_syn,
> > * ISV field.
> > */
> > if (!(template_syn & ARM_EL_ISV) || target_el != 2 || s1ptw) {
> > - syn = syn_data_abort_no_iss(same_el,
> > + syn = syn_data_abort_no_iss(same_el, 0,
> > ea, 0, s1ptw, is_write, fsc);
> > } else {
> > /* Fields: IL, ISV, SAS, SSE, SRT, SF and AR come from the
> > template
> > --
> > 1.8.3.1
> >
>
> thanks
> -- PMM
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH RESEND v15 08/10] target-arm: kvm64: inject synchronous External Abort
2018-11-21 14:33 [PATCH RESEND v15 08/10] target-arm: kvm64: inject synchronous External Abort gengdongjiu
@ 2018-11-23 18:45 ` Peter Maydell
2018-11-24 7:14 ` gengdongjiu
2018-11-26 11:50 ` Alex Bennée
0 siblings, 2 replies; 7+ messages in thread
From: Peter Maydell @ 2018-11-23 18:45 UTC (permalink / raw)
To: gengdongjiu
Cc: Eduardo Habkost, kvm-devel, Michael S. Tsirkin, Marc Zyngier,
Alex Bennée, Marcelo Tosatti, QEMU Developers, Shannon Zhao,
Zheng Xiang, qemu-arm, James Morse, Paolo Bonzini, Igor Mammedov,
Laszlo Ersek, Richard Henderson
On Wed, 21 Nov 2018 at 14:34, gengdongjiu <gengdongjiu@huawei.com> wrote:
>
> Hi Peter,
> Thanks for the review and comments.
>
> >
> > On 8 November 2018 at 10:29, Dongjiu Geng <gengdongjiu@huawei.com> wrote:
> > > +bool write_part_cpustate_to_list(ARMCPU *cpu, ptrdiff_t fieldoffset)
> >
> > What is this about? Nothing else in QEMU needs to mess with the cpustate synchronization. My first assumption is that you should not
> > need to do so either.
>
> We should change the guest CP15 ESR_EL1's value, the only method is to change the cpu->cpreg_values[] in QEMU, then QEMU call write_list_to_kvmstate()
> to set the cpu->cpreg_values[] to KVM which include the specified ESR_EL1 value, KVM do world switch, and then set the specified ESR_EL1's value to guest kernel.
Ah, I see. This is a bug in our current handling of the register
state, where we implicitly assume that nothing in QEMU will ever
want to change any system register values. This assumption is
now false -- kvm_arm_handle_debug() broke it -- so we need to
fix the code that does kvm_arch_put_registers(). There is a comment
in the kvm32.c version of that function about this. (The kvm64.c
version has the same assumption but doesn't comment on it.)
We should (ideally) fix this bug in the code that does register
syncing, without requiring places in QEMU that update system
registers to have to manually indicate which registers they have
changed. I'll have a think about how best to do this.
> About the detailed explanation, as shown in [2].
>
> kvm_arm_handle_debug() does not need to do this because QEMU does not need to change CP15 registers, such as ESR_EL1.
kvm_arm_handle_debug does change ESR_EL1: it is injecting an exception
and so should set the exception register. This happens when it
calls the do_interrupt() hook, because arm_cpu_do_interrupt_aarch64()
writes to env->cp15.esr_el[new_el].
I'm not entirely sure why this is working today, in fact.
Alex, did you test whether our debug-exception-injection
reports the correct ESR_EL1 to the guest ?
> > > +/* Inject synchronous external abort */ static void
> > > +kvm_inject_arm_sea(CPUState *c) {
> > > + ARMCPU *cpu = ARM_CPU(c);
> > > + CPUARMState *env = &cpu->env;
> > > + CPUClass *cc = CPU_GET_CLASS(c);
> > > + uint32_t esr;
> > > + int ret;
> > > +
> > > + /* This exception is synchronous data abort */
> > > + c->exception_index = EXCP_DATA_ABORT;
> > > + /* Inject the exception to guest EL1 */
> > > + env->exception.target_el = 1;
> >
> > These comments don't tell us anything that the code does not.
>
> Thanks, do you mean I need to remove it or add more detailed comments to it?
As a rule of thumb, comments should provide information to
the reader which they wouldn't get if they only had the code.
Comments often answer the "why do we do this" question, or
provide an overall summary of what the code is going to do,
or refer to an external source (a datasheet, an algorithm)
that is necessary to understand the code. It's better to
avoid comments that say "what the code is doing" at a line-by-line
level, because the code itself already answers the "what"
question at that level of detail.
thanks
-- PMM
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH RESEND v15 08/10] target-arm: kvm64: inject synchronous External Abort
2018-11-23 18:45 ` Peter Maydell
@ 2018-11-24 7:14 ` gengdongjiu
2018-11-26 11:50 ` Alex Bennée
1 sibling, 0 replies; 7+ messages in thread
From: gengdongjiu @ 2018-11-24 7:14 UTC (permalink / raw)
To: Peter Maydell
Cc: Eduardo Habkost, kvm-devel, Michael S. Tsirkin, Marc Zyngier,
Alex Bennée, Marcelo Tosatti, QEMU Developers, Shannon Zhao,
Zheng Xiang, qemu-arm, James Morse, Paolo Bonzini, Igor Mammedov,
Laszlo Ersek, Richard Henderson
Hi Peter,
On 2018/11/24 2:45, Peter Maydell wrote:
> On Wed, 21 Nov 2018 at 14:34, gengdongjiu <gengdongjiu@huawei.com> wrote:
>>
>> Hi Peter,
>> Thanks for the review and comments.
>>
>>>
>>> On 8 November 2018 at 10:29, Dongjiu Geng <gengdongjiu@huawei.com> wrote:
>>>> +bool write_part_cpustate_to_list(ARMCPU *cpu, ptrdiff_t fieldoffset)
>>>
>>> What is this about? Nothing else in QEMU needs to mess with the cpustate synchronization. My first assumption is that you should not
>>> need to do so either.
>>
>> We should change the guest CP15 ESR_EL1's value, the only method is to change the cpu->cpreg_values[] in QEMU, then QEMU call write_list_to_kvmstate()
>> to set the cpu->cpreg_values[] to KVM which include the specified ESR_EL1 value, KVM do world switch, and then set the specified ESR_EL1's value to guest kernel.
>
> Ah, I see. This is a bug in our current handling of the register
> state, where we implicitly assume that nothing in QEMU will ever
> want to change any system register values. This assumption is
> now false -- kvm_arm_handle_debug() broke it -- so we need to
> fix the code that does kvm_arch_put_registers(). There is a comment
> in the kvm32.c version of that function about this. (The kvm64.c
> version has the same assumption but doesn't comment on it.)
>
> We should (ideally) fix this bug in the code that does register
> syncing, without requiring places in QEMU that update system
> registers to have to manually indicate which registers they have
> changed. I'll have a think about how best to do this.
Ok, it is great that you will think about it, waiting for your wonderful solution
>
>> About the detailed explanation, as shown in [2].
>>
>> kvm_arm_handle_debug() does not need to do this because QEMU does not need to change CP15 registers, such as ESR_EL1.
>
> kvm_arm_handle_debug does change ESR_EL1: it is injecting an exception
> and so should set the exception register. This happens when it
> calls the do_interrupt() hook, because arm_cpu_do_interrupt_aarch64()
> writes to env->cp15.esr_el[new_el].
Yes, I see it, but the env->cp15.esr_el[new_el] shouldn't be successfully set to KVM when call kvm_arch_put_registers()
>
> I'm not entirely sure why this is working today, in fact.
> Alex, did you test whether our debug-exception-injection
> reports the correct ESR_EL1 to the guest ?
Alex?
>
>>>> +/* Inject synchronous external abort */ static void
>>>> +kvm_inject_arm_sea(CPUState *c) {
>>>> + ARMCPU *cpu = ARM_CPU(c);
>>>> + CPUARMState *env = &cpu->env;
>>>> + CPUClass *cc = CPU_GET_CLASS(c);
>>>> + uint32_t esr;
>>>> + int ret;
>>>> +
>>>> + /* This exception is synchronous data abort */
>>>> + c->exception_index = EXCP_DATA_ABORT;
>>>> + /* Inject the exception to guest EL1 */
>>>> + env->exception.target_el = 1;
>>>
>>> These comments don't tell us anything that the code does not.
>>
>> Thanks, do you mean I need to remove it or add more detailed comments to it?
>
> As a rule of thumb, comments should provide information to
> the reader which they wouldn't get if they only had the code.
> Comments often answer the "why do we do this" question, or
> provide an overall summary of what the code is going to do,
> or refer to an external source (a datasheet, an algorithm)
> that is necessary to understand the code. It's better to
> avoid comments that say "what the code is doing" at a line-by-line
> level, because the code itself already answers the "what"
> question at that level of detail.
sure, got it, thanks for the explanation and guidelines.
>
> thanks
> -- PMM
>
> .
>
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH RESEND v15 08/10] target-arm: kvm64: inject synchronous External Abort
2018-11-23 18:45 ` Peter Maydell
2018-11-24 7:14 ` gengdongjiu
@ 2018-11-26 11:50 ` Alex Bennée
1 sibling, 0 replies; 7+ messages in thread
From: Alex Bennée @ 2018-11-26 11:50 UTC (permalink / raw)
To: Peter Maydell
Cc: Eduardo Habkost, kvm-devel, Michael S. Tsirkin, Marc Zyngier,
Marcelo Tosatti, QEMU Developers, gengdongjiu, Shannon Zhao,
Zheng Xiang, qemu-arm, James Morse, Paolo Bonzini, Igor Mammedov,
Laszlo Ersek, Richard Henderson
Peter Maydell <peter.maydell@linaro.org> writes:
> On Wed, 21 Nov 2018 at 14:34, gengdongjiu <gengdongjiu@huawei.com> wrote:
>>
>> Hi Peter,
>> Thanks for the review and comments.
>>
>> >
>> > On 8 November 2018 at 10:29, Dongjiu Geng <gengdongjiu@huawei.com> wrote:
>> > > +bool write_part_cpustate_to_list(ARMCPU *cpu, ptrdiff_t fieldoffset)
>> >
>> > What is this about? Nothing else in QEMU needs to mess with the cpustate synchronization. My first assumption is that you should not
>> > need to do so either.
>>
>> We should change the guest CP15 ESR_EL1's value, the only method is to change the cpu->cpreg_values[] in QEMU, then QEMU call write_list_to_kvmstate()
>> to set the cpu->cpreg_values[] to KVM which include the specified ESR_EL1 value, KVM do world switch, and then set the specified ESR_EL1's value to guest kernel.
>
> Ah, I see. This is a bug in our current handling of the register
> state, where we implicitly assume that nothing in QEMU will ever
> want to change any system register values. This assumption is
> now false -- kvm_arm_handle_debug() broke it -- so we need to
> fix the code that does kvm_arch_put_registers(). There is a comment
> in the kvm32.c version of that function about this. (The kvm64.c
> version has the same assumption but doesn't comment on it.)
>
> We should (ideally) fix this bug in the code that does register
> syncing, without requiring places in QEMU that update system
> registers to have to manually indicate which registers they have
> changed. I'll have a think about how best to do this.
>
>> About the detailed explanation, as shown in [2].
>>
>> kvm_arm_handle_debug() does not need to do this because QEMU does not need to change CP15 registers, such as ESR_EL1.
>
> kvm_arm_handle_debug does change ESR_EL1: it is injecting an exception
> and so should set the exception register. This happens when it
> calls the do_interrupt() hook, because arm_cpu_do_interrupt_aarch64()
> writes to env->cp15.esr_el[new_el].
>
> I'm not entirely sure why this is working today, in fact.
> Alex, did you test whether our debug-exception-injection
> reports the correct ESR_EL1 to the guest ?
<snip>
I did not - I was mostly focusing in the host-debugging-the-guest test
case. I'll get a test rig up and check.
--
Alex Bennée
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH RESEND v15 08/10] target-arm: kvm64: inject synchronous External Abort
@ 2018-11-26 17:25 gengdongjiu
0 siblings, 0 replies; 7+ messages in thread
From: gengdongjiu @ 2018-11-26 17:25 UTC (permalink / raw)
To: Alex Bennée, Peter Maydell
Cc: Eduardo Habkost, kvm-devel, Michael S. Tsirkin, Marc Zyngier,
Marcelo Tosatti, QEMU Developers, Shannon Zhao, zhengxiang (A),
qemu-arm, James Morse, Paolo Bonzini, Igor Mammedov,
Laszlo Ersek, Richard Henderson
> >>
> >> Hi Peter,
> >> Thanks for the review and comments.
> >>
> >> >
> >> > On 8 November 2018 at 10:29, Dongjiu Geng <gengdongjiu@huawei.com> wrote:
> >> > > +bool write_part_cpustate_to_list(ARMCPU *cpu, ptrdiff_t
> >> > > +fieldoffset)
> >> >
> >> > What is this about? Nothing else in QEMU needs to mess with the
> >> > cpustate synchronization. My first assumption is that you should not need to do so either.
> >>
> >> We should change the guest CP15 ESR_EL1's value, the only method is
> >> to change the cpu->cpreg_values[] in QEMU, then QEMU call write_list_to_kvmstate() to set the cpu->cpreg_values[] to KVM which
> include the specified ESR_EL1 value, KVM do world switch, and then set the specified ESR_EL1's value to guest kernel.
> >
> > Ah, I see. This is a bug in our current handling of the register
> > state, where we implicitly assume that nothing in QEMU will ever want
> > to change any system register values. This assumption is now false --
> > kvm_arm_handle_debug() broke it -- so we need to fix the code that
> > does kvm_arch_put_registers(). There is a comment in the kvm32.c
> > version of that function about this. (The kvm64.c version has the same
> > assumption but doesn't comment on it.)
> >
> > We should (ideally) fix this bug in the code that does register
> > syncing, without requiring places in QEMU that update system registers
> > to have to manually indicate which registers they have changed. I'll
> > have a think about how best to do this.
> >
> >> About the detailed explanation, as shown in [2].
> >>
> >> kvm_arm_handle_debug() does not need to do this because QEMU does not need to change CP15 registers, such as ESR_EL1.
> >
> > kvm_arm_handle_debug does change ESR_EL1: it is injecting an exception
> > and so should set the exception register. This happens when it calls
> > the do_interrupt() hook, because arm_cpu_do_interrupt_aarch64() writes
> > to env->cp15.esr_el[new_el].
> >
> > I'm not entirely sure why this is working today, in fact.
> > Alex, did you test whether our debug-exception-injection reports the
> > correct ESR_EL1 to the guest ?
> <snip>
>
> I did not - I was mostly focusing in the host-debugging-the-guest test case. I'll get a test rig up and check.
Thanks, please test it in the KVM mode, not in the TCG mode. Waiting for your test result.
>
> --
> Alex Bennée
^ permalink raw reply [flat|nested] 7+ messages in thread
* [PATCH RESEND v15 00/10] Add ARMv8 RAS virtualization support in QEMU
@ 2018-11-08 10:29 Dongjiu Geng
2018-11-08 10:29 ` [PATCH RESEND v15 08/10] target-arm: kvm64: inject synchronous External Abort Dongjiu Geng
0 siblings, 1 reply; 7+ messages in thread
From: Dongjiu Geng @ 2018-11-08 10:29 UTC (permalink / raw)
To: imammedo, mst, peter.maydell, lersek, marc.zyngier, james.morse,
pbonzini, mtosatti, rth, zhengxiang9, ehabkost, kvm,
shannon.zhaosl, qemu-devel, qemu-arm, gengdongjiu
In the ARMv8 platform, the CPU error type are synchronous external
abort(SEA) and SError Interrupt (SEI). If exception happens to guest,
sometimes guest itself do the recovery is better, because host
does not know guest's detailed information. For example, if a guest
user-space application happen exception, host doe not which application
encounter errors.
For the ARMv8 SEA/SEI, KVM or host kernel will deliver SIGBUS or
use other interface to notify user space. After user space gets
the notification, it will record the CPER to guest GHES buffer
for guest and inject a exception or IRQ to KVM.
In the current implement, if the SIGBUS is BUS_MCEERR_AR, we will
treat it as synchronous exception, and use ARMv8 SEA notification type
to notify guest after recording CPER for guest;
This series patches are based on Qemu 2.12, which have two parts:
1. Generate APEI/GHES table and record CPER for guest in runtime.
2. Handle the SIGBUS signal, record the CPER and fill into guest memory,
then according to SIGBUS type(BUS_MCEERR_AR or BUS_MCEERR_AO), using
different ACPI notification type to notify guest.
Whole solution was suggested by James(james.morse@arm.com); APEI part solution is suggested by
Laszlo(lersek@redhat.com). Shown some discussion in [1].
This series patches have already tested on ARM64 platform with RAS feature enabled:
Show the APEI part verification result in [2]
Show the BUS_MCEERR_AR SIGBUS handling verification result in [3]
---
Change since v14:
1. Remove the BUS_MCEERR_AO handling logic because this asynchronous signal was masked by main thread
2. Address some Igor Mammedov's comments(ACPI part)
1) change the comments for the enum AcpiHestNotifyType definition and remove ditto in patch 1
2) change some patch commit messages and separate "APEI GHES table generation" patch to more patches.
3. Address some peter's comments(arm64 Synchronous External Abort injection)
1) change some code notes
2) using arm_current_el() for current EL
2) use the helper functions for those (syn_data_abort_*).
Change since v13:
1. Move the patches that set guest ESR and inject virtual SError out of this series
2. Clean and optimize the APEI part patches
3. Update the commit messages and add some comments for the code
Change since v12:
1. Address Paolo's comments to move HWPoisonPage definition to accel/kvm/kvm-all.c
2. Only call kvm_cpu_synchronize_state() when get the BUS_MCEERR_AR signal
3. Only add and enable GPIO-Signal and ARMv8 SEA two hardware error sources
4. Address Michael's comments to not sync SPDX from Linux kernel header file
Change since v11:
Address James's comments(james.morse@arm.com)
1. Check whether KVM has the capability to to set ESR instead of detecting host CPU RAS capability
2. For SIGBUS_MCEERR_AR SIGBUS, use Synchronous-External-Abort(SEA) notification type
for SIGBUS_MCEERR_AO SIGBUS, use GPIO-Signal notification
Address Shannon's comments(for ACPI part):
1. Unify hest_ghes.c and hest_ghes.h license declaration
2. Remove unnecessary including "qmp-commands.h" in hest_ghes.c
3. Unconditionally add guest APEI table based on James's comments(james.morse@arm.com)
4. Add a option to virt machine for migration compatibility. On new virt machine it's on
by default while off for old ones, we enabled it since 2.12
5. Refer to the ACPI spec version which introduces Hardware Error Notification first time
6. Add ACPI_HEST_NOTIFY_RESERVED notification type
Address Igor's comments(for ACPI part):
1. Add doc patch first which will describe how it's supposed to work between QEMU/firmware/guest
OS with expected flows.
2. Move APEI diagrams into doc/spec patch
3. Remove redundant g_malloc in ghes_record_cper()
4. Use build_append_int_noprefix() API to compose whole error status block and whole APEI table,
and try to get rid of most structures in patch 1, as they will be left unused after that
5. Reuse something like https://github.com/imammedo/qemu/commit/3d2fd6d13a3ea298d2ee814835495ce6241d085c
to build GAS
6. Remove much offsetof() in the function
7. Build independent tables first and only then build dependent tables passing to it pointers
to previously build table if necessary.
8. Redefine macro GHES_ACPI_HEST_NOTIFY_RESERVED to ACPI_HEST_ERROR_SOURCE_COUNT to avoid confusion
Address Peter Maydell's comments
1. linux-headers is done as a patch of their own created using scripts/update-linux-headers.sh run against a
mainline kernel tree
2. Tested whether this patchset builds OK on aarch32
3. Abstract Hwpoison page adding code out properly into a cpu-independent source file from target/i386/kvm.c,
such as kvm-all.c
4. Add doc-comment formatted documentation comment for new globally-visible function prototype in a header
---
[1]:
https://lkml.org/lkml/2017/2/27/246
https://patchwork.kernel.org/patch/9633105/
https://patchwork.kernel.org/patch/9925227/
[2]:
Note: the UEFI(QEMU_EFI.fd) is needed if guest want to use ACPI table.
After guest boot up, dump the APEI table, then can see the initialized table
(1) # iasl -p ./HEST -d /sys/firmware/acpi/tables/HEST
(2) # cat HEST.dsl
/*
* Intel ACPI Component Architecture
* AML/ASL+ Disassembler version 20170728 (64-bit version)
* Copyright (c) 2000 - 2017 Intel Corporation
*
* Disassembly of /sys/firmware/acpi/tables/HEST, Mon Sep 5 07:59:17 2016
*
* ACPI Data Table [HEST]
*
* Format: [HexOffset DecimalOffset ByteLength] FieldName : FieldValue
*/
..................................................................................
[308h 0776 2] Subtable Type : 000A [Generic Hardware Error Source V2]
[30Ah 0778 2] Source Id : 0001
[30Ch 0780 2] Related Source Id : FFFF
[30Eh 0782 1] Reserved : 00
[30Fh 0783 1] Enabled : 01
[310h 0784 4] Records To Preallocate : 00000001
[314h 0788 4] Max Sections Per Record : 00000001
[318h 0792 4] Max Raw Data Length : 00001000
[31Ch 0796 12] Error Status Address : [Generic Address Structure]
[31Ch 0796 1] Space ID : 00 [SystemMemory]
[31Dh 0797 1] Bit Width : 40
[31Eh 0798 1] Bit Offset : 00
[31Fh 0799 1] Encoded Access Width : 04 [QWord Access:64]
[320h 0800 8] Address : 00000000785D0040
[328h 0808 28] Notify : [Hardware Error Notification Structure]
[328h 0808 1] Notify Type : 08 [SEA]
[329h 0809 1] Notify Length : 1C
[32Ah 0810 2] Configuration Write Enable : 0000
[32Ch 0812 4] PollInterval : 00000000
[330h 0816 4] Vector : 00000000
[334h 0820 4] Polling Threshold Value : 00000000
[338h 0824 4] Polling Threshold Window : 00000000
[33Ch 0828 4] Error Threshold Value : 00000000
[340h 0832 4] Error Threshold Window : 00000000
[344h 0836 4] Error Status Block Length : 00001000
[348h 0840 12] Read Ack Register : [Generic Address Structure]
[348h 0840 1] Space ID : 00 [SystemMemory]
[349h 0841 1] Bit Width : 40
[34Ah 0842 1] Bit Offset : 00
[34Bh 0843 1] Encoded Access Width : 04 [QWord Access:64]
[34Ch 0844 8] Address : 00000000785D0098
[354h 0852 8] Read Ack Preserve : 00000000FFFFFFFE
[35Ch 0860 8] Read Ack Write : 0000000000000001
.....................................................................................
(3) After a synchronous external abort(SEA) happen, Qemu receive a SIGBUS and
filled the CPER into guest GHES memory. For example, according to above table,
the address that contains the physical address of a block of memory that holds
the error status data for this abort is 0x00000000785D0040
(4) the address for SEA notification error source is 0x785d80b0
(qemu) xp /1 0x00000000785D0040
00000000785d0040: 0x785d80b0
(5) check the content of generic error status block and generic error data entry
(qemu) xp /100x 0x785d80b0
00000000785d80b0: 0x00000001 0x00000000 0x00000000 0x00000098
00000000785d80c0: 0x00000000 0xa5bc1114 0x4ede6f64 0x833e63b8
00000000785d80d0: 0xb1837ced 0x00000000 0x00000300 0x00000050
00000000785d80e0: 0x00000000 0x00000000 0x00000000 0x00000000
00000000785d80f0: 0x00000000 0x00000000 0x00000000 0x00000000
00000000785d8100: 0x00000000 0x00000000 0x00000000 0x00004002
(6) check the OSPM's ACK value(for example SEA)
/* Before OSPM acknowledges the error, check the ACK value */
(qemu) xp /1 0x00000000785D0098
00000000785d00f0: 0x00000000
/* After OSPM acknowledges the error, check the ACK value, it change to 1 from 0 */
(qemu) xp /1 0x00000000785D0098
00000000785d00f0: 0x00000001
[3]: KVM deliver "BUS_MCEERR_AR" to Qemu, Qemu record the guest CPER and inject
synchronous external abort to notify guest, then guest do the recovery.
[ 1552.516170] Synchronous External Abort: synchronous external abort (0x92000410) at 0x000000003751c6b4
[ 1553.074073] {1}[Hardware Error]: Hardware error from APEI Generic Hardware Error Source: 8
[ 1553.081654] {1}[Hardware Error]: event severity: recoverable
[ 1554.034191] {1}[Hardware Error]: Error 0, type: recoverable
[ 1554.037934] {1}[Hardware Error]: section_type: memory error
[ 1554.513261] {1}[Hardware Error]: physical_address: 0x0000000040fa6000
[ 1554.513944] {1}[Hardware Error]: error_type: 0, unknown
[ 1555.041451] Memory failure: 0x40fa6: Killing mca-recover:1296 due to hardware memory corruption
[ 1555.373116] Memory failure: 0x40fa6: recovery action for dirty LRU page: Recovered
Dongjiu Geng (10):
ACPI: add some GHES structures and macros definition
acpi: add build_append_ghes_notify() helper for Hardware Error
Notification
acpi: add build_append_ghes_generic_data() helper for Generic Error
Data Entry
acpi: add build_append_ghes_generic_status() helper for Generic Error
Status Block
ACPI: Add APEI GHES table generation and CPER record support
docs: APEI GHES generation and CPER record description
KVM: Move related hwpoison page functions to accel/kvm/ folder
target-arm: kvm64: inject synchronous External Abort
hw/arm/virt: Add RAS platform version for migration
target-arm: kvm64: handle SIGBUS signal from kernel or KVM
accel/kvm/kvm-all.c | 33 ++++
default-configs/arm-softmmu.mak | 1 +
docs/specs/acpi_hest_ghes.txt | 97 ++++++++++++
hw/acpi/Makefile.objs | 1 +
hw/acpi/acpi_ghes.c | 335 ++++++++++++++++++++++++++++++++++++++++
hw/acpi/aml-build.c | 70 +++++++++
hw/arm/virt-acpi-build.c | 12 ++
hw/arm/virt.c | 4 +
include/exec/ram_addr.h | 5 +
include/hw/acpi/acpi-defs.h | 52 +++++++
include/hw/acpi/acpi_ghes.h | 86 +++++++++++
include/hw/acpi/aml-build.h | 21 +++
include/hw/arm/virt.h | 1 +
include/sysemu/kvm.h | 2 +-
target/arm/cpu.h | 2 +
target/arm/helper.c | 23 +++
target/arm/internals.h | 5 +-
target/arm/kvm.c | 3 +
target/arm/kvm64.c | 73 +++++++++
target/arm/op_helper.c | 2 +-
target/i386/kvm.c | 33 ----
21 files changed, 824 insertions(+), 37 deletions(-)
create mode 100644 docs/specs/acpi_hest_ghes.txt
create mode 100644 hw/acpi/acpi_ghes.c
create mode 100644 include/hw/acpi/acpi_ghes.h
--
1.8.3.1
^ permalink raw reply [flat|nested] 7+ messages in thread
* [PATCH RESEND v15 08/10] target-arm: kvm64: inject synchronous External Abort
2018-11-08 10:29 [PATCH RESEND v15 00/10] Add ARMv8 RAS virtualization support in QEMU Dongjiu Geng
@ 2018-11-08 10:29 ` Dongjiu Geng
2018-11-20 15:07 ` Peter Maydell
0 siblings, 1 reply; 7+ messages in thread
From: Dongjiu Geng @ 2018-11-08 10:29 UTC (permalink / raw)
To: imammedo, mst, peter.maydell, lersek, marc.zyngier, james.morse,
pbonzini, mtosatti, rth, zhengxiang9, ehabkost, kvm,
shannon.zhaosl, qemu-devel, qemu-arm, gengdongjiu
Add synchronous external abort injection logic, setup
exception type and syndrome value. When switch to guest,
guest will jump to the synchronous external abort vector
table entry.
The ESR_ELx.DFSC is set to synchronous external abort(0x10),
and ESR_ELx.FnV is set to not valid(0x1), which will tell
guest that FAR is not valid and holds an UNKNOWN value.
These value will be set to KVM register structures through
KVM_SET_ONE_REG IOCTL.
Signed-off-by: Dongjiu Geng <gengdongjiu@huawei.com>
---
Marc is against that KVM inject the synchronous external abort(SEA) in [1],
so user space how to inject it. The test result that injection SEA to guest by Qemu
is shown in [2].
[1]: https://lkml.org/lkml/2017/3/2/110
[2]:
Taking exception 4 [Data Abort]
...from EL0 to EL1
...with ESR 0x24/0x92000410
...with FAR 0x0
...with ELR 0x40cf04
...to EL1 PC 0xffffffc000084c00 PSTATE 0x3c5
after kvm_inject_arm_sea
Unhandled fault: synchronous external abort (0x92000410) at 0x0000007fa234c12c
CPU: 0 PID: 536 Comm: devmem Not tainted 4.1.0+ #20
Hardware name: linux,dummy-virt (DT)
task: ffffffc019ab2b00 ti: ffffffc008134000 task.ti: ffffffc008134000
PC is at 0x40cf04
LR is at 0x40cdec
pc : [<000000000040cf04>] lr : [<000000000040cdec>] pstate: 60000000
sp : 0000007ff7b24130
x29: 0000007ff7b24260 x28: 0000000000000000
x27: 00000000000000ad x26: 000000000049c000
x25: 000000000048904b x24: 000000000049c000
x23: 0000000040600000 x22: 0000007ff7b243a0
x21: 0000000000000002 x20: 0000000000000000
x19: 0000000000000020 x18: 0000000000000000
x17: 000000000049c6d0 x16: 0000007fa22c85c0
x15: 0000000000005798 x14: 0000007fa2205f1c
x13: 0000007fa241ccb0 x12: 0000000000000137
x11: 0000000000000000 x10: 0000000000000000
x9 : 0000000000000000 x8 : 00000000000000de
x7 : 0000000000000000 x6 : 0000000000002000
x5 : 0000000040600000 x4 : 0000000000000003
x3 : 0000000000000001 x2 : 0000000000000000
x1 : 0000000000000000 x0 : 0000007fa2418000
---
target/arm/cpu.h | 2 ++
target/arm/helper.c | 23 +++++++++++++++++++++++
target/arm/internals.h | 5 +++--
target/arm/kvm64.c | 39 +++++++++++++++++++++++++++++++++++++++
target/arm/op_helper.c | 2 +-
5 files changed, 68 insertions(+), 3 deletions(-)
diff --git a/target/arm/cpu.h b/target/arm/cpu.h
index b5eff79..502507d 100644
--- a/target/arm/cpu.h
+++ b/target/arm/cpu.h
@@ -2331,6 +2331,8 @@ bool write_list_to_cpustate(ARMCPU *cpu);
*/
bool write_cpustate_to_list(ARMCPU *cpu);
+bool write_part_cpustate_to_list(ARMCPU *cpu, ptrdiff_t fieldoffset);
+
#define ARM_CPUID_TI915T 0x54029152
#define ARM_CPUID_TI925T 0x54029252
diff --git a/target/arm/helper.c b/target/arm/helper.c
index 9630193..df078ff 100644
--- a/target/arm/helper.c
+++ b/target/arm/helper.c
@@ -263,6 +263,29 @@ static bool raw_accessors_invalid(const ARMCPRegInfo *ri)
return true;
}
+bool write_part_cpustate_to_list(ARMCPU *cpu, ptrdiff_t fieldoffset)
+{
+ const ARMCPRegInfo *ri;
+ uint32_t regidx, i;
+
+ for (i = 0; i < cpu->cpreg_array_len; i++) {
+ regidx = kvm_to_cpreg_id(cpu->cpreg_indexes[i]);
+ ri = get_arm_cp_reginfo(cpu->cp_regs, regidx);
+ if (!ri) {
+ continue;
+ }
+
+ if (ri->type & ARM_CP_NO_RAW) {
+ continue;
+ }
+ if (ri->fieldoffset == fieldoffset) {
+ cpu->cpreg_values[i] = read_raw_cp_reg(&cpu->env, ri);
+ return true;
+ }
+ }
+ return false;
+}
+
bool write_cpustate_to_list(ARMCPU *cpu)
{
/* Write the coprocessor state from cpu->env to the (index,value) list. */
diff --git a/target/arm/internals.h b/target/arm/internals.h
index 6c2bb2d..04ea074 100644
--- a/target/arm/internals.h
+++ b/target/arm/internals.h
@@ -415,13 +415,14 @@ static inline uint32_t syn_insn_abort(int same_el, int ea, int s1ptw, int fsc)
| ARM_EL_IL | (ea << 9) | (s1ptw << 7) | fsc;
}
-static inline uint32_t syn_data_abort_no_iss(int same_el,
+static inline uint32_t syn_data_abort_no_iss(int same_el, int fnv,
int ea, int cm, int s1ptw,
int wnr, int fsc)
{
return (EC_DATAABORT << ARM_EL_EC_SHIFT) | (same_el << ARM_EL_EC_SHIFT)
| ARM_EL_IL
- | (ea << 9) | (cm << 8) | (s1ptw << 7) | (wnr << 6) | fsc;
+ | (fnv << 10) | (ea << 9) | (cm << 8) | (s1ptw << 7)
+ | (wnr << 6) | fsc;
}
static inline uint32_t syn_data_abort_with_iss(int same_el,
diff --git a/target/arm/kvm64.c b/target/arm/kvm64.c
index 5de8ff0..0ca2b29 100644
--- a/target/arm/kvm64.c
+++ b/target/arm/kvm64.c
@@ -594,6 +594,45 @@ int kvm_arm_cpreg_level(uint64_t regidx)
return KVM_PUT_RUNTIME_STATE;
}
+/* Inject synchronous external abort */
+static void kvm_inject_arm_sea(CPUState *c)
+{
+ ARMCPU *cpu = ARM_CPU(c);
+ CPUARMState *env = &cpu->env;
+ CPUClass *cc = CPU_GET_CLASS(c);
+ uint32_t esr;
+ int ret;
+
+ /* This exception is synchronous data abort */
+ c->exception_index = EXCP_DATA_ABORT;
+ /* Inject the exception to guest EL1 */
+ env->exception.target_el = 1;
+
+ /* Set the DFSC to synchronous external abort and set FnV to not valid,
+ * this will tell guest the FAR_ELx is UNKNOWN for this abort.
+ */
+
+ /* This exception comes from lower or current exception level. */
+ if (arm_current_el(env) == PSTATE_MODE_EL0t) {
+ esr = syn_data_abort_no_iss(0, 1, 0, 0, 0, 0, 0x10);
+ } else {
+ esr = syn_data_abort_no_iss(1, 1, 0, 0, 0, 0, 0x10);
+ }
+
+ env->exception.syndrome = esr;
+
+ cc->do_interrupt(c);
+ if (kvm_enabled()) {
+ /* write ESR_EL1 from cpustate to list*/
+ ret = write_part_cpustate_to_list(cpu,
+ offsetof(CPUARMState, cp15.esr_el[1]));
+ if (!ret) {
+ fprintf(stderr, "<%s> failed to set esr_el1\n", __func__);
+ abort();
+ }
+ }
+}
+
#define AARCH64_CORE_REG(x) (KVM_REG_ARM64 | KVM_REG_SIZE_U64 | \
KVM_REG_ARM_CORE | KVM_REG_ARM_CORE_REG(x))
diff --git a/target/arm/op_helper.c b/target/arm/op_helper.c
index 90741f6..3f1d656 100644
--- a/target/arm/op_helper.c
+++ b/target/arm/op_helper.c
@@ -109,7 +109,7 @@ static inline uint32_t merge_syn_data_abort(uint32_t template_syn,
* ISV field.
*/
if (!(template_syn & ARM_EL_ISV) || target_el != 2 || s1ptw) {
- syn = syn_data_abort_no_iss(same_el,
+ syn = syn_data_abort_no_iss(same_el, 0,
ea, 0, s1ptw, is_write, fsc);
} else {
/* Fields: IL, ISV, SAS, SSE, SRT, SF and AR come from the template
--
1.8.3.1
^ permalink raw reply related [flat|nested] 7+ messages in thread
* Re: [PATCH RESEND v15 08/10] target-arm: kvm64: inject synchronous External Abort
2018-11-08 10:29 ` [PATCH RESEND v15 08/10] target-arm: kvm64: inject synchronous External Abort Dongjiu Geng
@ 2018-11-20 15:07 ` Peter Maydell
0 siblings, 0 replies; 7+ messages in thread
From: Peter Maydell @ 2018-11-20 15:07 UTC (permalink / raw)
To: Dongjiu Geng
Cc: Eduardo Habkost, kvm-devel, Michael S. Tsirkin, Marc Zyngier,
Marcelo Tosatti, QEMU Developers, Shannon Zhao, Zheng Xiang,
qemu-arm, James Morse, Paolo Bonzini, Igor Mammedov,
Laszlo Ersek, Richard Henderson
On 8 November 2018 at 10:29, Dongjiu Geng <gengdongjiu@huawei.com> wrote:
> Add synchronous external abort injection logic, setup
> exception type and syndrome value. When switch to guest,
> guest will jump to the synchronous external abort vector
> table entry.
>
> The ESR_ELx.DFSC is set to synchronous external abort(0x10),
> and ESR_ELx.FnV is set to not valid(0x1), which will tell
> guest that FAR is not valid and holds an UNKNOWN value.
> These value will be set to KVM register structures through
> KVM_SET_ONE_REG IOCTL.
>
> Signed-off-by: Dongjiu Geng <gengdongjiu@huawei.com>
> ---
> Marc is against that KVM inject the synchronous external abort(SEA) in [1],
> so user space how to inject it. The test result that injection SEA to guest by Qemu
> is shown in [2].
>
> [1]: https://lkml.org/lkml/2017/3/2/110
> [2]:
> Taking exception 4 [Data Abort]
> ...from EL0 to EL1
> ...with ESR 0x24/0x92000410
> ...with FAR 0x0
> ...with ELR 0x40cf04
> ...to EL1 PC 0xffffffc000084c00 PSTATE 0x3c5
> after kvm_inject_arm_sea
> Unhandled fault: synchronous external abort (0x92000410) at 0x0000007fa234c12c
> CPU: 0 PID: 536 Comm: devmem Not tainted 4.1.0+ #20
> Hardware name: linux,dummy-virt (DT)
> task: ffffffc019ab2b00 ti: ffffffc008134000 task.ti: ffffffc008134000
> PC is at 0x40cf04
> LR is at 0x40cdec
> pc : [<000000000040cf04>] lr : [<000000000040cdec>] pstate: 60000000
> sp : 0000007ff7b24130
> x29: 0000007ff7b24260 x28: 0000000000000000
> x27: 00000000000000ad x26: 000000000049c000
> x25: 000000000048904b x24: 000000000049c000
> x23: 0000000040600000 x22: 0000007ff7b243a0
> x21: 0000000000000002 x20: 0000000000000000
> x19: 0000000000000020 x18: 0000000000000000
> x17: 000000000049c6d0 x16: 0000007fa22c85c0
> x15: 0000000000005798 x14: 0000007fa2205f1c
> x13: 0000007fa241ccb0 x12: 0000000000000137
> x11: 0000000000000000 x10: 0000000000000000
> x9 : 0000000000000000 x8 : 00000000000000de
> x7 : 0000000000000000 x6 : 0000000000002000
> x5 : 0000000040600000 x4 : 0000000000000003
> x3 : 0000000000000001 x2 : 0000000000000000
> x1 : 0000000000000000 x0 : 0000007fa2418000
> ---
> target/arm/cpu.h | 2 ++
> target/arm/helper.c | 23 +++++++++++++++++++++++
> target/arm/internals.h | 5 +++--
> target/arm/kvm64.c | 39 +++++++++++++++++++++++++++++++++++++++
> target/arm/op_helper.c | 2 +-
> 5 files changed, 68 insertions(+), 3 deletions(-)
>
> diff --git a/target/arm/cpu.h b/target/arm/cpu.h
> index b5eff79..502507d 100644
> --- a/target/arm/cpu.h
> +++ b/target/arm/cpu.h
> @@ -2331,6 +2331,8 @@ bool write_list_to_cpustate(ARMCPU *cpu);
> */
> bool write_cpustate_to_list(ARMCPU *cpu);
>
> +bool write_part_cpustate_to_list(ARMCPU *cpu, ptrdiff_t fieldoffset);
> +
> #define ARM_CPUID_TI915T 0x54029152
> #define ARM_CPUID_TI925T 0x54029252
>
> diff --git a/target/arm/helper.c b/target/arm/helper.c
> index 9630193..df078ff 100644
> --- a/target/arm/helper.c
> +++ b/target/arm/helper.c
> @@ -263,6 +263,29 @@ static bool raw_accessors_invalid(const ARMCPRegInfo *ri)
> return true;
> }
>
> +bool write_part_cpustate_to_list(ARMCPU *cpu, ptrdiff_t fieldoffset)
> +{
> + const ARMCPRegInfo *ri;
> + uint32_t regidx, i;
> +
> + for (i = 0; i < cpu->cpreg_array_len; i++) {
> + regidx = kvm_to_cpreg_id(cpu->cpreg_indexes[i]);
> + ri = get_arm_cp_reginfo(cpu->cp_regs, regidx);
> + if (!ri) {
> + continue;
> + }
> +
> + if (ri->type & ARM_CP_NO_RAW) {
> + continue;
> + }
> + if (ri->fieldoffset == fieldoffset) {
> + cpu->cpreg_values[i] = read_raw_cp_reg(&cpu->env, ri);
> + return true;
> + }
> + }
> + return false;
> +}
What is this about? Nothing else in QEMU needs to mess with the
cpustate synchronization. My first assumption is that you should
not need to do so either.
> +
> bool write_cpustate_to_list(ARMCPU *cpu)
> {
> /* Write the coprocessor state from cpu->env to the (index,value) list. */
> diff --git a/target/arm/internals.h b/target/arm/internals.h
> index 6c2bb2d..04ea074 100644
> --- a/target/arm/internals.h
> +++ b/target/arm/internals.h
> @@ -415,13 +415,14 @@ static inline uint32_t syn_insn_abort(int same_el, int ea, int s1ptw, int fsc)
> | ARM_EL_IL | (ea << 9) | (s1ptw << 7) | fsc;
> }
>
> -static inline uint32_t syn_data_abort_no_iss(int same_el,
> +static inline uint32_t syn_data_abort_no_iss(int same_el, int fnv,
> int ea, int cm, int s1ptw,
> int wnr, int fsc)
> {
> return (EC_DATAABORT << ARM_EL_EC_SHIFT) | (same_el << ARM_EL_EC_SHIFT)
> | ARM_EL_IL
> - | (ea << 9) | (cm << 8) | (s1ptw << 7) | (wnr << 6) | fsc;
> + | (fnv << 10) | (ea << 9) | (cm << 8) | (s1ptw << 7)
> + | (wnr << 6) | fsc;
> }
>
> static inline uint32_t syn_data_abort_with_iss(int same_el,
> diff --git a/target/arm/kvm64.c b/target/arm/kvm64.c
> index 5de8ff0..0ca2b29 100644
> --- a/target/arm/kvm64.c
> +++ b/target/arm/kvm64.c
> @@ -594,6 +594,45 @@ int kvm_arm_cpreg_level(uint64_t regidx)
> return KVM_PUT_RUNTIME_STATE;
> }
>
> +/* Inject synchronous external abort */
> +static void kvm_inject_arm_sea(CPUState *c)
> +{
> + ARMCPU *cpu = ARM_CPU(c);
> + CPUARMState *env = &cpu->env;
> + CPUClass *cc = CPU_GET_CLASS(c);
> + uint32_t esr;
> + int ret;
> +
> + /* This exception is synchronous data abort */
> + c->exception_index = EXCP_DATA_ABORT;
> + /* Inject the exception to guest EL1 */
> + env->exception.target_el = 1;
These comments don't tell us anything that the code does not.
> +
> + /* Set the DFSC to synchronous external abort and set FnV to not valid,
> + * this will tell guest the FAR_ELx is UNKNOWN for this abort.
> + */
> +
> + /* This exception comes from lower or current exception level. */
> + if (arm_current_el(env) == PSTATE_MODE_EL0t) {
arm_current_el() returns an integer; you should be comparing it to
0,1,2,3, not to PSTATE_MODE_* constants.
> + esr = syn_data_abort_no_iss(0, 1, 0, 0, 0, 0, 0x10);
> + } else {
> + esr = syn_data_abort_no_iss(1, 1, 0, 0, 0, 0, 0x10);
> + }
Better written as
bool same_el;
[...]
same_el = arm_current_el(env) == env->exception.target_el;
env->exception.syndrome = syn_data_abort_no_iss(same_el, 1, 0, 0,
0, 0, 0x10);
> +
> + env->exception.syndrome = esr;
> +
> + cc->do_interrupt(c);
You need to take the iothread lock before calling do_interrupt.
Compare the code in kvm_arm_handle_debug().
> + if (kvm_enabled()) {
How can we get here if KVM is not enabled ?
> + /* write ESR_EL1 from cpustate to list*/
> + ret = write_part_cpustate_to_list(cpu,
> + offsetof(CPUARMState, cp15.esr_el[1]));
> + if (!ret) {
> + fprintf(stderr, "<%s> failed to set esr_el1\n", __func__);
> + abort();
> + }
kvm_arm_handle_debug() doesn't need to do this, I don't understand
why this would be different.
> + }
> +}
> +
> #define AARCH64_CORE_REG(x) (KVM_REG_ARM64 | KVM_REG_SIZE_U64 | \
> KVM_REG_ARM_CORE | KVM_REG_ARM_CORE_REG(x))
>
> diff --git a/target/arm/op_helper.c b/target/arm/op_helper.c
> index 90741f6..3f1d656 100644
> --- a/target/arm/op_helper.c
> +++ b/target/arm/op_helper.c
> @@ -109,7 +109,7 @@ static inline uint32_t merge_syn_data_abort(uint32_t template_syn,
> * ISV field.
> */
> if (!(template_syn & ARM_EL_ISV) || target_el != 2 || s1ptw) {
> - syn = syn_data_abort_no_iss(same_el,
> + syn = syn_data_abort_no_iss(same_el, 0,
> ea, 0, s1ptw, is_write, fsc);
> } else {
> /* Fields: IL, ISV, SAS, SSE, SRT, SF and AR come from the template
> --
> 1.8.3.1
>
thanks
-- PMM
^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2018-11-26 17:25 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-11-21 14:33 [PATCH RESEND v15 08/10] target-arm: kvm64: inject synchronous External Abort gengdongjiu
2018-11-23 18:45 ` Peter Maydell
2018-11-24 7:14 ` gengdongjiu
2018-11-26 11:50 ` Alex Bennée
-- strict thread matches above, loose matches on Subject: below --
2018-11-26 17:25 gengdongjiu
2018-11-08 10:29 [PATCH RESEND v15 00/10] Add ARMv8 RAS virtualization support in QEMU Dongjiu Geng
2018-11-08 10:29 ` [PATCH RESEND v15 08/10] target-arm: kvm64: inject synchronous External Abort Dongjiu Geng
2018-11-20 15:07 ` Peter Maydell
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).