Re: [RFC PATCH 01/11] target/riscv: Add CLIC CSR mintstatus

[Alistair Francis <alistair23@gmail.com>]

On Fri, Jul 2, 2021 at 4:11 PM LIU Zhiwei <zhiwei_liu@c-sky.com> wrote:
>
>
> On 2021/7/2 下午1:38, Alistair Francis wrote:
> > On Thu, Jul 1, 2021 at 6:45 PM Frank Chang <frank.chang@sifive.com> wrote:
> >> LIU Zhiwei <zhiwei_liu@c-sky.com> 於 2021年4月20日 週二 上午8:49寫道：
> >>>
> >>> On 2021/4/20 上午7:23, Alistair Francis wrote:
> >>>> On Fri, Apr 9, 2021 at 5:52 PM LIU Zhiwei <zhiwei_liu@c-sky.com> wrote:
> >>>>> CSR mintstatus holds the active interrupt level for each supported
> >>>>> privilege mode. sintstatus, and user, uintstatus, provide restricted
> >>>>> views of mintstatus.
> >>>>>
> >>>>> Signed-off-by: LIU Zhiwei <zhiwei_liu@c-sky.com>
> >>>>> ---
> >>>>>    target/riscv/cpu.h      |  2 ++
> >>>>>    target/riscv/cpu_bits.h | 11 +++++++++++
> >>>>>    target/riscv/csr.c      | 26 ++++++++++++++++++++++++++
> >>>>>    3 files changed, 39 insertions(+)
> >>>>>
> >>>>> diff --git a/target/riscv/cpu.h b/target/riscv/cpu.h
> >>>>> index 0a33d387ba..1a44ca62c7 100644
> >>>>> --- a/target/riscv/cpu.h
> >>>>> +++ b/target/riscv/cpu.h
> >>>>> @@ -159,6 +159,7 @@ struct CPURISCVState {
> >>>>>        target_ulong mip;
> >>>>>
> >>>>>        uint32_t miclaim;
> >>>>> +    uint32_t mintstatus; /* clic-spec */
> >>>>>
> >>>>>        target_ulong mie;
> >>>>>        target_ulong mideleg;
> >>>>> @@ -243,6 +244,7 @@ struct CPURISCVState {
> >>>>>
> >>>>>        /* Fields from here on are preserved across CPU reset. */
> >>>>>        QEMUTimer *timer; /* Internal timer */
> >>>>> +    void *clic;       /* clic interrupt controller */
> >>>> This should be the CLIC type.
> >>> OK.
> >>>
> >>> Actually there are many versions of CLIC in my branch as different
> >>> devices. But it is better to use CLIC type for the upstream version.
> >>
> >> Hi Alistair and Zhiwei,
> >>
> >> Replacing void *clic with RISCVCLICState *clic may create a circular loop
> >> because CPURISCVState is also referenced in riscv_clic.h.
> >>
> >> However, I would like to ask what is the best approach to add
> >> the reference of CLIC device in CPURISCVState struct?
> >>
> >> There may be different kinds of CLIC devices.
> >> AFAK, there was another RFC patchset trying to add void *eclic
> >> for Nuclei processor into CPURISCVState struct:
> >> https://patchwork.kernel.org/project/qemu-devel/patch/20210507081654.11056-2-wangjunqiang@iscas.ac.cn/
> >>
> >> Is it okay to add the device reference directly into CPURISCVState struct like that,
> >> or we should create some abstraction for these CLIC devices?
> >> (However, I'm not sure how big the differences are for these CLIC devices...)
> > I would prefer to not have the CLIC in the struct at all.
> >
> > Why is the CLIC required from the CPU?
>
> In my opinion,   the tight coupled interrupt controller, like NVIC in
> ARM,  is much different from other devices.
> CPU harts need to communicate with it through many ways.

Agreed.

The difference with RISC-V is that we already have multiple tightly
coupled interrupt controllers. We have the PLIC, CLIC, eCLIC and AIA,
not to mention possible vendor controllers. Do we really need a CPU
struct entry for every single one?

I would like to try and keep all of that logic outside of the CPU
state. It might not be possible (at least without being a mess) in
which case that's fine, but it's at least worth considering.

>
> We can store the CLIC instance in the struct CPURISCVState or a global
> variable.  Is that any other way?

We could split the device. So for example the CSRs and other parts
that relate to the CPU could be in the CPU while the register mappings
and GPIO lines could be it's own device.

Another option is to use GPIO lines to indicate the status, but for
anything too complex that will be messy.

>
> > I'm guessing we at least need it for CLIC CSR accesses. Could we
> > handle that in the CPU and avoid needing a reference to the CLIC?
> CSR access is one case. Other cases are:
>
> 1. When process an interrupt and decide ISP address,  we need to know
> current interrupt is vectored or not.
>
> 2.  When interrupt returned, we need to choose another interrupt to
> serve, so that it will not miss any interrupt.

Thanks!

Alistair

>
> Thanks,
> Zhiwei
> >
> > Alistair
> >
> >> Thanks,
> >> Frank Chang
> >>
> >>>
> >>>>>    };
> >>>>>
> >>>>>    OBJECT_DECLARE_TYPE(RISCVCPU, RISCVCPUClass,
> >>>>> diff --git a/target/riscv/cpu_bits.h b/target/riscv/cpu_bits.h
> >>>>> index caf4599207..c4ce6ec3d9 100644
> >>>>> --- a/target/riscv/cpu_bits.h
> >>>>> +++ b/target/riscv/cpu_bits.h
> >>>>> @@ -165,6 +165,7 @@
> >>>>>    #define CSR_MCAUSE          0x342
> >>>>>    #define CSR_MTVAL           0x343
> >>>>>    #define CSR_MIP             0x344
> >>>>> +#define CSR_MINTSTATUS      0x346 /* clic-spec-draft */
> >>>>>
> >>>>>    /* Legacy Machine Trap Handling (priv v1.9.1) */
> >>>>>    #define CSR_MBADADDR        0x343
> >>>>> @@ -183,6 +184,7 @@
> >>>>>    #define CSR_SCAUSE          0x142
> >>>>>    #define CSR_STVAL           0x143
> >>>>>    #define CSR_SIP             0x144
> >>>>> +#define CSR_SINTSTATUS      0x146 /* clic-spec-draft */
> >>>>>
> >>>>>    /* Legacy Supervisor Trap Handling (priv v1.9.1) */
> >>>>>    #define CSR_SBADADDR        0x143
> >>>>> @@ -585,6 +587,15 @@
> >>>>>    #define SIP_STIP                           MIP_STIP
> >>>>>    #define SIP_SEIP                           MIP_SEIP
> >>>>>
> >>>>> +/* mintstatus */
> >>>>> +#define MINTSTATUS_MIL                     0xff000000 /* mil[7:0] */
> >>>>> +#define MINTSTATUS_SIL                     0x0000ff00 /* sil[7:0] */
> >>>>> +#define MINTSTATUS_UIL                     0x000000ff /* uil[7:0] */
> >>>>> +
> >>>>> +/* sintstatus */
> >>>>> +#define SINTSTATUS_SIL                     0x0000ff00 /* sil[7:0] */
> >>>>> +#define SINTSTATUS_UIL                     0x000000ff /* uil[7:0] */
> >>>> The bit fields in the comments are out of date.
> >>> I didn't notice it.   Fix it in next version.
> >>>
> >>> Thanks.
> >>>
> >>> Zhiwei
> >>>
> >>>> Alistair
> >>>>
> >>>>> +
> >>>>>    /* MIE masks */
> >>>>>    #define MIE_SEIE                           (1 << IRQ_S_EXT)
> >>>>>    #define MIE_UEIE                           (1 << IRQ_U_EXT)
> >>>>> diff --git a/target/riscv/csr.c b/target/riscv/csr.c
> >>>>> index d2585395bf..320b18ab60 100644
> >>>>> --- a/target/riscv/csr.c
> >>>>> +++ b/target/riscv/csr.c
> >>>>> @@ -188,6 +188,12 @@ static int pmp(CPURISCVState *env, int csrno)
> >>>>>    {
> >>>>>        return -!riscv_feature(env, RISCV_FEATURE_PMP);
> >>>>>    }
> >>>>> +
> >>>>> +static int clic(CPURISCVState *env, int csrno)
> >>>>> +{
> >>>>> +    return !!env->clic;
> >>>>> +}
> >>>>> +
> >>>>>    #endif
> >>>>>
> >>>>>    /* User Floating-Point CSRs */
> >>>>> @@ -734,6 +740,12 @@ static int rmw_mip(CPURISCVState *env, int csrno, target_ulong *ret_value,
> >>>>>        return 0;
> >>>>>    }
> >>>>>
> >>>>> +static int read_mintstatus(CPURISCVState *env, int csrno, target_ulong *val)
> >>>>> +{
> >>>>> +    *val = env->mintstatus;
> >>>>> +    return 0;
> >>>>> +}
> >>>>> +
> >>>>>    /* Supervisor Trap Setup */
> >>>>>    static int read_sstatus(CPURISCVState *env, int csrno, target_ulong *val)
> >>>>>    {
> >>>>> @@ -893,6 +905,13 @@ static int rmw_sip(CPURISCVState *env, int csrno, target_ulong *ret_value,
> >>>>>        return ret;
> >>>>>    }
> >>>>>
> >>>>> +static int read_sintstatus(CPURISCVState *env, int csrno, target_ulong *val)
> >>>>> +{
> >>>>> +    target_ulong mask = SINTSTATUS_SIL | SINTSTATUS_UIL;
> >>>>> +    *val = env->mintstatus & mask;
> >>>>> +    return 0;
> >>>>> +}
> >>>>> +
> >>>>>    /* Supervisor Protection and Translation */
> >>>>>    static int read_satp(CPURISCVState *env, int csrno, target_ulong *val)
> >>>>>    {
> >>>>> @@ -1644,5 +1663,12 @@ riscv_csr_operations csr_ops[CSR_TABLE_SIZE] = {
> >>>>>        [CSR_MHPMCOUNTER29H] = { "mhpmcounter29h", any32,  read_zero },
> >>>>>        [CSR_MHPMCOUNTER30H] = { "mhpmcounter30h", any32,  read_zero },
> >>>>>        [CSR_MHPMCOUNTER31H] = { "mhpmcounter31h", any32,  read_zero },
> >>>>> +
> >>>>> +    /* Machine Mode Core Level Interrupt Controller */
> >>>>> +    [CSR_MINTSTATUS] = { "mintstatus", clic,  read_mintstatus },
> >>>>> +
> >>>>> +    /* Supervisor Mode Core Level Interrupt Controller */
> >>>>> +    [CSR_SINTSTATUS] = { "sintstatus", clic,  read_sintstatus },
> >>>>> +
> >>>>>    #endif /* !CONFIG_USER_ONLY */
> >>>>>    };
> >>>>> --
> >>>>> 2.25.1
> >>>>>
> >>>>>