[Qemu-devel] [PATCH] LEON3 IRQMP: Fix IRQ software ack

[Qemu-devel] [PATCH] LEON3 IRQMP: Fix IRQ software ack

[Jean-Christophe Dubois]

With the LEON3 IRQ controller IRQs can be acknoledged 2 ways:
* Explicitely by software writing to the CLEAR_OFFSET register
* Implicitely when the procesor is done running the trap handler attached
  to the IRQ.

The actual IRQMP code only allows the implicit processor triggered IRQ ack.
If software write explicitely to the CLEAR_OFFSET register, this will clear
the pending bit in the register value but this will not lower the onloing
raised IRQ with the processor. The IRQ will be kept raised to the LEON
processor until the related trap handler is run and the processor implicitely
ack the interrupt. So with the actual IRQMP code trap hadler have to be run
even if the software has already done its job by clearing the pending bit.

This feature has been tested on another LEON3 simulator (tsim_leon3 from
Gaisler) and it turns out that the Qemu implementation is not equivalent to
the tsim one. In tsim, if software does clear a pending interrupt before
the related interrupt handler is triggered the said interrupt handler will
not be called.

This patch bring the Qemu IRQMP implementation in line with the tsim
implementation by allowing IRQ to be acknoledged by software only.

Signed-off-by: Jean-Christophe Dubois <jcd@tribudubois.net>
---
 hw/intc/grlib_irqmp.c | 17 +++++++++++------
 1 file changed, 11 insertions(+), 6 deletions(-)

diff --git a/hw/intc/grlib_irqmp.c b/hw/intc/grlib_irqmp.c
index 94659ee256..d6f9cb3692 100644
--- a/hw/intc/grlib_irqmp.c
+++ b/hw/intc/grlib_irqmp.c
@@ -106,6 +106,15 @@ static void grlib_irqmp_check_irqs(IRQMPState *state)
     }
 }
 
+static void grlib_irqmp_ack_mask(IRQMPState *state, uint32_t mask)
+{
+    /* Clear registers */
+    state->pending  &= ~mask;
+    state->force[0] &= ~mask; /* Only CPU 0 (No SMP support) */
+
+    grlib_irqmp_check_irqs(state);
+}
+
 void grlib_irqmp_ack(DeviceState *dev, int intno)
 {
     IRQMP        *irqmp = GRLIB_IRQMP(dev);
@@ -120,11 +129,7 @@ void grlib_irqmp_ack(DeviceState *dev, int intno)
 
     trace_grlib_irqmp_ack(intno);
 
-    /* Clear registers */
-    state->pending  &= ~mask;
-    state->force[0] &= ~mask; /* Only CPU 0 (No SMP support) */
-
-    grlib_irqmp_check_irqs(state);
+    grlib_irqmp_ack_mask(state, mask);
 }
 
 void grlib_irqmp_set_irq(void *opaque, int irq, int level)
@@ -251,7 +256,7 @@ static void grlib_irqmp_write(void *opaque, hwaddr addr,
 
     case CLEAR_OFFSET:
         value &= ~1; /* clean up the value */
-        state->pending &= ~value;
+        grlib_irqmp_ack_mask(state, value);
         return;
 
     case MP_STATUS_OFFSET:
-- 
2.14.1

Re: [Qemu-devel] [PATCH] LEON3 IRQMP: Fix IRQ software ack

[Fabien Chouteau]

On 10/01/2018 21:43, Jean-Christophe Dubois wrote:
> With the LEON3 IRQ controller IRQs can be acknoledged 2 ways:
> * Explicitely by software writing to the CLEAR_OFFSET register
> * Implicitely when the procesor is done running the trap handler attached
>   to the IRQ.
> 

Thanks Jean-Christophe,

I tested the patch with our software and it works.

Reviewed-by: Fabien Chouteau <chouteau@adacore.com>

Re: [Qemu-devel] [PATCH] LEON3 IRQMP: Fix IRQ software ack

[Jean-Christophe Dubois]

Thanks Fabien,

Now, as a side question, could you tell me which reference LEON3 
platform is implemented by Qemu in leon3_generic?

It doesn't seem to match the one emulated by tsim.

Thanks.

JC

Le 2018-01-11 12:48, Fabien Chouteau a écrit :
> On 10/01/2018 21:43, Jean-Christophe Dubois wrote:
>> With the LEON3 IRQ controller IRQs can be acknoledged 2 ways:
>> * Explicitely by software writing to the CLEAR_OFFSET register
>> * Implicitely when the procesor is done running the trap handler 
>> attached
>>   to the IRQ.
>> 
> 
> Thanks Jean-Christophe,
> 
> I tested the patch with our software and it works.
> 
> Reviewed-by: Fabien Chouteau <chouteau@adacore.com>

Re: [Qemu-devel] [PATCH] LEON3 IRQMP: Fix IRQ software ack

[Fabien Chouteau]

On 11/01/2018 13:35, Jean-Christophe Dubois wrote:
> Thanks Fabien,
> 
> Now, as a side question, could you tell me which reference LEON3 platform is implemented by Qemu in leon3_generic?
> 

I think it was the based on the FPGA version of Leon3 I was using at the
time. The name leon3_generic comes from my will to make it a
configurable board where users could define the number and the location
of the different peripherals, I never had time to work on this.

Re: [Qemu-devel] [PATCH] LEON3 IRQMP: Fix IRQ software ack

[Jean-Christophe Dubois]

Le 2018-01-12 11:55, Fabien Chouteau a écrit :
> On 11/01/2018 13:35, Jean-Christophe Dubois wrote:
>> Thanks Fabien,
>> 
>> Now, as a side question, could you tell me which reference LEON3 
>> platform is implemented by Qemu in leon3_generic?
>> 
> 
> I think it was the based on the FPGA version of Leon3 I was using at 
> the
> time. The name leon3_generic comes from my will to make it a
> configurable board where users could define the number and the location
> of the different peripherals, I never had time to work on this.

I see. I am not sure how to bring configurability to Qemu. There is the 
possibility to describe the hw PTF with DTC/DTB or something similar. I 
think some people were working on it for the ARM Qemu platform (but I am 
not sure what happened to this initiative).

Now in the meantime, would it make sense to move leon3_generic to a tsim 
compatible platform?

This would allow to validate the same software on the 2 simulators 
(obviously it would not be compatible with your specific FPGA version 
for now).

JC

Re: [Qemu-devel] [PATCH] LEON3 IRQMP: Fix IRQ software ack

[Fabien Chouteau]

On 12/01/2018 15:10, Jean-Christophe Dubois wrote:
> Le 2018-01-12 11:55, Fabien Chouteau a écrit :
>> On 11/01/2018 13:35, Jean-Christophe Dubois wrote:
>>> Thanks Fabien,
>>>
>>> Now, as a side question, could you tell me which reference LEON3 platform is implemented by Qemu in leon3_generic?
>>>
>>
>> I think it was the based on the FPGA version of Leon3 I was using at the
>> time. The name leon3_generic comes from my will to make it a
>> configurable board where users could define the number and the location
>> of the different peripherals, I never had time to work on this.
> 
> I see. I am not sure how to bring configurability to Qemu. There is the possibility to describe the hw PTF with DTC/DTB or something similar. I think some people were working on it for the ARM Qemu platform (but I am not sure what happened to this initiative).
> 
> Now in the meantime, would it make sense to move leon3_generic to a tsim compatible platform?
> 

I don't think so, leon3_generic is compatible with a real hardware which
is also interesting for comparison.

> This would allow to validate the same software on the 2 simulators (obviously it would not be compatible with your specific FPGA version for now).
>

The Leon3 AMBA bus provides a way to discover the peripherals and their
address, so any system should be capable of supporting different
peripheral layouts.

Here's an example of AMBA discovery code from a very old project of mine
(don't judge me on this :) :
https://github.com/Fabien-Chouteau/kabitbol/blob/master/src/amba.c

There was a couple of patches submitted some times ago to add Leon3 AMBA
support in QEMU, I think it's time to bring them back...

Re: [Qemu-devel] [PATCH] LEON3 IRQMP: Fix IRQ software ack

[Jean-Christophe Dubois]

Le 2018-01-15 12:09, Fabien Chouteau a écrit :
> On 12/01/2018 15:10, Jean-Christophe Dubois wrote:
>> Le 2018-01-12 11:55, Fabien Chouteau a écrit :
>>> On 11/01/2018 13:35, Jean-Christophe Dubois wrote:
>>>> Thanks Fabien,
>>>> 
>>>> Now, as a side question, could you tell me which reference LEON3 
>>>> platform is implemented by Qemu in leon3_generic?
>>>> 
>>> 
>>> I think it was the based on the FPGA version of Leon3 I was using at 
>>> the
>>> time. The name leon3_generic comes from my will to make it a
>>> configurable board where users could define the number and the 
>>> location
>>> of the different peripherals, I never had time to work on this.
>> 
>> I see. I am not sure how to bring configurability to Qemu. There is 
>> the possibility to describe the hw PTF with DTC/DTB or something 
>> similar. I think some people were working on it for the ARM Qemu 
>> platform (but I am not sure what happened to this initiative).
>> 
>> Now in the meantime, would it make sense to move leon3_generic to a 
>> tsim compatible platform?
>> 
> 
> I don't think so, leon3_generic is compatible with a real hardware 
> which
> is also interesting for comparison.

What real hardware (gaisler reference platform) is it? Could you point 
to the public reference manual for this hardware?

Another possibility is to add a leon3_tsim platform into Qemu to support 
the compatibility with the tsim emulator. But the difference with 
leon3_generic would be minimal.

> 
>> This would allow to validate the same software on the 2 simulators 
>> (obviously it would not be compatible with your specific FPGA version 
>> for now).
>> 
> 
> The Leon3 AMBA bus provides a way to discover the peripherals and their
> address, so any system should be capable of supporting different
> peripheral layouts.
> 
> Here's an example of AMBA discovery code from a very old project of 
> mine
> (don't judge me on this :) :
> https://github.com/Fabien-Chouteau/kabitbol/blob/master/src/amba.c
> 
> There was a couple of patches submitted some times ago to add Leon3 
> AMBA
> support in QEMU, I think it's time to bring them back...

What you are talking about here is the possibility for the software 
running inside Qemu to probe/discover the hardware. For such feature 
Qemu should implement what is required for this AMBA discovery. But it 
does not solve how you decide at run time at what addresses are the 
various AMBA devices (how you decide to emulate a tsim platform or 
another one).

Note: For now I am not so much interested in the AMBA discovery as the 
type of software platform I am thinking about is embedded where the 
hardware is well known ahead of time. This discovery capability would 
make sense for more generic OS like linux or such. We cannot require all 
embedded OS to implement the AMBA discovery process.

Re: [Qemu-devel] [PATCH] LEON3 IRQMP: Fix IRQ software ack

[Jean-Christophe Dubois]

Le 2018-01-15 14:45, Jean-Christophe Dubois a écrit :
> Le 2018-01-15 12:09, Fabien Chouteau a écrit :
>> On 12/01/2018 15:10, Jean-Christophe Dubois wrote:
>>> Le 2018-01-12 11:55, Fabien Chouteau a écrit :
>>>> On 11/01/2018 13:35, Jean-Christophe Dubois wrote:
>>>>> Thanks Fabien,
>>>>> 
>>>>> Now, as a side question, could you tell me which reference LEON3 
>>>>> platform is implemented by Qemu in leon3_generic?
>>>>> 
>>>> 
>>>> I think it was the based on the FPGA version of Leon3 I was using at 
>>>> the
>>>> time. The name leon3_generic comes from my will to make it a
>>>> configurable board where users could define the number and the 
>>>> location
>>>> of the different peripherals, I never had time to work on this.
>>> 
>>> I see. I am not sure how to bring configurability to Qemu. There is 
>>> the possibility to describe the hw PTF with DTC/DTB or something 
>>> similar. I think some people were working on it for the ARM Qemu 
>>> platform (but I am not sure what happened to this initiative).
>>> 
>>> Now in the meantime, would it make sense to move leon3_generic to a 
>>> tsim compatible platform?
>>> 
>> 
>> I don't think so, leon3_generic is compatible with a real hardware 
>> which
>> is also interesting for comparison.
> 
> What real hardware (gaisler reference platform) is it? Could you point
> to the public reference manual for this hardware?
> 
> Another possibility is to add a leon3_tsim platform into Qemu to
> support the compatibility with the tsim emulator. But the difference
> with leon3_generic would be minimal.
> 
>> 
>>> This would allow to validate the same software on the 2 simulators 
>>> (obviously it would not be compatible with your specific FPGA version 
>>> for now).
>>> 
>> 
>> The Leon3 AMBA bus provides a way to discover the peripherals and 
>> their
>> address, so any system should be capable of supporting different
>> peripheral layouts.
>> 
>> Here's an example of AMBA discovery code from a very old project of 
>> mine
>> (don't judge me on this :) :
>> https://github.com/Fabien-Chouteau/kabitbol/blob/master/src/amba.c
>> 
>> There was a couple of patches submitted some times ago to add Leon3 
>> AMBA
>> support in QEMU, I think it's time to bring them back...
> 
> What you are talking about here is the possibility for the software
> running inside Qemu to probe/discover the hardware. For such feature
> Qemu should implement what is required for this AMBA discovery. But it
> does not solve how you decide at run time at what addresses are the
> various AMBA devices (how you decide to emulate a tsim platform or
> another one).
> 
> Note: For now I am not so much interested in the AMBA discovery as the
> type of software platform I am thinking about is embedded where the
> hardware is well known ahead of time. This discovery capability would
> make sense for more generic OS like linux or such. We cannot require
> all embedded OS to implement the AMBA discovery process.

For the configurability of Qemu, I was thinking of using something 
similar to the Qemu provided by Xilinx 
(http://www.wiki.xilinx.com/QEMU). Basically, you provide a DTB file as 
a Qemu command line argument and Qemu will build the various devices 
(including addresses and interrupts) based on the content of this file. 
Then when running an OS, it can provide the DTB file (for example to 
Linux) that match exactly the emulated platform.

This should allow to build "any" variation of the platform and to add 
devices as you need them. This makes sense for Xilinx (their customer 
are building custom platforms) and it would also make sense for LEON as 
the CPU core is usually integrated inside a custom SOC/FPGA.

JC

Re: [Qemu-devel] [PATCH] LEON3 IRQMP: Fix IRQ software ack

[Fabien Chouteau]

On 15/01/2018 18:27, Jean-Christophe Dubois wrote:
> Le 2018-01-15 14:45, Jean-Christophe Dubois a écrit :
>>
>> Note: For now I am not so much interested in the AMBA discovery as the
>> type of software platform I am thinking about is embedded where the
>> hardware is well known ahead of time. This discovery capability would
>> make sense for more generic OS like linux or such. We cannot require
>> all embedded OS to implement the AMBA discovery process.
> 
> For the configurability of Qemu, I was thinking of using something similar to the Qemu provided by Xilinx (http://www.wiki.xilinx.com/QEMU). Basically, you provide a DTB file as a Qemu command line argument and Qemu will build the various devices (including addresses and interrupts) based on the content of this file. Then when running an OS, it can provide the DTB file (for example to Linux) that match exactly the emulated platform.
> 
> This should allow to build "any" variation of the platform and to add devices as you need them. This makes sense for Xilinx (their customer are building custom platforms) and it would also make sense for LEON as the CPU core is usually integrated inside a custom SOC/FPGA.
> 

Makes sense, another intermediate solution is to add an argument to the
function leon3_generic_hw_init(), A struct that specifies the address of
the peripherals. That way it will be easy to add more machine definition
without bringing more maintenance burden.

Re: [Qemu-devel] [PATCH] LEON3 IRQMP: Fix IRQ software ack

[Alistair Francis]

On Mon, Jan 15, 2018 at 9:27 AM, Jean-Christophe Dubois
<jcd@tribudubois.net> wrote:
> Le 2018-01-15 14:45, Jean-Christophe Dubois a écrit :
>>
>> Le 2018-01-15 12:09, Fabien Chouteau a écrit :
>>>
>>> On 12/01/2018 15:10, Jean-Christophe Dubois wrote:
>>>>
>>>> Le 2018-01-12 11:55, Fabien Chouteau a écrit :
>>>>>
>>>>> On 11/01/2018 13:35, Jean-Christophe Dubois wrote:
>>>>>>
>>>>>> Thanks Fabien,
>>>>>>
>>>>>> Now, as a side question, could you tell me which reference LEON3
>>>>>> platform is implemented by Qemu in leon3_generic?
>>>>>>
>>>>>
>>>>> I think it was the based on the FPGA version of Leon3 I was using at
>>>>> the
>>>>> time. The name leon3_generic comes from my will to make it a
>>>>> configurable board where users could define the number and the location
>>>>> of the different peripherals, I never had time to work on this.
>>>>
>>>>
>>>> I see. I am not sure how to bring configurability to Qemu. There is the
>>>> possibility to describe the hw PTF with DTC/DTB or something similar. I
>>>> think some people were working on it for the ARM Qemu platform (but I am not
>>>> sure what happened to this initiative).
>>>>
>>>> Now in the meantime, would it make sense to move leon3_generic to a tsim
>>>> compatible platform?
>>>>
>>>
>>> I don't think so, leon3_generic is compatible with a real hardware which
>>> is also interesting for comparison.
>>
>>
>> What real hardware (gaisler reference platform) is it? Could you point
>> to the public reference manual for this hardware?
>>
>> Another possibility is to add a leon3_tsim platform into Qemu to
>> support the compatibility with the tsim emulator. But the difference
>> with leon3_generic would be minimal.
>>
>>>
>>>> This would allow to validate the same software on the 2 simulators
>>>> (obviously it would not be compatible with your specific FPGA version for
>>>> now).
>>>>
>>>
>>> The Leon3 AMBA bus provides a way to discover the peripherals and their
>>> address, so any system should be capable of supporting different
>>> peripheral layouts.
>>>
>>> Here's an example of AMBA discovery code from a very old project of mine
>>> (don't judge me on this :) :
>>> https://github.com/Fabien-Chouteau/kabitbol/blob/master/src/amba.c
>>>
>>> There was a couple of patches submitted some times ago to add Leon3 AMBA
>>> support in QEMU, I think it's time to bring them back...
>>
>>
>> What you are talking about here is the possibility for the software
>> running inside Qemu to probe/discover the hardware. For such feature
>> Qemu should implement what is required for this AMBA discovery. But it
>> does not solve how you decide at run time at what addresses are the
>> various AMBA devices (how you decide to emulate a tsim platform or
>> another one).
>>
>> Note: For now I am not so much interested in the AMBA discovery as the
>> type of software platform I am thinking about is embedded where the
>> hardware is well known ahead of time. This discovery capability would
>> make sense for more generic OS like linux or such. We cannot require
>> all embedded OS to implement the AMBA discovery process.
>
>
> For the configurability of Qemu, I was thinking of using something similar
> to the Qemu provided by Xilinx (http://www.wiki.xilinx.com/QEMU). Basically,
> you provide a DTB file as a Qemu command line argument and Qemu will build
> the various devices (including addresses and interrupts) based on the
> content of this file. Then when running an OS, it can provide the DTB file
> (for example to Linux) that match exactly the emulated platform.
>
> This should allow to build "any" variation of the platform and to add
> devices as you need them. This makes sense for Xilinx (their customer are
> building custom platforms) and it would also make sense for LEON as the CPU
> core is usually integrated inside a custom SOC/FPGA.

This is a cool solution and works pretty well for us, but it requires
very complex code and is not easy to debug or maintain, just something
to keep in mind.

We also have a lot of trouble working with Linux device trees. For our
more complex ZynqMP SoC we have our own non-standard device trees as
the Linux ones don't include enough information.

Alistair

>
> JC
>

Re: [Qemu-devel] [PATCH] LEON3 IRQMP: Fix IRQ software ack

[Mark Cave-Ayland]

On 11/01/18 11:48, Fabien Chouteau wrote:

> On 10/01/2018 21:43, Jean-Christophe Dubois wrote:
>> With the LEON3 IRQ controller IRQs can be acknoledged 2 ways:
>> * Explicitely by software writing to the CLEAR_OFFSET register
>> * Implicitely when the procesor is done running the trap handler attached
>>    to the IRQ.
>>
> 
> Thanks Jean-Christophe,
> 
> I tested the patch with our software and it works.
> 
> Reviewed-by: Fabien Chouteau <chouteau@adacore.com>

Thanks Fabien. Does that mean you would like me to take this patch for 
my qemu-sparc branch?


ATB,

Mark.

Re: [Qemu-devel] [PATCH] LEON3 IRQMP: Fix IRQ software ack

[Fabien Chouteau]

On 15/01/2018 19:16, Mark Cave-Ayland wrote:
> On 11/01/18 11:48, Fabien Chouteau wrote:
> 
>> On 10/01/2018 21:43, Jean-Christophe Dubois wrote:
>>> With the LEON3 IRQ controller IRQs can be acknoledged 2 ways:
>>> * Explicitely by software writing to the CLEAR_OFFSET register
>>> * Implicitely when the procesor is done running the trap handler attached
>>>    to the IRQ.
>>>
>>
>> Thanks Jean-Christophe,
>>
>> I tested the patch with our software and it works.
>>
>> Reviewed-by: Fabien Chouteau <chouteau@adacore.com>
> 
> Thanks Fabien. Does that mean you would like me to take this patch for my qemu-sparc branch?
> 

Yes please :)

Should I use Acked-by?

Re: [Qemu-devel] [PATCH] LEON3 IRQMP: Fix IRQ software ack

[Mark Cave-Ayland]

On 16/01/18 10:21, Fabien Chouteau wrote:

> On 15/01/2018 19:16, Mark Cave-Ayland wrote:
>> On 11/01/18 11:48, Fabien Chouteau wrote:
>>
>>> On 10/01/2018 21:43, Jean-Christophe Dubois wrote:
>>>> With the LEON3 IRQ controller IRQs can be acknoledged 2 ways:
>>>> * Explicitely by software writing to the CLEAR_OFFSET register
>>>> * Implicitely when the procesor is done running the trap handler attached
>>>>     to the IRQ.
>>>>
>>>
>>> Thanks Jean-Christophe,
>>>
>>> I tested the patch with our software and it works.
>>>
>>> Reviewed-by: Fabien Chouteau <chouteau@adacore.com>
>>
>> Thanks Fabien. Does that mean you would like me to take this patch for my qemu-sparc branch?
>>
> 
> Yes please :)
> 
> Should I use Acked-by?

I already have your Reviewed-by from a previous email so that's fine 
with me.


ATB,

Mark.

Re: [Qemu-devel] [PATCH] LEON3 IRQMP: Fix IRQ software ack

[Mark Cave-Ayland]

On 11/01/18 11:48, Fabien Chouteau wrote:

> On 10/01/2018 21:43, Jean-Christophe Dubois wrote:
>> With the LEON3 IRQ controller IRQs can be acknoledged 2 ways:
>> * Explicitely by software writing to the CLEAR_OFFSET register
>> * Implicitely when the procesor is done running the trap handler attached
>>    to the IRQ.
>>
> 
> Thanks Jean-Christophe,
> 
> I tested the patch with our software and it works.
> 
> Reviewed-by: Fabien Chouteau <chouteau@adacore.com>

Thanks. I've applied this (with some minor touch-ups to the commit 
message) to my qemu-sparc branch.


ATB,

Mark.

Re: [Qemu-devel] [PATCH] LEON3 IRQMP: Fix IRQ software ack

[Fabien Chouteau]

On 24/01/2018 20:51, Mark Cave-Ayland wrote:
> On 11/01/18 11:48, Fabien Chouteau wrote:
> 
>> On 10/01/2018 21:43, Jean-Christophe Dubois wrote:
>>> With the LEON3 IRQ controller IRQs can be acknoledged 2 ways:
>>> * Explicitely by software writing to the CLEAR_OFFSET register
>>> * Implicitely when the procesor is done running the trap handler attached
>>>    to the IRQ.
>>>
>>
>> Thanks Jean-Christophe,
>>
>> I tested the patch with our software and it works.
>>
>> Reviewed-by: Fabien Chouteau <chouteau@adacore.com>
> 
> Thanks. I've applied this (with some minor touch-ups to the commit message) to my qemu-sparc branch.
> 

Thanks you Mark!