Re: [RFC PATCH v2 13/22] xen/arm: its: Add virtual ITS command support

[Vijay Kilari <vijay.kilari@gmail.com>]

On Thu, Apr 2, 2015 at 7:17 PM, Julien Grall <julien.grall.oss@gmail.com> wrote:
> Hi Ian,
>
> On 02/04/2015 12:18, Ian Campbell wrote:
>>
>> On Thu, 2015-04-02 at 12:06 +0100, Julien Grall wrote:
>>
>>>> Can we just enqueue with the hardware and use the guest vcpu polling
>>>> loop to trigger us to check for completion?
>>>
>>>
>>> Enqueue may be long. I was thinking about suggesting to use a tasklet
>>> for processing ITS command.
>>
>>
>> We don't need to enqueue everything the guest gives us at once, we could
>> only do a subset and pickup the rest later as things complete at the
>> physical ITS.
>
>
> That would require more tracking. Anyway, I think that would work.

Sorry for late reply. Could not get time to work on this.

Approach 1: (Using completion interrupt)
----------------
1) Create dummy device for each virtual ITS when virtual its is
created for a domain
    OR Allocate one interrupt number for each virtual ITS from the one
single dummy device

2) Trap on CWRITER
3) Take vits lock
4)     Read a Virtual Command from Virtual Command Queue
5)     Translate Virtual command to Physical command
5)  Release vits lock
6) Take physical ITS lock
7)      Post physical commands + Append INT command
8) Release physical ITS lock
9) Return from CWRITER trap and let VCPU poll in EL1 (kernel) for CREADER update
    which indicates completion of command.
10) On receiving interrupt, Update CREADER of this virtual ITS

Pros:
    - VCPU polls in EL1.

Cons:
    - Complexity in creating & managing dummy device.

Approach 2:
----------------
I thought of below approach where in we reduce locking time and no need
of Interrupt.

1) Trap on CWRITER
2) Take vits lock
3)     Read _all_ or 2 Virtual Commands from Virtual Command Queue
4)     Translate _all_ or 2 Virtual commands to Physical commands
5)  Release vits lock
6) Take physical ITS lock
7)      Post physical commands
8) Release physical ITS lock
9) Poll for completion of command and return from CWRITER trap.

Pros:
    - Simple approach
Cons:
    - VCPU polls in EL2

>
>> I would expect Xen itself to use the second option at the host level,
>> which would then drive the completion via the vGITS_CREADR or the
>> guest's virtualised interrupt.
>>
>> That means the pCPU is free during the ITS processing, which is surely
>> what we want.
>
>
> Right, that would be the best solution for Xen.
>
> Although, the would mean diverging from Linux driver (see discussion on
> patch #6). But I think it's inevitable we can't have the same driver close
> to Linux.
>
>>> A guest would be buggy if it doesn't implement one of this solution. And
>>> therefore may not run on real h/w.
>>
>>
>> I was more concerned about it wedging the hypervisor somehow with a
>> large number of completed but not released operations.
>
>
> We don't have to worry about released operations. Once it acknowledge (via
> the above completion notification) the command can be discard in the ITS
> driver.
>
> Regards,
>
> --
> Julien Grall