Re: [Qemu-devel] implementing architectural timers using QEMU timers

[Max Filippov <jcmvbkbc@gmail.com>]

On Mon, Jan 9, 2017 at 7:41 AM, Alex Bligh <alex@alex.org.uk> wrote:
>> On 9 Jan 2017, at 15:18, Max Filippov <jcmvbkbc@gmail.com> wrote:
>> I'm trying to reimplement xtensa CCOUNT (cycle counter) and
>> CCOMPARE (CCOUNT-based timer interrupts) using QEMU
>> timers. That is CCOUNT value is derived from the
>> QEMU_CLOCK_VIRTUAL clock and CCOMPARE interrupts are
>> generated from the QEMU_CLOCK_VIRTUAL timer callbacks.
>> The code is here:
>>  https://github.com/OSLL/qemu-xtensa/commits/xtensa-ccount
>>
>> I've got the following issues doing that:
>>
>> - in non-icount mode I can often read CCOUNT and get a value
>>  that is greater than programmed CCOMPARE value, which
>>  means that QEMU timer must have been fired at that point, but
>>  no sign of timer callback being called. That is timer callback
>>  invocation lags behind due time.
>>
>>  Is my understanding correct that there's no hard expectations
>>  that firing of QEMU timers will be correctly sequenced with
>>  readings of QEMU clock?
>
> My understanding is that the intent of the API contract is that
>
> * The timer will not fire earlier than the time requested (but
>   can fire later)
>
> * Timers on the same clock and context will fire in the order
>   of their expiry time (with the ordering being undefined in
>   the event the expiry times are equal).
>
> Whether in practice any users make use of the second guarantee
> above I don't know.

Ok.

>> - I thought that could be improved in -icount mode, so I tried that.
>>  It is better with -icount, but it's still not 100% accurate. That is
>>  I was able to observe guest reading QEMU clock value that is
>>  past QEMU timer deadline before that timer callback was
>>  invoked.
>>
>>  That sounds like a bug to me, is it?
>
> Hmm... that would be a bug if it were guaranteed that the
> guest reads are always perfectly in sync with the timer
> itself. I don't know whether that is the case.

I guess that's my intuitive expectation of the determinism
offered by the -icount. Because if QEMU_CLOCK_VIRTUAL
timers are allowed to lag they cannot be used for deterministic
events, right? If that's intentional, then the question is what
can be used to generate an event at specific virtual time?

> Even though I authored a large timer patch, I found the icount
> stuff confusing.
>
>> - when guest sets a timer and halts itself waiting for timer
>>  interrupt with waiti opcode QEMU behaviour is very strange with
>>  -icount: regardless of the programmed timeout QEMU waits for
>>  about a second before it delivers interrupt, and, AFAICT,
>>  interrupt delivery it is not caused by the corresponding CCOUNT
>>  timer. I was able to track this issue down to the
>>  qemu_clock_use_for_deadline function, i.e. always returning true
>>  'fixes' that unwanted delay, but looking around the timer code
>>  I've got the impression that that's not the correct fix.
>>
>>  Any suggestions on how to fix that?
>
> This could be someone using timerlistgroup_deadline_ns rather than
> qemu_clock_deadline_ns_all.

Well, main_loop_wait uses timerlistgroup_deadline_ns. If I only
change this place to take all timers into account it works without
the unwanted timeout. But still the following comment above the
qemu_clock_use_for_deadline makes me wonder if such change
is legitimate:

 * Determine whether a clock should be used for deadline
 * calculations. Some clocks, for instance vm_clock with
 * use_icount set, do not count in nanoseconds. Such clocks
 * are not used for deadline calculations, and are presumed
 * to interrupt any poll using qemu_notify/aio_notify
 * etc.

> The comment at the top of qemu_clock_deadline_ns_all says:
>
> /* Calculate the soonest deadline across all timerlists attached
>  * to the clock. This is used for the icount timeout so we
>  * ignore whether or not the clock should be used in deadline
>  * calculations.
>  */
>
> I'm wondering whether there is something up with that logic.

-- 
Thanks.
-- Max