All of lore.kernel.org
 help / color / mirror / Atom feed
From: Peter Hilber <peter.hilber@opensynergy.com>
To: David Woodhouse <dwmw2@infradead.org>,
	Alexandre Belloni <alexandre.belloni@bootlin.com>
Cc: linux-kernel@vger.kernel.org, virtualization@lists.linux.dev,
	virtio-dev@lists.oasis-open.org,
	linux-arm-kernel@lists.infradead.org, linux-rtc@vger.kernel.org,
	"virtio-comment@lists.oasis-open.org"
	<virtio-comment@lists.oasis-open.org>,
	"Christopher S. Hall" <christopher.s.hall@intel.com>,
	Jason Wang <jasowang@redhat.com>,
	John Stultz <jstultz@google.com>,
	"Michael S. Tsirkin" <mst@redhat.com>,
	netdev@vger.kernel.org,
	Richard Cochran <richardcochran@gmail.com>,
	Stephen Boyd <sboyd@kernel.org>,
	Thomas Gleixner <tglx@linutronix.de>,
	Xuan Zhuo <xuanzhuo@linux.alibaba.com>,
	Marc Zyngier <maz@kernel.org>,
	Mark Rutland <mark.rutland@arm.com>,
	Daniel Lezcano <daniel.lezcano@linaro.org>,
	Alessandro Zummo <a.zummo@towertech.it>,
	"Ridoux, Julien" <ridouxj@amazon.com>
Subject: Re: [RFC PATCH v3 0/7] Add virtio_rtc module and related changes
Date: Wed, 13 Mar 2024 18:50:59 +0100	[thread overview]
Message-ID: <a0193f93-64cb-4935-b70f-aeee66eb6d44@opensynergy.com> (raw)
In-Reply-To: <96be7312f7bddaf06c690e082a8028fa8b511deb.camel@infradead.org>

On 13.03.24 15:06, David Woodhouse wrote:
> On Wed, 2024-03-13 at 13:58 +0100, Alexandre Belloni wrote:
>> The TSC or whatever CPU counter/clock that is used to keep the system
>> time is not an RTC, I don't get why it has to be exposed as such to the
>> guests. PTP is fine and precise, RTC is not.
> 
> Ah, I see. But the point of the virtio_rtc is not really to expose that
> CPU counter. The point is to report the wallclock time, just like an
> actual RTC. The real difference is the *precision*.
> 
> The virtio_rtc device has a facility to *also* expose the counter,
> because that's what we actually need to gain that precision...
> 
> Applications don't read the RTC every time they want to know what the
> time is. These days, they don't even make a system call; it's done
> entirely in userspace mode. The kernel exposes some shared memory,
> essentially saying "the counter was X at time Y, and runs at Z Hz".
> Then applications just read the CPU counter and do some arithmetic.
> 
> As we require more and more precision in the calibration, it becomes
> important to get *paired* readings of the CPU counter and the wallclock
> time at precisely the same moment. If the guest has to read one and
> then the other, potentially taking interrupts, getting preempted and
> suffering steal/SMI time in the middle, that introduces an error which
> is increasingly significant as we increasingly care about precision.
> 
> Peter's proposal exposes the pairs of {X,Y} and leaves *all* the guest
> kernels having to repeat readings over time and perform the calibration
> as the underlying hardware oscillator frequency (Z) drifts with
> temperature. I'm trying to get him to let the hypervisor expose the
> calibrated frequency Z too. Along with *error* bounds for ±δX and ±δZ.
> Which aside from reducing the duplication of effort, will *also* fix
> the problem of live migration where *all* those things suffer a step
> change and leave the guest with an inaccurate clock but not knowing it.

I am already convinced that this would work significantly better than the
{X,Y} pair (but would be a bit more effort to implement):

1. when accessed by user space, obviously

2. when backing the PTP clock, it saves CPU time and makes non-paired
   reads more precise.

I would just prefer to try upstreaming the {X,Y} pairing first. I think the
{X,Y,Z...} pairing could be discussed and developed in parallel.

Thanks for the comments,

Peter

WARNING: multiple messages have this Message-ID (diff)
From: Peter Hilber <peter.hilber@opensynergy.com>
To: David Woodhouse <dwmw2@infradead.org>,
	Alexandre Belloni <alexandre.belloni@bootlin.com>
Cc: linux-kernel@vger.kernel.org, virtualization@lists.linux.dev,
	virtio-dev@lists.oasis-open.org,
	linux-arm-kernel@lists.infradead.org, linux-rtc@vger.kernel.org,
	"virtio-comment@lists.oasis-open.org"
	<virtio-comment@lists.oasis-open.org>,
	"Christopher S. Hall" <christopher.s.hall@intel.com>,
	Jason Wang <jasowang@redhat.com>,
	John Stultz <jstultz@google.com>,
	"Michael S. Tsirkin" <mst@redhat.com>,
	netdev@vger.kernel.org,
	Richard Cochran <richardcochran@gmail.com>,
	Stephen Boyd <sboyd@kernel.org>,
	Thomas Gleixner <tglx@linutronix.de>,
	Xuan Zhuo <xuanzhuo@linux.alibaba.com>,
	Marc Zyngier <maz@kernel.org>,
	Mark Rutland <mark.rutland@arm.com>,
	Daniel Lezcano <daniel.lezcano@linaro.org>,
	Alessandro Zummo <a.zummo@towertech.it>,
	"Ridoux, Julien" <ridouxj@amazon.com>
Subject: Re: [RFC PATCH v3 0/7] Add virtio_rtc module and related changes
Date: Wed, 13 Mar 2024 18:50:59 +0100	[thread overview]
Message-ID: <a0193f93-64cb-4935-b70f-aeee66eb6d44@opensynergy.com> (raw)
In-Reply-To: <96be7312f7bddaf06c690e082a8028fa8b511deb.camel@infradead.org>

On 13.03.24 15:06, David Woodhouse wrote:
> On Wed, 2024-03-13 at 13:58 +0100, Alexandre Belloni wrote:
>> The TSC or whatever CPU counter/clock that is used to keep the system
>> time is not an RTC, I don't get why it has to be exposed as such to the
>> guests. PTP is fine and precise, RTC is not.
> 
> Ah, I see. But the point of the virtio_rtc is not really to expose that
> CPU counter. The point is to report the wallclock time, just like an
> actual RTC. The real difference is the *precision*.
> 
> The virtio_rtc device has a facility to *also* expose the counter,
> because that's what we actually need to gain that precision...
> 
> Applications don't read the RTC every time they want to know what the
> time is. These days, they don't even make a system call; it's done
> entirely in userspace mode. The kernel exposes some shared memory,
> essentially saying "the counter was X at time Y, and runs at Z Hz".
> Then applications just read the CPU counter and do some arithmetic.
> 
> As we require more and more precision in the calibration, it becomes
> important to get *paired* readings of the CPU counter and the wallclock
> time at precisely the same moment. If the guest has to read one and
> then the other, potentially taking interrupts, getting preempted and
> suffering steal/SMI time in the middle, that introduces an error which
> is increasingly significant as we increasingly care about precision.
> 
> Peter's proposal exposes the pairs of {X,Y} and leaves *all* the guest
> kernels having to repeat readings over time and perform the calibration
> as the underlying hardware oscillator frequency (Z) drifts with
> temperature. I'm trying to get him to let the hypervisor expose the
> calibrated frequency Z too. Along with *error* bounds for ±δX and ±δZ.
> Which aside from reducing the duplication of effort, will *also* fix
> the problem of live migration where *all* those things suffer a step
> change and leave the guest with an inaccurate clock but not knowing it.

I am already convinced that this would work significantly better than the
{X,Y} pair (but would be a bit more effort to implement):

1. when accessed by user space, obviously

2. when backing the PTP clock, it saves CPU time and makes non-paired
   reads more precise.

I would just prefer to try upstreaming the {X,Y} pairing first. I think the
{X,Y,Z...} pairing could be discussed and developed in parallel.

Thanks for the comments,

Peter

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

  parent reply	other threads:[~2024-03-13 17:51 UTC|newest]

Thread overview: 68+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-12-18  7:38 [virtio-dev] [RFC PATCH v3 0/7] Add virtio_rtc module and related changes Peter Hilber
2023-12-18  7:38 ` Peter Hilber
2023-12-18  7:38 ` Peter Hilber
2023-12-18  7:38 ` [RFC PATCH v3 1/7] timekeeping: Fix cross-timestamp interpolation on counter wrap Peter Hilber
2024-02-19 16:35   ` [tip: timers/core] " tip-bot2 for Peter Hilber
2023-12-18  7:38 ` [RFC PATCH v3 2/7] timekeeping: Fix cross-timestamp interpolation corner case decision Peter Hilber
2024-02-19 16:35   ` [tip: timers/core] " tip-bot2 for Peter Hilber
2023-12-18  7:38 ` [RFC PATCH v3 3/7] timekeeping: Fix cross-timestamp interpolation for non-x86 Peter Hilber
2024-02-19 16:35   ` [tip: timers/core] " tip-bot2 for Peter Hilber
2023-12-18  7:38 ` [virtio-dev] [RFC PATCH v3 4/7] virtio_rtc: Add module and driver core Peter Hilber
2023-12-18  7:38   ` Peter Hilber
2023-12-18  7:38   ` Peter Hilber
2023-12-18  7:38 ` [virtio-dev] [RFC PATCH v3 5/7] virtio_rtc: Add PTP clocks Peter Hilber
2023-12-18  7:38   ` Peter Hilber
2023-12-18  7:38   ` Peter Hilber
2023-12-18  7:38 ` [virtio-dev] [RFC PATCH v3 6/7] virtio_rtc: Add Arm Generic Timer cross-timestamping Peter Hilber
2023-12-18  7:38   ` Peter Hilber
2023-12-18  7:38   ` Peter Hilber
2023-12-18  7:38 ` [virtio-dev] [RFC PATCH v3 7/7] virtio_rtc: Add RTC class driver Peter Hilber
2023-12-18  7:38   ` Peter Hilber
2024-03-08 17:03   ` Alexandre Belloni
2024-03-11 18:28     ` Peter Hilber
2024-03-11 19:46       ` Alexandre Belloni
2024-03-13  9:13         ` Peter Hilber
2024-03-07 14:02 ` [RFC PATCH v3 0/7] Add virtio_rtc module and related changes David Woodhouse
2024-03-07 14:02   ` David Woodhouse
2024-03-08 10:32   ` Peter Hilber
2024-03-08 10:32     ` Peter Hilber
2024-03-08 12:33     ` David Woodhouse
2024-03-08 12:33       ` David Woodhouse
2024-03-11 18:24       ` Peter Hilber
2024-03-11 18:24         ` Peter Hilber
2024-03-12 17:15         ` David Woodhouse
2024-03-12 17:15           ` David Woodhouse
2024-03-13  9:45           ` Peter Hilber
2024-03-13  9:45             ` Peter Hilber
2024-03-13 11:18             ` Alexandre Belloni
2024-03-13 11:18               ` Alexandre Belloni
2024-03-13 12:29               ` David Woodhouse
2024-03-13 12:29                 ` David Woodhouse
2024-03-13 12:58                 ` Alexandre Belloni
2024-03-13 12:58                   ` Alexandre Belloni
2024-03-13 14:06                   ` David Woodhouse
2024-03-13 14:06                     ` David Woodhouse
2024-03-13 14:50                     ` Alexandre Belloni
2024-03-13 14:50                       ` Alexandre Belloni
2024-03-13 20:12                       ` Andrew Lunn
2024-03-13 20:12                         ` Andrew Lunn
2024-03-14  9:13                         ` Peter Hilber
2024-03-14  9:13                           ` Peter Hilber
2024-03-13 17:50                     ` Peter Hilber [this message]
2024-03-13 17:50                       ` Peter Hilber
2024-03-13 14:15               ` Peter Hilber
2024-03-13 14:15                 ` Peter Hilber
2024-03-13 12:45             ` David Woodhouse
2024-03-13 12:45               ` David Woodhouse
2024-03-13 17:50               ` Peter Hilber
2024-03-13 17:50                 ` Peter Hilber
2024-03-13 18:18                 ` David Woodhouse
2024-03-13 18:18                   ` David Woodhouse
2024-03-14 10:13                   ` Peter Hilber
2024-03-14 10:13                     ` Peter Hilber
2024-03-14 14:19                     ` David Woodhouse
2024-03-14 14:19                       ` David Woodhouse
2024-03-19 13:47                       ` Peter Hilber
2024-03-19 13:47                         ` Peter Hilber
2024-03-20 17:22                         ` David Woodhouse
2024-03-20 17:22                           ` David Woodhouse

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=a0193f93-64cb-4935-b70f-aeee66eb6d44@opensynergy.com \
    --to=peter.hilber@opensynergy.com \
    --cc=a.zummo@towertech.it \
    --cc=alexandre.belloni@bootlin.com \
    --cc=christopher.s.hall@intel.com \
    --cc=daniel.lezcano@linaro.org \
    --cc=dwmw2@infradead.org \
    --cc=jasowang@redhat.com \
    --cc=jstultz@google.com \
    --cc=linux-arm-kernel@lists.infradead.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-rtc@vger.kernel.org \
    --cc=mark.rutland@arm.com \
    --cc=maz@kernel.org \
    --cc=mst@redhat.com \
    --cc=netdev@vger.kernel.org \
    --cc=richardcochran@gmail.com \
    --cc=ridouxj@amazon.com \
    --cc=sboyd@kernel.org \
    --cc=tglx@linutronix.de \
    --cc=virtio-comment@lists.oasis-open.org \
    --cc=virtio-dev@lists.oasis-open.org \
    --cc=virtualization@lists.linux.dev \
    --cc=xuanzhuo@linux.alibaba.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.