From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751903AbdA1TFN (ORCPT ); Sat, 28 Jan 2017 14:05:13 -0500 Received: from mail-sn1nam01on0115.outbound.protection.outlook.com ([104.47.32.115]:49952 "EHLO NAM01-SN1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1751344AbdA1TFE (ORCPT ); Sat, 28 Jan 2017 14:05:04 -0500 From: KY Srinivasan To: Vitaly Kuznetsov , "devel@linuxdriverproject.org" CC: Thomas Gleixner , "linux-kernel@vger.kernel.org" , Haiyang Zhang , "John Stultz" , "Alex Ng (LIS)" , Stephen Hemminger , Olaf Hering , Richard Cochran , Radim Krcmar Subject: RE: [PATCH v4 2/2] hv_utils: implement Hyper-V PTP source Thread-Topic: [PATCH v4 2/2] hv_utils: implement Hyper-V PTP source Thread-Index: AQHScl6wonFE8Mps90mns6e/UTf1JKFOTdjQ Date: Sat, 28 Jan 2017 19:04:55 +0000 Message-ID: References: <20170119141636.3970-1-vkuznets@redhat.com> <20170119141636.3970-3-vkuznets@redhat.com> In-Reply-To: <20170119141636.3970-3-vkuznets@redhat.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=kys@microsoft.com; x-originating-ip: [2601:600:8c00:1040:b076:3b2a:409c:285e] x-ms-office365-filtering-correlation-id: 6e2b4616-eec1-4c25-b6f1-08d447b08c1b x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: UriScan:;BCL:0;PCL:0;RULEID:(22001)(48565401076);SRVR:DM5PR03MB2729; x-microsoft-exchange-diagnostics: 1;DM5PR03MB2729;7:8852lmMPXxR97mWjS2jHR/h6MKoQDLiL9HUy50YYpwuwNYRuz2R41TQ6LzKWGMlO42xF3Z5kYq3mFA8ASol6Uw9pKPS/VMR+qLRdHhSvXAnrynZIjxVUbBfvn8R1gXzsik/RWQpNd3NXKZjkGcJLlnBm6pHoQAb2TOl3j47KdaZ/zn8HHBa0EUumNmhktmR4aIo0unMvtRW0Sdbk+91/U2JUe8W2EjflWGApq2LtBjWowk4TMv6Pfj/8JLagajHqTyawCz5PnEhXPiTpp706lJHwCTRYIKQyXoUeuSSMB9n0nykrXd/LcBkzHKT9kDleSkO6zBzp82Bxig/OhuQAJSJC167X3iW4NozkwHEkgM5Lt7l3xC0l6udIVaeV8a6izIUdchGJnnwKDgf72eURZUBsAeeTA7YTrWYbBZ8n0uOVudbVbiRz4kzMYL/Zx6wrTfZApqkSWhP1U+yAKoDnQY9MTmbES2DJ+P8UnLGXm1k= x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(158342451672863)(278428928389397)(9452136761055); x-exchange-antispam-report-cfa-test: BCL:0;PCL:0;RULEID:(61425038)(6040375)(601004)(2401047)(8121501046)(5005006)(3002001)(10201501046)(6055026)(61426038)(61427038)(6041248)(20161123562025)(20161123558021)(20161123560025)(20161123564025)(20161123555025)(6072148)(6047074);SRVR:DM5PR03MB2729;BCL:0;PCL:0;RULEID:;SRVR:DM5PR03MB2729; x-forefront-prvs: 02015246A9 x-forefront-antispam-report: SFV:NSPM;SFS:(10019020)(6009001)(7916002)(39850400002)(39450400003)(39410400002)(39860400002)(39840400002)(13464003)(189002)(377454003)(199003)(97736004)(54906002)(55016002)(53936002)(10090500001)(10290500002)(9686003)(76176999)(305945005)(7736002)(5660300001)(54356999)(101416001)(3280700002)(99286003)(5001770100001)(6436002)(3660700001)(92566002)(5005710100001)(2900100001)(74316002)(2501003)(189998001)(8990500004)(6116002)(39060400001)(106356001)(77096006)(102836003)(38730400001)(8676002)(105586002)(25786008)(106116001)(8936002)(6506006)(81156014)(2950100002)(86362001)(68736007)(2906002)(7696004)(50986999)(33656002)(122556002)(86612001)(81166006)(229853002)(4326007);DIR:OUT;SFP:1102;SCL:1;SRVR:DM5PR03MB2729;H:DM5PR03MB2490.namprd03.prod.outlook.com;FPR:;SPF:None;PTR:InfoNoRecords;MX:1;A:1;LANG:en; spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 X-OriginatorOrg: microsoft.com X-MS-Exchange-CrossTenant-originalarrivaltime: 28 Jan 2017 19:04:55.4763 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 72f988bf-86f1-41af-91ab-2d7cd011db47 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR03MB2729 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by mail.home.local id v0SJ5KAx002335 > -----Original Message----- > From: Vitaly Kuznetsov [mailto:vkuznets@redhat.com] > Sent: Thursday, January 19, 2017 6:17 AM > To: devel@linuxdriverproject.org > Cc: Thomas Gleixner ; linux-kernel@vger.kernel.org; > Haiyang Zhang ; KY Srinivasan > ; John Stultz ; Alex Ng (LIS) > ; Stephen Hemminger > ; Olaf Hering ; Richard > Cochran ; Radim Krcmar > > Subject: [PATCH v4 2/2] hv_utils: implement Hyper-V PTP source > > With TimeSync version 4 protocol support we started updating system time > continuously through the whole lifetime of Hyper-V guests. Every 5 seconds > there is a time sample from the host which triggers do_settimeofday[64](). > While the time from the host is very accurate such adjustments may cause > issues: > - Time is jumping forward and backward, some applications may misbehave. > - In case an NTP server runs in parallel and uses something else for time > sync (network, PTP,...) system time will never converge. > - Systemd starts annoying you by printing "Time has been changed" every 5 > seconds to the system log. > > Instead of doing in-kernel time adjustments offload the work to an > NTP client by exposing TimeSync messages as a PTP device. Users may now > decide what they want to use as a source. > > I tested the solution with chrony, the config was: > > refclock PHC /dev/ptp0 poll 3 precision 1e-9 > > The result I'm seeing is accurate enough, the time delta between the guest > and the host is almost always within [-10us, +10us], the in-kernel solution > was giving us comparable results. > > I also tried implementing PPS device instead of PTP by using not currently > used Hyper-V synthetic timers (we use only one of four for clockevent) but > with PPS source only chrony wasn't able to give me the required accuracy, > the delta often more that 100us. > > Signed-off-by: Vitaly Kuznetsov Vitaly, This patch does not apply. Please rebase and send. K. Y > --- > drivers/hv/hv_util.c | 144 > ++++++++++++++++++++++++++++++++++++++++++--------- > 1 file changed, 119 insertions(+), 25 deletions(-) > > diff --git a/drivers/hv/hv_util.c b/drivers/hv/hv_util.c > index 94719eb..e1211b8 100644 > --- a/drivers/hv/hv_util.c > +++ b/drivers/hv/hv_util.c > @@ -27,6 +27,7 @@ > #include > #include > #include > +#include > > #include "hyperv_vmbus.h" > > @@ -179,31 +180,34 @@ struct adj_time_work { > u8 flags; > }; > > +static u64 get_timeadj_latency(u64 ref_time) > +{ > + u64 current_tick; > + > + if (ts_srv_version <= TS_VERSION_3) > + return 0; > + > + /* > + * Some latency has been introduced since Hyper-V generated > + * its time sample. Take that latency into account before > + * using TSC reference time sample from Hyper-V. > + * > + * This sample is given by TimeSync v4 and above hosts. > + */ > + rdmsrl(HV_X64_MSR_TIME_REF_COUNT, current_tick); > + return current_tick - ref_time; > +} > + > static void hv_set_host_time(struct work_struct *work) > { > struct adj_time_work *wrk; > - s64 host_tns; > - u64 newtime; > struct timespec64 host_ts; > + u64 newtime; > > wrk = container_of(work, struct adj_time_work, work); > > - newtime = wrk->host_time; > - if (ts_srv_version > TS_VERSION_3) { > - /* > - * Some latency has been introduced since Hyper-V > generated > - * its time sample. Take that latency into account before > - * using TSC reference time sample from Hyper-V. > - * > - * This sample is given by TimeSync v4 and above hosts. > - */ > - u64 current_tick; > - > - rdmsrl(HV_X64_MSR_TIME_REF_COUNT, current_tick); > - newtime += (current_tick - wrk->ref_time); > - } > - host_tns = (newtime - WLTIMEDELTA) * 100; > - host_ts = ns_to_timespec64(host_tns); > + newtime = wrk->host_time + get_timeadj_latency(wrk->ref_time); > + host_ts = ns_to_timespec64((newtime - WLTIMEDELTA) * 100); > > do_settimeofday64(&host_ts); > } > @@ -222,22 +226,52 @@ static void hv_set_host_time(struct work_struct > *work) > * to discipline the clock. > */ > static struct adj_time_work wrk; > -static inline void adj_guesttime(u64 hosttime, u64 reftime, u8 flags) > + > +/* > + * The last time sample, received from the host. PTP device responds to > + * requests by using this data and the current partition-wide time reference > + * count. > + */ > +static struct { > + u64 host_time; > + u64 ref_time; > + spinlock_t lock; > +} host_ts; > + > +static inline void adj_guesttime(u64 hosttime, u64 reftime, u8 adj_flags) > { > + unsigned long flags; > > /* > * This check is safe since we are executing in the > * interrupt context and time synch messages arre always > * delivered on the same CPU. > */ > - if (work_pending(&wrk.work)) > - return; > + if (adj_flags & ICTIMESYNCFLAG_SYNC) { > + if (work_pending(&wrk.work)) > + return; > > - wrk.host_time = hosttime; > - wrk.ref_time = reftime; > - wrk.flags = flags; > - if ((flags & (ICTIMESYNCFLAG_SYNC | ICTIMESYNCFLAG_SAMPLE)) != > 0) { > + wrk.host_time = hosttime; > + wrk.ref_time = reftime; > + wrk.flags = adj_flags; > schedule_work(&wrk.work); > + } else { > + spin_lock_irqsave(&host_ts.lock, flags); > + host_ts.host_time = hosttime; > + > + /* > + * Prior to version 4 TimeSync messages from the host don't > + * contain any reference time (the time when the time > sample > + * was generated), save the current time reference count > + * instead. This adds a small delta between the time sample > + * generation and the reception of the sample here to the > result > + * but it's the best thing we can do. > + */ > + if (ts_srv_version <= TS_VERSION_3) > + rdmsrl(HV_X64_MSR_TIME_REF_COUNT, > host_ts.ref_time); > + else > + host_ts.ref_time = reftime; > + spin_unlock_irqrestore(&host_ts.lock, flags); > } > } > > @@ -470,14 +504,74 @@ static struct hv_driver util_drv = { > .remove = util_remove, > }; > > +static int hv_ptp_enable(struct ptp_clock_info *info, > + struct ptp_clock_request *request, int on) > +{ > + return -EOPNOTSUPP; > +} > + > +static int hv_ptp_settime(struct ptp_clock_info *p, const struct timespec64 > *ts) > +{ > + return -EOPNOTSUPP; > +} > + > +static int hv_ptp_adjfreq(struct ptp_clock_info *ptp, s32 delta) > +{ > + return -EOPNOTSUPP; > +} > +static int hv_ptp_adjtime(struct ptp_clock_info *ptp, s64 delta) > +{ > + return -EOPNOTSUPP; > +} > + > +static int hv_ptp_gettime(struct ptp_clock_info *info, struct timespec64 > *ts) > +{ > + u64 newtime; > + unsigned long flags; > + > + spin_lock_irqsave(&host_ts.lock, flags); > + newtime = host_ts.host_time + > get_timeadj_latency(host_ts.ref_time); > + *ts = ns_to_timespec64((newtime - WLTIMEDELTA) * 100); > + spin_unlock_irqrestore(&host_ts.lock, flags); > + > + return 0; > +} > + > +static const struct ptp_clock_info ptp_hyperv_info = { > + .name = "hyperv", > + .enable = hv_ptp_enable, > + .adjtime = hv_ptp_adjtime, > + .adjfreq = hv_ptp_adjfreq, > + .gettime64 = hv_ptp_gettime, > + .settime64 = hv_ptp_settime, > + .owner = THIS_MODULE, > +}; > + > +static struct ptp_clock *hv_ptp_clock; > + > static int hv_timesync_init(struct hv_util_service *srv) > { > INIT_WORK(&wrk.work, hv_set_host_time); > + > + /* > + * ptp_clock_register() returns NULL when > CONFIG_PTP_1588_CLOCK is > + * disabled but the driver is still useful without the PTP device > + * as it still handles the ICTIMESYNCFLAG_SYNC case. > + */ > + hv_ptp_clock = ptp_clock_register(&ptp_hyperv_info, NULL); > + if (IS_ERR_OR_NULL(hv_ptp_clock)) { > + pr_err("cannot register PTP clock: %ld\n", > + PTR_ERR(hv_ptp_clock)); > + hv_ptp_clock = NULL; > + } > + > return 0; > } > > static void hv_timesync_deinit(void) > { > + if (hv_ptp_clock) > + ptp_clock_unregister(hv_ptp_clock); > cancel_work_sync(&wrk.work); > } > > -- > 2.9.3