From: Peter Zijlstra <peterz@infradead.org> To: mingo@kernel.org Cc: will@kernel.org, peterz@infradead.org, boqun.feng@gmail.com, mark.rutland@arm.com, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, seanjc@google.com, pbonzini@redhat.com, jgross@suse.com, srivatsa@csail.mit.edu, amakhalov@vmware.com, pv-drivers@vmware.com, wanpengli@tencent.com, vkuznets@redhat.com, boris.ostrovsky@oracle.com, rafael@kernel.org, daniel.lezcano@linaro.org, lpieralisi@kernel.org, frederic@kernel.org, paulmck@kernel.org, rostedt@goodmis.org, mhiramat@kernel.org, longman@redhat.com, juri.lelli@redhat.com, vincent.guittot@linaro.org, dietmar.eggemann@arm.com, bsegall@google.com, mgorman@suse.de, bristot@redhat.com, vschneid@redhat.com, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, virtualization@lists.linux-foundation.org, linux-pm@vger.kernel.org, linux-trace-kernel@vger.kernel.org, Uros Bizjak <ubizjak@gmail.com> Subject: [PATCH v2 6/9] x86/pvclock: improve atomic update of last_value in pvclock_clocksource_read Date: Thu, 26 Jan 2023 16:08:35 +0100 [thread overview] Message-ID: <20230126151323.643408110@infradead.org> (raw) In-Reply-To: 20230126150829.087606759@infradead.org From: Uros Bizjak <ubizjak@gmail.com> Improve atomic update of last_value in pvclock_clocksource_read: - Atomic update can be skipped if the "last_value" is already equal to "ret". - The detection of atomic update failure is not correct. The value, returned by atomic64_cmpxchg should be compared to the old value from the location to be updated. If these two are the same, then atomic update succeeded and "last_value" location is updated to "ret" in an atomic way. Otherwise, the atomic update failed and it should be retried with the value from "last_value" - exactly what atomic64_try_cmpxchg does in a correct and more optimal way. Signed-off-by: Uros Bizjak <ubizjak@gmail.com> Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org> Link: https://lkml.kernel.org/r/20230118202330.3740-1-ubizjak@gmail.com --- arch/x86/kernel/pvclock.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/arch/x86/kernel/pvclock.c b/arch/x86/kernel/pvclock.c index eda37df016f0..5a2a517dd61b 100644 --- a/arch/x86/kernel/pvclock.c +++ b/arch/x86/kernel/pvclock.c @@ -102,10 +102,9 @@ u64 pvclock_clocksource_read(struct pvclock_vcpu_time_info *src) */ last = atomic64_read(&last_value); do { - if (ret < last) + if (ret <= last) return last; - last = atomic64_cmpxchg(&last_value, last, ret); - } while (unlikely(last != ret)); + } while (!atomic64_try_cmpxchg(&last_value, &last, ret)); return ret; } -- 2.39.0
WARNING: multiple messages have this Message-ID (diff)
From: Peter Zijlstra <peterz@infradead.org> To: mingo@kernel.org Cc: mark.rutland@arm.com, juri.lelli@redhat.com, daniel.lezcano@linaro.org, wanpengli@tencent.com, kvm@vger.kernel.org, rafael@kernel.org, peterz@infradead.org, lpieralisi@kernel.org, dave.hansen@linux.intel.com, virtualization@lists.linux-foundation.org, bsegall@google.com, amakhalov@vmware.com, will@kernel.org, tglx@linutronix.de, vschneid@redhat.com, hpa@zytor.com, x86@kernel.org, Uros Bizjak <ubizjak@gmail.com>, pv-drivers@vmware.com, mingo@redhat.com, mgorman@suse.de, longman@redhat.com, frederic@kernel.org, linux-trace-kernel@vger.kernel.org, paulmck@kernel.org, linux-pm@vger.kernel.org, boqun.feng@gmail.com, rostedt@goodmis.org, bp@alien8.de, vincent.guittot@linaro.org, boris.ostrovsky@oracle.com, dietmar.eggemann@arm.com, jgross@suse.com, seanjc@google.com, linux-kernel@vger.kernel.org, mhiramat@kernel.org, pbonzini@redhat.com, bristot@redhat.com Subject: [PATCH v2 6/9] x86/pvclock: improve atomic update of last_value in pvclock_clocksource_read Date: Thu, 26 Jan 2023 16:08:35 +0100 [thread overview] Message-ID: <20230126151323.643408110@infradead.org> (raw) In-Reply-To: 20230126150829.087606759@infradead.org From: Uros Bizjak <ubizjak@gmail.com> Improve atomic update of last_value in pvclock_clocksource_read: - Atomic update can be skipped if the "last_value" is already equal to "ret". - The detection of atomic update failure is not correct. The value, returned by atomic64_cmpxchg should be compared to the old value from the location to be updated. If these two are the same, then atomic update succeeded and "last_value" location is updated to "ret" in an atomic way. Otherwise, the atomic update failed and it should be retried with the value from "last_value" - exactly what atomic64_try_cmpxchg does in a correct and more optimal way. Signed-off-by: Uros Bizjak <ubizjak@gmail.com> Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org> Link: https://lkml.kernel.org/r/20230118202330.3740-1-ubizjak@gmail.com --- arch/x86/kernel/pvclock.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/arch/x86/kernel/pvclock.c b/arch/x86/kernel/pvclock.c index eda37df016f0..5a2a517dd61b 100644 --- a/arch/x86/kernel/pvclock.c +++ b/arch/x86/kernel/pvclock.c @@ -102,10 +102,9 @@ u64 pvclock_clocksource_read(struct pvclock_vcpu_time_info *src) */ last = atomic64_read(&last_value); do { - if (ret < last) + if (ret <= last) return last; - last = atomic64_cmpxchg(&last_value, last, ret); - } while (unlikely(last != ret)); + } while (!atomic64_try_cmpxchg(&last_value, &last, ret)); return ret; } -- 2.39.0 _______________________________________________ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
next prev parent reply other threads:[~2023-01-26 15:15 UTC|newest] Thread overview: 29+ messages / expand[flat|nested] mbox.gz Atom feed top 2023-01-26 15:08 [PATCH v2 0/9] A few more cpuidle vs rcu fixes Peter Zijlstra 2023-01-26 15:08 ` Peter Zijlstra 2023-01-26 15:08 ` [PATCH v2 1/9] drivers: firmware: psci: Dont instrument suspend code Peter Zijlstra 2023-01-26 15:08 ` Peter Zijlstra 2023-01-31 14:22 ` [tip: sched/core] cpuidle: " tip-bot2 for Mark Rutland 2023-01-26 15:08 ` [PATCH v2 2/9] bug: Disable rcu_is_watching() during WARN/BUG Peter Zijlstra 2023-01-26 15:08 ` Peter Zijlstra 2023-01-31 14:22 ` [tip: sched/core] cpuidle: lib/bug: " tip-bot2 for Peter Zijlstra 2023-01-26 15:08 ` [PATCH v2 3/9] tracing: Warn about !rcu_is_watching() Peter Zijlstra 2023-01-26 15:08 ` Peter Zijlstra 2023-01-31 14:22 ` [tip: sched/core] cpuidle: " tip-bot2 for Peter Zijlstra 2023-01-26 15:08 ` [PATCH v2 4/9] tracing, preempt: Squash _rcuidle tracing Peter Zijlstra 2023-01-26 15:08 ` Peter Zijlstra 2023-01-31 8:50 ` [PATCH v2.1 " Peter Zijlstra 2023-01-31 8:50 ` Peter Zijlstra 2023-01-26 15:08 ` [PATCH v2 5/9] x86: Always inline arch_atomic64 Peter Zijlstra 2023-01-26 15:08 ` Peter Zijlstra 2023-01-31 14:22 ` [tip: sched/core] x86/atomics: Always inline arch_atomic64*() tip-bot2 for Peter Zijlstra 2023-01-26 15:08 ` Peter Zijlstra [this message] 2023-01-26 15:08 ` [PATCH v2 6/9] x86/pvclock: improve atomic update of last_value in pvclock_clocksource_read Peter Zijlstra 2023-01-26 15:08 ` [PATCH v2 7/9] x86: Mark sched_clock() noinstr Peter Zijlstra 2023-01-26 15:08 ` Peter Zijlstra 2023-01-31 14:22 ` [tip: sched/core] sched/clock/x86: " tip-bot2 for Peter Zijlstra 2023-01-26 15:08 ` [PATCH v2 8/9] sched/clock: Make local_clock() noinstr Peter Zijlstra 2023-01-26 15:08 ` Peter Zijlstra 2023-01-31 14:22 ` [tip: sched/core] " tip-bot2 for Peter Zijlstra 2023-01-26 15:08 ` [PATCH v2 9/9] cpuidle: Fix poll_idle() noinstr annotation Peter Zijlstra 2023-01-26 15:08 ` Peter Zijlstra 2023-01-29 23:23 ` [PATCH v2 0/9] A few more cpuidle vs rcu fixes Paul E. McKenney
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20230126151323.643408110@infradead.org \ --to=peterz@infradead.org \ --cc=amakhalov@vmware.com \ --cc=boqun.feng@gmail.com \ --cc=boris.ostrovsky@oracle.com \ --cc=bp@alien8.de \ --cc=bristot@redhat.com \ --cc=bsegall@google.com \ --cc=daniel.lezcano@linaro.org \ --cc=dave.hansen@linux.intel.com \ --cc=dietmar.eggemann@arm.com \ --cc=frederic@kernel.org \ --cc=hpa@zytor.com \ --cc=jgross@suse.com \ --cc=juri.lelli@redhat.com \ --cc=kvm@vger.kernel.org \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-pm@vger.kernel.org \ --cc=linux-trace-kernel@vger.kernel.org \ --cc=longman@redhat.com \ --cc=lpieralisi@kernel.org \ --cc=mark.rutland@arm.com \ --cc=mgorman@suse.de \ --cc=mhiramat@kernel.org \ --cc=mingo@kernel.org \ --cc=mingo@redhat.com \ --cc=paulmck@kernel.org \ --cc=pbonzini@redhat.com \ --cc=pv-drivers@vmware.com \ --cc=rafael@kernel.org \ --cc=rostedt@goodmis.org \ --cc=seanjc@google.com \ --cc=srivatsa@csail.mit.edu \ --cc=tglx@linutronix.de \ --cc=ubizjak@gmail.com \ --cc=vincent.guittot@linaro.org \ --cc=virtualization@lists.linux-foundation.org \ --cc=vkuznets@redhat.com \ --cc=vschneid@redhat.com \ --cc=wanpengli@tencent.com \ --cc=will@kernel.org \ --cc=x86@kernel.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.