From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0E2DBC433FE for ; Tue, 8 Feb 2022 11:26:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1357250AbiBHL0R (ORCPT ); Tue, 8 Feb 2022 06:26:17 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48326 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1345829AbiBHJ5C (ORCPT ); Tue, 8 Feb 2022 04:57:02 -0500 Received: from mail-lf1-x12c.google.com (mail-lf1-x12c.google.com [IPv6:2a00:1450:4864:20::12c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B2AC0C03FEC0 for ; Tue, 8 Feb 2022 01:57:00 -0800 (PST) Received: by mail-lf1-x12c.google.com with SMTP id x23so32363734lfc.0 for ; Tue, 08 Feb 2022 01:57:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=xnDB24LQeAUZkcfN3wL+FHV/aKMR05Pb76EsfKgOtQs=; b=gvGqrcbMb8DtdN8ZXcz6prSV4SyBxVL8iaSXK7eBe0hV86JPh8TEGaBs9LeBjH394n JV5aBERcpIRWYYa6clmXFDkBsubhJVVJXzuUPZt7JtaDR+AcLQ74PLG74djh/U6ThE9F jYLME7ZdZfWKpgrV+VImN+2PQV0ucGeZ80w1uSkw089d/t3KRCTq41dyBqhBLwUg08Y+ oFdo8CrrtiCXONlgKD+p0PBpktOXX9dbD5ax6mo9/6Hk50Eh3BsT6JiVSEF2I2T7Agvt DGIRHYiSRdYJMcE9MSwtJaWRukyds/9cMoCgWlB0nnL/HMht3xIjt98uC6O9o9xEKRbe 1vKw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=xnDB24LQeAUZkcfN3wL+FHV/aKMR05Pb76EsfKgOtQs=; b=5cwPntVFypG7hsfXiv9GHZz5sj7/VimqA2kD2bEUE6QUtG4b/lyosqYY9C3U79bfHR v4GxQ+JAb8P8ok5PQzebOqItq70o5pGbZBkGesBx9QBWmCQEaUKf5T7NPNqCOTfpJ1hj PV8r0tNENc7a1MpKPTSpNdTOx/bDIdS4VLULgjuyxQ+vynG6CMIBHFTwRmyIdhLRBZmb QOJGFBMJgrZKOm1cK3o3Gip+viRYdAPIkFbnMjU4mEy9YlYgzH2rkfF5KivZUOYSW0Ia jDaoIotKUonDTtlK9wvLu8IO3127+J9BcTWkC+JFPUXpjEt8PbK5JI9/Qg7WdvL5w214 7eCQ== X-Gm-Message-State: AOAM530Y2HN4GS03//dmvA11MnsKFLxIrGs5gUuGJ60xgAJV/A0slXRo o9MFBr4GQasUobX/HFmCE4jOnmsFEdXu4gnuKvmaYw== X-Google-Smtp-Source: ABdhPJxaGa+pLWFNUpDY6o6qT/j3Mq2c5YYfreeyurnEZrGUe6/lYOJubaJ+/Um6tzeBm0PoCUWy/A+7MB2iwbKR5iU= X-Received: by 2002:a05:6512:3181:: with SMTP id i1mr2552749lfe.286.1644314218724; Tue, 08 Feb 2022 01:56:58 -0800 (PST) MIME-Version: 1.0 References: <87mtp5q3gx.wl-maz@kernel.org> <87fsuxq049.wl-maz@kernel.org> <20210825150713.5rpwzm4grfn7akcw@gator.home> <877dg8ppnt.wl-maz@kernel.org> <20210827074011.ci2kzo4cnlp3qz7h@gator.home> <87ilyitt6e.wl-maz@kernel.org> <87lf3drmvp.wl-maz@kernel.org> <875yq88app.wl-maz@kernel.org> <878ruld72v.wl-maz@kernel.org> In-Reply-To: <878ruld72v.wl-maz@kernel.org> From: Oliver Upton Date: Tue, 8 Feb 2022 01:56:47 -0800 Message-ID: Subject: Re: KVM/arm64: Guest ABI changes do not appear rollback-safe To: Marc Zyngier Cc: Raghavendra Rao Ananta , Andrew Jones , kvmarm@lists.cs.columbia.edu, pshier@google.com, ricarkol@google.com, reijiw@google.com, jingzhangos@google.com, kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, james.morse@arm.com, Alexandru.Elisei@arm.com, suzuki.poulose@arm.com, Peter Maydell , Sean Christopherson Content-Type: text/plain; charset="UTF-8" Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Hi Marc, On Tue, Feb 8, 2022 at 1:46 AM Marc Zyngier wrote: > > > KVM currently restricts the vcpu features to be unified across vcpus, > > > but that's only an implementation choice. > > > > But that implementation choice has become ABI, no? How could support > > for asymmetry be added without requiring userspace opt-in or breaking > > existing VMMs that depend on feature unification? > > Of course, you'd need some sort of advertising of this new behaviour. > > One thing I would like to add to the current state of thing is an > indication of whether the effects of a sysreg being written from > userspace are global or local to a vcpu. You'd need a new capability, > and an extra flag added to the encoding of each register. Ah. I think that is a much more reasonable fit then. VMMs unaware of this can continue to migrate new bits (albeit at the cost of potentially higher lock contention for the per-VM stuff), and those that do can reap the benefits of writing such attributes exactly once. [...] > > > A device means yet another configuration and migration API. Don't you > > > think we have enough of those? The complexity of KVM/arm64 userspace > > > API is already insane, and extremely fragile. Adding to it will be a > > > validation nightmare (it already is, and I don't see anyone actively > > > helping with it). > > > > It seems equally fragile to introduce VM-wide serialization to vCPU > > UAPI that we know is in the live migration critical path for _any_ > > VMM. Without requiring userspace changes for all the new widgets under > > discussion we're effectively forcing VMMs to do something suboptimal. > > I'm perfectly happy with suboptimal to start with, and find ways to > make it better once we have a clear path forward. I just don't want to > conflate the two. Fair. My initial concern was that it didn't seem as though there was much room for growth/improvement with the one reg UAPI, but your suggestion definitely provides a ramp out to handle VM state once per VM. Thanks for following up :) -- Oliver