From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.4 required=3.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,USER_IN_DEF_DKIM_WL autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BFC9DC11D20 for ; Thu, 20 Feb 2020 20:44:46 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 8F994207FD for ; Thu, 20 Feb 2020 20:44:46 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="TVQhrreH" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729277AbgBTUoo (ORCPT ); Thu, 20 Feb 2020 15:44:44 -0500 Received: from mail-lj1-f196.google.com ([209.85.208.196]:33518 "EHLO mail-lj1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729231AbgBTUom (ORCPT ); Thu, 20 Feb 2020 15:44:42 -0500 Received: by mail-lj1-f196.google.com with SMTP id y6so5699398lji.0 for ; Thu, 20 Feb 2020 12:44:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=IWKv+CThwOSAcFDicDDkW6IxDXx0lTgPGsxbr+yUlKk=; b=TVQhrreHbf17hU30aWa/y+NBiuV9Q4d2JKpj0/fRjks4Rycw9ZM9syDOBQC+MZvyjA GasB9n67jaYQ3ymLvKx78uDjNdKsqdKks8gCJc5rsStSk4Tc1VqJyT56JCrcYMSkUwnC iQX+bBtvhysn0NRSvlgI4JTli8yYHDOizn0T/HTIs7sCCv0G0NQZn1Rm3bwNL9nZDyz+ a38lYxDmYOjSy3RuKpEiyESSCw3NCJZ7Dsi2GNBHrSBk6L2mtmjw8oYsNufrs8PaYam2 h2Z/HB9mljDnv2RM/IQdLQCSth1u6PEI5CcWGoWBgipkIP72A+Edrc2wSIAsScsw+O/u ogvw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=IWKv+CThwOSAcFDicDDkW6IxDXx0lTgPGsxbr+yUlKk=; b=gEinwP0vzVCEMiDAOgZC/KnBR1qJLWwDqWzaidNTxp11zCaCUu1m4y95zYEnbHqOl6 eb25jKAxqxH80gXAeh0TmHI3j8cBp6rnKZzcruScoBVdGJOwMLZVGdAjzUrBcXhQPwsh E6JGGCYC1J2fRqYXum0JjocIFMzFaaa1Nrd2KsnI2NcVJKKaQFzQKG80KRjd7UIVJgPQ d0Md56Lzy6qYUk/JDUP1VjuzCfpXDmNXpuRsRFtFe6akYquf2kKw3dq33CH/XfSDV+u9 WR3QMxntrg1X9210uMpGOSolWebP3EF83dQ+rPxz5/ofdJnM+vULs80ySTzMaHV4Bqsz AUIQ== X-Gm-Message-State: APjAAAWPCIwPdKI0l1V6EE523vOgqYsF0spMFgQTNGbbtJcCpbEaYGL9 0e/YlKBP6a6S7braQBTm8f1wcu0cKE0DSYnkzFwTRw== X-Google-Smtp-Source: APXvYqwooZIqlD5lJXNGmxCHDE66yRocSA5LDyvW8+AijnQERqL8v/ziSqz7UPiFAr1QEKFZ15WV464xfqknmXj3IPE= X-Received: by 2002:a05:651c:448:: with SMTP id g8mr20233966ljg.35.1582231477601; Thu, 20 Feb 2020 12:44:37 -0800 (PST) MIME-Version: 1.0 References: <52450536-AF7B-4206-8F05-CF387A216031@amacapital.net> <3de6e962-3277-ddbd-8c78-eaf754973928@amd.com> In-Reply-To: <3de6e962-3277-ddbd-8c78-eaf754973928@amd.com> From: Steve Rutherford Date: Thu, 20 Feb 2020 12:43:59 -0800 Message-ID: Subject: Re: [PATCH 10/12] mm: x86: Invoke hypercall when page encryption status is changed To: Brijesh Singh Cc: Andy Lutomirski , Ashish Kalra , Paolo Bonzini , Thomas Gleixner , Ingo Molnar , "H. Peter Anvin" , =?UTF-8?B?UmFkaW0gS3LEjW3DocWZ?= , Joerg Roedel , Borislav Petkov , Tom Lendacky , David Rientjes , x86@kernel.org, KVM list , LKML Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Thu, Feb 20, 2020 at 7:55 AM Brijesh Singh wrote= : > > > > On 2/19/20 8:12 PM, Andy Lutomirski wrote: > > > > > >> On Feb 19, 2020, at 5:58 PM, Steve Rutherford = wrote: > >> > >> =EF=BB=BFOn Wed, Feb 12, 2020 at 5:18 PM Ashish Kalra wrote: > >>> > >>> From: Brijesh Singh > >>> > >>> Invoke a hypercall when a memory region is changed from encrypted -> > >>> decrypted and vice versa. Hypervisor need to know the page encryption > >>> status during the guest migration. > >> > >> One messy aspect, which I think is fine in practice, is that this > >> presumes that pages are either treated as encrypted or decrypted. If > >> also done on SEV, the in-place re-encryption supported by SME would > >> break SEV migration. Linux doesn't do this now on SEV, and I don't > >> have an intuition for why Linux might want this, but we will need to > >> ensure it is never done in order to ensure that migration works down > >> the line. I don't believe the AMD manual promises this will work > >> anyway. > >> > >> Something feels a bit wasteful about having all future kernels > >> universally announce c-bit status when SEV is enabled, even if KVM > >> isn't listening, since it may be too old (or just not want to know). > >> Might be worth eliding the hypercalls if you get ENOSYS back? There > >> might be a better way of passing paravirt config metadata across than > >> just trying and seeing if the hypercall succeeds, but I'm not super > >> familiar with it. > > > > I actually think this should be a hard requirement to merge this. The h= ost needs to tell the guest that it supports this particular migration stra= tegy and the guest needs to tell the host that it is using it. And the gue= st needs a way to tell the host that it=E2=80=99s *not* using it right now = due to kexec, for example. > > > > I=E2=80=99m still uneasy about a guest being migrated in the window whe= re the hypercall tracking and the page encryption bit don=E2=80=99t match. = I guess maybe corruption in this window doesn=E2=80=99t matter? > > > > I don't think there is a corruption issue here. Let's consider the below > case: > > 1) A page is transmitted as C=3D1 (encrypted) > > 2) During the migration window, the page encryption bit is changed > to C=3D0 (decrypted) > > 3) #2 will cause a change in page table memory, thus dirty memory > the tracker will create retransmission of the page table memory. > > 4) The page itself will not be re-transmitted because there was > no change to the content of the page. > > On destination, the read from the page will get the ciphertext. > > The encryption bit change in the page table is used on the next access. > The user of the page needs to ensure that data is written with the > correct encryption bit before reading. > > thanks I think the issue results from a slightly different perspective than the one you are using. I think the situation Andy is interested in is when a c-bit change and a write happen close in time. There are five events, and the ordering matters: 1) Guest dirties the c-bit in the guest 2) Guest dirties the page 3) Host userspace observes the c-bit logs 4) Host userspace observes the page dirty logs 5) Host transmits the page If these are reordered to: 3) Host userspace observes the c-bit logs 1) Guest dirties the c-bit in the guest 2) Guest dirties the page 4) Host userspace observes the page dirty logs 5) Host transmits the page (from the wrong c-bit perspective!) Then the host will transmit a page with the wrong c-bit status and clear the dirty bit for that page. If the guest page is not retransmitted incidentally later, then this page will be corrupted. If you treat pages with dirty c-bits as dirty pages, then you will check the c-bit logs later and observe the dirty c-bit and retransmit. There might be some cleverness around enforcing that you always fetch the c-bit logs after fetching the dirty logs, but I haven't convinced myself that this works yet. I think it might, since then the c-bits are at least as fresh as the dirty bits. The main uncertainty that comes to mind for that strategy is if, on multi-vCPU VMs, the page dirtying event (from the new c-bit perspective) and the c-bit status change hypercall can themselves race. If a write from the new c-bit perspective can arrive before the c-bit status change arrives in the c-bit logs, we will need to treat pages with dirty c-bits as dirty pages. Note that I do agree that if the c-bit status flips, and no one writes to the page, it doesn't really matter if you retransmit that page. If a guest wants to read nonsense, it can.