From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-23.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_IN_DEF_DKIM_WL autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0E43CC11F66 for ; Wed, 14 Jul 2021 13:22:50 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 78058613BE for ; Wed, 14 Jul 2021 13:22:49 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 78058613BE Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 603B66B0083; Wed, 14 Jul 2021 09:22:49 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 5B38A6B0085; Wed, 14 Jul 2021 09:22:49 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 42D8C6B0088; Wed, 14 Jul 2021 09:22:49 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0084.hostedemail.com [216.40.44.84]) by kanga.kvack.org (Postfix) with ESMTP id 1C5166B0083 for ; Wed, 14 Jul 2021 09:22:49 -0400 (EDT) Received: from smtpin32.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 137481858CE12 for ; Wed, 14 Jul 2021 13:22:48 +0000 (UTC) X-FDA: 78361258416.32.44E71FA Received: from mail-qt1-f180.google.com (mail-qt1-f180.google.com [209.85.160.180]) by imf22.hostedemail.com (Postfix) with ESMTP id A09BD1938 for ; Wed, 14 Jul 2021 13:22:47 +0000 (UTC) Received: by mail-qt1-f180.google.com with SMTP id r17so1814092qtp.5 for ; Wed, 14 Jul 2021 06:22:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=8WaQzYZeQ7Li+6RRsoQKD67iI4SEvRFGh30cooQVpZw=; b=ALZqB1qoxrXn+jAy7RQaf+A7TkxQvjq4ulZG5YO0uUIcybxnZKm0rpHD53EIfe8Jk/ YpGITna9ARYShBitncZbkmgkFksLsZIy/+ZdAoeYLBTDx8C1DKoohAJPaBYnRKyySVGc ITV+cFCVYK57U2R6VkGW5CEThNcTLrXhcqZ3U/43GLOYRV5ZCgoQENAK9PMCgKrnE4Bx RHXyl1CDuy68G1JGKqXbsBkwwZFSyQIkgTYpbX3O1ZpSJIeE/kExE2YJ4ajcd9qm2a80 xOkPSt+FopxTdx7Z1GVBLUUUa3y0DyeLiL1Gd0HkNb5dMY0ayoba9xVy0Kz+ZAL5/J4w 3PIw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=8WaQzYZeQ7Li+6RRsoQKD67iI4SEvRFGh30cooQVpZw=; b=LWMJaOwf/9lw7KxDx5m0u/yMiIZZZKMoEFzgTxDT/lznP9DA3zifWmEo5X5Jec/7k+ VFqkhaesdTYzYquHZHAn8rMiYAY2rqrBzkRS9d6egYSdej8l8zfU8r8XI53L2eCQwZ0r ReHlOBMsG/+mK/jxiLZ3tOtZZFEgIx6DtfyrgCAxztskkaFUF5PTsFqZsrvx7QFFYE3g wfIiJ7y37jXXiyJJRhF7V/owU8TsOWy7UOT2pIqQNizUnah5Pk0/B/dZ1fPZyT8gmmU/ XRHPLHCjPzTKyvRj0WpzP8FjGOGOKbYij7gdkOSqNtFTPAyWbH8QIka1+pkgUp5icRhJ jMIw== X-Gm-Message-State: AOAM532NqoVTnAGj5r5lECz7cb24UOadgVE5qI5IghhZxiUaXeuyBTgJ CzsxXf0yMcPnirU9I5XNjePoUgN9KTZZf2nIm7II+g== X-Google-Smtp-Source: ABdhPJzUq99VHUKVJMfZn1C5QDzWePBXid78b3kd93+UIASO5flvGLThImwDSEJuQzURM3cOVvnjCPX/f4O4MxrE6VA= X-Received: by 2002:a05:622a:409:: with SMTP id n9mr9315541qtx.261.1626268966412; Wed, 14 Jul 2021 06:22:46 -0700 (PDT) MIME-Version: 1.0 References: <20210707183616.5620-1-brijesh.singh@amd.com> <20210707183616.5620-16-brijesh.singh@amd.com> In-Reply-To: <20210707183616.5620-16-brijesh.singh@amd.com> From: Marc Orr Date: Wed, 14 Jul 2021 06:22:35 -0700 Message-ID: Subject: Re: [PATCH Part2 RFC v4 15/40] crypto: ccp: Handle the legacy TMR allocation when SNP is enabled To: Brijesh Singh Cc: x86@kernel.org, linux-kernel@vger.kernel.org, kvm list , linux-efi@vger.kernel.org, platform-driver-x86@vger.kernel.org, linux-coco@lists.linux.dev, linux-mm@kvack.org, linux-crypto@vger.kernel.org, Thomas Gleixner , Ingo Molnar , Joerg Roedel , Tom Lendacky , "H. Peter Anvin" , Ard Biesheuvel , Paolo Bonzini , Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Andy Lutomirski , Dave Hansen , Sergio Lopez , Peter Gonda , Peter Zijlstra , Srinivas Pandruvada , David Rientjes , Dov Murik , Tobin Feldman-Fitzthum , Borislav Petkov , Michael Roth , Vlastimil Babka , tony.luck@intel.com, npmccallum@redhat.com, brijesh.ksingh@gmail.com, Alper Gun Content-Type: text/plain; charset="UTF-8" Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=google.com header.s=20161025 header.b=ALZqB1qo; spf=pass (imf22.hostedemail.com: domain of marcorr@google.com designates 209.85.160.180 as permitted sender) smtp.mailfrom=marcorr@google.com; dmarc=pass (policy=reject) header.from=google.com X-Stat-Signature: 88ct135y18ohfuqyfyf5h9ibdt58shtd X-Rspamd-Queue-Id: A09BD1938 X-Rspamd-Server: rspam01 X-HE-Tag: 1626268967-747508 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed, Jul 7, 2021 at 11:37 AM Brijesh Singh wrote: > > The behavior and requirement for the SEV-legacy command is altered when > the SNP firmware is in the INIT state. See SEV-SNP firmware specification > for more details. > > When SNP is INIT state, all the SEV-legacy commands that cause the > firmware to write memory must be in the firmware state. The TMR memory > is allocated by the host but updated by the firmware, so, it must be > in the firmware state. Additionally, the TMR memory must be a 2MB aligned > instead of the 1MB, and the TMR length need to be 2MB instead of 1MB. > The helper __snp_{alloc,free}_firmware_pages() can be used for allocating > and freeing the memory used by the firmware. > > While at it, provide API that can be used by others to allocate a page > that can be used by the firmware. The immediate user for this API will > be the KVM driver. The KVM driver to need to allocate a firmware context > page during the guest creation. The context page need to be updated > by the firmware. See the SEV-SNP specification for further details. > > Signed-off-by: Brijesh Singh > --- > drivers/crypto/ccp/sev-dev.c | 144 +++++++++++++++++++++++++++++++---- > include/linux/psp-sev.h | 11 +++ > 2 files changed, 142 insertions(+), 13 deletions(-) > > diff --git a/drivers/crypto/ccp/sev-dev.c b/drivers/crypto/ccp/sev-dev.c > index ad9a0c8111e0..bb07c68834a6 100644 > --- a/drivers/crypto/ccp/sev-dev.c > +++ b/drivers/crypto/ccp/sev-dev.c > @@ -54,6 +54,14 @@ static int psp_timeout; > #define SEV_ES_TMR_SIZE (1024 * 1024) > static void *sev_es_tmr; > > +/* When SEV-SNP is enabled the TMR need to be 2MB aligned and 2MB size. */ nit: "the TMR need" -> "the TMR needs" > +#define SEV_SNP_ES_TMR_SIZE (2 * 1024 * 1024) > + > +static size_t sev_es_tmr_size = SEV_ES_TMR_SIZE; > + > +static int __sev_do_cmd_locked(int cmd, void *data, int *psp_ret); > +static int sev_do_cmd(int cmd, void *data, int *psp_ret); > + > static inline bool sev_version_greater_or_equal(u8 maj, u8 min) > { > struct sev_device *sev = psp_master->sev_data; > @@ -151,6 +159,112 @@ static int sev_cmd_buffer_len(int cmd) > return 0; > } > > +static int snp_reclaim_page(struct page *page, bool locked) > +{ > + struct sev_data_snp_page_reclaim data = {}; Hmmm.. according to some things I read online, an empty initializer list is not legal in C. For example: https://stackoverflow.com/questions/17589533/is-an-empty-initializer-list-valid-c-code I'm sure this is compiling. Should we change this to `{0}`, which I believe will initialize all fields in this struct to zero, according to: https://stackoverflow.com/questions/11152160/initializing-a-struct-to-0? > + int ret, err; > + > + data.paddr = page_to_pfn(page) << PAGE_SHIFT; > + > + if (locked) > + ret = __sev_do_cmd_locked(SEV_CMD_SNP_PAGE_RECLAIM, &data, &err); > + else > + ret = sev_do_cmd(SEV_CMD_SNP_PAGE_RECLAIM, &data, &err); > + > + return ret; > +} > + > +static int snp_set_rmptable_state(unsigned long paddr, int npages, > + struct rmpupdate *val, bool locked, bool need_reclaim) > +{ > + unsigned long pfn = __sme_clr(paddr) >> PAGE_SHIFT; > + unsigned long pfn_end = pfn + npages; > + struct psp_device *psp = psp_master; > + struct sev_device *sev; > + int rc; > + > + if (!psp || !psp->sev_data) > + return 0; Should this return a non-zero value -- maybe `-ENODEV`? Otherwise, the `snp_alloc_firmware_page()` API will return a page that the caller believes is suitable to use with FW. My concern is that someone decides to use this API to stash a page very early on during kernel boot and that page becomes a time bomb. If we initialize `rc` to `-ENODEV` (or something similar), then every return in this function can be `return rc`. > + > + /* If SEV-SNP is initialized then add the page in RMP table. */ > + sev = psp->sev_data; > + if (!sev->snp_inited) > + return 0; Ditto. Should this turn a non-zero value? > + > + while (pfn < pfn_end) { > + if (need_reclaim) > + if (snp_reclaim_page(pfn_to_page(pfn), locked)) > + return -EFAULT; > + > + rc = rmpupdate(pfn_to_page(pfn), val); > + if (rc) > + return rc; > + > + pfn++; > + } > + > + return 0; > +} > + > +static struct page *__snp_alloc_firmware_pages(gfp_t gfp_mask, int order, bool locked) > +{ > + struct rmpupdate val = {}; `{}` -> `{0}`? (Not sure, see my previous comment.) > + unsigned long paddr; > + struct page *page; > + > + page = alloc_pages(gfp_mask, order); > + if (!page) > + return NULL; > + > + val.assigned = 1; > + val.immutable = 1; > + paddr = __pa((unsigned long)page_address(page)); > + > + if (snp_set_rmptable_state(paddr, 1 << order, &val, locked, false)) { > + pr_warn("Failed to set page state (leaking it)\n"); Maybe `WARN_ONCE` instead of `pr_warn`? It's both a big attention grabber and also rate limited. > + return NULL; > + } > + > + return page; > +} > + > +void *snp_alloc_firmware_page(gfp_t gfp_mask) > +{ > + struct page *page; > + > + page = __snp_alloc_firmware_pages(gfp_mask, 0, false); > + > + return page ? page_address(page) : NULL; > +} > +EXPORT_SYMBOL_GPL(snp_alloc_firmware_page); > > +static void __snp_free_firmware_pages(struct page *page, int order, bool locked) > +{ > + struct rmpupdate val = {}; `{}` -> `{0}`? (Not sure, see my previous comment.) > + unsigned long paddr; > + > + if (!page) > + return; > + > + paddr = __pa((unsigned long)page_address(page)); > + > + if (snp_set_rmptable_state(paddr, 1 << order, &val, locked, true)) { > + pr_warn("Failed to set page state (leaking it)\n"); WARN_ONCE? > + return; > + } > + > + __free_pages(page, order); > +} > + > +void snp_free_firmware_page(void *addr) > +{ > + if (!addr) > + return; > + > + __snp_free_firmware_pages(virt_to_page(addr), 0, false); > +} > +EXPORT_SYMBOL(snp_free_firmware_page); > + > static int __sev_do_cmd_locked(int cmd, void *data, int *psp_ret) > { > struct psp_device *psp = psp_master; > @@ -273,7 +387,7 @@ static int __sev_platform_init_locked(int *error) > > data.flags |= SEV_INIT_FLAGS_SEV_ES; > data.tmr_address = tmr_pa; > - data.tmr_len = SEV_ES_TMR_SIZE; > + data.tmr_len = sev_es_tmr_size; > } > > rc = __sev_do_cmd_locked(SEV_CMD_INIT, &data, error); > @@ -630,6 +744,8 @@ static int __sev_snp_init_locked(int *error) > sev->snp_inited = true; > dev_dbg(sev->dev, "SEV-SNP firmware initialized\n"); > > + sev_es_tmr_size = SEV_SNP_ES_TMR_SIZE; > + > return rc; > } > > @@ -1153,8 +1269,10 @@ static void sev_firmware_shutdown(struct sev_device *sev) > /* The TMR area was encrypted, flush it from the cache */ > wbinvd_on_all_cpus(); > > - free_pages((unsigned long)sev_es_tmr, > - get_order(SEV_ES_TMR_SIZE)); > + > + __snp_free_firmware_pages(virt_to_page(sev_es_tmr), > + get_order(sev_es_tmr_size), > + false); > sev_es_tmr = NULL; > } > > @@ -1204,16 +1322,6 @@ void sev_pci_init(void) > sev_update_firmware(sev->dev) == 0) > sev_get_api_version(); > > - /* Obtain the TMR memory area for SEV-ES use */ > - tmr_page = alloc_pages(GFP_KERNEL, get_order(SEV_ES_TMR_SIZE)); > - if (tmr_page) { > - sev_es_tmr = page_address(tmr_page); > - } else { > - sev_es_tmr = NULL; > - dev_warn(sev->dev, > - "SEV: TMR allocation failed, SEV-ES support unavailable\n"); > - } > - > /* > * If boot CPU supports the SNP, then first attempt to initialize > * the SNP firmware. > @@ -1229,6 +1337,16 @@ void sev_pci_init(void) > } > } > > + /* Obtain the TMR memory area for SEV-ES use */ > + tmr_page = __snp_alloc_firmware_pages(GFP_KERNEL, get_order(sev_es_tmr_size), false); > + if (tmr_page) { > + sev_es_tmr = page_address(tmr_page); > + } else { > + sev_es_tmr = NULL; > + dev_warn(sev->dev, > + "SEV: TMR allocation failed, SEV-ES support unavailable\n"); > + } > + > /* Initialize the platform */ > rc = sev_platform_init(&error); > if (rc && (error == SEV_RET_SECURE_DATA_INVALID)) { > diff --git a/include/linux/psp-sev.h b/include/linux/psp-sev.h > index 63ef766cbd7a..b72a74f6a4e9 100644 > --- a/include/linux/psp-sev.h > +++ b/include/linux/psp-sev.h > @@ -12,6 +12,8 @@ > #ifndef __PSP_SEV_H__ > #define __PSP_SEV_H__ > > +#include > + > #include > > #ifdef CONFIG_X86 > @@ -920,6 +922,8 @@ int snp_guest_dbg_decrypt(struct sev_data_snp_dbg *data, int *error); > > > void *psp_copy_user_blob(u64 uaddr, u32 len); > +void *snp_alloc_firmware_page(gfp_t mask); > +void snp_free_firmware_page(void *addr); > > #else /* !CONFIG_CRYPTO_DEV_SP_PSP */ > > @@ -961,6 +965,13 @@ static inline int snp_guest_dbg_decrypt(struct sev_data_snp_dbg *data, int *erro > return -ENODEV; > } > > +static inline void *snp_alloc_firmware_page(gfp_t mask) > +{ > + return NULL; > +} > + > +static inline void snp_free_firmware_page(void *addr) { } > + > #endif /* CONFIG_CRYPTO_DEV_SP_PSP */ > > #endif /* __PSP_SEV_H__ */ > -- > 2.17.1 > >