From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 38EC6C433C1 for ; Mon, 22 Mar 2021 10:14:59 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id BDF786196E for ; Mon, 22 Mar 2021 10:14:58 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org BDF786196E Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=shutemov.name Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 9EDC06B00A3; Mon, 22 Mar 2021 05:56:07 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9C58F6B00A4; Mon, 22 Mar 2021 05:56:07 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 82B656B00A5; Mon, 22 Mar 2021 05:56:07 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0227.hostedemail.com [216.40.44.227]) by kanga.kvack.org (Postfix) with ESMTP id 650306B00A3 for ; Mon, 22 Mar 2021 05:56:07 -0400 (EDT) Received: from smtpin37.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id C1C4A8249980 for ; Mon, 22 Mar 2021 10:14:57 +0000 (UTC) X-FDA: 77947101834.37.174E0EA Received: from mail-lf1-f46.google.com (mail-lf1-f46.google.com [209.85.167.46]) by imf28.hostedemail.com (Postfix) with ESMTP id 1D8B72000242 for ; Mon, 22 Mar 2021 10:14:57 +0000 (UTC) Received: by mail-lf1-f46.google.com with SMTP id a198so20337440lfd.7 for ; Mon, 22 Mar 2021 03:14:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=shutemov-name.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=rVYaAi1eY1kPyS3/JwBek8GPGY1ptAf+8QB+EHmsqAU=; b=0xAxeaYtjCj0z4GR/m+iCT1B87INTUY//rqJ+VAahUIwsVg8ZAvYyLV/C73kwEny9I 5bmP+Ze/gAocr73JJ11cXkYyXb0/Gf3Zd+r4R3eA58TMDA1eiMWU+rMh1/olatKyJLjo 4QKYJ/75INlA0ttbJG3z39uP+Gil0Esyvd0LSgK5Tvb012/4yqglr2y7lQCQCcZcuLcr HPKJqoA+/yI/PbJGB9nlNL9opAR7qW1DeNwoWJ9I18hjVB4+THSiBNsRwIPj7IuDMhNl fGocd8edYDD+GUhQKbrV1adoAPW9vMeyJWE2vDTyiWsTDO41apCkiT/tXVhgWd0t5/6t VLKA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=rVYaAi1eY1kPyS3/JwBek8GPGY1ptAf+8QB+EHmsqAU=; b=ADZ0NabinMnJ3FsJkH6/Q0ayv6Oeoq4DrGdEEWXLekWD+X+t+fWMOJVdCf2ofhWHMv r3YjOcra2P5UnJwBqeO9ri5Y1ZxVdQOOFzAO6sVAyA1/eOj/SOlSQiJ1tuAc3dlATVBw ch91wfoq5WQzkvGUk/Hw7VwUsnDViqZxhbI1XiVo+D7PE1xfbe+qdOwK2LvNPWmKCIzK LzDIVLF9bBpkI9b+sURfuDLaZP46/UBGnK0EDAXfOm1m9ohbkJEp9ID6nID2Px0Ppz74 riNo3nOqkLNdEMjfAgtc3OOAxFNeE50owCCMtP57RcsFO7MYfW6s6Vqxn8nxyqrMhVLV DZLQ== X-Gm-Message-State: AOAM530PC7R3DVNGhSLxWMr0q1daTq+goIOKs8IlRLTBQxizYnXK8xDW uzDd2mafoEITeT+Gqss1hIkRpw== X-Google-Smtp-Source: ABdhPJxKaAEo8nt6OXdUuX61A1wui+fxNSys6+2rWR6UBxN18woKeXmAqW98QKT5Ba+bYjAcU7cKIw== X-Received: by 2002:a05:6512:348c:: with SMTP id v12mr8454102lfr.271.1616408095388; Mon, 22 Mar 2021 03:14:55 -0700 (PDT) Received: from box.localdomain ([86.57.175.117]) by smtp.gmail.com with ESMTPSA id z10sm1523041lfe.114.2021.03.22.03.14.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 22 Mar 2021 03:14:54 -0700 (PDT) Received: by box.localdomain (Postfix, from userid 1000) id 23BA8101DEB; Mon, 22 Mar 2021 13:15:02 +0300 (+03) Date: Mon, 22 Mar 2021 13:15:02 +0300 From: "Kirill A. Shutemov" To: Yu-cheng Yu Cc: x86@kernel.org, "H. Peter Anvin" , Thomas Gleixner , Ingo Molnar , linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-api@vger.kernel.org, Arnd Bergmann , Andy Lutomirski , Balbir Singh , Borislav Petkov , Cyrill Gorcunov , Dave Hansen , Eugene Syromiatnikov , Florian Weimer , "H.J. Lu" , Jann Horn , Jonathan Corbet , Kees Cook , Mike Kravetz , Nadav Amit , Oleg Nesterov , Pavel Machek , Peter Zijlstra , Randy Dunlap , "Ravi V. Shankar" , Vedvyas Shanbhogue , Dave Martin , Weijiang Yang , Pengfei Xu , Haitao Huang Subject: Re: [PATCH v23 12/28] x86/mm: Update ptep_set_wrprotect() and pmdp_set_wrprotect() for transition from _PAGE_DIRTY to _PAGE_COW Message-ID: <20210322101502.b5hdy3qgyh6hf3sr@box> References: <20210316151054.5405-1-yu-cheng.yu@intel.com> <20210316151054.5405-13-yu-cheng.yu@intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20210316151054.5405-13-yu-cheng.yu@intel.com> X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 1D8B72000242 X-Stat-Signature: iiuz676r97tkr4zw3u1bz8g3mhad3ozz Received-SPF: none (shutemov.name>: No applicable sender policy available) receiver=imf28; identity=mailfrom; envelope-from=""; helo=mail-lf1-f46.google.com; client-ip=209.85.167.46 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1616408097-848771 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue, Mar 16, 2021 at 08:10:38AM -0700, Yu-cheng Yu wrote: > When Shadow Stack is introduced, [R/O + _PAGE_DIRTY] PTE is reserved for > shadow stack. Copy-on-write PTEs have [R/O + _PAGE_COW]. > > When a PTE goes from [R/W + _PAGE_DIRTY] to [R/O + _PAGE_COW], it could > become a transient shadow stack PTE in two cases: > > The first case is that some processors can start a write but end up seeing > a read-only PTE by the time they get to the Dirty bit, creating a transient > shadow stack PTE. However, this will not occur on processors supporting > Shadow Stack, and a TLB flush is not necessary. > > The second case is that when _PAGE_DIRTY is replaced with _PAGE_COW non- > atomically, a transient shadow stack PTE can be created as a result. > Thus, prevent that with cmpxchg. > > Dave Hansen, Jann Horn, Andy Lutomirski, and Peter Zijlstra provided many > insights to the issue. Jann Horn provided the cmpxchg solution. > > Signed-off-by: Yu-cheng Yu > Reviewed-by: Kees Cook > --- > arch/x86/include/asm/pgtable.h | 36 ++++++++++++++++++++++++++++++++++ > 1 file changed, 36 insertions(+) > > diff --git a/arch/x86/include/asm/pgtable.h b/arch/x86/include/asm/pgtable.h > index e1739f590ca6..46d9394b884f 100644 > --- a/arch/x86/include/asm/pgtable.h > +++ b/arch/x86/include/asm/pgtable.h > @@ -1306,6 +1306,24 @@ static inline pte_t ptep_get_and_clear_full(struct mm_struct *mm, > static inline void ptep_set_wrprotect(struct mm_struct *mm, > unsigned long addr, pte_t *ptep) > { > + /* > + * If Shadow Stack is enabled, pte_wrprotect() moves _PAGE_DIRTY > + * to _PAGE_COW (see comments at pte_wrprotect()). > + * When a thread reads a RW=1, Dirty=0 PTE and before changing it > + * to RW=0, Dirty=0, another thread could have written to the page > + * and the PTE is RW=1, Dirty=1 now. Use try_cmpxchg() to detect > + * PTE changes and update old_pte, then try again. > + */ > + if (cpu_feature_enabled(X86_FEATURE_SHSTK)) { > + pte_t old_pte, new_pte; > + > + old_pte = READ_ONCE(*ptep); > + do { > + new_pte = pte_wrprotect(old_pte); > + } while (!try_cmpxchg(&ptep->pte, &old_pte.pte, new_pte.pte)); I think this is wrong. You need to update old_pte on every loop iteration, otherwise you can get in to endless loop. The same issue for pmdp_set_wrprotect(). > + > + return; > + } > clear_bit(_PAGE_BIT_RW, (unsigned long *)&ptep->pte); > } > > @@ -1350,6 +1368,24 @@ static inline pud_t pudp_huge_get_and_clear(struct mm_struct *mm, > static inline void pmdp_set_wrprotect(struct mm_struct *mm, > unsigned long addr, pmd_t *pmdp) > { > + /* > + * If Shadow Stack is enabled, pmd_wrprotect() moves _PAGE_DIRTY > + * to _PAGE_COW (see comments at pmd_wrprotect()). > + * When a thread reads a RW=1, Dirty=0 PMD and before changing it > + * to RW=0, Dirty=0, another thread could have written to the page > + * and the PMD is RW=1, Dirty=1 now. Use try_cmpxchg() to detect > + * PMD changes and update old_pmd, then try again. > + */ > + if (cpu_feature_enabled(X86_FEATURE_SHSTK)) { > + pmd_t old_pmd, new_pmd; > + > + old_pmd = READ_ONCE(*pmdp); > + do { > + new_pmd = pmd_wrprotect(old_pmd); > + } while (!try_cmpxchg((pmdval_t *)pmdp, (pmdval_t *)&old_pmd, pmd_val(new_pmd))); > + > + return; > + } > clear_bit(_PAGE_BIT_RW, (unsigned long *)pmdp); > } > > -- > 2.21.0 > -- Kirill A. Shutemov