From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.0 required=3.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH, MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,UNPARSEABLE_RELAY, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8AF63C38A2A for ; Wed, 8 Apr 2020 05:07:35 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 588A620769 for ; Wed, 8 Apr 2020 05:07:35 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="rRyPLGp4" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727547AbgDHFHe (ORCPT ); Wed, 8 Apr 2020 01:07:34 -0400 Received: from userp2130.oracle.com ([156.151.31.86]:54140 "EHLO userp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727349AbgDHFHc (ORCPT ); Wed, 8 Apr 2020 01:07:32 -0400 Received: from pps.filterd (userp2130.oracle.com [127.0.0.1]) by userp2130.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 038553bq013448; Wed, 8 Apr 2020 05:07:19 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2020-01-29; bh=jtenwkhxJwQP2uq2B69E8Rt6lQLXw7OMtcgth/iQeLE=; b=rRyPLGp4AV4NTgpv3HbnBcQ7xi+6GgDMb68GyvXgeEhCjSMF6ysWqwYzf7YpGW/uJiQW Fl8krSNYEc/p+LC9gyJHE1S5WR1iuo0vMrP3nJHdB8saF1Jw3mo2oqfIgWRhYKvPF1p7 Uw2FQ74MmTPitiYxcvgIQ0kbhOLcwOuoFK5RFTT3vpoyc/2D0JHD7bMk8ZvhhaJGCrSm wC7lni75F9GI7ehsae7NyFMzWCwJjHBVrugyTPdblSdk89Z5pO5iQ3RDD2uiPIcdPfTJ ktNEWQUDZ2W1eLam26fvFBy1P/ka5AIDkRC8dGg8z918WC1UQ3hSAKntgBBIJe8Lk9lZ sg== Received: from aserp3020.oracle.com (aserp3020.oracle.com [141.146.126.70]) by userp2130.oracle.com with ESMTP id 3091m39155-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 08 Apr 2020 05:07:19 +0000 Received: from pps.filterd (aserp3020.oracle.com [127.0.0.1]) by aserp3020.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 03851WCB100753; Wed, 8 Apr 2020 05:05:18 GMT Received: from aserv0121.oracle.com (aserv0121.oracle.com [141.146.126.235]) by aserp3020.oracle.com with ESMTP id 3091m2hv20-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 08 Apr 2020 05:05:18 +0000 Received: from abhmp0012.oracle.com (abhmp0012.oracle.com [141.146.116.18]) by aserv0121.oracle.com (8.14.4/8.13.8) with ESMTP id 03855H45007452; Wed, 8 Apr 2020 05:05:17 GMT Received: from monad.ca.oracle.com (/10.156.75.81) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Tue, 07 Apr 2020 22:05:17 -0700 From: Ankur Arora To: linux-kernel@vger.kernel.org, x86@kernel.org Cc: peterz@infradead.org, hpa@zytor.com, jpoimboe@redhat.com, namit@vmware.com, mhiramat@kernel.org, jgross@suse.com, bp@alien8.de, vkuznets@redhat.com, pbonzini@redhat.com, boris.ostrovsky@oracle.com, mihai.carabas@oracle.com, kvm@vger.kernel.org, xen-devel@lists.xenproject.org, virtualization@lists.linux-foundation.org, Ankur Arora Subject: [RFC PATCH 13/26] x86/alternatives: Split __text_poke() Date: Tue, 7 Apr 2020 22:03:10 -0700 Message-Id: <20200408050323.4237-14-ankur.a.arora@oracle.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200408050323.4237-1-ankur.a.arora@oracle.com> References: <20200408050323.4237-1-ankur.a.arora@oracle.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9584 signatures=668685 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 phishscore=0 bulkscore=0 mlxscore=0 malwarescore=0 spamscore=0 adultscore=0 suspectscore=0 mlxlogscore=873 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2003020000 definitions=main-2004080037 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9584 signatures=668685 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 adultscore=0 impostorscore=0 malwarescore=0 lowpriorityscore=0 mlxlogscore=934 priorityscore=1501 clxscore=1015 bulkscore=0 phishscore=0 mlxscore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2003020000 definitions=main-2004080037 Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Separate __text_poke() into map, memcpy and unmap portions, (__text_poke_map(), __text_do_poke() and __text_poke_unmap().) Do this to separate the non-reentrant bits from the reentrant __text_do_poke(). __text_poke_map()/_unmap() modify poking_mm, poking_addr and do the pte-mapping and thus are non-reentrant. This allows __text_do_poke() to be safely called from an INT3 context with __text_poke_map()/_unmap() being called at the start and the end of the patching of a call-site instead of doing that for each stage of the three patching stages. Signed-off-by: Ankur Arora --- arch/x86/kernel/alternative.c | 46 +++++++++++++++++++++++++---------- 1 file changed, 33 insertions(+), 13 deletions(-) diff --git a/arch/x86/kernel/alternative.c b/arch/x86/kernel/alternative.c index 0344e49a4ade..337aad8c2521 100644 --- a/arch/x86/kernel/alternative.c +++ b/arch/x86/kernel/alternative.c @@ -805,13 +805,12 @@ void __init_or_module text_poke_early(void *addr, const void *opcode, __ro_after_init struct mm_struct *poking_mm; __ro_after_init unsigned long poking_addr; -static void __text_poke(void *addr, const void *opcode, size_t len) +static void __text_poke_map(void *addr, size_t len, + temp_mm_state_t *prev_mm, pte_t **ptep) { bool cross_page_boundary = offset_in_page(addr) + len > PAGE_SIZE; struct page *pages[2] = {NULL}; - temp_mm_state_t prev; - unsigned long flags; - pte_t pte, *ptep; + pte_t pte; pgprot_t pgprot; /* @@ -836,8 +835,6 @@ static void __text_poke(void *addr, const void *opcode, size_t len) */ BUG_ON(!pages[0] || (cross_page_boundary && !pages[1])); - local_irq_save(flags); - /* * Map the page without the global bit, as TLB flushing is done with * flush_tlb_mm_range(), which is intended for non-global PTEs. @@ -849,30 +846,42 @@ static void __text_poke(void *addr, const void *opcode, size_t len) * unlocked. This does mean that we need to be careful that no other * context (ex. INT3 handler) is simultaneously writing to this pte. */ - ptep = __get_unlocked_pte(poking_mm, poking_addr); + *ptep = __get_unlocked_pte(poking_mm, poking_addr); /* * This must not fail; preallocated in poking_init(). */ - VM_BUG_ON(!ptep); + VM_BUG_ON(!*ptep); pte = mk_pte(pages[0], pgprot); - set_pte_at(poking_mm, poking_addr, ptep, pte); + set_pte_at(poking_mm, poking_addr, *ptep, pte); if (cross_page_boundary) { pte = mk_pte(pages[1], pgprot); - set_pte_at(poking_mm, poking_addr + PAGE_SIZE, ptep + 1, pte); + set_pte_at(poking_mm, poking_addr + PAGE_SIZE, *ptep + 1, pte); } /* * Loading the temporary mm behaves as a compiler barrier, which * guarantees that the PTE will be set at the time memcpy() is done. */ - prev = use_temporary_mm(poking_mm); + *prev_mm = use_temporary_mm(poking_mm); +} +/* + * Do the actual poke. Needs to be re-entrant as this can be called + * via INT3 context as well. + */ +static void __text_do_poke(unsigned long offset, const void *opcode, size_t len) +{ kasan_disable_current(); - memcpy((u8 *)poking_addr + offset_in_page(addr), opcode, len); + memcpy((u8 *)poking_addr + offset, opcode, len); kasan_enable_current(); +} +static void __text_poke_unmap(void *addr, const void *opcode, size_t len, + temp_mm_state_t *prev_mm, pte_t *ptep) +{ + bool cross_page_boundary = offset_in_page(addr) + len > PAGE_SIZE; /* * Ensure that the PTE is only cleared after the instructions of memcpy * were issued by using a compiler barrier. @@ -888,7 +897,7 @@ static void __text_poke(void *addr, const void *opcode, size_t len) * instruction that already allows the core to see the updated version. * Xen-PV is assumed to serialize execution in a similar manner. */ - unuse_temporary_mm(prev); + unuse_temporary_mm(*prev_mm); /* * Flushing the TLB might involve IPIs, which would require enabled @@ -903,7 +912,18 @@ static void __text_poke(void *addr, const void *opcode, size_t len) * fundamentally screwy; there's nothing we can really do about that. */ BUG_ON(memcmp(addr, opcode, len)); +} +static void __text_poke(void *addr, const void *opcode, size_t len) +{ + temp_mm_state_t prev_mm; + unsigned long flags; + pte_t *ptep; + + local_irq_save(flags); + __text_poke_map(addr, len, &prev_mm, &ptep); + __text_do_poke(offset_in_page(addr), opcode, len); + __text_poke_unmap(addr, opcode, len, &prev_mm, ptep); local_irq_restore(flags); } -- 2.20.1 From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.5 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,UNPARSEABLE_RELAY,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 02EBBC352B6 for ; Wed, 8 Apr 2020 05:07:38 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C9AB020769 for ; Wed, 8 Apr 2020 05:07:37 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="rRyPLGp4" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C9AB020769 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=oracle.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1jM2vx-0004UY-4o; Wed, 08 Apr 2020 05:07:29 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1jM2vw-0004UD-7u for xen-devel@lists.xenproject.org; Wed, 08 Apr 2020 05:07:28 +0000 X-Inumbo-ID: d59d9162-7956-11ea-81bb-12813bfff9fa Received: from userp2130.oracle.com (unknown [156.151.31.86]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id d59d9162-7956-11ea-81bb-12813bfff9fa; Wed, 08 Apr 2020 05:07:25 +0000 (UTC) Received: from pps.filterd (userp2130.oracle.com [127.0.0.1]) by userp2130.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 038553bq013448; Wed, 8 Apr 2020 05:07:19 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2020-01-29; bh=jtenwkhxJwQP2uq2B69E8Rt6lQLXw7OMtcgth/iQeLE=; b=rRyPLGp4AV4NTgpv3HbnBcQ7xi+6GgDMb68GyvXgeEhCjSMF6ysWqwYzf7YpGW/uJiQW Fl8krSNYEc/p+LC9gyJHE1S5WR1iuo0vMrP3nJHdB8saF1Jw3mo2oqfIgWRhYKvPF1p7 Uw2FQ74MmTPitiYxcvgIQ0kbhOLcwOuoFK5RFTT3vpoyc/2D0JHD7bMk8ZvhhaJGCrSm wC7lni75F9GI7ehsae7NyFMzWCwJjHBVrugyTPdblSdk89Z5pO5iQ3RDD2uiPIcdPfTJ ktNEWQUDZ2W1eLam26fvFBy1P/ka5AIDkRC8dGg8z918WC1UQ3hSAKntgBBIJe8Lk9lZ sg== Received: from aserp3020.oracle.com (aserp3020.oracle.com [141.146.126.70]) by userp2130.oracle.com with ESMTP id 3091m39155-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 08 Apr 2020 05:07:19 +0000 Received: from pps.filterd (aserp3020.oracle.com [127.0.0.1]) by aserp3020.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 03851WCB100753; Wed, 8 Apr 2020 05:05:18 GMT Received: from aserv0121.oracle.com (aserv0121.oracle.com [141.146.126.235]) by aserp3020.oracle.com with ESMTP id 3091m2hv20-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 08 Apr 2020 05:05:18 +0000 Received: from abhmp0012.oracle.com (abhmp0012.oracle.com [141.146.116.18]) by aserv0121.oracle.com (8.14.4/8.13.8) with ESMTP id 03855H45007452; Wed, 8 Apr 2020 05:05:17 GMT Received: from monad.ca.oracle.com (/10.156.75.81) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Tue, 07 Apr 2020 22:05:17 -0700 From: Ankur Arora To: linux-kernel@vger.kernel.org, x86@kernel.org Subject: [RFC PATCH 13/26] x86/alternatives: Split __text_poke() Date: Tue, 7 Apr 2020 22:03:10 -0700 Message-Id: <20200408050323.4237-14-ankur.a.arora@oracle.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200408050323.4237-1-ankur.a.arora@oracle.com> References: <20200408050323.4237-1-ankur.a.arora@oracle.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9584 signatures=668685 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 phishscore=0 bulkscore=0 mlxscore=0 malwarescore=0 spamscore=0 adultscore=0 suspectscore=0 mlxlogscore=873 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2003020000 definitions=main-2004080037 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9584 signatures=668685 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 adultscore=0 impostorscore=0 malwarescore=0 lowpriorityscore=0 mlxlogscore=934 priorityscore=1501 clxscore=1015 bulkscore=0 phishscore=0 mlxscore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2003020000 definitions=main-2004080037 X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: jgross@suse.com, xen-devel@lists.xenproject.org, kvm@vger.kernel.org, peterz@infradead.org, hpa@zytor.com, Ankur Arora , virtualization@lists.linux-foundation.org, pbonzini@redhat.com, namit@vmware.com, mhiramat@kernel.org, jpoimboe@redhat.com, mihai.carabas@oracle.com, bp@alien8.de, vkuznets@redhat.com, boris.ostrovsky@oracle.com Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" Separate __text_poke() into map, memcpy and unmap portions, (__text_poke_map(), __text_do_poke() and __text_poke_unmap().) Do this to separate the non-reentrant bits from the reentrant __text_do_poke(). __text_poke_map()/_unmap() modify poking_mm, poking_addr and do the pte-mapping and thus are non-reentrant. This allows __text_do_poke() to be safely called from an INT3 context with __text_poke_map()/_unmap() being called at the start and the end of the patching of a call-site instead of doing that for each stage of the three patching stages. Signed-off-by: Ankur Arora --- arch/x86/kernel/alternative.c | 46 +++++++++++++++++++++++++---------- 1 file changed, 33 insertions(+), 13 deletions(-) diff --git a/arch/x86/kernel/alternative.c b/arch/x86/kernel/alternative.c index 0344e49a4ade..337aad8c2521 100644 --- a/arch/x86/kernel/alternative.c +++ b/arch/x86/kernel/alternative.c @@ -805,13 +805,12 @@ void __init_or_module text_poke_early(void *addr, const void *opcode, __ro_after_init struct mm_struct *poking_mm; __ro_after_init unsigned long poking_addr; -static void __text_poke(void *addr, const void *opcode, size_t len) +static void __text_poke_map(void *addr, size_t len, + temp_mm_state_t *prev_mm, pte_t **ptep) { bool cross_page_boundary = offset_in_page(addr) + len > PAGE_SIZE; struct page *pages[2] = {NULL}; - temp_mm_state_t prev; - unsigned long flags; - pte_t pte, *ptep; + pte_t pte; pgprot_t pgprot; /* @@ -836,8 +835,6 @@ static void __text_poke(void *addr, const void *opcode, size_t len) */ BUG_ON(!pages[0] || (cross_page_boundary && !pages[1])); - local_irq_save(flags); - /* * Map the page without the global bit, as TLB flushing is done with * flush_tlb_mm_range(), which is intended for non-global PTEs. @@ -849,30 +846,42 @@ static void __text_poke(void *addr, const void *opcode, size_t len) * unlocked. This does mean that we need to be careful that no other * context (ex. INT3 handler) is simultaneously writing to this pte. */ - ptep = __get_unlocked_pte(poking_mm, poking_addr); + *ptep = __get_unlocked_pte(poking_mm, poking_addr); /* * This must not fail; preallocated in poking_init(). */ - VM_BUG_ON(!ptep); + VM_BUG_ON(!*ptep); pte = mk_pte(pages[0], pgprot); - set_pte_at(poking_mm, poking_addr, ptep, pte); + set_pte_at(poking_mm, poking_addr, *ptep, pte); if (cross_page_boundary) { pte = mk_pte(pages[1], pgprot); - set_pte_at(poking_mm, poking_addr + PAGE_SIZE, ptep + 1, pte); + set_pte_at(poking_mm, poking_addr + PAGE_SIZE, *ptep + 1, pte); } /* * Loading the temporary mm behaves as a compiler barrier, which * guarantees that the PTE will be set at the time memcpy() is done. */ - prev = use_temporary_mm(poking_mm); + *prev_mm = use_temporary_mm(poking_mm); +} +/* + * Do the actual poke. Needs to be re-entrant as this can be called + * via INT3 context as well. + */ +static void __text_do_poke(unsigned long offset, const void *opcode, size_t len) +{ kasan_disable_current(); - memcpy((u8 *)poking_addr + offset_in_page(addr), opcode, len); + memcpy((u8 *)poking_addr + offset, opcode, len); kasan_enable_current(); +} +static void __text_poke_unmap(void *addr, const void *opcode, size_t len, + temp_mm_state_t *prev_mm, pte_t *ptep) +{ + bool cross_page_boundary = offset_in_page(addr) + len > PAGE_SIZE; /* * Ensure that the PTE is only cleared after the instructions of memcpy * were issued by using a compiler barrier. @@ -888,7 +897,7 @@ static void __text_poke(void *addr, const void *opcode, size_t len) * instruction that already allows the core to see the updated version. * Xen-PV is assumed to serialize execution in a similar manner. */ - unuse_temporary_mm(prev); + unuse_temporary_mm(*prev_mm); /* * Flushing the TLB might involve IPIs, which would require enabled @@ -903,7 +912,18 @@ static void __text_poke(void *addr, const void *opcode, size_t len) * fundamentally screwy; there's nothing we can really do about that. */ BUG_ON(memcmp(addr, opcode, len)); +} +static void __text_poke(void *addr, const void *opcode, size_t len) +{ + temp_mm_state_t prev_mm; + unsigned long flags; + pte_t *ptep; + + local_irq_save(flags); + __text_poke_map(addr, len, &prev_mm, &ptep); + __text_do_poke(offset_in_page(addr), opcode, len); + __text_poke_unmap(addr, opcode, len, &prev_mm, ptep); local_irq_restore(flags); } -- 2.20.1