From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7D570C04FF3 for ; Mon, 24 May 2021 16:56:20 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 3BAE6613F4 for ; Mon, 24 May 2021 16:56:20 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3BAE6613F4 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=y9xNshh3pFqboXr4VCitfiZdwyQTH8sTr2ybJhKyiFY=; b=g7mcp9gEvlW7Fb+vekHOry4hpD 74U4CnzJoSPudlp/deICxPPUTsiKW06ykEBMneQYCHSCQ2taFH9tC3M47rvtmpNpIFOajIRx7alBJ tUu9xo5MHXbQJp5U8KAr/EbeKsmpz6RPxOQgVFFZPMhmhm8SmNrfRlFzUcoCrqnykV7GeY/jxXDtf 2MfcilpnvrFJfub8sW0jlVaZNlVMebqVDCjnacDFJ8dtFUKOEiSViSB/2I9YfHq8vaJ3malylkdiL N7COeQk5Iah5NGJQCLvzaXi+9WWyciv7KNnZs2j841P1iNDFOdneeChKF8gsq55WvikiXFaEN+iCO h8iNqpfw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1llDqQ-001BER-8N; Mon, 24 May 2021 16:54:22 +0000 Received: from mail-qv1-xf4a.google.com ([2607:f8b0:4864:20::f4a]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1ll5yU-000lpN-5b for linux-arm-kernel@lists.infradead.org; Mon, 24 May 2021 08:30:11 +0000 Received: by mail-qv1-xf4a.google.com with SMTP id b24-20020a0cb3d80000b02901e78b82d74aso26655927qvf.20 for ; Mon, 24 May 2021 01:30:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=ENW5yXANABtXqS4LSFiicgp+oOZVaFiuGmSutRHOKLg=; b=n35s2r8zg4kPPfXcRIrKGVlVttKF3YXWcVa0/3Fr+uDsfMQIWKI9pfOdyP03lK7krH Hkspcig5ZwiFqx4RSHP5KbXxwlPENa5tHSv4SCgXfKkqlmbpSckLQlBxgf0wnL4yc3wN UXTwgOQdgJBy2CSBEAp+oEDDQENKlRc2Z9RB9PbkjOrySE+tUKVSbczyi64bPpqedvqW WG2xWgLk4aaXxyPyCk4gKtjMlXzPiS13j+mwjZ/g7SWCmEkEDnFRXy5M3e4I3BLQ42tu TWCfsWLIFj2NHSkZkSm5w26zJYVPDWLqQvFMO+IsRLDcRB6Lx2MG3m/pWNrFu51bFz4t R1aQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=ENW5yXANABtXqS4LSFiicgp+oOZVaFiuGmSutRHOKLg=; b=ODPEVyFzkzIm/8U7TbqrDjeq2MziWe+RYF9NEKceUTUvspdP+b6IjNIgIYHaf8W4UU fJEeGKmoqja52uihqf3SMjmO/N3AsQ87xdSq4tIhQZmhPuCZMlMXtIt2JnAFMXRWjFdy M1U9hhpjwqA7PKTMVr4LObvMD2qcRmdxy8zaQV6brvCLPp8o+ZPPiI8MdOjo2h8F+4v0 D/PPuLrY++qfnLWMQbMlMyuFyPsxM6aB5eLjhZsOrt6wCgAWStQ9VELROLKtKOX8pGYU rx852Ilf+4Q6HFCWjUUnnhFi/Gw4DLreNHxb4souH+aA/5wYF64y32Xp9wS1C8vmun5I wM0w== X-Gm-Message-State: AOAM533BAn9cc9PjNIN3ih5x0lWRfGhe8eiCMzgMY/yuC8jNnrpKfpxz yxf2YqXCUgEBtMVEBSXc1NoNfesRd1u8fqJUfImN2NdAUjF+NsgQ6eaMsrlcEV+ltiHUVj3sxq5 dH7QpqGx72xMLa8hno3sUZxeFa7wCrGXjaJ4K/sbyMPNGwIgVcQa6lIdfwpLbET/hR5Rz3AtEG8 ZBqA== X-Google-Smtp-Source: ABdhPJyMiQfBxCyMCt6KVGAbGahKIyWaGpH/V7RzNif0KbBFzEOTWNWWX2OxUzXqiuCi5yt6uFWwh+SU5Q== X-Received: from tabba.c.googlers.com ([fda3:e722:ac3:10:28:9cb1:c0a8:482]) (user=tabba job=sendgmr) by 2002:a0c:eed4:: with SMTP id h20mr28681086qvs.40.1621845007757; Mon, 24 May 2021 01:30:07 -0700 (PDT) Date: Mon, 24 May 2021 09:29:45 +0100 In-Reply-To: <20210524083001.2586635-1-tabba@google.com> Message-Id: <20210524083001.2586635-3-tabba@google.com> Mime-Version: 1.0 References: <20210524083001.2586635-1-tabba@google.com> X-Mailer: git-send-email 2.31.1.818.g46aad6cb9e-goog Subject: [PATCH v4 02/18] arm64: assembler: add conditional cache fixups From: Fuad Tabba To: linux-arm-kernel@lists.infradead.org Cc: will@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, maz@kernel.org, ardb@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, robin.murphy@arm.com, tabba@google.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210524_013010_234808_D8593895 X-CRM114-Status: GOOD ( 15.19 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Mark Rutland It would be helpful if we could use both `dcache_by_line_op` and `invalidate_icache_by_line` for user memory without accidentally fixing up unexpected faults when performing maintenance on kernel addresses. Let's make this possible by having both macros take an optional fixup label, and only generating an extable entry if a label is provided. At the same time, let's clean up the labels used to be globally unique using \@ as we do for other macros. There should be no functional change as a result of this patch. Signed-off-by: Mark Rutland Signed-off-by: Fuad Tabba Cc: Ard Biesheuvel Cc: Catalin Marinas Cc: Fuad Tabba Cc: Will Deacon --- arch/arm64/include/asm/assembler.h | 39 +++++++++++++++++++++--------- 1 file changed, 28 insertions(+), 11 deletions(-) diff --git a/arch/arm64/include/asm/assembler.h b/arch/arm64/include/asm/assembler.h index 6a0fbc599196..0a276b46ef50 100644 --- a/arch/arm64/include/asm/assembler.h +++ b/arch/arm64/include/asm/assembler.h @@ -130,15 +130,27 @@ alternative_endif .endm /* - * Emit an entry into the exception table + * Create an exception table entry for `insn`, which will branch to `fixup` + * when an unhandled fault is taken. */ - .macro _asm_extable, from, to + .macro _asm_extable, insn, fixup .pushsection __ex_table, "a" .align 3 - .long (\from - .), (\to - .) + .long (\insn - .), (\fixup - .) .popsection .endm +/* + * Create an exception table entry for `insn` if `fixup` is provided. Otherwise + * do nothing. + */ + .macro _cond_extable, insn, fixup + .ifnc \fixup, + _asm_extable \insn, \fixup + .endif + .endm + + #define USER(l, x...) \ 9999: x; \ _asm_extable 9999b, l @@ -383,6 +395,7 @@ alternative_cb_end * domain: domain used in dsb instruciton * addr: starting virtual address of the region * size: size of the region + * fixup: optional label to branch to on user fault * Corrupts: addr, size, tmp1, tmp2 */ .macro __dcache_op_workaround_clean_cache, op, addr @@ -393,12 +406,12 @@ alternative_else alternative_endif .endm - .macro dcache_by_line_op op, domain, addr, size, tmp1, tmp2 + .macro dcache_by_line_op op, domain, addr, size, tmp1, tmp2, fixup dcache_line_size \tmp1, \tmp2 add \size, \addr, \size sub \tmp2, \tmp1, #1 bic \addr, \addr, \tmp2 -9998: +.Ldcache_op\@: .ifc \op, cvau __dcache_op_workaround_clean_cache \op, \addr .else @@ -418,8 +431,10 @@ alternative_endif .endif add \addr, \addr, \tmp1 cmp \addr, \size - b.lo 9998b + b.lo .Ldcache_op\@ dsb \domain + + _cond_extable .Ldcache_op\@, \fixup .endm /* @@ -427,20 +442,22 @@ alternative_endif * [start, end) * * start, end: virtual addresses describing the region - * label: A label to branch to on user fault. + * fixup: optional label to branch to on user fault * Corrupts: tmp1, tmp2 */ - .macro invalidate_icache_by_line start, end, tmp1, tmp2, label + .macro invalidate_icache_by_line start, end, tmp1, tmp2, fixup icache_line_size \tmp1, \tmp2 sub \tmp2, \tmp1, #1 bic \tmp2, \start, \tmp2 -9997: -USER(\label, ic ivau, \tmp2) // invalidate I line PoU +.Licache_op\@: + ic ivau, \tmp2 // invalidate I line PoU add \tmp2, \tmp2, \tmp1 cmp \tmp2, \end - b.lo 9997b + b.lo .Licache_op\@ dsb ish isb + + _cond_extable .Licache_op\@, \fixup .endm /* -- 2.31.1.818.g46aad6cb9e-goog _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel