From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 3C0DFC433EF for ; Fri, 13 May 2022 13:39:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:Subject:Message-ID:Date:From: In-Reply-To:References:MIME-Version:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=bn8LmiaPA7FJsF0K7yxpKJt96E0LaTLx6Q9D69OXv7U=; b=xvf0Qt6+bJolc+ laK+6UbFAQOQgDcnwIclLGTVlGCtCGkOa5P7y2CYvbMMEmWtUBl9L484YMoiYlbBlnDuvoCqq+nJY 2yzdBDkC/JftQ8swxOBCXqfDHN5A2Jo0wMBz4b1ShMuegl7dgHKMLDZxhA64HZwM75ibedt39B2+S sIGmebKetXaGW0wAYpEiieNVFPZrTW9ZJYEHTf83oBrUgH7S0+CKvdImxjt8SFdh5WsMeeFlxVuki vlPhOl+cZywgCh08czF+vp/oHMjisbjZ8XOTdU4iuoBgMQa1+uBlZMISUhYHONtLM3EKMOJTQgSQr bb0FIPMqe8YEiQtbBWpg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1npVVQ-00GJx8-DT; Fri, 13 May 2022 13:38:56 +0000 Received: from ams.source.kernel.org ([2604:1380:4601:e00::1]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1npVVJ-00GJsu-7C for linux-riscv@lists.infradead.org; Fri, 13 May 2022 13:38:54 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 824D7B8302A for ; Fri, 13 May 2022 13:38:47 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 45354C34114 for ; Fri, 13 May 2022 13:38:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1652449126; bh=r9PmVNd6Xnk8/rSY8uOm78ASlLb6vOPRZNuIP7avU6E=; h=References:In-Reply-To:From:Date:Subject:To:Cc:From; b=HnMQM2GxrVqye11NA40NEXZEcEv8tSZHF2peR1LexkD7q2IPNUYbPEoHQk40z365l j/jUXwq3DbRBWb3S2d9IMufzAfpM8OwSNZlZ2fWKI1L8XiAXEavixQWMEQIvklOyXW qxczS3nsyBD+O5UbKFyrb+0BsdHAbLOnvG+VG+QPwIHXHVnzCzZHmgsX3OserGIuZa m1iU4OuGMGebWZ/jsh3PDueT745cbJu8TVTKA/ZTxtYkeyjNxUivPpvCeTmQN/wxSJ u07UgH7/Pbfno6Yxj/LXFNdkOWCACZGJMYhlr/H810Z7nk1F/Nv96igvL5I+GCmO5Q Qdaoec2QWjddw== Received: by mail-vk1-f173.google.com with SMTP id t12so4221455vkt.5 for ; Fri, 13 May 2022 06:38:46 -0700 (PDT) X-Gm-Message-State: AOAM531UpJBNVFCTwL9GCkHkxH+0ikBlcyb8/mJZrpWHcGu3nWs5wBq8 2wS/NfM3p6VtwVcSwntqWlbIT+Ythc8BU/0jW6Q= X-Google-Smtp-Source: ABdhPJxZMrege554eIIwgpSz2JaIN2X2uxgd4Rxqs+KIs2Hz3q+ywVFUmOLZg/+gVO1gdykuJctt1bGwwl38Gdxnpy4= X-Received: by 2002:a1f:9fc4:0:b0:345:5848:4f44 with SMTP id i187-20020a1f9fc4000000b0034558484f44mr2244788vke.2.1652449125146; Fri, 13 May 2022 06:38:45 -0700 (PDT) MIME-Version: 1.0 References: <20220511214132.2281431-1-heiko@sntech.de> <20220511214132.2281431-3-heiko@sntech.de> In-Reply-To: From: Guo Ren Date: Fri, 13 May 2022 21:38:34 +0800 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: [PATCH v2 2/3] riscv: Implement Zicbom-based cache management operations To: Anup Patel Cc: Heiko Stuebner , Palmer Dabbelt , Paul Walmsley , linux-riscv , "linux-kernel@vger.kernel.org List" , Wei Fu , Atish Patra , Nick Kossifidis , Samuel Holland , Christoph Muellner , Philipp Tomsich , Rob Herring , krzk+dt@kernel.org, DTML , Christoph Hellwig , Atish Patra X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220513_063849_626758_CBF273A7 X-CRM114-Status: GOOD ( 39.74 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Reviewed-by: Guo Ren On Thu, May 12, 2022 at 12:19 PM Anup Patel wrote: > > On Thu, May 12, 2022 at 3:11 AM Heiko Stuebner wrote: > > > > The Zicbom ISA-extension was ratified in november 2021 > > and introduces instructions for dcache invalidate, clean > > and flush operations. > > > > Implement cache management operations based on them. > > > > Of course not all cores will support this, so implement an > > alternative-based mechanism that replaces empty instructions > > with ones done around Zicbom instructions. > > > > We're using prebuild instructions for the Zicbom instructions > > for now, to not require a bleeding-edge compiler (gcc-12) > > for these somewhat simple instructions. > > > > Signed-off-by: Heiko Stuebner > > Cc: Christoph Hellwig > > Cc: Atish Patra > > Cc: Guo Ren > > Looks good to me. > > Reviewed-by: Anup Patel > > Regards, > Anup > > > --- > > arch/riscv/Kconfig | 15 +++++ > > arch/riscv/include/asm/cacheflush.h | 6 ++ > > arch/riscv/include/asm/errata_list.h | 39 +++++++++++- > > arch/riscv/include/asm/hwcap.h | 1 + > > arch/riscv/kernel/cpu.c | 1 + > > arch/riscv/kernel/cpufeature.c | 17 +++++ > > arch/riscv/kernel/setup.c | 2 + > > arch/riscv/mm/Makefile | 1 + > > arch/riscv/mm/dma-noncoherent.c | 92 ++++++++++++++++++++++++++++ > > 9 files changed, 173 insertions(+), 1 deletion(-) > > create mode 100644 arch/riscv/mm/dma-noncoherent.c > > > > diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig > > index 65285b980134..532db45367a7 100644 > > --- a/arch/riscv/Kconfig > > +++ b/arch/riscv/Kconfig > > @@ -363,6 +363,21 @@ config RISCV_ISA_SVPBMT > > > > If you don't know what to do here, say Y. > > > > +config RISCV_ISA_ZICBOM > > + bool "Zicbom extension support for non-coherent dma operation" > > + select ARCH_HAS_DMA_PREP_COHERENT > > + select ARCH_HAS_SYNC_DMA_FOR_DEVICE > > + select ARCH_HAS_SYNC_DMA_FOR_CPU > > + select ARCH_HAS_SETUP_DMA_OPS > > + select DMA_DIRECT_REMAP > > + select RISCV_ALTERNATIVE > > + default y > > + help > > + Adds support to dynamically detect the presence of the ZICBOM extension > > + (Cache Block Management Operations) and enable its usage. > > + > > + If you don't know what to do here, say Y. > > + > > config FPU > > bool "FPU support" > > default y > > diff --git a/arch/riscv/include/asm/cacheflush.h b/arch/riscv/include/asm/cacheflush.h > > index 23ff70350992..eb12d014b158 100644 > > --- a/arch/riscv/include/asm/cacheflush.h > > +++ b/arch/riscv/include/asm/cacheflush.h > > @@ -42,6 +42,12 @@ void flush_icache_mm(struct mm_struct *mm, bool local); > > > > #endif /* CONFIG_SMP */ > > > > +#ifdef CONFIG_RISCV_ISA_ZICBOM > > +void riscv_init_cbom_blocksize(void); > > +#else > > +static inline void riscv_init_cbom_blocksize(void) { } > > +#endif > > + > > /* > > * Bits in sys_riscv_flush_icache()'s flags argument. > > */ > > diff --git a/arch/riscv/include/asm/errata_list.h b/arch/riscv/include/asm/errata_list.h > > index 9e2888dbb5b1..eebcd4415049 100644 > > --- a/arch/riscv/include/asm/errata_list.h > > +++ b/arch/riscv/include/asm/errata_list.h > > @@ -20,7 +20,8 @@ > > #endif > > > > #define CPUFEATURE_SVPBMT 0 > > -#define CPUFEATURE_NUMBER 1 > > +#define CPUFEATURE_CMO 1 > > +#define CPUFEATURE_NUMBER 2 > > > > #ifdef __ASSEMBLY__ > > > > @@ -93,6 +94,42 @@ asm volatile(ALTERNATIVE( \ > > #define ALT_THEAD_PMA(_val) > > #endif > > > > +/* > > + * cbo.clean rs1 > > + * | 31 - 20 | 19 - 15 | 14 - 12 | 11 - 7 | 6 - 0 | > > + * 0...01 rs1 010 00000 0001111 > > + * > > + * cbo.flush rs1 > > + * | 31 - 20 | 19 - 15 | 14 - 12 | 11 - 7 | 6 - 0 | > > + * 0...10 rs1 010 00000 0001111 > > + * > > + * cbo.inval rs1 > > + * | 31 - 20 | 19 - 15 | 14 - 12 | 11 - 7 | 6 - 0 | > > + * 0...00 rs1 010 00000 0001111 > > + */ > > +#define CBO_INVAL_A0 ".long 0x15200F" > > +#define CBO_CLEAN_A0 ".long 0x25200F" > > +#define CBO_FLUSH_A0 ".long 0x05200F" > > + > > +#define ALT_CMO_OP(_op, _start, _size, _cachesize) \ > > +asm volatile(ALTERNATIVE( \ > > + "nop\n\t" \ > > + "nop\n\t" \ > > + "nop\n\t" \ > > + "nop\n\t" \ > > + "nop", \ > > + "mv a0, %1\n\t" \ > > + "j 2f\n\t" \ > > + "3:\n\t" \ > > + CBO_##_op##_A0 "\n\t" \ > > + "add a0, a0, %0\n\t" \ > > + "2:\n\t" \ > > + "bltu a0, %2, 3b\n\t", 0, \ > > + CPUFEATURE_CMO, CONFIG_RISCV_ISA_ZICBOM) \ > > + : : "r"(_cachesize), \ > > + "r"(ALIGN((_start), (_cachesize))), \ > > + "r"(ALIGN((_start) + (_size), (_cachesize)))) > > + > > #endif /* __ASSEMBLY__ */ > > > > #endif > > diff --git a/arch/riscv/include/asm/hwcap.h b/arch/riscv/include/asm/hwcap.h > > index 4e2486881840..6044e402003d 100644 > > --- a/arch/riscv/include/asm/hwcap.h > > +++ b/arch/riscv/include/asm/hwcap.h > > @@ -53,6 +53,7 @@ extern unsigned long elf_hwcap; > > enum riscv_isa_ext_id { > > RISCV_ISA_EXT_SSCOFPMF = RISCV_ISA_EXT_BASE, > > RISCV_ISA_EXT_SVPBMT, > > + RISCV_ISA_EXT_ZICBOM, > > RISCV_ISA_EXT_ID_MAX = RISCV_ISA_EXT_MAX, > > }; > > > > diff --git a/arch/riscv/kernel/cpu.c b/arch/riscv/kernel/cpu.c > > index 40c8776aec12..8f6fc15baa8e 100644 > > --- a/arch/riscv/kernel/cpu.c > > +++ b/arch/riscv/kernel/cpu.c > > @@ -89,6 +89,7 @@ int riscv_of_parent_hartid(struct device_node *node) > > static struct riscv_isa_ext_data isa_ext_arr[] = { > > __RISCV_ISA_EXT_DATA(sscofpmf, RISCV_ISA_EXT_SSCOFPMF), > > __RISCV_ISA_EXT_DATA(svpbmt, RISCV_ISA_EXT_SVPBMT), > > + __RISCV_ISA_EXT_DATA(zicbom, RISCV_ISA_EXT_ZICBOM), > > __RISCV_ISA_EXT_DATA("", RISCV_ISA_EXT_MAX), > > }; > > > > diff --git a/arch/riscv/kernel/cpufeature.c b/arch/riscv/kernel/cpufeature.c > > index dea3ea19deee..db3c02409a4a 100644 > > --- a/arch/riscv/kernel/cpufeature.c > > +++ b/arch/riscv/kernel/cpufeature.c > > @@ -199,6 +199,7 @@ void __init riscv_fill_hwcap(void) > > } else { > > SET_ISA_EXT_MAP("sscofpmf", RISCV_ISA_EXT_SSCOFPMF); > > SET_ISA_EXT_MAP("svpbmt", RISCV_ISA_EXT_SVPBMT); > > + SET_ISA_EXT_MAP("zicbom", RISCV_ISA_EXT_ZICBOM); > > } > > #undef SET_ISA_EXT_MAP > > } > > @@ -265,12 +266,28 @@ static bool __init_or_module cpufeature_svpbmt_check_func(unsigned int stage) > > return false; > > } > > > > +static bool __init_or_module cpufeature_cmo_check_func(unsigned int stage) > > +{ > > + switch (stage) { > > + case RISCV_ALTERNATIVES_EARLY_BOOT: > > + return false; > > + default: > > + return riscv_isa_extension_available(NULL, ZICBOM); > > + } > > + > > + return false; > > +} > > + > > static const struct cpufeature_info __initdata_or_module > > cpufeature_list[CPUFEATURE_NUMBER] = { > > { > > .name = "svpbmt", > > .check_func = cpufeature_svpbmt_check_func > > }, > > + { > > + .name = "cmo", > > + .check_func = cpufeature_cmo_check_func > > + }, > > }; > > > > static u32 __init_or_module cpufeature_probe(unsigned int stage) > > diff --git a/arch/riscv/kernel/setup.c b/arch/riscv/kernel/setup.c > > index 9162e9a824d2..cc5bfeba499a 100644 > > --- a/arch/riscv/kernel/setup.c > > +++ b/arch/riscv/kernel/setup.c > > @@ -22,6 +22,7 @@ > > #include > > > > #include > > +#include > > #include > > #include > > #include > > @@ -296,6 +297,7 @@ void __init setup_arch(char **cmdline_p) > > #endif > > > > riscv_fill_hwcap(); > > + riscv_init_cbom_blocksize(); > > apply_boot_alternatives(); > > } > > > > diff --git a/arch/riscv/mm/Makefile b/arch/riscv/mm/Makefile > > index ac7a25298a04..548f2f3c00e9 100644 > > --- a/arch/riscv/mm/Makefile > > +++ b/arch/riscv/mm/Makefile > > @@ -30,3 +30,4 @@ endif > > endif > > > > obj-$(CONFIG_DEBUG_VIRTUAL) += physaddr.o > > +obj-$(CONFIG_RISCV_ISA_ZICBOM) += dma-noncoherent.o > > diff --git a/arch/riscv/mm/dma-noncoherent.c b/arch/riscv/mm/dma-noncoherent.c > > new file mode 100644 > > index 000000000000..99decaa25324 > > --- /dev/null > > +++ b/arch/riscv/mm/dma-noncoherent.c > > @@ -0,0 +1,92 @@ > > +// SPDX-License-Identifier: GPL-2.0-only > > +/* > > + * RISC-V specific functions to support DMA for non-coherent devices > > + * > > + * Copyright (c) 2021 Western Digital Corporation or its affiliates. > > + */ > > + > > +#include > > +#include > > +#include > > +#include > > +#include > > +#include > > +#include > > +#include > > + > > +static unsigned int riscv_cbom_block_size = L1_CACHE_BYTES; > > + > > +void arch_sync_dma_for_device(phys_addr_t paddr, size_t size, enum dma_data_direction dir) > > +{ > > + switch (dir) { > > + case DMA_TO_DEVICE: > > + ALT_CMO_OP(CLEAN, (unsigned long)phys_to_virt(paddr), size, riscv_cbom_block_size); > > + break; > > + case DMA_FROM_DEVICE: > > + ALT_CMO_OP(INVAL, (unsigned long)phys_to_virt(paddr), size, riscv_cbom_block_size); > > + break; > > + case DMA_BIDIRECTIONAL: > > + ALT_CMO_OP(FLUSH, (unsigned long)phys_to_virt(paddr), size, riscv_cbom_block_size); > > + break; > > + default: > > + break; > > + } > > +} > > + > > +void arch_sync_dma_for_cpu(phys_addr_t paddr, size_t size, enum dma_data_direction dir) > > +{ > > + switch (dir) { > > + case DMA_TO_DEVICE: > > + break; > > + case DMA_FROM_DEVICE: > > + case DMA_BIDIRECTIONAL: > > + ALT_CMO_OP(INVAL, (unsigned long)phys_to_virt(paddr), size, riscv_cbom_block_size); > > + break; > > + default: > > + break; > > + } > > +} > > + > > +void arch_dma_prep_coherent(struct page *page, size_t size) > > +{ > > + void *flush_addr = page_address(page); > > + > > + memset(flush_addr, 0, size); > > + ALT_CMO_OP(FLUSH, (unsigned long)flush_addr, size, riscv_cbom_block_size); > > +} > > + > > +void arch_setup_dma_ops(struct device *dev, u64 dma_base, u64 size, > > + const struct iommu_ops *iommu, bool coherent) > > +{ > > + /* If a specific device is dma-coherent, set it here */ > > + dev->dma_coherent = coherent; > > +} > > + > > +void riscv_init_cbom_blocksize(void) > > +{ > > + struct device_node *node; > > + int ret; > > + u32 val; > > + > > + for_each_of_cpu_node(node) { > > + int hartid = riscv_of_processor_hartid(node); > > + int cbom_hartid; > > + > > + if (hartid < 0) > > + continue; > > + > > + /* set block-size for cbom extension if available */ > > + ret = of_property_read_u32(node, "riscv,cbom-block-size", &val); > > + if (ret) > > + continue; > > + > > + if (!riscv_cbom_block_size) { > > + riscv_cbom_block_size = val; > > + cbom_hartid = hartid; > > + } else { > > + if (riscv_cbom_block_size != val) > > + pr_warn("cbom-block-size mismatched between harts %d and %d\n", > > + cbom_hartid, hartid); > > + } > > + } > > +} > > -- > > 2.35.1 > > -- Best Regards Guo Ren ML: https://lore.kernel.org/linux-csky/ _______________________________________________ linux-riscv mailing list linux-riscv@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-riscv From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3AA3FC41535 for ; Fri, 13 May 2022 13:50:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1381020AbiEMNjQ (ORCPT ); Fri, 13 May 2022 09:39:16 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35632 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1381640AbiEMNix (ORCPT ); Fri, 13 May 2022 09:38:53 -0400 Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3110365A4; Fri, 13 May 2022 06:38:49 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id B9C66B8302B; Fri, 13 May 2022 13:38:47 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 5177BC36AEC; Fri, 13 May 2022 13:38:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1652449126; bh=r9PmVNd6Xnk8/rSY8uOm78ASlLb6vOPRZNuIP7avU6E=; h=References:In-Reply-To:From:Date:Subject:To:Cc:From; b=HnMQM2GxrVqye11NA40NEXZEcEv8tSZHF2peR1LexkD7q2IPNUYbPEoHQk40z365l j/jUXwq3DbRBWb3S2d9IMufzAfpM8OwSNZlZ2fWKI1L8XiAXEavixQWMEQIvklOyXW qxczS3nsyBD+O5UbKFyrb+0BsdHAbLOnvG+VG+QPwIHXHVnzCzZHmgsX3OserGIuZa m1iU4OuGMGebWZ/jsh3PDueT745cbJu8TVTKA/ZTxtYkeyjNxUivPpvCeTmQN/wxSJ u07UgH7/Pbfno6Yxj/LXFNdkOWCACZGJMYhlr/H810Z7nk1F/Nv96igvL5I+GCmO5Q Qdaoec2QWjddw== Received: by mail-vk1-f173.google.com with SMTP id e144so4218349vke.9; Fri, 13 May 2022 06:38:46 -0700 (PDT) X-Gm-Message-State: AOAM5328010VNaWx3EHC6wOSnQ1okUxfSjMtjc9fQK5m7wj31nv2x1BN a5XpQUsz7EZitPZS2hSTEaq4jT6WUBdHncQ/epk= X-Google-Smtp-Source: ABdhPJxZMrege554eIIwgpSz2JaIN2X2uxgd4Rxqs+KIs2Hz3q+ywVFUmOLZg/+gVO1gdykuJctt1bGwwl38Gdxnpy4= X-Received: by 2002:a1f:9fc4:0:b0:345:5848:4f44 with SMTP id i187-20020a1f9fc4000000b0034558484f44mr2244788vke.2.1652449125146; Fri, 13 May 2022 06:38:45 -0700 (PDT) MIME-Version: 1.0 References: <20220511214132.2281431-1-heiko@sntech.de> <20220511214132.2281431-3-heiko@sntech.de> In-Reply-To: From: Guo Ren Date: Fri, 13 May 2022 21:38:34 +0800 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: [PATCH v2 2/3] riscv: Implement Zicbom-based cache management operations To: Anup Patel Cc: Heiko Stuebner , Palmer Dabbelt , Paul Walmsley , linux-riscv , "linux-kernel@vger.kernel.org List" , Wei Fu , Atish Patra , Nick Kossifidis , Samuel Holland , Christoph Muellner , Philipp Tomsich , Rob Herring , krzk+dt@kernel.org, DTML , Christoph Hellwig , Atish Patra Content-Type: text/plain; charset="UTF-8" Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Reviewed-by: Guo Ren On Thu, May 12, 2022 at 12:19 PM Anup Patel wrote: > > On Thu, May 12, 2022 at 3:11 AM Heiko Stuebner wrote: > > > > The Zicbom ISA-extension was ratified in november 2021 > > and introduces instructions for dcache invalidate, clean > > and flush operations. > > > > Implement cache management operations based on them. > > > > Of course not all cores will support this, so implement an > > alternative-based mechanism that replaces empty instructions > > with ones done around Zicbom instructions. > > > > We're using prebuild instructions for the Zicbom instructions > > for now, to not require a bleeding-edge compiler (gcc-12) > > for these somewhat simple instructions. > > > > Signed-off-by: Heiko Stuebner > > Cc: Christoph Hellwig > > Cc: Atish Patra > > Cc: Guo Ren > > Looks good to me. > > Reviewed-by: Anup Patel > > Regards, > Anup > > > --- > > arch/riscv/Kconfig | 15 +++++ > > arch/riscv/include/asm/cacheflush.h | 6 ++ > > arch/riscv/include/asm/errata_list.h | 39 +++++++++++- > > arch/riscv/include/asm/hwcap.h | 1 + > > arch/riscv/kernel/cpu.c | 1 + > > arch/riscv/kernel/cpufeature.c | 17 +++++ > > arch/riscv/kernel/setup.c | 2 + > > arch/riscv/mm/Makefile | 1 + > > arch/riscv/mm/dma-noncoherent.c | 92 ++++++++++++++++++++++++++++ > > 9 files changed, 173 insertions(+), 1 deletion(-) > > create mode 100644 arch/riscv/mm/dma-noncoherent.c > > > > diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig > > index 65285b980134..532db45367a7 100644 > > --- a/arch/riscv/Kconfig > > +++ b/arch/riscv/Kconfig > > @@ -363,6 +363,21 @@ config RISCV_ISA_SVPBMT > > > > If you don't know what to do here, say Y. > > > > +config RISCV_ISA_ZICBOM > > + bool "Zicbom extension support for non-coherent dma operation" > > + select ARCH_HAS_DMA_PREP_COHERENT > > + select ARCH_HAS_SYNC_DMA_FOR_DEVICE > > + select ARCH_HAS_SYNC_DMA_FOR_CPU > > + select ARCH_HAS_SETUP_DMA_OPS > > + select DMA_DIRECT_REMAP > > + select RISCV_ALTERNATIVE > > + default y > > + help > > + Adds support to dynamically detect the presence of the ZICBOM extension > > + (Cache Block Management Operations) and enable its usage. > > + > > + If you don't know what to do here, say Y. > > + > > config FPU > > bool "FPU support" > > default y > > diff --git a/arch/riscv/include/asm/cacheflush.h b/arch/riscv/include/asm/cacheflush.h > > index 23ff70350992..eb12d014b158 100644 > > --- a/arch/riscv/include/asm/cacheflush.h > > +++ b/arch/riscv/include/asm/cacheflush.h > > @@ -42,6 +42,12 @@ void flush_icache_mm(struct mm_struct *mm, bool local); > > > > #endif /* CONFIG_SMP */ > > > > +#ifdef CONFIG_RISCV_ISA_ZICBOM > > +void riscv_init_cbom_blocksize(void); > > +#else > > +static inline void riscv_init_cbom_blocksize(void) { } > > +#endif > > + > > /* > > * Bits in sys_riscv_flush_icache()'s flags argument. > > */ > > diff --git a/arch/riscv/include/asm/errata_list.h b/arch/riscv/include/asm/errata_list.h > > index 9e2888dbb5b1..eebcd4415049 100644 > > --- a/arch/riscv/include/asm/errata_list.h > > +++ b/arch/riscv/include/asm/errata_list.h > > @@ -20,7 +20,8 @@ > > #endif > > > > #define CPUFEATURE_SVPBMT 0 > > -#define CPUFEATURE_NUMBER 1 > > +#define CPUFEATURE_CMO 1 > > +#define CPUFEATURE_NUMBER 2 > > > > #ifdef __ASSEMBLY__ > > > > @@ -93,6 +94,42 @@ asm volatile(ALTERNATIVE( \ > > #define ALT_THEAD_PMA(_val) > > #endif > > > > +/* > > + * cbo.clean rs1 > > + * | 31 - 20 | 19 - 15 | 14 - 12 | 11 - 7 | 6 - 0 | > > + * 0...01 rs1 010 00000 0001111 > > + * > > + * cbo.flush rs1 > > + * | 31 - 20 | 19 - 15 | 14 - 12 | 11 - 7 | 6 - 0 | > > + * 0...10 rs1 010 00000 0001111 > > + * > > + * cbo.inval rs1 > > + * | 31 - 20 | 19 - 15 | 14 - 12 | 11 - 7 | 6 - 0 | > > + * 0...00 rs1 010 00000 0001111 > > + */ > > +#define CBO_INVAL_A0 ".long 0x15200F" > > +#define CBO_CLEAN_A0 ".long 0x25200F" > > +#define CBO_FLUSH_A0 ".long 0x05200F" > > + > > +#define ALT_CMO_OP(_op, _start, _size, _cachesize) \ > > +asm volatile(ALTERNATIVE( \ > > + "nop\n\t" \ > > + "nop\n\t" \ > > + "nop\n\t" \ > > + "nop\n\t" \ > > + "nop", \ > > + "mv a0, %1\n\t" \ > > + "j 2f\n\t" \ > > + "3:\n\t" \ > > + CBO_##_op##_A0 "\n\t" \ > > + "add a0, a0, %0\n\t" \ > > + "2:\n\t" \ > > + "bltu a0, %2, 3b\n\t", 0, \ > > + CPUFEATURE_CMO, CONFIG_RISCV_ISA_ZICBOM) \ > > + : : "r"(_cachesize), \ > > + "r"(ALIGN((_start), (_cachesize))), \ > > + "r"(ALIGN((_start) + (_size), (_cachesize)))) > > + > > #endif /* __ASSEMBLY__ */ > > > > #endif > > diff --git a/arch/riscv/include/asm/hwcap.h b/arch/riscv/include/asm/hwcap.h > > index 4e2486881840..6044e402003d 100644 > > --- a/arch/riscv/include/asm/hwcap.h > > +++ b/arch/riscv/include/asm/hwcap.h > > @@ -53,6 +53,7 @@ extern unsigned long elf_hwcap; > > enum riscv_isa_ext_id { > > RISCV_ISA_EXT_SSCOFPMF = RISCV_ISA_EXT_BASE, > > RISCV_ISA_EXT_SVPBMT, > > + RISCV_ISA_EXT_ZICBOM, > > RISCV_ISA_EXT_ID_MAX = RISCV_ISA_EXT_MAX, > > }; > > > > diff --git a/arch/riscv/kernel/cpu.c b/arch/riscv/kernel/cpu.c > > index 40c8776aec12..8f6fc15baa8e 100644 > > --- a/arch/riscv/kernel/cpu.c > > +++ b/arch/riscv/kernel/cpu.c > > @@ -89,6 +89,7 @@ int riscv_of_parent_hartid(struct device_node *node) > > static struct riscv_isa_ext_data isa_ext_arr[] = { > > __RISCV_ISA_EXT_DATA(sscofpmf, RISCV_ISA_EXT_SSCOFPMF), > > __RISCV_ISA_EXT_DATA(svpbmt, RISCV_ISA_EXT_SVPBMT), > > + __RISCV_ISA_EXT_DATA(zicbom, RISCV_ISA_EXT_ZICBOM), > > __RISCV_ISA_EXT_DATA("", RISCV_ISA_EXT_MAX), > > }; > > > > diff --git a/arch/riscv/kernel/cpufeature.c b/arch/riscv/kernel/cpufeature.c > > index dea3ea19deee..db3c02409a4a 100644 > > --- a/arch/riscv/kernel/cpufeature.c > > +++ b/arch/riscv/kernel/cpufeature.c > > @@ -199,6 +199,7 @@ void __init riscv_fill_hwcap(void) > > } else { > > SET_ISA_EXT_MAP("sscofpmf", RISCV_ISA_EXT_SSCOFPMF); > > SET_ISA_EXT_MAP("svpbmt", RISCV_ISA_EXT_SVPBMT); > > + SET_ISA_EXT_MAP("zicbom", RISCV_ISA_EXT_ZICBOM); > > } > > #undef SET_ISA_EXT_MAP > > } > > @@ -265,12 +266,28 @@ static bool __init_or_module cpufeature_svpbmt_check_func(unsigned int stage) > > return false; > > } > > > > +static bool __init_or_module cpufeature_cmo_check_func(unsigned int stage) > > +{ > > + switch (stage) { > > + case RISCV_ALTERNATIVES_EARLY_BOOT: > > + return false; > > + default: > > + return riscv_isa_extension_available(NULL, ZICBOM); > > + } > > + > > + return false; > > +} > > + > > static const struct cpufeature_info __initdata_or_module > > cpufeature_list[CPUFEATURE_NUMBER] = { > > { > > .name = "svpbmt", > > .check_func = cpufeature_svpbmt_check_func > > }, > > + { > > + .name = "cmo", > > + .check_func = cpufeature_cmo_check_func > > + }, > > }; > > > > static u32 __init_or_module cpufeature_probe(unsigned int stage) > > diff --git a/arch/riscv/kernel/setup.c b/arch/riscv/kernel/setup.c > > index 9162e9a824d2..cc5bfeba499a 100644 > > --- a/arch/riscv/kernel/setup.c > > +++ b/arch/riscv/kernel/setup.c > > @@ -22,6 +22,7 @@ > > #include > > > > #include > > +#include > > #include > > #include > > #include > > @@ -296,6 +297,7 @@ void __init setup_arch(char **cmdline_p) > > #endif > > > > riscv_fill_hwcap(); > > + riscv_init_cbom_blocksize(); > > apply_boot_alternatives(); > > } > > > > diff --git a/arch/riscv/mm/Makefile b/arch/riscv/mm/Makefile > > index ac7a25298a04..548f2f3c00e9 100644 > > --- a/arch/riscv/mm/Makefile > > +++ b/arch/riscv/mm/Makefile > > @@ -30,3 +30,4 @@ endif > > endif > > > > obj-$(CONFIG_DEBUG_VIRTUAL) += physaddr.o > > +obj-$(CONFIG_RISCV_ISA_ZICBOM) += dma-noncoherent.o > > diff --git a/arch/riscv/mm/dma-noncoherent.c b/arch/riscv/mm/dma-noncoherent.c > > new file mode 100644 > > index 000000000000..99decaa25324 > > --- /dev/null > > +++ b/arch/riscv/mm/dma-noncoherent.c > > @@ -0,0 +1,92 @@ > > +// SPDX-License-Identifier: GPL-2.0-only > > +/* > > + * RISC-V specific functions to support DMA for non-coherent devices > > + * > > + * Copyright (c) 2021 Western Digital Corporation or its affiliates. > > + */ > > + > > +#include > > +#include > > +#include > > +#include > > +#include > > +#include > > +#include > > +#include > > + > > +static unsigned int riscv_cbom_block_size = L1_CACHE_BYTES; > > + > > +void arch_sync_dma_for_device(phys_addr_t paddr, size_t size, enum dma_data_direction dir) > > +{ > > + switch (dir) { > > + case DMA_TO_DEVICE: > > + ALT_CMO_OP(CLEAN, (unsigned long)phys_to_virt(paddr), size, riscv_cbom_block_size); > > + break; > > + case DMA_FROM_DEVICE: > > + ALT_CMO_OP(INVAL, (unsigned long)phys_to_virt(paddr), size, riscv_cbom_block_size); > > + break; > > + case DMA_BIDIRECTIONAL: > > + ALT_CMO_OP(FLUSH, (unsigned long)phys_to_virt(paddr), size, riscv_cbom_block_size); > > + break; > > + default: > > + break; > > + } > > +} > > + > > +void arch_sync_dma_for_cpu(phys_addr_t paddr, size_t size, enum dma_data_direction dir) > > +{ > > + switch (dir) { > > + case DMA_TO_DEVICE: > > + break; > > + case DMA_FROM_DEVICE: > > + case DMA_BIDIRECTIONAL: > > + ALT_CMO_OP(INVAL, (unsigned long)phys_to_virt(paddr), size, riscv_cbom_block_size); > > + break; > > + default: > > + break; > > + } > > +} > > + > > +void arch_dma_prep_coherent(struct page *page, size_t size) > > +{ > > + void *flush_addr = page_address(page); > > + > > + memset(flush_addr, 0, size); > > + ALT_CMO_OP(FLUSH, (unsigned long)flush_addr, size, riscv_cbom_block_size); > > +} > > + > > +void arch_setup_dma_ops(struct device *dev, u64 dma_base, u64 size, > > + const struct iommu_ops *iommu, bool coherent) > > +{ > > + /* If a specific device is dma-coherent, set it here */ > > + dev->dma_coherent = coherent; > > +} > > + > > +void riscv_init_cbom_blocksize(void) > > +{ > > + struct device_node *node; > > + int ret; > > + u32 val; > > + > > + for_each_of_cpu_node(node) { > > + int hartid = riscv_of_processor_hartid(node); > > + int cbom_hartid; > > + > > + if (hartid < 0) > > + continue; > > + > > + /* set block-size for cbom extension if available */ > > + ret = of_property_read_u32(node, "riscv,cbom-block-size", &val); > > + if (ret) > > + continue; > > + > > + if (!riscv_cbom_block_size) { > > + riscv_cbom_block_size = val; > > + cbom_hartid = hartid; > > + } else { > > + if (riscv_cbom_block_size != val) > > + pr_warn("cbom-block-size mismatched between harts %d and %d\n", > > + cbom_hartid, hartid); > > + } > > + } > > +} > > -- > > 2.35.1 > > -- Best Regards Guo Ren ML: https://lore.kernel.org/linux-csky/