From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 50F84C4332F for ; Wed, 1 Jun 2022 10:12:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1351921AbiFAKMF (ORCPT ); Wed, 1 Jun 2022 06:12:05 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44334 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1351839AbiFAKMD (ORCPT ); Wed, 1 Jun 2022 06:12:03 -0400 Received: from mail-ej1-x62b.google.com (mail-ej1-x62b.google.com [IPv6:2a00:1450:4864:20::62b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 431472528E for ; Wed, 1 Jun 2022 03:11:54 -0700 (PDT) Received: by mail-ej1-x62b.google.com with SMTP id q1so2687067ejz.9 for ; Wed, 01 Jun 2022 03:11:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=0VLAI4Ml+SYHKdIrxqb9eLy4RuGSKwigpqa2hE5EhyM=; b=KdTk+xeAXQ8wsm4aSQlIU20whz1IffM5ydaFwWfMSZfBnChnZFwu+RsXjFj/7lC3NP q/bLiDhiB8llQRYKV8jfOjp/zRGu6Oj7aq1VciRM8fxWt8W2JAlIC+0ozACUPthXCcCb 3+4dGTpxNvEDJRp6T2Gyub+mS2i0g4/3OKsgzwVK8gciPoDRCoRfMoXPRiqQVOH53s9O N7xkmRlxSVVmbRFheCIkjZ8sSmuhslQ/iBHFBq4e6JGUpWgBNKpiN1tUAT/vLqRQohnm 0jm1mAyL1DIbO1jE1V0i+NH1A10gTA3QYwUtWqS1imlYNfOYqA8KOG+/RGyWdhw1UV07 6n4g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=0VLAI4Ml+SYHKdIrxqb9eLy4RuGSKwigpqa2hE5EhyM=; b=rhMFBcsfT29AUhcqKl2YeG0VL/XUIERNSJKbMMZcdL73AFdYv8eDt3VclT5c0/RRF9 56XgQ4a7wntWhvRPLC74WCaO69NZXkpdpqUca/AEGXaUCGuKlTEuXEUXacX0e1ePmzja 9qOLv2HMUjyMHuCIFfxrhviXkDRRKh54Dhvwj+tjzKxcZ1RsI803N5eLf/hcNE9UlwxM eOXTK9In86Xs/AdPqnaeOG8uVriERgJDGBjO2fsXK6+qK3auUyfwhGc52ROZPy606d4P RBiOTehqerOljwrfhe4iQrGN5EASVsCWYpDJTPtbj9nSRfPqqWSSJbUG8+RQY9Y6Yv5i D1Zg== X-Gm-Message-State: AOAM531Dp5h6yXarKyi4+cY/xN0dJQ7m7gTBrWLx2R9XgByVKJ2ViCvB vCzWra2Q52TasCWP1ib3ApqhoA== X-Google-Smtp-Source: ABdhPJxYMIZaapgsvY80qG/83T9mL1n8nrcG58gH9s/gwMkjWMv3ztULv3t9VCh7k6hav4WBZ1rz0w== X-Received: by 2002:a17:907:16ac:b0:6fe:b7db:8af5 with SMTP id hc44-20020a17090716ac00b006feb7db8af5mr48421039ejc.433.1654078312673; Wed, 01 Jun 2022 03:11:52 -0700 (PDT) Received: from localhost.localdomain (xdsl-188-155-176-92.adslplus.ch. [188.155.176.92]) by smtp.gmail.com with ESMTPSA id m9-20020a170906580900b006feb7b1379dsm501373ejq.181.2022.06.01.03.11.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 01 Jun 2022 03:11:52 -0700 (PDT) From: Krzysztof Kozlowski To: Andy Gross , Bjorn Andersson , Krzysztof Kozlowski , Georgi Djakov , Rob Herring , Catalin Marinas , Will Deacon , linux-arm-msm@vger.kernel.org, linux-pm@vger.kernel.org, devicetree@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org Cc: Thara Gopinath Subject: [PATCH v4 4/4] arm64: dts: qcom: sdm845: Add CPU BWMON Date: Wed, 1 Jun 2022 12:11:40 +0200 Message-Id: <20220601101140.170504-5-krzysztof.kozlowski@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220601101140.170504-1-krzysztof.kozlowski@linaro.org> References: <20220601101140.170504-1-krzysztof.kozlowski@linaro.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org Add device node for CPU-memory BWMON device (bandwidth monitoring) on SDM845 measuring bandwidth between CPU (gladiator_noc) and Last Level Cache (memnoc). Usage of this BWMON allows to remove fixed bandwidth votes from cpufreq (CPU nodes) thus achieve high memory throughput even with lower CPU frequencies. Performance impact (SDM845-MTP RB3 board, linux next-20220422): 1. No noticeable impact when running with schedutil or performance governors. 2. When comparing to customized kernel with synced interconnects and without bandwidth votes from CPU freq, the sysbench memory tests show significant improvement with bwmon for blocksizes past the L3 cache. The results for such superficial comparison: sysbench memory test, results in MB/s (higher is better) bs kB | type | V | V+no bw votes | bwmon | benefit % 1 | W/seq | 14795 | 4816 | 4985 | 3.5% 64 | W/seq | 41987 | 10334 | 10433 | 1.0% 4096 | W/seq | 29768 | 8728 | 32007 | 266.7% 65536 | W/seq | 17711 | 4846 | 18399 | 279.6% 262144 | W/seq | 16112 | 4538 | 17429 | 284.1% 64 | R/seq | 61202 | 67092 | 66804 | -0.4% 4096 | R/seq | 23871 | 5458 | 24307 | 345.4% 65536 | R/seq | 18554 | 4240 | 18685 | 340.7% 262144 | R/seq | 17524 | 4207 | 17774 | 322.4% 64 | W/rnd | 2663 | 1098 | 1119 | 1.9% 65536 | W/rnd | 600 | 316 | 610 | 92.7% 64 | R/rnd | 4915 | 4784 | 4594 | -4.0% 65536 | R/rnd | 664 | 281 | 678 | 140.7% Legend: bs kB: block size in KB (small block size means only L1-3 caches are used type: R - read, W - write, seq - sequential, rnd - random V: vanilla (next-20220422) V + no bw votes: vanilla without bandwidth votes from CPU freq bwmon: bwmon without bandwidth votes from CPU freq benefit %: difference between vanilla without bandwidth votes and bwmon (higher is better) Co-developed-by: Thara Gopinath Signed-off-by: Thara Gopinath Signed-off-by: Krzysztof Kozlowski --- arch/arm64/boot/dts/qcom/sdm845.dtsi | 54 ++++++++++++++++++++++++++++ 1 file changed, 54 insertions(+) diff --git a/arch/arm64/boot/dts/qcom/sdm845.dtsi b/arch/arm64/boot/dts/qcom/sdm845.dtsi index 83e8b63f0910..adffb9c70566 100644 --- a/arch/arm64/boot/dts/qcom/sdm845.dtsi +++ b/arch/arm64/boot/dts/qcom/sdm845.dtsi @@ -2026,6 +2026,60 @@ llcc: system-cache-controller@1100000 { interrupts = ; }; + pmu@1436400 { + compatible = "qcom,sdm845-cpu-bwmon"; + reg = <0 0x01436400 0 0x600>; + + interrupts = ; + + interconnects = <&gladiator_noc MASTER_APPSS_PROC 3 &mem_noc SLAVE_EBI1 3>, + <&osm_l3 MASTER_OSM_L3_APPS &osm_l3 SLAVE_OSM_L3>; + interconnect-names = "ddr", "l3c"; + + operating-points-v2 = <&cpu_bwmon_opp_table>; + + cpu_bwmon_opp_table: opp-table { + compatible = "operating-points-v2"; + + /* + * The interconnect paths bandwidths taken from + * cpu4_opp_table bandwidth. + * They also match different tables from + * msm-4.9 downstream kernel: + * - the gladiator_noc-mem_noc from bandwidth + * table of qcom,llccbw (property qcom,bw-tbl); + * bus width: 4 bytes; + * - the OSM L3 from bandwidth table of + * qcom,cpu4-l3lat-mon (qcom,core-dev-table); + * bus width: 16 bytes; + */ + opp-0 { + opp-peak-kBps = <800000 4800000>; + }; + opp-1 { + opp-peak-kBps = <1804000 9216000>; + }; + opp-2 { + opp-peak-kBps = <2188000 11980800>; + }; + opp-3 { + opp-peak-kBps = <3072000 15052800>; + }; + opp-4 { + opp-peak-kBps = <4068000 19353600>; + }; + opp-5 { + opp-peak-kBps = <5412000 20889600>; + }; + opp-6 { + opp-peak-kBps = <6220000 22425600>; + }; + opp-7 { + opp-peak-kBps = <7216000 25497600>; + }; + }; + }; + pcie0: pci@1c00000 { compatible = "qcom,pcie-sdm845"; reg = <0 0x01c00000 0 0x2000>, -- 2.34.1 From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 345E7C433F5 for ; Wed, 1 Jun 2022 10:28:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=/HsbKhuzr7Vgg3zbKh/Qj8F6k8tq/bmGBvx6vsyoymo=; b=jNTHGfr/YVE+td Elna99+PqVwzafOG61vUhOo6tcojd7xlY1Ai+a6UUizUEKG0N6y3gHOLnt06w5l5LMgcejH/zkr3w 203q8bdhwUiE+MoRIS6GDvNODvkOLMHhfLbgjgKjLx2jKZz2xhJHaXryYvdMBbwF7o4q3PKJ2e0+z /Ol0NbJEXWIC+a0uZdPzBmYTYYGdehQGMFGZzAEvuDdiS3hxCt/iIkMECDqA1zKFnf8UPqawfnOAz K7h1oxN5DK21YX+nWnRV61yz2YKYFFEEwFgwg/atOHcu8/1VpvKR3Vh1WEWLemkfdVblVeBopXX3T EtMb5uqZFd2QinLCO1Lg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1nwLZI-00FI6Y-KJ; Wed, 01 Jun 2022 10:27:13 +0000 Received: from mail-ej1-x630.google.com ([2a00:1450:4864:20::630]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1nwLKU-00FBUh-Pt for linux-arm-kernel@lists.infradead.org; Wed, 01 Jun 2022 10:11:56 +0000 Received: by mail-ej1-x630.google.com with SMTP id rq11so2730506ejc.4 for ; Wed, 01 Jun 2022 03:11:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=0VLAI4Ml+SYHKdIrxqb9eLy4RuGSKwigpqa2hE5EhyM=; b=KdTk+xeAXQ8wsm4aSQlIU20whz1IffM5ydaFwWfMSZfBnChnZFwu+RsXjFj/7lC3NP q/bLiDhiB8llQRYKV8jfOjp/zRGu6Oj7aq1VciRM8fxWt8W2JAlIC+0ozACUPthXCcCb 3+4dGTpxNvEDJRp6T2Gyub+mS2i0g4/3OKsgzwVK8gciPoDRCoRfMoXPRiqQVOH53s9O N7xkmRlxSVVmbRFheCIkjZ8sSmuhslQ/iBHFBq4e6JGUpWgBNKpiN1tUAT/vLqRQohnm 0jm1mAyL1DIbO1jE1V0i+NH1A10gTA3QYwUtWqS1imlYNfOYqA8KOG+/RGyWdhw1UV07 6n4g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=0VLAI4Ml+SYHKdIrxqb9eLy4RuGSKwigpqa2hE5EhyM=; b=UJLMM0CagfYi2lnbXlXZpYI0AWdc1S6fE4V9gHMCmYN4AYdkhMR/Bl4Jn3DhbRAvBw fF8BOCwdOd1gfFnOVDz0eZVtDfOnV1ZYYdoKhzvX+p2MLRuJM7DxErPeSbl5LlV/T8En WIaC+FP7KpuC0Cfjri/12jC3hXBTCGgDZSPZLE1vODZgQ+ihu/MTA5irGGXZl4MEkww3 XP8nKVVFqIOH72YI47YRLEiTHIu95XT5acARjVWW44kiumwfkxW+yZxnjS4oqphXouew kflfRSzwdR99+v94ND0Xnk1qDoZqt2pZvMkOsYO5B52GAMfyDPXqXlJkYEe0anzVQKLk 1R4A== X-Gm-Message-State: AOAM530dJakZhGenllrrXp7tPuOR+spmNCvOrnA6X2MYu4uyzOxrz26e vaFhOEyHZKxu7PIBkWwN1dxc2w== X-Google-Smtp-Source: ABdhPJxYMIZaapgsvY80qG/83T9mL1n8nrcG58gH9s/gwMkjWMv3ztULv3t9VCh7k6hav4WBZ1rz0w== X-Received: by 2002:a17:907:16ac:b0:6fe:b7db:8af5 with SMTP id hc44-20020a17090716ac00b006feb7db8af5mr48421039ejc.433.1654078312673; Wed, 01 Jun 2022 03:11:52 -0700 (PDT) Received: from localhost.localdomain (xdsl-188-155-176-92.adslplus.ch. [188.155.176.92]) by smtp.gmail.com with ESMTPSA id m9-20020a170906580900b006feb7b1379dsm501373ejq.181.2022.06.01.03.11.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 01 Jun 2022 03:11:52 -0700 (PDT) From: Krzysztof Kozlowski To: Andy Gross , Bjorn Andersson , Krzysztof Kozlowski , Georgi Djakov , Rob Herring , Catalin Marinas , Will Deacon , linux-arm-msm@vger.kernel.org, linux-pm@vger.kernel.org, devicetree@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org Cc: Thara Gopinath Subject: [PATCH v4 4/4] arm64: dts: qcom: sdm845: Add CPU BWMON Date: Wed, 1 Jun 2022 12:11:40 +0200 Message-Id: <20220601101140.170504-5-krzysztof.kozlowski@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220601101140.170504-1-krzysztof.kozlowski@linaro.org> References: <20220601101140.170504-1-krzysztof.kozlowski@linaro.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220601_031154_878239_A2218C71 X-CRM114-Status: GOOD ( 15.40 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Add device node for CPU-memory BWMON device (bandwidth monitoring) on SDM845 measuring bandwidth between CPU (gladiator_noc) and Last Level Cache (memnoc). Usage of this BWMON allows to remove fixed bandwidth votes from cpufreq (CPU nodes) thus achieve high memory throughput even with lower CPU frequencies. Performance impact (SDM845-MTP RB3 board, linux next-20220422): 1. No noticeable impact when running with schedutil or performance governors. 2. When comparing to customized kernel with synced interconnects and without bandwidth votes from CPU freq, the sysbench memory tests show significant improvement with bwmon for blocksizes past the L3 cache. The results for such superficial comparison: sysbench memory test, results in MB/s (higher is better) bs kB | type | V | V+no bw votes | bwmon | benefit % 1 | W/seq | 14795 | 4816 | 4985 | 3.5% 64 | W/seq | 41987 | 10334 | 10433 | 1.0% 4096 | W/seq | 29768 | 8728 | 32007 | 266.7% 65536 | W/seq | 17711 | 4846 | 18399 | 279.6% 262144 | W/seq | 16112 | 4538 | 17429 | 284.1% 64 | R/seq | 61202 | 67092 | 66804 | -0.4% 4096 | R/seq | 23871 | 5458 | 24307 | 345.4% 65536 | R/seq | 18554 | 4240 | 18685 | 340.7% 262144 | R/seq | 17524 | 4207 | 17774 | 322.4% 64 | W/rnd | 2663 | 1098 | 1119 | 1.9% 65536 | W/rnd | 600 | 316 | 610 | 92.7% 64 | R/rnd | 4915 | 4784 | 4594 | -4.0% 65536 | R/rnd | 664 | 281 | 678 | 140.7% Legend: bs kB: block size in KB (small block size means only L1-3 caches are used type: R - read, W - write, seq - sequential, rnd - random V: vanilla (next-20220422) V + no bw votes: vanilla without bandwidth votes from CPU freq bwmon: bwmon without bandwidth votes from CPU freq benefit %: difference between vanilla without bandwidth votes and bwmon (higher is better) Co-developed-by: Thara Gopinath Signed-off-by: Thara Gopinath Signed-off-by: Krzysztof Kozlowski --- arch/arm64/boot/dts/qcom/sdm845.dtsi | 54 ++++++++++++++++++++++++++++ 1 file changed, 54 insertions(+) diff --git a/arch/arm64/boot/dts/qcom/sdm845.dtsi b/arch/arm64/boot/dts/qcom/sdm845.dtsi index 83e8b63f0910..adffb9c70566 100644 --- a/arch/arm64/boot/dts/qcom/sdm845.dtsi +++ b/arch/arm64/boot/dts/qcom/sdm845.dtsi @@ -2026,6 +2026,60 @@ llcc: system-cache-controller@1100000 { interrupts = ; }; + pmu@1436400 { + compatible = "qcom,sdm845-cpu-bwmon"; + reg = <0 0x01436400 0 0x600>; + + interrupts = ; + + interconnects = <&gladiator_noc MASTER_APPSS_PROC 3 &mem_noc SLAVE_EBI1 3>, + <&osm_l3 MASTER_OSM_L3_APPS &osm_l3 SLAVE_OSM_L3>; + interconnect-names = "ddr", "l3c"; + + operating-points-v2 = <&cpu_bwmon_opp_table>; + + cpu_bwmon_opp_table: opp-table { + compatible = "operating-points-v2"; + + /* + * The interconnect paths bandwidths taken from + * cpu4_opp_table bandwidth. + * They also match different tables from + * msm-4.9 downstream kernel: + * - the gladiator_noc-mem_noc from bandwidth + * table of qcom,llccbw (property qcom,bw-tbl); + * bus width: 4 bytes; + * - the OSM L3 from bandwidth table of + * qcom,cpu4-l3lat-mon (qcom,core-dev-table); + * bus width: 16 bytes; + */ + opp-0 { + opp-peak-kBps = <800000 4800000>; + }; + opp-1 { + opp-peak-kBps = <1804000 9216000>; + }; + opp-2 { + opp-peak-kBps = <2188000 11980800>; + }; + opp-3 { + opp-peak-kBps = <3072000 15052800>; + }; + opp-4 { + opp-peak-kBps = <4068000 19353600>; + }; + opp-5 { + opp-peak-kBps = <5412000 20889600>; + }; + opp-6 { + opp-peak-kBps = <6220000 22425600>; + }; + opp-7 { + opp-peak-kBps = <7216000 25497600>; + }; + }; + }; + pcie0: pci@1c00000 { compatible = "qcom,pcie-sdm845"; reg = <0 0x01c00000 0 0x2000>, -- 2.34.1 _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel