[PATCH 0/4] soc: qcom: icc-bwmon: Add support for llcc and cpu bwmon on sc7280

[PATCH 0/4] soc: qcom: icc-bwmon: Add support for llcc and cpu bwmon on sc7280

[Rajendra Nayak]

This patchset adds support for cpu bwmon (bwmon4) and llcc bwmon (bwmon5) found
on sc7280 SoC.

Patchset is based on top of series [1] that adds support for llcc bwmon on sdm845

[1]
https://patchwork.kernel.org/project/linux-arm-msm/list/?series=663695&state=*

Rajendra Nayak (4):
  dt-bindings: interconnect: qcom,msm8998-bwmon: Add support for sc7280
    BWMONs
  soc: qcom: icc-bwmon: add support for sc7280 LLCC BWMON
  soc: qcom: icc-bwmon: force clear counter/irq registers
  arm64: dts: qcom: sc7280: Add cpu and llcc BWMON

 .../interconnect/qcom,msm8998-bwmon.yaml      |  2 +
 arch/arm64/boot/dts/qcom/sc7280.dtsi          | 76 +++++++++++++++++++
 drivers/soc/qcom/icc-bwmon.c                  | 21 +++++
 3 files changed, 99 insertions(+)

-- 
2.17.1

[PATCH 1/4] dt-bindings: interconnect: qcom,msm8998-bwmon: Add support for sc7280 BWMONs

[Rajendra Nayak]

Add a compatible for the cpu BWMON (version 4) instance and one
for the llcc BWMON (version 5) found in sc7280 SoC.

Signed-off-by: Rajendra Nayak <quic_rjendra@quicinc.com>
---
 .../devicetree/bindings/interconnect/qcom,msm8998-bwmon.yaml    | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/Documentation/devicetree/bindings/interconnect/qcom,msm8998-bwmon.yaml b/Documentation/devicetree/bindings/interconnect/qcom,msm8998-bwmon.yaml
index 32e2892d736b..cac915c5c2aa 100644
--- a/Documentation/devicetree/bindings/interconnect/qcom,msm8998-bwmon.yaml
+++ b/Documentation/devicetree/bindings/interconnect/qcom,msm8998-bwmon.yaml
@@ -25,8 +25,10 @@ properties:
       - items:
           - enum:
               - qcom,sdm845-bwmon
+              - qcom,sc7280-bwmon
           - const: qcom,msm8998-bwmon
       - const: qcom,msm8998-bwmon       # BWMON v4
+      - const: qcom,sc7280-llcc-bwmon   # BWMON v5
       - const: qcom,sdm845-llcc-bwmon   # BWMON v5
 
   interconnects:
-- 
2.17.1

[PATCH 2/4] soc: qcom: icc-bwmon: add support for sc7280 LLCC BWMON

[Rajendra Nayak]

Add support for sc7280 BWMON instance measuring traffic between LLCC and
memory with the v5 register layout.

Signed-off-by: Rajendra Nayak <quic_rjendra@quicinc.com>
---
 drivers/soc/qcom/icc-bwmon.c | 15 +++++++++++++++
 1 file changed, 15 insertions(+)

diff --git a/drivers/soc/qcom/icc-bwmon.c b/drivers/soc/qcom/icc-bwmon.c
index 47c2c3e7bb3f..44a10009b45e 100644
--- a/drivers/soc/qcom/icc-bwmon.c
+++ b/drivers/soc/qcom/icc-bwmon.c
@@ -656,6 +656,18 @@ static const struct icc_bwmon_data sdm845_llcc_bwmon_data = {
 	.regmap_cfg = &sdm845_llcc_bwmon_regmap_cfg,
 };
 
+static const struct icc_bwmon_data sc7280_llcc_bwmon_data = {
+	.sample_ms = 4,
+	.count_unit_kb = 64,
+	.default_highbw_kbps = 800 * 1024, /* 800 MBps */
+	.default_medbw_kbps = 256 * 1024, /* 256 MBps */
+	.default_lowbw_kbps = 0,
+	.zone1_thres_count = 16,
+	.zone3_thres_count = 1,
+	.regmap_fields = sdm845_llcc_bwmon_reg_fields,
+	.regmap_cfg = &sdm845_llcc_bwmon_regmap_cfg,
+};
+
 static const struct of_device_id bwmon_of_match[] = {
 	{
 		.compatible = "qcom,msm8998-bwmon",
@@ -663,6 +675,9 @@ static const struct of_device_id bwmon_of_match[] = {
 	}, {
 		.compatible = "qcom,sdm845-llcc-bwmon",
 		.data = &sdm845_llcc_bwmon_data
+	}, {
+		.compatible = "qcom,sc7280-llcc-bwmon",
+		.data = &sc7280_llcc_bwmon_data
 	},
 	{}
 };
-- 
2.17.1

[PATCH 3/4] soc: qcom: icc-bwmon: force clear counter/irq registers

[Rajendra Nayak]

In some SoCs we have to force clear the counter/irq clear registers as
they are not self clearing after they are written into.
sc7280 seems to be one such SoC, handle this with a quirk flag.

Signed-off-by: Rajendra Nayak <quic_rjendra@quicinc.com>
---
 drivers/soc/qcom/icc-bwmon.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/drivers/soc/qcom/icc-bwmon.c b/drivers/soc/qcom/icc-bwmon.c
index 44a10009b45e..17cba2648ae7 100644
--- a/drivers/soc/qcom/icc-bwmon.c
+++ b/drivers/soc/qcom/icc-bwmon.c
@@ -115,6 +115,7 @@
 
 /* Quirks for specific BWMON types */
 #define BWMON_HAS_GLOBAL_IRQ			BIT(0)
+#define BWMON_NEEDS_FORCE_CLEAR			BIT(1)
 
 enum bwmon_fields {
 	F_GLOBAL_IRQ_CLEAR,
@@ -343,6 +344,8 @@ static void bwmon_clear_counters(struct icc_bwmon *bwmon, bool clear_all)
 	 * before we try to clear the IRQ or do any other counter operations.
 	 */
 	regmap_field_force_write(bwmon->regs[F_CLEAR], val);
+	if (bwmon->data->quirks & BWMON_NEEDS_FORCE_CLEAR)
+		regmap_field_force_write(bwmon->regs[F_CLEAR], 0);
 }
 
 static void bwmon_clear_irq(struct icc_bwmon *bwmon)
@@ -364,6 +367,8 @@ static void bwmon_clear_irq(struct icc_bwmon *bwmon)
 	 * interrupt is cleared.
 	 */
 	regmap_field_force_write(bwmon->regs[F_IRQ_CLEAR], BWMON_IRQ_ENABLE_MASK);
+	if (bwmon->data->quirks & BWMON_NEEDS_FORCE_CLEAR)
+		regmap_field_force_write(bwmon->regs[F_IRQ_CLEAR], 0);
 	if (bwmon->data->quirks & BWMON_HAS_GLOBAL_IRQ)
 		regmap_field_force_write(bwmon->regs[F_GLOBAL_IRQ_CLEAR],
 					 BWMON_V4_GLOBAL_IRQ_ENABLE_ENABLE);
@@ -664,6 +669,7 @@ static const struct icc_bwmon_data sc7280_llcc_bwmon_data = {
 	.default_lowbw_kbps = 0,
 	.zone1_thres_count = 16,
 	.zone3_thres_count = 1,
+	.quirks = BWMON_NEEDS_FORCE_CLEAR,
 	.regmap_fields = sdm845_llcc_bwmon_reg_fields,
 	.regmap_cfg = &sdm845_llcc_bwmon_regmap_cfg,
 };
-- 
2.17.1

[PATCH 4/4] arm64: dts: qcom: sc7280: Add cpu and llcc BWMON

[Rajendra Nayak]

Add cpu and llcc BWMON nodes and their corresponding
OPP tables for sc7280 SoC.

Signed-off-by: Rajendra Nayak <quic_rjendra@quicinc.com>
---
 arch/arm64/boot/dts/qcom/sc7280.dtsi | 76 ++++++++++++++++++++++++++++
 1 file changed, 76 insertions(+)

diff --git a/arch/arm64/boot/dts/qcom/sc7280.dtsi b/arch/arm64/boot/dts/qcom/sc7280.dtsi
index 13d7f267b289..a839ba968d13 100644
--- a/arch/arm64/boot/dts/qcom/sc7280.dtsi
+++ b/arch/arm64/boot/dts/qcom/sc7280.dtsi
@@ -3275,6 +3275,82 @@
 			};
 		};
 
+		pmu@9091000 {
+			compatible = "qcom,sc7280-llcc-bwmon";
+			reg = <0 0x9091000 0 0x1000>;
+
+			interrupts = <GIC_SPI 81 IRQ_TYPE_LEVEL_HIGH>;
+
+			interconnects = <&mc_virt MASTER_LLCC 3 &mc_virt SLAVE_EBI1 3>;
+
+			operating-points-v2 = <&llcc_bwmon_opp_table>;
+
+			llcc_bwmon_opp_table: opp-table {
+				compatible = "operating-points-v2";
+
+				opp-0 {
+					opp-peak-kBps = <800000>;
+				};
+				opp-1 {
+					opp-peak-kBps = <1804000>;
+				};
+				opp-2 {
+					opp-peak-kBps = <2188000>;
+				};
+				opp-3 {
+					opp-peak-kBps = <3072000>;
+				};
+				opp-4 {
+					opp-peak-kBps = <4068000>;
+				};
+				opp-5 {
+					opp-peak-kBps = <6220000>;
+				};
+				opp-6 {
+					opp-peak-kBps = <6832000>;
+				};
+				opp-7 {
+					opp-peak-kBps = <8532000>;
+				};
+			};
+		};
+
+		pmu@90b6000 {
+			compatible = "qcom,sc7280-cpu-bwmon", "qcom,msm8998-bwmon";
+			reg = <0 0x090b6400 0 0x600>;
+
+			interrupts = <GIC_SPI 581 IRQ_TYPE_LEVEL_HIGH>;
+
+			interconnects = <&gem_noc MASTER_APPSS_PROC 3 &gem_noc SLAVE_LLCC 3>;
+			operating-points-v2 = <&cpu_bwmon_opp_table>;
+
+			cpu_bwmon_opp_table: opp-table {
+				compatible = "operating-points-v2";
+
+				opp-0 {
+					opp-peak-kBps = <2400000>;
+				};
+				opp-1 {
+					opp-peak-kBps = <4800000>;
+				};
+				opp-2 {
+					opp-peak-kBps = <7456000>;
+				};
+				opp-3 {
+					opp-peak-kBps = <9600000>;
+				};
+				opp-4 {
+					opp-peak-kBps = <12896000>;
+				};
+				opp-5 {
+					opp-peak-kBps = <14928000>;
+				};
+				opp-6 {
+					opp-peak-kBps = <17056000>;
+				};
+			};
+		};
+
 		dc_noc: interconnect@90e0000 {
 			reg = <0 0x090e0000 0 0x5080>;
 			compatible = "qcom,sc7280-dc-noc";
-- 
2.17.1

Re: [PATCH 1/4] dt-bindings: interconnect: qcom,msm8998-bwmon: Add support for sc7280 BWMONs

[Krzysztof Kozlowski]

On 01/09/2022 15:47, Rajendra Nayak wrote:
> Add a compatible for the cpu BWMON (version 4) instance and one
> for the llcc BWMON (version 5) found in sc7280 SoC.

+Cc of few Linaro folks.

Awesome! I see bwmon is being used! Rajendra, do you have any particular
needs/todos for bwmon or other related pieces?


> 
> Signed-off-by: Rajendra Nayak <quic_rjendra@quicinc.com>
> ---
>  .../devicetree/bindings/interconnect/qcom,msm8998-bwmon.yaml    | 2 ++
>  1 file changed, 2 insertions(+)
> 
> diff --git a/Documentation/devicetree/bindings/interconnect/qcom,msm8998-bwmon.yaml b/Documentation/devicetree/bindings/interconnect/qcom,msm8998-bwmon.yaml
> index 32e2892d736b..cac915c5c2aa 100644
> --- a/Documentation/devicetree/bindings/interconnect/qcom,msm8998-bwmon.yaml
> +++ b/Documentation/devicetree/bindings/interconnect/qcom,msm8998-bwmon.yaml
> @@ -25,8 +25,10 @@ properties:
>        - items:
>            - enum:
>                - qcom,sdm845-bwmon
> +              - qcom,sc7280-bwmon

Could you put it in alphabetical order, so before sdm845?

With above:

Reviewed-by: Krzysztof Kozlowski <krzysztof.kozlowski@linaro.org>

Best regards,
Krzysztof

Re: [PATCH 2/4] soc: qcom: icc-bwmon: add support for sc7280 LLCC BWMON

[Krzysztof Kozlowski]

On 01/09/2022 15:47, Rajendra Nayak wrote:
> Add support for sc7280 BWMON instance measuring traffic between LLCC and
> memory with the v5 register layout.
> 
> Signed-off-by: Rajendra Nayak <quic_rjendra@quicinc.com>
> ---
>  drivers/soc/qcom/icc-bwmon.c | 15 +++++++++++++++
>  1 file changed, 15 insertions(+)
> 
> diff --git a/drivers/soc/qcom/icc-bwmon.c b/drivers/soc/qcom/icc-bwmon.c
> index 47c2c3e7bb3f..44a10009b45e 100644
> --- a/drivers/soc/qcom/icc-bwmon.c
> +++ b/drivers/soc/qcom/icc-bwmon.c
> @@ -656,6 +656,18 @@ static const struct icc_bwmon_data sdm845_llcc_bwmon_data = {
>  	.regmap_cfg = &sdm845_llcc_bwmon_regmap_cfg,
>  };
>  
> +static const struct icc_bwmon_data sc7280_llcc_bwmon_data = {
> +	.sample_ms = 4,
> +	.count_unit_kb = 64,

This makes me wonder if I put correct count unit for SDM845 LLCC...

Reviewed-by: Krzysztof Kozlowski <krzysztof.kozlowski@linaro.org>

Best regards,
Krzysztof

Re: [PATCH 3/4] soc: qcom: icc-bwmon: force clear counter/irq registers

[Krzysztof Kozlowski]

On 01/09/2022 15:47, Rajendra Nayak wrote:
> In some SoCs we have to force clear the counter/irq clear registers as
> they are not self clearing after they are written into.
> sc7280 seems to be one such SoC, handle this with a quirk flag.
> 
> Signed-off-by: Rajendra Nayak <quic_rjendra@quicinc.com>


Reviewed-by: Krzysztof Kozlowski <krzysztof.kozlowski@linaro.org>


Best regards,
Krzysztof

Re: [PATCH 4/4] arm64: dts: qcom: sc7280: Add cpu and llcc BWMON

[Krzysztof Kozlowski]

On 01/09/2022 15:47, Rajendra Nayak wrote:
> Add cpu and llcc BWMON nodes and their corresponding
> OPP tables for sc7280 SoC.
> 
> Signed-off-by: Rajendra Nayak <quic_rjendra@quicinc.com>


Reviewed-by: Krzysztof Kozlowski <krzysztof.kozlowski@linaro.org>


Best regards,
Krzysztof

Re: [PATCH 1/4] dt-bindings: interconnect: qcom,msm8998-bwmon: Add support for sc7280 BWMONs

[Rajendra Nayak]

On 9/1/2022 8:55 PM, Krzysztof Kozlowski wrote:
> On 01/09/2022 15:47, Rajendra Nayak wrote:
>> Add a compatible for the cpu BWMON (version 4) instance and one
>> for the llcc BWMON (version 5) found in sc7280 SoC.
> 
> +Cc of few Linaro folks.
> 
> Awesome! I see bwmon is being used! Rajendra, do you have any particular
> needs/todos for bwmon or other related pieces?

Thanks Krzysztof, nothing at the moment, things just worked out of the box
(except for the PATCH 3/4 that took a while to uncover)
We are still analyzing the perf/power numbers on sc7280 based ChromeOS devices,
along with some other yet to be posted changes to enable memory latency
based governors in firmware, but all is good with bwmon drivers for now.

> 
>>
>> Signed-off-by: Rajendra Nayak <quic_rjendra@quicinc.com>
>> ---
>>   .../devicetree/bindings/interconnect/qcom,msm8998-bwmon.yaml    | 2 ++
>>   1 file changed, 2 insertions(+)
>>
>> diff --git a/Documentation/devicetree/bindings/interconnect/qcom,msm8998-bwmon.yaml b/Documentation/devicetree/bindings/interconnect/qcom,msm8998-bwmon.yaml
>> index 32e2892d736b..cac915c5c2aa 100644
>> --- a/Documentation/devicetree/bindings/interconnect/qcom,msm8998-bwmon.yaml
>> +++ b/Documentation/devicetree/bindings/interconnect/qcom,msm8998-bwmon.yaml
>> @@ -25,8 +25,10 @@ properties:
>>         - items:
>>             - enum:
>>                 - qcom,sdm845-bwmon
>> +              - qcom,sc7280-bwmon
> 
> Could you put it in alphabetical order, so before sdm845?

ah right, will fix and repost, thanks.
  
> With above:
> 
> Reviewed-by: Krzysztof Kozlowski <krzysztof.kozlowski@linaro.org>
> 
> Best regards,
> Krzysztof

Re: [PATCH 2/4] soc: qcom: icc-bwmon: add support for sc7280 LLCC BWMON

[Rajendra Nayak]

On 9/1/2022 8:57 PM, Krzysztof Kozlowski wrote:
> On 01/09/2022 15:47, Rajendra Nayak wrote:
>> Add support for sc7280 BWMON instance measuring traffic between LLCC and
>> memory with the v5 register layout.
>>
>> Signed-off-by: Rajendra Nayak <quic_rjendra@quicinc.com>
>> ---
>>   drivers/soc/qcom/icc-bwmon.c | 15 +++++++++++++++
>>   1 file changed, 15 insertions(+)
>>
>> diff --git a/drivers/soc/qcom/icc-bwmon.c b/drivers/soc/qcom/icc-bwmon.c
>> index 47c2c3e7bb3f..44a10009b45e 100644
>> --- a/drivers/soc/qcom/icc-bwmon.c
>> +++ b/drivers/soc/qcom/icc-bwmon.c
>> @@ -656,6 +656,18 @@ static const struct icc_bwmon_data sdm845_llcc_bwmon_data = {
>>   	.regmap_cfg = &sdm845_llcc_bwmon_regmap_cfg,
>>   };
>>   
>> +static const struct icc_bwmon_data sc7280_llcc_bwmon_data = {
>> +	.sample_ms = 4,
>> +	.count_unit_kb = 64,
> 
> This makes me wonder if I put correct count unit for SDM845 LLCC...

These numbers seem to vary from SoC to SoC, I looked up the sdm845.dtsi from
the CAF kernels [1] and it seems like it should be 4096 instead of 1024?

[1] https://source.codeaurora.org/quic/la/kernel/msm-4.9/tree/arch/arm64/boot/dts/qcom/sdm845.dtsi
  
> 
> Reviewed-by: Krzysztof Kozlowski <krzysztof.kozlowski@linaro.org>
> 
> Best regards,
> Krzysztof

Re: [PATCH 2/4] soc: qcom: icc-bwmon: add support for sc7280 LLCC BWMON

[Krzysztof Kozlowski]

On 02/09/2022 06:04, Rajendra Nayak wrote:
> 
> 
> On 9/1/2022 8:57 PM, Krzysztof Kozlowski wrote:
>> On 01/09/2022 15:47, Rajendra Nayak wrote:
>>> Add support for sc7280 BWMON instance measuring traffic between LLCC and
>>> memory with the v5 register layout.
>>>
>>> Signed-off-by: Rajendra Nayak <quic_rjendra@quicinc.com>
>>> ---
>>>   drivers/soc/qcom/icc-bwmon.c | 15 +++++++++++++++
>>>   1 file changed, 15 insertions(+)
>>>
>>> diff --git a/drivers/soc/qcom/icc-bwmon.c b/drivers/soc/qcom/icc-bwmon.c
>>> index 47c2c3e7bb3f..44a10009b45e 100644
>>> --- a/drivers/soc/qcom/icc-bwmon.c
>>> +++ b/drivers/soc/qcom/icc-bwmon.c
>>> @@ -656,6 +656,18 @@ static const struct icc_bwmon_data sdm845_llcc_bwmon_data = {
>>>   	.regmap_cfg = &sdm845_llcc_bwmon_regmap_cfg,
>>>   };
>>>   
>>> +static const struct icc_bwmon_data sc7280_llcc_bwmon_data = {
>>> +	.sample_ms = 4,
>>> +	.count_unit_kb = 64,
>>
>> This makes me wonder if I put correct count unit for SDM845 LLCC...
> 
> These numbers seem to vary from SoC to SoC, 

Hm, then it is probably ok.

> I looked up the sdm845.dtsi from
> the CAF kernels [1] and it seems like it should be 4096 instead of 1024?

The SDM845 v2 DTSI was saying 64 kB for CPU bwmon and indeed 4 MB for
LLCC. I think I took 1 MB from default value from the msm-4.9 driver and
it matched my measurements. I need to test it again and maybe fix it to
4 MB.

Best regards,
Krzysztof