[PATCHv3 0/5] arm-cci500: Workaround pmu_event_set_period

[PATCHv3 0/5] arm-cci500: Workaround pmu_event_set_period

[Suzuki K. Poulose]

The CCI PMU driver sets the event counter to the half of the maximum
value(2^31) it can count before we start the counters via
pmu_event_set_period(). This is done to give us the best chance to
handle the overflow interrupt, taking care of extreme interrupt latencies.

However, CCI-500 comes with advanced power saving schemes, which disables
the clock to the event counters unless the counters are enabled to count
(PMCR.CEN). This prevents the driver from writing the period to the
counters before starting them.  Also, there is no way we can reset the
individual event counter to 0 (PMCR.RST resets all the counters, losing
their current readings). However the value of the counter is preserved and
could be read back, when the counters are not enabled.

So we cannot reliably use the counters and compute the number of events
generated during the sampling period since we don't have the value of the
counter at start.

Here are the possible solutions:

 1) Disable clock gating on CCI-500 by setting Control_Override_Reg[bit3].
    - The Control_Override_Reg is secure (and hence not programmable from
      Linux), and also has an impact on power consumption.

 2) Change the order of operations
	i.e,
	a) Program and enable individual counters
	b) Enable counting on all the counters by setting PMCR.CEN
	c) Write the period to the individual counters
	d) Disable the counters
    - This could cause in unnecessary noise in the other counters and is
      costly (we should repeat this for all enabled counters).

 3) Don't set the counter value, instead use the current count as the
    starting count and compute the delta at the end of sampling.

 4) Modified version of 2, which disables all the other counters, except
    the target counter, with the target counter programmed with an invalid
    event code(which guarantees that the counter won't change during the
    operation).

This patch implements option 4 for CCI-500. CCI-400 behavior remains
unchanged.

Changes since V2:
 - Rebased to 4.4-rc1 + Mark's patch to simply PMU syfs attributes [1]
 - Address comments on v2.
 - Split the introduction of write_counter hook to a separate patch

Changes since V1:
 - Choose 4 instead of 3 above, suggested by Mark Rutland

[1] http://lists.infradead.org/pipermail/linux-arm-kernel/2015-September/373129.html

Suzuki K. Poulose (5):
  arm-cci: Refactor CCI PMU enable/disable methods
  arm-cci: Get the status of a counter
  arm-cci: Add routines to enable/disable all counters
  arm-cci: Add hooks for pmu_write_counter
  arm-cci: CCI-500: Work around PMU counter writes

 drivers/bus/arm-cci.c |  143 ++++++++++++++++++++++++++++++++++++++++++++-----
 1 file changed, 131 insertions(+), 12 deletions(-)

-- 
1.7.9.5

[PATCHv3 1/5] arm-cci: Refactor CCI PMU enable/disable methods

[Suzuki K. Poulose]

This patch refactors the CCI PMU driver code a little bit to
make it easier share the code for enabling/disabling the CCI
PMU. This will be used by the hooks to work around the special cases
where writing to a counter is not always that easy(e.g, CCI-500)

No functional changes.

Cc: Punit Agrawal <punit.agrawal@arm.com>
Cc: Mark Rutland <mark.rutland@arm.com>
Signed-off-by: Suzuki K. Poulose <suzuki.poulose@arm.com>
---
 drivers/bus/arm-cci.c |   32 ++++++++++++++++++++++----------
 1 file changed, 22 insertions(+), 10 deletions(-)

diff --git a/drivers/bus/arm-cci.c b/drivers/bus/arm-cci.c
index ee47e6b..f5793b9 100644
--- a/drivers/bus/arm-cci.c
+++ b/drivers/bus/arm-cci.c
@@ -667,6 +667,26 @@ static u32 pmu_get_max_counters(void)
 		CCI_PMCR_NCNT_MASK) >> CCI_PMCR_NCNT_SHIFT;
 }
 
+/* Should be called with cci_pmu->hw_events->pmu_lock held */
+static void __cci_pmu_enable(void)
+{
+	u32 val;
+
+	/* Enable all the PMU counters. */
+	val = readl_relaxed(cci_ctrl_base + CCI_PMCR) | CCI_PMCR_CEN;
+	writel(val, cci_ctrl_base + CCI_PMCR);
+}
+
+/* Should be called with cci_pmu->hw_events->pmu_lock held */
+static void __cci_pmu_disable(void)
+{
+	u32 val;
+
+	/* Disable all the PMU counters. */
+	val = readl_relaxed(cci_ctrl_base + CCI_PMCR) & ~CCI_PMCR_CEN;
+	writel(val, cci_ctrl_base + CCI_PMCR);
+}
+
 static int pmu_get_event_idx(struct cci_pmu_hw_events *hw, struct perf_event *event)
 {
 	struct cci_pmu *cci_pmu = to_cci_pmu(event->pmu);
@@ -880,16 +900,12 @@ static void cci_pmu_enable(struct pmu *pmu)
 	struct cci_pmu_hw_events *hw_events = &cci_pmu->hw_events;
 	int enabled = bitmap_weight(hw_events->used_mask, cci_pmu->num_cntrs);
 	unsigned long flags;
-	u32 val;
 
 	if (!enabled)
 		return;
 
 	raw_spin_lock_irqsave(&hw_events->pmu_lock, flags);
-
-	/* Enable all the PMU counters. */
-	val = readl_relaxed(cci_ctrl_base + CCI_PMCR) | CCI_PMCR_CEN;
-	writel(val, cci_ctrl_base + CCI_PMCR);
+	__cci_pmu_enable();
 	raw_spin_unlock_irqrestore(&hw_events->pmu_lock, flags);
 
 }
@@ -899,13 +915,9 @@ static void cci_pmu_disable(struct pmu *pmu)
 	struct cci_pmu *cci_pmu = to_cci_pmu(pmu);
 	struct cci_pmu_hw_events *hw_events = &cci_pmu->hw_events;
 	unsigned long flags;
-	u32 val;
 
 	raw_spin_lock_irqsave(&hw_events->pmu_lock, flags);
-
-	/* Disable all the PMU counters. */
-	val = readl_relaxed(cci_ctrl_base + CCI_PMCR) & ~CCI_PMCR_CEN;
-	writel(val, cci_ctrl_base + CCI_PMCR);
+	__cci_pmu_disable();
 	raw_spin_unlock_irqrestore(&hw_events->pmu_lock, flags);
 }
 
-- 
1.7.9.5

[PATCHv3 2/5] arm-cci: Get the status of a counter

[Suzuki K. Poulose]

Add helper routines to get the counter status and the event
programmed on it.

Cc: Punit Agrawal <punit.agrawal@arm.com>
Cc: Mark Rutland <mark.rutland@arm.com>
Signed-off-by: Suzuki K. Poulose <suzuki.poulose@arm.com>
---
Changes since V2:
 - Rename pmu_counter_get_ctrl => pmu_counter_is_enabled()
---
 drivers/bus/arm-cci.c |   12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/drivers/bus/arm-cci.c b/drivers/bus/arm-cci.c
index f5793b9..48936c8 100644
--- a/drivers/bus/arm-cci.c
+++ b/drivers/bus/arm-cci.c
@@ -652,11 +652,23 @@ static void pmu_enable_counter(struct cci_pmu *cci_pmu, int idx)
 	pmu_write_register(cci_pmu, 1, idx, CCI_PMU_CNTR_CTRL);
 }
 
+static bool __maybe_unused
+pmu_counter_is_enabled(struct cci_pmu *cci_pmu, int idx)
+{
+	return (pmu_read_register(cci_pmu, idx, CCI_PMU_CNTR_CTRL) & 0x1) != 0;
+}
+
 static void pmu_set_event(struct cci_pmu *cci_pmu, int idx, unsigned long event)
 {
 	pmu_write_register(cci_pmu, event, idx, CCI_PMU_EVT_SEL);
 }
 
+static u32 __maybe_unused
+pmu_get_event(struct cci_pmu *cci_pmu, int idx)
+{
+	return pmu_read_register(cci_pmu, idx, CCI_PMU_EVT_SEL);
+}
+
 /*
  * Returns the number of programmable counters actually implemented
  * by the cci
-- 
1.7.9.5

[PATCHv3 3/5] arm-cci: Add routines to enable/disable all counters

[Suzuki K. Poulose]

Adds helper routines to disable the counter controls for
all the counters on the CCI PMU and restore it back, by
preserving the original state in caller provided mask.

Cc: Punit Agrawal <punit.agrawal@arm.com>
Cc: Mark Rutland <mark.rutland@arm.com>
Signed-off-by: Suzuki K. Poulose <suzuki.poulose@arm.com>
---
Changes since V2:
  - Rename the functions to pmu_restore_counters, pmu_disable_counters
  - Added comment describing why we use a private mask
---
 drivers/bus/arm-cci.c |   38 ++++++++++++++++++++++++++++++++++++++
 1 file changed, 38 insertions(+)

diff --git a/drivers/bus/arm-cci.c b/drivers/bus/arm-cci.c
index 48936c8..91a9d5d 100644
--- a/drivers/bus/arm-cci.c
+++ b/drivers/bus/arm-cci.c
@@ -669,6 +669,44 @@ pmu_get_event(struct cci_pmu *cci_pmu, int idx)
 	return pmu_read_register(cci_pmu, idx, CCI_PMU_EVT_SEL);
 }
 
+
+/*
+ * For all counters on the CCI-PMU, disable any 'enabled' counters,
+ * saving the changed counters in the mask, so that we can restore
+ * it later using pmu_restore_counters. The mask is private to the
+ * caller. We cannot rely on the used_mask maintained by the CCI_PMU
+ * as it only tells us if the counter is assigned to perf_event or not.
+ * The state of the perf_event cannot be locked by the PMU layer, hence
+ * we check the individual counter status (which can be locked by
+ * cci_pm->hw_events->pmu_lock).
+ */
+static void __maybe_unused
+pmu_disable_counters(struct cci_pmu *cci_pmu, unsigned long *mask)
+{
+	int i;
+
+	for (i = 0; i < cci_pmu->num_cntrs; i++) {
+		if (pmu_counter_is_enabled(cci_pmu, i)) {
+			set_bit(i, mask);
+			pmu_disable_counter(cci_pmu, i);
+		} else
+			clear_bit(i, mask);
+	}
+}
+
+/*
+ * Restore the status of the counters. Reversal of the pmu_disable_counters().
+ * For each counter set in the mask, enable the counter back.
+ */
+static void __maybe_unused
+pmu_restore_counters(struct cci_pmu *cci_pmu, unsigned long *mask)
+{
+	int i;
+
+	for_each_set_bit(i, mask, cci_pmu->num_cntrs)
+		pmu_enable_counter(cci_pmu, i);
+}
+
 /*
  * Returns the number of programmable counters actually implemented
  * by the cci
-- 
1.7.9.5

[PATCHv3 4/5] arm-cci: Add hooks for pmu_write_counter

[Suzuki K. Poulose]

Add support for model specific methods to write to the PMU
event counter. This will be used by CCI500/550 PMUs.

Cc: Punit Agrawal <punit.agrawal@arm.com>
Cc: Mark Rutland <mark.rutland@arm.com>
Signed-off-by: Suzuki K. Poulose <suzuki.poulose@arm.com>
---
 drivers/bus/arm-cci.c |   14 ++++++++++++--
 1 file changed, 12 insertions(+), 2 deletions(-)

diff --git a/drivers/bus/arm-cci.c b/drivers/bus/arm-cci.c
index 91a9d5d..88b612f 100644
--- a/drivers/bus/arm-cci.c
+++ b/drivers/bus/arm-cci.c
@@ -126,6 +126,7 @@ struct cci_pmu_model {
 	struct event_range event_ranges[CCI_IF_MAX];
 	int (*validate_hw_event)(struct cci_pmu *, unsigned long);
 	int (*get_event_idx)(struct cci_pmu *, struct cci_pmu_hw_events *, unsigned long);
+	void (*write_counter)(struct cci_pmu *, u32, int);
 };
 
 static struct cci_pmu_model cci_pmu_models[];
@@ -829,16 +830,25 @@ static u32 pmu_read_counter(struct perf_event *event)
 	return value;
 }
 
+static void __pmu_write_counter(struct cci_pmu *cci_pmu, u32 value, int idx)
+{
+	pmu_write_register(cci_pmu, value, idx, CCI_PMU_CNTR);
+}
+
 static void pmu_write_counter(struct perf_event *event, u32 value)
 {
 	struct cci_pmu *cci_pmu = to_cci_pmu(event->pmu);
 	struct hw_perf_event *hw_counter = &event->hw;
 	int idx = hw_counter->idx;
 
-	if (unlikely(!pmu_is_valid_counter(cci_pmu, idx)))
+	if (unlikely(!pmu_is_valid_counter(cci_pmu, idx))) {
 		dev_err(&cci_pmu->plat_device->dev, "Invalid CCI PMU counter %d\n", idx);
+		return;
+	}
+	if (cci_pmu->model->write_counter)
+		cci_pmu->model->write_counter(cci_pmu, value, idx);
 	else
-		pmu_write_register(cci_pmu, value, idx, CCI_PMU_CNTR);
+		__pmu_write_counter(cci_pmu, value, idx);
 }
 
 static u64 pmu_event_update(struct perf_event *event)
-- 
1.7.9.5

[PATCHv3 5/5] arm-cci: CCI-500: Work around PMU counter writes

[Suzuki K. Poulose]

The CCI PMU driver sets the event counter to the half of the maximum
value(2^31) it can count before we start the counters via
pmu_event_set_period(). This is done to give us the best chance to
handle the overflow interrupt, taking care of extreme interrupt latencies.

However, CCI-500 comes with advanced power saving schemes, which
disables the clock to the event counters unless the counters are enabled to
count (PMCR.CEN). This prevents the driver from writing the period to the
counters before starting them.  Also, there is no way we can reset the
individual event counter to 0 (PMCR.RST resets all the counters, losing
their current readings). However the value of the counter is preserved and
could be read back, when the counters are not enabled.

So we cannot reliably use the counters and compute the number of events
generated during the sampling period since we don't have the value of the
counter at start.

This patch works around this issue by changing writes to the counter
with the following steps.

 1) Disable all the counters (remembering any counters which were enabled)
 2) Save the current event and program the target counter to count an
    invalid event, which by spec is guaranteed to not-generate any events.
 3) Enable the target counter.
 4) Enable the CCI PMU
 5) Write to the target counter.
 6) Disable the CCI PMU and the target counter
 7) Restore the event back on the target counter.
 8) Restore the status of the all the counters

Cc: Punit Agrawal <punit.agrawal@arm.com>
Cc: Mark Rutland <mark.rutland@arm.com>
Signed-off-by: Suzuki K. Poulose <suzuki.poulose@arm.com>
---
 drivers/bus/arm-cci.c |   47 +++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 47 insertions(+)

diff --git a/drivers/bus/arm-cci.c b/drivers/bus/arm-cci.c
index 88b612f..6020a02 100644
--- a/drivers/bus/arm-cci.c
+++ b/drivers/bus/arm-cci.c
@@ -835,6 +835,52 @@ static void __pmu_write_counter(struct cci_pmu *cci_pmu, u32 value, int idx)
 	pmu_write_register(cci_pmu, value, idx, CCI_PMU_CNTR);
 }
 
+#ifdef CONFIG_ARM_CCI500_PMU
+
+/*
+ * CCI-500 has advanced power saving policies, which could gate the
+ * clocks to the PMU counters, which makes the writes to them ineffective.
+ * The only way to write to those counters is when the global counters
+ * are enabled and the particular counter is enabled.
+ *
+ * So we do the following :
+ *
+ * 1) Disable all the PMU counters, saving their current state
+ * 2) Save the programmed event, and write an invalid event code
+ *    to the event control register for the counter, so that the
+ *    counters are not modified.
+ * 3) Enable the counter control for the counter.
+ * 4) Enable the global PMU profiling
+ * 5) Set the counter value
+ * 6) Disable the counter, global PMU.
+ * 7) Restore the event in the target counter
+ * 8) Restore the status of the rest of the counters.
+ *
+ * We choose an event code which has very little chances of getting
+ * assigned a valid code for step(2). We use the highest possible
+ * event code (0x1f) for the master interface 0.
+ */
+#define CCI500_INVALID_EVENT	((CCI500_PORT_M0 << CCI500_PMU_EVENT_SOURCE_SHIFT) | \
+				 (CCI500_PMU_EVENT_CODE_MASK << CCI500_PMU_EVENT_CODE_SHIFT))
+static void cci500_pmu_write_counter(struct cci_pmu *cci_pmu, u32 value, int idx)
+{
+	unsigned long mask[BITS_TO_LONGS(cci_pmu->num_cntrs)];
+	u32 event;
+
+	pmu_disable_counters(cci_pmu, mask);
+	event = pmu_get_event(cci_pmu, idx);
+	pmu_set_event(cci_pmu, idx, CCI500_INVALID_EVENT);
+	pmu_enable_counter(cci_pmu, idx);
+	__cci_pmu_enable();
+	__pmu_write_counter(cci_pmu, value, idx);
+	__cci_pmu_disable();
+	pmu_disable_counter(cci_pmu, idx);
+	pmu_set_event(cci_pmu, idx, event);
+	pmu_restore_counters(cci_pmu, mask);
+}
+
+#endif	/* CONFIG_ARM_CCI500_PMU */
+
 static void pmu_write_counter(struct perf_event *event, u32 value)
 {
 	struct cci_pmu *cci_pmu = to_cci_pmu(event->pmu);
@@ -1422,6 +1468,7 @@ static struct cci_pmu_model cci_pmu_models[] = {
 			},
 		},
 		.validate_hw_event = cci500_validate_hw_event,
+		.write_counter	= cci500_pmu_write_counter,
 	},
 #endif
 };
-- 
1.7.9.5

Re: [PATCHv3 1/5] arm-cci: Refactor CCI PMU enable/disable methods

[Mark Rutland]

On Tue, Nov 17, 2015 at 06:03:23PM +0000, Suzuki K. Poulose wrote:
> This patch refactors the CCI PMU driver code a little bit to
> make it easier share the code for enabling/disabling the CCI
> PMU. This will be used by the hooks to work around the special cases
> where writing to a counter is not always that easy(e.g, CCI-500)
> 
> No functional changes.

Looks sensible to me:

Acked-by: Mark Rutland <mark.rutland@arm.com>

Mark.

> Cc: Punit Agrawal <punit.agrawal@arm.com>
> Cc: Mark Rutland <mark.rutland@arm.com>
> Signed-off-by: Suzuki K. Poulose <suzuki.poulose@arm.com>
> ---
>  drivers/bus/arm-cci.c |   32 ++++++++++++++++++++++----------
>  1 file changed, 22 insertions(+), 10 deletions(-)
> 
> diff --git a/drivers/bus/arm-cci.c b/drivers/bus/arm-cci.c
> index ee47e6b..f5793b9 100644
> --- a/drivers/bus/arm-cci.c
> +++ b/drivers/bus/arm-cci.c
> @@ -667,6 +667,26 @@ static u32 pmu_get_max_counters(void)
>  		CCI_PMCR_NCNT_MASK) >> CCI_PMCR_NCNT_SHIFT;
>  }
>  
> +/* Should be called with cci_pmu->hw_events->pmu_lock held */
> +static void __cci_pmu_enable(void)
> +{
> +	u32 val;
> +
> +	/* Enable all the PMU counters. */
> +	val = readl_relaxed(cci_ctrl_base + CCI_PMCR) | CCI_PMCR_CEN;
> +	writel(val, cci_ctrl_base + CCI_PMCR);
> +}
> +
> +/* Should be called with cci_pmu->hw_events->pmu_lock held */
> +static void __cci_pmu_disable(void)
> +{
> +	u32 val;
> +
> +	/* Disable all the PMU counters. */
> +	val = readl_relaxed(cci_ctrl_base + CCI_PMCR) & ~CCI_PMCR_CEN;
> +	writel(val, cci_ctrl_base + CCI_PMCR);
> +}
> +
>  static int pmu_get_event_idx(struct cci_pmu_hw_events *hw, struct perf_event *event)
>  {
>  	struct cci_pmu *cci_pmu = to_cci_pmu(event->pmu);
> @@ -880,16 +900,12 @@ static void cci_pmu_enable(struct pmu *pmu)
>  	struct cci_pmu_hw_events *hw_events = &cci_pmu->hw_events;
>  	int enabled = bitmap_weight(hw_events->used_mask, cci_pmu->num_cntrs);
>  	unsigned long flags;
> -	u32 val;
>  
>  	if (!enabled)
>  		return;
>  
>  	raw_spin_lock_irqsave(&hw_events->pmu_lock, flags);
> -
> -	/* Enable all the PMU counters. */
> -	val = readl_relaxed(cci_ctrl_base + CCI_PMCR) | CCI_PMCR_CEN;
> -	writel(val, cci_ctrl_base + CCI_PMCR);
> +	__cci_pmu_enable();
>  	raw_spin_unlock_irqrestore(&hw_events->pmu_lock, flags);
>  
>  }
> @@ -899,13 +915,9 @@ static void cci_pmu_disable(struct pmu *pmu)
>  	struct cci_pmu *cci_pmu = to_cci_pmu(pmu);
>  	struct cci_pmu_hw_events *hw_events = &cci_pmu->hw_events;
>  	unsigned long flags;
> -	u32 val;
>  
>  	raw_spin_lock_irqsave(&hw_events->pmu_lock, flags);
> -
> -	/* Disable all the PMU counters. */
> -	val = readl_relaxed(cci_ctrl_base + CCI_PMCR) & ~CCI_PMCR_CEN;
> -	writel(val, cci_ctrl_base + CCI_PMCR);
> +	__cci_pmu_disable();
>  	raw_spin_unlock_irqrestore(&hw_events->pmu_lock, flags);
>  }
>  
> -- 
> 1.7.9.5
> 

Re: [PATCHv3 3/5] arm-cci: Add routines to enable/disable all counters

[Mark Rutland]

On Tue, Nov 17, 2015 at 06:03:25PM +0000, Suzuki K. Poulose wrote:
> Adds helper routines to disable the counter controls for
> all the counters on the CCI PMU and restore it back, by
> preserving the original state in caller provided mask.
> 
> Cc: Punit Agrawal <punit.agrawal@arm.com>
> Cc: Mark Rutland <mark.rutland@arm.com>
> Signed-off-by: Suzuki K. Poulose <suzuki.poulose@arm.com>
> ---
> Changes since V2:
>   - Rename the functions to pmu_restore_counters, pmu_disable_counters
>   - Added comment describing why we use a private mask
> ---
>  drivers/bus/arm-cci.c |   38 ++++++++++++++++++++++++++++++++++++++
>  1 file changed, 38 insertions(+)
> 
> diff --git a/drivers/bus/arm-cci.c b/drivers/bus/arm-cci.c
> index 48936c8..91a9d5d 100644
> --- a/drivers/bus/arm-cci.c
> +++ b/drivers/bus/arm-cci.c
> @@ -669,6 +669,44 @@ pmu_get_event(struct cci_pmu *cci_pmu, int idx)
>  	return pmu_read_register(cci_pmu, idx, CCI_PMU_EVT_SEL);
>  }
>  
> +
> +/*
> + * For all counters on the CCI-PMU, disable any 'enabled' counters,
> + * saving the changed counters in the mask, so that we can restore
> + * it later using pmu_restore_counters. The mask is private to the
> + * caller. We cannot rely on the used_mask maintained by the CCI_PMU
> + * as it only tells us if the counter is assigned to perf_event or not.
> + * The state of the perf_event cannot be locked by the PMU layer, hence
> + * we check the individual counter status (which can be locked by
> + * cci_pm->hw_events->pmu_lock).
> + */
> +static void __maybe_unused
> +pmu_disable_counters(struct cci_pmu *cci_pmu, unsigned long *mask)
> +{
> +	int i;
> +
> +	for (i = 0; i < cci_pmu->num_cntrs; i++) {
> +		if (pmu_counter_is_enabled(cci_pmu, i)) {
> +			set_bit(i, mask);
> +			pmu_disable_counter(cci_pmu, i);
> +		} else
> +			clear_bit(i, mask);

Can we not assume a clean mask to begin with?

> +	}
> +}
> +
> +/*
> + * Restore the status of the counters. Reversal of the pmu_disable_counters().
> + * For each counter set in the mask, enable the counter back.
> + */
> +static void __maybe_unused
> +pmu_restore_counters(struct cci_pmu *cci_pmu, unsigned long *mask)

This would probably be better with s/restore/enable/ for consistency
with pmu_disable_counters.

Other than that this looks fine to me.

Mark.

> +{
> +	int i;
> +
> +	for_each_set_bit(i, mask, cci_pmu->num_cntrs)
> +		pmu_enable_counter(cci_pmu, i);
> +}
> +
>  /*
>   * Returns the number of programmable counters actually implemented
>   * by the cci
> -- 
> 1.7.9.5
> 

Re: [PATCHv3 2/5] arm-cci: Get the status of a counter

[Mark Rutland]

On Tue, Nov 17, 2015 at 06:03:24PM +0000, Suzuki K. Poulose wrote:
> Add helper routines to get the counter status and the event
> programmed on it.
> 
> Cc: Punit Agrawal <punit.agrawal@arm.com>
> Cc: Mark Rutland <mark.rutland@arm.com>
> Signed-off-by: Suzuki K. Poulose <suzuki.poulose@arm.com>
> ---
> Changes since V2:
>  - Rename pmu_counter_get_ctrl => pmu_counter_is_enabled()

Acked-by: Mark Rutland <mark.rutland@arm.com>

Mark.

> ---
>  drivers/bus/arm-cci.c |   12 ++++++++++++
>  1 file changed, 12 insertions(+)
> 
> diff --git a/drivers/bus/arm-cci.c b/drivers/bus/arm-cci.c
> index f5793b9..48936c8 100644
> --- a/drivers/bus/arm-cci.c
> +++ b/drivers/bus/arm-cci.c
> @@ -652,11 +652,23 @@ static void pmu_enable_counter(struct cci_pmu *cci_pmu, int idx)
>  	pmu_write_register(cci_pmu, 1, idx, CCI_PMU_CNTR_CTRL);
>  }
>  
> +static bool __maybe_unused
> +pmu_counter_is_enabled(struct cci_pmu *cci_pmu, int idx)
> +{
> +	return (pmu_read_register(cci_pmu, idx, CCI_PMU_CNTR_CTRL) & 0x1) != 0;
> +}
> +
>  static void pmu_set_event(struct cci_pmu *cci_pmu, int idx, unsigned long event)
>  {
>  	pmu_write_register(cci_pmu, event, idx, CCI_PMU_EVT_SEL);
>  }
>  
> +static u32 __maybe_unused
> +pmu_get_event(struct cci_pmu *cci_pmu, int idx)
> +{
> +	return pmu_read_register(cci_pmu, idx, CCI_PMU_EVT_SEL);
> +}
> +
>  /*
>   * Returns the number of programmable counters actually implemented
>   * by the cci
> -- 
> 1.7.9.5
> 

Re: [PATCHv3 3/5] arm-cci: Add routines to enable/disable all counters

[Suzuki K. Poulose]

On 10/12/15 15:32, Mark Rutland wrote:
> On Tue, Nov 17, 2015 at 06:03:25PM +0000, Suzuki K. Poulose wrote:


>> +static void __maybe_unused
>> +pmu_disable_counters(struct cci_pmu *cci_pmu, unsigned long *mask)
>> +{
>> +	int i;
>> +
>> +	for (i = 0; i < cci_pmu->num_cntrs; i++) {
>> +		if (pmu_counter_is_enabled(cci_pmu, i)) {
>> +			set_bit(i, mask);
>> +			pmu_disable_counter(cci_pmu, i);
>> +		} else
>> +			clear_bit(i, mask);
>
> Can we not assume a clean mask to begin with?

If we force the caller to pass a clean mask, yes we could. I am fine
with either approach.

>
>> +	}
>> +}
>> +
>> +/*
>> + * Restore the status of the counters. Reversal of the pmu_disable_counters().
>> + * For each counter set in the mask, enable the counter back.
>> + */
>> +static void __maybe_unused
>> +pmu_restore_counters(struct cci_pmu *cci_pmu, unsigned long *mask)
>
> This would probably be better with s/restore/enable/ for consistency
> with pmu_disable_counters.

I had thought as well, but then chose restore as we don't enable all the
counters. Given that we pass a mask argument, it is fine to change it to
enable and will do that in the next one.

>
> Other than that this looks fine to me.

Thanks for the review.

Cheers
Suzuki

Re: [PATCHv3 5/5] arm-cci: CCI-500: Work around PMU counter writes

[Mark Rutland]

On Tue, Nov 17, 2015 at 06:03:27PM +0000, Suzuki K. Poulose wrote:
> The CCI PMU driver sets the event counter to the half of the maximum
> value(2^31) it can count before we start the counters via
> pmu_event_set_period(). This is done to give us the best chance to
> handle the overflow interrupt, taking care of extreme interrupt latencies.
> 
> However, CCI-500 comes with advanced power saving schemes, which
> disables the clock to the event counters unless the counters are enabled to
> count (PMCR.CEN). This prevents the driver from writing the period to the
> counters before starting them.  Also, there is no way we can reset the
> individual event counter to 0 (PMCR.RST resets all the counters, losing
> their current readings). However the value of the counter is preserved and
> could be read back, when the counters are not enabled.
> 
> So we cannot reliably use the counters and compute the number of events
> generated during the sampling period since we don't have the value of the
> counter at start.
> 
> This patch works around this issue by changing writes to the counter
> with the following steps.
> 
>  1) Disable all the counters (remembering any counters which were enabled)
>  2) Save the current event and program the target counter to count an
>     invalid event, which by spec is guaranteed to not-generate any events.
>  3) Enable the target counter.
>  4) Enable the CCI PMU
>  5) Write to the target counter.
>  6) Disable the CCI PMU and the target counter
>  7) Restore the event back on the target counter.
>  8) Restore the status of the all the counters
> 
> Cc: Punit Agrawal <punit.agrawal@arm.com>
> Cc: Mark Rutland <mark.rutland@arm.com>
> Signed-off-by: Suzuki K. Poulose <suzuki.poulose@arm.com>
> ---
>  drivers/bus/arm-cci.c |   47 +++++++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 47 insertions(+)
> 
> diff --git a/drivers/bus/arm-cci.c b/drivers/bus/arm-cci.c
> index 88b612f..6020a02 100644
> --- a/drivers/bus/arm-cci.c
> +++ b/drivers/bus/arm-cci.c
> @@ -835,6 +835,52 @@ static void __pmu_write_counter(struct cci_pmu *cci_pmu, u32 value, int idx)
>  	pmu_write_register(cci_pmu, value, idx, CCI_PMU_CNTR);
>  }
>  
> +#ifdef CONFIG_ARM_CCI500_PMU
> +
> +/*
> + * CCI-500 has advanced power saving policies, which could gate the
> + * clocks to the PMU counters, which makes the writes to them ineffective.
> + * The only way to write to those counters is when the global counters
> + * are enabled and the particular counter is enabled.
> + *
> + * So we do the following :
> + *
> + * 1) Disable all the PMU counters, saving their current state
> + * 2) Save the programmed event, and write an invalid event code
> + *    to the event control register for the counter, so that the
> + *    counters are not modified.
> + * 3) Enable the counter control for the counter.
> + * 4) Enable the global PMU profiling
> + * 5) Set the counter value
> + * 6) Disable the counter, global PMU.
> + * 7) Restore the event in the target counter
> + * 8) Restore the status of the rest of the counters.
> + *
> + * We choose an event code which has very little chances of getting
> + * assigned a valid code for step(2). We use the highest possible
> + * event code (0x1f) for the master interface 0.
> + */
> +#define CCI500_INVALID_EVENT	((CCI500_PORT_M0 << CCI500_PMU_EVENT_SOURCE_SHIFT) | \
> +				 (CCI500_PMU_EVENT_CODE_MASK << CCI500_PMU_EVENT_CODE_SHIFT))
> +static void cci500_pmu_write_counter(struct cci_pmu *cci_pmu, u32 value, int idx)
> +{
> +	unsigned long mask[BITS_TO_LONGS(cci_pmu->num_cntrs)];
> +	u32 event;
> +
> +	pmu_disable_counters(cci_pmu, mask);
> +	event = pmu_get_event(cci_pmu, idx);
> +	pmu_set_event(cci_pmu, idx, CCI500_INVALID_EVENT);
> +	pmu_enable_counter(cci_pmu, idx);
> +	__cci_pmu_enable();
> +	__pmu_write_counter(cci_pmu, value, idx);
> +	__cci_pmu_disable();
> +	pmu_disable_counter(cci_pmu, idx);
> +	pmu_set_event(cci_pmu, idx, event);
> +	pmu_restore_counters(cci_pmu, mask);
> +}
> +
> +#endif	/* CONFIG_ARM_CCI500_PMU */
> +
>  static void pmu_write_counter(struct perf_event *event, u32 value)
>  {
>  	struct cci_pmu *cci_pmu = to_cci_pmu(event->pmu);
> @@ -1422,6 +1468,7 @@ static struct cci_pmu_model cci_pmu_models[] = {
>  			},
>  		},
>  		.validate_hw_event = cci500_validate_hw_event,
> +		.write_counter	= cci500_pmu_write_counter,
>  	},

This should work, but it seems very heavyweight given we do it for each
write.

Can we not amortize this by using the {start,commit,cancel}_txn hooks?

Either we can handle 1-4 and 6-8 in those, or we can copy everything
into a shadow state and apply it all in one go at commit_txn time.

Or is that not possible for some reason I've missed?

Thanks,
Mark.

Re: [PATCHv3 3/5] arm-cci: Add routines to enable/disable all counters

[Mark Rutland]

On Thu, Dec 10, 2015 at 03:42:41PM +0000, Suzuki K. Poulose wrote:
> On 10/12/15 15:32, Mark Rutland wrote:
> >On Tue, Nov 17, 2015 at 06:03:25PM +0000, Suzuki K. Poulose wrote:
> 
> 
> >>+static void __maybe_unused
> >>+pmu_disable_counters(struct cci_pmu *cci_pmu, unsigned long *mask)
> >>+{
> >>+	int i;
> >>+
> >>+	for (i = 0; i < cci_pmu->num_cntrs; i++) {
> >>+		if (pmu_counter_is_enabled(cci_pmu, i)) {
> >>+			set_bit(i, mask);
> >>+			pmu_disable_counter(cci_pmu, i);
> >>+		} else
> >>+			clear_bit(i, mask);
> >
> >Can we not assume a clean mask to begin with?
> 
> If we force the caller to pass a clean mask, yes we could. I am fine
> with either approach.
> 
> >
> >>+	}
> >>+}
> >>+
> >>+/*
> >>+ * Restore the status of the counters. Reversal of the pmu_disable_counters().
> >>+ * For each counter set in the mask, enable the counter back.
> >>+ */
> >>+static void __maybe_unused
> >>+pmu_restore_counters(struct cci_pmu *cci_pmu, unsigned long *mask)
> >
> >This would probably be better with s/restore/enable/ for consistency
> >with pmu_disable_counters.
> 
> I had thought as well, but then chose restore as we don't enable all the
> counters. Given that we pass a mask argument, it is fine to change it to
> enable and will do that in the next one.

How about s/disable/save/ instead, following local_irq_{save,restore} ?

It just felt odd having disable/restore as a pairing.

Mark

Re: [PATCHv3 5/5] arm-cci: CCI-500: Work around PMU counter writes

[Suzuki K. Poulose]

On 10/12/15 15:42, Mark Rutland wrote:
> On Tue, Nov 17, 2015 at 06:03:27PM +0000, Suzuki K. Poulose wrote:
>> The CCI PMU driver sets the event counter to the half of the maximum
>> value(2^31) it can count before we start the counters via
>> pmu_event_set_period(). This is done to give us the best chance to
>> handle the overflow interrupt, taking care of extreme interrupt latencies.


>
> This should work, but it seems very heavyweight given we do it for each
> write.
>
> Can we not amortize this by using the {start,commit,cancel}_txn hooks?
>
> Either we can handle 1-4 and 6-8 in those, or we can copy everything
> into a shadow state and apply it all in one go at commit_txn time.

I took a look at it. The only worrying part is, if pmu->add() will be
called outside *_txn().

from linux/perf_event.h:

         /*
          * Adds/Removes a counter to/from the PMU, can be done inside a
          * transaction, see the ->*_txn() methods.
          *

As of now it is only called within the transactions, but the comment somehow
doesn't look like enforces it.

Thoughts ?

Suzuki

Re: [PATCHv3 5/5] arm-cci: CCI-500: Work around PMU counter writes

[Peter Zijlstra]

On Fri, Dec 11, 2015 at 11:28:45AM +0000, Suzuki K. Poulose wrote:
> On 10/12/15 15:42, Mark Rutland wrote:

> >This should work, but it seems very heavyweight given we do it for each
> >write.
> >
> >Can we not amortize this by using the {start,commit,cancel}_txn hooks?
> >
> >Either we can handle 1-4 and 6-8 in those, or we can copy everything
> >into a shadow state and apply it all in one go at commit_txn time.
> 
> I took a look at it. The only worrying part is, if pmu->add() will be
> called outside *_txn().
> 
> from linux/perf_event.h:
> 
>         /*
>          * Adds/Removes a counter to/from the PMU, can be done inside a
>          * transaction, see the ->*_txn() methods.
>          *
> 
> As of now it is only called within the transactions, but the comment somehow
> doesn't look like enforces it.

Right, txn stuff is intended to be optional. However a txn
implementation must track if one is in progress, so the ::add() method
can check against that.

Also note that there exist a callchain into pmu->add() that does not
start a txn. See:

	__perf_event_enable()
		if (event != leader)
			event_sched_in()
				event->pmu->add()

That said, you can also use pmu->pmu_{en,dis}able() to batch stuff (x86
does this too), add/del, start/stop are guaranteed to be called with the
PMU disabled (as per the comments in struct pmu).


on x86:

For ::add(), we delay touching the hardware until ::pmu_enable() time.

!txn ::add() will do a schedulability test to see if the pmu had place
for the new event and then record the details of it.

txn ::add() will just record the details.

::commit_txn will do the schedulability test for the txn, if that fails
we undo bits.

::pmu_enable rewrites the hardware registers, moves events about if
needed and configures the new event.

Re: [PATCHv3 5/5] arm-cci: CCI-500: Work around PMU counter writes

[Mark Rutland]

On Fri, Dec 11, 2015 at 11:28:45AM +0000, Suzuki K. Poulose wrote:
> On 10/12/15 15:42, Mark Rutland wrote:
> >On Tue, Nov 17, 2015 at 06:03:27PM +0000, Suzuki K. Poulose wrote:
> >>The CCI PMU driver sets the event counter to the half of the maximum
> >>value(2^31) it can count before we start the counters via
> >>pmu_event_set_period(). This is done to give us the best chance to
> >>handle the overflow interrupt, taking care of extreme interrupt latencies.
> 
> 
> >
> >This should work, but it seems very heavyweight given we do it for each
> >write.
> >
> >Can we not amortize this by using the {start,commit,cancel}_txn hooks?
> >
> >Either we can handle 1-4 and 6-8 in those, or we can copy everything
> >into a shadow state and apply it all in one go at commit_txn time.
> 
> I took a look at it. The only worrying part is, if pmu->add() will be
> called outside *_txn().

It looks like that happns.

If we __perf_event_enable an events which is not a leader, we may call
event_sched_in (which will call pmu->add) outside of a transaction. The
__perf_event_disable path is similar w.r.t. pmu->del.

So it does look like we can't rely on being in a transaction there.

Assuming that's deliberate, we could follow the example of other PMU
drivers and keep track of whether or not we're in a transaction. If not,
we do all the heavyweight work inline.

Thanks,
Mark.