Linux-ARM-MSM Archive on lore.kernel.org
 help / color / Atom feed
* [PATCH] coresight: etm4x: Add config to exclude kernel mode tracing
@ 2020-10-15 12:45 Sai Prakash Ranjan
  2020-10-15 14:27 ` Suzuki K Poulose
  2020-10-15 16:02 ` Mathieu Poirier
  0 siblings, 2 replies; 12+ messages in thread
From: Sai Prakash Ranjan @ 2020-10-15 12:45 UTC (permalink / raw)
  To: Mathieu Poirier, Suzuki K Poulose, Mike Leach
  Cc: coresight, Stephen Boyd, Denis Nikitin, linux-arm-msm,
	linux-kernel, linux-arm-kernel, Sai Prakash Ranjan

On production systems with ETMs enabled, it is preferred to
exclude kernel mode(NS EL1) tracing for security concerns and
support only userspace(NS EL0) tracing. So provide an option
via kconfig to exclude kernel mode tracing if it is required.
This config is disabled by default and would not affect the
current configuration which has both kernel and userspace
tracing enabled by default.

Signed-off-by: Sai Prakash Ranjan <saiprakash.ranjan@codeaurora.org>
---
 drivers/hwtracing/coresight/Kconfig                | 9 +++++++++
 drivers/hwtracing/coresight/coresight-etm4x-core.c | 6 +++++-
 2 files changed, 14 insertions(+), 1 deletion(-)

diff --git a/drivers/hwtracing/coresight/Kconfig b/drivers/hwtracing/coresight/Kconfig
index c1198245461d..52435de8824c 100644
--- a/drivers/hwtracing/coresight/Kconfig
+++ b/drivers/hwtracing/coresight/Kconfig
@@ -110,6 +110,15 @@ config CORESIGHT_SOURCE_ETM4X
 	  To compile this driver as a module, choose M here: the
 	  module will be called coresight-etm4x.
 
+config CORESIGHT_ETM4X_EXCL_KERN
+	bool "Coresight ETM 4.x exclude kernel mode tracing"
+	depends on CORESIGHT_SOURCE_ETM4X
+	help
+	  This will exclude kernel mode(NS EL1) tracing if enabled. This option
+	  will be useful to provide more flexible options on production systems
+	  where only userspace(NS EL0) tracing might be preferred for security
+	  reasons.
+
 config CORESIGHT_STM
 	tristate "CoreSight System Trace Macrocell driver"
 	depends on (ARM && !(CPU_32v3 || CPU_32v4 || CPU_32v4T)) || ARM64
diff --git a/drivers/hwtracing/coresight/coresight-etm4x-core.c b/drivers/hwtracing/coresight/coresight-etm4x-core.c
index abd706b216ac..7e5669e5cd1f 100644
--- a/drivers/hwtracing/coresight/coresight-etm4x-core.c
+++ b/drivers/hwtracing/coresight/coresight-etm4x-core.c
@@ -832,6 +832,9 @@ static u64 etm4_get_ns_access_type(struct etmv4_config *config)
 {
 	u64 access_type = 0;
 
+	if (IS_ENABLED(CONFIG_CORESIGHT_ETM4X_EXCL_KERN))
+		config->mode |= ETM_MODE_EXCL_KERN;
+
 	/*
 	 * EXLEVEL_NS, bits[15:12]
 	 * The Exception levels are:
@@ -849,7 +852,8 @@ static u64 etm4_get_ns_access_type(struct etmv4_config *config)
 		access_type = ETM_EXLEVEL_NS_HYP;
 	}
 
-	if (config->mode & ETM_MODE_EXCL_USER)
+	if (config->mode & ETM_MODE_EXCL_USER &&
+	    !IS_ENABLED(CONFIG_CORESIGHT_ETM4X_EXCL_KERN))
 		access_type |= ETM_EXLEVEL_NS_APP;
 
 	return access_type;

base-commit: 3477326277451000bc667dfcc4fd0774c039184c
-- 
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member
of Code Aurora Forum, hosted by The Linux Foundation


^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: [PATCH] coresight: etm4x: Add config to exclude kernel mode tracing
  2020-10-15 12:45 [PATCH] coresight: etm4x: Add config to exclude kernel mode tracing Sai Prakash Ranjan
@ 2020-10-15 14:27 ` Suzuki K Poulose
  2020-10-16  8:30   ` Sai Prakash Ranjan
  2020-10-15 16:02 ` Mathieu Poirier
  1 sibling, 1 reply; 12+ messages in thread
From: Suzuki K Poulose @ 2020-10-15 14:27 UTC (permalink / raw)
  To: saiprakash.ranjan, mathieu.poirier, mike.leach, peterz
  Cc: coresight, swboyd, denik, linux-arm-msm, linux-kernel, linux-arm-kernel

Hi Sai,

On 10/15/2020 01:45 PM, Sai Prakash Ranjan wrote:
> On production systems with ETMs enabled, it is preferred to
> exclude kernel mode(NS EL1) tracing for security concerns and
> support only userspace(NS EL0) tracing. So provide an option
> via kconfig to exclude kernel mode tracing if it is required.
> This config is disabled by default and would not affect the
> current configuration which has both kernel and userspace
> tracing enabled by default.

While this solution works for ETM4x, I would prefer if we did
this in a more generic way. There are other hardware tracing
PMUs that provide instruction tracing (e.g, Intel PT, even ETM3x)
and it would be good to have a single option that works everywhere.

Something like EXCLUDE_KERNEL_HW_ITRACE, which can be honored by
all tracing drivers ?
> 
> Signed-off-by: Sai Prakash Ranjan <saiprakash.ranjan@codeaurora.org>
> ---
>   drivers/hwtracing/coresight/Kconfig                | 9 +++++++++
>   drivers/hwtracing/coresight/coresight-etm4x-core.c | 6 +++++-
>   2 files changed, 14 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/hwtracing/coresight/Kconfig b/drivers/hwtracing/coresight/Kconfig
> index c1198245461d..52435de8824c 100644
> --- a/drivers/hwtracing/coresight/Kconfig
> +++ b/drivers/hwtracing/coresight/Kconfig
> @@ -110,6 +110,15 @@ config CORESIGHT_SOURCE_ETM4X
>   	  To compile this driver as a module, choose M here: the
>   	  module will be called coresight-etm4x.
>   
> +config CORESIGHT_ETM4X_EXCL_KERN
> +	bool "Coresight ETM 4.x exclude kernel mode tracing"
> +	depends on CORESIGHT_SOURCE_ETM4X
> +	help
> +	  This will exclude kernel mode(NS EL1) tracing if enabled. This option
> +	  will be useful to provide more flexible options on production systems
> +	  where only userspace(NS EL0) tracing might be preferred for security
> +	  reasons.
> +
>   config CORESIGHT_STM
>   	tristate "CoreSight System Trace Macrocell driver"
>   	depends on (ARM && !(CPU_32v3 || CPU_32v4 || CPU_32v4T)) || ARM64
> diff --git a/drivers/hwtracing/coresight/coresight-etm4x-core.c b/drivers/hwtracing/coresight/coresight-etm4x-core.c
> index abd706b216ac..7e5669e5cd1f 100644
> --- a/drivers/hwtracing/coresight/coresight-etm4x-core.c
> +++ b/drivers/hwtracing/coresight/coresight-etm4x-core.c
> @@ -832,6 +832,9 @@ static u64 etm4_get_ns_access_type(struct etmv4_config *config)
>   {
>   	u64 access_type = 0;
>   
> +	if (IS_ENABLED(CONFIG_CORESIGHT_ETM4X_EXCL_KERN))
> +		config->mode |= ETM_MODE_EXCL_KERN;
> +

Rather than hacking the mode behind the back, could we always make sure that
mode is not set in the first place and return this back to the user with
proper errors (see below) ?

>   	/*
>   	 * EXLEVEL_NS, bits[15:12]
>   	 * The Exception levels are:
> @@ -849,7 +852,8 @@ static u64 etm4_get_ns_access_type(struct etmv4_config *config)
>   		access_type = ETM_EXLEVEL_NS_HYP;
>   	}
>   
> -	if (config->mode & ETM_MODE_EXCL_USER)
> +	if (config->mode & ETM_MODE_EXCL_USER &&
> +	    !IS_ENABLED(CONFIG_CORESIGHT_ETM4X_EXCL_KERN))
>   		access_type |= ETM_EXLEVEL_NS_APP;

Why is this needed ?

Also we should return an error if the sysfs mode ever tries to clear the mode bit
for kernel in config->mode.

Suzuki

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: [PATCH] coresight: etm4x: Add config to exclude kernel mode tracing
  2020-10-15 12:45 [PATCH] coresight: etm4x: Add config to exclude kernel mode tracing Sai Prakash Ranjan
  2020-10-15 14:27 ` Suzuki K Poulose
@ 2020-10-15 16:02 ` Mathieu Poirier
       [not found]   ` <CADDJ8CXS8gGuXL45vR6xiHwJhZNcUJPvHMVYSGR6LDETRPJFiQ@mail.gmail.com>
  1 sibling, 1 reply; 12+ messages in thread
From: Mathieu Poirier @ 2020-10-15 16:02 UTC (permalink / raw)
  To: Sai Prakash Ranjan
  Cc: Suzuki K Poulose, Mike Leach, coresight, Stephen Boyd,
	Denis Nikitin, linux-arm-msm, linux-kernel, linux-arm-kernel

On Thu, Oct 15, 2020 at 06:15:22PM +0530, Sai Prakash Ranjan wrote:
> On production systems with ETMs enabled, it is preferred to
> exclude kernel mode(NS EL1) tracing for security concerns and
> support only userspace(NS EL0) tracing. So provide an option
> via kconfig to exclude kernel mode tracing if it is required.
> This config is disabled by default and would not affect the
> current configuration which has both kernel and userspace
> tracing enabled by default.
>

One requires root access (or be part of a special trace group) to be able to use
the cs_etm PMU.  With this kind of elevated access restricting tracing at EL1
provides little in terms of security.

Thanks,
Mathieu
 
> Signed-off-by: Sai Prakash Ranjan <saiprakash.ranjan@codeaurora.org>
> ---
>  drivers/hwtracing/coresight/Kconfig                | 9 +++++++++
>  drivers/hwtracing/coresight/coresight-etm4x-core.c | 6 +++++-
>  2 files changed, 14 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/hwtracing/coresight/Kconfig b/drivers/hwtracing/coresight/Kconfig
> index c1198245461d..52435de8824c 100644
> --- a/drivers/hwtracing/coresight/Kconfig
> +++ b/drivers/hwtracing/coresight/Kconfig
> @@ -110,6 +110,15 @@ config CORESIGHT_SOURCE_ETM4X
>  	  To compile this driver as a module, choose M here: the
>  	  module will be called coresight-etm4x.
>  
> +config CORESIGHT_ETM4X_EXCL_KERN
> +	bool "Coresight ETM 4.x exclude kernel mode tracing"
> +	depends on CORESIGHT_SOURCE_ETM4X
> +	help
> +	  This will exclude kernel mode(NS EL1) tracing if enabled. This option
> +	  will be useful to provide more flexible options on production systems
> +	  where only userspace(NS EL0) tracing might be preferred for security
> +	  reasons.
> +
>  config CORESIGHT_STM
>  	tristate "CoreSight System Trace Macrocell driver"
>  	depends on (ARM && !(CPU_32v3 || CPU_32v4 || CPU_32v4T)) || ARM64
> diff --git a/drivers/hwtracing/coresight/coresight-etm4x-core.c b/drivers/hwtracing/coresight/coresight-etm4x-core.c
> index abd706b216ac..7e5669e5cd1f 100644
> --- a/drivers/hwtracing/coresight/coresight-etm4x-core.c
> +++ b/drivers/hwtracing/coresight/coresight-etm4x-core.c
> @@ -832,6 +832,9 @@ static u64 etm4_get_ns_access_type(struct etmv4_config *config)
>  {
>  	u64 access_type = 0;
>  
> +	if (IS_ENABLED(CONFIG_CORESIGHT_ETM4X_EXCL_KERN))
> +		config->mode |= ETM_MODE_EXCL_KERN;
> +
>  	/*
>  	 * EXLEVEL_NS, bits[15:12]
>  	 * The Exception levels are:
> @@ -849,7 +852,8 @@ static u64 etm4_get_ns_access_type(struct etmv4_config *config)
>  		access_type = ETM_EXLEVEL_NS_HYP;
>  	}
>  
> -	if (config->mode & ETM_MODE_EXCL_USER)
> +	if (config->mode & ETM_MODE_EXCL_USER &&
> +	    !IS_ENABLED(CONFIG_CORESIGHT_ETM4X_EXCL_KERN))
>  		access_type |= ETM_EXLEVEL_NS_APP;
>  
>  	return access_type;
> 
> base-commit: 3477326277451000bc667dfcc4fd0774c039184c
> -- 
> QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member
> of Code Aurora Forum, hosted by The Linux Foundation
> 

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: [PATCH] coresight: etm4x: Add config to exclude kernel mode tracing
       [not found]   ` <CADDJ8CXS8gGuXL45vR6xiHwJhZNcUJPvHMVYSGR6LDETRPJFiQ@mail.gmail.com>
@ 2020-10-16  7:24     ` Leo Yan
  2020-10-16  8:40       ` Sai Prakash Ranjan
  2020-10-16 11:11     ` Suzuki Poulose
  1 sibling, 1 reply; 12+ messages in thread
From: Leo Yan @ 2020-10-16  7:24 UTC (permalink / raw)
  To: Denis Nikitin
  Cc: Mathieu Poirier, Sai Prakash Ranjan, linux-arm-msm, coresight,
	linux-kernel, Stephen Boyd, linux-arm-kernel, Mike Leach

On Thu, Oct 15, 2020 at 11:40:05PM -0700, Denis Nikitin wrote:
> Hi Mathieu,
> 
> I think one of the use cases could be VMs.
> Is there isolation between EL1 guest kernels which we can control from perf
> in a system wide mode?

Sorry for suddenly jumping in.

For KVM, I think we need to implement mechanism for saving/restoring
CoreSight context for every guest OS, the CPU PMUs has implemented
related features [1].

Thanks,
Leo

[1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/arch/arm64/kvm/pmu.c

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: [PATCH] coresight: etm4x: Add config to exclude kernel mode tracing
  2020-10-15 14:27 ` Suzuki K Poulose
@ 2020-10-16  8:30   ` Sai Prakash Ranjan
  0 siblings, 0 replies; 12+ messages in thread
From: Sai Prakash Ranjan @ 2020-10-16  8:30 UTC (permalink / raw)
  To: Suzuki K Poulose
  Cc: mathieu.poirier, mike.leach, peterz, coresight, swboyd, denik,
	linux-arm-msm, linux-kernel, linux-arm-kernel

Hi Suzuki,

On 2020-10-15 19:57, Suzuki K Poulose wrote:
> Hi Sai,
> 
> On 10/15/2020 01:45 PM, Sai Prakash Ranjan wrote:
>> On production systems with ETMs enabled, it is preferred to
>> exclude kernel mode(NS EL1) tracing for security concerns and
>> support only userspace(NS EL0) tracing. So provide an option
>> via kconfig to exclude kernel mode tracing if it is required.
>> This config is disabled by default and would not affect the
>> current configuration which has both kernel and userspace
>> tracing enabled by default.
> 
> While this solution works for ETM4x, I would prefer if we did
> this in a more generic way. There are other hardware tracing
> PMUs that provide instruction tracing (e.g, Intel PT, even ETM3x)
> and it would be good to have a single option that works everywhere.
> 
> Something like EXCLUDE_KERNEL_HW_ITRACE, which can be honored by
> all tracing drivers ?

I can add this for ETM3x as well but I have zero idea regarding
Intel PTs, is there any code in those hwtracing PMUs actually
excluding kernel mode tracing currently?

>> 
>> Signed-off-by: Sai Prakash Ranjan <saiprakash.ranjan@codeaurora.org>
>> ---
>>   drivers/hwtracing/coresight/Kconfig                | 9 +++++++++
>>   drivers/hwtracing/coresight/coresight-etm4x-core.c | 6 +++++-
>>   2 files changed, 14 insertions(+), 1 deletion(-)
>> 
>> diff --git a/drivers/hwtracing/coresight/Kconfig 
>> b/drivers/hwtracing/coresight/Kconfig
>> index c1198245461d..52435de8824c 100644
>> --- a/drivers/hwtracing/coresight/Kconfig
>> +++ b/drivers/hwtracing/coresight/Kconfig
>> @@ -110,6 +110,15 @@ config CORESIGHT_SOURCE_ETM4X
>>   	  To compile this driver as a module, choose M here: the
>>   	  module will be called coresight-etm4x.
>>   +config CORESIGHT_ETM4X_EXCL_KERN
>> +	bool "Coresight ETM 4.x exclude kernel mode tracing"
>> +	depends on CORESIGHT_SOURCE_ETM4X
>> +	help
>> +	  This will exclude kernel mode(NS EL1) tracing if enabled. This 
>> option
>> +	  will be useful to provide more flexible options on production 
>> systems
>> +	  where only userspace(NS EL0) tracing might be preferred for 
>> security
>> +	  reasons.
>> +
>>   config CORESIGHT_STM
>>   	tristate "CoreSight System Trace Macrocell driver"
>>   	depends on (ARM && !(CPU_32v3 || CPU_32v4 || CPU_32v4T)) || ARM64
>> diff --git a/drivers/hwtracing/coresight/coresight-etm4x-core.c 
>> b/drivers/hwtracing/coresight/coresight-etm4x-core.c
>> index abd706b216ac..7e5669e5cd1f 100644
>> --- a/drivers/hwtracing/coresight/coresight-etm4x-core.c
>> +++ b/drivers/hwtracing/coresight/coresight-etm4x-core.c
>> @@ -832,6 +832,9 @@ static u64 etm4_get_ns_access_type(struct 
>> etmv4_config *config)
>>   {
>>   	u64 access_type = 0;
>>   +	if (IS_ENABLED(CONFIG_CORESIGHT_ETM4X_EXCL_KERN))
>> +		config->mode |= ETM_MODE_EXCL_KERN;
>> +
> 
> Rather than hacking the mode behind the back, could we always make sure 
> that
> mode is not set in the first place and return this back to the user 
> with
> proper errors (see below) ?
> 

Sure, this was the minimal change with which I could keep the
check in one place which would work for both sysfs and perf,
but I'll change as you suggested and move the check to
etm4_parse_event_config() and etm4_config_trace_mode() and
return errors properly.

>>   	/*
>>   	 * EXLEVEL_NS, bits[15:12]
>>   	 * The Exception levels are:
>> @@ -849,7 +852,8 @@ static u64 etm4_get_ns_access_type(struct 
>> etmv4_config *config)
>>   		access_type = ETM_EXLEVEL_NS_HYP;
>>   	}
>>   -	if (config->mode & ETM_MODE_EXCL_USER)
>> +	if (config->mode & ETM_MODE_EXCL_USER &&
>> +	    !IS_ENABLED(CONFIG_CORESIGHT_ETM4X_EXCL_KERN))
>>   		access_type |= ETM_EXLEVEL_NS_APP;
> 
> Why is this needed ?
> 

Yes this will not be required as excluding both doesn't make
sense and we print warning in that case already, will drop
this.

> Also we should return an error if the sysfs mode ever tries to clear
> the mode bit
> for kernel in config->mode.
> 

Yes will change as explained in above comment.

Thanks,
Sai

-- 
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a 
member
of Code Aurora Forum, hosted by The Linux Foundation

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: [PATCH] coresight: etm4x: Add config to exclude kernel mode tracing
  2020-10-16  7:24     ` Leo Yan
@ 2020-10-16  8:40       ` Sai Prakash Ranjan
  2020-10-16  9:24         ` Leo Yan
  0 siblings, 1 reply; 12+ messages in thread
From: Sai Prakash Ranjan @ 2020-10-16  8:40 UTC (permalink / raw)
  To: Leo Yan
  Cc: Denis Nikitin, Mathieu Poirier, linux-arm-msm, coresight,
	linux-kernel, Stephen Boyd, linux-arm-kernel, Mike Leach,
	Suzuki K Poulose

Hi Leo,

On 2020-10-16 12:54, Leo Yan wrote:
> On Thu, Oct 15, 2020 at 11:40:05PM -0700, Denis Nikitin wrote:
>> Hi Mathieu,
>> 
>> I think one of the use cases could be VMs.
>> Is there isolation between EL1 guest kernels which we can control from 
>> perf
>> in a system wide mode?
> 
> Sorry for suddenly jumping in.
> 
> For KVM, I think we need to implement mechanism for saving/restoring
> CoreSight context for every guest OS, the CPU PMUs has implemented
> related features [1].
> 
> Thanks,
> Leo
> 
> [1]
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/arch/arm64/kvm/pmu.c
> 

What happens to the sysfs mode of tracing? For that we would still
need a config right to exclude kernel mode tracing completely.

Thanks,
Sai

-- 
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a 
member
of Code Aurora Forum, hosted by The Linux Foundation

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: [PATCH] coresight: etm4x: Add config to exclude kernel mode tracing
  2020-10-16  8:40       ` Sai Prakash Ranjan
@ 2020-10-16  9:24         ` Leo Yan
  2020-10-16 10:30           ` Sai Prakash Ranjan
  2020-10-16 11:38           ` Suzuki Poulose
  0 siblings, 2 replies; 12+ messages in thread
From: Leo Yan @ 2020-10-16  9:24 UTC (permalink / raw)
  To: Sai Prakash Ranjan
  Cc: Denis Nikitin, Mathieu Poirier, linux-arm-msm, coresight,
	linux-kernel, Stephen Boyd, linux-arm-kernel, Mike Leach,
	Suzuki K Poulose

Hi Sai,

On Fri, Oct 16, 2020 at 02:10:47PM +0530, Sai Prakash Ranjan wrote:
> Hi Leo,
> 
> On 2020-10-16 12:54, Leo Yan wrote:
> > On Thu, Oct 15, 2020 at 11:40:05PM -0700, Denis Nikitin wrote:
> > > Hi Mathieu,
> > > 
> > > I think one of the use cases could be VMs.
> > > Is there isolation between EL1 guest kernels which we can control
> > > from perf
> > > in a system wide mode?
> > 
> > Sorry for suddenly jumping in.
> > 
> > For KVM, I think we need to implement mechanism for saving/restoring
> > CoreSight context for every guest OS, the CPU PMUs has implemented
> > related features [1].
> > 
> > Thanks,
> > Leo
> > 
> > [1]
> > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/arch/arm64/kvm/pmu.c
> > 
> 
> What happens to the sysfs mode of tracing? For that we would still
> need a config right to exclude kernel mode tracing completely.

IIUC, sysfs mode and perf mode both can apply the same approach, the
guest OS runs a thread context for the host, so when a guest OS is
switched in or out, the hypervisor can save/restore the context for
the guest OS; thus every guest OS will have its dedicated context and
trace data ideally.

Thanks,
Leo

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: [PATCH] coresight: etm4x: Add config to exclude kernel mode tracing
  2020-10-16  9:24         ` Leo Yan
@ 2020-10-16 10:30           ` Sai Prakash Ranjan
  2020-10-16 11:38           ` Suzuki Poulose
  1 sibling, 0 replies; 12+ messages in thread
From: Sai Prakash Ranjan @ 2020-10-16 10:30 UTC (permalink / raw)
  To: Leo Yan
  Cc: Denis Nikitin, Mathieu Poirier, Suzuki K Poulose, linux-arm-msm,
	coresight, linux-kernel, Stephen Boyd, linux-arm-kernel,
	Mike Leach

Hi Leo,

On 2020-10-16 14:54, Leo Yan wrote:
> Hi Sai,
> 
> On Fri, Oct 16, 2020 at 02:10:47PM +0530, Sai Prakash Ranjan wrote:
>> Hi Leo,
>> 
>> On 2020-10-16 12:54, Leo Yan wrote:
>> > On Thu, Oct 15, 2020 at 11:40:05PM -0700, Denis Nikitin wrote:
>> > > Hi Mathieu,
>> > >
>> > > I think one of the use cases could be VMs.
>> > > Is there isolation between EL1 guest kernels which we can control
>> > > from perf
>> > > in a system wide mode?
>> >
>> > Sorry for suddenly jumping in.
>> >
>> > For KVM, I think we need to implement mechanism for saving/restoring
>> > CoreSight context for every guest OS, the CPU PMUs has implemented
>> > related features [1].
>> >
>> > Thanks,
>> > Leo
>> >
>> > [1]
>> > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/arch/arm64/kvm/pmu.c
>> >
>> 
>> What happens to the sysfs mode of tracing? For that we would still
>> need a config right to exclude kernel mode tracing completely.
> 
> IIUC, sysfs mode and perf mode both can apply the same approach, the
> guest OS runs a thread context for the host, so when a guest OS is
> switched in or out, the hypervisor can save/restore the context for
> the guest OS; thus every guest OS will have its dedicated context and
> trace data ideally.
> 

Thanks for the explanation, so for this usecase then we would have to
implement something as you suggested, not sure how hard would that be
looking at my KVM knowledge(which at the moment is almost nil) when
compared to a kconfig ;)

Thanks,
Sai

-- 
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a 
member
of Code Aurora Forum, hosted by The Linux Foundation

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: [PATCH] coresight: etm4x: Add config to exclude kernel mode tracing
       [not found]   ` <CADDJ8CXS8gGuXL45vR6xiHwJhZNcUJPvHMVYSGR6LDETRPJFiQ@mail.gmail.com>
  2020-10-16  7:24     ` Leo Yan
@ 2020-10-16 11:11     ` Suzuki Poulose
  1 sibling, 0 replies; 12+ messages in thread
From: Suzuki Poulose @ 2020-10-16 11:11 UTC (permalink / raw)
  To: Denis Nikitin, Mathieu Poirier
  Cc: Sai Prakash Ranjan, Mike Leach, coresight, Stephen Boyd,
	linux-arm-msm, linux-kernel, linux-arm-kernel

On 10/16/20 7:40 AM, Denis Nikitin wrote:
> Hi Mathieu,
> 
> I think one of the use cases could be VMs.
> Is there isolation between EL1 guest kernels which we can control from 
> perf in a system wide mode?

The proposed solution doesn't solve this for VMs anyway. It only
excludes EL1 *OR* EL2, depending on the host kernel's running  EL.
We cannot support Virtual ETM access for VMs with memory mapped
accesses.

Unforutnately, trace filtering is the solution for preventing tracing
for EL1 guest/kernel (available from v8.4 Self Hosted extensions). Other
option is to add support for "exclude_guest" support for CoreSight for perf.
But again this can't be controlled by sysfs. And it can't be enforced 
for perf, if not specified. Again it all goes back to the root
permission hammer lock which Mathieu pointed out.


With the v8.4 Self hosted trace extensions, Guest and Host both could
control individually if they can be traced (both EL0 and EL1/2).

Suzuki

> 
> Thanks,
> Denis
> 
> On Thu, Oct 15, 2020 at 9:03 AM Mathieu Poirier 
> <mathieu.poirier@linaro.org <mailto:mathieu.poirier@linaro.org>> wrote:
> 
>     On Thu, Oct 15, 2020 at 06:15:22PM +0530, Sai Prakash Ranjan wrote:
>      > On production systems with ETMs enabled, it is preferred to
>      > exclude kernel mode(NS EL1) tracing for security concerns and
>      > support only userspace(NS EL0) tracing. So provide an option
>      > via kconfig to exclude kernel mode tracing if it is required.
>      > This config is disabled by default and would not affect the
>      > current configuration which has both kernel and userspace
>      > tracing enabled by default.
>      >
> 
>     One requires root access (or be part of a special trace group) to be
>     able to use
>     the cs_etm PMU.  With this kind of elevated access restricting
>     tracing at EL1
>     provides little in terms of security.
> 
>     Thanks,
>     Mathieu
> 
>      > Signed-off-by: Sai Prakash Ranjan
>     <saiprakash.ranjan@codeaurora.org
>     <mailto:saiprakash.ranjan@codeaurora.org>>
>      > ---
>      >  drivers/hwtracing/coresight/Kconfig                | 9 +++++++++
>      >  drivers/hwtracing/coresight/coresight-etm4x-core.c | 6 +++++-
>      >  2 files changed, 14 insertions(+), 1 deletion(-)
>      >
>      > diff --git a/drivers/hwtracing/coresight/Kconfig
>     b/drivers/hwtracing/coresight/Kconfig
>      > index c1198245461d..52435de8824c 100644
>      > --- a/drivers/hwtracing/coresight/Kconfig
>      > +++ b/drivers/hwtracing/coresight/Kconfig
>      > @@ -110,6 +110,15 @@ config CORESIGHT_SOURCE_ETM4X
>      >         To compile this driver as a module, choose M here: the
>      >         module will be called coresight-etm4x.
>      >
>      > +config CORESIGHT_ETM4X_EXCL_KERN
>      > +     bool "Coresight ETM 4.x exclude kernel mode tracing"
>      > +     depends on CORESIGHT_SOURCE_ETM4X
>      > +     help
>      > +       This will exclude kernel mode(NS EL1) tracing if enabled.
>     This option
>      > +       will be useful to provide more flexible options on
>     production systems
>      > +       where only userspace(NS EL0) tracing might be preferred
>     for security
>      > +       reasons.
>      > +
>      >  config CORESIGHT_STM
>      >       tristate "CoreSight System Trace Macrocell driver"
>      >       depends on (ARM && !(CPU_32v3 || CPU_32v4 || CPU_32v4T)) ||
>     ARM64
>      > diff --git a/drivers/hwtracing/coresight/coresight-etm4x-core.c
>     b/drivers/hwtracing/coresight/coresight-etm4x-core.c
>      > index abd706b216ac..7e5669e5cd1f 100644
>      > --- a/drivers/hwtracing/coresight/coresight-etm4x-core.c
>      > +++ b/drivers/hwtracing/coresight/coresight-etm4x-core.c
>      > @@ -832,6 +832,9 @@ static u64 etm4_get_ns_access_type(struct
>     etmv4_config *config)
>      >  {
>      >       u64 access_type = 0;
>      >
>      > +     if (IS_ENABLED(CONFIG_CORESIGHT_ETM4X_EXCL_KERN))
>      > +             config->mode |= ETM_MODE_EXCL_KERN;
>      > +
>      >       /*
>      >        * EXLEVEL_NS, bits[15:12]
>      >        * The Exception levels are:
>      > @@ -849,7 +852,8 @@ static u64 etm4_get_ns_access_type(struct
>     etmv4_config *config)
>      >               access_type = ETM_EXLEVEL_NS_HYP;
>      >       }
>      >
>      > -     if (config->mode & ETM_MODE_EXCL_USER)
>      > +     if (config->mode & ETM_MODE_EXCL_USER &&
>      > +         !IS_ENABLED(CONFIG_CORESIGHT_ETM4X_EXCL_KERN))
>      >               access_type |= ETM_EXLEVEL_NS_APP;
>      >
>      >       return access_type;
>      >
>      > base-commit: 3477326277451000bc667dfcc4fd0774c039184c
>      > --
>      > QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is
>     a member
>      > of Code Aurora Forum, hosted by The Linux Foundation
>      >
> 


^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: [PATCH] coresight: etm4x: Add config to exclude kernel mode tracing
  2020-10-16  9:24         ` Leo Yan
  2020-10-16 10:30           ` Sai Prakash Ranjan
@ 2020-10-16 11:38           ` Suzuki Poulose
  2020-10-16 13:14             ` Leo Yan
  1 sibling, 1 reply; 12+ messages in thread
From: Suzuki Poulose @ 2020-10-16 11:38 UTC (permalink / raw)
  To: Leo Yan, Sai Prakash Ranjan
  Cc: Denis Nikitin, Mathieu Poirier, linux-arm-msm, coresight,
	linux-kernel, Stephen Boyd, linux-arm-kernel, Mike Leach

On 10/16/20 10:24 AM, Leo Yan wrote:
> Hi Sai,
> 
> On Fri, Oct 16, 2020 at 02:10:47PM +0530, Sai Prakash Ranjan wrote:
>> Hi Leo,
>>
>> On 2020-10-16 12:54, Leo Yan wrote:
>>> On Thu, Oct 15, 2020 at 11:40:05PM -0700, Denis Nikitin wrote:
>>>> Hi Mathieu,
>>>>
>>>> I think one of the use cases could be VMs.
>>>> Is there isolation between EL1 guest kernels which we can control
>>>> from perf
>>>> in a system wide mode?
>>>
>>> Sorry for suddenly jumping in.
>>>
>>> For KVM, I think we need to implement mechanism for saving/restoring
>>> CoreSight context for every guest OS, the CPU PMUs has implemented
>>> related features [1].
>>>
>>> Thanks,
>>> Leo
>>>
>>> [1]
>>> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/arch/arm64/kvm/pmu.c
>>>

Its not as easy as the CPU PMU for virtualizing the ETMs (with memory
mapped access only), i.e supporting ETMs from VMs.
We could definitely stop/resume on guest entry/exit, to support 
attr.exclude_guest.

>>
>> What happens to the sysfs mode of tracing? For that we would still
>> need a config right to exclude kernel mode tracing completely.
> 
> IIUC, sysfs mode and perf mode both can apply the same approach, the
> guest OS runs a thread context for the host, so when a guest OS is
> switched in or out, the hypervisor can save/restore the context for
> the guest OS; thus every guest OS will have its dedicated context and
> trace data ideally.

I don't think Guest Context is something we can support as mentioned
above, at least for systems without sysreg access for ETMs (and 
virtualizing ETRs is a different story !)

Cheers
Suzuki

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: [PATCH] coresight: etm4x: Add config to exclude kernel mode tracing
  2020-10-16 11:38           ` Suzuki Poulose
@ 2020-10-16 13:14             ` Leo Yan
  2020-10-16 13:17               ` Suzuki Poulose
  0 siblings, 1 reply; 12+ messages in thread
From: Leo Yan @ 2020-10-16 13:14 UTC (permalink / raw)
  To: Suzuki Poulose
  Cc: Sai Prakash Ranjan, Denis Nikitin, Mathieu Poirier,
	linux-arm-msm, coresight, linux-kernel, Stephen Boyd,
	linux-arm-kernel, Mike Leach

On Fri, Oct 16, 2020 at 12:38:47PM +0100, Suzuki Kuruppassery Poulose wrote:

[...]

> > > What happens to the sysfs mode of tracing? For that we would still
> > > need a config right to exclude kernel mode tracing completely.
> > 
> > IIUC, sysfs mode and perf mode both can apply the same approach, the
> > guest OS runs a thread context for the host, so when a guest OS is
> > switched in or out, the hypervisor can save/restore the context for
> > the guest OS; thus every guest OS will have its dedicated context and
> > trace data ideally.
> 
> I don't think Guest Context is something we can support as mentioned
> above, at least for systems without sysreg access for ETMs (and virtualizing
> ETRs is a different story !)

Thanks for sharing thoughts, Suzuki.

I missed the device virtulisation.  Here should virtualize all devices
(includes CoreSight ETM/funnel/ETR/ETF)?  Or only need to virtualize
ETRs?

Obviously, this is a difficult task :)

Thanks,
Leo

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: [PATCH] coresight: etm4x: Add config to exclude kernel mode tracing
  2020-10-16 13:14             ` Leo Yan
@ 2020-10-16 13:17               ` Suzuki Poulose
  0 siblings, 0 replies; 12+ messages in thread
From: Suzuki Poulose @ 2020-10-16 13:17 UTC (permalink / raw)
  To: Leo Yan
  Cc: Sai Prakash Ranjan, Denis Nikitin, Mathieu Poirier,
	linux-arm-msm, coresight, linux-kernel, Stephen Boyd,
	linux-arm-kernel, Mike Leach

On 10/16/20 2:14 PM, Leo Yan wrote:
> On Fri, Oct 16, 2020 at 12:38:47PM +0100, Suzuki Kuruppassery Poulose wrote:
> 
> [...]
> 
>>>> What happens to the sysfs mode of tracing? For that we would still
>>>> need a config right to exclude kernel mode tracing completely.
>>>
>>> IIUC, sysfs mode and perf mode both can apply the same approach, the
>>> guest OS runs a thread context for the host, so when a guest OS is
>>> switched in or out, the hypervisor can save/restore the context for
>>> the guest OS; thus every guest OS will have its dedicated context and
>>> trace data ideally.
>>
>> I don't think Guest Context is something we can support as mentioned
>> above, at least for systems without sysreg access for ETMs (and virtualizing
>> ETRs is a different story !)
> 
> Thanks for sharing thoughts, Suzuki.
> 
> I missed the device virtulisation.  Here should virtualize all devices
> (includes CoreSight ETM/funnel/ETR/ETF)?  Or only need to virtualize
> ETRs?

I wouldn't even think of virtualizing the components without sysreg
access. So let us not worry about it :-)

Cheers
Suzuki

^ permalink raw reply	[flat|nested] 12+ messages in thread

end of thread, back to index

Thread overview: 12+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-10-15 12:45 [PATCH] coresight: etm4x: Add config to exclude kernel mode tracing Sai Prakash Ranjan
2020-10-15 14:27 ` Suzuki K Poulose
2020-10-16  8:30   ` Sai Prakash Ranjan
2020-10-15 16:02 ` Mathieu Poirier
     [not found]   ` <CADDJ8CXS8gGuXL45vR6xiHwJhZNcUJPvHMVYSGR6LDETRPJFiQ@mail.gmail.com>
2020-10-16  7:24     ` Leo Yan
2020-10-16  8:40       ` Sai Prakash Ranjan
2020-10-16  9:24         ` Leo Yan
2020-10-16 10:30           ` Sai Prakash Ranjan
2020-10-16 11:38           ` Suzuki Poulose
2020-10-16 13:14             ` Leo Yan
2020-10-16 13:17               ` Suzuki Poulose
2020-10-16 11:11     ` Suzuki Poulose

Linux-ARM-MSM Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-arm-msm/0 linux-arm-msm/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-arm-msm linux-arm-msm/ https://lore.kernel.org/linux-arm-msm \
		linux-arm-msm@vger.kernel.org
	public-inbox-index linux-arm-msm

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-arm-msm


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git