[PATCH 0/4] Add support to exclude kernel mode hardware assisted instruction tracing

[PATCH 0/4] Add support to exclude kernel mode hardware assisted instruction tracing

[Sai Prakash Ranjan]

Hardware assisted tracing families such as ARM Coresight, Intel PT
provides rich tracing capabilities including instruction level
tracing and accurate timestamps which are very useful for profiling
and also pose a significant security risk. One such example of
security risk is when kernel mode tracing is not excluded and these
hardware assisted tracing can be used to analyze cryptographic code
execution. In this case, even the root user must not be able to infer
anything.

To explain it more clearly, here is the quote from a member of the
security team (credits: Mattias Nissler),

"Consider a system where disk contents are encrypted and the encryption
key is set up by the user when mounting the file system. From that point
on the encryption key resides in the kernel. It seems reasonable to
expect that the disk encryption key be protected from exfiltration even
if the system later suffers a root compromise (or even against insiders
that have root access), at least as long as the attacker doesn't
manage to compromise the kernel."

Here the idea is to protect such important information from all users
including root users since root privileges does not have to mean full
control over the kernel [1] and root compromise does not have to be
the end of the world.

Currently we can exclude kernel mode tracing via perf_event_paranoid
sysctl but it has following limitations,

 * It is applicable to all PMUs and not just the ones supporting
   instruction tracing.
 * No option to restrict kernel mode instruction tracing by the
   root user.
 * Not possible to restrict kernel mode instruction tracing when the
   hardware assisted tracing IPs like ARM Coresight ETMs use an
   additional interface via sysfs for tracing in addition to perf
   interface.

So introduce a new config CONFIG_EXCLUDE_KERNEL_HW_ITRACE to exclude
kernel mode instruction tracing which will be generic and applicable
to all hardware tracing families and which can also be used with other
interfaces like sysfs in case of ETMs.

Patch 1 adds this new config and the support in perf core to exclude
kernel mode tracing for PMUs which support instruction mode tracing.

Patch 2 adds the perf evsel warning message when the perf tool users
attempt to perform a kernel mode instruction trace with the config
enabled to exclude the kernel mode tracing.

Patch 3 and Patch 4 adds the support for excluding kernel mode for
ARM Coresight ETM{4,3}XX sysfs mode using the newly introduced generic
config.

[1] https://lwn.net/Articles/796866/

Sai Prakash Ranjan (4):
  perf/core: Add support to exclude kernel mode instruction tracing
  perf evsel: Print warning for excluding kernel mode instruction
    tracing
  coresight: etm4x: Add support to exclude kernel mode tracing
  coresight: etm3x: Add support to exclude kernel mode tracing

 drivers/hwtracing/coresight/coresight-etm3x-core.c | 11 +++++++++++
 .../hwtracing/coresight/coresight-etm3x-sysfs.c    |  3 ++-
 drivers/hwtracing/coresight/coresight-etm4x-core.c | 14 +++++++++++++-
 .../hwtracing/coresight/coresight-etm4x-sysfs.c    |  3 ++-
 init/Kconfig                                       | 12 ++++++++++++
 kernel/events/core.c                               |  6 ++++++
 tools/perf/util/evsel.c                            |  3 ++-
 7 files changed, 48 insertions(+), 4 deletions(-)

-- 
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member
of Code Aurora Forum, hosted by The Linux Foundation

[PATCH 1/4] perf/core: Add support to exclude kernel mode instruction tracing

[Sai Prakash Ranjan]

Hardware assisted tracing families such as ARM Coresight, Intel PT
provides rich tracing capabilities including instruction level
tracing and accurate timestamps which are very useful for profiling
and also pose a significant security risk. One such example of
security risk is when kernel mode tracing is not excluded and these
hardware assisted tracing can be used to analyze cryptographic code
execution. In this case, even the root user must not be able to infer
anything.

To explain it more clearly in the words of a security team member
(credits: Mattias Nissler),

"Consider a system where disk contents are encrypted and the encryption
key is set up by the user when mounting the file system. From that point
on the encryption key resides in the kernel. It seems reasonable to
expect that the disk encryption key be protected from exfiltration even
if the system later suffers a root compromise (or even against insiders
that have root access), at least as long as the attacker doesn't
manage to compromise the kernel."

Here the idea is to protect such important information from all users
including root users since root privileges does not have to mean full
control over the kernel [1] and root compromise does not have to be
the end of the world.

Currently we can exclude kernel mode tracing via perf_event_paranoid
sysctl but it has following limitations,

 * It is applicable to all PMUs and not just the ones supporting
   instruction tracing.
 * No option to restrict kernel mode instruction tracing by the
   root user.
 * Not possible to restrict kernel mode instruction tracing when the
   hardware assisted tracing IPs like ARM Coresight ETMs use an
   additional interface via sysfs for tracing in addition to perf
   interface.

So introduce a new config CONFIG_EXCLUDE_KERNEL_HW_ITRACE to exclude
kernel mode instruction tracing which will be generic and applicable
to all hardware tracing families and which can also be used with other
interfaces like sysfs in case of ETMs.

[1] https://lwn.net/Articles/796866/

Suggested-by: Suzuki K Poulose <suzuki.poulose@arm.com>
Suggested-by: Al Grant <al.grant@arm.com>
Tested-by: Denis Nikitin <denik@chromium.org>
Link: https://lore.kernel.org/lkml/20201015124522.1876-1-saiprakash.ranjan@codeaurora.org/
Signed-off-by: Sai Prakash Ranjan <saiprakash.ranjan@codeaurora.org>
---
 init/Kconfig         | 12 ++++++++++++
 kernel/events/core.c |  6 ++++++
 2 files changed, 18 insertions(+)

diff --git a/init/Kconfig b/init/Kconfig
index af454a51f3c5..31b4d1f26bce 100644
--- a/init/Kconfig
+++ b/init/Kconfig
@@ -1832,6 +1832,18 @@ config DEBUG_PERF_USE_VMALLOC
 
 endmenu
 
+config EXCLUDE_KERNEL_HW_ITRACE
+	bool "Exclude kernel mode hardware assisted instruction tracing"
+	depends on PERF_EVENTS
+	help
+	  Exclude kernel mode instruction tracing by hardware tracing
+	  family such as ARM Coresight ETM, Intel PT and so on.
+
+	  This option allows to disable kernel mode instruction tracing
+	  offered by hardware assisted tracing for all users(including root)
+	  especially for production systems where only userspace tracing might
+	  be preferred for security reasons.
+
 config VM_EVENT_COUNTERS
 	default y
 	bool "Enable VM event counters for /proc/vmstat" if EXPERT
diff --git a/kernel/events/core.c b/kernel/events/core.c
index aece2fe19693..044a774cef6d 100644
--- a/kernel/events/core.c
+++ b/kernel/events/core.c
@@ -11866,6 +11866,12 @@ SYSCALL_DEFINE5(perf_event_open,
 		goto err_task;
 	}
 
+	if (IS_ENABLED(CONFIG_EXCLUDE_KERNEL_HW_ITRACE) &&
+	    (event->pmu->capabilities & PERF_PMU_CAP_ITRACE) && !attr.exclude_kernel) {
+		err = -EACCES;
+		goto err_alloc;
+	}
+
 	if (is_sampling_event(event)) {
 		if (event->pmu->capabilities & PERF_PMU_CAP_NO_INTERRUPT) {
 			err = -EOPNOTSUPP;
-- 
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member
of Code Aurora Forum, hosted by The Linux Foundation

[PATCH 2/4] perf evsel: Print warning for excluding kernel mode instruction tracing

[Sai Prakash Ranjan]

Add a warning message to check CONFIG_EXCLUDE_KERNEL_HW_ITRACE kernel
config which excludes kernel mode instruction tracing to help perf tool
users identify the perf event open failure when they attempt kernel mode
tracing with this config enabled.

Tested-by: Denis Nikitin <denik@chromium.org>
Signed-off-by: Sai Prakash Ranjan <saiprakash.ranjan@codeaurora.org>
---
 tools/perf/util/evsel.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/tools/perf/util/evsel.c b/tools/perf/util/evsel.c
index c26ea82220bd..09cc0349f883 100644
--- a/tools/perf/util/evsel.c
+++ b/tools/perf/util/evsel.c
@@ -2630,7 +2630,8 @@ int evsel__open_strerror(struct evsel *evsel, struct target *target,
 		 ">= 1: Disallow CPU event access\n"
 		 ">= 2: Disallow kernel profiling\n"
 		 "To make the adjusted perf_event_paranoid setting permanent preserve it\n"
-		 "in /etc/sysctl.conf (e.g. kernel.perf_event_paranoid = <setting>)",
+		 "in /etc/sysctl.conf (e.g. kernel.perf_event_paranoid = <setting>)\n\n"
+		 "Also check CONFIG_EXCLUDE_KERNEL_HW_ITRACE if kernel mode tracing is allowed.",
 		 perf_event_paranoid());
 	case ENOENT:
 		return scnprintf(msg, size, "The %s event is not supported.", evsel__name(evsel));
-- 
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member
of Code Aurora Forum, hosted by The Linux Foundation

[PATCH 3/4] coresight: etm4x: Add support to exclude kernel mode tracing

[Sai Prakash Ranjan]

On production systems with ETMs enabled, it is preferred to exclude
kernel mode(NS EL1) tracing for security concerns and support only
userspace(NS EL0) tracing. Perf subsystem interface for ETMs use
the newly introduced kernel config CONFIG_EXCLUDE_KERNEL_HW_ITRACE
to exclude kernel mode tracing, but there is an additional interface
via sysfs for ETMs which also needs to be handled to exclude kernel
mode tracing. So we use this same generic kernel config to handle
the sysfs mode of tracing. This config is disabled by default and
would not affect the current configuration which has both kernel and
userspace tracing enabled by default.

Tested-by: Denis Nikitin <denik@chromium.org>
Signed-off-by: Sai Prakash Ranjan <saiprakash.ranjan@codeaurora.org>
---
 drivers/hwtracing/coresight/coresight-etm4x-core.c | 14 +++++++++++++-
 .../hwtracing/coresight/coresight-etm4x-sysfs.c    |  3 ++-
 2 files changed, 15 insertions(+), 2 deletions(-)

diff --git a/drivers/hwtracing/coresight/coresight-etm4x-core.c b/drivers/hwtracing/coresight/coresight-etm4x-core.c
index b20b6ff17cf6..f94143057bb8 100644
--- a/drivers/hwtracing/coresight/coresight-etm4x-core.c
+++ b/drivers/hwtracing/coresight/coresight-etm4x-core.c
@@ -1052,12 +1052,16 @@ static void etm4_set_default(struct etmv4_config *config)
 		return;
 
 	/*
-	 * Make default initialisation trace everything
+	 * Make default initialisation trace everything when
+	 * CONFIG_EXCLUDE_KERNEL_HW_ITRACE is disabled.
 	 *
 	 * This is done by a minimum default config sufficient to enable
 	 * full instruction trace - with a default filter for trace all
 	 * achieved by having no filtering.
 	 */
+	if (IS_ENABLED(CONFIG_EXCLUDE_KERNEL_HW_ITRACE))
+		config->mode |= ETM_MODE_EXCL_KERN;
+
 	etm4_set_default_config(config);
 	etm4_set_default_filter(config);
 }
@@ -1195,6 +1199,7 @@ static int etm4_set_event_filters(struct etmv4_drvdata *drvdata,
 void etm4_config_trace_mode(struct etmv4_config *config)
 {
 	u32 mode;
+	struct etmv4_drvdata *drvdata = container_of(config, struct etmv4_drvdata, config);
 
 	mode = config->mode;
 	mode &= (ETM_MODE_EXCL_KERN | ETM_MODE_EXCL_USER);
@@ -1202,6 +1207,13 @@ void etm4_config_trace_mode(struct etmv4_config *config)
 	/* excluding kernel AND user space doesn't make sense */
 	WARN_ON_ONCE(mode == (ETM_MODE_EXCL_KERN | ETM_MODE_EXCL_USER));
 
+	if (!(mode & ETM_MODE_EXCL_KERN) && IS_ENABLED(CONFIG_EXCLUDE_KERNEL_HW_ITRACE)) {
+		dev_err(&drvdata->csdev->dev,
+			"Kernel mode tracing is not allowed, check your kernel config\n");
+		config->mode |= ETM_MODE_EXCL_KERN;
+		return;
+	}
+
 	/* nothing to do if neither flags are set */
 	if (!(mode & ETM_MODE_EXCL_KERN) && !(mode & ETM_MODE_EXCL_USER))
 		return;
diff --git a/drivers/hwtracing/coresight/coresight-etm4x-sysfs.c b/drivers/hwtracing/coresight/coresight-etm4x-sysfs.c
index 989ce7b8ade7..f1d19d69d151 100644
--- a/drivers/hwtracing/coresight/coresight-etm4x-sysfs.c
+++ b/drivers/hwtracing/coresight/coresight-etm4x-sysfs.c
@@ -426,7 +426,8 @@ static ssize_t mode_store(struct device *dev,
 	else
 		config->vinst_ctrl &= ~BIT(11);
 
-	if (config->mode & (ETM_MODE_EXCL_KERN | ETM_MODE_EXCL_USER))
+	if ((config->mode & (ETM_MODE_EXCL_KERN | ETM_MODE_EXCL_USER)) ||
+	    IS_ENABLED(CONFIG_EXCLUDE_KERNEL_HW_ITRACE))
 		etm4_config_trace_mode(config);
 
 	spin_unlock(&drvdata->spinlock);
-- 
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member
of Code Aurora Forum, hosted by The Linux Foundation

[PATCH 4/4] coresight: etm3x: Add support to exclude kernel mode tracing

[Sai Prakash Ranjan]

On production systems with ETMs enabled, it is preferred to exclude
kernel mode(NS EL1) tracing for security concerns and support only
userspace(NS EL0) tracing. Perf subsystem interface for ETMs use
the newly introduced kernel config CONFIG_EXCLUDE_KERNEL_HW_ITRACE
to exclude kernel mode tracing, but there is an additional interface
via sysfs for ETMs which also needs to be handled to exclude kernel
mode tracing. So we use this same generic kernel config to handle
the sysfs mode of tracing. This config is disabled by default and
would not affect the current configuration which has both kernel and
userspace tracing enabled by default.

Signed-off-by: Sai Prakash Ranjan <saiprakash.ranjan@codeaurora.org>
---
 drivers/hwtracing/coresight/coresight-etm3x-core.c  | 11 +++++++++++
 drivers/hwtracing/coresight/coresight-etm3x-sysfs.c |  3 ++-
 2 files changed, 13 insertions(+), 1 deletion(-)

diff --git a/drivers/hwtracing/coresight/coresight-etm3x-core.c b/drivers/hwtracing/coresight/coresight-etm3x-core.c
index 5bf5a5a4ce6d..4da3bfa66b70 100644
--- a/drivers/hwtracing/coresight/coresight-etm3x-core.c
+++ b/drivers/hwtracing/coresight/coresight-etm3x-core.c
@@ -195,6 +195,9 @@ void etm_set_default(struct etm_config *config)
 	if (WARN_ON_ONCE(!config))
 		return;
 
+	if (IS_ENABLED(CONFIG_EXCLUDE_KERNEL_HW_ITRACE))
+		config->mode |= ETM_MODE_EXCL_KERN;
+
 	/*
 	 * Taken verbatim from the TRM:
 	 *
@@ -239,6 +242,7 @@ void etm_set_default(struct etm_config *config)
 void etm_config_trace_mode(struct etm_config *config)
 {
 	u32 flags, mode;
+	struct etm_drvdata *drvdata = container_of(config, struct etm_drvdata, config);
 
 	mode = config->mode;
 
@@ -248,6 +252,13 @@ void etm_config_trace_mode(struct etm_config *config)
 	if (mode == (ETM_MODE_EXCL_KERN | ETM_MODE_EXCL_USER))
 		return;
 
+	if (!(mode & ETM_MODE_EXCL_KERN) && IS_ENABLED(CONFIG_EXCLUDE_KERNEL_HW_ITRACE)) {
+		dev_err(&drvdata->csdev->dev,
+			"Kernel mode tracing is not allowed, check your kernel config\n");
+		config->mode |= ETM_MODE_EXCL_KERN;
+		return;
+	}
+
 	/* nothing to do if neither flags are set */
 	if (!(mode & ETM_MODE_EXCL_KERN) && !(mode & ETM_MODE_EXCL_USER))
 		return;
diff --git a/drivers/hwtracing/coresight/coresight-etm3x-sysfs.c b/drivers/hwtracing/coresight/coresight-etm3x-sysfs.c
index e8c7649f123e..26642dafddbb 100644
--- a/drivers/hwtracing/coresight/coresight-etm3x-sysfs.c
+++ b/drivers/hwtracing/coresight/coresight-etm3x-sysfs.c
@@ -164,7 +164,8 @@ static ssize_t mode_store(struct device *dev,
 	else
 		config->ctrl &= ~ETMCR_RETURN_STACK;
 
-	if (config->mode & (ETM_MODE_EXCL_KERN | ETM_MODE_EXCL_USER))
+	if ((config->mode & (ETM_MODE_EXCL_KERN | ETM_MODE_EXCL_USER)) ||
+	    IS_ENABLED(CONFIG_EXCLUDE_KERNEL_HW_ITRACE))
 		etm_config_trace_mode(config);
 
 	spin_unlock(&drvdata->spinlock);
-- 
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member
of Code Aurora Forum, hosted by The Linux Foundation

Re: [PATCH 1/4] perf/core: Add support to exclude kernel mode instruction tracing

[Peter Zijlstra]

On Sat, Jan 30, 2021 at 12:35:10AM +0530, Sai Prakash Ranjan wrote:

> Here the idea is to protect such important information from all users
> including root users since root privileges does not have to mean full
> control over the kernel [1] and root compromise does not have to be
> the end of the world.

And yet, your thing lacks:

> +config EXCLUDE_KERNEL_HW_ITRACE
> +	bool "Exclude kernel mode hardware assisted instruction tracing"
> +	depends on PERF_EVENTS
	depends on SECURITY_LOCKDOWN

or whatever the appropriate symbol is.

> +	help
> +	  Exclude kernel mode instruction tracing by hardware tracing
> +	  family such as ARM Coresight ETM, Intel PT and so on.
> +
> +	  This option allows to disable kernel mode instruction tracing
> +	  offered by hardware assisted tracing for all users(including root)
> +	  especially for production systems where only userspace tracing might
> +	  be preferred for security reasons.

Also, colour me unconvinced, pretty much all kernel level PMU usage
can be employed to side-channel / infer crypto keys, why focus on
ITRACE over others?

Re: [PATCH 1/4] perf/core: Add support to exclude kernel mode instruction tracing

[Sai Prakash Ranjan]

Hi Peter,

On 2021-01-30 01:00, Peter Zijlstra wrote:
> On Sat, Jan 30, 2021 at 12:35:10AM +0530, Sai Prakash Ranjan wrote:
> 
>> Here the idea is to protect such important information from all users
>> including root users since root privileges does not have to mean full
>> control over the kernel [1] and root compromise does not have to be
>> the end of the world.
> 
> And yet, your thing lacks:
> 

I guess you mean this lacks an explanation as to why this only applies
to ITRACE and not others? See below.

>> +config EXCLUDE_KERNEL_HW_ITRACE
>> +	bool "Exclude kernel mode hardware assisted instruction tracing"
>> +	depends on PERF_EVENTS
> 	depends on SECURITY_LOCKDOWN
> 
> or whatever the appropriate symbol is.
> 

Ok I suppose you mean CONFIG_SECURITY_LOCKDOWN_LSM? But I don't see
how this new config has to depend on that? This can work independently
whether complete lockdown is enforced or not since it applies to only
hardware instruction tracing. Ideally this depends on several hardware
tracing configs such as ETMs and others but we don't need them because
we are already exposing PERF_PMU_CAP_ITRACE check in the events core.

>> +	help
>> +	  Exclude kernel mode instruction tracing by hardware tracing
>> +	  family such as ARM Coresight ETM, Intel PT and so on.
>> +
>> +	  This option allows to disable kernel mode instruction tracing
>> +	  offered by hardware assisted tracing for all users(including root)
>> +	  especially for production systems where only userspace tracing 
>> might
>> +	  be preferred for security reasons.
> 
> Also, colour me unconvinced, pretty much all kernel level PMU usage
> can be employed to side-channel / infer crypto keys, why focus on
> ITRACE over others?

Here ITRACE is not just instruction trace, it is meant for hardware 
assisted
instruction trace such as Intel PT, Intel BTS, ARM coresight etc. These 
provide
much more capabilities than normal instruction tracing whether its 
kernel level
or userspace. More specifically, these provide more accurate branch 
trace like
Intel PT LBR (Last Branch Record), Intel BTS(Branch Trace Store) which 
can be
used to decode the program flow more accurately with timestamps in real 
time
than other PMUs. Also there is cycle accurate tracing which can 
theoretically
be used for some speculative execution based attacks. Which other kernel 
level
PMUs can be used to get a full branch trace that is not locked down? If 
there
is one, then this should probably be applied to it as well.

Thanks,
Sai

-- 
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a 
member
of Code Aurora Forum, hosted by The Linux Foundation

Re: [PATCH 1/4] perf/core: Add support to exclude kernel mode instruction tracing

[Peter Zijlstra]

On Mon, Feb 01, 2021 at 01:11:04PM +0530, Sai Prakash Ranjan wrote:

> Ok I suppose you mean CONFIG_SECURITY_LOCKDOWN_LSM? But I don't see
> how this new config has to depend on that? This can work independently
> whether complete lockdown is enforced or not since it applies to only
> hardware instruction tracing. Ideally this depends on several hardware
> tracing configs such as ETMs and others but we don't need them because
> we are already exposing PERF_PMU_CAP_ITRACE check in the events core.

If you don't have lockdown, root pretty much owns the kernel, or am I
missing something?

> be used for some speculative execution based attacks. Which other
> kernel level PMUs can be used to get a full branch trace that is not
> locked down? If there is one, then this should probably be applied to
> it as well.

Just the regular counters. The information isn't as accurate, but given
enough goes you can infer plenty.

Just like all the SMT size-channel attacks.

Sure, PT and friends make it even easier, but I don't see a fundamental
distinction.

Re: [PATCH 1/4] perf/core: Add support to exclude kernel mode instruction tracing

[Sai Prakash Ranjan]

Hi Peter,

On 2021-02-01 19:11, Peter Zijlstra wrote:
> On Mon, Feb 01, 2021 at 01:11:04PM +0530, Sai Prakash Ranjan wrote:
> 
>> Ok I suppose you mean CONFIG_SECURITY_LOCKDOWN_LSM? But I don't see
>> how this new config has to depend on that? This can work independently
>> whether complete lockdown is enforced or not since it applies to only
>> hardware instruction tracing. Ideally this depends on several hardware
>> tracing configs such as ETMs and others but we don't need them because
>> we are already exposing PERF_PMU_CAP_ITRACE check in the events core.
> 
> If you don't have lockdown, root pretty much owns the kernel, or am I
> missing something?
> 

You are right in saying that without lockdown root would own kernel but
this config(EXCLUDE_KERNEL) will independently make sure that kernel
level pmu tracing is not allowed(we return -EACCES) even if LOCKDOWN
config is disabled. So I'm saying that we don't need to depend on
LOCKDOWN config, its good to have LOCKDOWN config enabled but perf
subsystem doesn't have to care about that.

>> be used for some speculative execution based attacks. Which other
>> kernel level PMUs can be used to get a full branch trace that is not
>> locked down? If there is one, then this should probably be applied to
>> it as well.
> 
> Just the regular counters. The information isn't as accurate, but given
> enough goes you can infer plenty.
> 
> Just like all the SMT size-channel attacks.
> 
> Sure, PT and friends make it even easier, but I don't see a fundamental
> distinction.

Right, we should then exclude all kernel level pmu tracing, is it fine?

if (IS_ENABLED(CONFIG_EXCLUDE_KERNEL_HW_ITRACE) && 
!attr.exclude_kernel))
     return -EACCES;

Thanks,
Sai

-- 
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a 
member
of Code Aurora Forum, hosted by The Linux Foundation

Re: [PATCH 1/4] perf/core: Add support to exclude kernel mode instruction tracing

[Sai Prakash Ranjan]

Hi Peter,

On 2021-02-02 11:41, Sai Prakash Ranjan wrote:
> Hi Peter,
> 
> On 2021-02-01 19:11, Peter Zijlstra wrote:
>> On Mon, Feb 01, 2021 at 01:11:04PM +0530, Sai Prakash Ranjan wrote:
>> 
>>> Ok I suppose you mean CONFIG_SECURITY_LOCKDOWN_LSM? But I don't see
>>> how this new config has to depend on that? This can work 
>>> independently
>>> whether complete lockdown is enforced or not since it applies to only
>>> hardware instruction tracing. Ideally this depends on several 
>>> hardware
>>> tracing configs such as ETMs and others but we don't need them 
>>> because
>>> we are already exposing PERF_PMU_CAP_ITRACE check in the events core.
>> 
>> If you don't have lockdown, root pretty much owns the kernel, or am I
>> missing something?
>> 
> 
> You are right in saying that without lockdown root would own kernel but
> this config(EXCLUDE_KERNEL) will independently make sure that kernel
> level pmu tracing is not allowed(we return -EACCES) even if LOCKDOWN
> config is disabled. So I'm saying that we don't need to depend on
> LOCKDOWN config, its good to have LOCKDOWN config enabled but perf
> subsystem doesn't have to care about that.
> 
>>> be used for some speculative execution based attacks. Which other
>>> kernel level PMUs can be used to get a full branch trace that is not
>>> locked down? If there is one, then this should probably be applied to
>>> it as well.
>> 
>> Just the regular counters. The information isn't as accurate, but 
>> given
>> enough goes you can infer plenty.
>> 
>> Just like all the SMT size-channel attacks.
>> 
>> Sure, PT and friends make it even easier, but I don't see a 
>> fundamental
>> distinction.
> 
> Right, we should then exclude all kernel level pmu tracing, is it fine?
> 
> if (IS_ENABLED(CONFIG_EXCLUDE_KERNEL_HW_ITRACE) && 
> !attr.exclude_kernel))
>     return -EACCES;
> 

Sorry for being pushy, but does the above make sense?

Thanks,
Sai

-- 
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a 
member
of Code Aurora Forum, hosted by The Linux Foundation

Re: [PATCH 3/4] coresight: etm4x: Add support to exclude kernel mode tracing

[Doug Anderson]

Hi,

On Fri, Jan 29, 2021 at 11:08 AM Sai Prakash Ranjan
<saiprakash.ranjan@codeaurora.org> wrote:
>
> @@ -1202,6 +1207,13 @@ void etm4_config_trace_mode(struct etmv4_config *config)
>         /* excluding kernel AND user space doesn't make sense */
>         WARN_ON_ONCE(mode == (ETM_MODE_EXCL_KERN | ETM_MODE_EXCL_USER));
>
> +       if (!(mode & ETM_MODE_EXCL_KERN) && IS_ENABLED(CONFIG_EXCLUDE_KERNEL_HW_ITRACE)) {
> +               dev_err(&drvdata->csdev->dev,
> +                       "Kernel mode tracing is not allowed, check your kernel config\n");
> +               config->mode |= ETM_MODE_EXCL_KERN;
> +               return;

So I'm not an expert on this code, but the above looks suspicious to
me.  Specifically you are still modifying "config->mode" even though
printing an "error" (dev_err, not dev_warn) and then skipping the rest
of this function.  Since you're skipping the rest of this function
you're not applying the access, right?  Naively I'd have expected one
of these:

1. Maybe the "dev_err" should be a "dev_warn" and then you shouldn't
"return".  In this case you're just implicitly adding
"ETM_MODE_EXCL_KERN" (and shouting) but then making things work.  Of
course, then what happens if the user already specified
"ETM_MODE_EXCL_USER" too?  As per the comment above that "doesn't make
sense".  ...so maybe the code wouldn't behave properly...

2. Maybe you should be modifying this function to return an error code.

3. Maybe you should just be updating the one caller of this function
to error check this right at the beginning of the function and then
fail the sysfs write if the user did the wrong thing.  Then in
etm4_config_trace_mode you could just have a WARN_ON_ONCE if the
kernel wasn't excluded...

-Doug

Re: [PATCH 3/4] coresight: etm4x: Add support to exclude kernel mode tracing

[Sai Prakash Ranjan]

Hi,

Thanks for taking a look, comments inline.

On 2021-02-23 01:44, Doug Anderson wrote:
> Hi,
> 
> On Fri, Jan 29, 2021 at 11:08 AM Sai Prakash Ranjan
> <saiprakash.ranjan@codeaurora.org> wrote:
>> 
>> @@ -1202,6 +1207,13 @@ void etm4_config_trace_mode(struct etmv4_config 
>> *config)
>>         /* excluding kernel AND user space doesn't make sense */
>>         WARN_ON_ONCE(mode == (ETM_MODE_EXCL_KERN | 
>> ETM_MODE_EXCL_USER));
>> 
>> +       if (!(mode & ETM_MODE_EXCL_KERN) && 
>> IS_ENABLED(CONFIG_EXCLUDE_KERNEL_HW_ITRACE)) {
>> +               dev_err(&drvdata->csdev->dev,
>> +                       "Kernel mode tracing is not allowed, check 
>> your kernel config\n");
>> +               config->mode |= ETM_MODE_EXCL_KERN;
>> +               return;
> 
> So I'm not an expert on this code, but the above looks suspicious to
> me.  Specifically you are still modifying "config->mode" even though
> printing an "error" (dev_err, not dev_warn) and then skipping the rest
> of this function.  Since you're skipping the rest of this function
> you're not applying the access, right?  Naively I'd have expected one
> of these:
> 
> 1. Maybe the "dev_err" should be a "dev_warn" and then you shouldn't
> "return".  In this case you're just implicitly adding
> "ETM_MODE_EXCL_KERN" (and shouting) but then making things work.  Of
> course, then what happens if the user already specified
> "ETM_MODE_EXCL_USER" too?  As per the comment above that "doesn't make
> sense".  ...so maybe the code wouldn't behave properly...
> 
> 2. Maybe you should be modifying this function to return an error code.
> 

mode_store() which is the caller of this function sets the config->mode
based on the value passed in the sysfs, so if the user passes the mode
which doesn't exclude the kernel even though the kernel config is 
enabled
and the code just sets it, then that is an error and the user should be
warned about, so I used dev_err, but I can change it to dev_warn if that
is preferred. And to make sysfs mode show the original mode after 
failure,
I set the mode in etm4_config_trace_mode().

But you are right, I am skipping to set the config for other mode bits
and returning which is wrong, will fix it as you suggest below.

> 3. Maybe you should just be updating the one caller of this function
> to error check this right at the beginning of the function and then
> fail the sysfs write if the user did the wrong thing.  Then in
> etm4_config_trace_mode you could just have a WARN_ON_ONCE if the
> kernel wasn't excluded...
> 

Right, will do this.

Thanks,
Sai

-- 
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a 
member
of Code Aurora Forum, hosted by The Linux Foundation