[PATCH v1 0/2] arm64: Enable BTI for the executable as well as the interpreter

[PATCH v1 0/2] arm64: Enable BTI for the executable as well as the interpreter

[Mark Brown]

Deployments of BTI on arm64 have run into issues interacting with
systemd's MemoryDenyWriteExecute feature.  Currently for dynamically
linked executables the kernel will only handle architecture specific
properties like BTI for the interpreter, the expectation is that the
interpreter will then handle any properties on the main executable.
For BTI this means remapping the executable segments PROT_EXEC |
PROT_BTI.

This interacts poorly with MemoryDenyWriteExecute since that is
implemented using a seccomp filter which prevents setting PROT_EXEC on
already mapped memory and lacks the context to be able to detect that
memory is already mapped with PROT_EXEC.  This series resolves this by
handling the BTI property for both the interpreter and the main
executable.

This does mean that we may get more code with BTI enabled if running on
a system without BTI support in the dynamic linker, this is expected to
be a safe configuration and testing seems to confirm that. It also
reduces the flexibility userspace has to disable BTI but it is expected
that for cases where there are problems which require BTI to be disabled
it is more likely that it will need to be disabled on a system level.

Mark Brown (2):
  elf: Allow architectures to parse properties on the main executable
  arm64: Enable BTI for main executable as well as the interpreter

 arch/arm64/include/asm/elf.h | 13 ++++++++++---
 arch/arm64/kernel/process.c  | 18 ++++++------------
 fs/binfmt_elf.c              | 25 +++++++++++++++++--------
 include/linux/elf.h          |  4 +++-
 4 files changed, 36 insertions(+), 24 deletions(-)


base-commit: d07f6ca923ea0927a1024dfccafc5b53b61cfecc
-- 
2.20.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH v1 1/2] elf: Allow architectures to parse properties on the main executable

[Mark Brown]

Currently the ELF code only attempts to parse properties on the image
that will start execution, either the interpreter or for statically linked
executables the main executable. The expectation is that any property
handling for the main executable will be done by the interpreter. This is
a bit inconsistent since we do map the executable and is causing problems
for the arm64 BTI support when used in conjunction with systemd's use of
seccomp to implement MemoryDenyWriteExecute which stops the dynamic linker
adjusting the permissions of executable segments.

Allow architectures to handle properties for both the dynamic linker and
main executable, adjusting arch_parse_elf_properties() to have an is_interp
flag as with arch_elf_adjust_prot() and calling it for both the main
executable and any intepreter.

Signed-off-by: Mark Brown <broonie@kernel.org>
---
 arch/arm64/include/asm/elf.h |  3 ++-
 fs/binfmt_elf.c              | 25 +++++++++++++++++--------
 include/linux/elf.h          |  4 +++-
 3 files changed, 22 insertions(+), 10 deletions(-)

diff --git a/arch/arm64/include/asm/elf.h b/arch/arm64/include/asm/elf.h
index 8d1c8dcb87fd..c8678a8c36d5 100644
--- a/arch/arm64/include/asm/elf.h
+++ b/arch/arm64/include/asm/elf.h
@@ -261,6 +261,7 @@ struct arch_elf_state {
 
 static inline int arch_parse_elf_property(u32 type, const void *data,
 					  size_t datasz, bool compat,
+					  bool is_interp,
 					  struct arch_elf_state *arch)
 {
 	/* No known properties for AArch32 yet */
@@ -273,7 +274,7 @@ static inline int arch_parse_elf_property(u32 type, const void *data,
 		if (datasz != sizeof(*p))
 			return -ENOEXEC;
 
-		if (system_supports_bti() &&
+		if (system_supports_bti() && is_interp &&
 		    (*p & GNU_PROPERTY_AARCH64_FEATURE_1_BTI))
 			arch->flags |= ARM64_ELF_BTI;
 	}
diff --git a/fs/binfmt_elf.c b/fs/binfmt_elf.c
index 187b3f2b9202..c8397664af39 100644
--- a/fs/binfmt_elf.c
+++ b/fs/binfmt_elf.c
@@ -716,7 +716,7 @@ static unsigned long load_elf_interp(struct elfhdr *interp_elf_ex,
  */
 
 static int parse_elf_property(const char *data, size_t *off, size_t datasz,
-			      struct arch_elf_state *arch,
+			      struct arch_elf_state *arch, bool is_interp,
 			      bool have_prev_type, u32 *prev_type)
 {
 	size_t o, step;
@@ -751,7 +751,8 @@ static int parse_elf_property(const char *data, size_t *off, size_t datasz,
 	*prev_type = pr->pr_type;
 
 	ret = arch_parse_elf_property(pr->pr_type, data + o,
-				      pr->pr_datasz, ELF_COMPAT, arch);
+				      pr->pr_datasz, ELF_COMPAT, is_interp,
+				      arch);
 	if (ret)
 		return ret;
 
@@ -764,7 +765,7 @@ static int parse_elf_property(const char *data, size_t *off, size_t datasz,
 #define NOTE_NAME_SZ (sizeof(GNU_PROPERTY_TYPE_0_NAME))
 
 static int parse_elf_properties(struct file *f, const struct elf_phdr *phdr,
-				struct arch_elf_state *arch)
+				bool is_interp, struct arch_elf_state *arch)
 {
 	union {
 		struct elf_note nhdr;
@@ -813,7 +814,8 @@ static int parse_elf_properties(struct file *f, const struct elf_phdr *phdr,
 	have_prev_type = false;
 	do {
 		ret = parse_elf_property(note.data, &off, datasz, arch,
-					 have_prev_type, &prev_type);
+					 is_interp, have_prev_type,
+					 &prev_type);
 		have_prev_type = true;
 	} while (!ret);
 
@@ -828,6 +830,7 @@ static int load_elf_binary(struct linux_binprm *bprm)
 	unsigned long error;
 	struct elf_phdr *elf_ppnt, *elf_phdata, *interp_elf_phdata = NULL;
 	struct elf_phdr *elf_property_phdata = NULL;
+	struct elf_phdr *interp_elf_property_phdata = NULL;
 	unsigned long elf_bss, elf_brk;
 	int bss_prot = 0;
 	int retval, i;
@@ -963,12 +966,11 @@ static int load_elf_binary(struct linux_binprm *bprm)
 			goto out_free_dentry;
 
 		/* Pass PT_LOPROC..PT_HIPROC headers to arch code */
-		elf_property_phdata = NULL;
 		elf_ppnt = interp_elf_phdata;
 		for (i = 0; i < interp_elf_ex->e_phnum; i++, elf_ppnt++)
 			switch (elf_ppnt->p_type) {
 			case PT_GNU_PROPERTY:
-				elf_property_phdata = elf_ppnt;
+				interp_elf_property_phdata = elf_ppnt;
 				break;
 
 			case PT_LOPROC ... PT_HIPROC:
@@ -979,10 +981,17 @@ static int load_elf_binary(struct linux_binprm *bprm)
 					goto out_free_dentry;
 				break;
 			}
+
+		retval = parse_elf_properties(interpreter,
+					      interp_elf_property_phdata,
+					      true, &arch_state);
+		if (retval)
+			goto out_free_dentry;
+
 	}
 
-	retval = parse_elf_properties(interpreter ?: bprm->file,
-				      elf_property_phdata, &arch_state);
+	retval = parse_elf_properties(bprm->file, elf_property_phdata,
+				      false, &arch_state);
 	if (retval)
 		goto out_free_dentry;
 
diff --git a/include/linux/elf.h b/include/linux/elf.h
index c9a46c4e183b..a20dcdcd86c5 100644
--- a/include/linux/elf.h
+++ b/include/linux/elf.h
@@ -88,13 +88,15 @@ struct arch_elf_state;
 #ifndef CONFIG_ARCH_USE_GNU_PROPERTY
 static inline int arch_parse_elf_property(u32 type, const void *data,
 					  size_t datasz, bool compat,
+					  bool is_interp,
 					  struct arch_elf_state *arch)
 {
 	return 0;
 }
 #else
 extern int arch_parse_elf_property(u32 type, const void *data, size_t datasz,
-				   bool compat, struct arch_elf_state *arch);
+				   bool compat, bool is_interp,
+				   struct arch_elf_state *arch);
 #endif
 
 #ifdef CONFIG_ARCH_HAVE_ELF_PROT
-- 
2.20.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH v1 2/2] arm64: Enable BTI for main executable as well as the interpreter

[Mark Brown]

Currently for dynamically linked ELF executables we only enable BTI for
the interpreter, expecting the interpreter to do this for the main
executable. This is a bit inconsistent since we do map main executable and
is causing issues with systemd's MemoryDenyWriteExecute feature which is
implemented using a seccomp filter which prevents setting PROT_EXEC on
already mapped memory and lacks the context to be able to detect that
memory is already mapped with PROT_EXEC.

Resolve this by checking the BTI property for the main executable and
enabling BTI if it is present when doing the initial mapping. This does
mean that we may get more code with BTI enabled if running on a system
without BTI support in the dynamic linker, this is expected to be a safe
configuration and testing seems to confirm that. It also reduces the
flexibility userspace has to disable BTI but it is expected that for cases
where there are problems which require BTI to be disabled it is more likely
that it will need to be disabled on a system level.

Signed-off-by: Mark Brown <broonie@kernel.org>
---
 arch/arm64/include/asm/elf.h | 14 ++++++++++----
 arch/arm64/kernel/process.c  | 18 ++++++------------
 2 files changed, 16 insertions(+), 16 deletions(-)

diff --git a/arch/arm64/include/asm/elf.h b/arch/arm64/include/asm/elf.h
index c8678a8c36d5..a6e9032b951a 100644
--- a/arch/arm64/include/asm/elf.h
+++ b/arch/arm64/include/asm/elf.h
@@ -253,7 +253,8 @@ struct arch_elf_state {
 	int flags;
 };
 
-#define ARM64_ELF_BTI		(1 << 0)
+#define ARM64_ELF_INTERP_BTI		(1 << 0)
+#define ARM64_ELF_EXEC_BTI		(1 << 1)
 
 #define INIT_ARCH_ELF_STATE {			\
 	.flags = 0,				\
@@ -274,9 +275,14 @@ static inline int arch_parse_elf_property(u32 type, const void *data,
 		if (datasz != sizeof(*p))
 			return -ENOEXEC;
 
-		if (system_supports_bti() && is_interp &&
-		    (*p & GNU_PROPERTY_AARCH64_FEATURE_1_BTI))
-			arch->flags |= ARM64_ELF_BTI;
+		if (system_supports_bti() &&
+		    (*p & GNU_PROPERTY_AARCH64_FEATURE_1_BTI)) {
+			if (is_interp) {
+				arch->flags |= ARM64_ELF_INTERP_BTI;
+			} else {
+				arch->flags |= ARM64_ELF_EXEC_BTI;
+			}
+		}
 	}
 
 	return 0;
diff --git a/arch/arm64/kernel/process.c b/arch/arm64/kernel/process.c
index b4bb67f17a2c..f7fff4a4c99f 100644
--- a/arch/arm64/kernel/process.c
+++ b/arch/arm64/kernel/process.c
@@ -744,19 +744,13 @@ asmlinkage void __sched arm64_preempt_schedule_irq(void)
 int arch_elf_adjust_prot(int prot, const struct arch_elf_state *state,
 			 bool has_interp, bool is_interp)
 {
-	/*
-	 * For dynamically linked executables the interpreter is
-	 * responsible for setting PROT_BTI on everything except
-	 * itself.
-	 */
-	if (is_interp != has_interp)
-		return prot;
+	if (prot & PROT_EXEC) {
+		if (state->flags & ARM64_ELF_INTERP_BTI && is_interp)
+			prot |= PROT_BTI;
 
-	if (!(state->flags & ARM64_ELF_BTI))
-		return prot;
-
-	if (prot & PROT_EXEC)
-		prot |= PROT_BTI;
+		if (state->flags & ARM64_ELF_EXEC_BTI && !is_interp)
+			prot |= PROT_BTI;
+	}
 
 	return prot;
 }
-- 
2.20.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v1 1/2] elf: Allow architectures to parse properties on the main executable

[Dave Martin]

On Fri, May 21, 2021 at 03:46:20PM +0100, Mark Brown wrote:
> Currently the ELF code only attempts to parse properties on the image
> that will start execution, either the interpreter or for statically linked
> executables the main executable. The expectation is that any property
> handling for the main executable will be done by the interpreter. This is
> a bit inconsistent since we do map the executable and is causing problems
> for the arm64 BTI support when used in conjunction with systemd's use of
> seccomp to implement MemoryDenyWriteExecute which stops the dynamic linker
> adjusting the permissions of executable segments.
> 
> Allow architectures to handle properties for both the dynamic linker and
> main executable, adjusting arch_parse_elf_properties() to have an is_interp
> flag as with arch_elf_adjust_prot() and calling it for both the main
> executable and any intepreter.
>
> Signed-off-by: Mark Brown <broonie@kernel.org>
> ---
>  arch/arm64/include/asm/elf.h |  3 ++-
>  fs/binfmt_elf.c              | 25 +++++++++++++++++--------
>  include/linux/elf.h          |  4 +++-
>  3 files changed, 22 insertions(+), 10 deletions(-)
> 
> diff --git a/arch/arm64/include/asm/elf.h b/arch/arm64/include/asm/elf.h
> index 8d1c8dcb87fd..c8678a8c36d5 100644
> --- a/arch/arm64/include/asm/elf.h
> +++ b/arch/arm64/include/asm/elf.h
> @@ -261,6 +261,7 @@ struct arch_elf_state {
>  
>  static inline int arch_parse_elf_property(u32 type, const void *data,
>  					  size_t datasz, bool compat,
> +					  bool is_interp,
>  					  struct arch_elf_state *arch)
>  {
>  	/* No known properties for AArch32 yet */
> @@ -273,7 +274,7 @@ static inline int arch_parse_elf_property(u32 type, const void *data,
>  		if (datasz != sizeof(*p))
>  			return -ENOEXEC;
>  
> -		if (system_supports_bti() &&
> +		if (system_supports_bti() && is_interp &&

Won't this cause BTI to be forced off for static binaries?

Perhaps this should be (has_interp == is_interp), as for
arch_elf_adjust_prot().  Seems gross though, since has_interp would
become useless after the next patch.  If there's no sensible way to
keep this bisectable, perhaps the patches can be merged instead.

(has_interp should probably also go away for arch_elf_adjust_prot() --
see my comments on the next patch).

>  		    (*p & GNU_PROPERTY_AARCH64_FEATURE_1_BTI))
>  			arch->flags |= ARM64_ELF_BTI;
>  	}
> diff --git a/fs/binfmt_elf.c b/fs/binfmt_elf.c
> index 187b3f2b9202..c8397664af39 100644
> --- a/fs/binfmt_elf.c
> +++ b/fs/binfmt_elf.c
> @@ -716,7 +716,7 @@ static unsigned long load_elf_interp(struct elfhdr *interp_elf_ex,
>   */
>  
>  static int parse_elf_property(const char *data, size_t *off, size_t datasz,
> -			      struct arch_elf_state *arch,
> +			      struct arch_elf_state *arch, bool is_interp,
>  			      bool have_prev_type, u32 *prev_type)
>  {
>  	size_t o, step;
> @@ -751,7 +751,8 @@ static int parse_elf_property(const char *data, size_t *off, size_t datasz,
>  	*prev_type = pr->pr_type;
>  
>  	ret = arch_parse_elf_property(pr->pr_type, data + o,
> -				      pr->pr_datasz, ELF_COMPAT, arch);
> +				      pr->pr_datasz, ELF_COMPAT, is_interp,
> +				      arch);
>  	if (ret)
>  		return ret;
>  
> @@ -764,7 +765,7 @@ static int parse_elf_property(const char *data, size_t *off, size_t datasz,
>  #define NOTE_NAME_SZ (sizeof(GNU_PROPERTY_TYPE_0_NAME))
>  
>  static int parse_elf_properties(struct file *f, const struct elf_phdr *phdr,
> -				struct arch_elf_state *arch)
> +				bool is_interp, struct arch_elf_state *arch)
>  {
>  	union {
>  		struct elf_note nhdr;
> @@ -813,7 +814,8 @@ static int parse_elf_properties(struct file *f, const struct elf_phdr *phdr,
>  	have_prev_type = false;
>  	do {
>  		ret = parse_elf_property(note.data, &off, datasz, arch,
> -					 have_prev_type, &prev_type);
> +					 is_interp, have_prev_type,
> +					 &prev_type);
>  		have_prev_type = true;
>  	} while (!ret);
>  
> @@ -828,6 +830,7 @@ static int load_elf_binary(struct linux_binprm *bprm)
>  	unsigned long error;
>  	struct elf_phdr *elf_ppnt, *elf_phdata, *interp_elf_phdata = NULL;
>  	struct elf_phdr *elf_property_phdata = NULL;
> +	struct elf_phdr *interp_elf_property_phdata = NULL;
>  	unsigned long elf_bss, elf_brk;
>  	int bss_prot = 0;
>  	int retval, i;
> @@ -963,12 +966,11 @@ static int load_elf_binary(struct linux_binprm *bprm)
>  			goto out_free_dentry;
>  
>  		/* Pass PT_LOPROC..PT_HIPROC headers to arch code */
> -		elf_property_phdata = NULL;
>  		elf_ppnt = interp_elf_phdata;
>  		for (i = 0; i < interp_elf_ex->e_phnum; i++, elf_ppnt++)
>  			switch (elf_ppnt->p_type) {
>  			case PT_GNU_PROPERTY:
> -				elf_property_phdata = elf_ppnt;
> +				interp_elf_property_phdata = elf_ppnt;

(Hmm, this actually looks a bit cleaner than just clobbering
elf_property_phdata with the interpreter properties as was done
previously.)

>  				break;
>  
>  			case PT_LOPROC ... PT_HIPROC:
> @@ -979,10 +981,17 @@ static int load_elf_binary(struct linux_binprm *bprm)
>  					goto out_free_dentry;
>  				break;
>  			}
> +
> +		retval = parse_elf_properties(interpreter,
> +					      interp_elf_property_phdata,
> +					      true, &arch_state);
> +		if (retval)
> +			goto out_free_dentry;
> +
>  	}
>  
> -	retval = parse_elf_properties(interpreter ?: bprm->file,
> -				      elf_property_phdata, &arch_state);
> +	retval = parse_elf_properties(bprm->file, elf_property_phdata,
> +				      false, &arch_state);
>  	if (retval)
>  		goto out_free_dentry;
>  
> diff --git a/include/linux/elf.h b/include/linux/elf.h
> index c9a46c4e183b..a20dcdcd86c5 100644
> --- a/include/linux/elf.h
> +++ b/include/linux/elf.h
> @@ -88,13 +88,15 @@ struct arch_elf_state;
>  #ifndef CONFIG_ARCH_USE_GNU_PROPERTY
>  static inline int arch_parse_elf_property(u32 type, const void *data,
>  					  size_t datasz, bool compat,
> +					  bool is_interp,
>  					  struct arch_elf_state *arch)
>  {
>  	return 0;
>  }
>  #else
>  extern int arch_parse_elf_property(u32 type, const void *data, size_t datasz,
> -				   bool compat, struct arch_elf_state *arch);
> +				   bool compat, bool is_interp,
> +				   struct arch_elf_state *arch);
>  #endif

Looks like a sensible change, modulo my comments above.

You may want to Cc Yu-cheng Yu <yu-cheng.yu@intel.com> when reposting,
since this would affect his patches (trivially though).

Cheers
---Dave

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v1 2/2] arm64: Enable BTI for main executable as well as the interpreter

[Dave Martin]

On Fri, May 21, 2021 at 03:46:21PM +0100, Mark Brown wrote:
> Currently for dynamically linked ELF executables we only enable BTI for
> the interpreter, expecting the interpreter to do this for the main
> executable. This is a bit inconsistent since we do map main executable and
> is causing issues with systemd's MemoryDenyWriteExecute feature which is
> implemented using a seccomp filter which prevents setting PROT_EXEC on
> already mapped memory and lacks the context to be able to detect that
> memory is already mapped with PROT_EXEC.

It's hard to know whether this is an extensibility fail in the
semantics of mprotect() (and so we were wrong to add PROT_BTI there in
line with my original proposal), or whether this is a case of systemd
doing something that is broken by design (if well-intentioned).  Since
there have been wacky arch-specific mprotect flags around for a fair
while I'd be tempted to argue the latter -- but then I am biased.


Anyway, although I'm a bit queasy about the cause of this patch, the
patch itself looks perfectly reasonable.  If nothing else, it makes
sense as a cleanup or optimisation, so that ld.so doesn't have to do a
bunch of mprotect() calls every time it loads a program.

Do we know how libcs will detect that they don't need to do the
mprotect() calls?  Do we need a detection mechanism at all?

Ignoring certain errors from mprotect() when ld.so is trying to set
PROT_BTI on the main executable's code pages is probably a reasonable,
backwards-compatible compromise here, but it seems a bit wasteful.

> Resolve this by checking the BTI property for the main executable and
> enabling BTI if it is present when doing the initial mapping. This does
> mean that we may get more code with BTI enabled if running on a system
> without BTI support in the dynamic linker, this is expected to be a safe
> configuration and testing seems to confirm that. It also reduces the

Ack, plus IIUC the architecture is designed so that everything works
providing that PROT_BTI is never set on non-BTI-aware code pages.  For
BTI-aware code, the sooner we set PROT_BTI the better I guess.

> flexibility userspace has to disable BTI but it is expected that for cases
> where there are problems which require BTI to be disabled it is more likely
> that it will need to be disabled on a system level.

There's no flexibility impact unless MemoryDenyWriteExecute is in force,
right?

Self-modifying programs (JITs etc.) already can't use that IIUC, so
shouldn't be affected.  That seems the main scenario where people are
likely to be twiddling PROT_{EXEC,WRITE,BTI} on existing pages.

If the main binary is marked as supporting BTI but breaks with
PROT_BTI, then that almost certainly means the toolchain, system
libraries or hardware are broken -- so it would be pointless to have an
elegant workaround.  A big global kill switch seems adequate to me.

> 
> Signed-off-by: Mark Brown <broonie@kernel.org>
> ---
>  arch/arm64/include/asm/elf.h | 14 ++++++++++----
>  arch/arm64/kernel/process.c  | 18 ++++++------------
>  2 files changed, 16 insertions(+), 16 deletions(-)
> 
> diff --git a/arch/arm64/include/asm/elf.h b/arch/arm64/include/asm/elf.h
> index c8678a8c36d5..a6e9032b951a 100644
> --- a/arch/arm64/include/asm/elf.h
> +++ b/arch/arm64/include/asm/elf.h
> @@ -253,7 +253,8 @@ struct arch_elf_state {
>  	int flags;
>  };
>  
> -#define ARM64_ELF_BTI		(1 << 0)
> +#define ARM64_ELF_INTERP_BTI		(1 << 0)
> +#define ARM64_ELF_EXEC_BTI		(1 << 1)
>  
>  #define INIT_ARCH_ELF_STATE {			\
>  	.flags = 0,				\
> @@ -274,9 +275,14 @@ static inline int arch_parse_elf_property(u32 type, const void *data,
>  		if (datasz != sizeof(*p))
>  			return -ENOEXEC;
>  
> -		if (system_supports_bti() && is_interp &&
> -		    (*p & GNU_PROPERTY_AARCH64_FEATURE_1_BTI))
> -			arch->flags |= ARM64_ELF_BTI;
> +		if (system_supports_bti() &&
> +		    (*p & GNU_PROPERTY_AARCH64_FEATURE_1_BTI)) {
> +			if (is_interp) {

Nit: can we drop the extra curlies?

> +				arch->flags |= ARM64_ELF_INTERP_BTI;
> +			} else {
> +				arch->flags |= ARM64_ELF_EXEC_BTI;
> +			}
> +		}
>  	}
>  
>  	return 0;
> diff --git a/arch/arm64/kernel/process.c b/arch/arm64/kernel/process.c
> index b4bb67f17a2c..f7fff4a4c99f 100644
> --- a/arch/arm64/kernel/process.c
> +++ b/arch/arm64/kernel/process.c
> @@ -744,19 +744,13 @@ asmlinkage void __sched arm64_preempt_schedule_irq(void)
>  int arch_elf_adjust_prot(int prot, const struct arch_elf_state *state,
>  			 bool has_interp, bool is_interp)
>  {
> -	/*
> -	 * For dynamically linked executables the interpreter is
> -	 * responsible for setting PROT_BTI on everything except
> -	 * itself.
> -	 */
> -	if (is_interp != has_interp)
> -		return prot;
> +	if (prot & PROT_EXEC) {
> +		if (state->flags & ARM64_ELF_INTERP_BTI && is_interp)
> +			prot |= PROT_BTI;
>  
> -	if (!(state->flags & ARM64_ELF_BTI))
> -		return prot;
> -
> -	if (prot & PROT_EXEC)
> -		prot |= PROT_BTI;
> +		if (state->flags & ARM64_ELF_EXEC_BTI && !is_interp)

Merge these ifs together somehow?  I'm happy either way, though.

> +			prot |= PROT_BTI;
> +	}

Since is_interp and has_interp were only needed for this logic in the
first place, I think we can probably drop those, maybe in a subsequent
patch.  Probably better to do it now before too much dust settles on
them.

Again, Cc Yu-cheng Yu if doing that, since it might affect his patches.

Reviewed-by: Dave Martin <Dave.Martin@arm.com>

(though if some of the suggested changes are made elsewhere, this will
probably need a minor respin).

Cheers
---Dave

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v1 2/2] arm64: Enable BTI for main executable as well as the interpreter

[Mark Brown]

[-- Attachment #1.1: Type: text/plain, Size: 1017 bytes --]

On Thu, Jun 03, 2021 at 04:40:35PM +0100, Dave Martin wrote:

> Do we know how libcs will detect that they don't need to do the
> mprotect() calls?  Do we need a detection mechanism at all?

> Ignoring certain errors from mprotect() when ld.so is trying to set
> PROT_BTI on the main executable's code pages is probably a reasonable,
> backwards-compatible compromise here, but it seems a bit wasteful.

I think the theory was that they would just do the mprotect() calls and
ignore any errors as they currently do, or declare that they depend on a
new enough kernel version I guess (not an option for glibc but might be
for others which didn't do BTI yet).

> > flexibility userspace has to disable BTI but it is expected that for cases
> > where there are problems which require BTI to be disabled it is more likely
> > that it will need to be disabled on a system level.

> There's no flexibility impact unless MemoryDenyWriteExecute is in force,
> right?

Right, or some other mechanism that has the same effect.

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

[-- Attachment #2: Type: text/plain, Size: 176 bytes --]

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v1 2/2] arm64: Enable BTI for main executable as well as the interpreter

[Catalin Marinas]

On Thu, Jun 03, 2021 at 05:51:34PM +0100, Mark Brown wrote:
> On Thu, Jun 03, 2021 at 04:40:35PM +0100, Dave Martin wrote:
> > Do we know how libcs will detect that they don't need to do the
> > mprotect() calls?  Do we need a detection mechanism at all?
> > 
> > Ignoring certain errors from mprotect() when ld.so is trying to set
> > PROT_BTI on the main executable's code pages is probably a reasonable,
> > backwards-compatible compromise here, but it seems a bit wasteful.
> 
> I think the theory was that they would just do the mprotect() calls and
> ignore any errors as they currently do, or declare that they depend on a
> new enough kernel version I guess (not an option for glibc but might be
> for others which didn't do BTI yet).

I think we discussed the possibility of an AT_FLAGS bit. Until recently,
this field was 0 but it gained a new bit now. If we are to expose this
to arch-specific things, it may need some reservations. Anyway, that's
an optimisation that can be added subsequently.

-- 
Catalin

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v1 1/2] elf: Allow architectures to parse properties on the main executable

[Mark Brown]

[-- Attachment #1.1: Type: text/plain, Size: 587 bytes --]

On Thu, Jun 03, 2021 at 04:40:24PM +0100, Dave Martin wrote:
> On Fri, May 21, 2021 at 03:46:20PM +0100, Mark Brown wrote:

> > -		if (system_supports_bti() &&
> > +		if (system_supports_bti() && is_interp &&

> Won't this cause BTI to be forced off for static binaries?

> Perhaps this should be (has_interp == is_interp), as for
> arch_elf_adjust_prot().  Seems gross though, since has_interp would
> become useless after the next patch.  If there's no sensible way to
> keep this bisectable, perhaps the patches can be merged instead.

Ugh, right.  I only tested the finished result.

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

[-- Attachment #2: Type: text/plain, Size: 176 bytes --]

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v1 2/2] arm64: Enable BTI for main executable as well as the interpreter

[Dave Martin]

On Thu, Jun 03, 2021 at 07:04:31PM +0100, Catalin Marinas via Libc-alpha wrote:
> On Thu, Jun 03, 2021 at 05:51:34PM +0100, Mark Brown wrote:
> > On Thu, Jun 03, 2021 at 04:40:35PM +0100, Dave Martin wrote:
> > > Do we know how libcs will detect that they don't need to do the
> > > mprotect() calls?  Do we need a detection mechanism at all?
> > > 
> > > Ignoring certain errors from mprotect() when ld.so is trying to set
> > > PROT_BTI on the main executable's code pages is probably a reasonable,
> > > backwards-compatible compromise here, but it seems a bit wasteful.
> > 
> > I think the theory was that they would just do the mprotect() calls and
> > ignore any errors as they currently do, or declare that they depend on a
> > new enough kernel version I guess (not an option for glibc but might be
> > for others which didn't do BTI yet).
> 
> I think we discussed the possibility of an AT_FLAGS bit. Until recently,
> this field was 0 but it gained a new bit now. If we are to expose this
> to arch-specific things, it may need some reservations. Anyway, that's
> an optimisation that can be added subsequently.

I suppose so, but AT_FLAGS doesn't seem appropriate somehow.

I wonder why we suddenly start considering adding a flag to AT_FLAGS
every few months, when it had sat empty for decades.  This may say
something about the current health of the kernel ABI, but I'm not sure
exactly what.

I think having mprotect() fail in a predictable way may be preferable
for now: glibc still only needs to probe with a single call and could
cache the knowledge after that.  Code outside libc / ld.so seems quite
unlikely to care about this.

Since only the executable segment(s) of the main binary need to be
protected, this should require only a very small number of mprotect()
calls in normal situations.  Although it feels a bit cruddy as a design,
cost-wise I think that extra overhead would be swamped by other noise in
realistic scenarios.  Often, there is just a single executable segment,
so the common case would probably require just one mprotect() call.  I
don't know if it ever gets much more complicated when using the
standard linker scripts.

Any ideas on how we would document this behaviour?  The kernel and libc
behaviour are 100% clear: you _are_ allowed to twiddle PROT_BTI on
executable mappings, and there is no legitimate (or even useful) reason
to disallow this.  It's only systemd deliberately breaking the API that
causes the behaviour seem by "userspace" to vary.

Cheers
---Dave

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v1 2/2] arm64: Enable BTI for main executable as well as the interpreter

[Catalin Marinas]

On Mon, Jun 07, 2021 at 12:25:38PM +0100, Dave P Martin wrote:
> On Thu, Jun 03, 2021 at 07:04:31PM +0100, Catalin Marinas via Libc-alpha wrote:
> > On Thu, Jun 03, 2021 at 05:51:34PM +0100, Mark Brown wrote:
> > > On Thu, Jun 03, 2021 at 04:40:35PM +0100, Dave Martin wrote:
> > > > Do we know how libcs will detect that they don't need to do the
> > > > mprotect() calls?  Do we need a detection mechanism at all?
> > > > 
> > > > Ignoring certain errors from mprotect() when ld.so is trying to set
> > > > PROT_BTI on the main executable's code pages is probably a reasonable,
> > > > backwards-compatible compromise here, but it seems a bit wasteful.
> > > 
> > > I think the theory was that they would just do the mprotect() calls and
> > > ignore any errors as they currently do, or declare that they depend on a
> > > new enough kernel version I guess (not an option for glibc but might be
> > > for others which didn't do BTI yet).
> > 
> > I think we discussed the possibility of an AT_FLAGS bit. Until recently,
> > this field was 0 but it gained a new bit now. If we are to expose this
> > to arch-specific things, it may need some reservations. Anyway, that's
> > an optimisation that can be added subsequently.
> 
> I suppose so, but AT_FLAGS doesn't seem appropriate somehow.
> 
> I wonder why we suddenly start considering adding a flag to AT_FLAGS
> every few months, when it had sat empty for decades.  This may say
> something about the current health of the kernel ABI, but I'm not sure
> exactly what.
> 
> I think having mprotect() fail in a predictable way may be preferable
> for now: glibc still only needs to probe with a single call and could
> cache the knowledge after that.  Code outside libc / ld.so seems quite
> unlikely to care about this.

I think that's the expected approach for now. If anyone complains about
an extra syscall, we can look into options but I wouldn't rush on doing
something.

> Any ideas on how we would document this behaviour?  The kernel and libc
> behaviour are 100% clear: you _are_ allowed to twiddle PROT_BTI on
> executable mappings, and there is no legitimate (or even useful) reason
> to disallow this.  It's only systemd deliberately breaking the API that
> causes the behaviour seem by "userspace" to vary.

I don't think we can document all the filters that can be added on top
various syscalls, so I'd leave it undocumented (or part of the systemd
documentation). It was a user space program (systemd) breaking another
user space program (well, anything with a new enough glibc). The kernel
ABI was still valid when /sbin/init started ;).

-- 
Catalin

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v1 2/2] arm64: Enable BTI for main executable as well as the interpreter

[Mark Brown]

[-- Attachment #1.1: Type: text/plain, Size: 773 bytes --]

On Mon, Jun 07, 2021 at 07:12:13PM +0100, Catalin Marinas wrote:

> I don't think we can document all the filters that can be added on top
> various syscalls, so I'd leave it undocumented (or part of the systemd
> documentation). It was a user space program (systemd) breaking another
> user space program (well, anything with a new enough glibc). The kernel
> ABI was still valid when /sbin/init started ;).

Indeed.  I think from a kernel point of view the main thing is to look
at why userspace feels the need to do things like this and see if
there's anything we can improve or do better with in future APIs, part
of the original discussion here was figuring out that there's not really
any other reasonable options for userspace to implement this check at
the minute.

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

[-- Attachment #2: Type: text/plain, Size: 176 bytes --]

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v1 2/2] arm64: Enable BTI for main executable as well as the interpreter

[Dave Martin]

On Tue, Jun 08, 2021 at 12:33:18PM +0100, Mark Brown via Libc-alpha wrote:
> On Mon, Jun 07, 2021 at 07:12:13PM +0100, Catalin Marinas wrote:
> 
> > I don't think we can document all the filters that can be added on top
> > various syscalls, so I'd leave it undocumented (or part of the systemd
> > documentation). It was a user space program (systemd) breaking another
> > user space program (well, anything with a new enough glibc). The kernel
> > ABI was still valid when /sbin/init started ;).
> 
> Indeed.  I think from a kernel point of view the main thing is to look
> at why userspace feels the need to do things like this and see if
> there's anything we can improve or do better with in future APIs, part
> of the original discussion here was figuring out that there's not really
> any other reasonable options for userspace to implement this check at
> the minute.

Ack, that would be my policy -- just wanted to make it explicit.
It would be good if there were better dialogue between the systemd
and kernel folks on this kind of thing.

SECCOMP makes it rather easy to (attempt to) paper over kernel/user API
design problems, which probably reduces the chance of the API ever being
fixed properly, if we're not careful...

Cheers
---Dave

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v1 2/2] arm64: Enable BTI for main executable as well as the interpreter

[Jeremy Linton]

On 6/8/21 10:19 AM, Dave Martin wrote:
> On Tue, Jun 08, 2021 at 12:33:18PM +0100, Mark Brown via Libc-alpha wrote:
>> On Mon, Jun 07, 2021 at 07:12:13PM +0100, Catalin Marinas wrote:
>>
>>> I don't think we can document all the filters that can be added on top
>>> various syscalls, so I'd leave it undocumented (or part of the systemd
>>> documentation). It was a user space program (systemd) breaking another
>>> user space program (well, anything with a new enough glibc). The kernel
>>> ABI was still valid when /sbin/init started ;).
>>
>> Indeed.  I think from a kernel point of view the main thing is to look
>> at why userspace feels the need to do things like this and see if
>> there's anything we can improve or do better with in future APIs, part
>> of the original discussion here was figuring out that there's not really
>> any other reasonable options for userspace to implement this check at
>> the minute.
> 
> Ack, that would be my policy -- just wanted to make it explicit.
> It would be good if there were better dialogue between the systemd
> and kernel folks on this kind of thing.
> 
> SECCOMP makes it rather easy to (attempt to) paper over kernel/user API
> design problems, which probably reduces the chance of the API ever being
> fixed properly, if we're not careful...

Well IMHO the problem is larger than just BTI here, what systemd is 
trying to do by fixing the exec state of a service is admirable but its 
a 90% solution without the entire linker/loader being in a more 
privileged context. While BTI makes finding a generic gadget that can 
call mprotect harder, it still seems like it might just be a little too 
easy. The secomp filter is providing a nice bonus by removing the 
ability to disable BTI via mprotect without also disabling X. So without 
moving more of the linker into the kernel its hard to see how one can 
really lock down X only pages.

Anyway, i'm testing this on rawhide now.

Thanks!



_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v1 2/2] arm64: Enable BTI for main executable as well as the interpreter

[Dave Martin]

On Tue, Jun 08, 2021 at 10:42:41AM -0500, Jeremy Linton wrote:
> On 6/8/21 10:19 AM, Dave Martin wrote:
> >On Tue, Jun 08, 2021 at 12:33:18PM +0100, Mark Brown via Libc-alpha wrote:
> >>On Mon, Jun 07, 2021 at 07:12:13PM +0100, Catalin Marinas wrote:
> >>
> >>>I don't think we can document all the filters that can be added on top
> >>>various syscalls, so I'd leave it undocumented (or part of the systemd
> >>>documentation). It was a user space program (systemd) breaking another
> >>>user space program (well, anything with a new enough glibc). The kernel
> >>>ABI was still valid when /sbin/init started ;).
> >>
> >>Indeed.  I think from a kernel point of view the main thing is to look
> >>at why userspace feels the need to do things like this and see if
> >>there's anything we can improve or do better with in future APIs, part
> >>of the original discussion here was figuring out that there's not really
> >>any other reasonable options for userspace to implement this check at
> >>the minute.
> >
> >Ack, that would be my policy -- just wanted to make it explicit.
> >It would be good if there were better dialogue between the systemd
> >and kernel folks on this kind of thing.
> >
> >SECCOMP makes it rather easy to (attempt to) paper over kernel/user API
> >design problems, which probably reduces the chance of the API ever being
> >fixed properly, if we're not careful...
> 
> Well IMHO the problem is larger than just BTI here, what systemd is trying
> to do by fixing the exec state of a service is admirable but its a 90%
> solution without the entire linker/loader being in a more privileged
> context. While BTI makes finding a generic gadget that can call mprotect
> harder, it still seems like it might just be a little too easy. The secomp
> filter is providing a nice bonus by removing the ability to disable BTI via
> mprotect without also disabling X. So without moving more of the linker into
> the kernel its hard to see how one can really lock down X only pages.
> 
> Anyway, i'm testing this on rawhide now.
> 
> Thanks!

Well, I agree that there are larger issues here.  But we need to be
realistic and try not to do too much damage to future maintainability.

Note, your "bonus" is really a feature-like bug.  This is what we
should be trying to avoid IMHO: if it's important, it needs to be
designed and guaranteed.  Something that works by accident is likely to
get broken again by accident in the future.

Cheers
---Dave

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel