[PATCH v2] execve: warn if process starts with executable stack

[PATCH v2] execve: warn if process starts with executable stack

[Alexey Dobriyan]

There were few episodes of silent downgrade to an executable stack over
years:

1) linking innocent looking assembly file will silently add executable
   stack if proper linker options is not given as well:

	$ cat f.S
	.intel_syntax noprefix
	.text
	.globl f
	f:
	        ret

	$ cat main.c
	void f(void);
	int main(void)
	{
	        f();
	        return 0;
	}

	$ gcc main.c f.S
	$ readelf -l ./a.out
	  GNU_STACK      0x0000000000000000 0x0000000000000000 0x0000000000000000
                         0x0000000000000000 0x0000000000000000  RWE    0x10
			 					 ^^^

2) converting C99 nested function into a closure
https://nullprogram.com/blog/2019/11/15/

	void intsort2(int *base, size_t nmemb, _Bool invert)
	{
	    int cmp(const void *a, const void *b)
	    {
	        int r = *(int *)a - *(int *)b;
	        return invert ? -r : r;
	    }
	    qsort(base, nmemb, sizeof(*base), cmp);
	}

will silently require stack trampolines while non-closure version will not.

Without doubt this behaviour is documented somewhere, add a warning so that
developers and users can at least notice. After so many years of x86_64 having
proper executable stack support it should not cause too many problems.

Signed-off-by: Alexey Dobriyan <adobriyan@gmail.com>
---

	v2: print pathname instead of comm/pid

 fs/exec.c |    5 +++++
 1 file changed, 5 insertions(+)

--- a/fs/exec.c
+++ b/fs/exec.c
@@ -761,6 +761,11 @@ int setup_arg_pages(struct linux_binprm *bprm,
 		goto out_unlock;
 	BUG_ON(prev != vma);
 
+	if (unlikely(vm_flags & VM_EXEC)) {
+		pr_warn_once("process '%pD4' started with executable stack\n",
+			     bprm->file);
+	}
+
 	/* Move stack pages down in memory. */
 	if (stack_shift) {
 		ret = shift_arg_pages(vma, stack_shift);

Re: [PATCH v2] execve: warn if process starts with executable stack

[Andrew Morton]

On Sun, 8 Dec 2019 20:19:18 +0300 Alexey Dobriyan <adobriyan@gmail.com> wrote:

> There were few episodes of silent downgrade to an executable stack over
> years:
> 
> 1) linking innocent looking assembly file will silently add executable
>    stack if proper linker options is not given as well:
> 
> 	$ cat f.S
> 	.intel_syntax noprefix
> 	.text
> 	.globl f
> 	f:
> 	        ret
> 
> 	$ cat main.c
> 	void f(void);
> 	int main(void)
> 	{
> 	        f();
> 	        return 0;
> 	}
> 
> 	$ gcc main.c f.S
> 	$ readelf -l ./a.out
> 	  GNU_STACK      0x0000000000000000 0x0000000000000000 0x0000000000000000
>                          0x0000000000000000 0x0000000000000000  RWE    0x10
> 			 					 ^^^
> 
> 2) converting C99 nested function into a closure
> https://nullprogram.com/blog/2019/11/15/
> 
> 	void intsort2(int *base, size_t nmemb, _Bool invert)
> 	{
> 	    int cmp(const void *a, const void *b)
> 	    {
> 	        int r = *(int *)a - *(int *)b;
> 	        return invert ? -r : r;
> 	    }
> 	    qsort(base, nmemb, sizeof(*base), cmp);
> 	}
> 
> will silently require stack trampolines while non-closure version will not.
> 
> Without doubt this behaviour is documented somewhere, add a warning so that
> developers and users can at least notice. After so many years of x86_64 having
> proper executable stack support it should not cause too many problems.

hm, OK, let's give it a trial run.

> --- a/fs/exec.c
> +++ b/fs/exec.c
> @@ -761,6 +761,11 @@ int setup_arg_pages(struct linux_binprm *bprm,
>  		goto out_unlock;
>  	BUG_ON(prev != vma);
>  
> +	if (unlikely(vm_flags & VM_EXEC)) {
> +		pr_warn_once("process '%pD4' started with executable stack\n",
> +			     bprm->file);
> +	}
> +
>  	/* Move stack pages down in memory. */
>  	if (stack_shift) {
>  		ret = shift_arg_pages(vma, stack_shift);

What are poor users supposed to do if this message comes out? 
Hopefully google the message and end up at this thread.  What do you
want to tell them?

Re: [PATCH v2] execve: warn if process starts with executable stack

[Alexey Dobriyan]

On Tue, Dec 10, 2019 at 05:47:26PM -0800, Andrew Morton wrote:
> On Sun, 8 Dec 2019 20:19:18 +0300 Alexey Dobriyan <adobriyan@gmail.com> wrote:
> 
> > There were few episodes of silent downgrade to an executable stack over
> > years:
> > 
> > 1) linking innocent looking assembly file will silently add executable
> >    stack if proper linker options is not given as well:
> > 
> > 	$ cat f.S
> > 	.intel_syntax noprefix
> > 	.text
> > 	.globl f
> > 	f:
> > 	        ret
> > 
> > 	$ cat main.c
> > 	void f(void);
> > 	int main(void)
> > 	{
> > 	        f();
> > 	        return 0;
> > 	}
> > 
> > 	$ gcc main.c f.S
> > 	$ readelf -l ./a.out
> > 	  GNU_STACK      0x0000000000000000 0x0000000000000000 0x0000000000000000
> >                          0x0000000000000000 0x0000000000000000  RWE    0x10
> > 			 					 ^^^
> > 
> > 2) converting C99 nested function into a closure
> > https://nullprogram.com/blog/2019/11/15/
> > 
> > 	void intsort2(int *base, size_t nmemb, _Bool invert)
> > 	{
> > 	    int cmp(const void *a, const void *b)
> > 	    {
> > 	        int r = *(int *)a - *(int *)b;
> > 	        return invert ? -r : r;
> > 	    }
> > 	    qsort(base, nmemb, sizeof(*base), cmp);
> > 	}
> > 
> > will silently require stack trampolines while non-closure version will not.
> > 
> > Without doubt this behaviour is documented somewhere, add a warning so that
> > developers and users can at least notice. After so many years of x86_64 having
> > proper executable stack support it should not cause too many problems.
> 
> hm, OK, let's give it a trial run.
> 
> > --- a/fs/exec.c
> > +++ b/fs/exec.c
> > @@ -761,6 +761,11 @@ int setup_arg_pages(struct linux_binprm *bprm,
> >  		goto out_unlock;
> >  	BUG_ON(prev != vma);
> >  
> > +	if (unlikely(vm_flags & VM_EXEC)) {
> > +		pr_warn_once("process '%pD4' started with executable stack\n",
> > +			     bprm->file);
> > +	}
> > +
> >  	/* Move stack pages down in memory. */
> >  	if (stack_shift) {
> >  		ret = shift_arg_pages(vma, stack_shift);
> 
> What are poor users supposed to do if this message comes out? 
> Hopefully google the message and end up at this thread.  What do you
> want to tell them?

Me? Nothing :-) They hopefully should file tickets against distros and ISV,
post egregious examples to oss-security.

Like they already do against this warning!
> ACPI: [Firmware Bug]: BIOS _OSI(Linux) query ignored

Re: [PATCH v2] execve: warn if process starts with executable stack

[Willy Tarreau]

On Wed, Dec 11, 2019 at 10:22:25AM +0300, Alexey Dobriyan wrote:
> On Tue, Dec 10, 2019 at 05:47:26PM -0800, Andrew Morton wrote:
> > On Sun, 8 Dec 2019 20:19:18 +0300 Alexey Dobriyan <adobriyan@gmail.com> wrote:
> > 
> > > There were few episodes of silent downgrade to an executable stack over
> > > years:
> > > 
> > > 1) linking innocent looking assembly file will silently add executable
> > >    stack if proper linker options is not given as well:
> > > 
> > > 	$ cat f.S
> > > 	.intel_syntax noprefix
> > > 	.text
> > > 	.globl f
> > > 	f:
> > > 	        ret
> > > 
> > > 	$ cat main.c
> > > 	void f(void);
> > > 	int main(void)
> > > 	{
> > > 	        f();
> > > 	        return 0;
> > > 	}
> > > 
> > > 	$ gcc main.c f.S
> > > 	$ readelf -l ./a.out
> > > 	  GNU_STACK      0x0000000000000000 0x0000000000000000 0x0000000000000000
> > >                          0x0000000000000000 0x0000000000000000  RWE    0x10
> > > 			 					 ^^^
> > > 
> > > 2) converting C99 nested function into a closure
> > > https://nullprogram.com/blog/2019/11/15/
> > > 
> > > 	void intsort2(int *base, size_t nmemb, _Bool invert)
> > > 	{
> > > 	    int cmp(const void *a, const void *b)
> > > 	    {
> > > 	        int r = *(int *)a - *(int *)b;
> > > 	        return invert ? -r : r;
> > > 	    }
> > > 	    qsort(base, nmemb, sizeof(*base), cmp);
> > > 	}
> > > 
> > > will silently require stack trampolines while non-closure version will not.
> > > 
> > > Without doubt this behaviour is documented somewhere, add a warning so that
> > > developers and users can at least notice. After so many years of x86_64 having
> > > proper executable stack support it should not cause too many problems.
> > 
> > hm, OK, let's give it a trial run.
> > 
> > > --- a/fs/exec.c
> > > +++ b/fs/exec.c
> > > @@ -761,6 +761,11 @@ int setup_arg_pages(struct linux_binprm *bprm,
> > >  		goto out_unlock;
> > >  	BUG_ON(prev != vma);
> > >  
> > > +	if (unlikely(vm_flags & VM_EXEC)) {
> > > +		pr_warn_once("process '%pD4' started with executable stack\n",
> > > +			     bprm->file);
> > > +	}
> > > +
> > >  	/* Move stack pages down in memory. */
> > >  	if (stack_shift) {
> > >  		ret = shift_arg_pages(vma, stack_shift);
> > 
> > What are poor users supposed to do if this message comes out? 
> > Hopefully google the message and end up at this thread.  What do you
> > want to tell them?
> 
> Me? Nothing :-) They hopefully should file tickets against distros and ISV,
> post egregious examples to oss-security.
> 
> Like they already do against this warning!
> > ACPI: [Firmware Bug]: BIOS _OSI(Linux) query ignored

Alexey, Andrew is right. Your message gives no instruction and users are
already flooded with messages they got used to ignore. A warning is made
to catch attention so it should give instructions. It can either say
"this application relies on insecure capabilities and might not work
anymore in the future, you should report this to its author", or "report
this to kernel developers if you think this warning is inappropriate".
Otherwise it will just go to /dev/null with all warning about bad blocks
on USB sticks and CPU core throttling under high temperature.

Willy

Re: [PATCH v2] execve: warn if process starts with executable stack

[Alexey Dobriyan]

On Wed, Dec 11, 2019 at 10:59:37AM +0100, Willy Tarreau wrote:
> On Wed, Dec 11, 2019 at 10:22:25AM +0300, Alexey Dobriyan wrote:
> > On Tue, Dec 10, 2019 at 05:47:26PM -0800, Andrew Morton wrote:
> > > On Sun, 8 Dec 2019 20:19:18 +0300 Alexey Dobriyan <adobriyan@gmail.com> wrote:
> > > 
> > > > There were few episodes of silent downgrade to an executable stack over
> > > > years:
> > > > 
> > > > 1) linking innocent looking assembly file will silently add executable
> > > >    stack if proper linker options is not given as well:
> > > > 
> > > > 	$ cat f.S
> > > > 	.intel_syntax noprefix
> > > > 	.text
> > > > 	.globl f
> > > > 	f:
> > > > 	        ret
> > > > 
> > > > 	$ cat main.c
> > > > 	void f(void);
> > > > 	int main(void)
> > > > 	{
> > > > 	        f();
> > > > 	        return 0;
> > > > 	}
> > > > 
> > > > 	$ gcc main.c f.S
> > > > 	$ readelf -l ./a.out
> > > > 	  GNU_STACK      0x0000000000000000 0x0000000000000000 0x0000000000000000
> > > >                          0x0000000000000000 0x0000000000000000  RWE    0x10
> > > > 			 					 ^^^
> > > > 
> > > > 2) converting C99 nested function into a closure
> > > > https://nullprogram.com/blog/2019/11/15/
> > > > 
> > > > 	void intsort2(int *base, size_t nmemb, _Bool invert)
> > > > 	{
> > > > 	    int cmp(const void *a, const void *b)
> > > > 	    {
> > > > 	        int r = *(int *)a - *(int *)b;
> > > > 	        return invert ? -r : r;
> > > > 	    }
> > > > 	    qsort(base, nmemb, sizeof(*base), cmp);
> > > > 	}
> > > > 
> > > > will silently require stack trampolines while non-closure version will not.
> > > > 
> > > > Without doubt this behaviour is documented somewhere, add a warning so that
> > > > developers and users can at least notice. After so many years of x86_64 having
> > > > proper executable stack support it should not cause too many problems.
> > > 
> > > hm, OK, let's give it a trial run.
> > > 
> > > > --- a/fs/exec.c
> > > > +++ b/fs/exec.c
> > > > @@ -761,6 +761,11 @@ int setup_arg_pages(struct linux_binprm *bprm,
> > > >  		goto out_unlock;
> > > >  	BUG_ON(prev != vma);
> > > >  
> > > > +	if (unlikely(vm_flags & VM_EXEC)) {
> > > > +		pr_warn_once("process '%pD4' started with executable stack\n",
> > > > +			     bprm->file);
> > > > +	}
> > > > +
> > > >  	/* Move stack pages down in memory. */
> > > >  	if (stack_shift) {
> > > >  		ret = shift_arg_pages(vma, stack_shift);
> > > 
> > > What are poor users supposed to do if this message comes out? 
> > > Hopefully google the message and end up at this thread.  What do you
> > > want to tell them?
> > 
> > Me? Nothing :-) They hopefully should file tickets against distros and ISV,
> > post egregious examples to oss-security.
> > 
> > Like they already do against this warning!
> > > ACPI: [Firmware Bug]: BIOS _OSI(Linux) query ignored
> 
> Alexey, Andrew is right. Your message gives no instruction and users are
> already flooded with messages they got used to ignore. A warning is made
> to catch attention so it should give instructions. It can either say
> "this application relies on insecure capabilities and might not work
> anymore in the future, you should report this to its author",

"Insecure" is an accusation and baseless until source code is seen.
This program is secure despite having executable stack.

	.globl _start
	_start:
		mov	eax, 60
		xor	edi, edi
		syscall

"Might not work" is false, because PT_GNU_STACK is not going anywhere and
the default is not flipped and applications with executable stack aren't
going to become rejected. This patch is not about starting deprecation
executable stacks programme.

Ideally distros which don't have post build check implement one after
seeing few reports and someone reports few cases to ISV who aren't
clueless as well.

> or "report
> this to kernel developers if you think this warning is inappropriate".

Reports are better be done by people who know what they are doing, as in
understand what executable stack is and what does it mean in reality.

> Otherwise it will just go to /dev/null with all warning about bad blocks
> on USB sticks and CPU core throttling under high temperature.

That's fine. You don't want bugreports from people who don't know what
is executable stack. Every security bug bounty program is flooded by
such people. This is why message is worded in a neutral way.

Re: [PATCH v2] execve: warn if process starts with executable stack

[Willy Tarreau]

On Wed, Dec 11, 2019 at 09:19:33PM +0300, Alexey Dobriyan wrote:
> Reports are better be done by people who know what they are doing, as in
> understand what executable stack is and what does it mean in reality.
> 
> > Otherwise it will just go to /dev/null with all warning about bad blocks
> > on USB sticks and CPU core throttling under high temperature.
> 
> That's fine. You don't want bugreports from people who don't know what
> is executable stack. Every security bug bounty program is flooded by
> such people. This is why message is worded in a neutral way.

Well we definitely don't have the same experience with user reports. I
was just suggesting, but since you apparently already have all the
responses you needed, I'm even wondering why the warning remains.

Willy

Re: [PATCH v2] execve: warn if process starts with executable stack

[Alexey Dobriyan]

On Wed, Dec 11, 2019 at 07:24:01PM +0100, Willy Tarreau wrote:
> On Wed, Dec 11, 2019 at 09:19:33PM +0300, Alexey Dobriyan wrote:
> > Reports are better be done by people who know what they are doing, as in
> > understand what executable stack is and what does it mean in reality.
> > 
> > > Otherwise it will just go to /dev/null with all warning about bad blocks
> > > on USB sticks and CPU core throttling under high temperature.
> > 
> > That's fine. You don't want bugreports from people who don't know what
> > is executable stack. Every security bug bounty program is flooded by
> > such people. This is why message is worded in a neutral way.
> 
> Well we definitely don't have the same experience with user reports. I
> was just suggesting, but since you apparently already have all the
> responses you needed, I'm even wondering why the warning remains.

Willy, whatever instructions for users you have in mind must be
different for different people. Developer should be told to add
"-Wl,-z,noexecstack" and more. Regular user (define "regular") should be
told to send bugreport if the program really needs executable stack
which again splits into two situations: exec stack was added knowingly
because it is some old program with lost source code or it was readded
by mistake.

"Complain to linux-kernel" is meaningless, kernel is not responsible.

What the message is even supposed to say?

It is not even pr_err.

Re: [PATCH v2] execve: warn if process starts with executable stack

[Dan Carpenter]

On Fri, Dec 13, 2019 at 12:25:20AM +0300, Alexey Dobriyan wrote:
> On Wed, Dec 11, 2019 at 07:24:01PM +0100, Willy Tarreau wrote:
> > On Wed, Dec 11, 2019 at 09:19:33PM +0300, Alexey Dobriyan wrote:
> > > Reports are better be done by people who know what they are doing, as in
> > > understand what executable stack is and what does it mean in reality.
> > > 
> > > > Otherwise it will just go to /dev/null with all warning about bad blocks
> > > > on USB sticks and CPU core throttling under high temperature.
> > > 
> > > That's fine. You don't want bugreports from people who don't know what
> > > is executable stack. Every security bug bounty program is flooded by
> > > such people. This is why message is worded in a neutral way.
> > 
> > Well we definitely don't have the same experience with user reports. I
> > was just suggesting, but since you apparently already have all the
> > responses you needed, I'm even wondering why the warning remains.
> 
> Willy, whatever instructions for users you have in mind must be
> different for different people. Developer should be told to add
> "-Wl,-z,noexecstack" and more. Regular user (define "regular") should be
> told to send bugreport if the program really needs executable stack
> which again splits into two situations: exec stack was added knowingly
> because it is some old program with lost source code or it was readded
> by mistake.
> 
> "Complain to linux-kernel" is meaningless, kernel is not responsible.
> 
> What the message is even supposed to say?
> 

You could direct people to a website and then update the instructions
as needed.

regards,
dan carpenter

Re: [PATCH v2] execve: warn if process starts with executable stack

[Willy Tarreau]

On Fri, Dec 13, 2019 at 12:56:34PM +0300, Dan Carpenter wrote:
> On Fri, Dec 13, 2019 at 12:25:20AM +0300, Alexey Dobriyan wrote:
> > "Complain to linux-kernel" is meaningless, kernel is not responsible.
> > 
> > What the message is even supposed to say?
> > 
> 
> You could direct people to a website and then update the instructions
> as needed.

Another possibility is to just log this as a debug message, and in this
case the user can feel free to ignore it. But a warning is something that
needs to be addressed and without instructions it's hard.

Regards,
Willy

Re: [PATCH v2] execve: warn if process starts with executable stack

[Kees Cook]

On Tue, Dec 10, 2019 at 05:47:26PM -0800, Andrew Morton wrote:
> On Sun, 8 Dec 2019 20:19:18 +0300 Alexey Dobriyan <adobriyan@gmail.com> wrote:
> 
> > There were few episodes of silent downgrade to an executable stack over
> > years:
> > 
> > 1) linking innocent looking assembly file will silently add executable
> >    stack if proper linker options is not given as well:
> > 
> > 	$ cat f.S
> > 	.intel_syntax noprefix
> > 	.text
> > 	.globl f
> > 	f:
> > 	        ret
> > 
> > 	$ cat main.c
> > 	void f(void);
> > 	int main(void)
> > 	{
> > 	        f();
> > 	        return 0;
> > 	}
> > 
> > 	$ gcc main.c f.S
> > 	$ readelf -l ./a.out
> > 	  GNU_STACK      0x0000000000000000 0x0000000000000000 0x0000000000000000
> >                          0x0000000000000000 0x0000000000000000  RWE    0x10
> > 			 					 ^^^
> > 
> > 2) converting C99 nested function into a closure
> > https://nullprogram.com/blog/2019/11/15/
> > 
> > 	void intsort2(int *base, size_t nmemb, _Bool invert)
> > 	{
> > 	    int cmp(const void *a, const void *b)
> > 	    {
> > 	        int r = *(int *)a - *(int *)b;
> > 	        return invert ? -r : r;
> > 	    }
> > 	    qsort(base, nmemb, sizeof(*base), cmp);
> > 	}
> > 
> > will silently require stack trampolines while non-closure version will not.
> > 
> > Without doubt this behaviour is documented somewhere, add a warning so that
> > developers and users can at least notice. After so many years of x86_64 having
> > proper executable stack support it should not cause too many problems.
> 
> hm, OK, let's give it a trial run.

So, I'm a fan of the sentiment here, but this check is bogus for
several architectural and programmatic combinations. Sometimes
it's an intentional choice, not an "accident". e.g. klibc
uses trampolines for its setjmp implementation:
https://lists.zytor.com/archives/klibc/2020-February/004271.html

So now all the klibc initrds are throwing this warning (and then never
warning again, so it's extra pointless for such machines as "legitimate"
cases will never be discovered).

I've got a series up to fix the much sneakier version of this
(READ_IMPLIES_EXEC[1]), but at this point, I think this problem is well
addressed by distros scanning for this kind of thing in their repos.
For example, Ubuntu did this years ago (and flagged klibc as having
this condition):
https://wiki.ubuntu.com/SecurityTeam/Roadmap/ExecutableStacks

> 
> > --- a/fs/exec.c
> > +++ b/fs/exec.c
> > @@ -761,6 +761,11 @@ int setup_arg_pages(struct linux_binprm *bprm,
> >  		goto out_unlock;
> >  	BUG_ON(prev != vma);
> >  
> > +	if (unlikely(vm_flags & VM_EXEC)) {
> > +		pr_warn_once("process '%pD4' started with executable stack\n",
> > +			     bprm->file);
> > +	}
> > +
> >  	/* Move stack pages down in memory. */
> >  	if (stack_shift) {
> >  		ret = shift_arg_pages(vma, stack_shift);
> 
> What are poor users supposed to do if this message comes out? 
> Hopefully google the message and end up at this thread.  What do you
> want to tell them?

And that's the other problem (for which I see there is a thread). The
user can't actually do anything here.

I would really like something like this check, but maintaining a whitelist
of architectures or binaries to ignore is going to be awful, and there
is a very non-zero amount of software is built this way intentionally...

-Kees

[1] https://lore.kernel.org/lkml/20200225051307.6401-1-keescook@chromium.org/#r

-- 
Kees Cook