[PATCH] Makefile: LTO: have linker check -Wframe-larger-than

[PATCH] Makefile: LTO: have linker check -Wframe-larger-than

[Nick Desaulniers]

-Wframe-larger-than= requires stack frame information, which the
frontend cannot provide. This diagnostic is emitted late during
compilation once stack frame size is available.

When building with LTO, the frontend simply lowers C to LLVM IR and does
not have stack frame information, so it cannot emit this diagnostic.
When the linker drives LTO, it restarts optimizations and lowers LLVM IR
to object code. At that point, it has stack frame information but
doesn't know to check for a specific max stack frame size.

I consider this a bug in LLVM that we need to fix. There are some
details we're working out related to LTO such as which value to use when
there are multiple different values specified per TU, or how to
propagate these to compiler synthesized routines properly, if at all.

Until it's fixed, ensure we don't miss these. At that point we can wrap
this in a compiler version guard or revert this based on the minimum
support version of Clang.

The error message is not generated during link:
  LTO     vmlinux.o
ld.lld: warning: stack size limit exceeded (8224) in foobarbaz

Cc: Sami Tolvanen <samitolvanen@google.com>
Reported-by: Candle Sun <candlesea@gmail.com>
Suggested-by: Fangrui Song <maskray@google.com>
Signed-off-by: Nick Desaulniers <ndesaulniers@google.com>
---
LTO users might want to `make clean` or `rm -rf .thinlto-cache` to test
this.

 Makefile | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/Makefile b/Makefile
index f9b54da2fca0..74566b1417b8 100644
--- a/Makefile
+++ b/Makefile
@@ -910,6 +910,11 @@ CC_FLAGS_LTO	+= -fvisibility=hidden
 
 # Limit inlining across translation units to reduce binary size
 KBUILD_LDFLAGS += -mllvm -import-instr-limit=5
+
+# Check for frame size exceeding threshold during prolog/epilog insertion.
+ifneq ($(CONFIG_FRAME_WARN),0)
+KBUILD_LDFLAGS	+= -plugin-opt=-warn-stack-size=$(CONFIG_FRAME_WARN)
+endif
 endif
 
 ifdef CONFIG_LTO
-- 
2.31.0.rc2.261.g7f71774620-goog

Re: [PATCH] Makefile: LTO: have linker check -Wframe-larger-than

[Nick Desaulniers]

On Thu, Mar 11, 2021 at 5:09 PM Nick Desaulniers
<ndesaulniers@google.com> wrote:
>
> -Wframe-larger-than= requires stack frame information, which the
> frontend cannot provide. This diagnostic is emitted late during
> compilation once stack frame size is available.
>
> When building with LTO, the frontend simply lowers C to LLVM IR and does
> not have stack frame information, so it cannot emit this diagnostic.
> When the linker drives LTO, it restarts optimizations and lowers LLVM IR
> to object code. At that point, it has stack frame information but
> doesn't know to check for a specific max stack frame size.
>
> I consider this a bug in LLVM that we need to fix. There are some
> details we're working out related to LTO such as which value to use when
> there are multiple different values specified per TU, or how to
> propagate these to compiler synthesized routines properly, if at all.
>
> Until it's fixed, ensure we don't miss these. At that point we can wrap
> this in a compiler version guard or revert this based on the minimum
> support version of Clang.
>
> The error message is not generated during link:
>   LTO     vmlinux.o
> ld.lld: warning: stack size limit exceeded (8224) in foobarbaz
>
> Cc: Sami Tolvanen <samitolvanen@google.com>
> Reported-by: Candle Sun <candlesea@gmail.com>
> Suggested-by: Fangrui Song <maskray@google.com>
> Signed-off-by: Nick Desaulniers <ndesaulniers@google.com>
> ---
> LTO users might want to `make clean` or `rm -rf .thinlto-cache` to test
> this.
>
>  Makefile | 5 +++++
>  1 file changed, 5 insertions(+)
>
> diff --git a/Makefile b/Makefile
> index f9b54da2fca0..74566b1417b8 100644
> --- a/Makefile
> +++ b/Makefile

Candle sent me a private message that we probably also want coverage
for kernel modules. Let me revise this and test/send a v2.

> @@ -910,6 +910,11 @@ CC_FLAGS_LTO       += -fvisibility=hidden
>
>  # Limit inlining across translation units to reduce binary size
>  KBUILD_LDFLAGS += -mllvm -import-instr-limit=5
> +
> +# Check for frame size exceeding threshold during prolog/epilog insertion.
> +ifneq ($(CONFIG_FRAME_WARN),0)
> +KBUILD_LDFLAGS += -plugin-opt=-warn-stack-size=$(CONFIG_FRAME_WARN)
> +endif
>  endif
>
>  ifdef CONFIG_LTO
> --
> 2.31.0.rc2.261.g7f71774620-goog
>


-- 
Thanks,
~Nick Desaulniers

Re: [PATCH] Makefile: LTO: have linker check -Wframe-larger-than

[Nick Desaulniers]

On Fri, Mar 12, 2021 at 9:55 AM Nick Desaulniers
<ndesaulniers@google.com> wrote:
>
> On Thu, Mar 11, 2021 at 5:09 PM Nick Desaulniers
> <ndesaulniers@google.com> wrote:
> >
> > -Wframe-larger-than= requires stack frame information, which the
> > frontend cannot provide. This diagnostic is emitted late during
> > compilation once stack frame size is available.
> >
> > When building with LTO, the frontend simply lowers C to LLVM IR and does
> > not have stack frame information, so it cannot emit this diagnostic.
> > When the linker drives LTO, it restarts optimizations and lowers LLVM IR
> > to object code. At that point, it has stack frame information but
> > doesn't know to check for a specific max stack frame size.
> >
> > I consider this a bug in LLVM that we need to fix. There are some
> > details we're working out related to LTO such as which value to use when
> > there are multiple different values specified per TU, or how to
> > propagate these to compiler synthesized routines properly, if at all.
> >
> > Until it's fixed, ensure we don't miss these. At that point we can wrap
> > this in a compiler version guard or revert this based on the minimum
> > support version of Clang.
> >
> > The error message is not generated during link:
> >   LTO     vmlinux.o
> > ld.lld: warning: stack size limit exceeded (8224) in foobarbaz
> >
> > Cc: Sami Tolvanen <samitolvanen@google.com>
> > Reported-by: Candle Sun <candlesea@gmail.com>
> > Suggested-by: Fangrui Song <maskray@google.com>
> > Signed-off-by: Nick Desaulniers <ndesaulniers@google.com>
> > ---
> > LTO users might want to `make clean` or `rm -rf .thinlto-cache` to test
> > this.
> >
> >  Makefile | 5 +++++
> >  1 file changed, 5 insertions(+)
> >
> > diff --git a/Makefile b/Makefile
> > index f9b54da2fca0..74566b1417b8 100644
> > --- a/Makefile
> > +++ b/Makefile
>
> Candle sent me a private message that we probably also want coverage
> for kernel modules. Let me revise this and test/send a v2.

False alarm, seems specific to Android's LTO support pre-5.11. I will
fix that in Android trees.  This patch is still relevant going
forward.

>
> > @@ -910,6 +910,11 @@ CC_FLAGS_LTO       += -fvisibility=hidden
> >
> >  # Limit inlining across translation units to reduce binary size
> >  KBUILD_LDFLAGS += -mllvm -import-instr-limit=5
> > +
> > +# Check for frame size exceeding threshold during prolog/epilog insertion.
> > +ifneq ($(CONFIG_FRAME_WARN),0)
> > +KBUILD_LDFLAGS += -plugin-opt=-warn-stack-size=$(CONFIG_FRAME_WARN)
> > +endif
> >  endif
> >
> >  ifdef CONFIG_LTO
> > --
> > 2.31.0.rc2.261.g7f71774620-goog
> >
>
>
> --
> Thanks,
> ~Nick Desaulniers



-- 
Thanks,
~Nick Desaulniers

RE: [PATCH] Makefile: LTO: have linker check -Wframe-larger-than

[David Laight]

From: Nick Desaulniers
> Sent: 12 March 2021 17:55
> 
> On Thu, Mar 11, 2021 at 5:09 PM Nick Desaulniers
> <ndesaulniers@google.com> wrote:
> >
> > -Wframe-larger-than= requires stack frame information, which the
> > frontend cannot provide. This diagnostic is emitted late during
> > compilation once stack frame size is available.
> >
> > When building with LTO, the frontend simply lowers C to LLVM IR and does
> > not have stack frame information, so it cannot emit this diagnostic.
> > When the linker drives LTO, it restarts optimizations and lowers LLVM IR
> > to object code. At that point, it has stack frame information but
> > doesn't know to check for a specific max stack frame size.

With LTO the linker ought to be able to do a stack frame check
across multiples functions in the call stack.

Clearly recursive calls cause issues.
Indirect ones as well - but does CFI include enough info
about what can be called from where to help?

	David

-
Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK
Registration No: 1397386 (Wales)

Re: [PATCH] Makefile: LTO: have linker check -Wframe-larger-than

[Kees Cook]

On Thu, 11 Mar 2021 17:09:41 -0800, Nick Desaulniers wrote:
> -Wframe-larger-than= requires stack frame information, which the
> frontend cannot provide. This diagnostic is emitted late during
> compilation once stack frame size is available.
> 
> When building with LTO, the frontend simply lowers C to LLVM IR and does
> not have stack frame information, so it cannot emit this diagnostic.
> When the linker drives LTO, it restarts optimizations and lowers LLVM IR
> to object code. At that point, it has stack frame information but
> doesn't know to check for a specific max stack frame size.
> 
> [...]

Applied to for-linus/clang/features, thanks! This should be in -next
tomorrow and I'll send it on for -rc4 at the end of the week.

[1/1] Makefile: LTO: have linker check -Wframe-larger-than
      https://git.kernel.org/kees/c/24845dcb170e

-- 
Kees Cook