[PATCH] lto.inc: Add global LTO distro policy file

[PATCH] lto.inc: Add global LTO distro policy file

[Khem Raj]

Distros which want to enable LTO can utilize this file, it only covers
packages from OE-Core, other layers should include there own exclusion
list for recipe which dont work with LTO

Signed-off-by: Khem Raj <raj.khem@gmail.com>
---
 meta/conf/distro/include/lto.inc | 23 +++++++++++++++++++++++
 1 file changed, 23 insertions(+)
 create mode 100644 meta/conf/distro/include/lto.inc

diff --git a/meta/conf/distro/include/lto.inc b/meta/conf/distro/include/lto.inc
new file mode 100644
index 0000000000..0cedc8deca
--- /dev/null
+++ b/meta/conf/distro/include/lto.inc
@@ -0,0 +1,23 @@
+LTO_pn-glibc = ""
+LTO_pn-gcc-runtime = ""
+LTO_pn-libgcc-initial = ""
+LTO_pn-libgcc = ""
+LTO_pn-libpam = ""
+LTO_pn-elfutils = ""
+LTO_pn-perl = ""
+LTO_pn-busybox = ""
+LTO_pn-libxcrypt = ""
+LTO_pn-curl = ""
+LTO_pn-libcap = ""
+LTO_pn-python3 = ""
+LTO_pn-libproxy = ""
+LTO_pn-libbsd = ""
+
+LTO ?= "-flto"
+
+SELECTED_OPTIMIZATION_append_class-target = " ${LTO}"
+TARGET_LDFLAGS_append_class-target = " ${LTO}"
+
+SELECTED_OPTIMIZATION[vardeps] += "LTO"
+
+PACKAGE_DEBUG_SPLIT_STYLE = "debug-without-src"
-- 
2.25.2

Re: [OE-core] [PATCH] lto.inc: Add global LTO distro policy file

[Richard Purdie]

On Mon, 2020-03-23 at 16:25 -0700, Khem Raj wrote:
> Distros which want to enable LTO can utilize this file, it only
> covers
> packages from OE-Core, other layers should include there own
> exclusion
> list for recipe which dont work with LTO
> 
> Signed-off-by: Khem Raj <raj.khem@gmail.com>
> ---
>  meta/conf/distro/include/lto.inc | 23 +++++++++++++++++++++++
>  1 file changed, 23 insertions(+)
>  create mode 100644 meta/conf/distro/include/lto.inc
> 
> diff --git a/meta/conf/distro/include/lto.inc
> b/meta/conf/distro/include/lto.inc
> new file mode 100644
> index 0000000000..0cedc8deca
> --- /dev/null
> +++ b/meta/conf/distro/include/lto.inc
> @@ -0,0 +1,23 @@
> +LTO_pn-glibc = ""
> +LTO_pn-gcc-runtime = ""
> +LTO_pn-libgcc-initial = ""
> +LTO_pn-libgcc = ""
> +LTO_pn-libpam = ""
> +LTO_pn-elfutils = ""
> +LTO_pn-perl = ""
> +LTO_pn-busybox = ""
> +LTO_pn-libxcrypt = ""
> +LTO_pn-curl = ""
> +LTO_pn-libcap = ""
> +LTO_pn-python3 = ""
> +LTO_pn-libproxy = ""
> +LTO_pn-libbsd = ""
> +
> +LTO ?= "-flto"
> +
> +SELECTED_OPTIMIZATION_append_class-target = " ${LTO}"
> +TARGET_LDFLAGS_append_class-target = " ${LTO}"
> +
> +SELECTED_OPTIMIZATION[vardeps] += "LTO"
> +
> +PACKAGE_DEBUG_SPLIT_STYLE = "debug-without-src"

Why do we need to specify the debug split style here?

Cheers,

Richard

Re: [OE-core] [PATCH] lto.inc: Add global LTO distro policy file

[Adrian Bunk]

On Mon, Mar 23, 2020 at 04:25:29PM -0700, Khem Raj wrote:
> Distros which want to enable LTO can utilize this file, it only covers
> packages from OE-Core, other layers should include there own exclusion
> list for recipe which dont work with LTO
>...
> +++ b/meta/conf/distro/include/lto.inc
> @@ -0,0 +1,23 @@
> +LTO_pn-glibc = ""
> +LTO_pn-gcc-runtime = ""
> +LTO_pn-libgcc-initial = ""
> +LTO_pn-libgcc = ""
> +LTO_pn-libpam = ""
> +LTO_pn-elfutils = ""
> +LTO_pn-perl = ""
> +LTO_pn-busybox = ""
> +LTO_pn-libxcrypt = ""
> +LTO_pn-curl = ""
> +LTO_pn-libcap = ""
> +LTO_pn-python3 = ""
> +LTO_pn-libproxy = ""
> +LTO_pn-libbsd = ""
>...

It would be better to have these in the individual recipes,
it is much easier to understand what is going on when all
recipe-specific information is in the recipe.

Otherwise this will end as an implementation of workarounds that
might not even be needed anymore, for a previous example with
several workarounds that were no longer needed see
https://git.openembedded.org/openembedded-core/commit/?id=8bbb17d2dcafa9b4e26941a55932f4be2782e1d1

cu
Adrian

Re: [OE-core] [PATCH] lto.inc: Add global LTO distro policy file

[Adrian Bunk]

On Mon, Mar 23, 2020 at 04:25:29PM -0700, Khem Raj wrote:
>...
> +LTO_pn-python3 = ""
>...

Building Python with LTO works by passing --with-lto to configure.

cu
Adrian

Re: [OE-core] [PATCH] lto.inc: Add global LTO distro policy file

[Richard Purdie]

On Tue, 2020-03-24 at 13:27 +0200, Adrian Bunk wrote:
> On Mon, Mar 23, 2020 at 04:25:29PM -0700, Khem Raj wrote:
> > Distros which want to enable LTO can utilize this file, it only
> > covers
> > packages from OE-Core, other layers should include there own
> > exclusion
> > list for recipe which dont work with LTO
> > ...
> > +++ b/meta/conf/distro/include/lto.inc
> > @@ -0,0 +1,23 @@
> > +LTO_pn-glibc = ""
> > +LTO_pn-gcc-runtime = ""
> > +LTO_pn-libgcc-initial = ""
> > +LTO_pn-libgcc = ""
> > +LTO_pn-libpam = ""
> > +LTO_pn-elfutils = ""
> > +LTO_pn-perl = ""
> > +LTO_pn-busybox = ""
> > +LTO_pn-libxcrypt = ""
> > +LTO_pn-curl = ""
> > +LTO_pn-libcap = ""
> > +LTO_pn-python3 = ""
> > +LTO_pn-libproxy = ""
> > +LTO_pn-libbsd = ""
> > ...
> 
> It would be better to have these in the individual recipes,
> it is much easier to understand what is going on when all
> recipe-specific information is in the recipe.

I disagree. These files do let someone working on specific enablement
see the status collectively and allow people to opt in/out of a given
development feature.

At some point in the future that can tip into making more sense in a
recipe but its a staged process and things often make sense to start
like this.

Cheers,

Richard

Re: [OE-core] [PATCH] lto.inc: Add global LTO distro policy file

[Adrian Bunk]

On Mon, Mar 23, 2020 at 04:25:29PM -0700, Khem Raj wrote:
> Distros which want to enable LTO can utilize this file, it only covers
> packages from OE-Core, other layers should include there own exclusion
> list for recipe which dont work with LTO
>...

More and more upstreams already default to LTO in their build system.

What is the recommended way to get the correct configure options to 
disable LTO in all such recipes?

The option is obviously recipe-specific, but there has to be some
global define the recipe can check for disable/default/enable.

cu
Adrian

Re: [OE-core] [PATCH] lto.inc: Add global LTO distro policy file

[Khem Raj]

On Tue, Mar 24, 2020 at 1:37 AM Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
>
> On Mon, 2020-03-23 at 16:25 -0700, Khem Raj wrote:
> > Distros which want to enable LTO can utilize this file, it only
> > covers
> > packages from OE-Core, other layers should include there own
> > exclusion
> > list for recipe which dont work with LTO
> >
> > Signed-off-by: Khem Raj <raj.khem@gmail.com>
> > ---
> >  meta/conf/distro/include/lto.inc | 23 +++++++++++++++++++++++
> >  1 file changed, 23 insertions(+)
> >  create mode 100644 meta/conf/distro/include/lto.inc
> >
> > diff --git a/meta/conf/distro/include/lto.inc
> > b/meta/conf/distro/include/lto.inc
> > new file mode 100644
> > index 0000000000..0cedc8deca
> > --- /dev/null
> > +++ b/meta/conf/distro/include/lto.inc
> > @@ -0,0 +1,23 @@
> > +LTO_pn-glibc = ""
> > +LTO_pn-gcc-runtime = ""
> > +LTO_pn-libgcc-initial = ""
> > +LTO_pn-libgcc = ""
> > +LTO_pn-libpam = ""
> > +LTO_pn-elfutils = ""
> > +LTO_pn-perl = ""
> > +LTO_pn-busybox = ""
> > +LTO_pn-libxcrypt = ""
> > +LTO_pn-curl = ""
> > +LTO_pn-libcap = ""
> > +LTO_pn-python3 = ""
> > +LTO_pn-libproxy = ""
> > +LTO_pn-libbsd = ""
> > +
> > +LTO ?= "-flto"
> > +
> > +SELECTED_OPTIMIZATION_append_class-target = " ${LTO}"
> > +TARGET_LDFLAGS_append_class-target = " ${LTO}"
> > +
> > +SELECTED_OPTIMIZATION[vardeps] += "LTO"
> > +
> > +PACKAGE_DEBUG_SPLIT_STYLE = "debug-without-src"
>
> Why do we need to specify the debug split style here?
>

debug file generator does  not generate debugsrc packages with lto i
have not investigates in
detail but its due to the fact that .o files are not normal elf objs

> Cheers,
>
> Richard
>
>

Re: [OE-core] [PATCH] lto.inc: Add global LTO distro policy file

[Khem Raj]

On Tue, Mar 24, 2020 at 4:37 AM Adrian Bunk <bunk@stusta.de> wrote:
>
> On Mon, Mar 23, 2020 at 04:25:29PM -0700, Khem Raj wrote:
> >...
> > +LTO_pn-python3 = ""
> >...
>
> Building Python with LTO works by passing --with-lto to configure.

I think per recipe knobs I did not work on yet but perhaps worth
adding them in next release
or may be as packageconfig dependent on lto distro feature
>
> cu
> Adrian

Re: [OE-core] [PATCH] lto.inc: Add global LTO distro policy file

[Khem Raj]

On Tue, Mar 24, 2020 at 4:50 AM Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
>
> On Tue, 2020-03-24 at 13:27 +0200, Adrian Bunk wrote:
> > On Mon, Mar 23, 2020 at 04:25:29PM -0700, Khem Raj wrote:
> > > Distros which want to enable LTO can utilize this file, it only
> > > covers
> > > packages from OE-Core, other layers should include there own
> > > exclusion
> > > list for recipe which dont work with LTO
> > > ...
> > > +++ b/meta/conf/distro/include/lto.inc
> > > @@ -0,0 +1,23 @@
> > > +LTO_pn-glibc = ""
> > > +LTO_pn-gcc-runtime = ""
> > > +LTO_pn-libgcc-initial = ""
> > > +LTO_pn-libgcc = ""
> > > +LTO_pn-libpam = ""
> > > +LTO_pn-elfutils = ""
> > > +LTO_pn-perl = ""
> > > +LTO_pn-busybox = ""
> > > +LTO_pn-libxcrypt = ""
> > > +LTO_pn-curl = ""
> > > +LTO_pn-libcap = ""
> > > +LTO_pn-python3 = ""
> > > +LTO_pn-libproxy = ""
> > > +LTO_pn-libbsd = ""
> > > ...
> >
> > It would be better to have these in the individual recipes,
> > it is much easier to understand what is going on when all
> > recipe-specific information is in the recipe.
>
> I disagree. These files do let someone working on specific enablement
> see the status collectively and allow people to opt in/out of a given
> development feature.
>
> At some point in the future that can tip into making more sense in a
> recipe but its a staged process and things often make sense to start
> like this.

yeah. Maybe there should be a distro feature knob this inc file should
add which can be used by recipes
for an exception like adding configure options
>
> Cheers,
>
> Richard
>