iw 5.0.1 broken when compiled with lto

iw 5.0.1 broken when compiled with lto

[Hauke Mehrtens]

Hi Johannes,

When I compile iw 5.0.1 with link time optimization for MIPS32r2 big
endian all commands are removed, it looks like this:
------------------------------------------------------------------------
root@OpenWrt:/# iw
Usage:  iw [options] command
Options:
        --debug         enable netlink debugging
        --version       show version (5.0.1)
Commands:

Commands that use the netdev ('dev') can also be given the
'wdev' instead to identify the device.

You can omit the 'phy' or 'dev' if the identification is unique,
e.g. "iw wlan0 info" or "iw phy0 info". (Don't when scripting.)

Do NOT screenscrape this tool, we don't consider its output stable.

root@OpenWrt:/#
------------------------------------------------------------------------

When I compile it without LTO it works like expected.
This was seen in OpenWrt in the iw-full version and when I remove all
the OpenWrt specific patches to iw. The iw-tiny version from OpenWrt
works like expected. iw 4.14 also works in both versions.

This was report to OpenWrt here:
https://github.com/openwrt/openwrt/pull/1865
http://lists.infradead.org/pipermail/openwrt-devel/2018-October/014220.html

With GCC 8.2 this also happens with iw 4.14.


The binary with lto is even bigger and they have the same number of
functions starting with __cmd:

$ ls -al iw-full-*
-rwxr-xr-x 1 hauke hauke 180164 Feb 26 21:55 iw-full-lto
-rwxr-xr-x 1 hauke hauke 179016 Feb 26 21:57 iw-full-wo-lto
$ nm iw-full-lto |fgrep __cmd |wc
     96     288    7259
$ nm iw-full-wo-lto |fgrep __cmd |wc
     96     288    7259

Hauke

Re: iw 5.0.1 broken when compiled with lto

[Hauke Mehrtens]

On 2/26/19 9:59 PM, Hauke Mehrtens wrote:
> Hi Johannes,
> 
> When I compile iw 5.0.1 with link time optimization for MIPS32r2 big
> endian all commands are removed, it looks like this:
> ------------------------------------------------------------------------
> root@OpenWrt:/# iw
> Usage:  iw [options] command
> Options:
>         --debug         enable netlink debugging
>         --version       show version (5.0.1)
> Commands:
> 
> Commands that use the netdev ('dev') can also be given the
> 'wdev' instead to identify the device.
> 
> You can omit the 'phy' or 'dev' if the identification is unique,
> e.g. "iw wlan0 info" or "iw phy0 info". (Don't when scripting.)
> 
> Do NOT screenscrape this tool, we don't consider its output stable.
> 
> root@OpenWrt:/#
> ------------------------------------------------------------------------
> 
> When I compile it without LTO it works like expected.
> This was seen in OpenWrt in the iw-full version and when I remove all
> the OpenWrt specific patches to iw. The iw-tiny version from OpenWrt
> works like expected. iw 4.14 also works in both versions.
> 
> This was report to OpenWrt here:
> https://github.com/openwrt/openwrt/pull/1865
> http://lists.infradead.org/pipermail/openwrt-devel/2018-October/014220.html
> 
> With GCC 8.2 this also happens with iw 4.14.
> 
> 
> The binary with lto is even bigger and they have the same number of
> functions starting with __cmd:
> 
> $ ls -al iw-full-*
> -rwxr-xr-x 1 hauke hauke 180164 Feb 26 21:55 iw-full-lto
> -rwxr-xr-x 1 hauke hauke 179016 Feb 26 21:57 iw-full-wo-lto
> $ nm iw-full-lto |fgrep __cmd |wc
>      96     288    7259
> $ nm iw-full-wo-lto |fgrep __cmd |wc
>      96     288    7259
> 
> Hauke

Paul Fertser did some further analysis of this problem and we can now
reproduce it also on normal x86 systems when the binary is compiled like
this:
CFLAGS=" -Os -flto" LDFLAGS=" -flto" make

-O3 or -Os and link time optimization is needed to see this problem.

With LTO the compiler randomly reorders the __section_* variables
including __section_set and __section_get and that leads to wrong offset
used to traverse the __cmd section.

Hauke

Re: iw 5.0.1 broken when compiled with lto

[Johannes Berg]

On Tue, 2019-02-26 at 23:34 +0100, Hauke Mehrtens wrote:
> 
> 
> Paul Fertser did some further analysis of this problem and we can now
> reproduce it also on normal x86 systems when the binary is compiled like
> this:
> CFLAGS=" -Os -flto" LDFLAGS=" -flto" make
> 
> -O3 or -Os and link time optimization is needed to see this problem.
> 
> With LTO the compiler randomly reorders the __section_* variables
> including __section_set and __section_get and that leads to wrong offset
> used to traverse the __cmd section.

Yes, looks like stuff going into the __cmd section is no longer
guaranteed to be in the same order as in the files, which sort of makes
sense. Therefore, the trick I used to determine how densely the linker
packs this no longer works.

However, we can use the same trick with a different section, so I've
just committed a fix for this.

johannes