Crosscompiling iproute2

Crosscompiling iproute2

[Frank Wunderlich]

Hi,

i want to crosscompile (a modified version of) iproute2 (git://git.kernel.org/pub/scm/network/iproute2/iproute2.git) for armhf

do you any idea how?
configure-script seems to ignore "--host=arm-linux-gnueabihf" like i'm using for nftables

i modified Makefile

-CC := gcc
+CC := arm-linux-gnueabihf-gcc

-SUBDIRS=lib ip tc bridge misc netem genl tipc devlink rdma dcb man vdpa
+#SUBDIRS=lib ip tc bridge misc netem genl tipc devlink rdma dcb man vdpa
+SUBDIRS=ip

and run make like this to use static linking:

make LDFLAGS=-static

but it seems ip always needs libutil

make[1]: *** No rule to make target '../lib/libutil.a', needed by 'ip'.  Stop.

if i include lib in SUBDIRS i get many errors about missing libs like selinux and mnl

regards Frank

Re: Crosscompiling iproute2

[Stephen Hemminger]

On Sun, 16 May 2021 16:51:59 +0200
Frank Wunderlich <frank-w@public-files.de> wrote:

> Hi,
> 
> i want to crosscompile (a modified version of) iproute2 (git://git.kernel.org/pub/scm/network/iproute2/iproute2.git) for armhf
> 
> do you any idea how?
> configure-script seems to ignore "--host=arm-linux-gnueabihf" like i'm using for nftables
> 
> i modified Makefile
> 
> -CC := gcc
> +CC := arm-linux-gnueabihf-gcc
> 
> -SUBDIRS=lib ip tc bridge misc netem genl tipc devlink rdma dcb man vdpa
> +#SUBDIRS=lib ip tc bridge misc netem genl tipc devlink rdma dcb man vdpa
> +SUBDIRS=ip
> 
> and run make like this to use static linking:
> 
> make LDFLAGS=-static
> 
> but it seems ip always needs libutil
> 
> make[1]: *** No rule to make target '../lib/libutil.a', needed by 'ip'.  Stop.
> 
> if i include lib in SUBDIRS i get many errors about missing libs like selinux and mnl
> 
> regards Frank

It is possible to mostly do a cross build if you do:

$ make CC="$CC" LD="$LD"
There are issues with netem local table generation

Aw: Re: Crosscompiling iproute2

[Frank Wunderlich]

> Gesendet: Sonntag, 16. Mai 2021 um 23:17 Uhr
> Von: "Stephen Hemminger" <stephen@networkplumber.org>

> It is possible to mostly do a cross build if you do:
>
> $ make CC="$CC" LD="$LD"
> There are issues with netem local table generation

Hi,

thank you for your answer, but with this way i got this:

./normal > normal.dist
./normal: error while loading shared libraries: libm.so.6: cannot open shared object file: No such file or directory

i guess it's the netem issue you've mentioned, imho i cannot install armhf-libs on ubuntu, so i disabled subdirs in Makefile beginning with netem

-SUBDIRS=lib ip tc bridge misc netem genl tipc devlink rdma dcb man vdpa
+SUBDIRS=lib ip tc bridge misc
+#netem genl tipc devlink rdma dcb man vdpa

it seems to did it now

$ file ip/ip
ip/ip: ELF 32-bit LSB executable, ARM, EABI5 version 1 (GNU/Linux), statically linked, BuildID[sha1]=b36e094bc8681713d91ffe3a085ad4d3c6a1c5ea, for GNU/Linux 3.2.0, not stripped

thank you

regards Frank

Re: Crosscompiling iproute2

[Stephen Hemminger]

On Mon, 17 May 2021 09:44:21 +0200
Frank Wunderlich <frank-w@public-files.de> wrote:

> > Gesendet: Sonntag, 16. Mai 2021 um 23:17 Uhr
> > Von: "Stephen Hemminger" <stephen@networkplumber.org>  
> 
> > It is possible to mostly do a cross build if you do:
> >
> > $ make CC="$CC" LD="$LD"
> > There are issues with netem local table generation  
> 
> Hi,
> 
> thank you for your answer, but with this way i got this:
> 
> ./normal > normal.dist
> ./normal: error while loading shared libraries: libm.so.6: cannot open shared object file: No such file or directory
> 
> i guess it's the netem issue you've mentioned, imho i cannot install armhf-libs on ubuntu, so i disabled subdirs in Makefile beginning with netem
> 
> -SUBDIRS=lib ip tc bridge misc netem genl tipc devlink rdma dcb man vdpa
> +SUBDIRS=lib ip tc bridge misc
> +#netem genl tipc devlink rdma dcb man vdpa
> 
> it seems to did it now
> 
> $ file ip/ip
> ip/ip: ELF 32-bit LSB executable, ARM, EABI5 version 1 (GNU/Linux), statically linked, BuildID[sha1]=b36e094bc8681713d91ffe3a085ad4d3c6a1c5ea, for GNU/Linux 3.2.0, not stripped
> 
> thank you
> 
> regards Frank

Cross compile needs to know the compiler for building non-cross tools as well.
This works for me:

make CC="$CC" LD="$LD" HOSTCC=gcc

Aw: Re: Crosscompiling iproute2

[Frank Wunderlich]

> Gesendet: Montag, 17. Mai 2021 um 21:36 Uhr
> Von: "Stephen Hemminger" <stephen@networkplumber.org>
> An: "Frank Wunderlich" <frank-w@public-files.de>
> Cc: netdev@vger.kernel.org
> Betreff: Re: Crosscompiling iproute2
> Cross compile needs to know the compiler for building non-cross tools as well.
> This works for me:
>
> make CC="$CC" LD="$LD" HOSTCC=gcc

Thanks, works for me too, also with dynamic linking (without "LDFLAGS=-static")

CROSS_COMPILE=arm-linux-gnueabihf-
make -j8 CC="${CROSS_COMPILE}gcc" LD="${CROSS_COMPILE}-ld" HOSTCC=gcc

only see warning

libnetlink.c:154:2: warning: #warning "libmnl required for error support" [-Wcpp]

but i guess i can ignore it

regards Frank

Re: Crosscompiling iproute2

[Frank Wunderlich]

Am 17. Mai 2021 21:36:28 MESZ schrieb Stephen Hemminger <stephen@networkplumber.org>:
>On Mon, 17 May 2021 09:44:21 +0200
>This works for me:
>
>make CC="$CC" LD="$LD" HOSTCC=gcc

Hi,

Currently have an issue i guess from install. After compile i install into local directory,pack it and unpack on target system (/usr/local/sbin).tried

https://github.com/frank-w/iproute2/blob/main/crosscompile.sh#L17

Basic ip commands work,but if i try e.g. this

ip link add name lanbr0 type bridge vlan_filtering 1 vlan_default_pvid 500

I get this:

Garbage instead of arguments "vlan_filtering ...". Try "ip link help".

I guess ip tries to call bridge binary from wrong path (tried $PRFX/usr/local/bin).

regards Frank

Re: Crosscompiling iproute2

[Stephen Hemminger]

On Mon, 24 May 2021 21:06:02 +0200
Frank Wunderlich <frank-w@public-files.de> wrote:

> Am 17. Mai 2021 21:36:28 MESZ schrieb Stephen Hemminger <stephen@networkplumber.org>:
> >On Mon, 17 May 2021 09:44:21 +0200
> >This works for me:
> >
> >make CC="$CC" LD="$LD" HOSTCC=gcc  
> 
> Hi,
> 
> Currently have an issue i guess from install. After compile i install into local directory,pack it and unpack on target system (/usr/local/sbin).tried
> 
> https://github.com/frank-w/iproute2/blob/main/crosscompile.sh#L17

> 
> Basic ip commands work,but if i try e.g. this
> 
> ip link add name lanbr0 type bridge vlan_filtering 1 vlan_default_pvid 500
> 
> I get this:
> 
> Garbage instead of arguments "vlan_filtering ...". Try "ip link help".
> 
> I guess ip tries to call bridge binary from wrong path (tried $PRFX/usr/local/bin).
> 
> regards Frank

No ip command does not call bridge.

More likely either your kernel is out of date with the ip command (ie new ip command is asking for
something kernel doesn't understand); or the iplink_bridge.c was not compiled as part of your compile;
or simple PATH issue
or your system is not handling dlopen(NULL) correctly.

What happens is that the "type" field in ip link triggers the code
to use dlopen as form of introspection (see get_link_kind)

Re: Crosscompiling iproute2

[Frank Wunderlich]

Am 24. Mai 2021 23:36:20 MESZ schrieb Stephen Hemminger <stephen@networkplumber.org>:
>On Mon, 24 May 2021 21:06:02 +0200
>Frank Wunderlich <frank-w@public-files.de> wrote:
>
>> Am 17. Mai 2021 21:36:28 MESZ schrieb Stephen Hemminger
><stephen@networkplumber.org>:
>> >On Mon, 17 May 2021 09:44:21 +0200
>> >This works for me:
>> >
>> >make CC="$CC" LD="$LD" HOSTCC=gcc  
>> 
>> Hi,
>> 
>> Currently have an issue i guess from install. After compile i install
>into local directory,pack it and unpack on target system
>(/usr/local/sbin).tried
>> 
>> https://github.com/frank-w/iproute2/blob/main/crosscompile.sh#L17
>
>> 
>> Basic ip commands work,but if i try e.g. this
>> 
>> ip link add name lanbr0 type bridge vlan_filtering 1
>vlan_default_pvid 500
>> 
>> I get this:
>> 
>> Garbage instead of arguments "vlan_filtering ...". Try "ip link
>help".
>> 
>> I guess ip tries to call bridge binary from wrong path (tried
>$PRFX/usr/local/bin).
>> 
>> regards Frank
>
>No ip command does not call bridge.
>
>More likely either your kernel is out of date with the ip command (ie
>new ip command is asking for
>something kernel doesn't understand);
I use 5.13-rc2 and can use the same command with debians ip command

>or the iplink_bridge.c was not
>compiled as part of your compile;
>or simple PATH issue
>or your system is not handling dlopen(NULL) correctly.

Which lib does ip load when using the vlanfiltering option?

>What happens is that the "type" field in ip link triggers the code
>to use dlopen as form of introspection (see get_link_kind)

I can use the command without vlan_filtering option (including type bridge).

Maybe missing libnml while compile can cause this? had disabled in config.mk and was not reset by make clean,manual delete causes build error,see my last mail

You can crosscompile only with CC,LD and HOSTCC set?

regards Frank

Re: Crosscompiling iproute2

[Stephen Hemminger]

On Tue, 25 May 2021 17:56:09 +0200
Frank Wunderlich <frank-w@public-files.de> wrote:

> Am 24. Mai 2021 23:36:20 MESZ schrieb Stephen Hemminger <stephen@networkplumber.org>:
> >On Mon, 24 May 2021 21:06:02 +0200
> >Frank Wunderlich <frank-w@public-files.de> wrote:
> >  
> >> Am 17. Mai 2021 21:36:28 MESZ schrieb Stephen Hemminger  
> ><stephen@networkplumber.org>:  
> >> >On Mon, 17 May 2021 09:44:21 +0200
> >> >This works for me:
> >> >
> >> >make CC="$CC" LD="$LD" HOSTCC=gcc    
> >> 
> >> Hi,
> >> 
> >> Currently have an issue i guess from install. After compile i install  
> >into local directory,pack it and unpack on target system
> >(/usr/local/sbin).tried  
> >> 
> >> https://github.com/frank-w/iproute2/blob/main/crosscompile.sh#L17  
> >  
> >> 
> >> Basic ip commands work,but if i try e.g. this
> >> 
> >> ip link add name lanbr0 type bridge vlan_filtering 1  
> >vlan_default_pvid 500  
> >> 
> >> I get this:
> >> 
> >> Garbage instead of arguments "vlan_filtering ...". Try "ip link  
> >help".  
> >> 
> >> I guess ip tries to call bridge binary from wrong path (tried  
> >$PRFX/usr/local/bin).  
> >> 
> >> regards Frank  
> >
> >No ip command does not call bridge.
> >
> >More likely either your kernel is out of date with the ip command (ie
> >new ip command is asking for
> >something kernel doesn't understand);  
> I use 5.13-rc2 and can use the same command with debians ip command
> 
> >or the iplink_bridge.c was not
> >compiled as part of your compile;
> >or simple PATH issue
> >or your system is not handling dlopen(NULL) correctly.  
> 
> Which lib does ip load when using the vlanfiltering option?
It is doing dlopen of itself, no other library

> 
> >What happens is that the "type" field in ip link triggers the code
> >to use dlopen as form of introspection (see get_link_kind)  
> 
> I can use the command without vlan_filtering option (including type bridge).
> 
> Maybe missing libnml while compile can cause this? had disabled in config.mk and was not reset by make clean,manual delete causes build error,see my last mail
> 
> You can crosscompile only with CC,LD and HOSTCC set?

libmnl is needed to get the error handling and a few other features.

Aw: Re: Crosscompiling iproute2

[Frank Wunderlich]

> Gesendet: Dienstag, 25. Mai 2021 um 18:08 Uhr
> Von: "Stephen Hemminger" <stephen@networkplumber.org>
> An: "Frank Wunderlich" <frank-w@public-files.de>
> Cc: netdev@vger.kernel.org
> Betreff: Re: Crosscompiling iproute2
>
> On Tue, 25 May 2021 17:56:09 +0200
> Frank Wunderlich <frank-w@public-files.de> wrote:
>
> > Am 24. Mai 2021 23:36:20 MESZ schrieb Stephen Hemminger <stephen@networkplumber.org>:
> > >On Mon, 24 May 2021 21:06:02 +0200
> > >Frank Wunderlich <frank-w@public-files.de> wrote:
> > >
> > >> Am 17. Mai 2021 21:36:28 MESZ schrieb Stephen Hemminger
> > ><stephen@networkplumber.org>:
> > >> >On Mon, 17 May 2021 09:44:21 +0200
> > >> >This works for me:
> > >> >
> > >> >make CC="$CC" LD="$LD" HOSTCC=gcc
> > >>
> > >> Hi,
> > >>
> > >> Currently have an issue i guess from install. After compile i install
> > >into local directory,pack it and unpack on target system
> > >(/usr/local/sbin).tried
> > >>
> > >> https://github.com/frank-w/iproute2/blob/main/crosscompile.sh#L17
> > >
> > >>
> > >> Basic ip commands work,but if i try e.g. this
> > >>
> > >> ip link add name lanbr0 type bridge vlan_filtering 1
> > >vlan_default_pvid 500
> > >>
> > >> I get this:
> > >>
> > >> Garbage instead of arguments "vlan_filtering ...". Try "ip link
> > >help".
> > >>
> > >> I guess ip tries to call bridge binary from wrong path (tried
> > >$PRFX/usr/local/bin).
> > >>
> > >> regards Frank
> > >
> > >No ip command does not call bridge.
> > >
> > >More likely either your kernel is out of date with the ip command (ie
> > >new ip command is asking for
> > >something kernel doesn't understand);
> > I use 5.13-rc2 and can use the same command with debians ip command
> >
> > >or the iplink_bridge.c was not
> > >compiled as part of your compile;
> > >or simple PATH issue
> > >or your system is not handling dlopen(NULL) correctly.
> >
> > Which lib does ip load when using the vlanfiltering option?
> It is doing dlopen of itself, no other library
>
> >
> > >What happens is that the "type" field in ip link triggers the code
> > >to use dlopen as form of introspection (see get_link_kind)

this seems to be the problem:

openat(AT_FDCWD, "/usr/lib/ip/link_bridge.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
write(2, "Garbage instead of arguments \"vl"..., 71Garbage instead of arguments "vlan_filtering ...". Try "ip link help".

i have no /usr/lib/ip directory, my package contains only lib-folder for tc (with dist files only because i use static linking). also there is no *.so in my building-directory

how should this built?

> > I can use the command without vlan_filtering option (including type bridge).
> >
> > Maybe missing libnml while compile can cause this? had disabled in config.mk and was not reset by make clean,manual delete causes build error,see my last mail
> >
> > You can crosscompile only with CC,LD and HOSTCC set?
>
> libmnl is needed to get the error handling and a few other features.

so not needed for compilation, right? on target-system i have it

/usr/local/lib/aarch64-linux-gnu:
        libmnl.so.0 -> libmnl.so.0.2.0

regards Frank

Re: Crosscompiling iproute2

[Stephen Hemminger]

On Tue, 25 May 2021 20:18:39 +0200
Frank Wunderlich <frank-w@public-files.de> wrote:

> > Gesendet: Dienstag, 25. Mai 2021 um 18:08 Uhr
> > Von: "Stephen Hemminger" <stephen@networkplumber.org>
> > An: "Frank Wunderlich" <frank-w@public-files.de>
> > Cc: netdev@vger.kernel.org
> > Betreff: Re: Crosscompiling iproute2
> >
> > On Tue, 25 May 2021 17:56:09 +0200
> > Frank Wunderlich <frank-w@public-files.de> wrote:
> >  
> > > Am 24. Mai 2021 23:36:20 MESZ schrieb Stephen Hemminger <stephen@networkplumber.org>:  
> > > >On Mon, 24 May 2021 21:06:02 +0200
> > > >Frank Wunderlich <frank-w@public-files.de> wrote:
> > > >  
> > > >> Am 17. Mai 2021 21:36:28 MESZ schrieb Stephen Hemminger  
> > > ><stephen@networkplumber.org>:  
> > > >> >On Mon, 17 May 2021 09:44:21 +0200
> > > >> >This works for me:
> > > >> >
> > > >> >make CC="$CC" LD="$LD" HOSTCC=gcc  
> > > >>
> > > >> Hi,
> > > >>
> > > >> Currently have an issue i guess from install. After compile i install  
> > > >into local directory,pack it and unpack on target system
> > > >(/usr/local/sbin).tried  
> > > >>
> > > >> https://github.com/frank-w/iproute2/blob/main/crosscompile.sh#L17  
> > > >  
> > > >>
> > > >> Basic ip commands work,but if i try e.g. this
> > > >>
> > > >> ip link add name lanbr0 type bridge vlan_filtering 1  
> > > >vlan_default_pvid 500  
> > > >>
> > > >> I get this:
> > > >>
> > > >> Garbage instead of arguments "vlan_filtering ...". Try "ip link  
> > > >help".  
> > > >>
> > > >> I guess ip tries to call bridge binary from wrong path (tried  
> > > >$PRFX/usr/local/bin).  
> > > >>
> > > >> regards Frank  
> > > >
> > > >No ip command does not call bridge.
> > > >
> > > >More likely either your kernel is out of date with the ip command (ie
> > > >new ip command is asking for
> > > >something kernel doesn't understand);  
> > > I use 5.13-rc2 and can use the same command with debians ip command
> > >  
> > > >or the iplink_bridge.c was not
> > > >compiled as part of your compile;
> > > >or simple PATH issue
> > > >or your system is not handling dlopen(NULL) correctly.  
> > >
> > > Which lib does ip load when using the vlanfiltering option?  
> > It is doing dlopen of itself, no other library
> >  
> > >  
> > > >What happens is that the "type" field in ip link triggers the code
> > > >to use dlopen as form of introspection (see get_link_kind)  
> 
> this seems to be the problem:
> 
> openat(AT_FDCWD, "/usr/lib/ip/link_bridge.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
> write(2, "Garbage instead of arguments \"vl"..., 71Garbage instead of arguments "vlan_filtering ...". Try "ip link help".
> 
> i have no /usr/lib/ip directory, my package contains only lib-folder for tc (with dist files only because i use static linking). also there is no *.so in my building-directory

That only gets called if you haven't got the bridge part in the original ip command.
The shared library stuff is for other non-static libraries to extend iproute2.
This is unused by most distro's and you shouldn't need it.

I think the bridge part was just not built in your version.

Aw: Re: Crosscompiling iproute2

[Frank Wunderlich]

Hi,
> Gesendet: Dienstag, 25. Mai 2021 um 23:37 Uhr
> Von: "Stephen Hemminger" <stephen@networkplumber.org>

> That only gets called if you haven't got the bridge part in the original ip command.
> The shared library stuff is for other non-static libraries to extend iproute2.
> This is unused by most distro's and you shouldn't need it.
>
> I think the bridge part was just not built in your version.

i see the compilation of iplink_bridge.o before linking ip, so i guess it should be compiled in

    CC       iplink_bridge.o
    CC       iplink_bridge_slave.o

i wonder why it tries to use a lib which was not compiled....

but i still need to disable mnl and selinux after creating config.mk to avoid linking errors (now i see only warning about error reporting).

i updated the compile-script, so you can see what i'm doing

https://github.com/frank-w/iproute2/blob/main/crosscompile.sh

regards Frank

Aw: Re: Crosscompiling iproute2

[Frank Wunderlich]

> Currently have an issue i guess from install. After compile i install into local directory,pack it and unpack on target system (/usr/local/sbin).tried
>
> https://github.com/frank-w/iproute2/blob/main/crosscompile.sh#L17
>
> Basic ip commands work,but if i try e.g. this
>
> ip link add name lanbr0 type bridge vlan_filtering 1 vlan_default_pvid 500
>
> I get this:
>
> Garbage instead of arguments "vlan_filtering ...". Try "ip link help".
>
> I guess ip tries to call bridge binary from wrong path (tried $PRFX/usr/local/bin).

based on Makefile i tried this install line after building

make DESTDIR=$PRFX PREFIX=/usr/local SBINDIR=/usr/local/sbin ARPDDIR=/var/lib/arpd CONFDIR=/etc/iproute2 install

it installs to the local folder instead of system-directories, but error on target is still the same :(

same line with binaries from debian working, so i'm sure the command is right (and have depencies in linux kernel available), except there was any change in code causing this

any idea?