【For help】 configure crossbar build tool in CMakelist.txt

【For help】 configure crossbar build tool in CMakelist.txt

[oulijun]

Hi, all

   I am working for debugging userspace code  of hns in rdma-core.git.

  I have met a cmake question now.  please give a help for me.

  if I use crossbar build tool aarch64-linux-gnu-gcc for building the directory(provider/hns),  what i should do it ?

My modification currently according to the others as fllows:

in the file : provider/hns/CMakelist.txt

set(CMAKE_C_COMPILER /opt/gcc-linaro-aarch64-linux-gnu-4.9-2014.09_linux/bin/aarch64-linux-gnu-gcc)

but the modification is fail

-- The C compiler identification is GNU 4.8.4
-- Check for working C compiler using: Ninja
-- Check for working C compiler using: Ninja -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- cmake_share:
-- GNU:
-- BUILD_BIN_DIR:/home/ubuntu/rdma-core/build/bin
-- CMAKE_INSTALL: /usr/local/var/run
-- Found PkgConfig: /usr/bin/pkg-config (found version "0.26")
-- rdma_build: Debug;Release;RelWithDebInfo;MinSizeRel
-- BUILD_INCLUDE_DIR:/home/ubuntu/rdma-core/build/include
-- Performing Test SUPPORTS_GNU99
-- Performing Test SUPPORTS_GNU99 - Success
-- Performing Test HAVE_C_WARNINGS
-- Performing Test HAVE_C_WARNINGS - Success
-- Performing Test HAVE_C_WMISSING_PROTOTYPES
-- Performing Test HAVE_C_WMISSING_PROTOTYPES - Success
-- Performing Test HAVE_C_WMISSING_DECLARATIONS
-- Performing Test HAVE_C_WMISSING_DECLARATIONS - Success
-- Performing Test HAVE_C_WWRITE_STRINGS
-- Performing Test HAVE_C_WWRITE_STRINGS - Success
-- Performing Test HAVE_C_WFORMAT_2
-- Performing Test HAVE_C_WFORMAT_2 - Success
-- Performing Test HAVE_C_WORKING_SHADOW
-- Performing Test HAVE_C_WORKING_SHADOW - Success
-- Performing Test HAVE_C_WORKING_MISSING_FIELD_INITIALIZERS
-- Performing Test HAVE_C_WORKING_MISSING_FIELD_INITIALIZERS - Failed
-- Performing Test HAVE_C_WNO_MISSING_FIELD_INITIALIZERS
-- Performing Test HAVE_C_WNO_MISSING_FIELD_INITIALIZERS - Success
-- Performing Test HAVE_NO_STRICT_ALIASING
-- Performing Test HAVE_NO_STRICT_ALIASING - Success
-- Performing Test HAS_CLOEXEC
-- Performing Test HAS_CLOEXEC - Success
-- Performing Test HAVE_FUNC_ATTRIBUTE_ALWAYS_INLINE
-- Performing Test HAVE_FUNC_ATTRIBUTE_ALWAYS_INLINE - Success
-- Performing Test SUPPORTS_AS_NEEDED
-- Performing Test SUPPORTS_AS_NEEDED - Success
-- Performing Test SUPPORTS_NO_UNDEFINED
-- Performing Test SUPPORTS_NO_UNDEFINED - Success
-- Performing Test _LDSYMVER_SUCCESS
-- Performing Test _LDSYMVER_SUCCESS - Success
-- Found LDSymVer: GNU
-- Looking for include file pthread.h
-- Looking for include file pthread.h - found
-- Looking for pthread_create
-- Looking for pthread_create - not found
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE
-- checking for modules 'libnl-3.0;libnl-route-3.0'
--   found libnl-3.0, version 3.2.21
--   found libnl-route-3.0, version 3.2.21
-- Performing Test HAVE_WORKING_IF_H
-- Performing Test HAVE_WORKING_IF_H - Failed
-- Check size of long
-- Check size of long - done
-- Looking for rdma/rdma_user_rxe.h
-- Looking for rdma/rdma_user_rxe.h - not found
-- Looking for valgrind/memcheck.h
-- Looking for valgrind/memcheck.h - found
-- Looking for valgrind/drd.h
-- Looking for valgrind/drd.h - found
-- Performing Test LIBC_HAS_LIBRT
-- Performing Test LIBC_HAS_LIBRT - Success
-- Performing Test HAVE_C_WSTRICT_PROTOTYPES
-- Performing Test HAVE_C_WSTRICT_PROTOTYPES - Success
-- Performing Test HAVE_C_WOLD_STYLE_DEFINITION
-- Performing Test HAVE_C_WOLD_STYLE_DEFINITION - Success
-- Performing Test HAVE_C_WREDUNDANT_DECLS
-- Performing Test HAVE_C_WREDUNDANT_DECLS - Failed
-- c_complier: /opt/gcc-linaro-aarch64-linux-gnu-4.9-2014.09_linux/bin/aarch64-linux-gnu-gcc
-- Missing Optional Items:
--  netlink/route/link.h and net/if.h NOT co-includable (old headers)
--  rdma/rdma_user_rxe.h NOT found (old system kernel headers)
--  -Wmissing-field-initializers does NOT work
--  -Wredundant-decls does NOT work
-- Configuring done
-- Generating done
-- Build files have been written to: /home/ubuntu/rdma-core/build
[115/164] Building C object providers/hns/CMakeFiles/hns-rdmav2.dir/hns_roce_u_hw_v1.c.o
FAILED: /usr/bin/cc  -Dhns_rdmav2_EXPORTS -std=gnu99 -Wall -Wextra -Wno-sign-compare -Wno-unused-parameter -Wmissing-prototypes -Wmissing-declarations -Wwrite-strings -Wformat=2 -Wshadow -Wno-missing-field-initializers -Wstrict-prototypes -Wold-style-definition -O2 -g  -fPIC -Iinclude -I/usr/include/libnl3 -MMD -MT providers/hns/CMakeFiles/hns-rdmav2.dir/hns_roce_u_hw_v1.c.o -MF "providers/hns/CMakeFiles/hns-rdmav2.dir/hns_roce_u_hw_v1.c.o.d" -o providers/hns/CMakeFiles/hns-rdmav2.dir/hns_roce_u_hw_v1.c.o -c ../providers/hns/hns_roce_u_hw_v1.c
../providers/hns/hns_roce_u_hw_v1.c: Assembler messages:
../providers/hns/hns_roce_u_hw_v1.c:830: Error: number of operands mismatch for `ds'
../providers/hns/hns_roce_u_hw_v1.c:264: Error: number of operands mismatch for `ds'
../providers/hns/hns_roce_u_hw_v1.c:413: Error: number of operands mismatch for `ds'
../providers/hns/hns_roce_u_hw_v1.c:586: Error: number of operands mismatch for `ds'
../providers/hns/hns_roce_u_hw_v1.c:630: Error: number of operands mismatch for `ds'
[115/164] Building C object ibacm/CMakeFiles/ibacm.dir/src/acm.c.o
ninja: build stopped: subcommand failed.



.

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: ???For help??? configure crossbar build tool in CMakelist.txt

[Jason Gunthorpe]

On Tue, Oct 18, 2016 at 06:55:08PM +0800, oulijun wrote:

>   if I use crossbar build tool aarch64-linux-gnu-gcc for building
>   the directory(provider/hns), what i should do it ?

You cannot cross compile only a part of the project, you must cross
compile everything.

$ mkdir build; cd build
$ CC=aarch64-linux-gnu-gcc cmake -GNinja
$ ninja

I haven't extensively tested cross compiling, you may run into
problems, in particular I know pkgconfig for libnl doesn't work
reliably when cross compiling.

You may need to build with -DENABLE_RESOLVE_NEIGH=0 if you don't have
a cross compiled libnl3 available.

> My modification currently according to the others as fllows:
> 
> in the file : provider/hns/CMakelist.txt
> 
> set(CMAKE_C_COMPILER /opt/gcc-linaro-aarch64-linux-gnu-4.9-2014.09_linux/bin/aarch64-linux-gnu-gcc)

This is not the right approach, the hns provider CMakelist should
detect the compiler is not ARM64 and just do nothing.

Perhaps something like this:

CHECK_C_SOURCE_COMPILES("
#ifndef __ARM64__
#error Failed
#endif
 int main(int argc,const char *argv[]) { return 1; }"
 HAVE_ARCH_ARM64)

if (HAVE_ARCH_ARM64)
 [..]
endif()
 
Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

some test question//Re: [For help] configure crossbar build tool in CMakelist.txt

[oulijun]

Hi, Jason

Indeed, you are right! when use the cmd: CC= aarch64-linux-gnu-gcc cmake -GNinja ..

the libnl doesn't work

the print log as follows:

ubuntu@62fe1357a077:~/rdma-core/build$ ninja
[27/164] Linking C executable bin/iwpmd
FAILED: : && /opt/gcc-linaro-aarch64-linux-gnu-4.9-2014.09_linux/bin/aarch64-linux-gnu-gcc -std=gnu99 -Wall -Wextra -Wno-sign-compare -Wno-unused-parameter -Wmissing-prototypes -Wmissing-declarations -Wwrite-strings -Wformat=2 -Wshadow -Wno-missing-field-initializers -Wstrict-prototypes -Wold-style-definition -O2 -g -Wl,--as-needed -Wl,--no-undefined iwpmd/CMakeFiles/iwpmd.dir/iwarp_pm_common.c.o iwpmd/CMakeFiles/iwpmd.dir/iwarp_pm_helper.c.o iwpmd/CMakeFiles/iwpmd.dir/iwarp_pm_server.c.o -o bin/iwpmd ccan/libccan.a -lnl-route-3 -lnl-3 -lpthread && :
/opt/gcc-linaro-aarch64-linux-gnu-4.9-2014.09_linux/bin/../lib/gcc/aarch64-linux-gnu/4.9.2/../../../../aarch64-linux-gnu/bin/ld: cannot find -lnl-route-3
/opt/gcc-linaro-aarch64-linux-gnu-4.9-2014.09_linux/bin/../lib/gcc/aarch64-linux-gnu/4.9.2/../../../../aarch64-linux-gnu/bin/ld: cannot find -lnl-3
collect2: error: ld returned 1 exit status
[27/164] Building C object libibverbs/CMakeFiles/ibverbs.dir/cmd.c.o
ninja: build stopped: subcommand failed.


when use the cmd: CC=aarch64-linux-gnu-gcc cmake -GNinja -DENABLE_RESOLVE_NEIGH=0 ..

the provider/hns/CMakelist.txt can be called correctly and generate the library libhns-rdmav2.so(not be tested), but it will generate warning with mlx5

the print log as follows:

Building C object providers/mlx5/CMakeFiles/mlx5-rdmav2.dir/cq.c.o
../providers/mlx5/cq.c:414:13: warning: function declaration isn't a prototype [-Wstrict-prototypes]
static void mlx5_stall_poll_cq()
^
../providers/mlx5/cq.c: In function 'mlx5_stall_poll_cq':
../providers/mlx5/cq.c:414:13: warning: old-style function definition [-Wold-style-definition]

In summary, I have two questions:

1. if don't use libnl(use -DENABLE_RESOLVE_NEIGH=0), it will not confluent the function of library?

2. the rdma-core.git use the default build method by used build.sh, and i need modify it by added the building item or modify the README.md for hns, is the right approach?


Thanks

在 2016/10/18 23:50, Jason Gunthorpe 写道:
> On Tue, Oct 18, 2016 at 06:55:08PM +0800, oulijun wrote:
> 
>>   if I use crossbar build tool aarch64-linux-gnu-gcc for building
>>   the directory(provider/hns), what i should do it ?
> 
> You cannot cross compile only a part of the project, you must cross
> compile everything.
> 
> $ mkdir build; cd build
> $ CC=aarch64-linux-gnu-gcc cmake -GNinja
> $ ninja
> 
> I haven't extensively tested cross compiling, you may run into
> problems, in particular I know pkgconfig for libnl doesn't work
> reliably when cross compiling.
> 
> You may need to build with -DENABLE_RESOLVE_NEIGH=0 if you don't have
> a cross compiled libnl3 available.
> 
>> My modification currently according to the others as fllows:
>>
>> in the file : provider/hns/CMakelist.txt
>>
>> set(CMAKE_C_COMPILER /opt/gcc-linaro-aarch64-linux-gnu-4.9-2014.09_linux/bin/aarch64-linux-gnu-gcc)
> 
> This is not the right approach, the hns provider CMakelist should
> detect the compiler is not ARM64 and just do nothing.
> 
> Perhaps something like this:
> 
> CHECK_C_SOURCE_COMPILES("
> #ifndef __ARM64__
> #error Failed
> #endif
>  int main(int argc,const char *argv[]) { return 1; }"
>  HAVE_ARCH_ARM64)
> 
> if (HAVE_ARCH_ARM64)
>  [..]
> endif()
>  
> Jason
> --
> To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 
> .
> 


--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: some test question//Re: [For help] configure crossbar build tool in CMakelist.txt

[Jason Gunthorpe]

On Wed, Oct 19, 2016 at 10:53:34AM +0800, oulijun wrote:

> when use the cmd: CC=aarch64-linux-gnu-gcc cmake -GNinja -DENABLE_RESOLVE_NEIGH=0 ..
> 
> the provider/hns/CMakelist.txt can be called correctly and generate
> the library libhns-rdmav2.so(not be tested), but it will generate
> warning with mlx5
> 
> the print log as follows:
> 
> Building C object providers/mlx5/CMakeFiles/mlx5-rdmav2.dir/cq.c.o
> ../providers/mlx5/cq.c:414:13: warning: function declaration isn't a prototype [-Wstrict-prototypes]
> static void mlx5_stall_poll_cq()

Thanks, you may be the first person to compile on ARM64, so I'm glad
to hear that is your only warning, I will get it fixed.

> In summary, I have two questions:
> 
> 1. if don't use libnl(use -DENABLE_RESOLVE_NEIGH=0), it will not
> confluent the function of library?

Correct, RoCEE will not work entirely as expected if you try and run
the libibverbs from such a build.

> 2. the rdma-core.git use the default build method by used build.sh,
> and i need modify it by added the building item or modify the
> README.md for hns, is the right approach?

No.

If you native compile on an actual ARM64 machine you will not have
problems, simply follow the README.md directions to install the
required packages on the actual machine.

Your problem is cross compiling. Cross compiling is hard. You need to
make libnl3 available in your cross compiler's 'sysroot'. I can't
really give you generic good directions for that..

One option is to cross compile libnl and 'cross-install' it into the
compiler tree.

Another option is to download the libnl3 ARM64 packages from Debian and
then unpack and rename directories 'just so' to make it work.

But if you don't intend to run the build binaries (eg when you go to
test, you will build natively on ARM64), then don't worry about it.

Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[Test fail]//Re: some test question//Re: [For help] configure crossbar build tool in CMakelist.txt

[oulijun]

Hi, Jason
   I am working for test the userspace code of hns in rdma-core.git with your guide.
   I have met some quesiton for debugging in my environment.
   after add some print lines in load_driver() in libibverbs/init.c, as follows:

   printf("[%s, %d] %s, %d\n", __func__, __LINE__, VERBS_PROVIDER_DIR, sizeof(VERBS_PROVIDER_DIR));
        /* If configured with a provider plugin path then try that next */
        if (sizeof(VERBS_PROVIDER_DIR) >= 1) {
                if (asprintf(&so_name, VERBS_PROVIDER_DIR "/lib%s" DLOPEN_TRAILER, name) <
                    0)
                        goto out_asprintf;
                printf("[%s, %d] so_name: %s\n", __func__, __LINE__, so_name);// @1
                printf("[%s, %d] so_name: %s\n", __func__, __LINE__, DLOPEN_TRAILER);//
                dlhandle = dlopen(so_name, RTLD_NOW);
                free(so_name);
                if (dlhandle)
                        return;
        }
        /* Otherwise use the system libary search path. This is the historical
           behavior of libibverbs */
        if (asprintf(&so_name, "lib%s" DLOPEN_TRAILER, name) < 0)
                goto out_asprintf;
        printf("[%s, %d] so_name: %s\n", __func__, __LINE__, so_name);//@2
        printf("[%s, %d] so_name: %s\n", __func__, __LINE__, DLOPEN_TRAILER);//

    using the cmd:
	CC=aarch64-linux-gnu-gcc cmake -GNinja -DENABLE_RESOLVE_NEIGH=0 -DVERBS_PROVIDER_DIR='' ..
	ninja

   after obtained the libhisi-rdmav2.so and hisi.driver, i put them into the /lib and /etc/libibverbs.d/
in my test environment.

Now, it is test fail. some question are not understanded.
1. when set the VERBS_PROVIDER_DIR to empty, the sizeof(VERBS_PROVIDER_DIR) should be 0 and the branch should not be run
2. the value of so_name should be /libhisi-rdmav2.so in @1 and libhisi-rdmav2.so in @2.  in fact, the value of
so_name is /libhisi and libhisi

the test print log as follows:

-rdmav2.soer, 218] so_name: /libhisi
[load_driver, 219] so_name: -rdmav2.so
-rdmav2.soer, 229] so_name: libhisi
[load_driver, 230] so_name: -rdmav2.so

在 2016/10/19 12:11, Jason Gunthorpe 写道:
> On Wed, Oct 19, 2016 at 10:53:34AM +0800, oulijun wrote:
> 
>> when use the cmd: CC=aarch64-linux-gnu-gcc cmake -GNinja -DENABLE_RESOLVE_NEIGH=0 ..
>>
>> the provider/hns/CMakelist.txt can be called correctly and generate
>> the library libhns-rdmav2.so(not be tested), but it will generate
>> warning with mlx5
>>
>> the print log as follows:
>>
>> Building C object providers/mlx5/CMakeFiles/mlx5-rdmav2.dir/cq.c.o
>> ../providers/mlx5/cq.c:414:13: warning: function declaration isn't a prototype [-Wstrict-prototypes]
>> static void mlx5_stall_poll_cq()
> 
> Thanks, you may be the first person to compile on ARM64, so I'm glad
> to hear that is your only warning, I will get it fixed.
> 
>> In summary, I have two questions:
>>
>> 1. if don't use libnl(use -DENABLE_RESOLVE_NEIGH=0), it will not
>> confluent the function of library?
> 
> Correct, RoCEE will not work entirely as expected if you try and run
> the libibverbs from such a build.
> 
>> 2. the rdma-core.git use the default build method by used build.sh,
>> and i need modify it by added the building item or modify the
>> README.md for hns, is the right approach?
> 
> No.
> 
> If you native compile on an actual ARM64 machine you will not have
> problems, simply follow the README.md directions to install the
> required packages on the actual machine.
> 
> Your problem is cross compiling. Cross compiling is hard. You need to
> make libnl3 available in your cross compiler's 'sysroot'. I can't
> really give you generic good directions for that..
> 
> One option is to cross compile libnl and 'cross-install' it into the
> compiler tree.
> 
> Another option is to download the libnl3 ARM64 packages from Debian and
> then unpack and rename directories 'just so' to make it work.
> 
> But if you don't intend to run the build binaries (eg when you go to
> test, you will build natively on ARM64), then don't worry about it.
> 
> Jason
> 
> .
> 


--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [Test fail]//Re: some test question//Re: [For help] configure crossbar build tool in CMakelist.txt

[Jason Gunthorpe]

On Thu, Oct 20, 2016 at 03:50:07PM +0800, oulijun wrote:
> 1. when set the VERBS_PROVIDER_DIR to empty, the
> sizeof(VERBS_PROVIDER_DIR) should be 0 and the branch should not be
> run

This is a mistake, sizeof('') is 1, so the if should be (> 1)' I will
fix it.

> 2. the value of so_name should be /libhisi-rdmav2.so in @1 and
> libhisi-rdmav2.so in @2.  in fact, the value of so_name is /libhisi
> and libhisi
> 
> the test print log as follows:
> 
> -rdmav2.soer, 218] so_name: /libhisi
> [load_driver, 219] so_name: -rdmav2.so
> -rdmav2.soer, 229] so_name: libhisi
> [load_driver, 230] so_name: -rdmav2.so

If you notice the -rdmav2.so has been placed at the start of the line,
this suggest you have a spurious '\r' character. This would come from
the .driver file.

Since you did not use the cmake install process you must have written
the .driver file yourself and used a DOS text editor. UNIX line
endings are required.

Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [Test fail]//Re: some test question//Re: [For help] configure crossbar build tool in CMakelist.txt

[oulijun]

在 2016/10/21 0:29, Jason Gunthorpe 写道:
> On Thu, Oct 20, 2016 at 03:50:07PM +0800, oulijun wrote:
>> 1. when set the VERBS_PROVIDER_DIR to empty, the
>> sizeof(VERBS_PROVIDER_DIR) should be 0 and the branch should not be
>> run
> 
> This is a mistake, sizeof('') is 1, so the if should be (> 1)' I will
> fix it.
> 
>> 2. the value of so_name should be /libhisi-rdmav2.so in @1 and
>> libhisi-rdmav2.so in @2.  in fact, the value of so_name is /libhisi
>> and libhisi
>>
>> the test print log as follows:
>>
>> -rdmav2.soer, 218] so_name: /libhisi
>> [load_driver, 219] so_name: -rdmav2.so
>> -rdmav2.soer, 229] so_name: libhisi
>> [load_driver, 230] so_name: -rdmav2.so
> 
> If you notice the -rdmav2.so has been placed at the start of the line,
> this suggest you have a spurious '\r' character. This would come from
> the .driver file.
> 
> Since you did not use the cmake install process you must have written
> the .driver file yourself and used a DOS text editor. UNIX line
> endings are required.
> 
> Jason
> 
Thanks, i see it.
> .
> 


--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html