* [meta-processor-sdk][PATCHv2 0/5] Pytorch GLOW AOT tool and demo example
@ 2019-08-23 13:58 Djordje Senicic
2019-08-23 13:58 ` [meta-processor-sdk][PATCHv2 1/5] mesa, llvm, meson: Update llvm to 8.0.1 plus define and use LLVM version globally Djordje Senicic
` (4 more replies)
0 siblings, 5 replies; 12+ messages in thread
From: Djordje Senicic @ 2019-08-23 13:58 UTC (permalink / raw)
To: meta-arago; +Cc: d-senicic1, Djordje Senicic
This is continuation of work from previous release that did not create nativesdk artifacts properly.
Pytorch GLOW xNN compiler allows Ahead-of-Time compilation of neural network models into CPU object file (for target architecture) using LLVM.
This requires LLVM compiler version 7 or later. Now, the 2020LTS LLVM candidate from oe-core master branch is imported, to avoid unnecessary future upgrades (LLVM8.0.1 version).
Also, there are several native tools that are part of GLOW project that need to be compiled using CLANG/LLVM compiler (from branch 8.0), that is separately pulled in for that
sole purpose (since oe-core does not include CLANG compiler).
Natively compiled tools are used for further building of nativesdk GLOW package which includes single AOT tool, image-classifier, that is deployed in linux-devkit.
End user could use this tool to compile its own xNN model into ARMv7 object file and link with its application. An example, demo-glow, is provided in top level Makefile and example-application.
Small, well known mnist xNN model and several test images, are included in the filesystem.
Djordje Senicic (5):
mesa,llvm,meson: Update llvm to 8.0.1 plus define and use LLVM version
globally
clang8-native: llvm and clang used only for compilation of native
tools for glow compiler
glow: Address devkit related RPATH issue to point to devkit shared
libraries instead of host native
demo-glow: Restore in top-level makefile examples after recipe cleanup
packagegroup-arago-tisdk: Include xNN glow based compiler tool into
devkit, and demo-glow example
recipes-apps/demo-glow/demo-glow.bb | 16 +-
recipes-apps/demo-glow/files/Makefile.build | 3 +-
...sdk-packagegroup-arago-tisdk-host.bbappend | 4 +
...group-arago-tisdk-addons-sdk-host.bbappend | 11 ++
...oup-arago-tisdk-addons-sdk-target.bbappend | 6 +
.../packagegroup-arago-tisdk-addons.bbappend | 7 +
...{llvm-glow_git.bb => clang8-native_git.bb} | 16 +-
...p-compilation-of-native-helper-tools.patch | 99 ++++++++++
recipes-devtools/glow/glow_git.bb | 62 ++++--
...ryInfo-Undefine-libc-functions-if-th.patch | 92 +++++++++
...-llvm-allow-env-override-of-exe-path.patch | 38 ++++
recipes-devtools/glow/llvm8_git.bb | 180 ++++++++++++++++++
.../ti-tisdk-makefile_1.0.bbappend | 4 +
13 files changed, 508 insertions(+), 30 deletions(-)
create mode 100644 recipes-core/packagegroups/nativesdk-packagegroup-arago-tisdk-host.bbappend
rename recipes-devtools/glow/{llvm-glow_git.bb => clang8-native_git.bb} (51%)
create mode 100644 recipes-devtools/glow/files/0002-Skip-compilation-of-native-helper-tools.patch
create mode 100644 recipes-devtools/glow/llvm8/0006-llvm-TargetLibraryInfo-Undefine-libc-functions-if-th.patch
create mode 100644 recipes-devtools/glow/llvm8/0007-llvm-allow-env-override-of-exe-path.patch
create mode 100644 recipes-devtools/glow/llvm8_git.bb
--
2.17.1
^ permalink raw reply [flat|nested] 12+ messages in thread* [meta-processor-sdk][PATCHv2 1/5] mesa, llvm, meson: Update llvm to 8.0.1 plus define and use LLVM version globally 2019-08-23 13:58 [meta-processor-sdk][PATCHv2 0/5] Pytorch GLOW AOT tool and demo example Djordje Senicic @ 2019-08-23 13:58 ` Djordje Senicic 2019-08-23 16:15 ` [EXTERNAL] " Jacob Stiffler 2019-08-23 13:58 ` [meta-processor-sdk][PATCHv2 2/5] clang8-native: llvm and clang used only for compilation of native tools for glow compiler Djordje Senicic ` (3 subsequent siblings) 4 siblings, 1 reply; 12+ messages in thread From: Djordje Senicic @ 2019-08-23 13:58 UTC (permalink / raw) To: meta-arago; +Cc: d-senicic1, Djordje Senicic, Khem Raj, Richard Purdie - Add missing dependency on libedit - Define LLVMVERSION on the same lines as GCCVERSION and other tools - Use LLVMVERSION in mesa and meson.bbclass to get llvm version instead of hardcoding it - Use llvm patches unmodified from meta-clang, helps in keeping them in sync - Define PREFERRED_VERSION for llvm, llvm-native, nativesdk-llvm Signed-off-by: Khem Raj <raj.khem@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> The recipe was imported from the OpenEmbedded server http://cgit.openembedded.org/openembedded-core/commit/meta/recipes-devtools/llvm/llvm_git.bb commit_id 3c08b638348abd543fc92baf56c28ca16ae6aac6 Currently this is needed for compiling pytorch glow with CPU backend which requires llvm7 or later. To avoid being used globally name before 2020LTS, name of the recipe is not llvm_git but llvm8_git Signed-off-by: Djordje Senicic <x0157990@ti.com> --- ...ryInfo-Undefine-libc-functions-if-th.patch | 92 +++++++++ ...-llvm-allow-env-override-of-exe-path.patch | 38 ++++ recipes-devtools/glow/llvm-glow_git.bb | 180 ++++++++++++++++-- 3 files changed, 298 insertions(+), 12 deletions(-) create mode 100644 recipes-devtools/glow/llvm-glow/0006-llvm-TargetLibraryInfo-Undefine-libc-functions-if-th.patch create mode 100644 recipes-devtools/glow/llvm-glow/0007-llvm-allow-env-override-of-exe-path.patch diff --git a/recipes-devtools/glow/llvm-glow/0006-llvm-TargetLibraryInfo-Undefine-libc-functions-if-th.patch b/recipes-devtools/glow/llvm-glow/0006-llvm-TargetLibraryInfo-Undefine-libc-functions-if-th.patch new file mode 100644 index 0000000..789c308 --- /dev/null +++ b/recipes-devtools/glow/llvm-glow/0006-llvm-TargetLibraryInfo-Undefine-libc-functions-if-th.patch @@ -0,0 +1,92 @@ +From 4c08879d2dfbe7face4e679ac8499dc7bff2dd20 Mon Sep 17 00:00:00 2001 +From: Khem Raj <raj.khem@gmail.com> +Date: Sat, 21 May 2016 00:33:20 +0000 +Subject: [PATCH 06/19] llvm: TargetLibraryInfo: Undefine libc functions if + they are macros + +musl defines some functions as macros and not inline functions +if this is the case then make sure to undefine them + +Upstream-Status: Pending +Signed-off-by: Khem Raj <raj.khem@gmail.com> +--- + .../llvm/Analysis/TargetLibraryInfo.def | 21 +++++++++++++++++++ + 1 file changed, 21 insertions(+) + +diff --git a/llvm/include/llvm/Analysis/TargetLibraryInfo.def b/llvm/include/llvm/Analysis/TargetLibraryInfo.def +index afed404f04c..876888656f2 100644 +--- a/llvm/include/llvm/Analysis/TargetLibraryInfo.def ++++ b/llvm/include/llvm/Analysis/TargetLibraryInfo.def +@@ -782,6 +782,9 @@ TLI_DEFINE_STRING_INTERNAL("fmodl") + TLI_DEFINE_ENUM_INTERNAL(fopen) + TLI_DEFINE_STRING_INTERNAL("fopen") + /// FILE *fopen64(const char *filename, const char *opentype) ++#ifdef fopen64 ++#undef fopen64 ++#endif + TLI_DEFINE_ENUM_INTERNAL(fopen64) + TLI_DEFINE_STRING_INTERNAL("fopen64") + /// int fork(); +@@ -829,6 +832,9 @@ TLI_DEFINE_STRING_INTERNAL("fseek") + /// int fseeko(FILE *stream, off_t offset, int whence); + TLI_DEFINE_ENUM_INTERNAL(fseeko) + TLI_DEFINE_STRING_INTERNAL("fseeko") ++#ifdef fseeko64 ++#undef fseeko64 ++#endif + /// int fseeko64(FILE *stream, off64_t offset, int whence) + TLI_DEFINE_ENUM_INTERNAL(fseeko64) + TLI_DEFINE_STRING_INTERNAL("fseeko64") +@@ -839,6 +845,9 @@ TLI_DEFINE_STRING_INTERNAL("fsetpos") + TLI_DEFINE_ENUM_INTERNAL(fstat) + TLI_DEFINE_STRING_INTERNAL("fstat") + /// int fstat64(int filedes, struct stat64 *buf) ++#ifdef fstat64 ++#undef fstat64 ++#endif + TLI_DEFINE_ENUM_INTERNAL(fstat64) + TLI_DEFINE_STRING_INTERNAL("fstat64") + /// int fstatvfs(int fildes, struct statvfs *buf); +@@ -854,6 +863,9 @@ TLI_DEFINE_STRING_INTERNAL("ftell") + TLI_DEFINE_ENUM_INTERNAL(ftello) + TLI_DEFINE_STRING_INTERNAL("ftello") + /// off64_t ftello64(FILE *stream) ++#ifdef ftello64 ++#undef ftello64 ++#endif + TLI_DEFINE_ENUM_INTERNAL(ftello64) + TLI_DEFINE_STRING_INTERNAL("ftello64") + /// int ftrylockfile(FILE *file); +@@ -980,6 +992,9 @@ TLI_DEFINE_STRING_INTERNAL("logl") + TLI_DEFINE_ENUM_INTERNAL(lstat) + TLI_DEFINE_STRING_INTERNAL("lstat") + /// int lstat64(const char *path, struct stat64 *buf); ++#ifdef lstat64 ++#undef lstat64 ++#endif + TLI_DEFINE_ENUM_INTERNAL(lstat64) + TLI_DEFINE_STRING_INTERNAL("lstat64") + /// void *malloc(size_t size); +@@ -1205,6 +1220,9 @@ TLI_DEFINE_STRING_INTERNAL("sscanf") + TLI_DEFINE_ENUM_INTERNAL(stat) + TLI_DEFINE_STRING_INTERNAL("stat") + /// int stat64(const char *path, struct stat64 *buf); ++#ifdef stat64 ++#undef stat64 ++#endif + TLI_DEFINE_ENUM_INTERNAL(stat64) + TLI_DEFINE_STRING_INTERNAL("stat64") + /// int statvfs(const char *path, struct statvfs *buf); +@@ -1340,6 +1358,9 @@ TLI_DEFINE_STRING_INTERNAL("times") + TLI_DEFINE_ENUM_INTERNAL(tmpfile) + TLI_DEFINE_STRING_INTERNAL("tmpfile") + /// FILE *tmpfile64(void) ++#ifdef tmpfile64 ++#undef tmpfile64 ++#endif + TLI_DEFINE_ENUM_INTERNAL(tmpfile64) + TLI_DEFINE_STRING_INTERNAL("tmpfile64") + /// int toascii(int c); +-- +2.22.0 + diff --git a/recipes-devtools/glow/llvm-glow/0007-llvm-allow-env-override-of-exe-path.patch b/recipes-devtools/glow/llvm-glow/0007-llvm-allow-env-override-of-exe-path.patch new file mode 100644 index 0000000..f47080c --- /dev/null +++ b/recipes-devtools/glow/llvm-glow/0007-llvm-allow-env-override-of-exe-path.patch @@ -0,0 +1,38 @@ +From b66d6f39a374b8df41e7235351e5dee2e81f440c Mon Sep 17 00:00:00 2001 +From: Martin Kelly <mkelly@xevo.com> +Date: Fri, 19 May 2017 00:22:57 -0700 +Subject: [PATCH 07/19] llvm: allow env override of exe path + +When using a native llvm-config from inside a sysroot, we need llvm-config to +return the libraries, include directories, etc. from inside the sysroot rather +than from the native sysroot. Thus provide an env override for calling +llvm-config from a target sysroot. + +Upstream-Status: Pending +Signed-off-by: Martin Kelly <mkelly@xevo.com> +Signed-off-by: Khem Raj <raj.khem@gmail.com> +--- + llvm/tools/llvm-config/llvm-config.cpp | 7 +++++++ + 1 file changed, 7 insertions(+) + +diff --git a/llvm/tools/llvm-config/llvm-config.cpp b/llvm/tools/llvm-config/llvm-config.cpp +index 7ef7c46a262..a4f7ed82c7b 100644 +--- a/llvm/tools/llvm-config/llvm-config.cpp ++++ b/llvm/tools/llvm-config/llvm-config.cpp +@@ -225,6 +225,13 @@ Typical components:\n\ + + /// Compute the path to the main executable. + std::string GetExecutablePath(const char *Argv0) { ++ // Hack for Yocto: we need to override the root path when we are using ++ // llvm-config from within a target sysroot. ++ const char *Sysroot = std::getenv("YOCTO_ALTERNATE_EXE_PATH"); ++ if (Sysroot != nullptr) { ++ return Sysroot; ++ } ++ + // This just needs to be some symbol in the binary; C++ doesn't + // allow taking the address of ::main however. + void *P = (void *)(intptr_t)GetExecutablePath; +-- +2.22.0 + diff --git a/recipes-devtools/glow/llvm-glow_git.bb b/recipes-devtools/glow/llvm-glow_git.bb index 58d392b..d3c2642 100644 --- a/recipes-devtools/glow/llvm-glow_git.bb +++ b/recipes-devtools/glow/llvm-glow_git.bb @@ -1,24 +1,180 @@ -DESCRIPTION = "LLVM release suggested for pytorch glow neural network compiler" -HOMEPAGE = "https://github.com/llvm-mirror/llvm" +# Copyright (C) 2017 Khem Raj <raj.khem@gmail.com> +# Released under the MIT license (see COPYING.MIT for the terms) + +DESCRIPTION = "The LLVM Compiler Infrastructure" +HOMEPAGE = "http://llvm.org" +LICENSE = "NCSA" SECTION = "devel" -LICENSE = "Apache-2.0" + LIC_FILES_CHKSUM = "file://LICENSE.TXT;md5=c6b766a4e85dd28301eeed54a6684648" -PV = "8.0" -PR = "r0" -DEPENDS = "ninja-native protobuf-native libpng-native protobuf-native protobuf-c-native gflags-native glog-native" +DEPENDS = "libffi libxml2 zlib libedit ninja-native llvm8-native" -SOLIBVER = "1" +RDEPENDS_${PN}_append_class-target = " ncurses-terminfo" + +inherit cmake pkgconfig + +PROVIDES += "llvm${PV}" -SRC_URI = "git://github.com/llvm-mirror/llvm.git;protocol=https;branch=release_80;rev=5563a6a769b48912b9c8052124520610966eda28;destsuffix=git/llvm \ - git://github.com/llvm-mirror/clang.git;protocol=https;branch=release_80;rev=ccfe04576c13497b9c422ceef0b6efe99077a392;destsuffix=git/clang \ - " +LLVM_RELEASE = "${PV}" +LLVM_DIR = "llvm${LLVM_RELEASE}" +SRCREV = "19a71f6bdf2dddb10764939e7f0ec2b98dba76c9" + +BRANCH = "release/${MAJOR_VERSION}.x" +MAJOR_VERSION = "8" +MINOR_VERSION = "0" +PATCH_VERSION = "1" +SOLIBVER = "1" +PV = "${MAJOR_VERSION}.${MINOR_VERSION}.${PATCH_VERSION}" +SRC_URI = "git://github.com/llvm/llvm-project.git;branch=${BRANCH} \ + file://0006-llvm-TargetLibraryInfo-Undefine-libc-functions-if-th.patch;striplevel=2 \ + file://0007-llvm-allow-env-override-of-exe-path.patch;striplevel=2 \ + " S = "${WORKDIR}/git/llvm" -EXTRA_OECMAKE = "-DCMAKE_BUILD_TYPE=Release -DLLVM_INSTALL_UTILS=ON -DLLVM_ENABLE_PROJECTS=clang" +LLVM_INSTALL_DIR = "${WORKDIR}/llvm-install" + +def get_llvm_arch(bb, d, arch_var): + import re + a = d.getVar(arch_var) + if re.match(r'(i.86|athlon|x86.64)$', a): return 'X86' + elif re.match(r'arm$', a): return 'ARM' + elif re.match(r'armeb$', a): return 'ARM' + elif re.match(r'aarch64$', a): return 'AArch64' + elif re.match(r'aarch64_be$', a): return 'AArch64' + elif re.match(r'mips(isa|)(32|64|)(r6|)(el|)$', a): return 'Mips' + elif re.match(r'p(pc|owerpc)(|64)', a): return 'PowerPC' + else: + raise bb.parse.SkipRecipe("Cannot map '%s' to a supported LLVM architecture" % a) + +def get_llvm_host_arch(bb, d): + return get_llvm_arch(bb, d, 'HOST_ARCH') + +# +# Default to build all OE-Core supported target arches (user overridable). +# +LLVM_TARGETS ?= "AMDGPU;${@get_llvm_host_arch(bb, d)}" + +ARM_INSTRUCTION_SET_armv5 = "arm" +ARM_INSTRUCTION_SET_armv4t = "arm" + +EXTRA_OECMAKE += "-DLLVM_ENABLE_ASSERTIONS=OFF \ + -DLLVM_ENABLE_EXPENSIVE_CHECKS=OFF \ + -DLLVM_ENABLE_PIC=ON \ + -DLLVM_BINDINGS_LIST='' \ + -DLLVM_LINK_LLVM_DYLIB=ON \ + -DLLVM_ENABLE_FFI=ON \ + -DLLVM_ENABLE_RTTI=ON \ + -DFFI_INCLUDE_DIR=$(pkg-config --variable=includedir libffi) \ + -DLLVM_OPTIMIZED_TABLEGEN=ON \ + -DLLVM_TARGETS_TO_BUILD='${LLVM_TARGETS}' \ + -DLLVM_TEMPORARILY_ALLOW_OLD_TOOLCHAIN=ON \ + -DPYTHON_EXECUTABLE=${HOSTTOOLS_DIR}/python2 \ + -G Ninja" + +EXTRA_OECMAKE_append_class-target = "\ + -DCMAKE_CROSSCOMPILING:BOOL=ON \ + -DLLVM_TABLEGEN=${STAGING_BINDIR_NATIVE}/llvm-tblgen${PV} \ + -DLLVM_CONFIG_PATH=${STAGING_BINDIR_NATIVE}/llvm-config${PV} \ + " + +EXTRA_OECMAKE_append_class-nativesdk = "\ + -DCMAKE_CROSSCOMPILING:BOOL=ON \ + -DLLVM_TABLEGEN=${STAGING_BINDIR_NATIVE}/llvm-tblgen${PV} \ + -DLLVM_CONFIG_PATH=${STAGING_BINDIR_NATIVE}/llvm-config${PV} \ + " + +CXXFLAGS_append_class-target_powerpc = " -mlongcall" + +do_configure_prepend() { +# Fix paths in llvm-config + sed -i "s|sys::path::parent_path(CurrentPath))\.str()|sys::path::parent_path(sys::path::parent_path(CurrentPath))).str()|g" ${S}/tools/llvm-config/llvm-config.cpp + sed -ri "s#/(bin|include|lib)(/?\")#/\1/${LLVM_DIR}\2#g" ${S}/tools/llvm-config/llvm-config.cpp + sed -ri "s#lib/${LLVM_DIR}#${baselib}/${LLVM_DIR}#g" ${S}/tools/llvm-config/llvm-config.cpp +} + +do_compile() { + ninja -v ${PARALLEL_MAKE} +} + +do_compile_class-native() { + ninja -v ${PARALLEL_MAKE} llvm-config llvm-tblgen +} + +do_install() { + DESTDIR=${LLVM_INSTALL_DIR} ninja -v install + install -D -m 0755 ${B}/bin/llvm-config ${D}${libdir}/${LLVM_DIR}/llvm-config + + install -d ${D}${bindir}/${LLVM_DIR} + cp -r ${LLVM_INSTALL_DIR}${bindir}/* ${D}${bindir}/${LLVM_DIR}/ + + install -d ${D}${includedir}/${LLVM_DIR} + cp -r ${LLVM_INSTALL_DIR}${includedir}/* ${D}${includedir}/${LLVM_DIR}/ + + install -d ${D}${libdir}/${LLVM_DIR} + + # The LLVM sources have "/lib" embedded and so we cannot completely rely on the ${libdir} variable + if [ -d ${LLVM_INSTALL_DIR}${libdir}/ ]; then + cp -r ${LLVM_INSTALL_DIR}${libdir}/* ${D}${libdir}/${LLVM_DIR}/ + elif [ -d ${LLVM_INSTALL_DIR}${prefix}/lib ]; then + cp -r ${LLVM_INSTALL_DIR}${prefix}/lib/* ${D}${libdir}/${LLVM_DIR}/ + elif [ -d ${LLVM_INSTALL_DIR}${prefix}/lib64 ]; then + cp -r ${LLVM_INSTALL_DIR}${prefix}/lib64/* ${D}${libdir}/${LLVM_DIR}/ + fi + + # Remove unnecessary cmake files + rm -rf ${D}${libdir}/${LLVM_DIR}/cmake + + ln -s ${LLVM_DIR}/libLLVM-${MAJOR_VERSION}${SOLIBSDEV} ${D}${libdir}/libLLVM-${MAJOR_VERSION}${SOLIBSDEV} + + # We'll have to delete the libLLVM.so due to multiple reasons... + rm -rf ${D}${libdir}/${LLVM_DIR}/libLLVM.so + rm -rf ${D}${libdir}/${LLVM_DIR}/libLTO.so +} + +do_install_class-native() { + install -D -m 0755 ${B}/bin/llvm-tblgen ${D}${bindir}/llvm-tblgen${PV} + install -D -m 0755 ${B}/bin/llvm-config ${D}${bindir}/llvm-config${PV} + install -D -m 0755 ${B}/lib/libLLVM-${MAJOR_VERSION}.so ${D}${libdir}/libLLVM-${MAJOR_VERSION}.so +} + +PACKAGES =+ "${PN}-bugpointpasses ${PN}-llvmhello ${PN}-libllvm ${PN}-liboptremarks ${PN}-liblto" + +RRECOMMENDS_${PN}-dev += "${PN}-bugpointpasses ${PN}-llvmhello ${PN}-liboptremarks" + +FILES_${PN}-bugpointpasses = "\ + ${libdir}/${LLVM_DIR}/BugpointPasses.so \ +" + +FILES_${PN}-libllvm = "\ + ${libdir}/${LLVM_DIR}/libLLVM-${MAJOR_VERSION}.so \ + ${libdir}/libLLVM-${MAJOR_VERSION}.so \ +" + +FILES_${PN}-liblto += "\ + ${libdir}/${LLVM_DIR}/libLTO.so.* \ +" + +FILES_${PN}-liboptremarks += "\ + ${libdir}/${LLVM_DIR}/libOptRemarks.so.* \ +" + +FILES_${PN}-llvmhello = "\ + ${libdir}/${LLVM_DIR}/LLVMHello.so \ +" + +FILES_${PN}-dev += " \ + ${libdir}/${LLVM_DIR}/llvm-config \ + ${libdir}/${LLVM_DIR}/libOptRemarks.so \ + ${libdir}/${LLVM_DIR}/libLLVM-${MAJOR_VERSION}.${MINOR_VERSION}.${PATCH_VERSION}.so \ +" + +FILES_${PN}-staticdev += "\ + ${libdir}/${LLVM_DIR}/*.a \ +" -inherit pkgconfig cmake +INSANE_SKIP_${PN}-libllvm += "dev-so" BBCLASSEXTEND = "native nativesdk" -- 2.17.1 ^ permalink raw reply related [flat|nested] 12+ messages in thread
* Re: [EXTERNAL] [meta-processor-sdk][PATCHv2 1/5] mesa, llvm, meson: Update llvm to 8.0.1 plus define and use LLVM version globally 2019-08-23 13:58 ` [meta-processor-sdk][PATCHv2 1/5] mesa, llvm, meson: Update llvm to 8.0.1 plus define and use LLVM version globally Djordje Senicic @ 2019-08-23 16:15 ` Jacob Stiffler 2019-08-23 17:34 ` Senicic, Djordje 0 siblings, 1 reply; 12+ messages in thread From: Jacob Stiffler @ 2019-08-23 16:15 UTC (permalink / raw) To: Djordje Senicic, meta-arago; +Cc: d-senicic1 Djordje, I get an error with this. Have you tried building nativesdk-llvm8? ERROR: Nothing PROVIDES 'nativesdk-libedit' (but virtual:nativesdk:/oe/bld/sources/meta-processor-sdk/recipes-devtools/glow/llvm8_git.bb DEPENDS on or otherwise requires it). And when using "git send-email", you can use the "--no-signed-off-by-cc" flag to prevent CCing those that have "signed-off" in the commit message. - Jake On 8/23/2019 9:58 AM, Djordje Senicic wrote: > - Add missing dependency on libedit > - Define LLVMVERSION on the same lines as GCCVERSION and other tools > - Use LLVMVERSION in mesa and meson.bbclass to get llvm version instead of > hardcoding it > - Use llvm patches unmodified from meta-clang, helps in keeping them in > sync > - Define PREFERRED_VERSION for llvm, llvm-native, nativesdk-llvm > > Signed-off-by: Khem Raj <raj.khem@gmail.com> > Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> > > The recipe was imported from the OpenEmbedded server > http://cgit.openembedded.org/openembedded-core/commit/meta/recipes-devtools/llvm/llvm_git.bb > commit_id 3c08b638348abd543fc92baf56c28ca16ae6aac6 > Currently this is needed for compiling pytorch glow with CPU backend which requires llvm7 or later. > To avoid being used globally name before 2020LTS, name of the recipe is not llvm_git but llvm8_git > > Signed-off-by: Djordje Senicic <x0157990@ti.com> > --- > ...ryInfo-Undefine-libc-functions-if-th.patch | 92 +++++++++ > ...-llvm-allow-env-override-of-exe-path.patch | 38 ++++ > recipes-devtools/glow/llvm-glow_git.bb | 180 ++++++++++++++++-- > 3 files changed, 298 insertions(+), 12 deletions(-) > create mode 100644 recipes-devtools/glow/llvm-glow/0006-llvm-TargetLibraryInfo-Undefine-libc-functions-if-th.patch > create mode 100644 recipes-devtools/glow/llvm-glow/0007-llvm-allow-env-override-of-exe-path.patch > > diff --git a/recipes-devtools/glow/llvm-glow/0006-llvm-TargetLibraryInfo-Undefine-libc-functions-if-th.patch b/recipes-devtools/glow/llvm-glow/0006-llvm-TargetLibraryInfo-Undefine-libc-functions-if-th.patch > new file mode 100644 > index 0000000..789c308 > --- /dev/null > +++ b/recipes-devtools/glow/llvm-glow/0006-llvm-TargetLibraryInfo-Undefine-libc-functions-if-th.patch > @@ -0,0 +1,92 @@ > +From 4c08879d2dfbe7face4e679ac8499dc7bff2dd20 Mon Sep 17 00:00:00 2001 > +From: Khem Raj <raj.khem@gmail.com> > +Date: Sat, 21 May 2016 00:33:20 +0000 > +Subject: [PATCH 06/19] llvm: TargetLibraryInfo: Undefine libc functions if > + they are macros > + > +musl defines some functions as macros and not inline functions > +if this is the case then make sure to undefine them > + > +Upstream-Status: Pending > +Signed-off-by: Khem Raj <raj.khem@gmail.com> > +--- > + .../llvm/Analysis/TargetLibraryInfo.def | 21 +++++++++++++++++++ > + 1 file changed, 21 insertions(+) > + > +diff --git a/llvm/include/llvm/Analysis/TargetLibraryInfo.def b/llvm/include/llvm/Analysis/TargetLibraryInfo.def > +index afed404f04c..876888656f2 100644 > +--- a/llvm/include/llvm/Analysis/TargetLibraryInfo.def > ++++ b/llvm/include/llvm/Analysis/TargetLibraryInfo.def > +@@ -782,6 +782,9 @@ TLI_DEFINE_STRING_INTERNAL("fmodl") > + TLI_DEFINE_ENUM_INTERNAL(fopen) > + TLI_DEFINE_STRING_INTERNAL("fopen") > + /// FILE *fopen64(const char *filename, const char *opentype) > ++#ifdef fopen64 > ++#undef fopen64 > ++#endif > + TLI_DEFINE_ENUM_INTERNAL(fopen64) > + TLI_DEFINE_STRING_INTERNAL("fopen64") > + /// int fork(); > +@@ -829,6 +832,9 @@ TLI_DEFINE_STRING_INTERNAL("fseek") > + /// int fseeko(FILE *stream, off_t offset, int whence); > + TLI_DEFINE_ENUM_INTERNAL(fseeko) > + TLI_DEFINE_STRING_INTERNAL("fseeko") > ++#ifdef fseeko64 > ++#undef fseeko64 > ++#endif > + /// int fseeko64(FILE *stream, off64_t offset, int whence) > + TLI_DEFINE_ENUM_INTERNAL(fseeko64) > + TLI_DEFINE_STRING_INTERNAL("fseeko64") > +@@ -839,6 +845,9 @@ TLI_DEFINE_STRING_INTERNAL("fsetpos") > + TLI_DEFINE_ENUM_INTERNAL(fstat) > + TLI_DEFINE_STRING_INTERNAL("fstat") > + /// int fstat64(int filedes, struct stat64 *buf) > ++#ifdef fstat64 > ++#undef fstat64 > ++#endif > + TLI_DEFINE_ENUM_INTERNAL(fstat64) > + TLI_DEFINE_STRING_INTERNAL("fstat64") > + /// int fstatvfs(int fildes, struct statvfs *buf); > +@@ -854,6 +863,9 @@ TLI_DEFINE_STRING_INTERNAL("ftell") > + TLI_DEFINE_ENUM_INTERNAL(ftello) > + TLI_DEFINE_STRING_INTERNAL("ftello") > + /// off64_t ftello64(FILE *stream) > ++#ifdef ftello64 > ++#undef ftello64 > ++#endif > + TLI_DEFINE_ENUM_INTERNAL(ftello64) > + TLI_DEFINE_STRING_INTERNAL("ftello64") > + /// int ftrylockfile(FILE *file); > +@@ -980,6 +992,9 @@ TLI_DEFINE_STRING_INTERNAL("logl") > + TLI_DEFINE_ENUM_INTERNAL(lstat) > + TLI_DEFINE_STRING_INTERNAL("lstat") > + /// int lstat64(const char *path, struct stat64 *buf); > ++#ifdef lstat64 > ++#undef lstat64 > ++#endif > + TLI_DEFINE_ENUM_INTERNAL(lstat64) > + TLI_DEFINE_STRING_INTERNAL("lstat64") > + /// void *malloc(size_t size); > +@@ -1205,6 +1220,9 @@ TLI_DEFINE_STRING_INTERNAL("sscanf") > + TLI_DEFINE_ENUM_INTERNAL(stat) > + TLI_DEFINE_STRING_INTERNAL("stat") > + /// int stat64(const char *path, struct stat64 *buf); > ++#ifdef stat64 > ++#undef stat64 > ++#endif > + TLI_DEFINE_ENUM_INTERNAL(stat64) > + TLI_DEFINE_STRING_INTERNAL("stat64") > + /// int statvfs(const char *path, struct statvfs *buf); > +@@ -1340,6 +1358,9 @@ TLI_DEFINE_STRING_INTERNAL("times") > + TLI_DEFINE_ENUM_INTERNAL(tmpfile) > + TLI_DEFINE_STRING_INTERNAL("tmpfile") > + /// FILE *tmpfile64(void) > ++#ifdef tmpfile64 > ++#undef tmpfile64 > ++#endif > + TLI_DEFINE_ENUM_INTERNAL(tmpfile64) > + TLI_DEFINE_STRING_INTERNAL("tmpfile64") > + /// int toascii(int c); > +-- > +2.22.0 > + > diff --git a/recipes-devtools/glow/llvm-glow/0007-llvm-allow-env-override-of-exe-path.patch b/recipes-devtools/glow/llvm-glow/0007-llvm-allow-env-override-of-exe-path.patch > new file mode 100644 > index 0000000..f47080c > --- /dev/null > +++ b/recipes-devtools/glow/llvm-glow/0007-llvm-allow-env-override-of-exe-path.patch > @@ -0,0 +1,38 @@ > +From b66d6f39a374b8df41e7235351e5dee2e81f440c Mon Sep 17 00:00:00 2001 > +From: Martin Kelly <mkelly@xevo.com> > +Date: Fri, 19 May 2017 00:22:57 -0700 > +Subject: [PATCH 07/19] llvm: allow env override of exe path > + > +When using a native llvm-config from inside a sysroot, we need llvm-config to > +return the libraries, include directories, etc. from inside the sysroot rather > +than from the native sysroot. Thus provide an env override for calling > +llvm-config from a target sysroot. > + > +Upstream-Status: Pending > +Signed-off-by: Martin Kelly <mkelly@xevo.com> > +Signed-off-by: Khem Raj <raj.khem@gmail.com> > +--- > + llvm/tools/llvm-config/llvm-config.cpp | 7 +++++++ > + 1 file changed, 7 insertions(+) > + > +diff --git a/llvm/tools/llvm-config/llvm-config.cpp b/llvm/tools/llvm-config/llvm-config.cpp > +index 7ef7c46a262..a4f7ed82c7b 100644 > +--- a/llvm/tools/llvm-config/llvm-config.cpp > ++++ b/llvm/tools/llvm-config/llvm-config.cpp > +@@ -225,6 +225,13 @@ Typical components:\n\ > + > + /// Compute the path to the main executable. > + std::string GetExecutablePath(const char *Argv0) { > ++ // Hack for Yocto: we need to override the root path when we are using > ++ // llvm-config from within a target sysroot. > ++ const char *Sysroot = std::getenv("YOCTO_ALTERNATE_EXE_PATH"); > ++ if (Sysroot != nullptr) { > ++ return Sysroot; > ++ } > ++ > + // This just needs to be some symbol in the binary; C++ doesn't > + // allow taking the address of ::main however. > + void *P = (void *)(intptr_t)GetExecutablePath; > +-- > +2.22.0 > + > diff --git a/recipes-devtools/glow/llvm-glow_git.bb b/recipes-devtools/glow/llvm-glow_git.bb > index 58d392b..d3c2642 100644 > --- a/recipes-devtools/glow/llvm-glow_git.bb > +++ b/recipes-devtools/glow/llvm-glow_git.bb > @@ -1,24 +1,180 @@ > -DESCRIPTION = "LLVM release suggested for pytorch glow neural network compiler" > -HOMEPAGE = "https://github.com/llvm-mirror/llvm" > +# Copyright (C) 2017 Khem Raj <raj.khem@gmail.com> > +# Released under the MIT license (see COPYING.MIT for the terms) > + > +DESCRIPTION = "The LLVM Compiler Infrastructure" > +HOMEPAGE = "http://llvm.org" > +LICENSE = "NCSA" > SECTION = "devel" > -LICENSE = "Apache-2.0" > + > LIC_FILES_CHKSUM = "file://LICENSE.TXT;md5=c6b766a4e85dd28301eeed54a6684648" > -PV = "8.0" > -PR = "r0" > > -DEPENDS = "ninja-native protobuf-native libpng-native protobuf-native protobuf-c-native gflags-native glog-native" > +DEPENDS = "libffi libxml2 zlib libedit ninja-native llvm8-native" > > -SOLIBVER = "1" > +RDEPENDS_${PN}_append_class-target = " ncurses-terminfo" > + > +inherit cmake pkgconfig > + > +PROVIDES += "llvm${PV}" > > -SRC_URI = "git://github.com/llvm-mirror/llvm.git;protocol=https;branch=release_80;rev=5563a6a769b48912b9c8052124520610966eda28;destsuffix=git/llvm \ > - git://github.com/llvm-mirror/clang.git;protocol=https;branch=release_80;rev=ccfe04576c13497b9c422ceef0b6efe99077a392;destsuffix=git/clang \ > - " > +LLVM_RELEASE = "${PV}" > +LLVM_DIR = "llvm${LLVM_RELEASE}" > > +SRCREV = "19a71f6bdf2dddb10764939e7f0ec2b98dba76c9" > + > +BRANCH = "release/${MAJOR_VERSION}.x" > +MAJOR_VERSION = "8" > +MINOR_VERSION = "0" > +PATCH_VERSION = "1" > +SOLIBVER = "1" > +PV = "${MAJOR_VERSION}.${MINOR_VERSION}.${PATCH_VERSION}" > +SRC_URI = "git://github.com/llvm/llvm-project.git;branch=${BRANCH} \ > + file://0006-llvm-TargetLibraryInfo-Undefine-libc-functions-if-th.patch;striplevel=2 \ > + file://0007-llvm-allow-env-override-of-exe-path.patch;striplevel=2 \ > + " > > S = "${WORKDIR}/git/llvm" > > -EXTRA_OECMAKE = "-DCMAKE_BUILD_TYPE=Release -DLLVM_INSTALL_UTILS=ON -DLLVM_ENABLE_PROJECTS=clang" > +LLVM_INSTALL_DIR = "${WORKDIR}/llvm-install" > + > +def get_llvm_arch(bb, d, arch_var): > + import re > + a = d.getVar(arch_var) > + if re.match(r'(i.86|athlon|x86.64)$', a): return 'X86' > + elif re.match(r'arm$', a): return 'ARM' > + elif re.match(r'armeb$', a): return 'ARM' > + elif re.match(r'aarch64$', a): return 'AArch64' > + elif re.match(r'aarch64_be$', a): return 'AArch64' > + elif re.match(r'mips(isa|)(32|64|)(r6|)(el|)$', a): return 'Mips' > + elif re.match(r'p(pc|owerpc)(|64)', a): return 'PowerPC' > + else: > + raise bb.parse.SkipRecipe("Cannot map '%s' to a supported LLVM architecture" % a) > + > +def get_llvm_host_arch(bb, d): > + return get_llvm_arch(bb, d, 'HOST_ARCH') > + > +# > +# Default to build all OE-Core supported target arches (user overridable). > +# > +LLVM_TARGETS ?= "AMDGPU;${@get_llvm_host_arch(bb, d)}" > + > +ARM_INSTRUCTION_SET_armv5 = "arm" > +ARM_INSTRUCTION_SET_armv4t = "arm" > + > +EXTRA_OECMAKE += "-DLLVM_ENABLE_ASSERTIONS=OFF \ > + -DLLVM_ENABLE_EXPENSIVE_CHECKS=OFF \ > + -DLLVM_ENABLE_PIC=ON \ > + -DLLVM_BINDINGS_LIST='' \ > + -DLLVM_LINK_LLVM_DYLIB=ON \ > + -DLLVM_ENABLE_FFI=ON \ > + -DLLVM_ENABLE_RTTI=ON \ > + -DFFI_INCLUDE_DIR=$(pkg-config --variable=includedir libffi) \ > + -DLLVM_OPTIMIZED_TABLEGEN=ON \ > + -DLLVM_TARGETS_TO_BUILD='${LLVM_TARGETS}' \ > + -DLLVM_TEMPORARILY_ALLOW_OLD_TOOLCHAIN=ON \ > + -DPYTHON_EXECUTABLE=${HOSTTOOLS_DIR}/python2 \ > + -G Ninja" > + > +EXTRA_OECMAKE_append_class-target = "\ > + -DCMAKE_CROSSCOMPILING:BOOL=ON \ > + -DLLVM_TABLEGEN=${STAGING_BINDIR_NATIVE}/llvm-tblgen${PV} \ > + -DLLVM_CONFIG_PATH=${STAGING_BINDIR_NATIVE}/llvm-config${PV} \ > + " > + > +EXTRA_OECMAKE_append_class-nativesdk = "\ > + -DCMAKE_CROSSCOMPILING:BOOL=ON \ > + -DLLVM_TABLEGEN=${STAGING_BINDIR_NATIVE}/llvm-tblgen${PV} \ > + -DLLVM_CONFIG_PATH=${STAGING_BINDIR_NATIVE}/llvm-config${PV} \ > + " > + > +CXXFLAGS_append_class-target_powerpc = " -mlongcall" > + > +do_configure_prepend() { > +# Fix paths in llvm-config > + sed -i "s|sys::path::parent_path(CurrentPath))\.str()|sys::path::parent_path(sys::path::parent_path(CurrentPath))).str()|g" ${S}/tools/llvm-config/llvm-config.cpp > + sed -ri "s#/(bin|include|lib)(/?\")#/\1/${LLVM_DIR}\2#g" ${S}/tools/llvm-config/llvm-config.cpp > + sed -ri "s#lib/${LLVM_DIR}#${baselib}/${LLVM_DIR}#g" ${S}/tools/llvm-config/llvm-config.cpp > +} > + > +do_compile() { > + ninja -v ${PARALLEL_MAKE} > +} > + > +do_compile_class-native() { > + ninja -v ${PARALLEL_MAKE} llvm-config llvm-tblgen > +} > + > +do_install() { > + DESTDIR=${LLVM_INSTALL_DIR} ninja -v install > + install -D -m 0755 ${B}/bin/llvm-config ${D}${libdir}/${LLVM_DIR}/llvm-config > + > + install -d ${D}${bindir}/${LLVM_DIR} > + cp -r ${LLVM_INSTALL_DIR}${bindir}/* ${D}${bindir}/${LLVM_DIR}/ > + > + install -d ${D}${includedir}/${LLVM_DIR} > + cp -r ${LLVM_INSTALL_DIR}${includedir}/* ${D}${includedir}/${LLVM_DIR}/ > + > + install -d ${D}${libdir}/${LLVM_DIR} > + > + # The LLVM sources have "/lib" embedded and so we cannot completely rely on the ${libdir} variable > + if [ -d ${LLVM_INSTALL_DIR}${libdir}/ ]; then > + cp -r ${LLVM_INSTALL_DIR}${libdir}/* ${D}${libdir}/${LLVM_DIR}/ > + elif [ -d ${LLVM_INSTALL_DIR}${prefix}/lib ]; then > + cp -r ${LLVM_INSTALL_DIR}${prefix}/lib/* ${D}${libdir}/${LLVM_DIR}/ > + elif [ -d ${LLVM_INSTALL_DIR}${prefix}/lib64 ]; then > + cp -r ${LLVM_INSTALL_DIR}${prefix}/lib64/* ${D}${libdir}/${LLVM_DIR}/ > + fi > + > + # Remove unnecessary cmake files > + rm -rf ${D}${libdir}/${LLVM_DIR}/cmake > + > + ln -s ${LLVM_DIR}/libLLVM-${MAJOR_VERSION}${SOLIBSDEV} ${D}${libdir}/libLLVM-${MAJOR_VERSION}${SOLIBSDEV} > + > + # We'll have to delete the libLLVM.so due to multiple reasons... > + rm -rf ${D}${libdir}/${LLVM_DIR}/libLLVM.so > + rm -rf ${D}${libdir}/${LLVM_DIR}/libLTO.so > +} > + > +do_install_class-native() { > + install -D -m 0755 ${B}/bin/llvm-tblgen ${D}${bindir}/llvm-tblgen${PV} > + install -D -m 0755 ${B}/bin/llvm-config ${D}${bindir}/llvm-config${PV} > + install -D -m 0755 ${B}/lib/libLLVM-${MAJOR_VERSION}.so ${D}${libdir}/libLLVM-${MAJOR_VERSION}.so > +} > + > +PACKAGES =+ "${PN}-bugpointpasses ${PN}-llvmhello ${PN}-libllvm ${PN}-liboptremarks ${PN}-liblto" > + > +RRECOMMENDS_${PN}-dev += "${PN}-bugpointpasses ${PN}-llvmhello ${PN}-liboptremarks" > + > +FILES_${PN}-bugpointpasses = "\ > + ${libdir}/${LLVM_DIR}/BugpointPasses.so \ > +" > + > +FILES_${PN}-libllvm = "\ > + ${libdir}/${LLVM_DIR}/libLLVM-${MAJOR_VERSION}.so \ > + ${libdir}/libLLVM-${MAJOR_VERSION}.so \ > +" > + > +FILES_${PN}-liblto += "\ > + ${libdir}/${LLVM_DIR}/libLTO.so.* \ > +" > + > +FILES_${PN}-liboptremarks += "\ > + ${libdir}/${LLVM_DIR}/libOptRemarks.so.* \ > +" > + > +FILES_${PN}-llvmhello = "\ > + ${libdir}/${LLVM_DIR}/LLVMHello.so \ > +" > + > +FILES_${PN}-dev += " \ > + ${libdir}/${LLVM_DIR}/llvm-config \ > + ${libdir}/${LLVM_DIR}/libOptRemarks.so \ > + ${libdir}/${LLVM_DIR}/libLLVM-${MAJOR_VERSION}.${MINOR_VERSION}.${PATCH_VERSION}.so \ > +" > + > +FILES_${PN}-staticdev += "\ > + ${libdir}/${LLVM_DIR}/*.a \ > +" > > -inherit pkgconfig cmake > +INSANE_SKIP_${PN}-libllvm += "dev-so" > > BBCLASSEXTEND = "native nativesdk" ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [EXTERNAL] [meta-processor-sdk][PATCHv2 1/5] mesa, llvm, meson: Update llvm to 8.0.1 plus define and use LLVM version globally 2019-08-23 16:15 ` [EXTERNAL] " Jacob Stiffler @ 2019-08-23 17:34 ` Senicic, Djordje 2019-08-23 18:06 ` Jacob Stiffler 0 siblings, 1 reply; 12+ messages in thread From: Senicic, Djordje @ 2019-08-23 17:34 UTC (permalink / raw) To: Stiffler, Jacob, meta-arago I could build both nativesdk-libedit and nativesdk-llvm7 There is: ./meta-openembedded/meta-oe/recipes-devtools/libedit/libedit_20180525-3.1.bb -----Original Message----- From: Stiffler, Jacob Sent: Friday, August 23, 2019 12:16 PM To: Senicic, Djordje; meta-arago@arago-project.org Cc: Senicic, Djordje Subject: Re: [EXTERNAL] [meta-arago] [meta-processor-sdk][PATCHv2 1/5] mesa, llvm, meson: Update llvm to 8.0.1 plus define and use LLVM version globally Djordje, I get an error with this. Have you tried building nativesdk-llvm8? ERROR: Nothing PROVIDES 'nativesdk-libedit' (but virtual:nativesdk:/oe/bld/sources/meta-processor-sdk/recipes-devtools/glow/llvm8_git.bb DEPENDS on or otherwise requires it). And when using "git send-email", you can use the "--no-signed-off-by-cc" flag to prevent CCing those that have "signed-off" in the commit message. - Jake On 8/23/2019 9:58 AM, Djordje Senicic wrote: > - Add missing dependency on libedit > - Define LLVMVERSION on the same lines as GCCVERSION and other tools > - Use LLVMVERSION in mesa and meson.bbclass to get llvm version instead of > hardcoding it > - Use llvm patches unmodified from meta-clang, helps in keeping them in > sync > - Define PREFERRED_VERSION for llvm, llvm-native, nativesdk-llvm > > Signed-off-by: Khem Raj <raj.khem@gmail.com> > Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> > > The recipe was imported from the OpenEmbedded server > http://cgit.openembedded.org/openembedded-core/commit/meta/recipes-devtools/llvm/llvm_git.bb > commit_id 3c08b638348abd543fc92baf56c28ca16ae6aac6 > Currently this is needed for compiling pytorch glow with CPU backend which requires llvm7 or later. > To avoid being used globally name before 2020LTS, name of the recipe is not llvm_git but llvm8_git > > Signed-off-by: Djordje Senicic <x0157990@ti.com> > --- > ...ryInfo-Undefine-libc-functions-if-th.patch | 92 +++++++++ > ...-llvm-allow-env-override-of-exe-path.patch | 38 ++++ > recipes-devtools/glow/llvm-glow_git.bb | 180 ++++++++++++++++-- > 3 files changed, 298 insertions(+), 12 deletions(-) > create mode 100644 recipes-devtools/glow/llvm-glow/0006-llvm-TargetLibraryInfo-Undefine-libc-functions-if-th.patch > create mode 100644 recipes-devtools/glow/llvm-glow/0007-llvm-allow-env-override-of-exe-path.patch > > diff --git a/recipes-devtools/glow/llvm-glow/0006-llvm-TargetLibraryInfo-Undefine-libc-functions-if-th.patch b/recipes-devtools/glow/llvm-glow/0006-llvm-TargetLibraryInfo-Undefine-libc-functions-if-th.patch > new file mode 100644 > index 0000000..789c308 > --- /dev/null > +++ b/recipes-devtools/glow/llvm-glow/0006-llvm-TargetLibraryInfo-Undefine-libc-functions-if-th.patch > @@ -0,0 +1,92 @@ > +From 4c08879d2dfbe7face4e679ac8499dc7bff2dd20 Mon Sep 17 00:00:00 2001 > +From: Khem Raj <raj.khem@gmail.com> > +Date: Sat, 21 May 2016 00:33:20 +0000 > +Subject: [PATCH 06/19] llvm: TargetLibraryInfo: Undefine libc functions if > + they are macros > + > +musl defines some functions as macros and not inline functions > +if this is the case then make sure to undefine them > + > +Upstream-Status: Pending > +Signed-off-by: Khem Raj <raj.khem@gmail.com> > +--- > + .../llvm/Analysis/TargetLibraryInfo.def | 21 +++++++++++++++++++ > + 1 file changed, 21 insertions(+) > + > +diff --git a/llvm/include/llvm/Analysis/TargetLibraryInfo.def b/llvm/include/llvm/Analysis/TargetLibraryInfo.def > +index afed404f04c..876888656f2 100644 > +--- a/llvm/include/llvm/Analysis/TargetLibraryInfo.def > ++++ b/llvm/include/llvm/Analysis/TargetLibraryInfo.def > +@@ -782,6 +782,9 @@ TLI_DEFINE_STRING_INTERNAL("fmodl") > + TLI_DEFINE_ENUM_INTERNAL(fopen) > + TLI_DEFINE_STRING_INTERNAL("fopen") > + /// FILE *fopen64(const char *filename, const char *opentype) > ++#ifdef fopen64 > ++#undef fopen64 > ++#endif > + TLI_DEFINE_ENUM_INTERNAL(fopen64) > + TLI_DEFINE_STRING_INTERNAL("fopen64") > + /// int fork(); > +@@ -829,6 +832,9 @@ TLI_DEFINE_STRING_INTERNAL("fseek") > + /// int fseeko(FILE *stream, off_t offset, int whence); > + TLI_DEFINE_ENUM_INTERNAL(fseeko) > + TLI_DEFINE_STRING_INTERNAL("fseeko") > ++#ifdef fseeko64 > ++#undef fseeko64 > ++#endif > + /// int fseeko64(FILE *stream, off64_t offset, int whence) > + TLI_DEFINE_ENUM_INTERNAL(fseeko64) > + TLI_DEFINE_STRING_INTERNAL("fseeko64") > +@@ -839,6 +845,9 @@ TLI_DEFINE_STRING_INTERNAL("fsetpos") > + TLI_DEFINE_ENUM_INTERNAL(fstat) > + TLI_DEFINE_STRING_INTERNAL("fstat") > + /// int fstat64(int filedes, struct stat64 *buf) > ++#ifdef fstat64 > ++#undef fstat64 > ++#endif > + TLI_DEFINE_ENUM_INTERNAL(fstat64) > + TLI_DEFINE_STRING_INTERNAL("fstat64") > + /// int fstatvfs(int fildes, struct statvfs *buf); > +@@ -854,6 +863,9 @@ TLI_DEFINE_STRING_INTERNAL("ftell") > + TLI_DEFINE_ENUM_INTERNAL(ftello) > + TLI_DEFINE_STRING_INTERNAL("ftello") > + /// off64_t ftello64(FILE *stream) > ++#ifdef ftello64 > ++#undef ftello64 > ++#endif > + TLI_DEFINE_ENUM_INTERNAL(ftello64) > + TLI_DEFINE_STRING_INTERNAL("ftello64") > + /// int ftrylockfile(FILE *file); > +@@ -980,6 +992,9 @@ TLI_DEFINE_STRING_INTERNAL("logl") > + TLI_DEFINE_ENUM_INTERNAL(lstat) > + TLI_DEFINE_STRING_INTERNAL("lstat") > + /// int lstat64(const char *path, struct stat64 *buf); > ++#ifdef lstat64 > ++#undef lstat64 > ++#endif > + TLI_DEFINE_ENUM_INTERNAL(lstat64) > + TLI_DEFINE_STRING_INTERNAL("lstat64") > + /// void *malloc(size_t size); > +@@ -1205,6 +1220,9 @@ TLI_DEFINE_STRING_INTERNAL("sscanf") > + TLI_DEFINE_ENUM_INTERNAL(stat) > + TLI_DEFINE_STRING_INTERNAL("stat") > + /// int stat64(const char *path, struct stat64 *buf); > ++#ifdef stat64 > ++#undef stat64 > ++#endif > + TLI_DEFINE_ENUM_INTERNAL(stat64) > + TLI_DEFINE_STRING_INTERNAL("stat64") > + /// int statvfs(const char *path, struct statvfs *buf); > +@@ -1340,6 +1358,9 @@ TLI_DEFINE_STRING_INTERNAL("times") > + TLI_DEFINE_ENUM_INTERNAL(tmpfile) > + TLI_DEFINE_STRING_INTERNAL("tmpfile") > + /// FILE *tmpfile64(void) > ++#ifdef tmpfile64 > ++#undef tmpfile64 > ++#endif > + TLI_DEFINE_ENUM_INTERNAL(tmpfile64) > + TLI_DEFINE_STRING_INTERNAL("tmpfile64") > + /// int toascii(int c); > +-- > +2.22.0 > + > diff --git a/recipes-devtools/glow/llvm-glow/0007-llvm-allow-env-override-of-exe-path.patch b/recipes-devtools/glow/llvm-glow/0007-llvm-allow-env-override-of-exe-path.patch > new file mode 100644 > index 0000000..f47080c > --- /dev/null > +++ b/recipes-devtools/glow/llvm-glow/0007-llvm-allow-env-override-of-exe-path.patch > @@ -0,0 +1,38 @@ > +From b66d6f39a374b8df41e7235351e5dee2e81f440c Mon Sep 17 00:00:00 2001 > +From: Martin Kelly <mkelly@xevo.com> > +Date: Fri, 19 May 2017 00:22:57 -0700 > +Subject: [PATCH 07/19] llvm: allow env override of exe path > + > +When using a native llvm-config from inside a sysroot, we need llvm-config to > +return the libraries, include directories, etc. from inside the sysroot rather > +than from the native sysroot. Thus provide an env override for calling > +llvm-config from a target sysroot. > + > +Upstream-Status: Pending > +Signed-off-by: Martin Kelly <mkelly@xevo.com> > +Signed-off-by: Khem Raj <raj.khem@gmail.com> > +--- > + llvm/tools/llvm-config/llvm-config.cpp | 7 +++++++ > + 1 file changed, 7 insertions(+) > + > +diff --git a/llvm/tools/llvm-config/llvm-config.cpp b/llvm/tools/llvm-config/llvm-config.cpp > +index 7ef7c46a262..a4f7ed82c7b 100644 > +--- a/llvm/tools/llvm-config/llvm-config.cpp > ++++ b/llvm/tools/llvm-config/llvm-config.cpp > +@@ -225,6 +225,13 @@ Typical components:\n\ > + > + /// Compute the path to the main executable. > + std::string GetExecutablePath(const char *Argv0) { > ++ // Hack for Yocto: we need to override the root path when we are using > ++ // llvm-config from within a target sysroot. > ++ const char *Sysroot = std::getenv("YOCTO_ALTERNATE_EXE_PATH"); > ++ if (Sysroot != nullptr) { > ++ return Sysroot; > ++ } > ++ > + // This just needs to be some symbol in the binary; C++ doesn't > + // allow taking the address of ::main however. > + void *P = (void *)(intptr_t)GetExecutablePath; > +-- > +2.22.0 > + > diff --git a/recipes-devtools/glow/llvm-glow_git.bb b/recipes-devtools/glow/llvm-glow_git.bb > index 58d392b..d3c2642 100644 > --- a/recipes-devtools/glow/llvm-glow_git.bb > +++ b/recipes-devtools/glow/llvm-glow_git.bb > @@ -1,24 +1,180 @@ > -DESCRIPTION = "LLVM release suggested for pytorch glow neural network compiler" > -HOMEPAGE = "https://github.com/llvm-mirror/llvm" > +# Copyright (C) 2017 Khem Raj <raj.khem@gmail.com> > +# Released under the MIT license (see COPYING.MIT for the terms) > + > +DESCRIPTION = "The LLVM Compiler Infrastructure" > +HOMEPAGE = "http://llvm.org" > +LICENSE = "NCSA" > SECTION = "devel" > -LICENSE = "Apache-2.0" > + > LIC_FILES_CHKSUM = "file://LICENSE.TXT;md5=c6b766a4e85dd28301eeed54a6684648" > -PV = "8.0" > -PR = "r0" > > -DEPENDS = "ninja-native protobuf-native libpng-native protobuf-native protobuf-c-native gflags-native glog-native" > +DEPENDS = "libffi libxml2 zlib libedit ninja-native llvm8-native" > > -SOLIBVER = "1" > +RDEPENDS_${PN}_append_class-target = " ncurses-terminfo" > + > +inherit cmake pkgconfig > + > +PROVIDES += "llvm${PV}" > > -SRC_URI = "git://github.com/llvm-mirror/llvm.git;protocol=https;branch=release_80;rev=5563a6a769b48912b9c8052124520610966eda28;destsuffix=git/llvm \ > - git://github.com/llvm-mirror/clang.git;protocol=https;branch=release_80;rev=ccfe04576c13497b9c422ceef0b6efe99077a392;destsuffix=git/clang \ > - " > +LLVM_RELEASE = "${PV}" > +LLVM_DIR = "llvm${LLVM_RELEASE}" > > +SRCREV = "19a71f6bdf2dddb10764939e7f0ec2b98dba76c9" > + > +BRANCH = "release/${MAJOR_VERSION}.x" > +MAJOR_VERSION = "8" > +MINOR_VERSION = "0" > +PATCH_VERSION = "1" > +SOLIBVER = "1" > +PV = "${MAJOR_VERSION}.${MINOR_VERSION}.${PATCH_VERSION}" > +SRC_URI = "git://github.com/llvm/llvm-project.git;branch=${BRANCH} \ > + file://0006-llvm-TargetLibraryInfo-Undefine-libc-functions-if-th.patch;striplevel=2 \ > + file://0007-llvm-allow-env-override-of-exe-path.patch;striplevel=2 \ > + " > > S = "${WORKDIR}/git/llvm" > > -EXTRA_OECMAKE = "-DCMAKE_BUILD_TYPE=Release -DLLVM_INSTALL_UTILS=ON -DLLVM_ENABLE_PROJECTS=clang" > +LLVM_INSTALL_DIR = "${WORKDIR}/llvm-install" > + > +def get_llvm_arch(bb, d, arch_var): > + import re > + a = d.getVar(arch_var) > + if re.match(r'(i.86|athlon|x86.64)$', a): return 'X86' > + elif re.match(r'arm$', a): return 'ARM' > + elif re.match(r'armeb$', a): return 'ARM' > + elif re.match(r'aarch64$', a): return 'AArch64' > + elif re.match(r'aarch64_be$', a): return 'AArch64' > + elif re.match(r'mips(isa|)(32|64|)(r6|)(el|)$', a): return 'Mips' > + elif re.match(r'p(pc|owerpc)(|64)', a): return 'PowerPC' > + else: > + raise bb.parse.SkipRecipe("Cannot map '%s' to a supported LLVM architecture" % a) > + > +def get_llvm_host_arch(bb, d): > + return get_llvm_arch(bb, d, 'HOST_ARCH') > + > +# > +# Default to build all OE-Core supported target arches (user overridable). > +# > +LLVM_TARGETS ?= "AMDGPU;${@get_llvm_host_arch(bb, d)}" > + > +ARM_INSTRUCTION_SET_armv5 = "arm" > +ARM_INSTRUCTION_SET_armv4t = "arm" > + > +EXTRA_OECMAKE += "-DLLVM_ENABLE_ASSERTIONS=OFF \ > + -DLLVM_ENABLE_EXPENSIVE_CHECKS=OFF \ > + -DLLVM_ENABLE_PIC=ON \ > + -DLLVM_BINDINGS_LIST='' \ > + -DLLVM_LINK_LLVM_DYLIB=ON \ > + -DLLVM_ENABLE_FFI=ON \ > + -DLLVM_ENABLE_RTTI=ON \ > + -DFFI_INCLUDE_DIR=$(pkg-config --variable=includedir libffi) \ > + -DLLVM_OPTIMIZED_TABLEGEN=ON \ > + -DLLVM_TARGETS_TO_BUILD='${LLVM_TARGETS}' \ > + -DLLVM_TEMPORARILY_ALLOW_OLD_TOOLCHAIN=ON \ > + -DPYTHON_EXECUTABLE=${HOSTTOOLS_DIR}/python2 \ > + -G Ninja" > + > +EXTRA_OECMAKE_append_class-target = "\ > + -DCMAKE_CROSSCOMPILING:BOOL=ON \ > + -DLLVM_TABLEGEN=${STAGING_BINDIR_NATIVE}/llvm-tblgen${PV} \ > + -DLLVM_CONFIG_PATH=${STAGING_BINDIR_NATIVE}/llvm-config${PV} \ > + " > + > +EXTRA_OECMAKE_append_class-nativesdk = "\ > + -DCMAKE_CROSSCOMPILING:BOOL=ON \ > + -DLLVM_TABLEGEN=${STAGING_BINDIR_NATIVE}/llvm-tblgen${PV} \ > + -DLLVM_CONFIG_PATH=${STAGING_BINDIR_NATIVE}/llvm-config${PV} \ > + " > + > +CXXFLAGS_append_class-target_powerpc = " -mlongcall" > + > +do_configure_prepend() { > +# Fix paths in llvm-config > + sed -i "s|sys::path::parent_path(CurrentPath))\.str()|sys::path::parent_path(sys::path::parent_path(CurrentPath))).str()|g" ${S}/tools/llvm-config/llvm-config.cpp > + sed -ri "s#/(bin|include|lib)(/?\")#/\1/${LLVM_DIR}\2#g" ${S}/tools/llvm-config/llvm-config.cpp > + sed -ri "s#lib/${LLVM_DIR}#${baselib}/${LLVM_DIR}#g" ${S}/tools/llvm-config/llvm-config.cpp > +} > + > +do_compile() { > + ninja -v ${PARALLEL_MAKE} > +} > + > +do_compile_class-native() { > + ninja -v ${PARALLEL_MAKE} llvm-config llvm-tblgen > +} > + > +do_install() { > + DESTDIR=${LLVM_INSTALL_DIR} ninja -v install > + install -D -m 0755 ${B}/bin/llvm-config ${D}${libdir}/${LLVM_DIR}/llvm-config > + > + install -d ${D}${bindir}/${LLVM_DIR} > + cp -r ${LLVM_INSTALL_DIR}${bindir}/* ${D}${bindir}/${LLVM_DIR}/ > + > + install -d ${D}${includedir}/${LLVM_DIR} > + cp -r ${LLVM_INSTALL_DIR}${includedir}/* ${D}${includedir}/${LLVM_DIR}/ > + > + install -d ${D}${libdir}/${LLVM_DIR} > + > + # The LLVM sources have "/lib" embedded and so we cannot completely rely on the ${libdir} variable > + if [ -d ${LLVM_INSTALL_DIR}${libdir}/ ]; then > + cp -r ${LLVM_INSTALL_DIR}${libdir}/* ${D}${libdir}/${LLVM_DIR}/ > + elif [ -d ${LLVM_INSTALL_DIR}${prefix}/lib ]; then > + cp -r ${LLVM_INSTALL_DIR}${prefix}/lib/* ${D}${libdir}/${LLVM_DIR}/ > + elif [ -d ${LLVM_INSTALL_DIR}${prefix}/lib64 ]; then > + cp -r ${LLVM_INSTALL_DIR}${prefix}/lib64/* ${D}${libdir}/${LLVM_DIR}/ > + fi > + > + # Remove unnecessary cmake files > + rm -rf ${D}${libdir}/${LLVM_DIR}/cmake > + > + ln -s ${LLVM_DIR}/libLLVM-${MAJOR_VERSION}${SOLIBSDEV} ${D}${libdir}/libLLVM-${MAJOR_VERSION}${SOLIBSDEV} > + > + # We'll have to delete the libLLVM.so due to multiple reasons... > + rm -rf ${D}${libdir}/${LLVM_DIR}/libLLVM.so > + rm -rf ${D}${libdir}/${LLVM_DIR}/libLTO.so > +} > + > +do_install_class-native() { > + install -D -m 0755 ${B}/bin/llvm-tblgen ${D}${bindir}/llvm-tblgen${PV} > + install -D -m 0755 ${B}/bin/llvm-config ${D}${bindir}/llvm-config${PV} > + install -D -m 0755 ${B}/lib/libLLVM-${MAJOR_VERSION}.so ${D}${libdir}/libLLVM-${MAJOR_VERSION}.so > +} > + > +PACKAGES =+ "${PN}-bugpointpasses ${PN}-llvmhello ${PN}-libllvm ${PN}-liboptremarks ${PN}-liblto" > + > +RRECOMMENDS_${PN}-dev += "${PN}-bugpointpasses ${PN}-llvmhello ${PN}-liboptremarks" > + > +FILES_${PN}-bugpointpasses = "\ > + ${libdir}/${LLVM_DIR}/BugpointPasses.so \ > +" > + > +FILES_${PN}-libllvm = "\ > + ${libdir}/${LLVM_DIR}/libLLVM-${MAJOR_VERSION}.so \ > + ${libdir}/libLLVM-${MAJOR_VERSION}.so \ > +" > + > +FILES_${PN}-liblto += "\ > + ${libdir}/${LLVM_DIR}/libLTO.so.* \ > +" > + > +FILES_${PN}-liboptremarks += "\ > + ${libdir}/${LLVM_DIR}/libOptRemarks.so.* \ > +" > + > +FILES_${PN}-llvmhello = "\ > + ${libdir}/${LLVM_DIR}/LLVMHello.so \ > +" > + > +FILES_${PN}-dev += " \ > + ${libdir}/${LLVM_DIR}/llvm-config \ > + ${libdir}/${LLVM_DIR}/libOptRemarks.so \ > + ${libdir}/${LLVM_DIR}/libLLVM-${MAJOR_VERSION}.${MINOR_VERSION}.${PATCH_VERSION}.so \ > +" > + > +FILES_${PN}-staticdev += "\ > + ${libdir}/${LLVM_DIR}/*.a \ > +" > > -inherit pkgconfig cmake > +INSANE_SKIP_${PN}-libllvm += "dev-so" > > BBCLASSEXTEND = "native nativesdk" ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [EXTERNAL] [meta-processor-sdk][PATCHv2 1/5] mesa, llvm, meson: Update llvm to 8.0.1 plus define and use LLVM version globally 2019-08-23 17:34 ` Senicic, Djordje @ 2019-08-23 18:06 ` Jacob Stiffler 2019-08-23 18:17 ` Senicic, Djordje 0 siblings, 1 reply; 12+ messages in thread From: Jacob Stiffler @ 2019-08-23 18:06 UTC (permalink / raw) To: Senicic, Djordje, meta-arago Did you manually add BBCLASSEXTEND to libedit? I do not see it on thud. On 8/23/2019 1:34 PM, Senicic, Djordje wrote: > I could build both nativesdk-libedit and nativesdk-llvm7 > There is: ./meta-openembedded/meta-oe/recipes-devtools/libedit/libedit_20180525-3.1.bb > > -----Original Message----- > From: Stiffler, Jacob > Sent: Friday, August 23, 2019 12:16 PM > To: Senicic, Djordje; meta-arago@arago-project.org > Cc: Senicic, Djordje > Subject: Re: [EXTERNAL] [meta-arago] [meta-processor-sdk][PATCHv2 1/5] mesa, llvm, meson: Update llvm to 8.0.1 plus define and use LLVM version globally > > Djordje, > > I get an error with this. Have you tried building nativesdk-llvm8? > > ERROR: Nothing PROVIDES 'nativesdk-libedit' (but > virtual:nativesdk:/oe/bld/sources/meta-processor-sdk/recipes-devtools/glow/llvm8_git.bb > DEPENDS on or otherwise requires it). > > > And when using "git send-email", you can use the "--no-signed-off-by-cc" > flag to prevent CCing those that have "signed-off" in the commit message. > > > - Jake > > > On 8/23/2019 9:58 AM, Djordje Senicic wrote: >> - Add missing dependency on libedit >> - Define LLVMVERSION on the same lines as GCCVERSION and other tools >> - Use LLVMVERSION in mesa and meson.bbclass to get llvm version instead of >> hardcoding it >> - Use llvm patches unmodified from meta-clang, helps in keeping them in >> sync >> - Define PREFERRED_VERSION for llvm, llvm-native, nativesdk-llvm >> >> Signed-off-by: Khem Raj <raj.khem@gmail.com> >> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> >> >> The recipe was imported from the OpenEmbedded server >> http://cgit.openembedded.org/openembedded-core/commit/meta/recipes-devtools/llvm/llvm_git.bb >> commit_id 3c08b638348abd543fc92baf56c28ca16ae6aac6 >> Currently this is needed for compiling pytorch glow with CPU backend which requires llvm7 or later. >> To avoid being used globally name before 2020LTS, name of the recipe is not llvm_git but llvm8_git >> >> Signed-off-by: Djordje Senicic <x0157990@ti.com> >> --- >> ...ryInfo-Undefine-libc-functions-if-th.patch | 92 +++++++++ >> ...-llvm-allow-env-override-of-exe-path.patch | 38 ++++ >> recipes-devtools/glow/llvm-glow_git.bb | 180 ++++++++++++++++-- >> 3 files changed, 298 insertions(+), 12 deletions(-) >> create mode 100644 recipes-devtools/glow/llvm-glow/0006-llvm-TargetLibraryInfo-Undefine-libc-functions-if-th.patch >> create mode 100644 recipes-devtools/glow/llvm-glow/0007-llvm-allow-env-override-of-exe-path.patch >> >> diff --git a/recipes-devtools/glow/llvm-glow/0006-llvm-TargetLibraryInfo-Undefine-libc-functions-if-th.patch b/recipes-devtools/glow/llvm-glow/0006-llvm-TargetLibraryInfo-Undefine-libc-functions-if-th.patch >> new file mode 100644 >> index 0000000..789c308 >> --- /dev/null >> +++ b/recipes-devtools/glow/llvm-glow/0006-llvm-TargetLibraryInfo-Undefine-libc-functions-if-th.patch >> @@ -0,0 +1,92 @@ >> +From 4c08879d2dfbe7face4e679ac8499dc7bff2dd20 Mon Sep 17 00:00:00 2001 >> +From: Khem Raj <raj.khem@gmail.com> >> +Date: Sat, 21 May 2016 00:33:20 +0000 >> +Subject: [PATCH 06/19] llvm: TargetLibraryInfo: Undefine libc functions if >> + they are macros >> + >> +musl defines some functions as macros and not inline functions >> +if this is the case then make sure to undefine them >> + >> +Upstream-Status: Pending >> +Signed-off-by: Khem Raj <raj.khem@gmail.com> >> +--- >> + .../llvm/Analysis/TargetLibraryInfo.def | 21 +++++++++++++++++++ >> + 1 file changed, 21 insertions(+) >> + >> +diff --git a/llvm/include/llvm/Analysis/TargetLibraryInfo.def b/llvm/include/llvm/Analysis/TargetLibraryInfo.def >> +index afed404f04c..876888656f2 100644 >> +--- a/llvm/include/llvm/Analysis/TargetLibraryInfo.def >> ++++ b/llvm/include/llvm/Analysis/TargetLibraryInfo.def >> +@@ -782,6 +782,9 @@ TLI_DEFINE_STRING_INTERNAL("fmodl") >> + TLI_DEFINE_ENUM_INTERNAL(fopen) >> + TLI_DEFINE_STRING_INTERNAL("fopen") >> + /// FILE *fopen64(const char *filename, const char *opentype) >> ++#ifdef fopen64 >> ++#undef fopen64 >> ++#endif >> + TLI_DEFINE_ENUM_INTERNAL(fopen64) >> + TLI_DEFINE_STRING_INTERNAL("fopen64") >> + /// int fork(); >> +@@ -829,6 +832,9 @@ TLI_DEFINE_STRING_INTERNAL("fseek") >> + /// int fseeko(FILE *stream, off_t offset, int whence); >> + TLI_DEFINE_ENUM_INTERNAL(fseeko) >> + TLI_DEFINE_STRING_INTERNAL("fseeko") >> ++#ifdef fseeko64 >> ++#undef fseeko64 >> ++#endif >> + /// int fseeko64(FILE *stream, off64_t offset, int whence) >> + TLI_DEFINE_ENUM_INTERNAL(fseeko64) >> + TLI_DEFINE_STRING_INTERNAL("fseeko64") >> +@@ -839,6 +845,9 @@ TLI_DEFINE_STRING_INTERNAL("fsetpos") >> + TLI_DEFINE_ENUM_INTERNAL(fstat) >> + TLI_DEFINE_STRING_INTERNAL("fstat") >> + /// int fstat64(int filedes, struct stat64 *buf) >> ++#ifdef fstat64 >> ++#undef fstat64 >> ++#endif >> + TLI_DEFINE_ENUM_INTERNAL(fstat64) >> + TLI_DEFINE_STRING_INTERNAL("fstat64") >> + /// int fstatvfs(int fildes, struct statvfs *buf); >> +@@ -854,6 +863,9 @@ TLI_DEFINE_STRING_INTERNAL("ftell") >> + TLI_DEFINE_ENUM_INTERNAL(ftello) >> + TLI_DEFINE_STRING_INTERNAL("ftello") >> + /// off64_t ftello64(FILE *stream) >> ++#ifdef ftello64 >> ++#undef ftello64 >> ++#endif >> + TLI_DEFINE_ENUM_INTERNAL(ftello64) >> + TLI_DEFINE_STRING_INTERNAL("ftello64") >> + /// int ftrylockfile(FILE *file); >> +@@ -980,6 +992,9 @@ TLI_DEFINE_STRING_INTERNAL("logl") >> + TLI_DEFINE_ENUM_INTERNAL(lstat) >> + TLI_DEFINE_STRING_INTERNAL("lstat") >> + /// int lstat64(const char *path, struct stat64 *buf); >> ++#ifdef lstat64 >> ++#undef lstat64 >> ++#endif >> + TLI_DEFINE_ENUM_INTERNAL(lstat64) >> + TLI_DEFINE_STRING_INTERNAL("lstat64") >> + /// void *malloc(size_t size); >> +@@ -1205,6 +1220,9 @@ TLI_DEFINE_STRING_INTERNAL("sscanf") >> + TLI_DEFINE_ENUM_INTERNAL(stat) >> + TLI_DEFINE_STRING_INTERNAL("stat") >> + /// int stat64(const char *path, struct stat64 *buf); >> ++#ifdef stat64 >> ++#undef stat64 >> ++#endif >> + TLI_DEFINE_ENUM_INTERNAL(stat64) >> + TLI_DEFINE_STRING_INTERNAL("stat64") >> + /// int statvfs(const char *path, struct statvfs *buf); >> +@@ -1340,6 +1358,9 @@ TLI_DEFINE_STRING_INTERNAL("times") >> + TLI_DEFINE_ENUM_INTERNAL(tmpfile) >> + TLI_DEFINE_STRING_INTERNAL("tmpfile") >> + /// FILE *tmpfile64(void) >> ++#ifdef tmpfile64 >> ++#undef tmpfile64 >> ++#endif >> + TLI_DEFINE_ENUM_INTERNAL(tmpfile64) >> + TLI_DEFINE_STRING_INTERNAL("tmpfile64") >> + /// int toascii(int c); >> +-- >> +2.22.0 >> + >> diff --git a/recipes-devtools/glow/llvm-glow/0007-llvm-allow-env-override-of-exe-path.patch b/recipes-devtools/glow/llvm-glow/0007-llvm-allow-env-override-of-exe-path.patch >> new file mode 100644 >> index 0000000..f47080c >> --- /dev/null >> +++ b/recipes-devtools/glow/llvm-glow/0007-llvm-allow-env-override-of-exe-path.patch >> @@ -0,0 +1,38 @@ >> +From b66d6f39a374b8df41e7235351e5dee2e81f440c Mon Sep 17 00:00:00 2001 >> +From: Martin Kelly <mkelly@xevo.com> >> +Date: Fri, 19 May 2017 00:22:57 -0700 >> +Subject: [PATCH 07/19] llvm: allow env override of exe path >> + >> +When using a native llvm-config from inside a sysroot, we need llvm-config to >> +return the libraries, include directories, etc. from inside the sysroot rather >> +than from the native sysroot. Thus provide an env override for calling >> +llvm-config from a target sysroot. >> + >> +Upstream-Status: Pending >> +Signed-off-by: Martin Kelly <mkelly@xevo.com> >> +Signed-off-by: Khem Raj <raj.khem@gmail.com> >> +--- >> + llvm/tools/llvm-config/llvm-config.cpp | 7 +++++++ >> + 1 file changed, 7 insertions(+) >> + >> +diff --git a/llvm/tools/llvm-config/llvm-config.cpp b/llvm/tools/llvm-config/llvm-config.cpp >> +index 7ef7c46a262..a4f7ed82c7b 100644 >> +--- a/llvm/tools/llvm-config/llvm-config.cpp >> ++++ b/llvm/tools/llvm-config/llvm-config.cpp >> +@@ -225,6 +225,13 @@ Typical components:\n\ >> + >> + /// Compute the path to the main executable. >> + std::string GetExecutablePath(const char *Argv0) { >> ++ // Hack for Yocto: we need to override the root path when we are using >> ++ // llvm-config from within a target sysroot. >> ++ const char *Sysroot = std::getenv("YOCTO_ALTERNATE_EXE_PATH"); >> ++ if (Sysroot != nullptr) { >> ++ return Sysroot; >> ++ } >> ++ >> + // This just needs to be some symbol in the binary; C++ doesn't >> + // allow taking the address of ::main however. >> + void *P = (void *)(intptr_t)GetExecutablePath; >> +-- >> +2.22.0 >> + >> diff --git a/recipes-devtools/glow/llvm-glow_git.bb b/recipes-devtools/glow/llvm-glow_git.bb >> index 58d392b..d3c2642 100644 >> --- a/recipes-devtools/glow/llvm-glow_git.bb >> +++ b/recipes-devtools/glow/llvm-glow_git.bb >> @@ -1,24 +1,180 @@ >> -DESCRIPTION = "LLVM release suggested for pytorch glow neural network compiler" >> -HOMEPAGE = "https://github.com/llvm-mirror/llvm" >> +# Copyright (C) 2017 Khem Raj <raj.khem@gmail.com> >> +# Released under the MIT license (see COPYING.MIT for the terms) >> + >> +DESCRIPTION = "The LLVM Compiler Infrastructure" >> +HOMEPAGE = "http://llvm.org" >> +LICENSE = "NCSA" >> SECTION = "devel" >> -LICENSE = "Apache-2.0" >> + >> LIC_FILES_CHKSUM = "file://LICENSE.TXT;md5=c6b766a4e85dd28301eeed54a6684648" >> -PV = "8.0" >> -PR = "r0" >> >> -DEPENDS = "ninja-native protobuf-native libpng-native protobuf-native protobuf-c-native gflags-native glog-native" >> +DEPENDS = "libffi libxml2 zlib libedit ninja-native llvm8-native" >> >> -SOLIBVER = "1" >> +RDEPENDS_${PN}_append_class-target = " ncurses-terminfo" >> + >> +inherit cmake pkgconfig >> + >> +PROVIDES += "llvm${PV}" >> >> -SRC_URI = "git://github.com/llvm-mirror/llvm.git;protocol=https;branch=release_80;rev=5563a6a769b48912b9c8052124520610966eda28;destsuffix=git/llvm \ >> - git://github.com/llvm-mirror/clang.git;protocol=https;branch=release_80;rev=ccfe04576c13497b9c422ceef0b6efe99077a392;destsuffix=git/clang \ >> - " >> +LLVM_RELEASE = "${PV}" >> +LLVM_DIR = "llvm${LLVM_RELEASE}" >> >> +SRCREV = "19a71f6bdf2dddb10764939e7f0ec2b98dba76c9" >> + >> +BRANCH = "release/${MAJOR_VERSION}.x" >> +MAJOR_VERSION = "8" >> +MINOR_VERSION = "0" >> +PATCH_VERSION = "1" >> +SOLIBVER = "1" >> +PV = "${MAJOR_VERSION}.${MINOR_VERSION}.${PATCH_VERSION}" >> +SRC_URI = "git://github.com/llvm/llvm-project.git;branch=${BRANCH} \ >> + file://0006-llvm-TargetLibraryInfo-Undefine-libc-functions-if-th.patch;striplevel=2 \ >> + file://0007-llvm-allow-env-override-of-exe-path.patch;striplevel=2 \ >> + " >> >> S = "${WORKDIR}/git/llvm" >> >> -EXTRA_OECMAKE = "-DCMAKE_BUILD_TYPE=Release -DLLVM_INSTALL_UTILS=ON -DLLVM_ENABLE_PROJECTS=clang" >> +LLVM_INSTALL_DIR = "${WORKDIR}/llvm-install" >> + >> +def get_llvm_arch(bb, d, arch_var): >> + import re >> + a = d.getVar(arch_var) >> + if re.match(r'(i.86|athlon|x86.64)$', a): return 'X86' >> + elif re.match(r'arm$', a): return 'ARM' >> + elif re.match(r'armeb$', a): return 'ARM' >> + elif re.match(r'aarch64$', a): return 'AArch64' >> + elif re.match(r'aarch64_be$', a): return 'AArch64' >> + elif re.match(r'mips(isa|)(32|64|)(r6|)(el|)$', a): return 'Mips' >> + elif re.match(r'p(pc|owerpc)(|64)', a): return 'PowerPC' >> + else: >> + raise bb.parse.SkipRecipe("Cannot map '%s' to a supported LLVM architecture" % a) >> + >> +def get_llvm_host_arch(bb, d): >> + return get_llvm_arch(bb, d, 'HOST_ARCH') >> + >> +# >> +# Default to build all OE-Core supported target arches (user overridable). >> +# >> +LLVM_TARGETS ?= "AMDGPU;${@get_llvm_host_arch(bb, d)}" >> + >> +ARM_INSTRUCTION_SET_armv5 = "arm" >> +ARM_INSTRUCTION_SET_armv4t = "arm" >> + >> +EXTRA_OECMAKE += "-DLLVM_ENABLE_ASSERTIONS=OFF \ >> + -DLLVM_ENABLE_EXPENSIVE_CHECKS=OFF \ >> + -DLLVM_ENABLE_PIC=ON \ >> + -DLLVM_BINDINGS_LIST='' \ >> + -DLLVM_LINK_LLVM_DYLIB=ON \ >> + -DLLVM_ENABLE_FFI=ON \ >> + -DLLVM_ENABLE_RTTI=ON \ >> + -DFFI_INCLUDE_DIR=$(pkg-config --variable=includedir libffi) \ >> + -DLLVM_OPTIMIZED_TABLEGEN=ON \ >> + -DLLVM_TARGETS_TO_BUILD='${LLVM_TARGETS}' \ >> + -DLLVM_TEMPORARILY_ALLOW_OLD_TOOLCHAIN=ON \ >> + -DPYTHON_EXECUTABLE=${HOSTTOOLS_DIR}/python2 \ >> + -G Ninja" >> + >> +EXTRA_OECMAKE_append_class-target = "\ >> + -DCMAKE_CROSSCOMPILING:BOOL=ON \ >> + -DLLVM_TABLEGEN=${STAGING_BINDIR_NATIVE}/llvm-tblgen${PV} \ >> + -DLLVM_CONFIG_PATH=${STAGING_BINDIR_NATIVE}/llvm-config${PV} \ >> + " >> + >> +EXTRA_OECMAKE_append_class-nativesdk = "\ >> + -DCMAKE_CROSSCOMPILING:BOOL=ON \ >> + -DLLVM_TABLEGEN=${STAGING_BINDIR_NATIVE}/llvm-tblgen${PV} \ >> + -DLLVM_CONFIG_PATH=${STAGING_BINDIR_NATIVE}/llvm-config${PV} \ >> + " >> + >> +CXXFLAGS_append_class-target_powerpc = " -mlongcall" >> + >> +do_configure_prepend() { >> +# Fix paths in llvm-config >> + sed -i "s|sys::path::parent_path(CurrentPath))\.str()|sys::path::parent_path(sys::path::parent_path(CurrentPath))).str()|g" ${S}/tools/llvm-config/llvm-config.cpp >> + sed -ri "s#/(bin|include|lib)(/?\")#/\1/${LLVM_DIR}\2#g" ${S}/tools/llvm-config/llvm-config.cpp >> + sed -ri "s#lib/${LLVM_DIR}#${baselib}/${LLVM_DIR}#g" ${S}/tools/llvm-config/llvm-config.cpp >> +} >> + >> +do_compile() { >> + ninja -v ${PARALLEL_MAKE} >> +} >> + >> +do_compile_class-native() { >> + ninja -v ${PARALLEL_MAKE} llvm-config llvm-tblgen >> +} >> + >> +do_install() { >> + DESTDIR=${LLVM_INSTALL_DIR} ninja -v install >> + install -D -m 0755 ${B}/bin/llvm-config ${D}${libdir}/${LLVM_DIR}/llvm-config >> + >> + install -d ${D}${bindir}/${LLVM_DIR} >> + cp -r ${LLVM_INSTALL_DIR}${bindir}/* ${D}${bindir}/${LLVM_DIR}/ >> + >> + install -d ${D}${includedir}/${LLVM_DIR} >> + cp -r ${LLVM_INSTALL_DIR}${includedir}/* ${D}${includedir}/${LLVM_DIR}/ >> + >> + install -d ${D}${libdir}/${LLVM_DIR} >> + >> + # The LLVM sources have "/lib" embedded and so we cannot completely rely on the ${libdir} variable >> + if [ -d ${LLVM_INSTALL_DIR}${libdir}/ ]; then >> + cp -r ${LLVM_INSTALL_DIR}${libdir}/* ${D}${libdir}/${LLVM_DIR}/ >> + elif [ -d ${LLVM_INSTALL_DIR}${prefix}/lib ]; then >> + cp -r ${LLVM_INSTALL_DIR}${prefix}/lib/* ${D}${libdir}/${LLVM_DIR}/ >> + elif [ -d ${LLVM_INSTALL_DIR}${prefix}/lib64 ]; then >> + cp -r ${LLVM_INSTALL_DIR}${prefix}/lib64/* ${D}${libdir}/${LLVM_DIR}/ >> + fi >> + >> + # Remove unnecessary cmake files >> + rm -rf ${D}${libdir}/${LLVM_DIR}/cmake >> + >> + ln -s ${LLVM_DIR}/libLLVM-${MAJOR_VERSION}${SOLIBSDEV} ${D}${libdir}/libLLVM-${MAJOR_VERSION}${SOLIBSDEV} >> + >> + # We'll have to delete the libLLVM.so due to multiple reasons... >> + rm -rf ${D}${libdir}/${LLVM_DIR}/libLLVM.so >> + rm -rf ${D}${libdir}/${LLVM_DIR}/libLTO.so >> +} >> + >> +do_install_class-native() { >> + install -D -m 0755 ${B}/bin/llvm-tblgen ${D}${bindir}/llvm-tblgen${PV} >> + install -D -m 0755 ${B}/bin/llvm-config ${D}${bindir}/llvm-config${PV} >> + install -D -m 0755 ${B}/lib/libLLVM-${MAJOR_VERSION}.so ${D}${libdir}/libLLVM-${MAJOR_VERSION}.so >> +} >> + >> +PACKAGES =+ "${PN}-bugpointpasses ${PN}-llvmhello ${PN}-libllvm ${PN}-liboptremarks ${PN}-liblto" >> + >> +RRECOMMENDS_${PN}-dev += "${PN}-bugpointpasses ${PN}-llvmhello ${PN}-liboptremarks" >> + >> +FILES_${PN}-bugpointpasses = "\ >> + ${libdir}/${LLVM_DIR}/BugpointPasses.so \ >> +" >> + >> +FILES_${PN}-libllvm = "\ >> + ${libdir}/${LLVM_DIR}/libLLVM-${MAJOR_VERSION}.so \ >> + ${libdir}/libLLVM-${MAJOR_VERSION}.so \ >> +" >> + >> +FILES_${PN}-liblto += "\ >> + ${libdir}/${LLVM_DIR}/libLTO.so.* \ >> +" >> + >> +FILES_${PN}-liboptremarks += "\ >> + ${libdir}/${LLVM_DIR}/libOptRemarks.so.* \ >> +" >> + >> +FILES_${PN}-llvmhello = "\ >> + ${libdir}/${LLVM_DIR}/LLVMHello.so \ >> +" >> + >> +FILES_${PN}-dev += " \ >> + ${libdir}/${LLVM_DIR}/llvm-config \ >> + ${libdir}/${LLVM_DIR}/libOptRemarks.so \ >> + ${libdir}/${LLVM_DIR}/libLLVM-${MAJOR_VERSION}.${MINOR_VERSION}.${PATCH_VERSION}.so \ >> +" >> + >> +FILES_${PN}-staticdev += "\ >> + ${libdir}/${LLVM_DIR}/*.a \ >> +" >> >> -inherit pkgconfig cmake >> +INSANE_SKIP_${PN}-libllvm += "dev-so" >> >> BBCLASSEXTEND = "native nativesdk" ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [EXTERNAL] [meta-processor-sdk][PATCHv2 1/5] mesa, llvm, meson: Update llvm to 8.0.1 plus define and use LLVM version globally 2019-08-23 18:06 ` Jacob Stiffler @ 2019-08-23 18:17 ` Senicic, Djordje 0 siblings, 0 replies; 12+ messages in thread From: Senicic, Djordje @ 2019-08-23 18:17 UTC (permalink / raw) To: Stiffler, Jacob, meta-arago That is correct - this was manually modified instead of creating bbappend in meta-processor-sdk. I'll submit PATCHv3 1/5, to address this gap -----Original Message----- From: Stiffler, Jacob Sent: Friday, August 23, 2019 2:06 PM To: Senicic, Djordje; meta-arago@arago-project.org Subject: Re: [EXTERNAL] [meta-arago] [meta-processor-sdk][PATCHv2 1/5] mesa, llvm, meson: Update llvm to 8.0.1 plus define and use LLVM version globally Did you manually add BBCLASSEXTEND to libedit? I do not see it on thud. On 8/23/2019 1:34 PM, Senicic, Djordje wrote: > I could build both nativesdk-libedit and nativesdk-llvm7 > There is: ./meta-openembedded/meta-oe/recipes-devtools/libedit/libedit_20180525-3.1.bb > > -----Original Message----- > From: Stiffler, Jacob > Sent: Friday, August 23, 2019 12:16 PM > To: Senicic, Djordje; meta-arago@arago-project.org > Cc: Senicic, Djordje > Subject: Re: [EXTERNAL] [meta-arago] [meta-processor-sdk][PATCHv2 1/5] mesa, llvm, meson: Update llvm to 8.0.1 plus define and use LLVM version globally > > Djordje, > > I get an error with this. Have you tried building nativesdk-llvm8? > > ERROR: Nothing PROVIDES 'nativesdk-libedit' (but > virtual:nativesdk:/oe/bld/sources/meta-processor-sdk/recipes-devtools/glow/llvm8_git.bb > DEPENDS on or otherwise requires it). > > > And when using "git send-email", you can use the "--no-signed-off-by-cc" > flag to prevent CCing those that have "signed-off" in the commit message. > > > - Jake > > > On 8/23/2019 9:58 AM, Djordje Senicic wrote: >> - Add missing dependency on libedit >> - Define LLVMVERSION on the same lines as GCCVERSION and other tools >> - Use LLVMVERSION in mesa and meson.bbclass to get llvm version instead of >> hardcoding it >> - Use llvm patches unmodified from meta-clang, helps in keeping them in >> sync >> - Define PREFERRED_VERSION for llvm, llvm-native, nativesdk-llvm >> >> Signed-off-by: Khem Raj <raj.khem@gmail.com> >> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> >> >> The recipe was imported from the OpenEmbedded server >> http://cgit.openembedded.org/openembedded-core/commit/meta/recipes-devtools/llvm/llvm_git.bb >> commit_id 3c08b638348abd543fc92baf56c28ca16ae6aac6 >> Currently this is needed for compiling pytorch glow with CPU backend which requires llvm7 or later. >> To avoid being used globally name before 2020LTS, name of the recipe is not llvm_git but llvm8_git >> >> Signed-off-by: Djordje Senicic <x0157990@ti.com> >> --- >> ...ryInfo-Undefine-libc-functions-if-th.patch | 92 +++++++++ >> ...-llvm-allow-env-override-of-exe-path.patch | 38 ++++ >> recipes-devtools/glow/llvm-glow_git.bb | 180 ++++++++++++++++-- >> 3 files changed, 298 insertions(+), 12 deletions(-) >> create mode 100644 recipes-devtools/glow/llvm-glow/0006-llvm-TargetLibraryInfo-Undefine-libc-functions-if-th.patch >> create mode 100644 recipes-devtools/glow/llvm-glow/0007-llvm-allow-env-override-of-exe-path.patch >> >> diff --git a/recipes-devtools/glow/llvm-glow/0006-llvm-TargetLibraryInfo-Undefine-libc-functions-if-th.patch b/recipes-devtools/glow/llvm-glow/0006-llvm-TargetLibraryInfo-Undefine-libc-functions-if-th.patch >> new file mode 100644 >> index 0000000..789c308 >> --- /dev/null >> +++ b/recipes-devtools/glow/llvm-glow/0006-llvm-TargetLibraryInfo-Undefine-libc-functions-if-th.patch >> @@ -0,0 +1,92 @@ >> +From 4c08879d2dfbe7face4e679ac8499dc7bff2dd20 Mon Sep 17 00:00:00 2001 >> +From: Khem Raj <raj.khem@gmail.com> >> +Date: Sat, 21 May 2016 00:33:20 +0000 >> +Subject: [PATCH 06/19] llvm: TargetLibraryInfo: Undefine libc functions if >> + they are macros >> + >> +musl defines some functions as macros and not inline functions >> +if this is the case then make sure to undefine them >> + >> +Upstream-Status: Pending >> +Signed-off-by: Khem Raj <raj.khem@gmail.com> >> +--- >> + .../llvm/Analysis/TargetLibraryInfo.def | 21 +++++++++++++++++++ >> + 1 file changed, 21 insertions(+) >> + >> +diff --git a/llvm/include/llvm/Analysis/TargetLibraryInfo.def b/llvm/include/llvm/Analysis/TargetLibraryInfo.def >> +index afed404f04c..876888656f2 100644 >> +--- a/llvm/include/llvm/Analysis/TargetLibraryInfo.def >> ++++ b/llvm/include/llvm/Analysis/TargetLibraryInfo.def >> +@@ -782,6 +782,9 @@ TLI_DEFINE_STRING_INTERNAL("fmodl") >> + TLI_DEFINE_ENUM_INTERNAL(fopen) >> + TLI_DEFINE_STRING_INTERNAL("fopen") >> + /// FILE *fopen64(const char *filename, const char *opentype) >> ++#ifdef fopen64 >> ++#undef fopen64 >> ++#endif >> + TLI_DEFINE_ENUM_INTERNAL(fopen64) >> + TLI_DEFINE_STRING_INTERNAL("fopen64") >> + /// int fork(); >> +@@ -829,6 +832,9 @@ TLI_DEFINE_STRING_INTERNAL("fseek") >> + /// int fseeko(FILE *stream, off_t offset, int whence); >> + TLI_DEFINE_ENUM_INTERNAL(fseeko) >> + TLI_DEFINE_STRING_INTERNAL("fseeko") >> ++#ifdef fseeko64 >> ++#undef fseeko64 >> ++#endif >> + /// int fseeko64(FILE *stream, off64_t offset, int whence) >> + TLI_DEFINE_ENUM_INTERNAL(fseeko64) >> + TLI_DEFINE_STRING_INTERNAL("fseeko64") >> +@@ -839,6 +845,9 @@ TLI_DEFINE_STRING_INTERNAL("fsetpos") >> + TLI_DEFINE_ENUM_INTERNAL(fstat) >> + TLI_DEFINE_STRING_INTERNAL("fstat") >> + /// int fstat64(int filedes, struct stat64 *buf) >> ++#ifdef fstat64 >> ++#undef fstat64 >> ++#endif >> + TLI_DEFINE_ENUM_INTERNAL(fstat64) >> + TLI_DEFINE_STRING_INTERNAL("fstat64") >> + /// int fstatvfs(int fildes, struct statvfs *buf); >> +@@ -854,6 +863,9 @@ TLI_DEFINE_STRING_INTERNAL("ftell") >> + TLI_DEFINE_ENUM_INTERNAL(ftello) >> + TLI_DEFINE_STRING_INTERNAL("ftello") >> + /// off64_t ftello64(FILE *stream) >> ++#ifdef ftello64 >> ++#undef ftello64 >> ++#endif >> + TLI_DEFINE_ENUM_INTERNAL(ftello64) >> + TLI_DEFINE_STRING_INTERNAL("ftello64") >> + /// int ftrylockfile(FILE *file); >> +@@ -980,6 +992,9 @@ TLI_DEFINE_STRING_INTERNAL("logl") >> + TLI_DEFINE_ENUM_INTERNAL(lstat) >> + TLI_DEFINE_STRING_INTERNAL("lstat") >> + /// int lstat64(const char *path, struct stat64 *buf); >> ++#ifdef lstat64 >> ++#undef lstat64 >> ++#endif >> + TLI_DEFINE_ENUM_INTERNAL(lstat64) >> + TLI_DEFINE_STRING_INTERNAL("lstat64") >> + /// void *malloc(size_t size); >> +@@ -1205,6 +1220,9 @@ TLI_DEFINE_STRING_INTERNAL("sscanf") >> + TLI_DEFINE_ENUM_INTERNAL(stat) >> + TLI_DEFINE_STRING_INTERNAL("stat") >> + /// int stat64(const char *path, struct stat64 *buf); >> ++#ifdef stat64 >> ++#undef stat64 >> ++#endif >> + TLI_DEFINE_ENUM_INTERNAL(stat64) >> + TLI_DEFINE_STRING_INTERNAL("stat64") >> + /// int statvfs(const char *path, struct statvfs *buf); >> +@@ -1340,6 +1358,9 @@ TLI_DEFINE_STRING_INTERNAL("times") >> + TLI_DEFINE_ENUM_INTERNAL(tmpfile) >> + TLI_DEFINE_STRING_INTERNAL("tmpfile") >> + /// FILE *tmpfile64(void) >> ++#ifdef tmpfile64 >> ++#undef tmpfile64 >> ++#endif >> + TLI_DEFINE_ENUM_INTERNAL(tmpfile64) >> + TLI_DEFINE_STRING_INTERNAL("tmpfile64") >> + /// int toascii(int c); >> +-- >> +2.22.0 >> + >> diff --git a/recipes-devtools/glow/llvm-glow/0007-llvm-allow-env-override-of-exe-path.patch b/recipes-devtools/glow/llvm-glow/0007-llvm-allow-env-override-of-exe-path.patch >> new file mode 100644 >> index 0000000..f47080c >> --- /dev/null >> +++ b/recipes-devtools/glow/llvm-glow/0007-llvm-allow-env-override-of-exe-path.patch >> @@ -0,0 +1,38 @@ >> +From b66d6f39a374b8df41e7235351e5dee2e81f440c Mon Sep 17 00:00:00 2001 >> +From: Martin Kelly <mkelly@xevo.com> >> +Date: Fri, 19 May 2017 00:22:57 -0700 >> +Subject: [PATCH 07/19] llvm: allow env override of exe path >> + >> +When using a native llvm-config from inside a sysroot, we need llvm-config to >> +return the libraries, include directories, etc. from inside the sysroot rather >> +than from the native sysroot. Thus provide an env override for calling >> +llvm-config from a target sysroot. >> + >> +Upstream-Status: Pending >> +Signed-off-by: Martin Kelly <mkelly@xevo.com> >> +Signed-off-by: Khem Raj <raj.khem@gmail.com> >> +--- >> + llvm/tools/llvm-config/llvm-config.cpp | 7 +++++++ >> + 1 file changed, 7 insertions(+) >> + >> +diff --git a/llvm/tools/llvm-config/llvm-config.cpp b/llvm/tools/llvm-config/llvm-config.cpp >> +index 7ef7c46a262..a4f7ed82c7b 100644 >> +--- a/llvm/tools/llvm-config/llvm-config.cpp >> ++++ b/llvm/tools/llvm-config/llvm-config.cpp >> +@@ -225,6 +225,13 @@ Typical components:\n\ >> + >> + /// Compute the path to the main executable. >> + std::string GetExecutablePath(const char *Argv0) { >> ++ // Hack for Yocto: we need to override the root path when we are using >> ++ // llvm-config from within a target sysroot. >> ++ const char *Sysroot = std::getenv("YOCTO_ALTERNATE_EXE_PATH"); >> ++ if (Sysroot != nullptr) { >> ++ return Sysroot; >> ++ } >> ++ >> + // This just needs to be some symbol in the binary; C++ doesn't >> + // allow taking the address of ::main however. >> + void *P = (void *)(intptr_t)GetExecutablePath; >> +-- >> +2.22.0 >> + >> diff --git a/recipes-devtools/glow/llvm-glow_git.bb b/recipes-devtools/glow/llvm-glow_git.bb >> index 58d392b..d3c2642 100644 >> --- a/recipes-devtools/glow/llvm-glow_git.bb >> +++ b/recipes-devtools/glow/llvm-glow_git.bb >> @@ -1,24 +1,180 @@ >> -DESCRIPTION = "LLVM release suggested for pytorch glow neural network compiler" >> -HOMEPAGE = "https://github.com/llvm-mirror/llvm" >> +# Copyright (C) 2017 Khem Raj <raj.khem@gmail.com> >> +# Released under the MIT license (see COPYING.MIT for the terms) >> + >> +DESCRIPTION = "The LLVM Compiler Infrastructure" >> +HOMEPAGE = "http://llvm.org" >> +LICENSE = "NCSA" >> SECTION = "devel" >> -LICENSE = "Apache-2.0" >> + >> LIC_FILES_CHKSUM = "file://LICENSE.TXT;md5=c6b766a4e85dd28301eeed54a6684648" >> -PV = "8.0" >> -PR = "r0" >> >> -DEPENDS = "ninja-native protobuf-native libpng-native protobuf-native protobuf-c-native gflags-native glog-native" >> +DEPENDS = "libffi libxml2 zlib libedit ninja-native llvm8-native" >> >> -SOLIBVER = "1" >> +RDEPENDS_${PN}_append_class-target = " ncurses-terminfo" >> + >> +inherit cmake pkgconfig >> + >> +PROVIDES += "llvm${PV}" >> >> -SRC_URI = "git://github.com/llvm-mirror/llvm.git;protocol=https;branch=release_80;rev=5563a6a769b48912b9c8052124520610966eda28;destsuffix=git/llvm \ >> - git://github.com/llvm-mirror/clang.git;protocol=https;branch=release_80;rev=ccfe04576c13497b9c422ceef0b6efe99077a392;destsuffix=git/clang \ >> - " >> +LLVM_RELEASE = "${PV}" >> +LLVM_DIR = "llvm${LLVM_RELEASE}" >> >> +SRCREV = "19a71f6bdf2dddb10764939e7f0ec2b98dba76c9" >> + >> +BRANCH = "release/${MAJOR_VERSION}.x" >> +MAJOR_VERSION = "8" >> +MINOR_VERSION = "0" >> +PATCH_VERSION = "1" >> +SOLIBVER = "1" >> +PV = "${MAJOR_VERSION}.${MINOR_VERSION}.${PATCH_VERSION}" >> +SRC_URI = "git://github.com/llvm/llvm-project.git;branch=${BRANCH} \ >> + file://0006-llvm-TargetLibraryInfo-Undefine-libc-functions-if-th.patch;striplevel=2 \ >> + file://0007-llvm-allow-env-override-of-exe-path.patch;striplevel=2 \ >> + " >> >> S = "${WORKDIR}/git/llvm" >> >> -EXTRA_OECMAKE = "-DCMAKE_BUILD_TYPE=Release -DLLVM_INSTALL_UTILS=ON -DLLVM_ENABLE_PROJECTS=clang" >> +LLVM_INSTALL_DIR = "${WORKDIR}/llvm-install" >> + >> +def get_llvm_arch(bb, d, arch_var): >> + import re >> + a = d.getVar(arch_var) >> + if re.match(r'(i.86|athlon|x86.64)$', a): return 'X86' >> + elif re.match(r'arm$', a): return 'ARM' >> + elif re.match(r'armeb$', a): return 'ARM' >> + elif re.match(r'aarch64$', a): return 'AArch64' >> + elif re.match(r'aarch64_be$', a): return 'AArch64' >> + elif re.match(r'mips(isa|)(32|64|)(r6|)(el|)$', a): return 'Mips' >> + elif re.match(r'p(pc|owerpc)(|64)', a): return 'PowerPC' >> + else: >> + raise bb.parse.SkipRecipe("Cannot map '%s' to a supported LLVM architecture" % a) >> + >> +def get_llvm_host_arch(bb, d): >> + return get_llvm_arch(bb, d, 'HOST_ARCH') >> + >> +# >> +# Default to build all OE-Core supported target arches (user overridable). >> +# >> +LLVM_TARGETS ?= "AMDGPU;${@get_llvm_host_arch(bb, d)}" >> + >> +ARM_INSTRUCTION_SET_armv5 = "arm" >> +ARM_INSTRUCTION_SET_armv4t = "arm" >> + >> +EXTRA_OECMAKE += "-DLLVM_ENABLE_ASSERTIONS=OFF \ >> + -DLLVM_ENABLE_EXPENSIVE_CHECKS=OFF \ >> + -DLLVM_ENABLE_PIC=ON \ >> + -DLLVM_BINDINGS_LIST='' \ >> + -DLLVM_LINK_LLVM_DYLIB=ON \ >> + -DLLVM_ENABLE_FFI=ON \ >> + -DLLVM_ENABLE_RTTI=ON \ >> + -DFFI_INCLUDE_DIR=$(pkg-config --variable=includedir libffi) \ >> + -DLLVM_OPTIMIZED_TABLEGEN=ON \ >> + -DLLVM_TARGETS_TO_BUILD='${LLVM_TARGETS}' \ >> + -DLLVM_TEMPORARILY_ALLOW_OLD_TOOLCHAIN=ON \ >> + -DPYTHON_EXECUTABLE=${HOSTTOOLS_DIR}/python2 \ >> + -G Ninja" >> + >> +EXTRA_OECMAKE_append_class-target = "\ >> + -DCMAKE_CROSSCOMPILING:BOOL=ON \ >> + -DLLVM_TABLEGEN=${STAGING_BINDIR_NATIVE}/llvm-tblgen${PV} \ >> + -DLLVM_CONFIG_PATH=${STAGING_BINDIR_NATIVE}/llvm-config${PV} \ >> + " >> + >> +EXTRA_OECMAKE_append_class-nativesdk = "\ >> + -DCMAKE_CROSSCOMPILING:BOOL=ON \ >> + -DLLVM_TABLEGEN=${STAGING_BINDIR_NATIVE}/llvm-tblgen${PV} \ >> + -DLLVM_CONFIG_PATH=${STAGING_BINDIR_NATIVE}/llvm-config${PV} \ >> + " >> + >> +CXXFLAGS_append_class-target_powerpc = " -mlongcall" >> + >> +do_configure_prepend() { >> +# Fix paths in llvm-config >> + sed -i "s|sys::path::parent_path(CurrentPath))\.str()|sys::path::parent_path(sys::path::parent_path(CurrentPath))).str()|g" ${S}/tools/llvm-config/llvm-config.cpp >> + sed -ri "s#/(bin|include|lib)(/?\")#/\1/${LLVM_DIR}\2#g" ${S}/tools/llvm-config/llvm-config.cpp >> + sed -ri "s#lib/${LLVM_DIR}#${baselib}/${LLVM_DIR}#g" ${S}/tools/llvm-config/llvm-config.cpp >> +} >> + >> +do_compile() { >> + ninja -v ${PARALLEL_MAKE} >> +} >> + >> +do_compile_class-native() { >> + ninja -v ${PARALLEL_MAKE} llvm-config llvm-tblgen >> +} >> + >> +do_install() { >> + DESTDIR=${LLVM_INSTALL_DIR} ninja -v install >> + install -D -m 0755 ${B}/bin/llvm-config ${D}${libdir}/${LLVM_DIR}/llvm-config >> + >> + install -d ${D}${bindir}/${LLVM_DIR} >> + cp -r ${LLVM_INSTALL_DIR}${bindir}/* ${D}${bindir}/${LLVM_DIR}/ >> + >> + install -d ${D}${includedir}/${LLVM_DIR} >> + cp -r ${LLVM_INSTALL_DIR}${includedir}/* ${D}${includedir}/${LLVM_DIR}/ >> + >> + install -d ${D}${libdir}/${LLVM_DIR} >> + >> + # The LLVM sources have "/lib" embedded and so we cannot completely rely on the ${libdir} variable >> + if [ -d ${LLVM_INSTALL_DIR}${libdir}/ ]; then >> + cp -r ${LLVM_INSTALL_DIR}${libdir}/* ${D}${libdir}/${LLVM_DIR}/ >> + elif [ -d ${LLVM_INSTALL_DIR}${prefix}/lib ]; then >> + cp -r ${LLVM_INSTALL_DIR}${prefix}/lib/* ${D}${libdir}/${LLVM_DIR}/ >> + elif [ -d ${LLVM_INSTALL_DIR}${prefix}/lib64 ]; then >> + cp -r ${LLVM_INSTALL_DIR}${prefix}/lib64/* ${D}${libdir}/${LLVM_DIR}/ >> + fi >> + >> + # Remove unnecessary cmake files >> + rm -rf ${D}${libdir}/${LLVM_DIR}/cmake >> + >> + ln -s ${LLVM_DIR}/libLLVM-${MAJOR_VERSION}${SOLIBSDEV} ${D}${libdir}/libLLVM-${MAJOR_VERSION}${SOLIBSDEV} >> + >> + # We'll have to delete the libLLVM.so due to multiple reasons... >> + rm -rf ${D}${libdir}/${LLVM_DIR}/libLLVM.so >> + rm -rf ${D}${libdir}/${LLVM_DIR}/libLTO.so >> +} >> + >> +do_install_class-native() { >> + install -D -m 0755 ${B}/bin/llvm-tblgen ${D}${bindir}/llvm-tblgen${PV} >> + install -D -m 0755 ${B}/bin/llvm-config ${D}${bindir}/llvm-config${PV} >> + install -D -m 0755 ${B}/lib/libLLVM-${MAJOR_VERSION}.so ${D}${libdir}/libLLVM-${MAJOR_VERSION}.so >> +} >> + >> +PACKAGES =+ "${PN}-bugpointpasses ${PN}-llvmhello ${PN}-libllvm ${PN}-liboptremarks ${PN}-liblto" >> + >> +RRECOMMENDS_${PN}-dev += "${PN}-bugpointpasses ${PN}-llvmhello ${PN}-liboptremarks" >> + >> +FILES_${PN}-bugpointpasses = "\ >> + ${libdir}/${LLVM_DIR}/BugpointPasses.so \ >> +" >> + >> +FILES_${PN}-libllvm = "\ >> + ${libdir}/${LLVM_DIR}/libLLVM-${MAJOR_VERSION}.so \ >> + ${libdir}/libLLVM-${MAJOR_VERSION}.so \ >> +" >> + >> +FILES_${PN}-liblto += "\ >> + ${libdir}/${LLVM_DIR}/libLTO.so.* \ >> +" >> + >> +FILES_${PN}-liboptremarks += "\ >> + ${libdir}/${LLVM_DIR}/libOptRemarks.so.* \ >> +" >> + >> +FILES_${PN}-llvmhello = "\ >> + ${libdir}/${LLVM_DIR}/LLVMHello.so \ >> +" >> + >> +FILES_${PN}-dev += " \ >> + ${libdir}/${LLVM_DIR}/llvm-config \ >> + ${libdir}/${LLVM_DIR}/libOptRemarks.so \ >> + ${libdir}/${LLVM_DIR}/libLLVM-${MAJOR_VERSION}.${MINOR_VERSION}.${PATCH_VERSION}.so \ >> +" >> + >> +FILES_${PN}-staticdev += "\ >> + ${libdir}/${LLVM_DIR}/*.a \ >> +" >> >> -inherit pkgconfig cmake >> +INSANE_SKIP_${PN}-libllvm += "dev-so" >> >> BBCLASSEXTEND = "native nativesdk" ^ permalink raw reply [flat|nested] 12+ messages in thread
* [meta-processor-sdk][PATCHv2 2/5] clang8-native: llvm and clang used only for compilation of native tools for glow compiler 2019-08-23 13:58 [meta-processor-sdk][PATCHv2 0/5] Pytorch GLOW AOT tool and demo example Djordje Senicic 2019-08-23 13:58 ` [meta-processor-sdk][PATCHv2 1/5] mesa, llvm, meson: Update llvm to 8.0.1 plus define and use LLVM version globally Djordje Senicic @ 2019-08-23 13:58 ` Djordje Senicic 2019-08-23 13:58 ` [meta-processor-sdk][PATCHv2 3/5] glow: Address devkit related RPATH issue to point to devkit shared libraries instead of host native Djordje Senicic ` (2 subsequent siblings) 4 siblings, 0 replies; 12+ messages in thread From: Djordje Senicic @ 2019-08-23 13:58 UTC (permalink / raw) To: meta-arago; +Cc: d-senicic1, Djordje Senicic - Both clang and llvm from branch 8.0 are used only for pytoch glow package. This is to compile several native tools (source inside glow project) that are needed for nativesdk glow package, and demo-glow (one of these native tools is used to compile a model into armv7 object file). Signed-off-by: Djordje Senicic <x0157990@ti.com> --- recipes-devtools/glow/clang8-native_git.bb | 28 +++++++++++++++++++ ...ryInfo-Undefine-libc-functions-if-th.patch | 0 ...-llvm-allow-env-override-of-exe-path.patch | 0 .../glow/{llvm-glow_git.bb => llvm8_git.bb} | 0 4 files changed, 28 insertions(+) create mode 100644 recipes-devtools/glow/clang8-native_git.bb rename recipes-devtools/glow/{llvm-glow => llvm8}/0006-llvm-TargetLibraryInfo-Undefine-libc-functions-if-th.patch (100%) rename recipes-devtools/glow/{llvm-glow => llvm8}/0007-llvm-allow-env-override-of-exe-path.patch (100%) rename recipes-devtools/glow/{llvm-glow_git.bb => llvm8_git.bb} (100%) diff --git a/recipes-devtools/glow/clang8-native_git.bb b/recipes-devtools/glow/clang8-native_git.bb new file mode 100644 index 0000000..fadf35c --- /dev/null +++ b/recipes-devtools/glow/clang8-native_git.bb @@ -0,0 +1,28 @@ +DESCRIPTION = "CLANG/LLVM compiler used only for native tools of pytorch glow neural network compiler " +HOMEPAGE = "https://github.com/llvm-mirror/llvm" +SECTION = "devel" +LICENSE = "Apache-2.0" +LIC_FILES_CHKSUM = "file://LICENSE.TXT;md5=c6b766a4e85dd28301eeed54a6684648" +PV = "8.0" +PR = "r0" + +DEPENDS = "ninja protobuf libpng protobuf protobuf-c gflags glog " + +SOLIBVER = "1" + +BRANCH="release_80" + +SRC_URI = "git://github.com/llvm-mirror/llvm.git;protocol=https;branch=${BRANCH};destsuffix=git/llvm;name=llvm \ + git://github.com/llvm-mirror/clang.git;protocol=https;branch=${BRANCH};destsuffix=git/clang;name=clang \ + " + +SRCREV_llvm = "5563a6a769b48912b9c8052124520610966eda28" +SRCREV_clang = "ccfe04576c13497b9c422ceef0b6efe99077a392" + + +S = "${WORKDIR}/git/llvm" + +EXTRA_OECMAKE = "-DCMAKE_BUILD_TYPE=Release -DLLVM_INSTALL_UTILS=ON -DLLVM_ENABLE_PROJECTS=clang" + +inherit native pkgconfig cmake + diff --git a/recipes-devtools/glow/llvm-glow/0006-llvm-TargetLibraryInfo-Undefine-libc-functions-if-th.patch b/recipes-devtools/glow/llvm8/0006-llvm-TargetLibraryInfo-Undefine-libc-functions-if-th.patch similarity index 100% rename from recipes-devtools/glow/llvm-glow/0006-llvm-TargetLibraryInfo-Undefine-libc-functions-if-th.patch rename to recipes-devtools/glow/llvm8/0006-llvm-TargetLibraryInfo-Undefine-libc-functions-if-th.patch diff --git a/recipes-devtools/glow/llvm-glow/0007-llvm-allow-env-override-of-exe-path.patch b/recipes-devtools/glow/llvm8/0007-llvm-allow-env-override-of-exe-path.patch similarity index 100% rename from recipes-devtools/glow/llvm-glow/0007-llvm-allow-env-override-of-exe-path.patch rename to recipes-devtools/glow/llvm8/0007-llvm-allow-env-override-of-exe-path.patch diff --git a/recipes-devtools/glow/llvm-glow_git.bb b/recipes-devtools/glow/llvm8_git.bb similarity index 100% rename from recipes-devtools/glow/llvm-glow_git.bb rename to recipes-devtools/glow/llvm8_git.bb -- 2.17.1 ^ permalink raw reply related [flat|nested] 12+ messages in thread
* [meta-processor-sdk][PATCHv2 3/5] glow: Address devkit related RPATH issue to point to devkit shared libraries instead of host native 2019-08-23 13:58 [meta-processor-sdk][PATCHv2 0/5] Pytorch GLOW AOT tool and demo example Djordje Senicic 2019-08-23 13:58 ` [meta-processor-sdk][PATCHv2 1/5] mesa, llvm, meson: Update llvm to 8.0.1 plus define and use LLVM version globally Djordje Senicic 2019-08-23 13:58 ` [meta-processor-sdk][PATCHv2 2/5] clang8-native: llvm and clang used only for compilation of native tools for glow compiler Djordje Senicic @ 2019-08-23 13:58 ` Djordje Senicic 2019-09-18 19:39 ` [EXTERNAL] " Jacob Stiffler 2019-08-23 13:58 ` [meta-processor-sdk][PATCHv2 4/5] demo-glow: Restore in top-level makefile examples after recipe cleanup Djordje Senicic 2019-08-23 13:58 ` [meta-processor-sdk][PATCHv2 5/5] packagegroup-arago-tisdk: Include xNN glow based compiler tool into devkit, and demo-glow example Djordje Senicic 4 siblings, 1 reply; 12+ messages in thread From: Djordje Senicic @ 2019-08-23 13:58 UTC (permalink / raw) To: meta-arago; +Cc: d-senicic1, Djordje Senicic - Pytorch Glow is xNN compiler tool and artifacts are required only in devkit. Glow build procedure includes creation of native tools that are used in later phases of build process. These tools (include-bin, InstrGen, NodeGen) are created by native compilation, then used by nativesdk. Image-clasification tool created by native compilation is used by demo-glow to compile network into armv7 object file that is included in target filesystem. - RPATH for image-classification tool (created by glow project) is modified to be $ORIGIN based (i.e. relative location) - Update dependencies Signed-off-by: Djordje Senicic <x0157990@ti.com> --- ...p-compilation-of-native-helper-tools.patch | 99 +++++++++++++++++++ recipes-devtools/glow/glow_git.bb | 62 +++++++++--- 2 files changed, 145 insertions(+), 16 deletions(-) create mode 100644 recipes-devtools/glow/files/0002-Skip-compilation-of-native-helper-tools.patch diff --git a/recipes-devtools/glow/files/0002-Skip-compilation-of-native-helper-tools.patch b/recipes-devtools/glow/files/0002-Skip-compilation-of-native-helper-tools.patch new file mode 100644 index 0000000..6fd4a4d --- /dev/null +++ b/recipes-devtools/glow/files/0002-Skip-compilation-of-native-helper-tools.patch @@ -0,0 +1,99 @@ +From 923f66a209c329b2d5ab245e1c02f91fa6705d4c Mon Sep 17 00:00:00 2001 +From: Djordje Senicic <x0157990@ti.com> +Date: Thu, 1 Aug 2019 04:58:24 -0400 +Subject: [PATCH] Skip compilation of native helper tools + +Upstream-Status: Inappropriate [other] + +- Applicable for limited use (demo) of AOT image-classifier example only + +Signed-off-by: Djordje Senicic <x0157990@ti.com> +--- + lib/Backends/CPU/CMakeLists.txt | 2 +- + lib/Backends/OpenCL/CMakeLists.txt | 12 ++++++------ + lib/Graph/CMakeLists.txt | 3 +-- + lib/IR/CMakeLists.txt | 3 +-- + 4 files changed, 9 insertions(+), 11 deletions(-) + +diff --git a/lib/Backends/CPU/CMakeLists.txt b/lib/Backends/CPU/CMakeLists.txt +index 1e734539..0d5c5e60 100644 +--- a/lib/Backends/CPU/CMakeLists.txt ++++ b/lib/Backends/CPU/CMakeLists.txt +@@ -53,7 +53,7 @@ add_custom_command( + file(MAKE_DIRECTORY ${CMAKE_BINARY_DIR}/glow/CPU) + add_custom_command( + OUTPUT ${CMAKE_BINARY_DIR}/glow/CPU/libjit_bc.inc +- COMMAND include-bin "${CMAKE_BINARY_DIR}/CPU/libjit.bc" "${CMAKE_BINARY_DIR}/glow/CPU/libjit_bc.inc" ++ COMMAND ${GLOW_BINARY_DIR}/bin/include-bin "${CMAKE_BINARY_DIR}/CPU/libjit.bc" "${CMAKE_BINARY_DIR}/glow/CPU/libjit_bc.inc" + DEPENDS ${GLOW_BINARY_DIR}/CPU/libjit.bc + WORKING_DIRECTORY "${CMAKE_CURRENT_LIST_DIR}") + +diff --git a/lib/Backends/OpenCL/CMakeLists.txt b/lib/Backends/OpenCL/CMakeLists.txt +index fd9f965c..e37b4b35 100644 +--- a/lib/Backends/OpenCL/CMakeLists.txt ++++ b/lib/Backends/OpenCL/CMakeLists.txt +@@ -8,25 +8,25 @@ add_custom_target(make_opencl_dir + + add_custom_command( + OUTPUT "${OpenCL_DIR}/kernels.cl.inc" +- COMMAND include-bin ++ COMMAND ${GLOW_BINARY_DIR}/bin/include-bin + "${CMAKE_CURRENT_SOURCE_DIR}/kernels.cl" + "${OpenCL_DIR}/kernels.cl.inc" +- DEPENDS make_opencl_dir include-bin "${CMAKE_CURRENT_SOURCE_DIR}/kernels.cl") ++ DEPENDS make_opencl_dir ${GLOW_BINARY_DIR}/bin/include-bin "${CMAKE_CURRENT_SOURCE_DIR}/kernels.cl") + + add_custom_command( + OUTPUT "${OpenCL_DIR}/kernels_fwd_conv.cl.inc" +- COMMAND include-bin ++ COMMAND ${GLOW_BINARY_DIR}/bin/include-bin + "${CMAKE_CURRENT_SOURCE_DIR}/kernels_fwd_conv.cl" + "${OpenCL_DIR}/kernels_fwd_conv.cl.inc" +- DEPENDS make_opencl_dir include-bin ++ DEPENDS make_opencl_dir ${GLOW_BINARY_DIR}/bin/include-bin + "${CMAKE_CURRENT_SOURCE_DIR}/kernels_fwd_conv.cl") + + add_custom_command( + OUTPUT "${OpenCL_DIR}/kernels_fwd_quantized_conv.cl.inc" +- COMMAND include-bin ++ COMMAND ${GLOW_BINARY_DIR}/bin/include-bin + "${CMAKE_CURRENT_SOURCE_DIR}/kernels_fwd_quantized_conv.cl" + "${OpenCL_DIR}/kernels_fwd_quantized_conv.cl.inc" +- DEPENDS make_opencl_dir include-bin ++ DEPENDS make_opencl_dir ${GLOW_BINARY_DIR}/bin/include-bin + "${CMAKE_CURRENT_SOURCE_DIR}/kernels_fwd_quantized_conv.cl") + + add_library(OpenCLBackend +diff --git a/lib/Graph/CMakeLists.txt b/lib/Graph/CMakeLists.txt +index 84589761..f0976259 100644 +--- a/lib/Graph/CMakeLists.txt ++++ b/lib/Graph/CMakeLists.txt +@@ -7,8 +7,7 @@ add_custom_command(OUTPUT + "${NODES_HDR}" + "${NODES_SRC}" + "${NODES_DEF}" +- COMMAND NodeGen ${NODES_HDR} ${NODES_SRC} ${NODES_DEF} +- DEPENDS NodeGen ++ COMMAND ${GLOW_BINARY_DIR}/bin/NodeGen ${NODES_HDR} ${NODES_SRC} ${NODES_DEF} + COMMENT "NodeGen: Generating nodes." VERBATIM) + add_custom_target(AutoGenNode + DEPENDS +diff --git a/lib/IR/CMakeLists.txt b/lib/IR/CMakeLists.txt +index 7660308d..d5861bbb 100644 +--- a/lib/IR/CMakeLists.txt ++++ b/lib/IR/CMakeLists.txt +@@ -13,10 +13,9 @@ add_custom_command(OUTPUT + "${INSTR_BLD_HDR}" + "${INSTR_BLD_SRC}" + "${INSTR_IR_GEN}" +- COMMAND InstrGen ++ COMMAND ${GLOW_BINARY_DIR}/bin/InstrGen + "${INSTR_HDR}" "${INSTR_SRC}" "${INSTR_DEF}" + "${INSTR_BLD_HDR}" "${INSTR_BLD_SRC}" "${INSTR_IR_GEN}" +- DEPENDS InstrGen + COMMENT "InstrGen: Generating instructions." VERBATIM) + add_custom_target(AutoGenInstr + DEPENDS +-- +2.17.1 + diff --git a/recipes-devtools/glow/glow_git.bb b/recipes-devtools/glow/glow_git.bb index 3aea4b1..6ac8044 100644 --- a/recipes-devtools/glow/glow_git.bb +++ b/recipes-devtools/glow/glow_git.bb @@ -8,13 +8,12 @@ PV = "0.1+git${SRCPV}" PR = "r0" SRCREV_FORMAT = "glow" -DEPENDS_class-native = "zlib zlib-native llvm-glow-native ninja-native protobuf-native libpng-native protobuf-native protobuf-c-native gflags-native glog-native" -DEPENDS_class-nativesdk = "ncurses protobuf libpng glow-native zlib llvm-glow-native ninja-native protobuf-native libpng-native protobuf-c-native gflags-native glog-native" +DEPENDS_class-native = "clang8 zlib libpng ninja protobuf protobuf-c gflags glog " +DEPENDS_class-nativesdk = "ncurses protobuf zlib gflags glog libpng llvm8 glow-native ninja-native " -RDEPENDS_${PN}_class-nativesdk = "libpng ncurses glog" +RDEPENDS_${PN}_class-nativesdk = " ncurses protobuf " SRC_URI = "git://github.com/pytorch/glow.git;protocol=https;destsuffix=git;name=glow \ - file://0001-Fix-path-and-dependencies-to-intermediate-executable.patch \ git://github.com/stp/OutputCheck.git;protocol=https;destsuffix=git/tests/OutputCheck;name=outcheck \ git://github.com/google/benchmark.git;protocol=https;destsuffix=git/tests/googlebenchmark;name=gbench \ git://github.com/google/googletest.git;protocol=https;destsuffix=git/tests/googletest;name=gtest \ @@ -26,6 +25,11 @@ SRC_URI = "git://github.com/pytorch/glow.git;protocol=https;destsuffix=git;name= git://github.com/wjakob/clang-cindex-python3.git;protocol=https;destsuffix=git/thirdparty/onnx/third_party/pybind11/tools/clang;name=onnxclang \ " +SRC_URI_append_class-native = " file://0001-Fix-path-and-dependencies-to-intermediate-executable.patch \ + " +SRC_URI_append_class-nativesdk = "file://0002-Skip-compilation-of-native-helper-tools.patch \ + " + SRCREV_glow = "ba8061367aea0b472b596448018107ca6e87929e" SRCREV_outcheck = "eab62a5dd5129f6a4ebfbe4bbe41d35611f7c48d" SRCREV_gbench = "090faecb454fbd6e6e17a75ef8146acb037118d4" @@ -41,23 +45,49 @@ S = "${WORKDIR}/git" inherit pkgconfig cmake -EXTRA_OECMAKE += " -DCMAKE_BUILD_TYPE=Release -DCMAKE_EXE_LINKER_FLAGS="-Wl,-dynamic-linker=/lib64/ld-linux-x86-64.so.2 " " +CP_ARGS="-Prf --preserve=mode,timestamps --no-preserve=ownership" + +do_configure_class-native() { + cd ${B} + cmake ${S} -G Ninja -DCMAKE_BUILD_TYPE=Release -DCMAKE_EXE_LINKER_FLAGS="-Wl,-z,origin -ldl -lz -ltinfo -pthread " +} + +do_compile_class-native() { + cd ${B} + ninja include-bin InstrGen NodeGen image-classifier +} + +do_install_class-native() { + install -d ${D}${datadir}/glow/bin + install -d ${D}${datadir}/glow/tests/images + chrpath -d ${WORKDIR}/build/bin/* + cp ${CP_ARGS} ${B}/bin/* ${D}${datadir}/glow/bin/. + cp ${CP_ARGS} ${S}/tests/images/mnist ${D}${datadir}/glow/tests/images +} + -do_compile() { - cmake_runcmake_build --target ${OECMAKE_TARGET_COMPILE} +do_configure_class-nativesdk() { + cd ${B} + cmake ${S} -G Ninja -DCMAKE_BUILD_TYPE=Release -DCMAKE_EXE_LINKER_FLAGS="-Wl,-z,origin -ldl -lz -ltinfo -pthread " -DRUN_HAVE_STD_REGEX=0 -DRUN_HAVE_POSIX_REGEX=0 } -do_install () { - install -d ${D}${datadir}/glow +do_compile_class-nativesdk() { + cd ${B} + cp ${CP_ARGS} ${STAGING_DIR_NATIVE}/usr/share/glow/bin/* ${WORKDIR}/build/bin + ninja image-classifier +} + +do_install_class-nativesdk() { + install -d ${D}${datadir}/glow/bin install -d ${D}${datadir}/glow/tests/images - cp -Prf --preserve=mode,timestamps ${WORKDIR}/build/bin ${D}${datadir}/glow - cp -Prf --preserve=mode,timestamps ${WORKDIR}/build/tests/images/EmotionSampleImages ${D}${datadir}/glow/tests/images - cp -Prf --preserve=mode,timestamps ${WORKDIR}/build/tests/images/imagenet ${D}${datadir}/glow/tests/images - cp -Prf --preserve=mode,timestamps ${WORKDIR}/build/tests/images/imagenet_299 ${D}${datadir}/glow/tests/images - cp -Prf --preserve=mode,timestamps ${WORKDIR}/build/tests/images/mnist ${D}${datadir}/glow/tests/images - cp -Prf --preserve=mode,timestamps ${WORKDIR}/build/tests/images/other ${D}${datadir}/glow/tests/images + install -m 0755 ${B}/bin/image-classifier ${D}${datadir}/glow/bin/image-classifier + chrpath -r "\$ORIGIN/../../../lib:\$ORIGIN/../../../../lib" ${D}${datadir}/glow/bin/image-classifier + cp ${CP_ARGS} ${S}/tests/images/imagenet ${D}${datadir}/glow/tests/images + cp ${CP_ARGS} ${S}/tests/images/imagenet_299 ${D}${datadir}/glow/tests/images + cp ${CP_ARGS} ${S}/tests/images/mnist ${D}${datadir}/glow/tests/images } + BBCLASSEXTEND = "native nativesdk" -FILES_${PN} = "${datadir}" +FILES_${PN} = "${datadir}/glow" -- 2.17.1 ^ permalink raw reply related [flat|nested] 12+ messages in thread
* Re: [EXTERNAL] [meta-processor-sdk][PATCHv2 3/5] glow: Address devkit related RPATH issue to point to devkit shared libraries instead of host native 2019-08-23 13:58 ` [meta-processor-sdk][PATCHv2 3/5] glow: Address devkit related RPATH issue to point to devkit shared libraries instead of host native Djordje Senicic @ 2019-09-18 19:39 ` Jacob Stiffler 0 siblings, 0 replies; 12+ messages in thread From: Jacob Stiffler @ 2019-09-18 19:39 UTC (permalink / raw) To: Djordje Senicic, meta-arago; +Cc: d-senicic1 I get an error when configuring glow-native. Log data follows: | DEBUG: Executing shell function do_configure | -- The C compiler identification is GNU 5.3.1 | -- The CXX compiler identification is GNU 5.3.1 | -- Check for working C compiler: /oe/bld/build-CORTEX_1/arago-tmp-external-arm-toolchain/hosttools/gcc | -- Check for working C compiler: /oe/bld/build-CORTEX_1/arago-tmp-external-arm-toolchain/hosttools/gcc -- broken | CMake Error at /oe/bld/build-CORTEX_1/arago-tmp-external-arm-toolchain/work/x86_64-linux/glow-native/0.1+gitAUTOINC+ba8061367a-r0/recipe-sysroot-native/usr/share/cmake-3.12/Modules/CMakeTestCCompiler.cmake:52 (message): | The C compiler | | "/oe/bld/build-CORTEX_1/arago-tmp-external-arm-toolchain/hosttools/gcc" | | is not able to compile a simple test program. | | It fails with the following output: | | Change Dir: /oe/bld/build-CORTEX_1/arago-tmp-external-arm-toolchain/work/x86_64-linux/glow-native/0.1+gitAUTOINC+ba8061367a-r0/build/CMakeFiles/CMakeTmp | | Run Build Command:"/oe/bld/build-CORTEX_1/arago-tmp-external-arm-toolchain/work/x86_64-linux/glow-native/0.1+gitAUTOINC+ba8061367a-r0/recipe-sysroot-native/usr/bin/ninja" "cmTC_cc2e4" | [1/2] Building C object CMakeFiles/cmTC_cc2e4.dir/testCCompiler.c.o | [2/2] Linking C executable cmTC_cc2e4 | FAILED: cmTC_cc2e4 | : && /oe/bld/build-CORTEX_1/arago-tmp-external-arm-toolchain/hosttools/gcc -isystem/oe/bld/build-CORTEX_1/arago-tmp-external-arm-toolchain/work/x86_64-linux/glow-native/0.1+gitAUTOINC+ba8061367a-r0/recipe-sysroot-native/usr/include -O2 -pipe -Wl,-z,origin -ldl -lz -ltinfo -pthread CMakeFiles/cmTC_cc2e4.dir/testCCompiler.c.o -o cmTC_cc2e4 && : | /oe/bld/build-CORTEX_1/arago-tmp-external-arm-toolchain/hosttools/ld: cannot find -lz | /oe/bld/build-CORTEX_1/arago-tmp-external-arm-toolchain/hosttools/ld: cannot find -ltinfo | collect2: error: ld returned 1 exit status | ninja: build stopped: subcommand failed. | | | | | CMake will not be able to correctly generate this project. | Call Stack (most recent call first): | CMakeLists.txt:5 (project) | | | -- Configuring incomplete, errors occurred! On 8/23/2019 9:58 AM, Djordje Senicic wrote: > - Pytorch Glow is xNN compiler tool and artifacts are required only in devkit. > Glow build procedure includes creation of native tools that are used in later > phases of build process. These tools (include-bin, InstrGen, NodeGen) are created > by native compilation, then used by nativesdk. Image-clasification tool created by native > compilation is used by demo-glow to compile network into armv7 object file that is included in > target filesystem. > - RPATH for image-classification tool (created by glow project) is modified > to be $ORIGIN based (i.e. relative location) > - Update dependencies > > Signed-off-by: Djordje Senicic <x0157990@ti.com> > --- > ...p-compilation-of-native-helper-tools.patch | 99 +++++++++++++++++++ > recipes-devtools/glow/glow_git.bb | 62 +++++++++--- > 2 files changed, 145 insertions(+), 16 deletions(-) > create mode 100644 recipes-devtools/glow/files/0002-Skip-compilation-of-native-helper-tools.patch > > diff --git a/recipes-devtools/glow/files/0002-Skip-compilation-of-native-helper-tools.patch b/recipes-devtools/glow/files/0002-Skip-compilation-of-native-helper-tools.patch > new file mode 100644 > index 0000000..6fd4a4d > --- /dev/null > +++ b/recipes-devtools/glow/files/0002-Skip-compilation-of-native-helper-tools.patch > @@ -0,0 +1,99 @@ > +From 923f66a209c329b2d5ab245e1c02f91fa6705d4c Mon Sep 17 00:00:00 2001 > +From: Djordje Senicic <x0157990@ti.com> > +Date: Thu, 1 Aug 2019 04:58:24 -0400 > +Subject: [PATCH] Skip compilation of native helper tools > + > +Upstream-Status: Inappropriate [other] > + > +- Applicable for limited use (demo) of AOT image-classifier example only > + > +Signed-off-by: Djordje Senicic <x0157990@ti.com> > +--- > + lib/Backends/CPU/CMakeLists.txt | 2 +- > + lib/Backends/OpenCL/CMakeLists.txt | 12 ++++++------ > + lib/Graph/CMakeLists.txt | 3 +-- > + lib/IR/CMakeLists.txt | 3 +-- > + 4 files changed, 9 insertions(+), 11 deletions(-) > + > +diff --git a/lib/Backends/CPU/CMakeLists.txt b/lib/Backends/CPU/CMakeLists.txt > +index 1e734539..0d5c5e60 100644 > +--- a/lib/Backends/CPU/CMakeLists.txt > ++++ b/lib/Backends/CPU/CMakeLists.txt > +@@ -53,7 +53,7 @@ add_custom_command( > + file(MAKE_DIRECTORY ${CMAKE_BINARY_DIR}/glow/CPU) > + add_custom_command( > + OUTPUT ${CMAKE_BINARY_DIR}/glow/CPU/libjit_bc.inc > +- COMMAND include-bin "${CMAKE_BINARY_DIR}/CPU/libjit.bc" "${CMAKE_BINARY_DIR}/glow/CPU/libjit_bc.inc" > ++ COMMAND ${GLOW_BINARY_DIR}/bin/include-bin "${CMAKE_BINARY_DIR}/CPU/libjit.bc" "${CMAKE_BINARY_DIR}/glow/CPU/libjit_bc.inc" > + DEPENDS ${GLOW_BINARY_DIR}/CPU/libjit.bc > + WORKING_DIRECTORY "${CMAKE_CURRENT_LIST_DIR}") > + > +diff --git a/lib/Backends/OpenCL/CMakeLists.txt b/lib/Backends/OpenCL/CMakeLists.txt > +index fd9f965c..e37b4b35 100644 > +--- a/lib/Backends/OpenCL/CMakeLists.txt > ++++ b/lib/Backends/OpenCL/CMakeLists.txt > +@@ -8,25 +8,25 @@ add_custom_target(make_opencl_dir > + > + add_custom_command( > + OUTPUT "${OpenCL_DIR}/kernels.cl.inc" > +- COMMAND include-bin > ++ COMMAND ${GLOW_BINARY_DIR}/bin/include-bin > + "${CMAKE_CURRENT_SOURCE_DIR}/kernels.cl" > + "${OpenCL_DIR}/kernels.cl.inc" > +- DEPENDS make_opencl_dir include-bin "${CMAKE_CURRENT_SOURCE_DIR}/kernels.cl") > ++ DEPENDS make_opencl_dir ${GLOW_BINARY_DIR}/bin/include-bin "${CMAKE_CURRENT_SOURCE_DIR}/kernels.cl") > + > + add_custom_command( > + OUTPUT "${OpenCL_DIR}/kernels_fwd_conv.cl.inc" > +- COMMAND include-bin > ++ COMMAND ${GLOW_BINARY_DIR}/bin/include-bin > + "${CMAKE_CURRENT_SOURCE_DIR}/kernels_fwd_conv.cl" > + "${OpenCL_DIR}/kernels_fwd_conv.cl.inc" > +- DEPENDS make_opencl_dir include-bin > ++ DEPENDS make_opencl_dir ${GLOW_BINARY_DIR}/bin/include-bin > + "${CMAKE_CURRENT_SOURCE_DIR}/kernels_fwd_conv.cl") > + > + add_custom_command( > + OUTPUT "${OpenCL_DIR}/kernels_fwd_quantized_conv.cl.inc" > +- COMMAND include-bin > ++ COMMAND ${GLOW_BINARY_DIR}/bin/include-bin > + "${CMAKE_CURRENT_SOURCE_DIR}/kernels_fwd_quantized_conv.cl" > + "${OpenCL_DIR}/kernels_fwd_quantized_conv.cl.inc" > +- DEPENDS make_opencl_dir include-bin > ++ DEPENDS make_opencl_dir ${GLOW_BINARY_DIR}/bin/include-bin > + "${CMAKE_CURRENT_SOURCE_DIR}/kernels_fwd_quantized_conv.cl") > + > + add_library(OpenCLBackend > +diff --git a/lib/Graph/CMakeLists.txt b/lib/Graph/CMakeLists.txt > +index 84589761..f0976259 100644 > +--- a/lib/Graph/CMakeLists.txt > ++++ b/lib/Graph/CMakeLists.txt > +@@ -7,8 +7,7 @@ add_custom_command(OUTPUT > + "${NODES_HDR}" > + "${NODES_SRC}" > + "${NODES_DEF}" > +- COMMAND NodeGen ${NODES_HDR} ${NODES_SRC} ${NODES_DEF} > +- DEPENDS NodeGen > ++ COMMAND ${GLOW_BINARY_DIR}/bin/NodeGen ${NODES_HDR} ${NODES_SRC} ${NODES_DEF} > + COMMENT "NodeGen: Generating nodes." VERBATIM) > + add_custom_target(AutoGenNode > + DEPENDS > +diff --git a/lib/IR/CMakeLists.txt b/lib/IR/CMakeLists.txt > +index 7660308d..d5861bbb 100644 > +--- a/lib/IR/CMakeLists.txt > ++++ b/lib/IR/CMakeLists.txt > +@@ -13,10 +13,9 @@ add_custom_command(OUTPUT > + "${INSTR_BLD_HDR}" > + "${INSTR_BLD_SRC}" > + "${INSTR_IR_GEN}" > +- COMMAND InstrGen > ++ COMMAND ${GLOW_BINARY_DIR}/bin/InstrGen > + "${INSTR_HDR}" "${INSTR_SRC}" "${INSTR_DEF}" > + "${INSTR_BLD_HDR}" "${INSTR_BLD_SRC}" "${INSTR_IR_GEN}" > +- DEPENDS InstrGen > + COMMENT "InstrGen: Generating instructions." VERBATIM) > + add_custom_target(AutoGenInstr > + DEPENDS > +-- > +2.17.1 > + > diff --git a/recipes-devtools/glow/glow_git.bb b/recipes-devtools/glow/glow_git.bb > index 3aea4b1..6ac8044 100644 > --- a/recipes-devtools/glow/glow_git.bb > +++ b/recipes-devtools/glow/glow_git.bb > @@ -8,13 +8,12 @@ PV = "0.1+git${SRCPV}" > PR = "r0" > SRCREV_FORMAT = "glow" > > -DEPENDS_class-native = "zlib zlib-native llvm-glow-native ninja-native protobuf-native libpng-native protobuf-native protobuf-c-native gflags-native glog-native" > -DEPENDS_class-nativesdk = "ncurses protobuf libpng glow-native zlib llvm-glow-native ninja-native protobuf-native libpng-native protobuf-c-native gflags-native glog-native" > +DEPENDS_class-native = "clang8 zlib libpng ninja protobuf protobuf-c gflags glog " > +DEPENDS_class-nativesdk = "ncurses protobuf zlib gflags glog libpng llvm8 glow-native ninja-native " > > -RDEPENDS_${PN}_class-nativesdk = "libpng ncurses glog" > +RDEPENDS_${PN}_class-nativesdk = " ncurses protobuf " > > SRC_URI = "git://github.com/pytorch/glow.git;protocol=https;destsuffix=git;name=glow \ > - file://0001-Fix-path-and-dependencies-to-intermediate-executable.patch \ > git://github.com/stp/OutputCheck.git;protocol=https;destsuffix=git/tests/OutputCheck;name=outcheck \ > git://github.com/google/benchmark.git;protocol=https;destsuffix=git/tests/googlebenchmark;name=gbench \ > git://github.com/google/googletest.git;protocol=https;destsuffix=git/tests/googletest;name=gtest \ > @@ -26,6 +25,11 @@ SRC_URI = "git://github.com/pytorch/glow.git;protocol=https;destsuffix=git;name= > git://github.com/wjakob/clang-cindex-python3.git;protocol=https;destsuffix=git/thirdparty/onnx/third_party/pybind11/tools/clang;name=onnxclang \ > " > > +SRC_URI_append_class-native = " file://0001-Fix-path-and-dependencies-to-intermediate-executable.patch \ > + " > +SRC_URI_append_class-nativesdk = "file://0002-Skip-compilation-of-native-helper-tools.patch \ > + " > + > SRCREV_glow = "ba8061367aea0b472b596448018107ca6e87929e" > SRCREV_outcheck = "eab62a5dd5129f6a4ebfbe4bbe41d35611f7c48d" > SRCREV_gbench = "090faecb454fbd6e6e17a75ef8146acb037118d4" > @@ -41,23 +45,49 @@ S = "${WORKDIR}/git" > > inherit pkgconfig cmake > > -EXTRA_OECMAKE += " -DCMAKE_BUILD_TYPE=Release -DCMAKE_EXE_LINKER_FLAGS="-Wl,-dynamic-linker=/lib64/ld-linux-x86-64.so.2 " " > +CP_ARGS="-Prf --preserve=mode,timestamps --no-preserve=ownership" > + > +do_configure_class-native() { > + cd ${B} > + cmake ${S} -G Ninja -DCMAKE_BUILD_TYPE=Release -DCMAKE_EXE_LINKER_FLAGS="-Wl,-z,origin -ldl -lz -ltinfo -pthread " > +} > + > +do_compile_class-native() { > + cd ${B} > + ninja include-bin InstrGen NodeGen image-classifier > +} > + > +do_install_class-native() { > + install -d ${D}${datadir}/glow/bin > + install -d ${D}${datadir}/glow/tests/images > + chrpath -d ${WORKDIR}/build/bin/* > + cp ${CP_ARGS} ${B}/bin/* ${D}${datadir}/glow/bin/. > + cp ${CP_ARGS} ${S}/tests/images/mnist ${D}${datadir}/glow/tests/images > +} > + > > -do_compile() { > - cmake_runcmake_build --target ${OECMAKE_TARGET_COMPILE} > +do_configure_class-nativesdk() { > + cd ${B} > + cmake ${S} -G Ninja -DCMAKE_BUILD_TYPE=Release -DCMAKE_EXE_LINKER_FLAGS="-Wl,-z,origin -ldl -lz -ltinfo -pthread " -DRUN_HAVE_STD_REGEX=0 -DRUN_HAVE_POSIX_REGEX=0 > } > > -do_install () { > - install -d ${D}${datadir}/glow > +do_compile_class-nativesdk() { > + cd ${B} > + cp ${CP_ARGS} ${STAGING_DIR_NATIVE}/usr/share/glow/bin/* ${WORKDIR}/build/bin > + ninja image-classifier > +} > + > +do_install_class-nativesdk() { > + install -d ${D}${datadir}/glow/bin > install -d ${D}${datadir}/glow/tests/images > - cp -Prf --preserve=mode,timestamps ${WORKDIR}/build/bin ${D}${datadir}/glow > - cp -Prf --preserve=mode,timestamps ${WORKDIR}/build/tests/images/EmotionSampleImages ${D}${datadir}/glow/tests/images > - cp -Prf --preserve=mode,timestamps ${WORKDIR}/build/tests/images/imagenet ${D}${datadir}/glow/tests/images > - cp -Prf --preserve=mode,timestamps ${WORKDIR}/build/tests/images/imagenet_299 ${D}${datadir}/glow/tests/images > - cp -Prf --preserve=mode,timestamps ${WORKDIR}/build/tests/images/mnist ${D}${datadir}/glow/tests/images > - cp -Prf --preserve=mode,timestamps ${WORKDIR}/build/tests/images/other ${D}${datadir}/glow/tests/images > + install -m 0755 ${B}/bin/image-classifier ${D}${datadir}/glow/bin/image-classifier > + chrpath -r "\$ORIGIN/../../../lib:\$ORIGIN/../../../../lib" ${D}${datadir}/glow/bin/image-classifier > + cp ${CP_ARGS} ${S}/tests/images/imagenet ${D}${datadir}/glow/tests/images > + cp ${CP_ARGS} ${S}/tests/images/imagenet_299 ${D}${datadir}/glow/tests/images > + cp ${CP_ARGS} ${S}/tests/images/mnist ${D}${datadir}/glow/tests/images > } > > + > BBCLASSEXTEND = "native nativesdk" > > -FILES_${PN} = "${datadir}" > +FILES_${PN} = "${datadir}/glow" ^ permalink raw reply [flat|nested] 12+ messages in thread
* [meta-processor-sdk][PATCHv2 4/5] demo-glow: Restore in top-level makefile examples after recipe cleanup 2019-08-23 13:58 [meta-processor-sdk][PATCHv2 0/5] Pytorch GLOW AOT tool and demo example Djordje Senicic ` (2 preceding siblings ...) 2019-08-23 13:58 ` [meta-processor-sdk][PATCHv2 3/5] glow: Address devkit related RPATH issue to point to devkit shared libraries instead of host native Djordje Senicic @ 2019-08-23 13:58 ` Djordje Senicic 2019-08-23 16:17 ` [EXTERNAL] " Jacob Stiffler 2019-08-23 13:58 ` [meta-processor-sdk][PATCHv2 5/5] packagegroup-arago-tisdk: Include xNN glow based compiler tool into devkit, and demo-glow example Djordje Senicic 4 siblings, 1 reply; 12+ messages in thread From: Djordje Senicic @ 2019-08-23 13:58 UTC (permalink / raw) To: meta-arago; +Cc: d-senicic1, Djordje Senicic - demo-glow is added back to top level example-applications Signed-off-by: Djordje Senicic <x0157990@ti.com> --- recipes-apps/demo-glow/demo-glow.bb | 16 +++++++++------- recipes-apps/demo-glow/files/Makefile.build | 3 ++- .../ti-tisdk-makefile_1.0.bbappend | 4 ++++ 3 files changed, 15 insertions(+), 8 deletions(-) diff --git a/recipes-apps/demo-glow/demo-glow.bb b/recipes-apps/demo-glow/demo-glow.bb index 21d6bdf..76bd224 100644 --- a/recipes-apps/demo-glow/demo-glow.bb +++ b/recipes-apps/demo-glow/demo-glow.bb @@ -7,7 +7,7 @@ LIC_FILES_CHKSUM = "file://LICENSE;md5=86d3f3a95c324c9479bd8986968f4327" PV = "1.0" PR = "r1" -DEPENDS = "glow-native libpng" +DEPENDS = "glow-native libpng protobuf-native " SRC_URI = "https://raw.githubusercontent.com/pytorch/glow/1af031c9c040dd4ff843d9b50f83e71f6e605631/examples/bundles/lenet_mnist/main.cpp;name=lenet_mnist_main;subdir=src \ file://0001-fix-missing-string.patch \ @@ -23,6 +23,8 @@ SRC_URI[lenet_mnist_license.md5sum] = "86d3f3a95c324c9479bd8986968f4327" S = "${WORKDIR}/src" +CP_ARGS="-Prf --preserve=mode,timestamps --no-preserve=ownership" + do_compile() { install -d ${WORKDIR}/build LD_LIBRARY_PATH=${STAGING_DIR_NATIVE}/usr/lib ${STAGING_DIR_NATIVE}/usr/share/glow/bin/image-classifier ${STAGING_DIR_NATIVE}/usr/share/glow/tests/images/mnist/1_1008.png -m=mnist.onnx -image-mode=0to1 -model-input-name=data_0 -emit-bundle ${WORKDIR}/build/ -cpu -target armv7l-unknown-linux-gnueabihf -network-name="lenet_mnist" @@ -32,12 +34,12 @@ do_compile() { do_install () { install -d ${D}${datadir}/glow/infer install -d ${D}${datadir}/glow/tests - cp -Prf --preserve=mode,timestamps ${WORKDIR}/build/*.weights ${D}${datadir}/glow/infer - cp -Prf --preserve=mode,timestamps ${WORKDIR}/build/infer_mnist ${D}${datadir}/glow/infer - cp -Prf --preserve=mode,timestamps ${WORKDIR}/build/lenet_mnist.o ${D}${datadir}/glow/infer - cp -Prf --preserve=mode,timestamps ${WORKDIR}/src/makefile ${D}${datadir}/glow/infer - cp -Prf --preserve=mode,timestamps ${WORKDIR}/src/main.cpp ${D}${datadir}/glow/infer - cp -Prf --preserve=mode,timestamps ${STAGING_DIR_NATIVE}/usr/share/glow/tests/images/mnist/*.png ${D}${datadir}/glow/tests/. + cp ${CP_ARGS} ${WORKDIR}/build/*.weights ${D}${datadir}/glow/infer + cp $(CP_ARGS) ${WORKDIR}/build/infer_mnist ${D}${datadir}/glow/infer + cp ${CP_ARGS} ${WORKDIR}/build/lenet_mnist.o ${D}${datadir}/glow/infer + cp ${CP_ARGS} ${WORKDIR}/src/makefile ${D}${datadir}/glow/infer + cp ${CP_ARGS} ${WORKDIR}/src/main.cpp ${D}${datadir}/glow/infer + cp ${CP_ARGS} ${STAGING_DIR_NATIVE}/usr/share/glow/tests/images/mnist/*.png ${D}${datadir}/glow/tests/. } FILES_${PN} += "${datadir}/glow" diff --git a/recipes-apps/demo-glow/files/Makefile.build b/recipes-apps/demo-glow/files/Makefile.build index 044bbf9..baf1fc0 100644 --- a/recipes-apps/demo-glow/files/Makefile.build +++ b/recipes-apps/demo-glow/files/Makefile.build @@ -6,7 +6,8 @@ DESTDIR ?= all: release lenet_mnist: - LD_LIBRARY_PATH=${SDK_PATH_NATIVE}/usr/lib ${SDK_PATH_NATIVE}/usr/share/glow/bin/image-classifier ${SDK_PATH_NATIVE}/usr/share/glow/tests/images/mnist/1_1008.png -m=mnist.onnx -image-mode=0to1 -model-input-name=data_0 -emit-bundle ./ -cpu -target armv7l-unknown-linux-gnueabihf -network-name="lenet_mnist" + @. ${ENV_SETUP}; \ + ${SDK_PATH_NATIVE}/usr/share/glow/bin/image-classifier ${SDK_PATH_NATIVE}/usr/share/glow/tests/images/mnist/1_1008.png -m=mnist.onnx -image-mode=0to1 -model-input-name=data_0 -emit-bundle ./ -cpu -target armv7l-unknown-linux-gnueabihf -network-name="lenet_mnist" release: lenet_mnist @. ${ENV_SETUP}; \ diff --git a/recipes-tisdk/ti-tisdk-makefile/ti-tisdk-makefile_1.0.bbappend b/recipes-tisdk/ti-tisdk-makefile/ti-tisdk-makefile_1.0.bbappend index acd396b..ead9d53 100644 --- a/recipes-tisdk/ti-tisdk-makefile/ti-tisdk-makefile_1.0.bbappend +++ b/recipes-tisdk/ti-tisdk-makefile/ti-tisdk-makefile_1.0.bbappend @@ -10,6 +10,7 @@ SRC_URI_append = "\ file://Makefile_tiovx-app-host \ file://Makefile_tidl-examples \ file://Makefile_pru-adc \ + file://Makefile_demo-glow \ " SRC_URI_append_omap-a15 = " file://Makefile_big-data-ipc-demo" @@ -29,13 +30,16 @@ MAKEFILES_append_omap-a15 = " dual-camera-demo \ " MAKEFILES_append_omap-a15 = " video-graphics-test \ + demo-glow \ " MAKEFILES_append_ti43x = " evse-hmi \ + demo-glow \ " MAKEFILES_append_ti33x = " evse-hmi \ protection-relays-hmi \ + demo-glow \ " MAKEFILES_append_k3 = " evse-hmi" -- 2.17.1 ^ permalink raw reply related [flat|nested] 12+ messages in thread
* Re: [EXTERNAL] [meta-processor-sdk][PATCHv2 4/5] demo-glow: Restore in top-level makefile examples after recipe cleanup 2019-08-23 13:58 ` [meta-processor-sdk][PATCHv2 4/5] demo-glow: Restore in top-level makefile examples after recipe cleanup Djordje Senicic @ 2019-08-23 16:17 ` Jacob Stiffler 0 siblings, 0 replies; 12+ messages in thread From: Jacob Stiffler @ 2019-08-23 16:17 UTC (permalink / raw) To: Djordje Senicic, meta-arago; +Cc: d-senicic1 This fails to apply, can you rebase? On 8/23/2019 9:58 AM, Djordje Senicic wrote: > - demo-glow is added back to top level example-applications > > Signed-off-by: Djordje Senicic <x0157990@ti.com> > --- > recipes-apps/demo-glow/demo-glow.bb | 16 +++++++++------- > recipes-apps/demo-glow/files/Makefile.build | 3 ++- > .../ti-tisdk-makefile_1.0.bbappend | 4 ++++ > 3 files changed, 15 insertions(+), 8 deletions(-) > > diff --git a/recipes-apps/demo-glow/demo-glow.bb b/recipes-apps/demo-glow/demo-glow.bb > index 21d6bdf..76bd224 100644 > --- a/recipes-apps/demo-glow/demo-glow.bb > +++ b/recipes-apps/demo-glow/demo-glow.bb > @@ -7,7 +7,7 @@ LIC_FILES_CHKSUM = "file://LICENSE;md5=86d3f3a95c324c9479bd8986968f4327" > PV = "1.0" > PR = "r1" > > -DEPENDS = "glow-native libpng" > +DEPENDS = "glow-native libpng protobuf-native " > > SRC_URI = "https://raw.githubusercontent.com/pytorch/glow/1af031c9c040dd4ff843d9b50f83e71f6e605631/examples/bundles/lenet_mnist/main.cpp;name=lenet_mnist_main;subdir=src \ > file://0001-fix-missing-string.patch \ > @@ -23,6 +23,8 @@ SRC_URI[lenet_mnist_license.md5sum] = "86d3f3a95c324c9479bd8986968f4327" > > S = "${WORKDIR}/src" > > +CP_ARGS="-Prf --preserve=mode,timestamps --no-preserve=ownership" > + > do_compile() { > install -d ${WORKDIR}/build > LD_LIBRARY_PATH=${STAGING_DIR_NATIVE}/usr/lib ${STAGING_DIR_NATIVE}/usr/share/glow/bin/image-classifier ${STAGING_DIR_NATIVE}/usr/share/glow/tests/images/mnist/1_1008.png -m=mnist.onnx -image-mode=0to1 -model-input-name=data_0 -emit-bundle ${WORKDIR}/build/ -cpu -target armv7l-unknown-linux-gnueabihf -network-name="lenet_mnist" > @@ -32,12 +34,12 @@ do_compile() { > do_install () { > install -d ${D}${datadir}/glow/infer > install -d ${D}${datadir}/glow/tests > - cp -Prf --preserve=mode,timestamps ${WORKDIR}/build/*.weights ${D}${datadir}/glow/infer > - cp -Prf --preserve=mode,timestamps ${WORKDIR}/build/infer_mnist ${D}${datadir}/glow/infer > - cp -Prf --preserve=mode,timestamps ${WORKDIR}/build/lenet_mnist.o ${D}${datadir}/glow/infer > - cp -Prf --preserve=mode,timestamps ${WORKDIR}/src/makefile ${D}${datadir}/glow/infer > - cp -Prf --preserve=mode,timestamps ${WORKDIR}/src/main.cpp ${D}${datadir}/glow/infer > - cp -Prf --preserve=mode,timestamps ${STAGING_DIR_NATIVE}/usr/share/glow/tests/images/mnist/*.png ${D}${datadir}/glow/tests/. > + cp ${CP_ARGS} ${WORKDIR}/build/*.weights ${D}${datadir}/glow/infer > + cp $(CP_ARGS) ${WORKDIR}/build/infer_mnist ${D}${datadir}/glow/infer > + cp ${CP_ARGS} ${WORKDIR}/build/lenet_mnist.o ${D}${datadir}/glow/infer > + cp ${CP_ARGS} ${WORKDIR}/src/makefile ${D}${datadir}/glow/infer > + cp ${CP_ARGS} ${WORKDIR}/src/main.cpp ${D}${datadir}/glow/infer > + cp ${CP_ARGS} ${STAGING_DIR_NATIVE}/usr/share/glow/tests/images/mnist/*.png ${D}${datadir}/glow/tests/. > } > > FILES_${PN} += "${datadir}/glow" > diff --git a/recipes-apps/demo-glow/files/Makefile.build b/recipes-apps/demo-glow/files/Makefile.build > index 044bbf9..baf1fc0 100644 > --- a/recipes-apps/demo-glow/files/Makefile.build > +++ b/recipes-apps/demo-glow/files/Makefile.build > @@ -6,7 +6,8 @@ DESTDIR ?= > all: release > > lenet_mnist: > - LD_LIBRARY_PATH=${SDK_PATH_NATIVE}/usr/lib ${SDK_PATH_NATIVE}/usr/share/glow/bin/image-classifier ${SDK_PATH_NATIVE}/usr/share/glow/tests/images/mnist/1_1008.png -m=mnist.onnx -image-mode=0to1 -model-input-name=data_0 -emit-bundle ./ -cpu -target armv7l-unknown-linux-gnueabihf -network-name="lenet_mnist" > + @. ${ENV_SETUP}; \ > + ${SDK_PATH_NATIVE}/usr/share/glow/bin/image-classifier ${SDK_PATH_NATIVE}/usr/share/glow/tests/images/mnist/1_1008.png -m=mnist.onnx -image-mode=0to1 -model-input-name=data_0 -emit-bundle ./ -cpu -target armv7l-unknown-linux-gnueabihf -network-name="lenet_mnist" > > release: lenet_mnist > @. ${ENV_SETUP}; \ > diff --git a/recipes-tisdk/ti-tisdk-makefile/ti-tisdk-makefile_1.0.bbappend b/recipes-tisdk/ti-tisdk-makefile/ti-tisdk-makefile_1.0.bbappend > index acd396b..ead9d53 100644 > --- a/recipes-tisdk/ti-tisdk-makefile/ti-tisdk-makefile_1.0.bbappend > +++ b/recipes-tisdk/ti-tisdk-makefile/ti-tisdk-makefile_1.0.bbappend > @@ -10,6 +10,7 @@ SRC_URI_append = "\ > file://Makefile_tiovx-app-host \ > file://Makefile_tidl-examples \ > file://Makefile_pru-adc \ > + file://Makefile_demo-glow \ > " > > SRC_URI_append_omap-a15 = " file://Makefile_big-data-ipc-demo" > @@ -29,13 +30,16 @@ MAKEFILES_append_omap-a15 = " dual-camera-demo \ > " > > MAKEFILES_append_omap-a15 = " video-graphics-test \ > + demo-glow \ > " > > MAKEFILES_append_ti43x = " evse-hmi \ > + demo-glow \ > " > > MAKEFILES_append_ti33x = " evse-hmi \ > protection-relays-hmi \ > + demo-glow \ > " > > MAKEFILES_append_k3 = " evse-hmi" ^ permalink raw reply [flat|nested] 12+ messages in thread
* [meta-processor-sdk][PATCHv2 5/5] packagegroup-arago-tisdk: Include xNN glow based compiler tool into devkit, and demo-glow example 2019-08-23 13:58 [meta-processor-sdk][PATCHv2 0/5] Pytorch GLOW AOT tool and demo example Djordje Senicic ` (3 preceding siblings ...) 2019-08-23 13:58 ` [meta-processor-sdk][PATCHv2 4/5] demo-glow: Restore in top-level makefile examples after recipe cleanup Djordje Senicic @ 2019-08-23 13:58 ` Djordje Senicic 4 siblings, 0 replies; 12+ messages in thread From: Djordje Senicic @ 2019-08-23 13:58 UTC (permalink / raw) To: meta-arago; +Cc: d-senicic1, Djordje Senicic - image-classifier tool which converts xNN model into executable object code and sample test images are included into devkit - An example in demo-glow which convets lenet mnist model into object code and links into application is added to target filesystem and also top level example-applications Signed-off-by: Djordje Senicic <x0157990@ti.com> --- .../nativesdk-packagegroup-arago-tisdk-host.bbappend | 4 ++++ .../packagegroup-arago-tisdk-addons-sdk-host.bbappend | 11 +++++++++++ ...ackagegroup-arago-tisdk-addons-sdk-target.bbappend | 6 ++++++ .../packagegroup-arago-tisdk-addons.bbappend | 7 +++++++ 4 files changed, 28 insertions(+) create mode 100644 recipes-core/packagegroups/nativesdk-packagegroup-arago-tisdk-host.bbappend diff --git a/recipes-core/packagegroups/nativesdk-packagegroup-arago-tisdk-host.bbappend b/recipes-core/packagegroups/nativesdk-packagegroup-arago-tisdk-host.bbappend new file mode 100644 index 0000000..b35282e --- /dev/null +++ b/recipes-core/packagegroups/nativesdk-packagegroup-arago-tisdk-host.bbappend @@ -0,0 +1,4 @@ +PR_append = ".tisdk0" + +EXTRA_TI_TOOLS_append = " nativesdk-glow " + diff --git a/recipes-core/packagegroups/packagegroup-arago-tisdk-addons-sdk-host.bbappend b/recipes-core/packagegroups/packagegroup-arago-tisdk-addons-sdk-host.bbappend index a8a0b2e..6349bcd 100644 --- a/recipes-core/packagegroups/packagegroup-arago-tisdk-addons-sdk-host.bbappend +++ b/recipes-core/packagegroups/packagegroup-arago-tisdk-addons-sdk-host.bbappend @@ -18,3 +18,14 @@ EXTRA_LIBS_append_dra7xx = " \ UTILS_append_am335x-evm = " pru-adc-src" +EXTRA_PACKAGES_append_omap-a15 = " \ + demo-glow-src \ +" + +EXTRA_PACKAGES_append_ti43x = " \ + demo-glow-src \ +" + +EXTRA_PACKAGES_append_ti33x = " \ + demo-glow-src \ +" diff --git a/recipes-core/packagegroups/packagegroup-arago-tisdk-addons-sdk-target.bbappend b/recipes-core/packagegroups/packagegroup-arago-tisdk-addons-sdk-target.bbappend index 08f20e4..6d8a49e 100644 --- a/recipes-core/packagegroups/packagegroup-arago-tisdk-addons-sdk-target.bbappend +++ b/recipes-core/packagegroups/packagegroup-arago-tisdk-addons-sdk-target.bbappend @@ -77,3 +77,9 @@ EXTRA_LIBS_append_dra7xx = " \ tiovx-sys-iface \ " +EXTRA_PACKAGES_append_ti33x = " demo-glow-dev " + +EXTRA_PACKAGES_append_ti43x = " demo-glow-dev " + +EXTRA_PACKAGES_append_omap-a15 = " demo-glow-dev " + diff --git a/recipes-core/packagegroups/packagegroup-arago-tisdk-addons.bbappend b/recipes-core/packagegroups/packagegroup-arago-tisdk-addons.bbappend index 0cf0322..f365dc6 100644 --- a/recipes-core/packagegroups/packagegroup-arago-tisdk-addons.bbappend +++ b/recipes-core/packagegroups/packagegroup-arago-tisdk-addons.bbappend @@ -26,3 +26,10 @@ EXTRA_PACKAGES_append_armv7a = " zbar" UTILS_append = " net-snmp net-snmp-server-snmpd" EXTRA_PACKAGES_append_dra7xx = " tiovx-app-host tiovx-app-host-examples" + +EXTRA_PACKAGES_append_ti33x = " demo-glow " + +EXTRA_PACKAGES_append_ti43x = " demo-glow " + +EXTRA_PACKAGES_append_omap-a15 = " demo-glow " + -- 2.17.1 ^ permalink raw reply related [flat|nested] 12+ messages in thread
end of thread, other threads:[~2019-09-18 19:41 UTC | newest] Thread overview: 12+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2019-08-23 13:58 [meta-processor-sdk][PATCHv2 0/5] Pytorch GLOW AOT tool and demo example Djordje Senicic 2019-08-23 13:58 ` [meta-processor-sdk][PATCHv2 1/5] mesa, llvm, meson: Update llvm to 8.0.1 plus define and use LLVM version globally Djordje Senicic 2019-08-23 16:15 ` [EXTERNAL] " Jacob Stiffler 2019-08-23 17:34 ` Senicic, Djordje 2019-08-23 18:06 ` Jacob Stiffler 2019-08-23 18:17 ` Senicic, Djordje 2019-08-23 13:58 ` [meta-processor-sdk][PATCHv2 2/5] clang8-native: llvm and clang used only for compilation of native tools for glow compiler Djordje Senicic 2019-08-23 13:58 ` [meta-processor-sdk][PATCHv2 3/5] glow: Address devkit related RPATH issue to point to devkit shared libraries instead of host native Djordje Senicic 2019-09-18 19:39 ` [EXTERNAL] " Jacob Stiffler 2019-08-23 13:58 ` [meta-processor-sdk][PATCHv2 4/5] demo-glow: Restore in top-level makefile examples after recipe cleanup Djordje Senicic 2019-08-23 16:17 ` [EXTERNAL] " Jacob Stiffler 2019-08-23 13:58 ` [meta-processor-sdk][PATCHv2 5/5] packagegroup-arago-tisdk: Include xNN glow based compiler tool into devkit, and demo-glow example Djordje Senicic
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.