* [Buildroot] [PATCH v2 2/6] package/llvm: install target binary/debug tools
2018-11-17 3:56 [Buildroot] [PATCH v2 1/6] package/compiler-rt: new package Matt Weber
@ 2018-11-17 3:56 ` Matt Weber
2018-11-18 19:14 ` Romain Naour
2018-11-17 3:56 ` [Buildroot] [PATCH v2 3/6] testing/infra/builder: build with target and environment Matt Weber
` (5 subsequent siblings)
6 siblings, 1 reply; 22+ messages in thread
From: Matt Weber @ 2018-11-17 3:56 UTC (permalink / raw)
To: buildroot
The compiler-rt fuzzer and address sanitizer tools require additional
LLVM binary tools installed to allow stack trace decoding actively during
executable analysis.
https://github.com/google/sanitizers/wiki/AddressSanitizerCallStack
Signed-off-by: Matthew Weber <matthew.weber@rockwellcollins.com>
Reviewed-by: Romain Naour <romain.naour@smile.fr>
---
Changes
v1 -> v2
- None
---
package/llvm/llvm.mk | 11 ++++++++++-
1 file changed, 10 insertions(+), 1 deletion(-)
diff --git a/package/llvm/llvm.mk b/package/llvm/llvm.mk
index 1f9bd44..79477f0 100644
--- a/package/llvm/llvm.mk
+++ b/package/llvm/llvm.mk
@@ -197,8 +197,17 @@ HOST_LLVM_CONF_OPTS += \
# We need to activate LLVM_INCLUDE_TOOLS, otherwise it does not generate
# libLLVM.so
LLVM_CONF_OPTS += \
- -DLLVM_INCLUDE_TOOLS=ON \
+ -DLLVM_INCLUDE_TOOLS=ON
+
+# The llvm-symbolizer binary is used by the Compiler-RT Fuzzer
+# and AddressSanitizer tools for stack traces.
+ifeq ($(BR2_PACKAGE_COMPILER_RT),y)
+LLVM_CONF_OPTS += \
+ -DLLVM_BUILD_TOOLS=ON
+else
+LLVM_CONF_OPTS += \
-DLLVM_BUILD_TOOLS=OFF
+endif
# Compiler-rt not in the source tree.
# llvm runtime libraries are not in the source tree.
--
1.9.1
^ permalink raw reply related [flat|nested] 22+ messages in thread* [Buildroot] [PATCH v2 2/6] package/llvm: install target binary/debug tools
2018-11-17 3:56 ` [Buildroot] [PATCH v2 2/6] package/llvm: install target binary/debug tools Matt Weber
@ 2018-11-18 19:14 ` Romain Naour
2018-11-19 1:48 ` Matthew Weber
0 siblings, 1 reply; 22+ messages in thread
From: Romain Naour @ 2018-11-18 19:14 UTC (permalink / raw)
To: buildroot
Hi Matt,
Le 17/11/2018 ? 04:56, Matt Weber a ?crit?:
> The compiler-rt fuzzer and address sanitizer tools require additional
> LLVM binary tools installed to allow stack trace decoding actively during
> executable analysis.
>
> https://github.com/google/sanitizers/wiki/AddressSanitizerCallStack
>
> Signed-off-by: Matthew Weber <matthew.weber@rockwellcollins.com>
> Reviewed-by: Romain Naour <romain.naour@smile.fr>
For some reason I get an error while building llvm package tools.
x86_64-linux-g++: WARNING: unsafe header/library path used in cross-compilation:
'-I/usr/include/libxml2'
Can you have a look?
Best regards,
Romain
> ---
> Changes
> v1 -> v2
> - None
> ---
> package/llvm/llvm.mk | 11 ++++++++++-
> 1 file changed, 10 insertions(+), 1 deletion(-)
>
> diff --git a/package/llvm/llvm.mk b/package/llvm/llvm.mk
> index 1f9bd44..79477f0 100644
> --- a/package/llvm/llvm.mk
> +++ b/package/llvm/llvm.mk
> @@ -197,8 +197,17 @@ HOST_LLVM_CONF_OPTS += \
> # We need to activate LLVM_INCLUDE_TOOLS, otherwise it does not generate
> # libLLVM.so
> LLVM_CONF_OPTS += \
> - -DLLVM_INCLUDE_TOOLS=ON \
> + -DLLVM_INCLUDE_TOOLS=ON
> +
> +# The llvm-symbolizer binary is used by the Compiler-RT Fuzzer
> +# and AddressSanitizer tools for stack traces.
> +ifeq ($(BR2_PACKAGE_COMPILER_RT),y)
> +LLVM_CONF_OPTS += \
> + -DLLVM_BUILD_TOOLS=ON
> +else
> +LLVM_CONF_OPTS += \
> -DLLVM_BUILD_TOOLS=OFF
> +endif
>
> # Compiler-rt not in the source tree.
> # llvm runtime libraries are not in the source tree.
>
^ permalink raw reply [flat|nested] 22+ messages in thread
* [Buildroot] [PATCH v2 2/6] package/llvm: install target binary/debug tools
2018-11-18 19:14 ` Romain Naour
@ 2018-11-19 1:48 ` Matthew Weber
2018-11-19 8:09 ` Romain Naour
0 siblings, 1 reply; 22+ messages in thread
From: Matthew Weber @ 2018-11-19 1:48 UTC (permalink / raw)
To: buildroot
Romain,
On Sun, Nov 18, 2018 at 1:14 PM Romain Naour <romain.naour@smile.fr> wrote:
>
> Hi Matt,
>
> Le 17/11/2018 ? 04:56, Matt Weber a ?crit :
> > The compiler-rt fuzzer and address sanitizer tools require additional
> > LLVM binary tools installed to allow stack trace decoding actively during
> > executable analysis.
> >
> > https://github.com/google/sanitizers/wiki/AddressSanitizerCallStack
> >
> > Signed-off-by: Matthew Weber <matthew.weber@rockwellcollins.com>
> > Reviewed-by: Romain Naour <romain.naour@smile.fr>
>
> For some reason I get an error while building llvm package tools.
>
> x86_64-linux-g++: WARNING: unsafe header/library path used in cross-compilation:
> '-I/usr/include/libxml2'
>
> Can you have a look?
I went back and checked my logs on the clang compiler-rt run-test
build for that warning and didn't find it. I then checked and my
config and it has the unsafe header/library as error. Was your build
test just applying my patches and doing an explicit 'make llvm'?
Thanks
Matt
^ permalink raw reply [flat|nested] 22+ messages in thread
* [Buildroot] [PATCH v2 2/6] package/llvm: install target binary/debug tools
2018-11-19 1:48 ` Matthew Weber
@ 2018-11-19 8:09 ` Romain Naour
2018-11-19 13:04 ` Matthew Weber
0 siblings, 1 reply; 22+ messages in thread
From: Romain Naour @ 2018-11-19 8:09 UTC (permalink / raw)
To: buildroot
Hi Matt,
Le lun. 19 nov. 2018 ? 02:48, Matthew Weber <
matthew.weber@rockwellcollins.com> a ?crit :
> Romain,
>
> On Sun, Nov 18, 2018 at 1:14 PM Romain Naour <romain.naour@smile.fr>
> wrote:
> >
> > Hi Matt,
> >
> > Le 17/11/2018 ? 04:56, Matt Weber a ?crit :
> > > The compiler-rt fuzzer and address sanitizer tools require additional
> > > LLVM binary tools installed to allow stack trace decoding actively
> during
> > > executable analysis.
> > >
> > > https://github.com/google/sanitizers/wiki/AddressSanitizerCallStack
> > >
> > > Signed-off-by: Matthew Weber <matthew.weber@rockwellcollins.com>
> > > Reviewed-by: Romain Naour <romain.naour@smile.fr>
> >
> > For some reason I get an error while building llvm package tools.
> >
> > x86_64-linux-g++: WARNING: unsafe header/library path used in
> cross-compilation:
> > '-I/usr/include/libxml2'
> >
> > Can you have a look?
>
> I went back and checked my logs on the clang compiler-rt run-test
> build for that warning and didn't find it. I then checked and my
> config and it has the unsafe header/library as error. Was your build
> test just applying my patches and doing an explicit 'make llvm'?
>
Indeed,
Some tools use libxml2, so you have to add target libxml2 package as build
dependency.
Best regards,
Romain
> Thanks
> Matt
> _______________________________________________
> buildroot mailing list
> buildroot at busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20181119/ae4b7cff/attachment.html>
^ permalink raw reply [flat|nested] 22+ messages in thread
* [Buildroot] [PATCH v2 2/6] package/llvm: install target binary/debug tools
2018-11-19 8:09 ` Romain Naour
@ 2018-11-19 13:04 ` Matthew Weber
2018-11-20 22:36 ` Romain Naour
0 siblings, 1 reply; 22+ messages in thread
From: Matthew Weber @ 2018-11-19 13:04 UTC (permalink / raw)
To: buildroot
Romain,
On Mon, Nov 19, 2018 at 2:10 AM Romain Naour <romain.naour@gmail.com> wrote:
>
> Hi Matt,
>
> Le lun. 19 nov. 2018 ? 02:48, Matthew Weber <matthew.weber@rockwellcollins.com> a ?crit :
>>
>> Romain,
>>
>> On Sun, Nov 18, 2018 at 1:14 PM Romain Naour <romain.naour@smile.fr> wrote:
>> >
>> > Hi Matt,
>> >
>> > Le 17/11/2018 ? 04:56, Matt Weber a ?crit :
>> > > The compiler-rt fuzzer and address sanitizer tools require additional
>> > > LLVM binary tools installed to allow stack trace decoding actively during
>> > > executable analysis.
>> > >
>> > > https://github.com/google/sanitizers/wiki/AddressSanitizerCallStack
>> > >
>> > > Signed-off-by: Matthew Weber <matthew.weber@rockwellcollins.com>
>> > > Reviewed-by: Romain Naour <romain.naour@smile.fr>
>> >
>> > For some reason I get an error while building llvm package tools.
>> >
>> > x86_64-linux-g++: WARNING: unsafe header/library path used in cross-compilation:
>> > '-I/usr/include/libxml2'
>> >
>> > Can you have a look?
>>
>> I went back and checked my logs on the clang compiler-rt run-test
>> build for that warning and didn't find it. I then checked and my
>> config and it has the unsafe header/library as error. Was your build
>> test just applying my patches and doing an explicit 'make llvm'?
>
>
> Indeed,
> Some tools use libxml2, so you have to add target libxml2 package as build dependency.
>
Would you mind sharing your verbose output and equivalent of cmake
config guess/status?
^ permalink raw reply [flat|nested] 22+ messages in thread
* [Buildroot] [PATCH v2 2/6] package/llvm: install target binary/debug tools
2018-11-19 13:04 ` Matthew Weber
@ 2018-11-20 22:36 ` Romain Naour
2018-11-20 22:44 ` Matthew Weber
2018-11-21 3:12 ` Matthew Weber
0 siblings, 2 replies; 22+ messages in thread
From: Romain Naour @ 2018-11-20 22:36 UTC (permalink / raw)
To: buildroot
Hi Matt,
Le 19/11/2018 ? 14:04, Matthew Weber a ?crit?:
> Romain,
>
> On Mon, Nov 19, 2018 at 2:10 AM Romain Naour <romain.naour@gmail.com> wrote:
>>
>> Hi Matt,
>>
>> Le lun. 19 nov. 2018 ? 02:48, Matthew Weber <matthew.weber@rockwellcollins.com> a ?crit :
>>>
>>> Romain,
>>>
>>> On Sun, Nov 18, 2018 at 1:14 PM Romain Naour <romain.naour@smile.fr> wrote:
>>>>
>>>> Hi Matt,
>>>>
>>>> Le 17/11/2018 ? 04:56, Matt Weber a ?crit :
>>>>> The compiler-rt fuzzer and address sanitizer tools require additional
>>>>> LLVM binary tools installed to allow stack trace decoding actively during
>>>>> executable analysis.
>>>>>
>>>>> https://github.com/google/sanitizers/wiki/AddressSanitizerCallStack
>>>>>
>>>>> Signed-off-by: Matthew Weber <matthew.weber@rockwellcollins.com>
>>>>> Reviewed-by: Romain Naour <romain.naour@smile.fr>
>>>>
>>>> For some reason I get an error while building llvm package tools.
>>>>
>>>> x86_64-linux-g++: WARNING: unsafe header/library path used in cross-compilation:
>>>> '-I/usr/include/libxml2'
>>>>
>>>> Can you have a look?
>>>
>>> I went back and checked my logs on the clang compiler-rt run-test
>>> build for that warning and didn't find it. I then checked and my
>>> config and it has the unsafe header/library as error. Was your build
>>> test just applying my patches and doing an explicit 'make llvm'?
>>
>>
>> Indeed,
>> Some tools use libxml2, so you have to add target libxml2 package as build dependency.
>>
>
> Would you mind sharing your verbose output and equivalent of cmake
> config guess/status?
>
I need to rebuild from scratch since I've fixed the build locally.
I'll restart a new one next week-end.
Best regards,
Romain
^ permalink raw reply [flat|nested] 22+ messages in thread
* [Buildroot] [PATCH v2 2/6] package/llvm: install target binary/debug tools
2018-11-20 22:36 ` Romain Naour
@ 2018-11-20 22:44 ` Matthew Weber
2018-11-21 3:12 ` Matthew Weber
1 sibling, 0 replies; 22+ messages in thread
From: Matthew Weber @ 2018-11-20 22:44 UTC (permalink / raw)
To: buildroot
Romain,
On Tue, Nov 20, 2018, 16:37 Romain Naour <romain.naour@gmail.com wrote:
> Hi Matt,
>
> Le 19/11/2018 ? 14:04, Matthew Weber a ?crit :
> > Romain,
> >
> > On Mon, Nov 19, 2018 at 2:10 AM Romain Naour <romain.naour@gmail.com>
> wrote:
> >>
> >> Hi Matt,
> >>
> >> Le lun. 19 nov. 2018 ? 02:48, Matthew Weber <
> matthew.weber at rockwellcollins.com> a ?crit :
> >>>
> >>> Romain,
> >>>
> >>> On Sun, Nov 18, 2018 at 1:14 PM Romain Naour <romain.naour@smile.fr>
> wrote:
> >>>>
> >>>> Hi Matt,
> >>>>
> >>>> Le 17/11/2018 ? 04:56, Matt Weber a ?crit :
> >>>>> The compiler-rt fuzzer and address sanitizer tools require additional
> >>>>> LLVM binary tools installed to allow stack trace decoding actively
> during
> >>>>> executable analysis.
> >>>>>
> >>>>> https://github.com/google/sanitizers/wiki/AddressSanitizerCallStack
> >>>>>
> >>>>> Signed-off-by: Matthew Weber <matthew.weber@rockwellcollins.com>
> >>>>> Reviewed-by: Romain Naour <romain.naour@smile.fr>
> >>>>
> >>>> For some reason I get an error while building llvm package tools.
> >>>>
> >>>> x86_64-linux-g++: WARNING: unsafe header/library path used in
> cross-compilation:
> >>>> '-I/usr/include/libxml2'
> >>>>
> >>>> Can you have a look?
> >>>
> >>> I went back and checked my logs on the clang compiler-rt run-test
> >>> build for that warning and didn't find it. I then checked and my
> >>> config and it has the unsafe header/library as error. Was your build
> >>> test just applying my patches and doing an explicit 'make llvm'?
> >>
> >>
> >> Indeed,
> >> Some tools use libxml2, so you have to add target libxml2 package as
> build dependency.
> >>
> >
> > Would you mind sharing your verbose output and equivalent of cmake
> > config guess/status?
> >
>
> I need to rebuild from scratch since I've fixed the build locally.
> I'll restart a new one next week-end.
>
When I searched the llvm build, I see there are some xml dependencies. I
hadn't got far enough to find the variables in llvm build to adjust or if I
just need to make sure libxml2 is built first.
>
>
>
>
> Best regards,
> Romain
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20181120/f2b25b9f/attachment.html>
^ permalink raw reply [flat|nested] 22+ messages in thread
* [Buildroot] [PATCH v2 2/6] package/llvm: install target binary/debug tools
2018-11-20 22:36 ` Romain Naour
2018-11-20 22:44 ` Matthew Weber
@ 2018-11-21 3:12 ` Matthew Weber
2018-11-21 21:18 ` Romain Naour
1 sibling, 1 reply; 22+ messages in thread
From: Matthew Weber @ 2018-11-21 3:12 UTC (permalink / raw)
To: buildroot
On Tue, Nov 20, 2018 at 4:37 PM Romain Naour <romain.naour@gmail.com> wrote:
>
> Hi Matt,
>
> Le 19/11/2018 ? 14:04, Matthew Weber a ?crit :
> > Romain,
> >
> > On Mon, Nov 19, 2018 at 2:10 AM Romain Naour <romain.naour@gmail.com> wrote:
> >>
> >> Hi Matt,
> >>
> >> Le lun. 19 nov. 2018 ? 02:48, Matthew Weber <matthew.weber@rockwellcollins.com> a ?crit :
> >>>
> >>> Romain,
> >>>
> >>> On Sun, Nov 18, 2018 at 1:14 PM Romain Naour <romain.naour@smile.fr> wrote:
> >>>>
> >>>> Hi Matt,
> >>>>
> >>>> Le 17/11/2018 ? 04:56, Matt Weber a ?crit :
> >>>>> The compiler-rt fuzzer and address sanitizer tools require additional
> >>>>> LLVM binary tools installed to allow stack trace decoding actively during
> >>>>> executable analysis.
> >>>>>
> >>>>> https://github.com/google/sanitizers/wiki/AddressSanitizerCallStack
> >>>>>
> >>>>> Signed-off-by: Matthew Weber <matthew.weber@rockwellcollins.com>
> >>>>> Reviewed-by: Romain Naour <romain.naour@smile.fr>
> >>>>
> >>>> For some reason I get an error while building llvm package tools.
> >>>>
> >>>> x86_64-linux-g++: WARNING: unsafe header/library path used in cross-compilation:
> >>>> '-I/usr/include/libxml2'
> >>>>
> >>>> Can you have a look?
> >>>
> >>> I went back and checked my logs on the clang compiler-rt run-test
> >>> build for that warning and didn't find it. I then checked and my
> >>> config and it has the unsafe header/library as error. Was your build
> >>> test just applying my patches and doing an explicit 'make llvm'?
> >>
> >>
> >> Indeed,
> >> Some tools use libxml2, so you have to add target libxml2 package as build dependency.
> >>
> >
> > Would you mind sharing your verbose output and equivalent of cmake
> > config guess/status?
> >
>
> I need to rebuild from scratch since I've fixed the build locally.
> I'll restart a new one next week-end.
>
Something like the following added to package/llvm/llvm.mk? (Still
haven't see this on my end. I messed with my host system a bit to get
libxml2 support and headers in place, no luck.)
# LLVM requires libxml2
HOST_LLVM_CONF_OPTS += \
-DLIBXML2_LIBRARIES=$(HOST_DIR)/lib \
-DLIBXML2_INCLUDE_DIR=$(HOST_DIR)/include/libxml2
HOST_LLVM_DEPENDENCIES += host-libxml2
LLVM_CONF_OPTS += \
-DLIBXML2_LIBRARIES=$(STAGING_DIR)/usr/lib \
-DLIBXML2_INCLUDE_DIR=$(STAGING_DIR)/usr/include/libxml2
LLVM_DEPENDENCIES += libxml2
The interesting thing is that it looks like just a windows manifest
manager source file uses the libxml2. Unsure of it's role....
Matt
^ permalink raw reply [flat|nested] 22+ messages in thread* [Buildroot] [PATCH v2 2/6] package/llvm: install target binary/debug tools
2018-11-21 3:12 ` Matthew Weber
@ 2018-11-21 21:18 ` Romain Naour
2018-11-22 3:14 ` Matthew Weber
0 siblings, 1 reply; 22+ messages in thread
From: Romain Naour @ 2018-11-21 21:18 UTC (permalink / raw)
To: buildroot
Hi Matt,
Le 21/11/2018 ? 04:12, Matthew Weber a ?crit?:
> On Tue, Nov 20, 2018 at 4:37 PM Romain Naour <romain.naour@gmail.com> wrote:
>>
>> Hi Matt,
>>
>> Le 19/11/2018 ? 14:04, Matthew Weber a ?crit :
>>> Romain,
>>>
>>> On Mon, Nov 19, 2018 at 2:10 AM Romain Naour <romain.naour@gmail.com> wrote:
>>>>
>>>> Hi Matt,
>>>>
>>>> Le lun. 19 nov. 2018 ? 02:48, Matthew Weber <matthew.weber@rockwellcollins.com> a ?crit :
>>>>>
>>>>> Romain,
>>>>>
>>>>> On Sun, Nov 18, 2018 at 1:14 PM Romain Naour <romain.naour@smile.fr> wrote:
>>>>>>
>>>>>> Hi Matt,
>>>>>>
>>>>>> Le 17/11/2018 ? 04:56, Matt Weber a ?crit :
>>>>>>> The compiler-rt fuzzer and address sanitizer tools require additional
>>>>>>> LLVM binary tools installed to allow stack trace decoding actively during
>>>>>>> executable analysis.
>>>>>>>
>>>>>>> https://github.com/google/sanitizers/wiki/AddressSanitizerCallStack
>>>>>>>
>>>>>>> Signed-off-by: Matthew Weber <matthew.weber@rockwellcollins.com>
>>>>>>> Reviewed-by: Romain Naour <romain.naour@smile.fr>
>>>>>>
>>>>>> For some reason I get an error while building llvm package tools.
>>>>>>
>>>>>> x86_64-linux-g++: WARNING: unsafe header/library path used in cross-compilation:
>>>>>> '-I/usr/include/libxml2'
>>>>>>
>>>>>> Can you have a look?
>>>>>
>>>>> I went back and checked my logs on the clang compiler-rt run-test
>>>>> build for that warning and didn't find it. I then checked and my
>>>>> config and it has the unsafe header/library as error. Was your build
>>>>> test just applying my patches and doing an explicit 'make llvm'?
>>>>
>>>>
>>>> Indeed,
>>>> Some tools use libxml2, so you have to add target libxml2 package as build dependency.
>>>>
>>>
>>> Would you mind sharing your verbose output and equivalent of cmake
>>> config guess/status?
>>>
>>
>> I need to rebuild from scratch since I've fixed the build locally.
>> I'll restart a new one next week-end.
>>
>
> Something like the following added to package/llvm/llvm.mk? (Still
> haven't see this on my end. I messed with my host system a bit to get
> libxml2 support and headers in place, no luck.)
>
> # LLVM requires libxml2
> HOST_LLVM_CONF_OPTS += \
> -DLIBXML2_LIBRARIES=$(HOST_DIR)/lib \
> -DLIBXML2_INCLUDE_DIR=$(HOST_DIR)/include/libxml2
> HOST_LLVM_DEPENDENCIES += host-libxml2
> LLVM_CONF_OPTS += \
> -DLIBXML2_LIBRARIES=$(STAGING_DIR)/usr/lib \
> -DLIBXML2_INCLUDE_DIR=$(STAGING_DIR)/usr/include/libxml2
> LLVM_DEPENDENCIES += libxml2
I reproduced the issue: https://pastebin.com/s8PZvuGt
The CMakeCache.txt content seems interesting:
//Path to a file.
LIBXML2_INCLUDE_DIR:PATH=LIBXML2_INCLUDE_DIR-NOTFOUND
//Path to a library.
LIBXML2_LIBRARY:FILEPATH=LIBXML2_LIBRARY-NOTFOUND
//Path to a program.
LIBXML2_XMLLINT_EXECUTABLE:FILEPATH=/usr/bin/xmllint
//Use libxml2 if available. Can be ON, OFF, or FORCE_ON
LLVM_ENABLE_LIBXML2:STRING=ON
LLVM_ENABLE_LIBXML2 allow to define LLVM_LIBXML2_ENABLED
$ grep -r LLVM_LIBXML2_ENABLED in the build directory:
cmake/modules/CMakeFiles/LLVMConfig.cmake:set(LLVM_LIBXML2_ENABLED 0)
NATIVE/cmake/modules/CMakeFiles/LLVMConfig.cmake:set(LLVM_LIBXML2_ENABLED 1)
NATIVE/lib/cmake/llvm/LLVMConfig.cmake:set(LLVM_LIBXML2_ENABLED 1)
NATIVE/include/llvm/Config/config.h:#define LLVM_LIBXML2_ENABLED 1
lib/cmake/llvm/LLVMConfig.cmake:set(LLVM_LIBXML2_ENABLED 0)
include/llvm/Config/config.h:/* #undef LLVM_LIBXML2_ENABLED */
It seems that llvm try to build something for the host (NATIVE) while building
target llvm package. Sadly it use the cross-compiler to build them and that is
why we hit the paranoid wrapper.
>
> The interesting thing is that it looks like just a windows manifest
> manager source file uses the libxml2. Unsure of it's role....
Then we can just add -DLLVM_ENABLE_LIBXML2=OFF for host-llvm and llvm.
Best regards,
Romain
>
>
> Matt
>
^ permalink raw reply [flat|nested] 22+ messages in thread
* [Buildroot] [PATCH v2 2/6] package/llvm: install target binary/debug tools
2018-11-21 21:18 ` Romain Naour
@ 2018-11-22 3:14 ` Matthew Weber
2018-11-22 4:05 ` Matthew Weber
0 siblings, 1 reply; 22+ messages in thread
From: Matthew Weber @ 2018-11-22 3:14 UTC (permalink / raw)
To: buildroot
Romain,
On Wed, Nov 21, 2018 at 3:19 PM Romain Naour <romain.naour@gmail.com> wrote:
>
> Hi Matt,
>
> Le 21/11/2018 ? 04:12, Matthew Weber a ?crit :
> > On Tue, Nov 20, 2018 at 4:37 PM Romain Naour <romain.naour@gmail.com> wrote:
> >>
> >> Hi Matt,
> >>
> >> Le 19/11/2018 ? 14:04, Matthew Weber a ?crit :
> >>> Romain,
> >>>
> >>> On Mon, Nov 19, 2018 at 2:10 AM Romain Naour <romain.naour@gmail.com> wrote:
> >>>>
> >>>> Hi Matt,
> >>>>
> >>>> Le lun. 19 nov. 2018 ? 02:48, Matthew Weber <matthew.weber@rockwellcollins.com> a ?crit :
> >>>>>
> >>>>> Romain,
> >>>>>
> >>>>> On Sun, Nov 18, 2018 at 1:14 PM Romain Naour <romain.naour@smile.fr> wrote:
> >>>>>>
> >>>>>> Hi Matt,
> >>>>>>
> >>>>>> Le 17/11/2018 ? 04:56, Matt Weber a ?crit :
> >>>>>>> The compiler-rt fuzzer and address sanitizer tools require additional
> >>>>>>> LLVM binary tools installed to allow stack trace decoding actively during
> >>>>>>> executable analysis.
> >>>>>>>
> >>>>>>> https://github.com/google/sanitizers/wiki/AddressSanitizerCallStack
> >>>>>>>
> >>>>>>> Signed-off-by: Matthew Weber <matthew.weber@rockwellcollins.com>
> >>>>>>> Reviewed-by: Romain Naour <romain.naour@smile.fr>
> >>>>>>
> >>>>>> For some reason I get an error while building llvm package tools.
> >>>>>>
> >>>>>> x86_64-linux-g++: WARNING: unsafe header/library path used in cross-compilation:
> >>>>>> '-I/usr/include/libxml2'
> >>>>>>
> >>>>>> Can you have a look?
> >>>>>
> >>>>> I went back and checked my logs on the clang compiler-rt run-test
> >>>>> build for that warning and didn't find it. I then checked and my
> >>>>> config and it has the unsafe header/library as error. Was your build
> >>>>> test just applying my patches and doing an explicit 'make llvm'?
> >>>>
> >>>>
> >>>> Indeed,
> >>>> Some tools use libxml2, so you have to add target libxml2 package as build dependency.
> >>>>
> >>>
> >>> Would you mind sharing your verbose output and equivalent of cmake
> >>> config guess/status?
> >>>
> >>
> >> I need to rebuild from scratch since I've fixed the build locally.
> >> I'll restart a new one next week-end.
> >>
> >
> > Something like the following added to package/llvm/llvm.mk? (Still
> > haven't see this on my end. I messed with my host system a bit to get
> > libxml2 support and headers in place, no luck.)
> >
> > # LLVM requires libxml2
> > HOST_LLVM_CONF_OPTS += \
> > -DLIBXML2_LIBRARIES=$(HOST_DIR)/lib \
> > -DLIBXML2_INCLUDE_DIR=$(HOST_DIR)/include/libxml2
> > HOST_LLVM_DEPENDENCIES += host-libxml2
> > LLVM_CONF_OPTS += \
> > -DLIBXML2_LIBRARIES=$(STAGING_DIR)/usr/lib \
> > -DLIBXML2_INCLUDE_DIR=$(STAGING_DIR)/usr/include/libxml2
> > LLVM_DEPENDENCIES += libxml2
>
> I reproduced the issue: https://pastebin.com/s8PZvuGt
>
Ah ok, I see you have the AMD GPU stuff enabled. I'll enable that and
see if I can reproduce the path poisoning. Then do some builds
with/without explicitly disabling via -DLLVM_ENABLE_LIBXML2.
Matt
^ permalink raw reply [flat|nested] 22+ messages in thread
* [Buildroot] [PATCH v2 2/6] package/llvm: install target binary/debug tools
2018-11-22 3:14 ` Matthew Weber
@ 2018-11-22 4:05 ` Matthew Weber
0 siblings, 0 replies; 22+ messages in thread
From: Matthew Weber @ 2018-11-22 4:05 UTC (permalink / raw)
To: buildroot
Romain,
On Wed, Nov 21, 2018 at 9:14 PM Matthew Weber
<matthew.weber@rockwellcollins.com> wrote:
>
> Romain,
>
> On Wed, Nov 21, 2018 at 3:19 PM Romain Naour <romain.naour@gmail.com> wrote:
> >
> > Hi Matt,
> >
> > Le 21/11/2018 ? 04:12, Matthew Weber a ?crit :
> > > On Tue, Nov 20, 2018 at 4:37 PM Romain Naour <romain.naour@gmail.com> wrote:
> > >>
> > >> Hi Matt,
> > >>
> > >> Le 19/11/2018 ? 14:04, Matthew Weber a ?crit :
> > >>> Romain,
> > >>>
> > >>> On Mon, Nov 19, 2018 at 2:10 AM Romain Naour <romain.naour@gmail.com> wrote:
> > >>>>
> > >>>> Hi Matt,
> > >>>>
> > >>>> Le lun. 19 nov. 2018 ? 02:48, Matthew Weber <matthew.weber@rockwellcollins.com> a ?crit :
> > >>>>>
> > >>>>> Romain,
> > >>>>>
> > >>>>> On Sun, Nov 18, 2018 at 1:14 PM Romain Naour <romain.naour@smile.fr> wrote:
> > >>>>>>
> > >>>>>> Hi Matt,
> > >>>>>>
> > >>>>>> Le 17/11/2018 ? 04:56, Matt Weber a ?crit :
> > >>>>>>> The compiler-rt fuzzer and address sanitizer tools require additional
> > >>>>>>> LLVM binary tools installed to allow stack trace decoding actively during
> > >>>>>>> executable analysis.
> > >>>>>>>
> > >>>>>>> https://github.com/google/sanitizers/wiki/AddressSanitizerCallStack
> > >>>>>>>
> > >>>>>>> Signed-off-by: Matthew Weber <matthew.weber@rockwellcollins.com>
> > >>>>>>> Reviewed-by: Romain Naour <romain.naour@smile.fr>
> > >>>>>>
> > >>>>>> For some reason I get an error while building llvm package tools.
> > >>>>>>
> > >>>>>> x86_64-linux-g++: WARNING: unsafe header/library path used in cross-compilation:
> > >>>>>> '-I/usr/include/libxml2'
> > >>>>>>
> > >>>>>> Can you have a look?
> > >>>>>
> > >>>>> I went back and checked my logs on the clang compiler-rt run-test
> > >>>>> build for that warning and didn't find it. I then checked and my
> > >>>>> config and it has the unsafe header/library as error. Was your build
> > >>>>> test just applying my patches and doing an explicit 'make llvm'?
> > >>>>
> > >>>>
> > >>>> Indeed,
> > >>>> Some tools use libxml2, so you have to add target libxml2 package as build dependency.
> > >>>>
> > >>>
> > >>> Would you mind sharing your verbose output and equivalent of cmake
> > >>> config guess/status?
> > >>>
> > >>
> > >> I need to rebuild from scratch since I've fixed the build locally.
> > >> I'll restart a new one next week-end.
> > >>
> > >
> > > Something like the following added to package/llvm/llvm.mk? (Still
> > > haven't see this on my end. I messed with my host system a bit to get
> > > libxml2 support and headers in place, no luck.)
> > >
> > > # LLVM requires libxml2
> > > HOST_LLVM_CONF_OPTS += \
> > > -DLIBXML2_LIBRARIES=$(HOST_DIR)/lib \
> > > -DLIBXML2_INCLUDE_DIR=$(HOST_DIR)/include/libxml2
> > > HOST_LLVM_DEPENDENCIES += host-libxml2
> > > LLVM_CONF_OPTS += \
> > > -DLIBXML2_LIBRARIES=$(STAGING_DIR)/usr/lib \
> > > -DLIBXML2_INCLUDE_DIR=$(STAGING_DIR)/usr/include/libxml2
> > > LLVM_DEPENDENCIES += libxml2
> >
> > I reproduced the issue: https://pastebin.com/s8PZvuGt
> >
>
> Ah ok, I see you have the AMD GPU stuff enabled. I'll enable that and
> see if I can reproduce the path poisoning. Then do some builds
> with/without explicitly disabling via -DLLVM_ENABLE_LIBXML2.
>
I still can't reproduce it. Would you mind sharing your defconfig?
I'll send a v4 with the -DLLVM_ENABLE_LIBXML2=off. See where you get
with that.
Matt
^ permalink raw reply [flat|nested] 22+ messages in thread
* [Buildroot] [PATCH v2 3/6] testing/infra/builder: build with target and environment
2018-11-17 3:56 [Buildroot] [PATCH v2 1/6] package/compiler-rt: new package Matt Weber
2018-11-17 3:56 ` [Buildroot] [PATCH v2 2/6] package/llvm: install target binary/debug tools Matt Weber
@ 2018-11-17 3:56 ` Matt Weber
2018-11-18 21:35 ` Ricardo Martincoski
2018-11-17 3:56 ` [Buildroot] [PATCH v2 4/6] testing/infra/basetest: support br2-external Matt Weber
` (4 subsequent siblings)
6 siblings, 1 reply; 22+ messages in thread
From: Matt Weber @ 2018-11-17 3:56 UTC (permalink / raw)
To: buildroot
From: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
Make the builder able to call 'VAR1=1 make VAR2=2 target'.
Allow to send extra parameters to be added to the end of make command
line. It can be used for these purposes:
- to configure a br2-external, by passing 'BR2_EXTERNAL="dir"';
- to specify a make target, such as 'foo-source'.
Allow to add variables to the environment in which make runs. It can be
used to override values from environment, such as 'BR2_DL_DIR="dl"'.
This change will be needed when adding a common class to test the git
download infra.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Matt Weber <matthew.weber@rockwellcollins.com>
---
Changes
v2
- New from: http://patchwork.ozlabs.org/patch/912351/
---
support/testing/infra/builder.py | 38 ++++++++++++++++++++++++++++++++++----
1 file changed, 34 insertions(+), 4 deletions(-)
diff --git a/support/testing/infra/builder.py b/support/testing/infra/builder.py
index fc318fe..4d9c512 100644
--- a/support/testing/infra/builder.py
+++ b/support/testing/infra/builder.py
@@ -12,7 +12,18 @@ class Builder(object):
self.builddir = builddir
self.logfile = infra.open_log_file(builddir, "build", logtofile)
- def configure(self):
+ def configure(self, make_extra_opts=[], make_extra_env={}):
+ """
+ Configure the build.
+
+ make_extra_opts: a list of arguments to be passed to the make
+ command.
+ e.g. make_extra_opts=["BR2_EXTERNAL=/path"]
+
+ make_extra_env: a dict of variables to be appended (or replaced)
+ in the environment that calls make.
+ e.g. make_extra_env={"BR2_DL_DIR": "/path"}
+ """
if not os.path.isdir(self.builddir):
os.makedirs(self.builddir)
@@ -25,22 +36,41 @@ class Builder(object):
self.logfile.flush()
env = {"PATH": os.environ["PATH"]}
+ env.update(make_extra_env)
+
cmd = ["make",
- "O={}".format(self.builddir),
- "olddefconfig"]
+ "O={}".format(self.builddir)]
+ cmd += make_extra_opts
+ cmd += ["olddefconfig"]
+
ret = subprocess.call(cmd, stdout=self.logfile, stderr=self.logfile,
env=env)
if ret != 0:
raise SystemError("Cannot olddefconfig")
- def build(self):
+ def build(self, make_extra_opts=[], make_extra_env={}):
+ """
+ Perform the build.
+
+ make_extra_opts: a list of arguments to be passed to the make
+ command. It can include a make target.
+ e.g. make_extra_opts=["foo-source"]
+
+ make_extra_env: a dict of variables to be appended (or replaced)
+ in the environment that calls make.
+ e.g. make_extra_env={"BR2_DL_DIR": "/path"}
+ """
env = {"PATH": os.environ["PATH"]}
if "http_proxy" in os.environ:
self.logfile.write("Using system proxy: " +
os.environ["http_proxy"] + "\n")
env['http_proxy'] = os.environ["http_proxy"]
env['https_proxy'] = os.environ["http_proxy"]
+ env.update(make_extra_env)
+
cmd = ["make", "-C", self.builddir]
+ cmd += make_extra_opts
+
ret = subprocess.call(cmd, stdout=self.logfile, stderr=self.logfile,
env=env)
if ret != 0:
--
1.9.1
^ permalink raw reply related [flat|nested] 22+ messages in thread* [Buildroot] [PATCH v2 3/6] testing/infra/builder: build with target and environment
2018-11-17 3:56 ` [Buildroot] [PATCH v2 3/6] testing/infra/builder: build with target and environment Matt Weber
@ 2018-11-18 21:35 ` Ricardo Martincoski
0 siblings, 0 replies; 22+ messages in thread
From: Ricardo Martincoski @ 2018-11-18 21:35 UTC (permalink / raw)
To: buildroot
Hello,
Another nit that can be fixed while applying.
On Sat, Nov 17, 2018 at 01:56 AM, Matt Weber wrote:
> From: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
>
> Make the builder able to call 'VAR1=1 make VAR2=2 target'.
>
> Allow to send extra parameters to be added to the end of make command
> line. It can be used for these purposes:
> - to configure a br2-external, by passing 'BR2_EXTERNAL="dir"';
> - to specify a make target, such as 'foo-source'.
>
> Allow to add variables to the environment in which make runs. It can be
> used to override values from environment, such as 'BR2_DL_DIR="dl"'.
>
> This change will be needed when adding a common class to test the git
> download infra.
This is not untrue.
But in the context of your series, perhaps it could be replaced by:
"
This change will be needed when adding support to br2-externals to the
test infra.
"
And who applies could even tweak this to a better phrase.
>
> Signed-off-by: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
> Cc: Arnout Vandecappelle <arnout@mind.be>
> Signed-off-by: Matt Weber <matthew.weber@rockwellcollins.com>
It feels weird to add a tag to my own patch :-)
But I reviewed it and the contents are OK.
Regards,
Ricardo
^ permalink raw reply [flat|nested] 22+ messages in thread
* [Buildroot] [PATCH v2 4/6] testing/infra/basetest: support br2-external
2018-11-17 3:56 [Buildroot] [PATCH v2 1/6] package/compiler-rt: new package Matt Weber
2018-11-17 3:56 ` [Buildroot] [PATCH v2 2/6] package/llvm: install target binary/debug tools Matt Weber
2018-11-17 3:56 ` [Buildroot] [PATCH v2 3/6] testing/infra/builder: build with target and environment Matt Weber
@ 2018-11-17 3:56 ` Matt Weber
2018-11-18 21:41 ` Ricardo Martincoski
2018-11-17 3:56 ` [Buildroot] [PATCH v2 5/6] testing/tests: CLANG compiler-rt runtime test Matt Weber
` (3 subsequent siblings)
6 siblings, 1 reply; 22+ messages in thread
From: Matt Weber @ 2018-11-17 3:56 UTC (permalink / raw)
To: buildroot
From: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
This change will be needed when adding a common class to test the git
download infra.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Matt Weber <matthew.weber@rockwellcollins.com>
---
Changes
v2
- New from: http://patchwork.ozlabs.org/patch/912354/
---
support/testing/infra/basetest.py | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/support/testing/infra/basetest.py b/support/testing/infra/basetest.py
index 5014fef..8413943 100644
--- a/support/testing/infra/basetest.py
+++ b/support/testing/infra/basetest.py
@@ -30,6 +30,7 @@ MINIMAL_CONFIG = \
class BRTest(unittest.TestCase):
config = None
+ br2_external = list()
downloaddir = None
outputdir = None
logtofile = True
@@ -58,7 +59,7 @@ class BRTest(unittest.TestCase):
if not self.b.is_finished():
self.show_msg("Building")
- self.b.configure()
+ self.b.configure(["BR2_EXTERNAL={}".format(":".join(self.br2_external))])
self.b.build()
self.show_msg("Building done")
--
1.9.1
^ permalink raw reply related [flat|nested] 22+ messages in thread* [Buildroot] [PATCH v2 4/6] testing/infra/basetest: support br2-external
2018-11-17 3:56 ` [Buildroot] [PATCH v2 4/6] testing/infra/basetest: support br2-external Matt Weber
@ 2018-11-18 21:41 ` Ricardo Martincoski
0 siblings, 0 replies; 22+ messages in thread
From: Ricardo Martincoski @ 2018-11-18 21:41 UTC (permalink / raw)
To: buildroot
Hello,
Another nit that can be fixed while applying.
On Sat, Nov 17, 2018 at 01:56 AM, Matt Weber wrote:
> From: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
>
> This change will be needed when adding a common class to test the git
> download infra.
Just like in patch 3, this is not untrue.
But in the context of your series, perhaps it could be replaced by:
"
Some upcoming test cases can use one or more br2-external trees as
fixtures that provide packages used only in runtime tests.
Add support for br2-external into the BRTest class. Any test case can
then provide a list of paths for be used as br2-external trees during
the build of the image to be tested.
"
And who applies it could even tweak this to a better phrase.
>
> Signed-off-by: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
> Cc: Arnout Vandecappelle <arnout@mind.be>
> Signed-off-by: Matt Weber <matthew.weber@rockwellcollins.com>
> ---
I reviewed it and the contents are OK.
Regards,
Ricardo
^ permalink raw reply [flat|nested] 22+ messages in thread
* [Buildroot] [PATCH v2 5/6] testing/tests: CLANG compiler-rt runtime test
2018-11-17 3:56 [Buildroot] [PATCH v2 1/6] package/compiler-rt: new package Matt Weber
` (2 preceding siblings ...)
2018-11-17 3:56 ` [Buildroot] [PATCH v2 4/6] testing/infra/basetest: support br2-external Matt Weber
@ 2018-11-17 3:56 ` Matt Weber
2018-11-18 21:50 ` Ricardo Martincoski
2018-11-17 3:56 ` [Buildroot] [PATCH v2 6/6] llvm/clang: add note about version bumping together Matt Weber
` (2 subsequent siblings)
6 siblings, 1 reply; 22+ messages in thread
From: Matt Weber @ 2018-11-17 3:56 UTC (permalink / raw)
To: buildroot
This patch adds a test case that
1) Builds the complete LLVM and CLANG set of host tools
2) Cross-compiles the compiler-rt runtime using CLANG
3) Builds a cross-compiled application using CLANG and the libfuzzer
compiler-rt library.
4) Executes the fuzz application (part of the libfuzzer package) on
target and checks expected output
Signed-off-by: Matthew Weber <matthew.weber@rockwellcollins.com>
Cc: Ricardo Martincoski <ricardo.martincoski@gmail.com>
---
Changes
v1 -> v2
[Ricardo
- updated yml with test case
- moved emulator launch cmd to test case from infra
- defconfig in the order provided by savedefconfig
- indent defconfig
- add full package for test code to the test br2-external
- consolidated to one class only
---
.gitlab-ci.yml | 1 +
.../br2-external/clang-compiler-rt/Config.in | 1 +
.../br2-external/clang-compiler-rt/external.desc | 1 +
.../br2-external/clang-compiler-rt/external.mk | 1 +
.../clang-compiler-rt/package/libfuzzer/Config.in | 7 ++++
.../package/libfuzzer/libfuzzer.hash | 2 +
.../package/libfuzzer/libfuzzer.mk | 24 +++++++++++
support/testing/tests/package/test_clang.py | 46 ++++++++++++++++++++++
8 files changed, 83 insertions(+)
create mode 100644 support/testing/tests/package/br2-external/clang-compiler-rt/Config.in
create mode 100644 support/testing/tests/package/br2-external/clang-compiler-rt/external.desc
create mode 100644 support/testing/tests/package/br2-external/clang-compiler-rt/external.mk
create mode 100644 support/testing/tests/package/br2-external/clang-compiler-rt/package/libfuzzer/Config.in
create mode 100644 support/testing/tests/package/br2-external/clang-compiler-rt/package/libfuzzer/libfuzzer.hash
create mode 100644 support/testing/tests/package/br2-external/clang-compiler-rt/package/libfuzzer/libfuzzer.mk
create mode 100644 support/testing/tests/package/test_clang.py
diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index c78f2ea..2f2125f 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -309,6 +309,7 @@ tests.init.test_systemd.TestInitSystemSystemdRoNetworkd: *runtime_test
tests.init.test_systemd.TestInitSystemSystemdRwFull: *runtime_test
tests.init.test_systemd.TestInitSystemSystemdRwIfupdown: *runtime_test
tests.init.test_systemd.TestInitSystemSystemdRwNetworkd: *runtime_test
+tests.package.test_clang.TestClangCompilerRT: *runtime_test
tests.package.test_dropbear.TestDropbear: *runtime_test
tests.package.test_ipython.TestIPythonPy2: *runtime_test
tests.package.test_ipython.TestIPythonPy3: *runtime_test
diff --git a/support/testing/tests/package/br2-external/clang-compiler-rt/Config.in b/support/testing/tests/package/br2-external/clang-compiler-rt/Config.in
new file mode 100644
index 0000000..e1f9f8c
--- /dev/null
+++ b/support/testing/tests/package/br2-external/clang-compiler-rt/Config.in
@@ -0,0 +1 @@
+source "$BR2_EXTERNAL_CLANG_COMPILER_RT_PATH/package/libfuzzer/Config.in"
diff --git a/support/testing/tests/package/br2-external/clang-compiler-rt/external.desc b/support/testing/tests/package/br2-external/clang-compiler-rt/external.desc
new file mode 100644
index 0000000..92df859
--- /dev/null
+++ b/support/testing/tests/package/br2-external/clang-compiler-rt/external.desc
@@ -0,0 +1 @@
+name: CLANG_COMPILER_RT
diff --git a/support/testing/tests/package/br2-external/clang-compiler-rt/external.mk b/support/testing/tests/package/br2-external/clang-compiler-rt/external.mk
new file mode 100644
index 0000000..6fa55c1
--- /dev/null
+++ b/support/testing/tests/package/br2-external/clang-compiler-rt/external.mk
@@ -0,0 +1 @@
+include $(sort $(wildcard $(BR2_EXTERNAL_CLANG_COMPILER_RT_PATH)/package/*/*.mk))
diff --git a/support/testing/tests/package/br2-external/clang-compiler-rt/package/libfuzzer/Config.in b/support/testing/tests/package/br2-external/clang-compiler-rt/package/libfuzzer/Config.in
new file mode 100644
index 0000000..5af5d1b
--- /dev/null
+++ b/support/testing/tests/package/br2-external/clang-compiler-rt/package/libfuzzer/Config.in
@@ -0,0 +1,7 @@
+config BR2_PACKAGE_LIBFUZZER
+ bool "libfuzzer"
+ help
+ This is a set of tests (benchmarks) for fuzzing
+ engines (fuzzers).
+
+ https://github.com/google/fuzzer-test-suite
diff --git a/support/testing/tests/package/br2-external/clang-compiler-rt/package/libfuzzer/libfuzzer.hash b/support/testing/tests/package/br2-external/clang-compiler-rt/package/libfuzzer/libfuzzer.hash
new file mode 100644
index 0000000..6baf576
--- /dev/null
+++ b/support/testing/tests/package/br2-external/clang-compiler-rt/package/libfuzzer/libfuzzer.hash
@@ -0,0 +1,2 @@
+sha256 c0addb4d7f0447fc9fd7c80e5721fafe4c137f29a8ebd94c5fef7e1d6a2c944c libfuzzer-64e4d9aa19a8d33b61882154addbf8419d7416e1.tar.gz
+sha256 cfc7749b96f63bd31c3c42b5c471bf756814053e847c10f3eb003417bc523d30 LICENSE
diff --git a/support/testing/tests/package/br2-external/clang-compiler-rt/package/libfuzzer/libfuzzer.mk b/support/testing/tests/package/br2-external/clang-compiler-rt/package/libfuzzer/libfuzzer.mk
new file mode 100644
index 0000000..7d6f83b
--- /dev/null
+++ b/support/testing/tests/package/br2-external/clang-compiler-rt/package/libfuzzer/libfuzzer.mk
@@ -0,0 +1,24 @@
+################################################################################
+#
+# libfuzzer
+#
+################################################################################
+
+LIBFUZZER_VERSION = 64e4d9aa19a8d33b61882154addbf8419d7416e1
+LIBFUZZER_SITE = $(call github,google,fuzzer-test-suite,$(LIBFUZZER_VERSION))
+LIBFUZZER_LICENSE = Apache-2.0
+LIBFUZZER_LICENSE_FILES = LICENSE
+LIBFUZZER_DEPENDENCIES = compiler-rt
+
+define LIBFUZZER_BUILD_CMDS
+ $(HOST_DIR)/bin/clang++ -mcpu=cortex-a53 --sysroot=$(STAGING_DIR) \
+ -B $(HOST_DIR)/opt/ext-toolchain -fsanitize=address,fuzzer \
+ $(@D)/tutorial/fuzz_me.cc \
+ -o $(@D)/fuzz_me
+endef
+
+define LIBFUZZER_INSTALL_TARGET_CMDS
+ $(INSTALL) -D -m 755 $(@D)/fuzz_me $(TARGET_DIR)/usr/bin/fuzz_me
+endef
+
+$(eval $(generic-package))
diff --git a/support/testing/tests/package/test_clang.py b/support/testing/tests/package/test_clang.py
new file mode 100644
index 0000000..84a2528
--- /dev/null
+++ b/support/testing/tests/package/test_clang.py
@@ -0,0 +1,46 @@
+import os
+
+import infra.basetest
+
+FUZZ_TIMEOUT = 120
+
+
+class TestClangCompilerRT(infra.basetest.BRTest):
+ br2_external = [infra.filepath("tests/package/br2-external/clang-compiler-rt")]
+ config = \
+ """
+ BR2_aarch64=y
+ BR2_TOOLCHAIN_EXTERNAL=y
+ BR2_TARGET_GENERIC_GETTY_PORT="ttyAMA0"
+ BR2_LINUX_KERNEL=y
+ BR2_LINUX_KERNEL_CUSTOM_VERSION=y
+ BR2_LINUX_KERNEL_CUSTOM_VERSION_VALUE="4.16.7"
+ BR2_LINUX_KERNEL_USE_CUSTOM_CONFIG=y
+ BR2_LINUX_KERNEL_CUSTOM_CONFIG_FILE="board/qemu/aarch64-virt/linux.config"
+ BR2_LINUX_KERNEL_NEEDS_HOST_OPENSSL=y
+ BR2_PACKAGE_COMPILER_RT=y
+ BR2_PACKAGE_LLVM=y
+ BR2_TARGET_ROOTFS_CPIO=y
+ BR2_TARGET_ROOTFS_CPIO_GZIP=y
+ # BR2_TARGET_ROOTFS_TAR is not set
+ BR2_PACKAGE_LIBFUZZER=y
+ """
+
+ def login(self):
+ img = os.path.join(self.builddir, "images", "rootfs.cpio.gz")
+ kern = os.path.join(self.builddir, "images", "Image")
+ # Sanitizers overallocate memory and the minimum that seemed to work was 512MB
+ self.emulator.boot(arch="aarch64",
+ kernel=kern,
+ kernel_cmdline=["console=ttyAMA0"],
+ options=["-M", "virt", "-cpu", "cortex-a53", "-m", "512", "-initrd", img])
+ self.emulator.login()
+
+ def test_run(self):
+ self.login()
+
+ # The test case verifies both that the application executes and that
+ # the symbolizer is working to decode the stack trace
+ cmd = "fuzz_me 2>&1 | grep _M_replace"
+ _, exit_code = self.emulator.run(cmd, FUZZ_TIMEOUT)
+ self.assertEqual(exit_code, 0)
--
1.9.1
^ permalink raw reply related [flat|nested] 22+ messages in thread* [Buildroot] [PATCH v2 5/6] testing/tests: CLANG compiler-rt runtime test
2018-11-17 3:56 ` [Buildroot] [PATCH v2 5/6] testing/tests: CLANG compiler-rt runtime test Matt Weber
@ 2018-11-18 21:50 ` Ricardo Martincoski
0 siblings, 0 replies; 22+ messages in thread
From: Ricardo Martincoski @ 2018-11-18 21:50 UTC (permalink / raw)
To: buildroot
Hello,
IMO, much nicer the way you did, using the project in the package instead of
adding a copy of the .cc file!
On Sat, Nov 17, 2018 at 01:56 AM, Matt Weber wrote:
> This patch adds a test case that
> 1) Builds the complete LLVM and CLANG set of host tools
> 2) Cross-compiles the compiler-rt runtime using CLANG
> 3) Builds a cross-compiled application using CLANG and the libfuzzer
> compiler-rt library.
> 4) Executes the fuzz application (part of the libfuzzer package) on
> target and checks expected output
>
> Signed-off-by: Matthew Weber <matthew.weber@rockwellcollins.com>
> Cc: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Reviewed-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
[with the entire series applied on next branch:
https://gitlab.com/RicardoMartincoski/buildroot/-/jobs/121908178
NOTE: this test case takes longer than test_rust, ~2 hours in my host
machine with all tarballs previously downloaded]
Tested-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Regards,
Ricardo
^ permalink raw reply [flat|nested] 22+ messages in thread
* [Buildroot] [PATCH v2 6/6] llvm/clang: add note about version bumping together
2018-11-17 3:56 [Buildroot] [PATCH v2 1/6] package/compiler-rt: new package Matt Weber
` (3 preceding siblings ...)
2018-11-17 3:56 ` [Buildroot] [PATCH v2 5/6] testing/tests: CLANG compiler-rt runtime test Matt Weber
@ 2018-11-17 3:56 ` Matt Weber
2018-11-18 21:53 ` Ricardo Martincoski
2018-11-18 21:32 ` [Buildroot] [PATCH v2 1/6] package/compiler-rt: new package Ricardo Martincoski
2018-11-18 22:28 ` Romain Naour
6 siblings, 1 reply; 22+ messages in thread
From: Matt Weber @ 2018-11-17 3:56 UTC (permalink / raw)
To: buildroot
Signed-off-by: Matthew Weber <matthew.weber@rockwellcollins.com>
Acked-by: Romain Naour <romain.naour@smile.fr>
---
Changes
v1 -> v2
- None
---
package/clang/clang.mk | 1 +
package/llvm/llvm.mk | 1 +
2 files changed, 2 insertions(+)
diff --git a/package/clang/clang.mk b/package/clang/clang.mk
index f9d4ae3..16f93ee 100644
--- a/package/clang/clang.mk
+++ b/package/clang/clang.mk
@@ -4,6 +4,7 @@
#
################################################################################
+# LLVM and Clang should be version bumped together
CLANG_VERSION = 7.0.0
CLANG_SITE = http://llvm.org/releases/$(CLANG_VERSION)
CLANG_SOURCE = cfe-$(CLANG_VERSION).src.tar.xz
diff --git a/package/llvm/llvm.mk b/package/llvm/llvm.mk
index 79477f0..56f6ed9 100644
--- a/package/llvm/llvm.mk
+++ b/package/llvm/llvm.mk
@@ -4,6 +4,7 @@
#
################################################################################
+# LLVM and Clang should be version bumped together
LLVM_VERSION = 7.0.0
LLVM_SITE = http://llvm.org/releases/$(LLVM_VERSION)
LLVM_SOURCE = llvm-$(LLVM_VERSION).src.tar.xz
--
1.9.1
^ permalink raw reply related [flat|nested] 22+ messages in thread* [Buildroot] [PATCH v2 6/6] llvm/clang: add note about version bumping together
2018-11-17 3:56 ` [Buildroot] [PATCH v2 6/6] llvm/clang: add note about version bumping together Matt Weber
@ 2018-11-18 21:53 ` Ricardo Martincoski
0 siblings, 0 replies; 22+ messages in thread
From: Ricardo Martincoski @ 2018-11-18 21:53 UTC (permalink / raw)
To: buildroot
Hello,
Here again, the changes could be made while applying.
But please double-check, maybe I missed something.
On Sat, Nov 17, 2018 at 01:56 AM, Matt Weber wrote:
[snip]
> +++ b/package/clang/clang.mk
> @@ -4,6 +4,7 @@
> #
> ################################################################################
>
> +# LLVM and Clang should be version bumped together
As I replied to patch 1, shouldn't compiler-rt be added to this list?
> CLANG_VERSION = 7.0.0
> CLANG_SITE = http://llvm.org/releases/$(CLANG_VERSION)
> CLANG_SOURCE = cfe-$(CLANG_VERSION).src.tar.xz
> diff --git a/package/llvm/llvm.mk b/package/llvm/llvm.mk
> index 79477f0..56f6ed9 100644
> --- a/package/llvm/llvm.mk
> +++ b/package/llvm/llvm.mk
> @@ -4,6 +4,7 @@
> #
> ################################################################################
>
> +# LLVM and Clang should be version bumped together
Same here.
Regards,
Ricardo
^ permalink raw reply [flat|nested] 22+ messages in thread
* [Buildroot] [PATCH v2 1/6] package/compiler-rt: new package
2018-11-17 3:56 [Buildroot] [PATCH v2 1/6] package/compiler-rt: new package Matt Weber
` (4 preceding siblings ...)
2018-11-17 3:56 ` [Buildroot] [PATCH v2 6/6] llvm/clang: add note about version bumping together Matt Weber
@ 2018-11-18 21:32 ` Ricardo Martincoski
2018-11-18 22:28 ` Romain Naour
6 siblings, 0 replies; 22+ messages in thread
From: Ricardo Martincoski @ 2018-11-18 21:32 UTC (permalink / raw)
To: buildroot
Hello,
This is not a full review. I just spotted some nits while reviewing/testing the
runtime test in the series.
I guess/hope all of them can be fixed while applying. No need to resend just for
these.
On Sat, Nov 17, 2018 at 01:56 AM, Matt Weber wrote:
> This patch adds support for the compiler-rt (CLANG runtime) libary.
s/libary/library
> It builds a set of static libraries and installs them into the
> CLANG/LLVM toolchain resource folder.
>
> Signed-off-by: Matthew Weber <matthew.weber@rockwellcollins.com>
> Cc: Romain Naour <romain.naour@smile.fr>
> ---
[snip]
> +++ b/package/compiler-rt/compiler-rt.mk
> @@ -0,0 +1,35 @@
> +################################################################################
> +#
> +# compiler-rt
> +#
> +################################################################################
> +
> +# Compiler-RT should be bumped together with LLVM and Clang as the run-time is
> +# tied to the version of those tools
OK, but then shouldn't be better to also mention Compiler-RT in the comments
added to LLVM and Clang in patch 6?
> +COMPILER_RT_VERSION = 7.0.0
> +COMPILER_RT_SOURCE = compiler-rt-$(COMPILER_RT_VERSION).src.tar.xz
> +COMPILER_RT_SITE = http://llvm.org/releases/$(COMPILER_RT_VERSION)
> +COMPILER_RT_LICENSE = NCSA MIT
> +COMPILER_RT_LICENSE_FILES = LICENSE.TXT
> +COMPILER_RT_DEPENDENCIES = host-clang llvm
> +
> +COMPILER_RT_INSTALL_STAGING = YES
> +COMPILER_RT_INSTALL_TARGET = NO
> +
> +
I was testing your series and I saw the job check-package in GitLab CI failed:
https://gitlab.com/RicardoMartincoski/buildroot/-/jobs/121907593
package/compiler-rt/compiler-rt.mk:19: consecutive empty lines
> +# -DCMAKE_INSTALL_PREFIX="" allows the COMPILER_RT_INSTALL_STAGING_CMDS to
> +# provide the complete path to compilier-rt for installation of the runtime
s/compilier/compiler
Regards,
Ricardo
^ permalink raw reply [flat|nested] 22+ messages in thread* [Buildroot] [PATCH v2 1/6] package/compiler-rt: new package
2018-11-17 3:56 [Buildroot] [PATCH v2 1/6] package/compiler-rt: new package Matt Weber
` (5 preceding siblings ...)
2018-11-18 21:32 ` [Buildroot] [PATCH v2 1/6] package/compiler-rt: new package Ricardo Martincoski
@ 2018-11-18 22:28 ` Romain Naour
6 siblings, 0 replies; 22+ messages in thread
From: Romain Naour @ 2018-11-18 22:28 UTC (permalink / raw)
To: buildroot
Le 17/11/2018 ? 04:56, Matt Weber a ?crit?:
> This patch adds support for the compiler-rt (CLANG runtime) libary.
> It builds a set of static libraries and installs them into the
> CLANG/LLVM toolchain resource folder.
>
> Signed-off-by: Matthew Weber <matthew.weber@rockwellcollins.com>
> Cc: Romain Naour <romain.naour@smile.fr>
> ---
> Changes
> v1 -> v2
> [Romain
> - Removed unnecessary host-cmake dependency
> ---
> DEVELOPERS | 1 +
> package/Config.in | 1 +
> package/compiler-rt/Config.in | 12 ++++++++++++
> package/compiler-rt/compiler-rt.hash | 3 +++
> package/compiler-rt/compiler-rt.mk | 35 +++++++++++++++++++++++++++++++++++
> 5 files changed, 52 insertions(+)
> create mode 100644 package/compiler-rt/Config.in
> create mode 100644 package/compiler-rt/compiler-rt.hash
> create mode 100644 package/compiler-rt/compiler-rt.mk
>
> diff --git a/DEVELOPERS b/DEVELOPERS
> index 53467da..e78d649 100644
> --- a/DEVELOPERS
> +++ b/DEVELOPERS
> @@ -1360,6 +1360,7 @@ F: package/bridge-utils/
> F: package/checkpolicy/
> F: package/checksec/
> F: package/cgroupfs-mount/
> +F: package/compiler-rt/
> F: package/crda/
> F: package/devmem2/
> F: package/dnsmasq/
> diff --git a/package/Config.in b/package/Config.in
> index b60e770..73ddc2d 100644
> --- a/package/Config.in
> +++ b/package/Config.in
> @@ -1602,6 +1602,7 @@ menu "Other"
> source "package/clapack/Config.in"
> source "package/classpath/Config.in"
> source "package/cmocka/Config.in"
> + source "package/compiler-rt/Config.in"
> source "package/cppcms/Config.in"
> source "package/cracklib/Config.in"
> source "package/dawgdic/Config.in"
> diff --git a/package/compiler-rt/Config.in b/package/compiler-rt/Config.in
> new file mode 100644
> index 0000000..e15d3aa
> --- /dev/null
> +++ b/package/compiler-rt/Config.in
> @@ -0,0 +1,12 @@
> +config BR2_PACKAGE_COMPILER_RT
> + bool "compiler-rt"
> + depends on BR2_PACKAGE_LLVM
> + help
> + A collection of runtime libraries primarily used by clang and
> + llvm to provide builtins, sanitizer runtimes, and profiling
> + at runtime.
> +
> + https://compiler-rt.llvm.org/
> +
> +comment "compiler-rt requires llvm to be enabled"
> + depends on !BR2_PACKAGE_LLVM
> diff --git a/package/compiler-rt/compiler-rt.hash b/package/compiler-rt/compiler-rt.hash
> new file mode 100644
> index 0000000..df6ed19
> --- /dev/null
> +++ b/package/compiler-rt/compiler-rt.hash
> @@ -0,0 +1,3 @@
> +# Locally computed:
> +sha256 bdec7fe3cf2c85f55656c07dfb0bd93ae46f2b3dd8f33ff3ad6e7586f4c670d6 compiler-rt-7.0.0.src.tar.xz
> +sha256 417541d990edb3f96327ac03cb67e52eac80fc5c3e7afc69213cd04d7c3b9b27 LICENSE.TXT
> diff --git a/package/compiler-rt/compiler-rt.mk b/package/compiler-rt/compiler-rt.mk
> new file mode 100644
> index 0000000..dd962fb
> --- /dev/null
> +++ b/package/compiler-rt/compiler-rt.mk
> @@ -0,0 +1,35 @@
> +################################################################################
> +#
> +# compiler-rt
> +#
> +################################################################################
> +
> +# Compiler-RT should be bumped together with LLVM and Clang as the run-time is
> +# tied to the version of those tools
> +COMPILER_RT_VERSION = 7.0.0
> +COMPILER_RT_SOURCE = compiler-rt-$(COMPILER_RT_VERSION).src.tar.xz
> +COMPILER_RT_SITE = http://llvm.org/releases/$(COMPILER_RT_VERSION)
> +COMPILER_RT_LICENSE = NCSA MIT
> +COMPILER_RT_LICENSE_FILES = LICENSE.TXT
> +COMPILER_RT_DEPENDENCIES = host-clang llvm
> +
> +COMPILER_RT_INSTALL_STAGING = YES
> +COMPILER_RT_INSTALL_TARGET = NO
> +
> +
> +# -DCMAKE_INSTALL_PREFIX="" allows the COMPILER_RT_INSTALL_STAGING_CMDS to
> +# provide the complete path to compilier-rt for installation of the runtime
> +# libraries into the host-{clang,llvm} resources directory. The "resources"
> +# directory is loosely document at this point and will probably need revisited
> +# when making the llvm/clang tools SDK relocatable.
> +COMPILER_RT_CONF_OPTS=-DCOMPILER_RT_INCLUDE_TESTS=ON \
> + -DCOMPILER_RT_STANDALONE_BUILD=ON \
> + -DCOMPILER_RT_DEFAULT_TARGET_TRIPLE=$(GNU_TARGET_NAME) \
> + -DLLVM_CONFIG_PATH=$(HOST_DIR)/usr/bin/llvm-config \
> + -DCMAKE_INSTALL_PREFIX=""
Can you try with -DCOMPILER_RT_INSTALL_PATH=$(HOST_DIR) instead ?
Best regards,
Romain
> +
> +define COMPILER_RT_INSTALL_STAGING_CMDS
> + $(TARGET_MAKE_ENV) $(MAKE) -C $(@D) DESTDIR=$(HOST_DIR)/lib/clang/$(HOST_CLANG_VERSION)/ install/fast
> +endef
> +
> +$(eval $(cmake-package))
>
^ permalink raw reply [flat|nested] 22+ messages in thread