[PATCH] gcc-sanitizers: allow empty for lsan and tsan packages

[PATCH] gcc-sanitizers: allow empty for lsan and tsan packages

[kai.kang]

From: Kai Kang <kai.kang@windriver.com>

gcc checks threads and leak sanitizers support when do configure. If not
supported, packages of tsan and lsan are empty. If add such package to
image it fails to build:

| ERROR: core-image-minimal-1.0-r0 do_rootfs: libtsan-dev not found in the feeds
| ERROR: core-image-minimal-1.0-r0 do_rootfs: This is often caused by an empty
| package declared in a recipe's PACKAGES variable. (Empty packages are not
| constructed unless ALLOW_EMPTY_<pkg> = '1' is used.)

Set ALLOW_EMPTY for these packages to resolve the issue.

Signed-off-by: Kai Kang <kai.kang@windriver.com>
---
 meta/recipes-devtools/gcc/gcc-sanitizers.inc | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/meta/recipes-devtools/gcc/gcc-sanitizers.inc b/meta/recipes-devtools/gcc/gcc-sanitizers.inc
index df4e297..03d23ec 100644
--- a/meta/recipes-devtools/gcc/gcc-sanitizers.inc
+++ b/meta/recipes-devtools/gcc/gcc-sanitizers.inc
@@ -59,6 +59,13 @@ PACKAGES += "libasan libubsan liblsan libtsan"
 PACKAGES += "libasan-dev libubsan-dev liblsan-dev libtsan-dev"
 PACKAGES += "libasan-staticdev libubsan-staticdev liblsan-staticdev libtsan-staticdev"
 
+ALLOW_EMPTY_${MLPREFIX}liblsan = "1"
+ALLOW_EMPTY_${MLPREFIX}liblsan-dev = "1"
+ALLOW_EMPTY_${MLPREFIX}liblsan-staticdev = "1"
+ALLOW_EMPTY_${MLPREFIX}libtsan = "1"
+ALLOW_EMPTY_${MLPREFIX}libtsan-dev = "1"
+ALLOW_EMPTY_${MLPREFIX}libtsan-staticdev = "1"
+
 RDEPENDS_libasan += "libstdc++"
 RDEPENDS_libubsan += "libstdc++"
 RDEPENDS_liblsan += "libstdc++"
-- 
2.9.3

Re: [PATCH] gcc-sanitizers: allow empty for lsan and tsan packages

[Burton, Ross]

[-- Attachment #1: Type: text/plain, Size: 663 bytes --]

On 12 October 2016 at 03:37, <kai.kang@windriver.com> wrote:

> gcc checks threads and leak sanitizers support when do configure. If not
> supported, packages of tsan and lsan are empty. If add such package to
> image it fails to build:
>
> | ERROR: core-image-minimal-1.0-r0 do_rootfs: libtsan-dev not found in the
> feeds
> | ERROR: core-image-minimal-1.0-r0 do_rootfs: This is often caused by an
> empty
> | package declared in a recipe's PACKAGES variable. (Empty packages are not
> | constructed unless ALLOW_EMPTY_<pkg> = '1' is used.)
>

Not sure I'm understand where those dependencies are coming from, how would
I replicate this?

Ross

[-- Attachment #2: Type: text/html, Size: 1103 bytes --]

Re: [PATCH] gcc-sanitizers: allow empty for lsan and tsan packages

[Kang Kai]

[-- Attachment #1: Type: text/plain, Size: 2022 bytes --]

On 2016年10月12日 17:26, Burton, Ross wrote:
>
> On 12 October 2016 at 03:37, <kai.kang@windriver.com 
> <mailto:kai.kang@windriver.com>> wrote:
>
>     gcc checks threads and leak sanitizers support when do configure.
>     If not
>     supported, packages of tsan and lsan are empty. If add such package to
>     image it fails to build:
>
>     | ERROR: core-image-minimal-1.0-r0 do_rootfs: libtsan-dev not
>     found in the feeds
>     | ERROR: core-image-minimal-1.0-r0 do_rootfs: This is often caused
>     by an empty
>     | package declared in a recipe's PACKAGES variable. (Empty
>     packages are not
>     | constructed unless ALLOW_EMPTY_<pkg> = '1' is used.)
>
>
> Not sure I'm understand where those dependencies are coming from, how 
> would I replicate this?

gcc checks tsan and lsan support in source file 
gcc/libsanitizer/configure.tgt, it seems only support x86_64. Then tsan 
and lsan related packages will be empty for other target.
For qemuarm project, add in local.conf:

IMAGE_INSTALL_append = " libtsan"

Then run 'bitbake core-image-minimal", it shows errors:

ERROR: core-image-minimal-1.0-r0 do_rootfs: libtsan not found in the 
feeds (qemuarm armv5te armv5e armv5t armv5 armv4t armv4 arm noarch any 
all) in /buildarea3/kkang/poky-20141111/builds/qemuarm-Aug05/tmp/deploy/rpm.
ERROR: core-image-minimal-1.0-r0 do_rootfs: This is often caused by an 
empty package declared in a recipe's PACKAGES variable. (Empty packages 
are not constructed unless ALLOW_EMPTY_<pkg> = '1' is used.)
ERROR: core-image-minimal-1.0-r0 do_rootfs: Function failed: do_rootfs
ERROR: Logfile of failure stored in: 
/buildarea3/kkang/poky-20141111/builds/qemuarm-Aug05/tmp/work/qemuarm-poky-linux-gnueabi/core-image-minimal/1.0-r0/temp/log.do_rootfs.24600
ERROR: Task 
(/buildarea3/kkang/poky-20141111/poky/meta/recipes-core/images/core-image-minimal.bb:do_rootfs) 
failed with exit code '1'


Regards,
Kai



>
> Ross


-- 
Regards,
Neil | Kai Kang


[-- Attachment #2: Type: text/html, Size: 3699 bytes --]

Re: [PATCH] gcc-sanitizers: allow empty for lsan and tsan packages

[Burton, Ross]

[-- Attachment #1: Type: text/plain, Size: 532 bytes --]

On 13 October 2016 at 15:40, Kang Kai <Kai.Kang@windriver.com> wrote:

> gcc checks tsan and lsan support in source file gcc/libsanitizer/configure.tgt,
> it seems only support x86_64. Then tsan and lsan related packages will be
> empty for other target.
> For qemuarm project, add in local.conf:
>
> IMAGE_INSTALL_append = " libtsan"
>

If libtsan is x86-64 only, why not only add it for x86-64 images?  The
package not existing on machines it can't be built for is more obvious than
existing but being empty.

Ross

[-- Attachment #2: Type: text/html, Size: 965 bytes --]

Re: [PATCH] gcc-sanitizers: allow empty for lsan and tsan packages

[Kang Kai]

[-- Attachment #1: Type: text/plain, Size: 1294 bytes --]

On 2016年10月13日 22:59, Burton, Ross wrote:
>
> On 13 October 2016 at 15:40, Kang Kai <Kai.Kang@windriver.com 
> <mailto:Kai.Kang@windriver.com>> wrote:
>
>     gcc checks tsan and lsan support in source file
>     gcc/libsanitizer/configure.tgt, it seems only support x86_64. Then
>     tsan and lsan related packages will be empty for other target.
>     For qemuarm project, add in local.conf:
>
>     IMAGE_INSTALL_append = " libtsan"
>
>
> If libtsan is x86-64 only, why not only add it for x86-64 images?  The 
> package not existing on machines it can't be built for is more obvious 
> than existing but being empty.

The real scenario is for x86-64 kernel with x86 rootfs. We created a 
template to enable gcc sanitize support. It just simple adds sanitize 
related packages to image:

LIBX_DEV ?= ""
LIBX_DEV_x86-64 = "liblsan-dev libtsan-dev"

IMAGE_INSTALL += " \
     gcc-sanitizers \
     libasan-dev \
     libubsan-dev \
     ${LIBX_DEV} \
"

When multilib is enabled, it fails for x86-64 kernel with x86 rootfs 
such as lib32-xxx-image on qemux86-64. Var IMAGE_INSTALL will be expand 
with prefix lib32 but lib32-lsan* and lib32-tsan* are empty then cause 
the failures.

Regards,
Kai

>
> Ross


-- 
Regards,
Neil | Kai Kang


[-- Attachment #2: Type: text/html, Size: 2997 bytes --]

Re: [PATCH] gcc-sanitizers: allow empty for lsan and tsan packages

[Khem Raj]

On Fri, Oct 14, 2016 at 11:02 AM, Kang Kai <Kai.Kang@windriver.com> wrote:
> On 2016年10月13日 22:59, Burton, Ross wrote:
>
>
> On 13 October 2016 at 15:40, Kang Kai <Kai.Kang@windriver.com> wrote:
>>
>> gcc checks tsan and lsan support in source file
>> gcc/libsanitizer/configure.tgt, it seems only support x86_64. Then tsan and
>> lsan related packages will be empty for other target.
>> For qemuarm project, add in local.conf:
>>
>> IMAGE_INSTALL_append = " libtsan"
>
>
> If libtsan is x86-64 only, why not only add it for x86-64 images?  The
> package not existing on machines it can't be built for is more obvious than
> existing but being empty.
>
>
> The real scenario is for x86-64 kernel with x86 rootfs. We created a
> template to enable gcc sanitize support. It just simple adds sanitize
> related packages to image:
>
> LIBX_DEV ?= ""
> LIBX_DEV_x86-64 = "liblsan-dev libtsan-dev"
>
> IMAGE_INSTALL += " \
>     gcc-sanitizers \
>     libasan-dev \
>     libubsan-dev \
>     ${LIBX_DEV} \
> "
>
> When multilib is enabled, it fails for x86-64 kernel with x86 rootfs such as
> lib32-xxx-image on qemux86-64. Var IMAGE_INSTALL will be expand with prefix
> lib32 but lib32-lsan* and lib32-tsan* are empty then cause the failures.

perhaps we should build sanitizers for multilib too ?

>
> Regards,
> Kai
>
>
> Ross
>
>
>
> --
> Regards,
> Neil | Kai Kang

Re: [PATCH] gcc-sanitizers: allow empty for lsan and tsan packages

[Kang Kai]

On 2016年10月14日 17:32, Khem Raj wrote:
> On Fri, Oct 14, 2016 at 11:02 AM, Kang Kai <Kai.Kang@windriver.com> wrote:
>> On 2016年10月13日 22:59, Burton, Ross wrote:
>>
>>
>> On 13 October 2016 at 15:40, Kang Kai <Kai.Kang@windriver.com> wrote:
>>> gcc checks tsan and lsan support in source file
>>> gcc/libsanitizer/configure.tgt, it seems only support x86_64. Then tsan and
>>> lsan related packages will be empty for other target.
>>> For qemuarm project, add in local.conf:
>>>
>>> IMAGE_INSTALL_append = " libtsan"
>>
>> If libtsan is x86-64 only, why not only add it for x86-64 images?  The
>> package not existing on machines it can't be built for is more obvious than
>> existing but being empty.
>>
>>
>> The real scenario is for x86-64 kernel with x86 rootfs. We created a
>> template to enable gcc sanitize support. It just simple adds sanitize
>> related packages to image:
>>
>> LIBX_DEV ?= ""
>> LIBX_DEV_x86-64 = "liblsan-dev libtsan-dev"
>>
>> IMAGE_INSTALL += " \
>>      gcc-sanitizers \
>>      libasan-dev \
>>      libubsan-dev \
>>      ${LIBX_DEV} \
>> "
>>
>> When multilib is enabled, it fails for x86-64 kernel with x86 rootfs such as
>> lib32-xxx-image on qemux86-64. Var IMAGE_INSTALL will be expand with prefix
>> lib32 but lib32-lsan* and lib32-tsan* are empty then cause the failures.
> perhaps we should build sanitizers for multilib too ?

The problem is that thread and leak sanitizers are not supported for x86 
by checking in gcc/libsanitizer/configure.tgt:


# Filter out unsupported systems.
TSAN_TARGET_DEPENDENT_OBJECTS=
case "${target}" in
   x86_64-*-linux* | i?86-*-linux*)
         if test x$ac_cv_sizeof_void_p = x8; then
                 TSAN_SUPPORTED=yes
                 LSAN_SUPPORTED=yes
                 TSAN_TARGET_DEPENDENT_OBJECTS=tsan_rtl_amd64.lo
         fi
         ;;
...

that causes packages lib32-lsan* and lib32-tsan* are empty.


--Kai

>
>> Regards,
>> Kai
>>
>>
>> Ross
>>
>>
>>
>> --
>> Regards,
>> Neil | Kai Kang


-- 
Regards,
Neil | Kai Kang

Re: [PATCH] gcc-sanitizers: allow empty for lsan and tsan packages

[Khem Raj]

> On Oct 16, 2016, at 10:52 PM, Kang Kai <Kai.Kang@windriver.com> wrote:
> 
>> On 2016年10月14日 17:32, Khem Raj wrote:
>>> On Fri, Oct 14, 2016 at 11:02 AM, Kang Kai <Kai.Kang@windriver.com> wrote:
>>> On 2016年10月13日 22:59, Burton, Ross wrote:
>>> 
>>> 
>>>> On 13 October 2016 at 15:40, Kang Kai <Kai.Kang@windriver.com> wrote:
>>>> gcc checks tsan and lsan support in source file
>>>> gcc/libsanitizer/configure.tgt, it seems only support x86_64. Then tsan and
>>>> lsan related packages will be empty for other target.
>>>> For qemuarm project, add in local.conf:
>>>> 
>>>> IMAGE_INSTALL_append = " libtsan"
>>> 
>>> If libtsan is x86-64 only, why not only add it for x86-64 images?  The
>>> package not existing on machines it can't be built for is more obvious than
>>> existing but being empty.
>>> 
>>> 
>>> The real scenario is for x86-64 kernel with x86 rootfs. We created a
>>> template to enable gcc sanitize support. It just simple adds sanitize
>>> related packages to image:
>>> 
>>> LIBX_DEV ?= ""
>>> LIBX_DEV_x86-64 = "liblsan-dev libtsan-dev"
>>> 
>>> IMAGE_INSTALL += " \
>>>     gcc-sanitizers \
>>>     libasan-dev \
>>>     libubsan-dev \
>>>     ${LIBX_DEV} \
>>> "
>>> 
>>> When multilib is enabled, it fails for x86-64 kernel with x86 rootfs such as
>>> lib32-xxx-image on qemux86-64. Var IMAGE_INSTALL will be expand with prefix
>>> lib32 but lib32-lsan* and lib32-tsan* are empty then cause the failures.
>> perhaps we should build sanitizers for multilib too ?
> 
> The problem is that thread and leak sanitizers are not supported for x86 by checking in gcc/libsanitizer/configure.tgt:
> 
> 
> # Filter out unsupported systems.
> TSAN_TARGET_DEPENDENT_OBJECTS=
> case "${target}" in
>  x86_64-*-linux* | i?86-*-linux*)
>        if test x$ac_cv_sizeof_void_p = x8; then
>                TSAN_SUPPORTED=yes
>                LSAN_SUPPORTED=yes
>                TSAN_TARGET_DEPENDENT_OBJECTS=tsan_rtl_amd64.lo
>        fi
>        ;;
> ...
> 
> that causes packages lib32-lsan* and lib32-tsan* are empty.

Ok then may be keeping the empty packages is ok even though not optimal
> 
> 
> --Kai
> 
>> 
>>> Regards,
>>> Kai
>>> 
>>> 
>>> Ross
>>> 
>>> 
>>> 
>>> --
>>> Regards,
>>> Neil | Kai Kang
> 
> 
> -- not 
> Regards,
> Neil | Kai Kang
>