advice on recipe for shared lib mDNSResponder

advice on recipe for shared lib mDNSResponder

[Steve Pavao]

Hello,

I am fairly new to Yocto, yet have been able to successfully add a custom kernel object to my Yocto poky build, no problem.  However, I am having some difficulty adding a shared library, namely mDNSResponder.

Right now, my recipe is very simple and does not use autotools.  There are just a few tweaks for cross-building which I’ve added to the supplied mDNSResponder Makefile for mDNSPosix, in order to target 64-bit ARM hardware.  Here is the build error I encounter.  Obviously the environment is not being set 100% correctly, because it can not find stdio.h.  Is there an easy way to avoid this problem?

- Steve Pavao
Korg R&D

Log data follows:
| DEBUG: Executing shell function do_compile
| NOTE: make -j 4
| ERROR: oe_runmake failed
| make os=EmbeddedLinuxAarch64 Daemon libdns_sd -C mDNSPosix
| make[1]: warning: jobserver unavailable: using -j1.  Add '+' to parent make rule.
| make[1]: Entering directory '/data/development/lfs/yocto/poky/build/tmp/work/aarch64-poky-linux/mDNSResponder/333.10-r0/mDNSPosix'
| aarch64-poky-linux-gcc -I../mDNSCore -I../mDNSShared -Iobjects/prod/EmbeddedLinuxAarch64 -fwrapv -W -Wall -DPID_FILE=\"/var/run/mdnsd.pid\" -DMDNS_UDS_SERVERPATH=\"/var/run/mdnsd\" -DNOT_HAVE_SA_LEN -DUSES_NETLINK -DHAVE_LINUX -DTARGET_OS_LINUX -fno-strict-aliasing -Os -DMDNS_DEBUGMSGS=0   -c -o objects/prod/EmbeddedLinuxAarch64/PosixDaemon.c.o PosixDaemon.c
| PosixDaemon.c:31:19: fatal error: stdio.h: No such file or directory
|  #include <stdio.h>
|                    ^
| compilation terminated.
| Makefile:553: recipe for target 'objects/prod/EmbeddedLinuxAarch64/PosixDaemon.c.o' failed
| make[1]: *** [objects/prod/EmbeddedLinuxAarch64/PosixDaemon.c.o] Error 1

Re: advice on recipe for shared lib mDNSResponder

[Andre McCurdy]

On Wed, Sep 27, 2017 at 3:58 PM, Steve Pavao <stevep@korgrd.com> wrote:
> Hello,
>
> I am fairly new to Yocto, yet have been able to successfully add a custom kernel object to my Yocto poky build, no problem.  However, I am having some difficulty adding a shared library, namely mDNSResponder.
>
> Right now, my recipe is very simple and does not use autotools.  There are just a few tweaks for cross-building which I’ve added to the supplied mDNSResponder Makefile for mDNSPosix, in order to target 64-bit ARM hardware.  Here is the build error I encounter.  Obviously the environment is not being set 100% correctly, because it can not find stdio.h.  Is there an easy way to avoid this problem?

You need to ensure that the Makefiles etc for the package you are
building respect the ${CC} environment variable defined by OE. It
includes not only the name of the cross compiler but also important
command line options, e.g. tuning for the correct target CPU and the
correct --sysroot option. From your build log the --sysroot option is
missing, therefore the compiler can not find standard include files
etc.

Unfortunately packages with build with custom Makefiles are so diverse
that there's no single approach to making them work. Sometimes you can
force CC=${CC} etc via the make command line to over-ride incorrect
defaults in the Makefile, sometime the Makefile needs to be patched,
etc. Even if the build succeeds it's wise to carefully check the build
log to ensure that all calls to the compiler, linker, etc contain the
flags defined by OE.

Note however that according to the layer index there does already seem
to be at least one recipe for mDNSResponder. It includes a Makefile
patch which looks like it will address your issue (and some others):

  http://git.yoctoproject.org/cgit/cgit.cgi/meta-intel-iot-middleware/tree/recipes-connectivity/mdns/mdns_544.bb?h=master

>
> - Steve Pavao
> Korg R&D
>
> Log data follows:
> | DEBUG: Executing shell function do_compile
> | NOTE: make -j 4
> | ERROR: oe_runmake failed
> | make os=EmbeddedLinuxAarch64 Daemon libdns_sd -C mDNSPosix
> | make[1]: warning: jobserver unavailable: using -j1.  Add '+' to parent make rule.
> | make[1]: Entering directory '/data/development/lfs/yocto/poky/build/tmp/work/aarch64-poky-linux/mDNSResponder/333.10-r0/mDNSPosix'
> | aarch64-poky-linux-gcc -I../mDNSCore -I../mDNSShared -Iobjects/prod/EmbeddedLinuxAarch64 -fwrapv -W -Wall -DPID_FILE=\"/var/run/mdnsd.pid\" -DMDNS_UDS_SERVERPATH=\"/var/run/mdnsd\" -DNOT_HAVE_SA_LEN -DUSES_NETLINK -DHAVE_LINUX -DTARGET_OS_LINUX -fno-strict-aliasing -Os -DMDNS_DEBUGMSGS=0   -c -o objects/prod/EmbeddedLinuxAarch64/PosixDaemon.c.o PosixDaemon.c
> | PosixDaemon.c:31:19: fatal error: stdio.h: No such file or directory
> |  #include <stdio.h>
> |                    ^
> | compilation terminated.
> | Makefile:553: recipe for target 'objects/prod/EmbeddedLinuxAarch64/PosixDaemon.c.o' failed
> | make[1]: *** [objects/prod/EmbeddedLinuxAarch64/PosixDaemon.c.o] Error 1
> --
> _______________________________________________
> yocto mailing list
> yocto@yoctoproject.org
> https://lists.yoctoproject.org/listinfo/yocto

Re: advice on recipe for shared lib mDNSResponder

[Burton, Ross]

[-- Attachment #1: Type: text/plain, Size: 483 bytes --]

On 27 September 2017 at 23:58, Steve Pavao <stevep@korgrd.com> wrote:

> I am fairly new to Yocto, yet have been able to successfully add a custom
> kernel object to my Yocto poky build, no problem.  However, I am having
> some difficulty adding a shared library, namely mDNSResponder.
>

Andre has already replied with a good answer, but I'll point out that
Avahi, part of oe-core, has a mDNSResponder compatibility library so you
may be able to use that instead.

Ross

[-- Attachment #2: Type: text/html, Size: 805 bytes --]

Re: advice on recipe for shared lib mDNSResponder

[Steve Pavao]

Hello,

Thanks so much for the great and useful information from you and Ross.

I was unable to successfully build that recipe here, though.  I’ll check with the maintainers to ask why.

All I did was clone the meta-intel-iot-middleware repo, add that layer to bblayers.conf, and add mdns to IMAGE_INSTALL_append to get this far.

- Steve Pavao
Korg R&D


Build Configuration:
BB_VERSION        = "1.34.0"
BUILD_SYS         = "x86_64-linux"
NATIVELSBSTRING   = "universal"
TARGET_SYS        = "aarch64-poky-linux"
MACHINE           = "raspberrypi3-64"
DISTRO            = "poky"
DISTRO_VERSION    = "2.3.2"
TUNE_FEATURES     = "aarch64"
TARGET_FPU        = ""
meta              
meta-poky         
meta-yocto-bsp    = "pyro:072430b9b3a78b318b66371c36e2986d2ed5cba4"
meta-oe           
meta-python       
meta-multimedia   = "pyro:5e82995148a2844c6f483ae5ddd1438d87ea9fb7"
meta-intel-iot-middleware = "master:fc8eabfa4fb54802d3f97123b9d2954450175e33"
meta-raspberrypi  = "pyro:c981f01a2cdb3299eb57920cffbb5dfd4248209a"

Initialising tasks: 100% |#############################################################################################################| Time: 0:00:10
NOTE: Executing SetScene Tasks
NOTE: Executing RunQueue Tasks
ERROR: mdns-544-r1 do_compile: oe_runmake failed
ERROR: mdns-544-r1 do_compile: Function failed: do_compile (log file is located at /data/development/lfs/yocto/poky/build/tmp/work/aarch64-poky-linux/mdns/544-r1/temp/log.do_compile.10472)
ERROR: Logfile of failure stored in: /data/development/lfs/yocto/poky/build/tmp/work/aarch64-poky-linux/mdns/544-r1/temp/log.do_compile.10472
Log data follows:
| DEBUG: Executing shell function do_compile
| NOTE: make os=linux DEBUG=0
| ERROR: oe_runmake failed
| make: cc: Command not found
| Makefile:494: recipe for target 'objects/prod/PosixDaemon.c.o' failed
| make: *** [objects/prod/PosixDaemon.c.o] Error 127
| WARNING: exit code 1 from a shell command.
| ERROR: Function failed: do_compile (log file is located at /data/development/lfs/yocto/poky/build/tmp/work/aarch64-poky-linux/mdns/544-r1/temp/log.do_compile.10472)
ERROR: Task (/data/development/lfs/yocto/poky/meta-intel-iot-middleware/recipes-connectivity/mdns/mdns_544.bb:do_compile) failed with exit code '1'


> On Sep 27, 2017, at 7:17 PM, Andre McCurdy <armccurdy@gmail.com> wrote:
> 
> On Wed, Sep 27, 2017 at 3:58 PM, Steve Pavao <stevep@korgrd.com> wrote:
>> Hello,
>> 
>> I am fairly new to Yocto, yet have been able to successfully add a custom kernel object to my Yocto poky build, no problem. However, I am having some difficulty adding a shared library, namely mDNSResponder.
>> 
>> Right now, my recipe is very simple and does not use autotools.  There are just a few tweaks for cross-building which I’ve added to the supplied mDNSResponder Makefile for mDNSPosix, in order to target 64-bit ARM hardware.  Here is the build error I encounter.  Obviously the environment is not being set 100% correctly, because it can not find stdio.h.  Is there an easy way to avoid this problem?
> 
> You need to ensure that the Makefiles etc for the package you are
> building respect the ${CC} environment variable defined by OE. It
> includes not only the name of the cross compiler but also important
> command line options, e.g. tuning for the correct target CPU and the
> correct --sysroot option. From your build log the --sysroot option is
> missing, therefore the compiler can not find standard include files
> etc.
> 
> Unfortunately packages with build with custom Makefiles are so diverse
> that there's no single approach to making them work. Sometimes you can
> force CC=${CC} etc via the make command line to over-ride incorrect
> defaults in the Makefile, sometime the Makefile needs to be patched,
> etc. Even if the build succeeds it's wise to carefully check the build
> log to ensure that all calls to the compiler, linker, etc contain the
> flags defined by OE.
> 
> Note however that according to the layer index there does already seem
> to be at least one recipe for mDNSResponder. It includes a Makefile
> patch which looks like it will address your issue (and some others):
> 
>  http://git.yoctoproject.org/cgit/cgit.cgi/meta-intel-iot-middleware/tree/recipes-connectivity/mdns/mdns_544.bb?h=master
> 
>> 
>> - Steve Pavao
>> Korg R&D
>> 
>> Log data follows:
>> | DEBUG: Executing shell function do_compile
>> | NOTE: make -j 4
>> | ERROR: oe_runmake failed
>> | make os=EmbeddedLinuxAarch64 Daemon libdns_sd -C mDNSPosix
>> | make[1]: warning: jobserver unavailable: using -j1.  Add '+' to parent make rule.
>> | make[1]: Entering directory '/data/development/lfs/yocto/poky/build/tmp/work/aarch64-poky-linux/mDNSResponder/333.10-r0/mDNSPosix'
>> | aarch64-poky-linux-gcc -I../mDNSCore -I../mDNSShared -Iobjects/prod/EmbeddedLinuxAarch64 -fwrapv -W -Wall -DPID_FILE=\"/var/run/mdnsd.pid\" -DMDNS_UDS_SERVERPATH=\"/var/run/mdnsd\" -DNOT_HAVE_SA_LEN -DUSES_NETLINK -DHAVE_LINUX -DTARGET_OS_LINUX -fno-strict-aliasing -Os -DMDNS_DEBUGMSGS=0   -c -o objects/prod/EmbeddedLinuxAarch64/PosixDaemon.c.o PosixDaemon.c
>> | PosixDaemon.c:31:19: fatal error: stdio.h: No such file or directory
>> |  #include <stdio.h>
>> |                    ^
>> | compilation terminated.
>> | Makefile:553: recipe for target 'objects/prod/EmbeddedLinuxAarch64/PosixDaemon.c.o' failed
>> | make[1]: *** [objects/prod/EmbeddedLinuxAarch64/PosixDaemon.c.o] Error 1
>> --
>> _______________________________________________
>> yocto mailing list
>> yocto@yoctoproject.org
>> https://lists.yoctoproject.org/listinfo/yocto

Re: advice on recipe for shared lib mDNSResponder

[Paul Eggleton]

Hi Steve,

On Friday, 29 September 2017 5:29:32 AM NZDT Steve Pavao wrote:
> I was unable to successfully build that recipe here, though.  I’ll check
> with the maintainers to ask why.
> 
> All I did was clone the meta-intel-iot-middleware repo, add that layer to
> bblayers.conf, and add mdns to IMAGE_INSTALL_append to get this far.

FYI the meta-intel-iot-middleware repository is no longer being actively
maintained, however coincidentally I am currently in the process of updating
and moving the useful pieces elsewhere. mdns wasn't on my list, however I have
just tried to fix it up, here is what I have so far:

http://git.yoctoproject.org/cgit/cgit.cgi/meta-intel-iot-middleware/commit/?h=paule/mdns-upgrade-1&id=7389dd6b1d3e69a05b8fa62673ba03b2a91073ff

Unfortunately I hit a linking issue - the dns-sd binary gets linked to the
unversioned libdns_sd.so symlink, which (correctly) is packaged in the -dev
package, but as a result you get a do_package_qa failure (and probably
runtime issues). Moving the .so symlink into the main package wouldn't be
the right thing to do, so the fix should be made elsewhere. I don't have
any more time to poke at it but I'm guessing it's not that hard to fix if
you're familiar with this kind of problem. (The issues we've been dealing with
here are unfortunately par for the course when building makefile-only
software.)

I'd probably recommend going the avahi route as suggested by Ross in his
reply, but if you absolutely need mDNSResponder then maybe this will get you
some way towards that.

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre

Re: advice on recipe for shared lib mDNSResponder

[Andre McCurdy]

On Thu, Sep 28, 2017 at 3:30 PM, Paul Eggleton
<paul.eggleton@linux.intel.com> wrote:
> Hi Steve,
>
> On Friday, 29 September 2017 5:29:32 AM NZDT Steve Pavao wrote:
>> I was unable to successfully build that recipe here, though.  I’ll check
>> with the maintainers to ask why.
>>
>> All I did was clone the meta-intel-iot-middleware repo, add that layer to
>> bblayers.conf, and add mdns to IMAGE_INSTALL_append to get this far.
>
> FYI the meta-intel-iot-middleware repository is no longer being actively
> maintained, however coincidentally I am currently in the process of updating
> and moving the useful pieces elsewhere. mdns wasn't on my list, however I have
> just tried to fix it up, here is what I have so far:
>
> http://git.yoctoproject.org/cgit/cgit.cgi/meta-intel-iot-middleware/commit/?h=paule/mdns-upgrade-1&id=7389dd6b1d3e69a05b8fa62673ba03b2a91073ff
>
> Unfortunately I hit a linking issue - the dns-sd binary gets linked to the
> unversioned libdns_sd.so symlink,

If the lib isn't versioned, what does the symlink point to?

> which (correctly) is packaged in the -dev
> package, but as a result you get a do_package_qa failure (and probably
> runtime issues). Moving the .so symlink into the main package wouldn't be
> the right thing to do, so the fix should be made elsewhere. I don't have
> any more time to poke at it but I'm guessing it's not that hard to fix if
> you're familiar with this kind of problem. (The issues we've been dealing with
> here are unfortunately par for the course when building makefile-only
> software.)
>
> I'd probably recommend going the avahi route as suggested by Ross in his
> reply, but if you absolutely need mDNSResponder then maybe this will get you
> some way towards that.
>
> Cheers,
> Paul
>
> --
>
> Paul Eggleton
> Intel Open Source Technology Centre

Re: advice on recipe for shared lib mDNSResponder

[Andre McCurdy]

On Thu, Sep 28, 2017 at 5:11 PM, Andre McCurdy <armccurdy@gmail.com> wrote:
> On Thu, Sep 28, 2017 at 3:30 PM, Paul Eggleton
> <paul.eggleton@linux.intel.com> wrote:
>> Hi Steve,
>>
>> On Friday, 29 September 2017 5:29:32 AM NZDT Steve Pavao wrote:
>>> I was unable to successfully build that recipe here, though.  I’ll check
>>> with the maintainers to ask why.
>>>
>>> All I did was clone the meta-intel-iot-middleware repo, add that layer to
>>> bblayers.conf, and add mdns to IMAGE_INSTALL_append to get this far.
>>
>> FYI the meta-intel-iot-middleware repository is no longer being actively
>> maintained, however coincidentally I am currently in the process of updating
>> and moving the useful pieces elsewhere. mdns wasn't on my list, however I have
>> just tried to fix it up, here is what I have so far:
>>
>> http://git.yoctoproject.org/cgit/cgit.cgi/meta-intel-iot-middleware/commit/?h=paule/mdns-upgrade-1&id=7389dd6b1d3e69a05b8fa62673ba03b2a91073ff
>>
>> Unfortunately I hit a linking issue - the dns-sd binary gets linked to the
>> unversioned libdns_sd.so symlink,
>
> If the lib isn't versioned, what does the symlink point to?

OK, so the lib is versioned (ie there is a libdns_sd.so.1 which the
symlink points to) but the problem is that libdns_sd.so.1 doesn't set
a soname. Fix would be to add:

  -Wl,-soname,libdns_sd.so.1

to the linker command line when libdns_sd.so is linked.

Re: advice on recipe for shared lib mDNSResponder

[Paul Eggleton]

On Friday, 29 September 2017 1:44:13 PM NZDT Andre McCurdy wrote:
> On Thu, Sep 28, 2017 at 5:11 PM, Andre McCurdy <armccurdy@gmail.com> wrote:
> > On Thu, Sep 28, 2017 at 3:30 PM, Paul Eggleton
> > <paul.eggleton@linux.intel.com> wrote:
> >> Hi Steve,
> >>
> >> On Friday, 29 September 2017 5:29:32 AM NZDT Steve Pavao wrote:
> >>> I was unable to successfully build that recipe here, though.  I’ll check
> >>> with the maintainers to ask why.
> >>>
> >>> All I did was clone the meta-intel-iot-middleware repo, add that layer to
> >>> bblayers.conf, and add mdns to IMAGE_INSTALL_append to get this far.
> >>
> >> FYI the meta-intel-iot-middleware repository is no longer being actively
> >> maintained, however coincidentally I am currently in the process of updating
> >> and moving the useful pieces elsewhere. mdns wasn't on my list, however I have
> >> just tried to fix it up, here is what I have so far:
> >>
> >> http://git.yoctoproject.org/cgit/cgit.cgi/meta-intel-iot-middleware/commit/?h=paule/mdns-upgrade-1&id=7389dd6b1d3e69a05b8fa62673ba03b2a91073ff
> >>
> >> Unfortunately I hit a linking issue - the dns-sd binary gets linked to the
> >> unversioned libdns_sd.so symlink,
> >
> > If the lib isn't versioned, what does the symlink point to?
> 
> OK, so the lib is versioned (ie there is a libdns_sd.so.1 which the
> symlink points to) but the problem is that libdns_sd.so.1 doesn't set
> a soname. Fix would be to add:
> 
>   -Wl,-soname,libdns_sd.so.1

That did the trick. I'm ashamed to say that despite years of working on
recipes I had yet to delve into this particular area, so thanks for filling in
a gap in my knowledge.

I've pushed a new commit:

  http://git.yoctoproject.org/cgit/cgit.cgi/meta-intel-iot-middleware/commit/?h=paule/mdns-upgrade-1&id=fab9ceda3d748e66d0273b08a4e6f7fa427763ea

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre