[Buildroot] [autobuild.buildroot.net] Your daily results for 2021-02-04

[Buildroot] [autobuild.buildroot.net] Your daily results for 2021-02-04

[Heiko Thiery]

Hi all,

Am Fr., 5. Feb. 2021 um 09:18 Uhr schrieb Thomas Petazzoni
<thomas.petazzoni@bootlin.com>:
>
> Hello,
>
> Autobuilder failures
> ====================
>
> Below is a list of build failures reported by the Buildroot
> autobuilders in relation to packages or CPU
> architectures you are in charge of. Please help us
> improving the quality of Buildroot by investigating
> those build failures and sending patches to fix
> them.Thanks!
>
> Results for the 'master' branch
> -------------------------------
>
> Build failures related to your packages:
>
>     arch     |             reason             |                                       url
> -------------+--------------------------------+---------------------------------------------------------------------------------
>    xtensa    |        netopeer2-1.1.53        | http://autobuild.buildroot.net/results/e21834d4d2ee580f00f0fdcbd3728787148c0da9
>    nios2     |        netopeer2-1.1.53        | http://autobuild.buildroot.net/results/e9bfe730da3faec5884e8950855a34541c3408a9
> microblazeel |        netopeer2-1.1.53        | http://autobuild.buildroot.net/results/99ca5aa18074b19f2e6a311c2d38addb697f552b
>   powerpc    |        netopeer2-1.1.53        | http://autobuild.buildroot.net/results/2b9ad4626a1b6918a8e8d586521626306f5f8133
> microblazeel |        netopeer2-1.1.53        | http://autobuild.buildroot.net/results/65108399fb6d7b02d812bc15d5074bfd404764c5
>    nios2     |        netopeer2-1.1.53        | http://autobuild.buildroot.net/results/6730e1e6ceebec94a9ad27deac0a4d1bf4661536
>   powerpc    |        netopeer2-1.1.53        | http://autobuild.buildroot.net/results/16bf1c3b4059a159480d23e6e132dcff05bd9b96
> powerpc64le  |        netopeer2-1.1.53        | http://autobuild.buildroot.net/results/985aac8fd746c64e4ad0e093028164d8ae637409
>    mipsel    |        netopeer2-1.1.53        | http://autobuild.buildroot.net/results/da0e36543cf68e4e9bbe819945a884193f33819a

I checked the reason for the build failure on the netopeer2 package.
It is caused by some files that are created in /dev/shm/sr_* during
the installation process.

I tried to find a solution for that. My first intention was to do a
PRE_INSTALL_HOOK that deletes these files before the installation. But
YANN disclaimed that because we should never delete files in
/dev/shm/. This could lead to failures when doing concurrent parallel
builds.

To be more detailed what is going on:
The netopeer2 package can install the required yang models for runtime
during installation. Therefore an additional script (setup.sh) is
invoked. There the sysrepocfg host tool is used to do the installation
of these yang models. sysrepo will then create this /dev/shm files and
leave them. But with the updated netopeer2 package the shm files are
incompatible and the build errors appear.

So I see here 3 possible solutions:
1. do the PRE_INSTALL_HOOK to remove the files every time (disclaimed by Yann).
2. remove this files by hand (no long term solution).
3. disable the installation of the yang modules .. but then we have a
non functional installation available and we leave the installation of
the yang modules to the user.

What do you think?

-- 
Heiko

[Buildroot] [autobuild.buildroot.net] Your daily results for 2021-02-04

[Peter Seiderer]

Hello Heiko,

On Fri, 5 Feb 2021 15:42:46 +0100, Heiko Thiery <heiko.thiery@gmail.com> wrote:

> Hi all,
>
> Am Fr., 5. Feb. 2021 um 09:18 Uhr schrieb Thomas Petazzoni
> <thomas.petazzoni@bootlin.com>:
> >
> > Hello,
> >
> > Autobuilder failures
> > ====================
> >
> > Below is a list of build failures reported by the Buildroot
> > autobuilders in relation to packages or CPU
> > architectures you are in charge of. Please help us
> > improving the quality of Buildroot by investigating
> > those build failures and sending patches to fix
> > them.Thanks!
> >
> > Results for the 'master' branch
> > -------------------------------
> >
> > Build failures related to your packages:
> >
> >     arch     |             reason             |                                       url
> > -------------+--------------------------------+---------------------------------------------------------------------------------
> >    xtensa    |        netopeer2-1.1.53        | http://autobuild.buildroot.net/results/e21834d4d2ee580f00f0fdcbd3728787148c0da9
> >    nios2     |        netopeer2-1.1.53        | http://autobuild.buildroot.net/results/e9bfe730da3faec5884e8950855a34541c3408a9
> > microblazeel |        netopeer2-1.1.53        | http://autobuild.buildroot.net/results/99ca5aa18074b19f2e6a311c2d38addb697f552b
> >   powerpc    |        netopeer2-1.1.53        | http://autobuild.buildroot.net/results/2b9ad4626a1b6918a8e8d586521626306f5f8133
> > microblazeel |        netopeer2-1.1.53        | http://autobuild.buildroot.net/results/65108399fb6d7b02d812bc15d5074bfd404764c5
> >    nios2     |        netopeer2-1.1.53        | http://autobuild.buildroot.net/results/6730e1e6ceebec94a9ad27deac0a4d1bf4661536
> >   powerpc    |        netopeer2-1.1.53        | http://autobuild.buildroot.net/results/16bf1c3b4059a159480d23e6e132dcff05bd9b96
> > powerpc64le  |        netopeer2-1.1.53        | http://autobuild.buildroot.net/results/985aac8fd746c64e4ad0e093028164d8ae637409
> >    mipsel    |        netopeer2-1.1.53        | http://autobuild.buildroot.net/results/da0e36543cf68e4e9bbe819945a884193f33819a
>
> I checked the reason for the build failure on the netopeer2 package.
> It is caused by some files that are created in /dev/shm/sr_* during
> the installation process.
>
> I tried to find a solution for that. My first intention was to do a
> PRE_INSTALL_HOOK that deletes these files before the installation. But
> YANN disclaimed that because we should never delete files in
> /dev/shm/. This could lead to failures when doing concurrent parallel
> builds.
>
> To be more detailed what is going on:
> The netopeer2 package can install the required yang models for runtime
> during installation. Therefore an additional script (setup.sh) is
> invoked. There the sysrepocfg host tool is used to do the installation
> of these yang models. sysrepo will then create this /dev/shm files and
> leave them. But with the updated netopeer2 package the shm files are
> incompatible and the build errors appear.
>
> So I see here 3 possible solutions:
> 1. do the PRE_INSTALL_HOOK to remove the files every time (disclaimed by Yann).
> 2. remove this files by hand (no long term solution).
> 3. disable the installation of the yang modules .. but then we have a
> non functional installation available and we leave the installation of
> the yang modules to the user.
>
> What do you think?
>

Fix the script to cleanup afterwards? The /dev/shm/sr_* files are created
on the host, are they needed afterwards/at-run-time on the target?

OpenWRT seems to disable buildtime-module-install ([1]) for a similar case...

Regards,
Peter

[1] https://github.com/openwrt/packages/blob/openwrt-18.06/net/sysrepo/patches/002-remove-buildtime-module-install

[Buildroot] [autobuild.buildroot.net] Your daily results for 2021-02-04

[Peter Korsgaard]

>>>>> "Heiko" == Heiko Thiery <heiko.thiery@gmail.com> writes:

Hi,

 >> mipsel    |        netopeer2-1.1.53        | http://autobuild.buildroot.net/results/da0e36543cf68e4e9bbe819945a884193f33819a

 > I checked the reason for the build failure on the netopeer2 package.
 > It is caused by some files that are created in /dev/shm/sr_* during
 > the installation process.

 > I tried to find a solution for that. My first intention was to do a
 > PRE_INSTALL_HOOK that deletes these files before the installation. But
 > YANN disclaimed that because we should never delete files in
 > /dev/shm/. This could lead to failures when doing concurrent parallel
 > builds.

 > To be more detailed what is going on:
 > The netopeer2 package can install the required yang models for runtime
 > during installation. Therefore an additional script (setup.sh) is
 > invoked. There the sysrepocfg host tool is used to do the installation
 > of these yang models. sysrepo will then create this /dev/shm files and
 > leave them. But with the updated netopeer2 package the shm files are
 > incompatible and the build errors appear.

 > So I see here 3 possible solutions:
 > 1. do the PRE_INSTALL_HOOK to remove the files every time (disclaimed by Yann).
 > 2. remove this files by hand (no long term solution).
 > 3. disable the installation of the yang modules .. but then we have a
 > non functional installation available and we leave the installation of
 > the yang modules to the user.

Ideally the package should be fixed to create those files in
$DESTDIR/dev/shm rather than mess around with the host /dev/shm.

But as /dev/shm is not persistent, how does this work at runtime? What
do those files do exactly?

-- 
Bye, Peter Korsgaard

[Buildroot] [autobuild.buildroot.net] Your daily results for 2021-02-04

[Yann E. MORIN]

Heiko, All,

On 2021-02-05 15:42 +0100, Heiko Thiery spake thusly:
> Am Fr., 5. Feb. 2021 um 09:18 Uhr schrieb Thomas Petazzoni
> <thomas.petazzoni@bootlin.com>:
> >    xtensa    |        netopeer2-1.1.53        | http://autobuild.buildroot.net/results/e21834d4d2ee580f00f0fdcbd3728787148c0da9
> >    nios2     |        netopeer2-1.1.53        | http://autobuild.buildroot.net/results/e9bfe730da3faec5884e8950855a34541c3408a9
> > microblazeel |        netopeer2-1.1.53        | http://autobuild.buildroot.net/results/99ca5aa18074b19f2e6a311c2d38addb697f552b
> >   powerpc    |        netopeer2-1.1.53        | http://autobuild.buildroot.net/results/2b9ad4626a1b6918a8e8d586521626306f5f8133
> > microblazeel |        netopeer2-1.1.53        | http://autobuild.buildroot.net/results/65108399fb6d7b02d812bc15d5074bfd404764c5
> >    nios2     |        netopeer2-1.1.53        | http://autobuild.buildroot.net/results/6730e1e6ceebec94a9ad27deac0a4d1bf4661536
> >   powerpc    |        netopeer2-1.1.53        | http://autobuild.buildroot.net/results/16bf1c3b4059a159480d23e6e132dcff05bd9b96
> > powerpc64le  |        netopeer2-1.1.53        | http://autobuild.buildroot.net/results/985aac8fd746c64e4ad0e093028164d8ae637409
> >    mipsel    |        netopeer2-1.1.53        | http://autobuild.buildroot.net/results/da0e36543cf68e4e9bbe819945a884193f33819a
> 
> I checked the reason for the build failure on the netopeer2 package.
> It is caused by some files that are created in /dev/shm/sr_* during
> the installation process.
> 
> I tried to find a solution for that. My first intention was to do a
> PRE_INSTALL_HOOK that deletes these files before the installation. But
> YANN disclaimed that because we should never delete files in

*Yann, please. ;-)

So, first, there was a misunderstanding on my side. I thought /dev/shm
would contain system-level nodes. But I was confused, probably with pts
et al. Anyway, /dev/shm is the place where glibc will first look at to
create the files that backs shared memoru (shm). Removing files in there
is in fact not an issue per-se (although leftover files means something
is not cleaning up nicely behind them, and misses calling shm_unlink()).

Second, once I noticed that, my oposition to removing files in there is
that it can remove more than expected, and espcially, it may removes
frils from another, concurrent run. So we can not removes files using
wildcards.

Removing /dev/shm/sr_* is wrong.

> /dev/shm/. This could lead to failures when doing concurrent parallel
> builds.

That.

> To be more detailed what is going on:
> The netopeer2 package can install the required yang models for runtime
> during installation. Therefore an additional script (setup.sh) is
> invoked. There the sysrepocfg host tool is used to do the installation
> of these yang models. sysrepo will then create this /dev/shm files and
> leave them. But with the updated netopeer2 package the shm files are
> incompatible and the build errors appear.
> 
> So I see here 3 possible solutions:
> 1. do the PRE_INSTALL_HOOK to remove the files every time (disclaimed by Yann).
> 2. remove this files by hand (no long term solution).
> 3. disable the installation of the yang modules .. but then we have a
> non functional installation available and we leave the installation of
> the yang modules to the user.
> 
> What do you think?

4. Fix sysrepo to clean up when it quits.

And also make sure nothing is left running that still have those shared
ememory opened (e.g. a forked sysrepo still lurking in the background?)

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [autobuild.buildroot.net] Your daily results for 2021-02-04

[Yann E. MORIN]

Peter, All,

On 2021-02-05 19:35 +0100, Peter Korsgaard spake thusly:
> >>>>> "Heiko" == Heiko Thiery <heiko.thiery@gmail.com> writes:
>  >> mipsel    |        netopeer2-1.1.53        | http://autobuild.buildroot.net/results/da0e36543cf68e4e9bbe819945a884193f33819a
[--SNIP--]
>  > So I see here 3 possible solutions:
>  > 1. do the PRE_INSTALL_HOOK to remove the files every time (disclaimed by Yann).
>  > 2. remove this files by hand (no long term solution).
>  > 3. disable the installation of the yang modules .. but then we have a
>  > non functional installation available and we leave the installation of
>  > the yang modules to the user.
> 
> Ideally the package should be fixed to create those files in
> $DESTDIR/dev/shm rather than mess around with the host /dev/shm.
> 
> But as /dev/shm is not persistent, how does this work at runtime? What
> do those files do exactly?

They are shared memory.

From shm_overview(7):

    On Linux, shared memory objects are created in a (tmpfs(5)) virtual
    filesystem, normally mounted under /dev/shm. [...]

There is in fact nothing wrong in creating such shm objects during the
build.

What is wrong, is leaving them lingering about...

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [autobuild.buildroot.net] Your daily results for 2021-02-04

[Peter Korsgaard]

>>>>> "Yann" == Yann E MORIN <yann.morin.1998@free.fr> writes:

Hi,

 >> Ideally the package should be fixed to create those files in
 >> $DESTDIR/dev/shm rather than mess around with the host /dev/shm.
 >> 
 >> But as /dev/shm is not persistent, how does this work at runtime? What
 >> do those files do exactly?

 > They are shared memory.

 > From shm_overview(7):

 >     On Linux, shared memory objects are created in a (tmpfs(5)) virtual
 >     filesystem, normally mounted under /dev/shm. [...]

That I get.

 > There is in fact nothing wrong in creating such shm objects during the
 > build.

From the logs it looks like they are created during the installation,
rather than during the build.

 > What is wrong, is leaving them lingering about...

Indeed. I don't particular understand why the installation step needs to
fiddle with shared memory objects in the first place, but Ok.

So these objects are only needed temporarily during installation and not
later at runtime?

-- 
Bye, Peter Korsgaard

[Buildroot] [autobuild.buildroot.net] Your daily results for 2021-02-04

[Peter Seiderer]

On Fri, 5 Feb 2021 22:34:53 +0100, "Yann E. MORIN" <yann.morin.1998@free.fr> wrote:

> Peter, All,
>
> On 2021-02-05 19:35 +0100, Peter Korsgaard spake thusly:
> > >>>>> "Heiko" == Heiko Thiery <heiko.thiery@gmail.com> writes:
> >  >> mipsel    |        netopeer2-1.1.53        | http://autobuild.buildroot.net/results/da0e36543cf68e4e9bbe819945a884193f33819a
> [--SNIP--]
> >  > So I see here 3 possible solutions:
> >  > 1. do the PRE_INSTALL_HOOK to remove the files every time (disclaimed by Yann).
> >  > 2. remove this files by hand (no long term solution).
> >  > 3. disable the installation of the yang modules .. but then we have a
> >  > non functional installation available and we leave the installation of
> >  > the yang modules to the user.
> >
> > Ideally the package should be fixed to create those files in
> > $DESTDIR/dev/shm rather than mess around with the host /dev/shm.
> >
> > But as /dev/shm is not persistent, how does this work at runtime? What
> > do those files do exactly?
>
> They are shared memory.
>
> From shm_overview(7):
>
>     On Linux, shared memory objects are created in a (tmpfs(5)) virtual
>     filesystem, normally mounted under /dev/shm. [...]
>
> There is in fact nothing wrong in creating such shm objects during the
> build.
>
> What is wrong, is leaving them lingering about...

What happens with two independent buildroot builds both doing netopeer2 install
at the same time? Compete/interfere on the same files in /dev/shm/sh_*? Think
the only proper solution is to avoid the shm files or create them inside the
buildroot working directory (and cleanup afterwards)...

Regards,
Peter

>
> Regards,
> Yann E. MORIN.
>

[Buildroot] [autobuild.buildroot.net] Your daily results for 2021-02-04

[Yann E. MORIN]

Heiko, All,

On 2021-02-05 15:42 +0100, Heiko Thiery spake thusly:
> Am Fr., 5. Feb. 2021 um 09:18 Uhr schrieb Thomas Petazzoni
> <thomas.petazzoni@bootlin.com>:
> >    xtensa    |        netopeer2-1.1.53        | http://autobuild.buildroot.net/results/e21834d4d2ee580f00f0fdcbd3728787148c0da9
> I checked the reason for the build failure on the netopeer2 package.
> It is caused by some files that are created in /dev/shm/sr_* during
> the installation process.

Turns out, the 'sr_' prefix can be customised at runtime:

    $ SYSREPO_SHM_PREFIX=GRRR make netopeer2-reinstall
    $ ls /dev/shm
    /dev/shm/GRRR_ext                                     /dev/shm/GRRR_ietf-ssh-server.operational
    /dev/shm/GRRR_ietf-crypto-types.operational           /dev/shm/GRRR_ietf-ssh-server.running
    [--SNIP--]
    /dev/shm/GRRR_ietf-netconf-nmda.running               /dev/shm/GRRR_main
    [--SNIP--]
    /dev/shm/GRRR_ietf-origin.operational                 /dev/shm/GRRR_yang.operational
    /dev/shm/GRRR_ietf-origin.running                     /dev/shm/GRRR_yang.running

Unfortunately, that can not be used be specify a sub-directory, or an
alternate location...

So, by carefully choosing a prefix, we can at least identify what files
we must remove after the fact, and we can ensure that two concurrent
builds will not use the same files.

For example, totally untested:

    diff --git a/package/netopeer2/netopeer2.mk b/package/netopeer2/netopeer2.mk
    index bc02e0dc93..dd10f76cba 100644
    --- a/package/netopeer2/netopeer2.mk
    +++ b/package/netopeer2/netopeer2.mk
    @@ -13,9 +13,20 @@ NETOPEER2_DEPENDENCIES = libnetconf2 libyang sysrepo
     
     NETOPEER2_CONF_OPTS = -DBUILD_CLI=$(if $(BR2_PACKAGE_NETOPEER2_CLI),ON,OFF)
     
    +NETOPEER2_SHM_PREFIX = sr_buildroot$(subst /,_,$(CONFIG_DIR))_
    +NETOPEER2_MAKE_ENV = SYSREPO_SHM_PREFIX=$(NETOPEER2_SHM_PREFIX)
    +
     define NETOPEER2_INSTALL_INIT_SYSV
     	$(INSTALL) -m 755 -D package/netopeer2/S52netopeer2 \
     		$(TARGET_DIR)/etc/init.d/S52netopeer2
     endef
     
    +# The host sysrepo used to install the netopeer2 modules will leave
    +# its shared memory files lingering about. Clean up in its stead...
    +define NETOPEER2_CLEANUP
    +	rm -f /dev/shm/$(NETOPEER2_SHM_PREFIX)*
    +endef
    +NETOPEER2_PRE_INSTALL_TARGET_HOOKS += NETOPEER2_CLEANUP
    +NETOPEER2_POST_INSTALL_TARGET_HOOKS += NETOPEER2_CLEANUP
    +
     $(eval $(cmake-package))

(note: I hand-edited the patch, so it may be completely unusable as-is
now, but you at least get the idea...)

Regards,
Yann E. MORIN.

> I tried to find a solution for that. My first intention was to do a
> PRE_INSTALL_HOOK that deletes these files before the installation. But
> YANN disclaimed that because we should never delete files in
> /dev/shm/. This could lead to failures when doing concurrent parallel
> builds.
> 
> To be more detailed what is going on:
> The netopeer2 package can install the required yang models for runtime
> during installation. Therefore an additional script (setup.sh) is
> invoked. There the sysrepocfg host tool is used to do the installation
> of these yang models. sysrepo will then create this /dev/shm files and
> leave them. But with the updated netopeer2 package the shm files are
> incompatible and the build errors appear.
> 
> So I see here 3 possible solutions:
> 1. do the PRE_INSTALL_HOOK to remove the files every time (disclaimed by Yann).
> 2. remove this files by hand (no long term solution).
> 3. disable the installation of the yang modules .. but then we have a
> non functional installation available and we leave the installation of
> the yang modules to the user.
> 
> What do you think?
> 
> -- 
> Heiko

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [autobuild.buildroot.net] Your daily results for 2021-02-04

[Heiko Thiery]

Hi Peter,

Am Fr., 5. Feb. 2021 um 23:43 Uhr schrieb Peter Seiderer <ps.report@gmx.net>:
>
> On Fri, 5 Feb 2021 22:34:53 +0100, "Yann E. MORIN" <yann.morin.1998@free.fr> wrote:
>
> > Peter, All,
> >
> > On 2021-02-05 19:35 +0100, Peter Korsgaard spake thusly:
> > > >>>>> "Heiko" == Heiko Thiery <heiko.thiery@gmail.com> writes:
> > >  >> mipsel    |        netopeer2-1.1.53        | http://autobuild.buildroot.net/results/da0e36543cf68e4e9bbe819945a884193f33819a
> > [--SNIP--]
> > >  > So I see here 3 possible solutions:
> > >  > 1. do the PRE_INSTALL_HOOK to remove the files every time (disclaimed by Yann).
> > >  > 2. remove this files by hand (no long term solution).
> > >  > 3. disable the installation of the yang modules .. but then we have a
> > >  > non functional installation available and we leave the installation of
> > >  > the yang modules to the user.
> > >
> > > Ideally the package should be fixed to create those files in
> > > $DESTDIR/dev/shm rather than mess around with the host /dev/shm.
> > >
> > > But as /dev/shm is not persistent, how does this work at runtime? What
> > > do those files do exactly?
> >
> > They are shared memory.
> >
> > From shm_overview(7):
> >
> >     On Linux, shared memory objects are created in a (tmpfs(5)) virtual
> >     filesystem, normally mounted under /dev/shm. [...]
> >
> > There is in fact nothing wrong in creating such shm objects during the
> > build.
> >
> > What is wrong, is leaving them lingering about...
>
> What happens with two independent buildroot builds both doing netopeer2 install
> at the same time? Compete/interfere on the same files in /dev/shm/sh_*? Think
> the only proper solution is to avoid the shm files or create them inside the
> buildroot working directory (and cleanup afterwards)...

This is what Yann also mentioned and this should be fixed with the
proposed fix/workaround from Yann.

Thank you,
Heiko

[Buildroot] [autobuild.buildroot.net] Your daily results for 2021-02-04

[Heiko Thiery]

Hi Peter,

Am Fr., 5. Feb. 2021 um 22:41 Uhr schrieb Peter Korsgaard <peter@korsgaard.com>:
>
> >>>>> "Yann" == Yann E MORIN <yann.morin.1998@free.fr> writes:
>
> Hi,
>
>  >> Ideally the package should be fixed to create those files in
>  >> $DESTDIR/dev/shm rather than mess around with the host /dev/shm.
>  >>
>  >> But as /dev/shm is not persistent, how does this work at runtime? What
>  >> do those files do exactly?
>
>  > They are shared memory.
>
>  > From shm_overview(7):
>
>  >     On Linux, shared memory objects are created in a (tmpfs(5)) virtual
>  >     filesystem, normally mounted under /dev/shm. [...]
>
> That I get.
>
>  > There is in fact nothing wrong in creating such shm objects during the
>  > build.
>
> From the logs it looks like they are created during the installation,
> rather than during the build.
>
>  > What is wrong, is leaving them lingering about...
>
> Indeed. I don't particular understand why the installation step needs to
> fiddle with shared memory objects in the first place, but Ok.

The reason is that sysrepo and sysrepoctl create these files for
handling the modules. Indeed it is not 100% clear to me if these files
are really needed when installing the modules. But I think this by
design of syrepo.

> So these objects are only needed temporarily during installation and not
> later at runtime?

In this case these files are only temporarily needed during the
installation. Later on the target these files are also created for
runtime.

-- 
Heiko

[Buildroot] [autobuild.buildroot.net] Your daily results for 2021-02-04

[Heiko Thiery]

Hi Yann and all,

Am Sa., 6. Feb. 2021 um 00:37 Uhr schrieb Yann E. MORIN
<yann.morin.1998@free.fr>:
>
> Heiko, All,
>
> On 2021-02-05 15:42 +0100, Heiko Thiery spake thusly:
> > Am Fr., 5. Feb. 2021 um 09:18 Uhr schrieb Thomas Petazzoni
> > <thomas.petazzoni@bootlin.com>:
> > >    xtensa    |        netopeer2-1.1.53        | http://autobuild.buildroot.net/results/e21834d4d2ee580f00f0fdcbd3728787148c0da9
> > I checked the reason for the build failure on the netopeer2 package.
> > It is caused by some files that are created in /dev/shm/sr_* during
> > the installation process.
>
> Turns out, the 'sr_' prefix can be customised at runtime:
>
>     $ SYSREPO_SHM_PREFIX=GRRR make netopeer2-reinstall
>     $ ls /dev/shm
>     /dev/shm/GRRR_ext                                     /dev/shm/GRRR_ietf-ssh-server.operational
>     /dev/shm/GRRR_ietf-crypto-types.operational           /dev/shm/GRRR_ietf-ssh-server.running
>     [--SNIP--]
>     /dev/shm/GRRR_ietf-netconf-nmda.running               /dev/shm/GRRR_main
>     [--SNIP--]
>     /dev/shm/GRRR_ietf-origin.operational                 /dev/shm/GRRR_yang.operational
>     /dev/shm/GRRR_ietf-origin.running                     /dev/shm/GRRR_yang.running
>
> Unfortunately, that can not be used be specify a sub-directory, or an
> alternate location...
>
> So, by carefully choosing a prefix, we can at least identify what files
> we must remove after the fact, and we can ensure that two concurrent
> builds will not use the same files.

Great. I was not aware of this possibility. This seems to be the
easiest way to fix the issue from a buildroot point of view.

> For example, totally untested:
>
>     diff --git a/package/netopeer2/netopeer2.mk b/package/netopeer2/netopeer2.mk
>     index bc02e0dc93..dd10f76cba 100644
>     --- a/package/netopeer2/netopeer2.mk
>     +++ b/package/netopeer2/netopeer2.mk
>     @@ -13,9 +13,20 @@ NETOPEER2_DEPENDENCIES = libnetconf2 libyang sysrepo
>
>      NETOPEER2_CONF_OPTS = -DBUILD_CLI=$(if $(BR2_PACKAGE_NETOPEER2_CLI),ON,OFF)
>
>     +NETOPEER2_SHM_PREFIX = sr_buildroot$(subst /,_,$(CONFIG_DIR))_
>     +NETOPEER2_MAKE_ENV = SYSREPO_SHM_PREFIX=$(NETOPEER2_SHM_PREFIX)
>     +
>      define NETOPEER2_INSTALL_INIT_SYSV
>         $(INSTALL) -m 755 -D package/netopeer2/S52netopeer2 \
>                 $(TARGET_DIR)/etc/init.d/S52netopeer2
>      endef
>
>     +# The host sysrepo used to install the netopeer2 modules will leave
>     +# its shared memory files lingering about. Clean up in its stead...
>     +define NETOPEER2_CLEANUP
>     +   rm -f /dev/shm/$(NETOPEER2_SHM_PREFIX)*
>     +endef
>     +NETOPEER2_PRE_INSTALL_TARGET_HOOKS += NETOPEER2_CLEANUP
>     +NETOPEER2_POST_INSTALL_TARGET_HOOKS += NETOPEER2_CLEANUP
>     +
>      $(eval $(cmake-package))
>
> (note: I hand-edited the patch, so it may be completely unusable as-is
> now, but you at least get the idea...)

I used your snipped and (after fixing some small things) it works. So
I will prepare a patch and send it.

By the way ... after that I also do a patch for the host-sysrepo
dependency and add the executable location of sysrepoctl as MAKE_ENV.

Thanks to all for the help!

--
Heiko