[PATCH] rootfs: remove ldconfig auxiliary cache where appropriate

[PATCH] rootfs: remove ldconfig auxiliary cache where appropriate

[Damian Wrobel]

Removes the /var/cache/ldconfig auxiliary cache directory from
the rootfs when:
 - read-only-rootfs is in DISTRO_FEATURES,
 - ldconfig is not in DISTRO_FEATURES.

In both cases the /var/cache/ldconfig/aux-cache is useless.

Signed-off-by: Damian Wrobel <dwrobel@ertelnet.rybnik.pl>
---
 meta/lib/oe/rootfs.py | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/meta/lib/oe/rootfs.py b/meta/lib/oe/rootfs.py
index ad9fd77c8b..80673fab3f 100644
--- a/meta/lib/oe/rootfs.py
+++ b/meta/lib/oe/rootfs.py
@@ -302,6 +302,15 @@ class Rootfs(object, metaclass=ABCMeta):
             self._exec_shell_cmd(['ldconfig', '-r', self.image_rootfs, '-c',
                                   'new', '-v', '-X'])
 
+        image_rorfs = bb.utils.contains("IMAGE_FEATURES", "read-only-rootfs",
+                                        True, False, self.d)
+        ldconfig_in_features = bb.utils.contains("IMAGE_FEATURES", "ldconfig",
+                                                 True, False, self.d)
+        if image_rorfs or not ldconfig_in_features:
+            ldconfig_cache_dir = os.path.join(self.image_rootfs, "var/cache/ldconfig")
+            bb.note("Removing ldconfig auxiliary cache...")
+            shutil.rmtree(ldconfig_cache_dir)
+
     def _check_for_kernel_modules(self, modules_dir):
         for root, dirs, files in os.walk(modules_dir, topdown=True):
             for name in files:
-- 
2.31.1

Re: [OE-core] [PATCH] rootfs: remove ldconfig auxiliary cache where appropriate

[Andre McCurdy]

On Thu, Jul 22, 2021 at 2:48 AM Damian Wrobel
<dwrobel@ertelnet.rybnik.pl> wrote:
>
> Removes the /var/cache/ldconfig auxiliary cache directory from
> the rootfs when:
>  - read-only-rootfs is in DISTRO_FEATURES,
>  - ldconfig is not in DISTRO_FEATURES.
>
> In both cases the /var/cache/ldconfig/aux-cache is useless.

Should there be a test for package management being supported in the
image too? A read-only rootfs is one case where it's not possible to
install ldconfig support at run time but a writeable rootfs without
package management support would seem to be another?

> Signed-off-by: Damian Wrobel <dwrobel@ertelnet.rybnik.pl>
> ---
>  meta/lib/oe/rootfs.py | 9 +++++++++
>  1 file changed, 9 insertions(+)
>
> diff --git a/meta/lib/oe/rootfs.py b/meta/lib/oe/rootfs.py
> index ad9fd77c8b..80673fab3f 100644
> --- a/meta/lib/oe/rootfs.py
> +++ b/meta/lib/oe/rootfs.py
> @@ -302,6 +302,15 @@ class Rootfs(object, metaclass=ABCMeta):
>              self._exec_shell_cmd(['ldconfig', '-r', self.image_rootfs, '-c',
>                                    'new', '-v', '-X'])
>
> +        image_rorfs = bb.utils.contains("IMAGE_FEATURES", "read-only-rootfs",
> +                                        True, False, self.d)
> +        ldconfig_in_features = bb.utils.contains("IMAGE_FEATURES", "ldconfig",
> +                                                 True, False, self.d)
> +        if image_rorfs or not ldconfig_in_features:
> +            ldconfig_cache_dir = os.path.join(self.image_rootfs, "var/cache/ldconfig")
> +            bb.note("Removing ldconfig auxiliary cache...")
> +            shutil.rmtree(ldconfig_cache_dir)
> +
>      def _check_for_kernel_modules(self, modules_dir):
>          for root, dirs, files in os.walk(modules_dir, topdown=True):
>              for name in files:
> --
> 2.31.1
>
>
> 
>

Re: [OE-core] [PATCH] rootfs: remove ldconfig auxiliary cache where appropriate

[Alexandre Belloni]

On 22/07/2021 11:48:22+0200, Damian Wrobel wrote:
> Removes the /var/cache/ldconfig auxiliary cache directory from
> the rootfs when:
>  - read-only-rootfs is in DISTRO_FEATURES,
>  - ldconfig is not in DISTRO_FEATURES.
> 
> In both cases the /var/cache/ldconfig/aux-cache is useless.
> 

This breaks the do_rootfs task of most of the builds with:

Exception: FileNotFoundError: [Errno 2] No such file or directory: '/home/pokybuild/yocto-worker/qemuarm64-ltp/build/build/tmp/work/qemuarm64-poky-linux/core-image-sato/1.0-r0/rootfs/var/cache/ldconfig'
ERROR: Logfile of failure stored in: /home/pokybuild/yocto-worker/qemuarm64-ltp/build/build/tmp/work/qemuarm64-poky-linux/core-image-sato/1.0-r0/temp/log.do_rootfs.51846
NOTE: recipe core-image-sato-1.0-r0: task do_rootfs: Failed
ERROR: Task (/home/pokybuild/yocto-worker/qemuarm64-ltp/build/meta/recipes-sato/images/core-image-sato.bb:do_rootfs) failed with exit code '1'

Example here:
https://autobuilder.yoctoproject.org/typhoon/#/builders/96/builds/1983/steps/11/logs/stdio

> Signed-off-by: Damian Wrobel <dwrobel@ertelnet.rybnik.pl>
> ---
>  meta/lib/oe/rootfs.py | 9 +++++++++
>  1 file changed, 9 insertions(+)
> 
> diff --git a/meta/lib/oe/rootfs.py b/meta/lib/oe/rootfs.py
> index ad9fd77c8b..80673fab3f 100644
> --- a/meta/lib/oe/rootfs.py
> +++ b/meta/lib/oe/rootfs.py
> @@ -302,6 +302,15 @@ class Rootfs(object, metaclass=ABCMeta):
>              self._exec_shell_cmd(['ldconfig', '-r', self.image_rootfs, '-c',
>                                    'new', '-v', '-X'])
>  
> +        image_rorfs = bb.utils.contains("IMAGE_FEATURES", "read-only-rootfs",
> +                                        True, False, self.d)
> +        ldconfig_in_features = bb.utils.contains("IMAGE_FEATURES", "ldconfig",
> +                                                 True, False, self.d)
> +        if image_rorfs or not ldconfig_in_features:
> +            ldconfig_cache_dir = os.path.join(self.image_rootfs, "var/cache/ldconfig")
> +            bb.note("Removing ldconfig auxiliary cache...")
> +            shutil.rmtree(ldconfig_cache_dir)
> +
>      def _check_for_kernel_modules(self, modules_dir):
>          for root, dirs, files in os.walk(modules_dir, topdown=True):
>              for name in files:
> -- 
> 2.31.1
> 

> 
> 
> 


-- 
Alexandre Belloni, co-owner and COO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

Re: [OE-core] [PATCH] rootfs: remove ldconfig auxiliary cache where appropriate

[Damian Wrobel]

 ---- On Thu, 22 Jul 2021 21:00:15 +0200 Andre McCurdy <armccurdy@gmail.com> wrote ----
 > On Thu, Jul 22, 2021 at 2:48 AM Damian Wrobel
 > <dwrobel@ertelnet.rybnik.pl> wrote:
 > >
 > > Removes the /var/cache/ldconfig auxiliary cache directory from
 > > the rootfs when:
 > >  - read-only-rootfs is in DISTRO_FEATURES,
 > >  - ldconfig is not in DISTRO_FEATURES.
 > >
 > > In both cases the /var/cache/ldconfig/aux-cache is useless.
 > 
 > Should there be a test for package management being supported in the
 > image too? A read-only rootfs is one case where it's not possible to
 > install ldconfig support at run time but a writeable rootfs without
 > package management support would seem to be another?

It seems to be perfectly fine to use the ldconfig without having any package management system.

--
Regards,
Damian

 > 
 > > Signed-off-by: Damian Wrobel <dwrobel@ertelnet.rybnik.pl>
 > > ---
 > >  meta/lib/oe/rootfs.py | 9 +++++++++
 > >  1 file changed, 9 insertions(+)
 > >
 > > diff --git a/meta/lib/oe/rootfs.py b/meta/lib/oe/rootfs.py
 > > index ad9fd77c8b..80673fab3f 100644
 > > --- a/meta/lib/oe/rootfs.py
 > > +++ b/meta/lib/oe/rootfs.py
 > > @@ -302,6 +302,15 @@ class Rootfs(object, metaclass=ABCMeta):
 > >              self._exec_shell_cmd(['ldconfig', '-r', self.image_rootfs, '-c',
 > >                                    'new', '-v', '-X'])
 > >
 > > +        image_rorfs = bb.utils.contains("IMAGE_FEATURES", "read-only-rootfs",
 > > +                                        True, False, self.d)
 > > +        ldconfig_in_features = bb.utils.contains("IMAGE_FEATURES", "ldconfig",
 > > +                                                 True, False, self.d)
 > > +        if image_rorfs or not ldconfig_in_features:
 > > +            ldconfig_cache_dir = os.path.join(self.image_rootfs, "var/cache/ldconfig")
 > > +            bb.note("Removing ldconfig auxiliary cache...")
 > > +            shutil.rmtree(ldconfig_cache_dir)
 > > +
 > >      def _check_for_kernel_modules(self, modules_dir):
 > >          for root, dirs, files in os.walk(modules_dir, topdown=True):
 > >              for name in files:
 > > --
 > > 2.31.1
 > >
 > >
 > > 
 > >
 > 

Re: [OE-core] [PATCH] rootfs: remove ldconfig auxiliary cache where appropriate

[Damian Wrobel]

 ---- On Fri, 23 Jul 2021 00:37:08 +0200 Alexandre Belloni <alexandre.belloni@bootlin.com> wrote ----
 > On 22/07/2021 11:48:22+0200, Damian Wrobel wrote:
 > > Removes the /var/cache/ldconfig auxiliary cache directory from
 > > the rootfs when:
 > >  - read-only-rootfs is in DISTRO_FEATURES,
 > >  - ldconfig is not in DISTRO_FEATURES.
 > > 
 > > In both cases the /var/cache/ldconfig/aux-cache is useless.
 > > 
 > 
 > This breaks the do_rootfs task of most of the builds with:

Thank you for the feedback.
I will send v2 with an additional check and remove the directory only if it exists.

--
Regards,
Damian
 > 
 > Exception: FileNotFoundError: [Errno 2] No such file or directory: '/home/pokybuild/yocto-worker/qemuarm64-ltp/build/build/tmp/work/qemuarm64-poky-linux/core-image-sato/1.0-r0/rootfs/var/cache/ldconfig'
 > ERROR: Logfile of failure stored in: /home/pokybuild/yocto-worker/qemuarm64-ltp/build/build/tmp/work/qemuarm64-poky-linux/core-image-sato/1.0-r0/temp/log.do_rootfs.51846
 > NOTE: recipe core-image-sato-1.0-r0: task do_rootfs: Failed
 > ERROR: Task (/home/pokybuild/yocto-worker/qemuarm64-ltp/build/meta/recipes-sato/images/core-image-sato.bb:do_rootfs) failed with exit code '1'
 > 
 > Example here:
 > https://autobuilder.yoctoproject.org/typhoon/#/builders/96/builds/1983/steps/11/logs/stdio
 > 
 > > Signed-off-by: Damian Wrobel <dwrobel@ertelnet.rybnik.pl>
 > > ---
 > >  meta/lib/oe/rootfs.py | 9 +++++++++
 > >  1 file changed, 9 insertions(+)
 > > 
 > > diff --git a/meta/lib/oe/rootfs.py b/meta/lib/oe/rootfs.py
 > > index ad9fd77c8b..80673fab3f 100644
 > > --- a/meta/lib/oe/rootfs.py
 > > +++ b/meta/lib/oe/rootfs.py
 > > @@ -302,6 +302,15 @@ class Rootfs(object, metaclass=ABCMeta):
 > >              self._exec_shell_cmd(['ldconfig', '-r', self.image_rootfs, '-c',
 > >                                    'new', '-v', '-X'])
 > >  
 > > +        image_rorfs = bb.utils.contains("IMAGE_FEATURES", "read-only-rootfs",
 > > +                                        True, False, self.d)
 > > +        ldconfig_in_features = bb.utils.contains("IMAGE_FEATURES", "ldconfig",
 > > +                                                 True, False, self.d)
 > > +        if image_rorfs or not ldconfig_in_features:
 > > +            ldconfig_cache_dir = os.path.join(self.image_rootfs, "var/cache/ldconfig")
 > > +            bb.note("Removing ldconfig auxiliary cache...")
 > > +            shutil.rmtree(ldconfig_cache_dir)
 > > +
 > >      def _check_for_kernel_modules(self, modules_dir):
 > >          for root, dirs, files in os.walk(modules_dir, topdown=True):
 > >              for name in files:
 > > -- 
 > > 2.31.1
 > > 
 > 
 > > 
 > > 
 > > 
 > 
 > 
 > -- 
 > Alexandre Belloni, co-owner and COO, Bootlin
 > Embedded Linux and Kernel engineering
 > https://bootlin.com
 > 
 > 
 > 
 > 

[PATCH v2] rootfs: remove ldconfig auxiliary cache where appropriate

[Damian Wrobel]

Removes the /var/cache/ldconfig auxiliary cache directory from
the rootfs when:
 - read-only-rootfs is in DISTRO_FEATURES,
 - ldconfig is not in DISTRO_FEATURES.

In both cases the /var/cache/ldconfig/aux-cache is useless.

Signed-off-by: Damian Wrobel <dwrobel@ertelnet.rybnik.pl>
---
 meta/lib/oe/rootfs.py | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/meta/lib/oe/rootfs.py b/meta/lib/oe/rootfs.py
index ad9fd77c8b..8e63ebac9b 100644
--- a/meta/lib/oe/rootfs.py
+++ b/meta/lib/oe/rootfs.py
@@ -302,6 +302,16 @@ class Rootfs(object, metaclass=ABCMeta):
             self._exec_shell_cmd(['ldconfig', '-r', self.image_rootfs, '-c',
                                   'new', '-v', '-X'])
 
+        image_rorfs = bb.utils.contains("IMAGE_FEATURES", "read-only-rootfs",
+                                        True, False, self.d)
+        ldconfig_in_features = bb.utils.contains("IMAGE_FEATURES", "ldconfig",
+                                                 True, False, self.d)
+        if image_rorfs or not ldconfig_in_features:
+            ldconfig_cache_dir = os.path.join(self.image_rootfs, "var/cache/ldconfig")
+            if os.path.exists(ldconfig_cache_dir):
+                bb.note("Removing ldconfig auxiliary cache...")
+                shutil.rmtree(ldconfig_cache_dir)
+
     def _check_for_kernel_modules(self, modules_dir):
         for root, dirs, files in os.walk(modules_dir, topdown=True):
             for name in files:
-- 
2.31.1

Re: [OE-core] [PATCH] rootfs: remove ldconfig auxiliary cache where appropriate

[Andre McCurdy]

On Fri, Jul 23, 2021 at 2:16 AM Damian Wrobel
<dwrobel@ertelnet.rybnik.pl> wrote:
>
>  ---- On Thu, 22 Jul 2021 21:00:15 +0200 Andre McCurdy <armccurdy@gmail.com> wrote ----
>  > On Thu, Jul 22, 2021 at 2:48 AM Damian Wrobel
>  > <dwrobel@ertelnet.rybnik.pl> wrote:
>  > >
>  > > Removes the /var/cache/ldconfig auxiliary cache directory from
>  > > the rootfs when:
>  > >  - read-only-rootfs is in DISTRO_FEATURES,
>  > >  - ldconfig is not in DISTRO_FEATURES.
>  > >
>  > > In both cases the /var/cache/ldconfig/aux-cache is useless.
>  >
>  > Should there be a test for package management being supported in the
>  > image too? A read-only rootfs is one case where it's not possible to
>  > install ldconfig support at run time but a writeable rootfs without
>  > package management support would seem to be another?
>
> It seems to be perfectly fine to use the ldconfig without having any package management system.

Of course. But if ldconfig support is disabled at build time (ie the
ldconfig distro feature is disabled) and there is no package
management support (so ldconfig can not be installed at run time) then
is there any need for /var/cache/ldconfig/aux-cache ?

Re: [OE-core] [PATCH v2] rootfs: remove ldconfig auxiliary cache where appropriate

[Andre McCurdy]

On Fri, Jul 23, 2021 at 2:22 AM Damian Wrobel
<dwrobel@ertelnet.rybnik.pl> wrote:
>
> Removes the /var/cache/ldconfig auxiliary cache directory from
> the rootfs when:
>  - read-only-rootfs is in DISTRO_FEATURES,
>  - ldconfig is not in DISTRO_FEATURES.

Comments say DISTRO_FEATURES but code is checking IMAGE_FEATURES in both cases.

ldconfig is a distro feature and read-only-rootfs is an image feature,
so both the comments and code seem to be wrong...

> In both cases the /var/cache/ldconfig/aux-cache is useless.
>
> Signed-off-by: Damian Wrobel <dwrobel@ertelnet.rybnik.pl>
> ---
>  meta/lib/oe/rootfs.py | 10 ++++++++++
>  1 file changed, 10 insertions(+)
>
> diff --git a/meta/lib/oe/rootfs.py b/meta/lib/oe/rootfs.py
> index ad9fd77c8b..8e63ebac9b 100644
> --- a/meta/lib/oe/rootfs.py
> +++ b/meta/lib/oe/rootfs.py
> @@ -302,6 +302,16 @@ class Rootfs(object, metaclass=ABCMeta):
>              self._exec_shell_cmd(['ldconfig', '-r', self.image_rootfs, '-c',
>                                    'new', '-v', '-X'])
>
> +        image_rorfs = bb.utils.contains("IMAGE_FEATURES", "read-only-rootfs",
> +                                        True, False, self.d)
> +        ldconfig_in_features = bb.utils.contains("IMAGE_FEATURES", "ldconfig",
> +                                                 True, False, self.d)
> +        if image_rorfs or not ldconfig_in_features:
> +            ldconfig_cache_dir = os.path.join(self.image_rootfs, "var/cache/ldconfig")
> +            if os.path.exists(ldconfig_cache_dir):
> +                bb.note("Removing ldconfig auxiliary cache...")
> +                shutil.rmtree(ldconfig_cache_dir)
> +
>      def _check_for_kernel_modules(self, modules_dir):
>          for root, dirs, files in os.walk(modules_dir, topdown=True):
>              for name in files:
> --
> 2.31.1
>
>
> 
>

Re: [OE-core] [PATCH v2] rootfs: remove ldconfig auxiliary cache where appropriate

[Damian Wrobel]

 ---- On Fri, 23 Jul 2021 20:21:28 +0200 Andre McCurdy <armccurdy@gmail.com> wrote ----
 > On Fri, Jul 23, 2021 at 2:22 AM Damian Wrobel
 > <dwrobel@ertelnet.rybnik.pl> wrote:
 > >
 > > Removes the /var/cache/ldconfig auxiliary cache directory from
 > > the rootfs when:
 > >  - read-only-rootfs is in DISTRO_FEATURES,
 > >  - ldconfig is not in DISTRO_FEATURES.
 > 
 > Comments say DISTRO_FEATURES but code is checking IMAGE_FEATURES in both cases.
 > 
 > ldconfig is a distro feature and read-only-rootfs is an image feature,
 > so both the comments and code seem to be wrong...
Copy-paste side effect. Thanks for pointing this out. Fixed in v3.

--
Regards,
Damian
 > 
 > > In both cases the /var/cache/ldconfig/aux-cache is useless.
 > >
 > > Signed-off-by: Damian Wrobel <dwrobel@ertelnet.rybnik.pl>
 > > ---
 > >  meta/lib/oe/rootfs.py | 10 ++++++++++
 > >  1 file changed, 10 insertions(+)
 > >
 > > diff --git a/meta/lib/oe/rootfs.py b/meta/lib/oe/rootfs.py
 > > index ad9fd77c8b..8e63ebac9b 100644
 > > --- a/meta/lib/oe/rootfs.py
 > > +++ b/meta/lib/oe/rootfs.py
 > > @@ -302,6 +302,16 @@ class Rootfs(object, metaclass=ABCMeta):
 > >              self._exec_shell_cmd(['ldconfig', '-r', self.image_rootfs, '-c',
 > >                                    'new', '-v', '-X'])
 > >
 > > +        image_rorfs = bb.utils.contains("IMAGE_FEATURES", "read-only-rootfs",
 > > +                                        True, False, self.d)
 > > +        ldconfig_in_features = bb.utils.contains("IMAGE_FEATURES", "ldconfig",
 > > +                                                 True, False, self.d)
 > > +        if image_rorfs or not ldconfig_in_features:
 > > +            ldconfig_cache_dir = os.path.join(self.image_rootfs, "var/cache/ldconfig")
 > > +            if os.path.exists(ldconfig_cache_dir):
 > > +                bb.note("Removing ldconfig auxiliary cache...")
 > > +                shutil.rmtree(ldconfig_cache_dir)
 > > +
 > >      def _check_for_kernel_modules(self, modules_dir):
 > >          for root, dirs, files in os.walk(modules_dir, topdown=True):
 > >              for name in files:
 > > --
 > > 2.31.1
 > >
 > >
 > > 
 > >
 > 
 > 
 > 
 > 

[PATCH v3] rootfs: remove ldconfig auxiliary cache where appropriate

[Damian Wrobel]

Removes the /var/cache/ldconfig auxiliary cache directory from
the rootfs when:
 - read-only-rootfs is in IMAGE_FEATURES,
 - ldconfig is not in DISTRO_FEATURES.

In both cases the /var/cache/ldconfig/aux-cache is useless.

Signed-off-by: Damian Wrobel <dwrobel@ertelnet.rybnik.pl>
---
 meta/lib/oe/rootfs.py | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/meta/lib/oe/rootfs.py b/meta/lib/oe/rootfs.py
index ad9fd77c8b..4ea5adb9f4 100644
--- a/meta/lib/oe/rootfs.py
+++ b/meta/lib/oe/rootfs.py
@@ -302,6 +302,16 @@ class Rootfs(object, metaclass=ABCMeta):
             self._exec_shell_cmd(['ldconfig', '-r', self.image_rootfs, '-c',
                                   'new', '-v', '-X'])
 
+        image_rorfs = bb.utils.contains("IMAGE_FEATURES", "read-only-rootfs",
+                                        True, False, self.d)
+        ldconfig_in_features = bb.utils.contains("DISTRO_FEATURES", "ldconfig",
+                                                 True, False, self.d)
+        if image_rorfs or not ldconfig_in_features:
+            ldconfig_cache_dir = os.path.join(self.image_rootfs, "var/cache/ldconfig")
+            if os.path.exists(ldconfig_cache_dir):
+                bb.note("Removing ldconfig auxiliary cache...")
+                shutil.rmtree(ldconfig_cache_dir)
+
     def _check_for_kernel_modules(self, modules_dir):
         for root, dirs, files in os.walk(modules_dir, topdown=True):
             for name in files:
-- 
2.31.1