[OE-core][PATCH 0/1] wic: warn on usage of Y2038 affected file systems

[OE-core][PATCH 0/1] wic: warn on usage of Y2038 affected file systems

[florian.bezdeka]

From: Florian Bezdeka <florian.bezdeka@siemens.com>

The following patch is the summary of a nice journey through the file 
system jungle regarding Y2038 problem. It all began with a warning which 
is reported by kernels >= 5.4:

ext4 filesystem being mounted at (mountpoint) supports timestamps until
2038 (0x7fffffff)

When reading this warning I was surprised. Shouldn't a modern file
system like ext4 be Y2038-safe? As it turned out it depends on the
inode size if an ext4 file system is safe or not. So why was the
inode size not sufficient in my case?

The inode size is chosen during file system generation and depends on
the size of the file system that is going to be created. For details
let's have a look at `man mke2fs`:

-T usage-type[,...]
    Specify how the filesystem is going to be used, so that mke2fs can
    choose optimal filesystem parameters for that use. The usage types
    that are supported are defined in the configuration file
    /etc/mke2fs.conf. The user may specify one or more usage types
    using a comma separated list.

    If this option is is not specified, mke2fs will pick a single
    default usage type based on the size of the filesystem to be
    created. If the filesystem size is less than 3 megabytes, mke2fs
    will use the filesystem type floppy. If the filesystem size is
    greater than or equal to 3 but less than 512 megabytes, mke2fs(8)
    will use the filesystem type small.

The relevant parts from /etc/mke2fs.conf:
[fs_types]
...
        small = {
                blocksize = 1024
                inode_size = 128
                inode_ratio = 4096
        }
...

So whenever you create an ext4 file system with less than 512MB in
size you will end up with 128 byte inodes and your file system is
not Y2038-safe.

Some words to the other filesystems supported by wic:
 - ext2 and ext3 will overflow in 2038, 
   The user will be warned as well
 - squashfs will overflow in 2106
 - btrfs uses u64 for time representations, so Y2038 safe

Florian Bezdeka (1):
  wic: Warn if an ext filesystem affected by the Y2038 problem is used

 scripts/lib/wic/misc.py      |  1 +
 scripts/lib/wic/partition.py | 38 ++++++++++++++++++++++++++++++++++++
 2 files changed, 39 insertions(+)

-- 
2.29.2

[OE-core][PATCH 1/1] wic: Warn if an ext filesystem affected by the Y2038 problem is used

[florian.bezdeka]

From: Florian Bezdeka <florian.bezdeka@siemens.com>

We are getting closer and closer to the year 2038 where the 32 bit
time_t overflow will happen. While products (= embedded systems) with an
expected life time of 15 years are still save the situation may change
if your system has to survive the next 20 years.

ext2 and ext3 file systems are always affected by the time overflow, so
let's warn the user if these file systems are still being used.

If ext4 is affected depends on the inode size chosen during file system
creation. At least 256 bytes are necessary to be safe. As ext4 is
used very often (and partitions may be created small first and extended 
later) this might be an issue for many users.

Signed-off-by: Florian Bezdeka <florian.bezdeka@siemens.com>
---
 scripts/lib/wic/misc.py      |  1 +
 scripts/lib/wic/partition.py | 38 ++++++++++++++++++++++++++++++++++++
 2 files changed, 39 insertions(+)

diff --git a/scripts/lib/wic/misc.py b/scripts/lib/wic/misc.py
index 75b219cd3f..57c042c503 100644
--- a/scripts/lib/wic/misc.py
+++ b/scripts/lib/wic/misc.py
@@ -26,6 +26,7 @@ logger = logging.getLogger('wic')
 
 # executable -> recipe pairs for exec_native_cmd
 NATIVE_RECIPES = {"bmaptool": "bmap-tools",
+                  "dumpe2fs": "e2fsprogs",
                   "grub-mkimage": "grub-efi",
                   "isohybrid": "syslinux",
                   "mcopy": "mtools",
diff --git a/scripts/lib/wic/partition.py b/scripts/lib/wic/partition.py
index e574f40c47..85f9847047 100644
--- a/scripts/lib/wic/partition.py
+++ b/scripts/lib/wic/partition.py
@@ -298,6 +298,8 @@ class Partition():
         mkfs_cmd = "fsck.%s -pvfD %s" % (self.fstype, rootfs)
         exec_native_cmd(mkfs_cmd, native_sysroot, pseudo=pseudo)
 
+        self.check_for_Y2038_problem(rootfs, native_sysroot)
+
     def prepare_rootfs_btrfs(self, rootfs, cr_workdir, oe_builddir, rootfs_dir,
                              native_sysroot, pseudo):
         """
@@ -388,6 +390,8 @@ class Partition():
             (self.fstype, extraopts, label_str, self.fsuuid, rootfs)
         exec_native_cmd(mkfs_cmd, native_sysroot)
 
+        self.check_for_Y2038_problem(rootfs, native_sysroot)
+
     def prepare_empty_partition_btrfs(self, rootfs, oe_builddir,
                                       native_sysroot):
         """
@@ -449,3 +453,37 @@ class Partition():
 
         mkswap_cmd = "mkswap %s -U %s %s" % (label_str, self.fsuuid, path)
         exec_native_cmd(mkswap_cmd, native_sysroot)
+
+    def check_for_Y2038_problem(self, rootfs, native_sysroot):
+        """
+        Check if the filesystem is affected by the Y2038 problem
+        (Y2038 problem = 32 bit time_t overflow in January 2038)
+        """
+        def get_err_str(part):
+            err = "The {} filesystem {} has no Y2038 support."
+            if part.mountpoint:
+                args = [part.fstype, "mounted at %s" % part.mountpoint]
+            elif part.label:
+                args = [part.fstype, "labeled '%s'" % part.label]
+            elif part.part_name:
+                args = [part.fstype, "in partition '%s'" % part.part_name]
+            else:
+                args = [part.fstype, "in partition %s" % part.num]
+            return err.format(*args)
+
+        # ext2 and ext3 are always affected by the Y2038 problem
+        if self.fstype in ["ext2", "ext3"]:
+            logger.warn(get_err_str(self))
+            return
+
+        ret, out = exec_native_cmd("dumpe2fs %s" % rootfs, native_sysroot)
+
+        # if ext4 is affected by the Y2038 problem depends on the inode size
+        for line in out.splitlines():
+            if line.startswith("Inode size:"):
+                size = int(line.split(":")[1].strip())
+                if size < 256:
+                    logger.warn("%s Inodes (of size %d) are too small." %
+                                (get_err_str(self), size))
+                break
+
-- 
2.29.2

Re: [OE-core][PATCH 0/1] wic: warn on usage of Y2038 affected file systems

[Richard Purdie]

On Thu, 2021-02-18 at 13:06 +0000, florian.bezdeka@siemens.com wrote:
> From: Florian Bezdeka <florian.bezdeka@siemens.com>
> 
> The following patch is the summary of a nice journey through the file 
> system jungle regarding Y2038 problem. It all began with a warning which 
> is reported by kernels >= 5.4:
> 
> ext4 filesystem being mounted at (mountpoint) supports timestamps until
> 2038 (0x7fffffff)
> 
> When reading this warning I was surprised. Shouldn't a modern file
> system like ext4 be Y2038-safe? As it turned out it depends on the
> inode size if an ext4 file system is safe or not. So why was the
> inode size not sufficient in my case?
> 
> The inode size is chosen during file system generation and depends on
> the size of the file system that is going to be created. For details
> let's have a look at `man mke2fs`:
> 
> -T usage-type[,...]
>     Specify how the filesystem is going to be used, so that mke2fs can
>     choose optimal filesystem parameters for that use. The usage types
>     that are supported are defined in the configuration file
>     /etc/mke2fs.conf. The user may specify one or more usage types
>     using a comma separated list.
> 
>     If this option is is not specified, mke2fs will pick a single
>     default usage type based on the size of the filesystem to be
>     created. If the filesystem size is less than 3 megabytes, mke2fs
>     will use the filesystem type floppy. If the filesystem size is
>     greater than or equal to 3 but less than 512 megabytes, mke2fs(8)
>     will use the filesystem type small.
> 
> The relevant parts from /etc/mke2fs.conf:
> [fs_types]
> ...
>         small = {
>                 blocksize = 1024
>                 inode_size = 128
>                 inode_ratio = 4096
>         }
> ...
> 
> So whenever you create an ext4 file system with less than 512MB in
> size you will end up with 128 byte inodes and your file system is
> not Y2038-safe.
> 
> Some words to the other filesystems supported by wic:
>  - ext2 and ext3 will overflow in 2038, 
>    The user will be warned as well
>  - squashfs will overflow in 2106
>  - btrfs uses u64 for time representations, so Y2038 safe


Thanks, this is probably something we need to do. I did apply it in the 
test builds and they do show a few warnings:

https://autobuilder.yoctoproject.org/typhoon/#/builders/58/builds/3052

Before we could merge it, we really need to fix this. Are there parameters
we should add to our default ext4 to make it Y2038 safe and avoid the 
warnings?

Cheers,

Richard

Re: [OE-core][PATCH 0/1] wic: warn on usage of Y2038 affected file systems

[florian.bezdeka]

Hi Richard,

On Fri, 2021-02-19 at 10:55 +0000, Richard Purdie wrote:
> On Thu, 2021-02-18 at 13:06 +0000, florian.bezdeka@siemens.com wrote:
> > From: Florian Bezdeka <florian.bezdeka@siemens.com>
> > 
> > The following patch is the summary of a nice journey through the file 
> > system jungle regarding Y2038 problem. It all began with a warning which 
> > is reported by kernels >= 5.4:
> > 
> > ext4 filesystem being mounted at (mountpoint) supports timestamps until
> > 2038 (0x7fffffff)
> > 
> > When reading this warning I was surprised. Shouldn't a modern file
> > system like ext4 be Y2038-safe? As it turned out it depends on the
> > inode size if an ext4 file system is safe or not. So why was the
> > inode size not sufficient in my case?
> > 
> > The inode size is chosen during file system generation and depends on
> > the size of the file system that is going to be created. For details
> > let's have a look at `man mke2fs`:
> > 
> > -T usage-type[,...]
> >     Specify how the filesystem is going to be used, so that mke2fs can
> >     choose optimal filesystem parameters for that use. The usage types
> >     that are supported are defined in the configuration file
> >     /etc/mke2fs.conf. The user may specify one or more usage types
> >     using a comma separated list.
> > 
> >     If this option is is not specified, mke2fs will pick a single
> >     default usage type based on the size of the filesystem to be
> >     created. If the filesystem size is less than 3 megabytes, mke2fs
> >     will use the filesystem type floppy. If the filesystem size is
> >     greater than or equal to 3 but less than 512 megabytes, mke2fs(8)
> >     will use the filesystem type small.
> > 
> > The relevant parts from /etc/mke2fs.conf:
> > [fs_types]
> > ...
> >         small = {
> >                 blocksize = 1024
> >                 inode_size = 128
> >                 inode_ratio = 4096
> >         }
> > ...
> > 
> > So whenever you create an ext4 file system with less than 512MB in
> > size you will end up with 128 byte inodes and your file system is
> > not Y2038-safe.
> > 
> > Some words to the other filesystems supported by wic:
> >  - ext2 and ext3 will overflow in 2038, 
> >    The user will be warned as well
> >  - squashfs will overflow in 2106
> >  - btrfs uses u64 for time representations, so Y2038 safe
> 
> 
> Thanks, this is probably something we need to do. I did apply it in the 
> test builds and they do show a few warnings:
> 
(sorry for breaking the link, but hopefully our infrastructure is being
fixed soon)
> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fautobuilder.yoctoproject.org%2Ftyphoon%2F%23%2Fbuilders%2F58%2Fbuilds%2F3052&amp;data=04%7C01%7Cflorian.bezdeka%40siemens.com%7C35ed532b2cf14a3fb99e08d8d4c4eae5%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637493289505466921%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=1fZIR%2BFOFjTL%2BXaIjZJAVgg6x2YcV6UN6te1Akuvj%2FI%3D&amp;reserved=0
> 
> Before we could merge it, we really need to fix this. Are there parameters
> we should add to our default ext4 to make it Y2038 safe and avoid the 
> warnings?

Sorry for the warnings in the CI system. I didn't notice the .wks file
that is used for that test, but that is actually doing what I wanted to
achieve: Warn users that there system is affected by Y2038 problem.

There are a couple of possibilities now:
  - Adding `--mkfs-extraopts "-t ext4"` to the affected partition
    This tells mke2fs to use the parameters used for "normal" ext4 file
    systems instead of the "optimizations" for "small" ones. (There
    was some additional information in the cover letter)

  - Adding `--mkfs-extraopts "-I 256"` to tell mke2fs to use the bigger
    inodes.

I decided to not apply one of this options per default. Depending on
the target system and use case there might be (performance) side
effects.

Should I come up with a v2 and adding one of the mentioned options to
the .wks file? In this case both of them are possible. Or should I just
sent out an additional patch?

Best regards,
Florian
> 
> Cheers,
> 
> Richard
> 
> 

Re: [OE-core][PATCH 0/1] wic: warn on usage of Y2038 affected file systems

[Richard Purdie]

On Fri, 2021-02-19 at 11:19 +0000, florian.bezdeka@siemens.com wrote:
> Hi Richard,
> 
> On Fri, 2021-02-19 at 10:55 +0000, Richard Purdie wrote:
> > On Thu, 2021-02-18 at 13:06 +0000, florian.bezdeka@siemens.com wrote:
> > > From: Florian Bezdeka <florian.bezdeka@siemens.com>
> > > 
> > > The following patch is the summary of a nice journey through the file 
> > > system jungle regarding Y2038 problem. It all began with a warning which 
> > > is reported by kernels >= 5.4:
> > > 
> > > ext4 filesystem being mounted at (mountpoint) supports timestamps until
> > > 2038 (0x7fffffff)
> > > 
> > > When reading this warning I was surprised. Shouldn't a modern file
> > > system like ext4 be Y2038-safe? As it turned out it depends on the
> > > inode size if an ext4 file system is safe or not. So why was the
> > > inode size not sufficient in my case?
> > > 
> > > The inode size is chosen during file system generation and depends on
> > > the size of the file system that is going to be created. For details
> > > let's have a look at `man mke2fs`:
> > > 
> > > -T usage-type[,...]
> > >     Specify how the filesystem is going to be used, so that mke2fs can
> > >     choose optimal filesystem parameters for that use. The usage types
> > >     that are supported are defined in the configuration file
> > >     /etc/mke2fs.conf. The user may specify one or more usage types
> > >     using a comma separated list.
> > > 
> > >     If this option is is not specified, mke2fs will pick a single
> > >     default usage type based on the size of the filesystem to be
> > >     created. If the filesystem size is less than 3 megabytes, mke2fs
> > >     will use the filesystem type floppy. If the filesystem size is
> > >     greater than or equal to 3 but less than 512 megabytes, mke2fs(8)
> > >     will use the filesystem type small.
> > > 
> > > The relevant parts from /etc/mke2fs.conf:
> > > [fs_types]
> > > ...
> > >         small = {
> > >                 blocksize = 1024
> > >                 inode_size = 128
> > >                 inode_ratio = 4096
> > >         }
> > > ...
> > > 
> > > So whenever you create an ext4 file system with less than 512MB in
> > > size you will end up with 128 byte inodes and your file system is
> > > not Y2038-safe.
> > > 
> > > Some words to the other filesystems supported by wic:
> > >  - ext2 and ext3 will overflow in 2038, 
> > >    The user will be warned as well
> > >  - squashfs will overflow in 2106
> > >  - btrfs uses u64 for time representations, so Y2038 safe
> > 
> > 
> > Thanks, this is probably something we need to do. I did apply it in the 
> > test builds and they do show a few warnings:
> > 
> (sorry for breaking the link, but hopefully our infrastructure is being
> fixed soon)
> > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fautobuilder.yoctoproject.org%2Ftyphoon%2F%23%2Fbuilders%2F58%2Fbuilds%2F3052&amp;data=04%7C01%7Cflorian.bezdeka%40siemens.com%7C35ed532b2cf14a3fb99e08d8d4c4eae5%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637493289505466921%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=1fZIR%2BFOFjTL%2BXaIjZJAVgg6x2YcV6UN6te1Akuvj%2FI%3D&amp;reserved=0
> > 
> > Before we could merge it, we really need to fix this. Are there parameters
> > we should add to our default ext4 to make it Y2038 safe and avoid the 
> > warnings?
> 
> Sorry for the warnings in the CI system. I didn't notice the .wks file
> that is used for that test, but that is actually doing what I wanted to
> achieve: Warn users that there system is affected by Y2038 problem.

Its fine, it was a good test that it works in some ways! :)

> There are a couple of possibilities now:
>   - Adding `--mkfs-extraopts "-t ext4"` to the affected partition
>     This tells mke2fs to use the parameters used for "normal" ext4 file
>     systems instead of the "optimizations" for "small" ones. (There
>     was some additional information in the cover letter)
> 
>   - Adding `--mkfs-extraopts "-I 256"` to tell mke2fs to use the bigger
>     inodes.
> 
> I decided to not apply one of this options per default. Depending on
> the target system and use case there might be (performance) side
> effects.
> 
> Should I come up with a v2 and adding one of the mentioned options to
> the .wks file? In this case both of them are possible. Or should I just
> sent out an additional patch?

In this case a v2 with the additional change might be good, then if people
run into the same warning and find the commit adding it, they'd find
an idea of how to potentially fix it too?

Cheers,

Richard

[OE-core][PATCH v2] wic: Warn if an ext filesystem affected by the Y2038 problem is used

[florian.bezdeka]

From: Florian Bezdeka <florian.bezdeka@siemens.com>

We are getting closer and closer to the year 2038 where the 32 bit
time_t overflow will happen. While products (= embedded systems) with an
expected life time of 15 years are still save the situation may change
if your system has to survive the next 20 years.

ext2 and ext3 file systems are always affected by the time overflow, so
let's warn the user if these file systems are still being used.

If ext4 is affected depends on the inode size chosen during file system
creation. At least 256 bytes are necessary to be safe. As ext4 is
used very often (and partitions may be created small first and extended
later) this might be an issue for many users.

One partition of mkefidisk.wks was already affected by the Y2038
problem. By using `--mkfs-extraopts "-T default"` we tell mke2fs to use the
default usage-type, so disabling the auto-detection of some tuning
parameters that were dependent on the file system size.

Signed-off-by: Florian Bezdeka <florian.bezdeka@siemens.com>
---

 Changes in v2:
  - Fixed a warning reported by CI system coming from a partition in 
    /scripts/lib/wic/canned-wks/mkefidisk.wks. One of the partitions was
    already affected by the Y2038 problem.

 scripts/lib/wic/canned-wks/mkefidisk.wks |  2 +-
 scripts/lib/wic/misc.py                  |  1 +
 scripts/lib/wic/partition.py             | 38 ++++++++++++++++++++++++
 3 files changed, 40 insertions(+), 1 deletion(-)

diff --git a/scripts/lib/wic/canned-wks/mkefidisk.wks b/scripts/lib/wic/canned-wks/mkefidisk.wks
index 9f534fe184..d1878e23e5 100644
--- a/scripts/lib/wic/canned-wks/mkefidisk.wks
+++ b/scripts/lib/wic/canned-wks/mkefidisk.wks
@@ -4,7 +4,7 @@
 
 part /boot --source bootimg-efi --sourceparams="loader=grub-efi" --ondisk sda --label msdos --active --align 1024
 
-part / --source rootfs --ondisk sda --fstype=ext4 --label platform --align 1024 --use-uuid
+part / --source rootfs --ondisk sda --fstype=ext4 --mkfs-extraopts "-T default"  --label platform --align 1024 --use-uuid
 
 part swap --ondisk sda --size 44 --label swap1 --fstype=swap
 
diff --git a/scripts/lib/wic/misc.py b/scripts/lib/wic/misc.py
index 75b219cd3f..57c042c503 100644
--- a/scripts/lib/wic/misc.py
+++ b/scripts/lib/wic/misc.py
@@ -26,6 +26,7 @@ logger = logging.getLogger('wic')
 
 # executable -> recipe pairs for exec_native_cmd
 NATIVE_RECIPES = {"bmaptool": "bmap-tools",
+                  "dumpe2fs": "e2fsprogs",
                   "grub-mkimage": "grub-efi",
                   "isohybrid": "syslinux",
                   "mcopy": "mtools",
diff --git a/scripts/lib/wic/partition.py b/scripts/lib/wic/partition.py
index e574f40c47..85f9847047 100644
--- a/scripts/lib/wic/partition.py
+++ b/scripts/lib/wic/partition.py
@@ -298,6 +298,8 @@ class Partition():
         mkfs_cmd = "fsck.%s -pvfD %s" % (self.fstype, rootfs)
         exec_native_cmd(mkfs_cmd, native_sysroot, pseudo=pseudo)
 
+        self.check_for_Y2038_problem(rootfs, native_sysroot)
+
     def prepare_rootfs_btrfs(self, rootfs, cr_workdir, oe_builddir, rootfs_dir,
                              native_sysroot, pseudo):
         """
@@ -388,6 +390,8 @@ class Partition():
             (self.fstype, extraopts, label_str, self.fsuuid, rootfs)
         exec_native_cmd(mkfs_cmd, native_sysroot)
 
+        self.check_for_Y2038_problem(rootfs, native_sysroot)
+
     def prepare_empty_partition_btrfs(self, rootfs, oe_builddir,
                                       native_sysroot):
         """
@@ -449,3 +453,37 @@ class Partition():
 
         mkswap_cmd = "mkswap %s -U %s %s" % (label_str, self.fsuuid, path)
         exec_native_cmd(mkswap_cmd, native_sysroot)
+
+    def check_for_Y2038_problem(self, rootfs, native_sysroot):
+        """
+        Check if the filesystem is affected by the Y2038 problem
+        (Y2038 problem = 32 bit time_t overflow in January 2038)
+        """
+        def get_err_str(part):
+            err = "The {} filesystem {} has no Y2038 support."
+            if part.mountpoint:
+                args = [part.fstype, "mounted at %s" % part.mountpoint]
+            elif part.label:
+                args = [part.fstype, "labeled '%s'" % part.label]
+            elif part.part_name:
+                args = [part.fstype, "in partition '%s'" % part.part_name]
+            else:
+                args = [part.fstype, "in partition %s" % part.num]
+            return err.format(*args)
+
+        # ext2 and ext3 are always affected by the Y2038 problem
+        if self.fstype in ["ext2", "ext3"]:
+            logger.warn(get_err_str(self))
+            return
+
+        ret, out = exec_native_cmd("dumpe2fs %s" % rootfs, native_sysroot)
+
+        # if ext4 is affected by the Y2038 problem depends on the inode size
+        for line in out.splitlines():
+            if line.startswith("Inode size:"):
+                size = int(line.split(":")[1].strip())
+                if size < 256:
+                    logger.warn("%s Inodes (of size %d) are too small." %
+                                (get_err_str(self), size))
+                break
+
-- 
2.29.2

Re: [OE-core][PATCH v2] wic: Warn if an ext filesystem affected by the Y2038 problem is used

[Richard Purdie]

On Mon, 2021-02-22 at 11:42 +0000, florian.bezdeka@siemens.com wrote:
> From: Florian Bezdeka <florian.bezdeka@siemens.com>
> 
> We are getting closer and closer to the year 2038 where the 32 bit
> time_t overflow will happen. While products (= embedded systems) with an
> expected life time of 15 years are still save the situation may change
> if your system has to survive the next 20 years.
> 
> ext2 and ext3 file systems are always affected by the time overflow, so
> let's warn the user if these file systems are still being used.
> 
> If ext4 is affected depends on the inode size chosen during file system
> creation. At least 256 bytes are necessary to be safe. As ext4 is
> used very often (and partitions may be created small first and extended
> later) this might be an issue for many users.
> 
> One partition of mkefidisk.wks was already affected by the Y2038
> problem. By using `--mkfs-extraopts "-T default"` we tell mke2fs to use the
> default usage-type, so disabling the auto-detection of some tuning
> parameters that were dependent on the file system size.
> 
> Signed-off-by: Florian Bezdeka <florian.bezdeka@siemens.com>
> ---
> 
>  Changes in v2:
>   - Fixed a warning reported by CI system coming from a partition in 
>     /scripts/lib/wic/canned-wks/mkefidisk.wks. One of the partitions was
>     already affected by the Y2038 problem.

I added this and reran but its still showing some warnings unfortunately:

https://autobuilder.yoctoproject.org/typhoon/#/builders/58/builds/3065

Cheers,

Richard

Re: [OE-core][PATCH v2] wic: Warn if an ext filesystem affected by the Y2038 problem is used

[florian.bezdeka]

On Mon, 2021-02-22 at 15:59 +0000, Richard Purdie wrote:
> On Mon, 2021-02-22 at 11:42 +0000, florian.bezdeka@siemens.com wrote:
> > From: Florian Bezdeka <florian.bezdeka@siemens.com>
> > 
> > We are getting closer and closer to the year 2038 where the 32 bit
> > time_t overflow will happen. While products (= embedded systems) with an
> > expected life time of 15 years are still save the situation may change
> > if your system has to survive the next 20 years.
> > 
> > ext2 and ext3 file systems are always affected by the time overflow, so
> > let's warn the user if these file systems are still being used.
> > 
> > If ext4 is affected depends on the inode size chosen during file system
> > creation. At least 256 bytes are necessary to be safe. As ext4 is
> > used very often (and partitions may be created small first and extended
> > later) this might be an issue for many users.
> > 
> > One partition of mkefidisk.wks was already affected by the Y2038
> > problem. By using `--mkfs-extraopts "-T default"` we tell mke2fs to use the
> > default usage-type, so disabling the auto-detection of some tuning
> > parameters that were dependent on the file system size.
> > 
> > Signed-off-by: Florian Bezdeka <florian.bezdeka@siemens.com>
> > ---
> > 
> >  Changes in v2:
> >   - Fixed a warning reported by CI system coming from a partition in 
> >     /scripts/lib/wic/canned-wks/mkefidisk.wks. One of the partitions was
> >     already affected by the Y2038 problem.
> 
> I added this and reran but its still showing some warnings unfortunately:
> 
> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fautobuilder.yoctoproject.org%2Ftyphoon%2F%23%2Fbuilders%2F58%2Fbuilds%2F3065&amp;data=04%7C01%7Cflorian.bezdeka%40siemens.com%7C9fd84254f5d74deaec6208d8d74d5279%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637496074381666346%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=g32gFNEDvmIK2me%2BKKJEBwEIcdGMffd4TuhIMjmrZtU%3D&amp;reserved=0
> 

I see... Is it possible to see `/etc/mke2fs.conf` from within the
buildchroot that is actually calling `wic` somehow?

> Cheers,
> 
> Richard
> 

[OE-core][PATCH v3] wic: Warn if an ext filesystem affected by the Y2038 problem is used

[florian.bezdeka]

From: Florian Bezdeka <florian.bezdeka@siemens.com>

We are getting closer and closer to the year 2038 where the 32 bit
time_t overflow will happen. While products (= embedded systems) with an
expected life time of 15 years are still save the situation may change
if your system has to survive the next 20 years.

ext2 and ext3 file systems are always affected by the time overflow, so
let's warn the user if these file systems are still being used.

If ext4 is affected depends on the inode size chosen during file system
creation. At least 256 bytes are necessary to be safe. As ext4 is
used very often (and partitions may be created small first and extended
later) this might be an issue for many users.

One partition of mkefidisk.wks was already affected by the Y2038
problem. By using `--mkfs-extraopts "-I 256"` we tell mke2fs to use
inodes of size 256 bytes to be Y2038 safe.

Signed-off-by: Florian Bezdeka <florian.bezdeka@siemens.com>
---
 
Changes in v2:                                                                     
  - Fixed a warning reported by CI system coming from a partition in
    /scripts/lib/wic/canned-wks/mkefidisk.wks. One of the partitions was
    already affected by the Y2038 problem.

Changes in v3:
  - Moved to "-I 256" to become independent from the /etc/mke2fs.conf
    file (or even from the mke2fs compiled in defaults)

scripts/lib/wic/canned-wks/mkefidisk.wks |  2 +-
 scripts/lib/wic/misc.py                  |  1 +
 scripts/lib/wic/partition.py             | 38 ++++++++++++++++++++++++
 3 files changed, 40 insertions(+), 1 deletion(-)

diff --git a/scripts/lib/wic/canned-wks/mkefidisk.wks b/scripts/lib/wic/canned-wks/mkefidisk.wks
index 9f534fe184..627a03beca 100644
--- a/scripts/lib/wic/canned-wks/mkefidisk.wks
+++ b/scripts/lib/wic/canned-wks/mkefidisk.wks
@@ -4,7 +4,7 @@
 
 part /boot --source bootimg-efi --sourceparams="loader=grub-efi" --ondisk sda --label msdos --active --align 1024
 
-part / --source rootfs --ondisk sda --fstype=ext4 --label platform --align 1024 --use-uuid
+part / --source rootfs --ondisk sda --fstype=ext4 --mkfs-extraopts "-I 256"  --label platform --align 1024 --use-uuid
 
 part swap --ondisk sda --size 44 --label swap1 --fstype=swap
 
diff --git a/scripts/lib/wic/misc.py b/scripts/lib/wic/misc.py
index 75b219cd3f..57c042c503 100644
--- a/scripts/lib/wic/misc.py
+++ b/scripts/lib/wic/misc.py
@@ -26,6 +26,7 @@ logger = logging.getLogger('wic')
 
 # executable -> recipe pairs for exec_native_cmd
 NATIVE_RECIPES = {"bmaptool": "bmap-tools",
+                  "dumpe2fs": "e2fsprogs",
                   "grub-mkimage": "grub-efi",
                   "isohybrid": "syslinux",
                   "mcopy": "mtools",
diff --git a/scripts/lib/wic/partition.py b/scripts/lib/wic/partition.py
index e574f40c47..85f9847047 100644
--- a/scripts/lib/wic/partition.py
+++ b/scripts/lib/wic/partition.py
@@ -298,6 +298,8 @@ class Partition():
         mkfs_cmd = "fsck.%s -pvfD %s" % (self.fstype, rootfs)
         exec_native_cmd(mkfs_cmd, native_sysroot, pseudo=pseudo)
 
+        self.check_for_Y2038_problem(rootfs, native_sysroot)
+
     def prepare_rootfs_btrfs(self, rootfs, cr_workdir, oe_builddir, rootfs_dir,
                              native_sysroot, pseudo):
         """
@@ -388,6 +390,8 @@ class Partition():
             (self.fstype, extraopts, label_str, self.fsuuid, rootfs)
         exec_native_cmd(mkfs_cmd, native_sysroot)
 
+        self.check_for_Y2038_problem(rootfs, native_sysroot)
+
     def prepare_empty_partition_btrfs(self, rootfs, oe_builddir,
                                       native_sysroot):
         """
@@ -449,3 +453,37 @@ class Partition():
 
         mkswap_cmd = "mkswap %s -U %s %s" % (label_str, self.fsuuid, path)
         exec_native_cmd(mkswap_cmd, native_sysroot)
+
+    def check_for_Y2038_problem(self, rootfs, native_sysroot):
+        """
+        Check if the filesystem is affected by the Y2038 problem
+        (Y2038 problem = 32 bit time_t overflow in January 2038)
+        """
+        def get_err_str(part):
+            err = "The {} filesystem {} has no Y2038 support."
+            if part.mountpoint:
+                args = [part.fstype, "mounted at %s" % part.mountpoint]
+            elif part.label:
+                args = [part.fstype, "labeled '%s'" % part.label]
+            elif part.part_name:
+                args = [part.fstype, "in partition '%s'" % part.part_name]
+            else:
+                args = [part.fstype, "in partition %s" % part.num]
+            return err.format(*args)
+
+        # ext2 and ext3 are always affected by the Y2038 problem
+        if self.fstype in ["ext2", "ext3"]:
+            logger.warn(get_err_str(self))
+            return
+
+        ret, out = exec_native_cmd("dumpe2fs %s" % rootfs, native_sysroot)
+
+        # if ext4 is affected by the Y2038 problem depends on the inode size
+        for line in out.splitlines():
+            if line.startswith("Inode size:"):
+                size = int(line.split(":")[1].strip())
+                if size < 256:
+                    logger.warn("%s Inodes (of size %d) are too small." %
+                                (get_err_str(self), size))
+                break
+
-- 
2.29.2

Re: [OE-core][PATCH v3] wic: Warn if an ext filesystem affected by the Y2038 problem is used

[Richard Purdie]

On Tue, 2021-02-23 at 15:30 +0000, florian.bezdeka@siemens.com wrote:
> From: Florian Bezdeka <florian.bezdeka@siemens.com>
> 
> We are getting closer and closer to the year 2038 where the 32 bit
> time_t overflow will happen. While products (= embedded systems) with an
> expected life time of 15 years are still save the situation may change
> if your system has to survive the next 20 years.
> 
> ext2 and ext3 file systems are always affected by the time overflow, so
> let's warn the user if these file systems are still being used.
> 
> If ext4 is affected depends on the inode size chosen during file system
> creation. At least 256 bytes are necessary to be safe. As ext4 is
> used very often (and partitions may be created small first and extended
> later) this might be an issue for many users.
> 
> One partition of mkefidisk.wks was already affected by the Y2038
> problem. By using `--mkfs-extraopts "-I 256"` we tell mke2fs to use
> inodes of size 256 bytes to be Y2038 safe.
> 
> Signed-off-by: Florian Bezdeka <florian.bezdeka@siemens.com>
> ---
>  
> 
> Changes in v2:                                                                     
>   - Fixed a warning reported by CI system coming from a partition in
>     /scripts/lib/wic/canned-wks/mkefidisk.wks. One of the partitions was
>     already affected by the Y2038 problem.
> 
> Changes in v3:
>   - Moved to "-I 256" to become independent from the /etc/mke2fs.conf
>     file (or even from the mke2fs compiled in defaults)

CI tests are still showing the warnings unfortunately:

https://autobuilder.yoctoproject.org/typhoon/#/builders/58/builds/3079

Cheers,

Richard

[OE-core][PATCH v4] wic: Warn if an ext filesystem affected by the Y2038 problem is used

[florian.bezdeka]

We are getting closer and closer to the year 2038 where the 32 bit
time_t overflow will happen. While products (= embedded systems) with an
expected life time of 15 years are still save the situation may change
if your system has to survive the next 20 years.

ext2 and ext3 filesystems are always affected by the time overflow, so
let's warn the user if these filesystems are still being used.

If ext4 is affected depends on the inode size chosen during filesystem
creation. At least 256 bytes are necessary to be safe. As ext4 is
used very often (and partitions may be created small first and extended
later) this might be an issue for many users.

Some filesystems created during CI runs were already affected by the Y2038
problem. By using `--mkfs-extraopts "-T default"` we tell mke2fs not to
auto-detect the usage type based on the filesystem size. mke2fs will use
the default values for tuning parameters instead. The inode size is one
of these parameters.

Signed-off-by: Florian Bezdeka <florian.bezdeka@siemens.com>
---

  Changes in v4:
    - Updated the commit message to reflect current state
    - Fixed more affected filesystems that were reported by the CI system
      (directdisk-gpt.wks and directdisk.wks)
    - Moving back to "-T default", so disabling usage-type detection based
      on the filesystem size

  Changes in v3:
    - Moved to "-I 256" to become independent from the /etc/mke2fs.conf
      file (or even from the mke2fs compiled in defaults)

  Changes in v2: 
    - Fixed a warning reported by CI system coming from a partition in
      /scripts/lib/wic/canned-wks/mkefidisk.wks. One of the partitions was
      already affected by the Y2038 problem.
                                                                                    
 
 scripts/lib/wic/canned-wks/common.wks.inc     |  2 +-
 scripts/lib/wic/canned-wks/directdisk-gpt.wks |  2 +-
 scripts/lib/wic/canned-wks/mkefidisk.wks      |  2 +-
 scripts/lib/wic/misc.py                       |  1 +
 scripts/lib/wic/partition.py                  | 38 +++++++++++++++++++
 5 files changed, 42 insertions(+), 3 deletions(-)

diff --git a/scripts/lib/wic/canned-wks/common.wks.inc b/scripts/lib/wic/canned-wks/common.wks.inc
index 89880b417b..4fd29fa8c1 100644
--- a/scripts/lib/wic/canned-wks/common.wks.inc
+++ b/scripts/lib/wic/canned-wks/common.wks.inc
@@ -1,3 +1,3 @@
 # This file is included into 3 canned wks files from this directory
 part /boot --source bootimg-pcbios --ondisk sda --label boot --active --align 1024
-part / --source rootfs --use-uuid --fstype=ext4 --label platform --align 1024
+part / --source rootfs --use-uuid --fstype=ext4 --mkfs-extraopts "-T default" --label platform --align 1024
diff --git a/scripts/lib/wic/canned-wks/directdisk-gpt.wks b/scripts/lib/wic/canned-wks/directdisk-gpt.wks
index 8d7d8de6ea..cf16c0c30b 100644
--- a/scripts/lib/wic/canned-wks/directdisk-gpt.wks
+++ b/scripts/lib/wic/canned-wks/directdisk-gpt.wks
@@ -4,7 +4,7 @@
 
 
 part /boot --source bootimg-pcbios --ondisk sda --label boot --active --align 1024
-part / --source rootfs --ondisk sda --fstype=ext4 --label platform --align 1024 --use-uuid
+part / --source rootfs --ondisk sda --fstype=ext4 --mkfs-extraopts "-T default" --label platform --align 1024 --use-uuid
 
 bootloader  --ptable gpt --timeout=0  --append="rootwait rootfstype=ext4 video=vesafb vga=0x318 console=tty0 console=ttyS0,115200n8"
 
diff --git a/scripts/lib/wic/canned-wks/mkefidisk.wks b/scripts/lib/wic/canned-wks/mkefidisk.wks
index 9f534fe184..d1878e23e5 100644
--- a/scripts/lib/wic/canned-wks/mkefidisk.wks
+++ b/scripts/lib/wic/canned-wks/mkefidisk.wks
@@ -4,7 +4,7 @@
 
 part /boot --source bootimg-efi --sourceparams="loader=grub-efi" --ondisk sda --label msdos --active --align 1024
 
-part / --source rootfs --ondisk sda --fstype=ext4 --label platform --align 1024 --use-uuid
+part / --source rootfs --ondisk sda --fstype=ext4 --mkfs-extraopts "-T default"  --label platform --align 1024 --use-uuid
 
 part swap --ondisk sda --size 44 --label swap1 --fstype=swap
 
diff --git a/scripts/lib/wic/misc.py b/scripts/lib/wic/misc.py
index 75b219cd3f..57c042c503 100644
--- a/scripts/lib/wic/misc.py
+++ b/scripts/lib/wic/misc.py
@@ -26,6 +26,7 @@ logger = logging.getLogger('wic')
 
 # executable -> recipe pairs for exec_native_cmd
 NATIVE_RECIPES = {"bmaptool": "bmap-tools",
+                  "dumpe2fs": "e2fsprogs",
                   "grub-mkimage": "grub-efi",
                   "isohybrid": "syslinux",
                   "mcopy": "mtools",
diff --git a/scripts/lib/wic/partition.py b/scripts/lib/wic/partition.py
index e574f40c47..85f9847047 100644
--- a/scripts/lib/wic/partition.py
+++ b/scripts/lib/wic/partition.py
@@ -298,6 +298,8 @@ class Partition():
         mkfs_cmd = "fsck.%s -pvfD %s" % (self.fstype, rootfs)
         exec_native_cmd(mkfs_cmd, native_sysroot, pseudo=pseudo)
 
+        self.check_for_Y2038_problem(rootfs, native_sysroot)
+
     def prepare_rootfs_btrfs(self, rootfs, cr_workdir, oe_builddir, rootfs_dir,
                              native_sysroot, pseudo):
         """
@@ -388,6 +390,8 @@ class Partition():
             (self.fstype, extraopts, label_str, self.fsuuid, rootfs)
         exec_native_cmd(mkfs_cmd, native_sysroot)
 
+        self.check_for_Y2038_problem(rootfs, native_sysroot)
+
     def prepare_empty_partition_btrfs(self, rootfs, oe_builddir,
                                       native_sysroot):
         """
@@ -449,3 +453,37 @@ class Partition():
 
         mkswap_cmd = "mkswap %s -U %s %s" % (label_str, self.fsuuid, path)
         exec_native_cmd(mkswap_cmd, native_sysroot)
+
+    def check_for_Y2038_problem(self, rootfs, native_sysroot):
+        """
+        Check if the filesystem is affected by the Y2038 problem
+        (Y2038 problem = 32 bit time_t overflow in January 2038)
+        """
+        def get_err_str(part):
+            err = "The {} filesystem {} has no Y2038 support."
+            if part.mountpoint:
+                args = [part.fstype, "mounted at %s" % part.mountpoint]
+            elif part.label:
+                args = [part.fstype, "labeled '%s'" % part.label]
+            elif part.part_name:
+                args = [part.fstype, "in partition '%s'" % part.part_name]
+            else:
+                args = [part.fstype, "in partition %s" % part.num]
+            return err.format(*args)
+
+        # ext2 and ext3 are always affected by the Y2038 problem
+        if self.fstype in ["ext2", "ext3"]:
+            logger.warn(get_err_str(self))
+            return
+
+        ret, out = exec_native_cmd("dumpe2fs %s" % rootfs, native_sysroot)
+
+        # if ext4 is affected by the Y2038 problem depends on the inode size
+        for line in out.splitlines():
+            if line.startswith("Inode size:"):
+                size = int(line.split(":")[1].strip())
+                if size < 256:
+                    logger.warn("%s Inodes (of size %d) are too small." %
+                                (get_err_str(self), size))
+                break
+
-- 
2.29.2