* [OE-core][PATCH 0/1] wic: warn on usage of Y2038 affected file systems @ 2021-02-18 13:06 florian.bezdeka 2021-02-18 13:06 ` [OE-core][PATCH 1/1] wic: Warn if an ext filesystem affected by the Y2038 problem is used florian.bezdeka 2021-02-19 10:55 ` [OE-core][PATCH 0/1] wic: warn on usage of Y2038 affected file systems Richard Purdie 0 siblings, 2 replies; 11+ messages in thread From: florian.bezdeka @ 2021-02-18 13:06 UTC (permalink / raw) To: openembedded-core; +Cc: florian.bezdeka From: Florian Bezdeka <florian.bezdeka@siemens.com> The following patch is the summary of a nice journey through the file system jungle regarding Y2038 problem. It all began with a warning which is reported by kernels >= 5.4: ext4 filesystem being mounted at (mountpoint) supports timestamps until 2038 (0x7fffffff) When reading this warning I was surprised. Shouldn't a modern file system like ext4 be Y2038-safe? As it turned out it depends on the inode size if an ext4 file system is safe or not. So why was the inode size not sufficient in my case? The inode size is chosen during file system generation and depends on the size of the file system that is going to be created. For details let's have a look at `man mke2fs`: -T usage-type[,...] Specify how the filesystem is going to be used, so that mke2fs can choose optimal filesystem parameters for that use. The usage types that are supported are defined in the configuration file /etc/mke2fs.conf. The user may specify one or more usage types using a comma separated list. If this option is is not specified, mke2fs will pick a single default usage type based on the size of the filesystem to be created. If the filesystem size is less than 3 megabytes, mke2fs will use the filesystem type floppy. If the filesystem size is greater than or equal to 3 but less than 512 megabytes, mke2fs(8) will use the filesystem type small. The relevant parts from /etc/mke2fs.conf: [fs_types] ... small = { blocksize = 1024 inode_size = 128 inode_ratio = 4096 } ... So whenever you create an ext4 file system with less than 512MB in size you will end up with 128 byte inodes and your file system is not Y2038-safe. Some words to the other filesystems supported by wic: - ext2 and ext3 will overflow in 2038, The user will be warned as well - squashfs will overflow in 2106 - btrfs uses u64 for time representations, so Y2038 safe Florian Bezdeka (1): wic: Warn if an ext filesystem affected by the Y2038 problem is used scripts/lib/wic/misc.py | 1 + scripts/lib/wic/partition.py | 38 ++++++++++++++++++++++++++++++++++++ 2 files changed, 39 insertions(+) -- 2.29.2 ^ permalink raw reply [flat|nested] 11+ messages in thread
* [OE-core][PATCH 1/1] wic: Warn if an ext filesystem affected by the Y2038 problem is used 2021-02-18 13:06 [OE-core][PATCH 0/1] wic: warn on usage of Y2038 affected file systems florian.bezdeka @ 2021-02-18 13:06 ` florian.bezdeka 2021-02-19 10:55 ` [OE-core][PATCH 0/1] wic: warn on usage of Y2038 affected file systems Richard Purdie 1 sibling, 0 replies; 11+ messages in thread From: florian.bezdeka @ 2021-02-18 13:06 UTC (permalink / raw) To: openembedded-core; +Cc: florian.bezdeka From: Florian Bezdeka <florian.bezdeka@siemens.com> We are getting closer and closer to the year 2038 where the 32 bit time_t overflow will happen. While products (= embedded systems) with an expected life time of 15 years are still save the situation may change if your system has to survive the next 20 years. ext2 and ext3 file systems are always affected by the time overflow, so let's warn the user if these file systems are still being used. If ext4 is affected depends on the inode size chosen during file system creation. At least 256 bytes are necessary to be safe. As ext4 is used very often (and partitions may be created small first and extended later) this might be an issue for many users. Signed-off-by: Florian Bezdeka <florian.bezdeka@siemens.com> --- scripts/lib/wic/misc.py | 1 + scripts/lib/wic/partition.py | 38 ++++++++++++++++++++++++++++++++++++ 2 files changed, 39 insertions(+) diff --git a/scripts/lib/wic/misc.py b/scripts/lib/wic/misc.py index 75b219cd3f..57c042c503 100644 --- a/scripts/lib/wic/misc.py +++ b/scripts/lib/wic/misc.py @@ -26,6 +26,7 @@ logger = logging.getLogger('wic') # executable -> recipe pairs for exec_native_cmd NATIVE_RECIPES = {"bmaptool": "bmap-tools", + "dumpe2fs": "e2fsprogs", "grub-mkimage": "grub-efi", "isohybrid": "syslinux", "mcopy": "mtools", diff --git a/scripts/lib/wic/partition.py b/scripts/lib/wic/partition.py index e574f40c47..85f9847047 100644 --- a/scripts/lib/wic/partition.py +++ b/scripts/lib/wic/partition.py @@ -298,6 +298,8 @@ class Partition(): mkfs_cmd = "fsck.%s -pvfD %s" % (self.fstype, rootfs) exec_native_cmd(mkfs_cmd, native_sysroot, pseudo=pseudo) + self.check_for_Y2038_problem(rootfs, native_sysroot) + def prepare_rootfs_btrfs(self, rootfs, cr_workdir, oe_builddir, rootfs_dir, native_sysroot, pseudo): """ @@ -388,6 +390,8 @@ class Partition(): (self.fstype, extraopts, label_str, self.fsuuid, rootfs) exec_native_cmd(mkfs_cmd, native_sysroot) + self.check_for_Y2038_problem(rootfs, native_sysroot) + def prepare_empty_partition_btrfs(self, rootfs, oe_builddir, native_sysroot): """ @@ -449,3 +453,37 @@ class Partition(): mkswap_cmd = "mkswap %s -U %s %s" % (label_str, self.fsuuid, path) exec_native_cmd(mkswap_cmd, native_sysroot) + + def check_for_Y2038_problem(self, rootfs, native_sysroot): + """ + Check if the filesystem is affected by the Y2038 problem + (Y2038 problem = 32 bit time_t overflow in January 2038) + """ + def get_err_str(part): + err = "The {} filesystem {} has no Y2038 support." + if part.mountpoint: + args = [part.fstype, "mounted at %s" % part.mountpoint] + elif part.label: + args = [part.fstype, "labeled '%s'" % part.label] + elif part.part_name: + args = [part.fstype, "in partition '%s'" % part.part_name] + else: + args = [part.fstype, "in partition %s" % part.num] + return err.format(*args) + + # ext2 and ext3 are always affected by the Y2038 problem + if self.fstype in ["ext2", "ext3"]: + logger.warn(get_err_str(self)) + return + + ret, out = exec_native_cmd("dumpe2fs %s" % rootfs, native_sysroot) + + # if ext4 is affected by the Y2038 problem depends on the inode size + for line in out.splitlines(): + if line.startswith("Inode size:"): + size = int(line.split(":")[1].strip()) + if size < 256: + logger.warn("%s Inodes (of size %d) are too small." % + (get_err_str(self), size)) + break + -- 2.29.2 ^ permalink raw reply related [flat|nested] 11+ messages in thread
* Re: [OE-core][PATCH 0/1] wic: warn on usage of Y2038 affected file systems 2021-02-18 13:06 [OE-core][PATCH 0/1] wic: warn on usage of Y2038 affected file systems florian.bezdeka 2021-02-18 13:06 ` [OE-core][PATCH 1/1] wic: Warn if an ext filesystem affected by the Y2038 problem is used florian.bezdeka @ 2021-02-19 10:55 ` Richard Purdie 2021-02-19 11:19 ` florian.bezdeka 1 sibling, 1 reply; 11+ messages in thread From: Richard Purdie @ 2021-02-19 10:55 UTC (permalink / raw) To: florian.bezdeka, openembedded-core On Thu, 2021-02-18 at 13:06 +0000, florian.bezdeka@siemens.com wrote: > From: Florian Bezdeka <florian.bezdeka@siemens.com> > > The following patch is the summary of a nice journey through the file > system jungle regarding Y2038 problem. It all began with a warning which > is reported by kernels >= 5.4: > > ext4 filesystem being mounted at (mountpoint) supports timestamps until > 2038 (0x7fffffff) > > When reading this warning I was surprised. Shouldn't a modern file > system like ext4 be Y2038-safe? As it turned out it depends on the > inode size if an ext4 file system is safe or not. So why was the > inode size not sufficient in my case? > > The inode size is chosen during file system generation and depends on > the size of the file system that is going to be created. For details > let's have a look at `man mke2fs`: > > -T usage-type[,...] > Specify how the filesystem is going to be used, so that mke2fs can > choose optimal filesystem parameters for that use. The usage types > that are supported are defined in the configuration file > /etc/mke2fs.conf. The user may specify one or more usage types > using a comma separated list. > > If this option is is not specified, mke2fs will pick a single > default usage type based on the size of the filesystem to be > created. If the filesystem size is less than 3 megabytes, mke2fs > will use the filesystem type floppy. If the filesystem size is > greater than or equal to 3 but less than 512 megabytes, mke2fs(8) > will use the filesystem type small. > > The relevant parts from /etc/mke2fs.conf: > [fs_types] > ... > small = { > blocksize = 1024 > inode_size = 128 > inode_ratio = 4096 > } > ... > > So whenever you create an ext4 file system with less than 512MB in > size you will end up with 128 byte inodes and your file system is > not Y2038-safe. > > Some words to the other filesystems supported by wic: > - ext2 and ext3 will overflow in 2038, > The user will be warned as well > - squashfs will overflow in 2106 > - btrfs uses u64 for time representations, so Y2038 safe Thanks, this is probably something we need to do. I did apply it in the test builds and they do show a few warnings: https://autobuilder.yoctoproject.org/typhoon/#/builders/58/builds/3052 Before we could merge it, we really need to fix this. Are there parameters we should add to our default ext4 to make it Y2038 safe and avoid the warnings? Cheers, Richard ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [OE-core][PATCH 0/1] wic: warn on usage of Y2038 affected file systems 2021-02-19 10:55 ` [OE-core][PATCH 0/1] wic: warn on usage of Y2038 affected file systems Richard Purdie @ 2021-02-19 11:19 ` florian.bezdeka 2021-02-19 11:36 ` Richard Purdie 0 siblings, 1 reply; 11+ messages in thread From: florian.bezdeka @ 2021-02-19 11:19 UTC (permalink / raw) To: richard.purdie, openembedded-core; +Cc: jan.kiszka, henning.schild Hi Richard, On Fri, 2021-02-19 at 10:55 +0000, Richard Purdie wrote: > On Thu, 2021-02-18 at 13:06 +0000, florian.bezdeka@siemens.com wrote: > > From: Florian Bezdeka <florian.bezdeka@siemens.com> > > > > The following patch is the summary of a nice journey through the file > > system jungle regarding Y2038 problem. It all began with a warning which > > is reported by kernels >= 5.4: > > > > ext4 filesystem being mounted at (mountpoint) supports timestamps until > > 2038 (0x7fffffff) > > > > When reading this warning I was surprised. Shouldn't a modern file > > system like ext4 be Y2038-safe? As it turned out it depends on the > > inode size if an ext4 file system is safe or not. So why was the > > inode size not sufficient in my case? > > > > The inode size is chosen during file system generation and depends on > > the size of the file system that is going to be created. For details > > let's have a look at `man mke2fs`: > > > > -T usage-type[,...] > > Specify how the filesystem is going to be used, so that mke2fs can > > choose optimal filesystem parameters for that use. The usage types > > that are supported are defined in the configuration file > > /etc/mke2fs.conf. The user may specify one or more usage types > > using a comma separated list. > > > > If this option is is not specified, mke2fs will pick a single > > default usage type based on the size of the filesystem to be > > created. If the filesystem size is less than 3 megabytes, mke2fs > > will use the filesystem type floppy. If the filesystem size is > > greater than or equal to 3 but less than 512 megabytes, mke2fs(8) > > will use the filesystem type small. > > > > The relevant parts from /etc/mke2fs.conf: > > [fs_types] > > ... > > small = { > > blocksize = 1024 > > inode_size = 128 > > inode_ratio = 4096 > > } > > ... > > > > So whenever you create an ext4 file system with less than 512MB in > > size you will end up with 128 byte inodes and your file system is > > not Y2038-safe. > > > > Some words to the other filesystems supported by wic: > > - ext2 and ext3 will overflow in 2038, > > The user will be warned as well > > - squashfs will overflow in 2106 > > - btrfs uses u64 for time representations, so Y2038 safe > > > Thanks, this is probably something we need to do. I did apply it in the > test builds and they do show a few warnings: > (sorry for breaking the link, but hopefully our infrastructure is being fixed soon) > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fautobuilder.yoctoproject.org%2Ftyphoon%2F%23%2Fbuilders%2F58%2Fbuilds%2F3052&data=04%7C01%7Cflorian.bezdeka%40siemens.com%7C35ed532b2cf14a3fb99e08d8d4c4eae5%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637493289505466921%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=1fZIR%2BFOFjTL%2BXaIjZJAVgg6x2YcV6UN6te1Akuvj%2FI%3D&reserved=0 > > Before we could merge it, we really need to fix this. Are there parameters > we should add to our default ext4 to make it Y2038 safe and avoid the > warnings? Sorry for the warnings in the CI system. I didn't notice the .wks file that is used for that test, but that is actually doing what I wanted to achieve: Warn users that there system is affected by Y2038 problem. There are a couple of possibilities now: - Adding `--mkfs-extraopts "-t ext4"` to the affected partition This tells mke2fs to use the parameters used for "normal" ext4 file systems instead of the "optimizations" for "small" ones. (There was some additional information in the cover letter) - Adding `--mkfs-extraopts "-I 256"` to tell mke2fs to use the bigger inodes. I decided to not apply one of this options per default. Depending on the target system and use case there might be (performance) side effects. Should I come up with a v2 and adding one of the mentioned options to the .wks file? In this case both of them are possible. Or should I just sent out an additional patch? Best regards, Florian > > Cheers, > > Richard > > ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [OE-core][PATCH 0/1] wic: warn on usage of Y2038 affected file systems 2021-02-19 11:19 ` florian.bezdeka @ 2021-02-19 11:36 ` Richard Purdie 2021-02-22 11:42 ` [OE-core][PATCH v2] wic: Warn if an ext filesystem affected by the Y2038 problem is used florian.bezdeka 0 siblings, 1 reply; 11+ messages in thread From: Richard Purdie @ 2021-02-19 11:36 UTC (permalink / raw) To: florian.bezdeka, openembedded-core; +Cc: jan.kiszka, henning.schild On Fri, 2021-02-19 at 11:19 +0000, florian.bezdeka@siemens.com wrote: > Hi Richard, > > On Fri, 2021-02-19 at 10:55 +0000, Richard Purdie wrote: > > On Thu, 2021-02-18 at 13:06 +0000, florian.bezdeka@siemens.com wrote: > > > From: Florian Bezdeka <florian.bezdeka@siemens.com> > > > > > > The following patch is the summary of a nice journey through the file > > > system jungle regarding Y2038 problem. It all began with a warning which > > > is reported by kernels >= 5.4: > > > > > > ext4 filesystem being mounted at (mountpoint) supports timestamps until > > > 2038 (0x7fffffff) > > > > > > When reading this warning I was surprised. Shouldn't a modern file > > > system like ext4 be Y2038-safe? As it turned out it depends on the > > > inode size if an ext4 file system is safe or not. So why was the > > > inode size not sufficient in my case? > > > > > > The inode size is chosen during file system generation and depends on > > > the size of the file system that is going to be created. For details > > > let's have a look at `man mke2fs`: > > > > > > -T usage-type[,...] > > > Specify how the filesystem is going to be used, so that mke2fs can > > > choose optimal filesystem parameters for that use. The usage types > > > that are supported are defined in the configuration file > > > /etc/mke2fs.conf. The user may specify one or more usage types > > > using a comma separated list. > > > > > > If this option is is not specified, mke2fs will pick a single > > > default usage type based on the size of the filesystem to be > > > created. If the filesystem size is less than 3 megabytes, mke2fs > > > will use the filesystem type floppy. If the filesystem size is > > > greater than or equal to 3 but less than 512 megabytes, mke2fs(8) > > > will use the filesystem type small. > > > > > > The relevant parts from /etc/mke2fs.conf: > > > [fs_types] > > > ... > > > small = { > > > blocksize = 1024 > > > inode_size = 128 > > > inode_ratio = 4096 > > > } > > > ... > > > > > > So whenever you create an ext4 file system with less than 512MB in > > > size you will end up with 128 byte inodes and your file system is > > > not Y2038-safe. > > > > > > Some words to the other filesystems supported by wic: > > > - ext2 and ext3 will overflow in 2038, > > > The user will be warned as well > > > - squashfs will overflow in 2106 > > > - btrfs uses u64 for time representations, so Y2038 safe > > > > > > Thanks, this is probably something we need to do. I did apply it in the > > test builds and they do show a few warnings: > > > (sorry for breaking the link, but hopefully our infrastructure is being > fixed soon) > > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fautobuilder.yoctoproject.org%2Ftyphoon%2F%23%2Fbuilders%2F58%2Fbuilds%2F3052&data=04%7C01%7Cflorian.bezdeka%40siemens.com%7C35ed532b2cf14a3fb99e08d8d4c4eae5%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637493289505466921%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=1fZIR%2BFOFjTL%2BXaIjZJAVgg6x2YcV6UN6te1Akuvj%2FI%3D&reserved=0 > > > > Before we could merge it, we really need to fix this. Are there parameters > > we should add to our default ext4 to make it Y2038 safe and avoid the > > warnings? > > Sorry for the warnings in the CI system. I didn't notice the .wks file > that is used for that test, but that is actually doing what I wanted to > achieve: Warn users that there system is affected by Y2038 problem. Its fine, it was a good test that it works in some ways! :) > There are a couple of possibilities now: > - Adding `--mkfs-extraopts "-t ext4"` to the affected partition > This tells mke2fs to use the parameters used for "normal" ext4 file > systems instead of the "optimizations" for "small" ones. (There > was some additional information in the cover letter) > > - Adding `--mkfs-extraopts "-I 256"` to tell mke2fs to use the bigger > inodes. > > I decided to not apply one of this options per default. Depending on > the target system and use case there might be (performance) side > effects. > > Should I come up with a v2 and adding one of the mentioned options to > the .wks file? In this case both of them are possible. Or should I just > sent out an additional patch? In this case a v2 with the additional change might be good, then if people run into the same warning and find the commit adding it, they'd find an idea of how to potentially fix it too? Cheers, Richard ^ permalink raw reply [flat|nested] 11+ messages in thread
* [OE-core][PATCH v2] wic: Warn if an ext filesystem affected by the Y2038 problem is used 2021-02-19 11:36 ` Richard Purdie @ 2021-02-22 11:42 ` florian.bezdeka 2021-02-22 15:59 ` Richard Purdie 0 siblings, 1 reply; 11+ messages in thread From: florian.bezdeka @ 2021-02-22 11:42 UTC (permalink / raw) To: openembedded-core Cc: richard.purdie, henning.schild, jan.kiszka, florian.bezdeka From: Florian Bezdeka <florian.bezdeka@siemens.com> We are getting closer and closer to the year 2038 where the 32 bit time_t overflow will happen. While products (= embedded systems) with an expected life time of 15 years are still save the situation may change if your system has to survive the next 20 years. ext2 and ext3 file systems are always affected by the time overflow, so let's warn the user if these file systems are still being used. If ext4 is affected depends on the inode size chosen during file system creation. At least 256 bytes are necessary to be safe. As ext4 is used very often (and partitions may be created small first and extended later) this might be an issue for many users. One partition of mkefidisk.wks was already affected by the Y2038 problem. By using `--mkfs-extraopts "-T default"` we tell mke2fs to use the default usage-type, so disabling the auto-detection of some tuning parameters that were dependent on the file system size. Signed-off-by: Florian Bezdeka <florian.bezdeka@siemens.com> --- Changes in v2: - Fixed a warning reported by CI system coming from a partition in /scripts/lib/wic/canned-wks/mkefidisk.wks. One of the partitions was already affected by the Y2038 problem. scripts/lib/wic/canned-wks/mkefidisk.wks | 2 +- scripts/lib/wic/misc.py | 1 + scripts/lib/wic/partition.py | 38 ++++++++++++++++++++++++ 3 files changed, 40 insertions(+), 1 deletion(-) diff --git a/scripts/lib/wic/canned-wks/mkefidisk.wks b/scripts/lib/wic/canned-wks/mkefidisk.wks index 9f534fe184..d1878e23e5 100644 --- a/scripts/lib/wic/canned-wks/mkefidisk.wks +++ b/scripts/lib/wic/canned-wks/mkefidisk.wks @@ -4,7 +4,7 @@ part /boot --source bootimg-efi --sourceparams="loader=grub-efi" --ondisk sda --label msdos --active --align 1024 -part / --source rootfs --ondisk sda --fstype=ext4 --label platform --align 1024 --use-uuid +part / --source rootfs --ondisk sda --fstype=ext4 --mkfs-extraopts "-T default" --label platform --align 1024 --use-uuid part swap --ondisk sda --size 44 --label swap1 --fstype=swap diff --git a/scripts/lib/wic/misc.py b/scripts/lib/wic/misc.py index 75b219cd3f..57c042c503 100644 --- a/scripts/lib/wic/misc.py +++ b/scripts/lib/wic/misc.py @@ -26,6 +26,7 @@ logger = logging.getLogger('wic') # executable -> recipe pairs for exec_native_cmd NATIVE_RECIPES = {"bmaptool": "bmap-tools", + "dumpe2fs": "e2fsprogs", "grub-mkimage": "grub-efi", "isohybrid": "syslinux", "mcopy": "mtools", diff --git a/scripts/lib/wic/partition.py b/scripts/lib/wic/partition.py index e574f40c47..85f9847047 100644 --- a/scripts/lib/wic/partition.py +++ b/scripts/lib/wic/partition.py @@ -298,6 +298,8 @@ class Partition(): mkfs_cmd = "fsck.%s -pvfD %s" % (self.fstype, rootfs) exec_native_cmd(mkfs_cmd, native_sysroot, pseudo=pseudo) + self.check_for_Y2038_problem(rootfs, native_sysroot) + def prepare_rootfs_btrfs(self, rootfs, cr_workdir, oe_builddir, rootfs_dir, native_sysroot, pseudo): """ @@ -388,6 +390,8 @@ class Partition(): (self.fstype, extraopts, label_str, self.fsuuid, rootfs) exec_native_cmd(mkfs_cmd, native_sysroot) + self.check_for_Y2038_problem(rootfs, native_sysroot) + def prepare_empty_partition_btrfs(self, rootfs, oe_builddir, native_sysroot): """ @@ -449,3 +453,37 @@ class Partition(): mkswap_cmd = "mkswap %s -U %s %s" % (label_str, self.fsuuid, path) exec_native_cmd(mkswap_cmd, native_sysroot) + + def check_for_Y2038_problem(self, rootfs, native_sysroot): + """ + Check if the filesystem is affected by the Y2038 problem + (Y2038 problem = 32 bit time_t overflow in January 2038) + """ + def get_err_str(part): + err = "The {} filesystem {} has no Y2038 support." + if part.mountpoint: + args = [part.fstype, "mounted at %s" % part.mountpoint] + elif part.label: + args = [part.fstype, "labeled '%s'" % part.label] + elif part.part_name: + args = [part.fstype, "in partition '%s'" % part.part_name] + else: + args = [part.fstype, "in partition %s" % part.num] + return err.format(*args) + + # ext2 and ext3 are always affected by the Y2038 problem + if self.fstype in ["ext2", "ext3"]: + logger.warn(get_err_str(self)) + return + + ret, out = exec_native_cmd("dumpe2fs %s" % rootfs, native_sysroot) + + # if ext4 is affected by the Y2038 problem depends on the inode size + for line in out.splitlines(): + if line.startswith("Inode size:"): + size = int(line.split(":")[1].strip()) + if size < 256: + logger.warn("%s Inodes (of size %d) are too small." % + (get_err_str(self), size)) + break + -- 2.29.2 ^ permalink raw reply related [flat|nested] 11+ messages in thread
* Re: [OE-core][PATCH v2] wic: Warn if an ext filesystem affected by the Y2038 problem is used 2021-02-22 11:42 ` [OE-core][PATCH v2] wic: Warn if an ext filesystem affected by the Y2038 problem is used florian.bezdeka @ 2021-02-22 15:59 ` Richard Purdie 2021-02-22 16:31 ` florian.bezdeka 2021-02-23 15:30 ` [OE-core][PATCH v3] " florian.bezdeka 0 siblings, 2 replies; 11+ messages in thread From: Richard Purdie @ 2021-02-22 15:59 UTC (permalink / raw) To: florian.bezdeka, openembedded-core; +Cc: henning.schild, jan.kiszka On Mon, 2021-02-22 at 11:42 +0000, florian.bezdeka@siemens.com wrote: > From: Florian Bezdeka <florian.bezdeka@siemens.com> > > We are getting closer and closer to the year 2038 where the 32 bit > time_t overflow will happen. While products (= embedded systems) with an > expected life time of 15 years are still save the situation may change > if your system has to survive the next 20 years. > > ext2 and ext3 file systems are always affected by the time overflow, so > let's warn the user if these file systems are still being used. > > If ext4 is affected depends on the inode size chosen during file system > creation. At least 256 bytes are necessary to be safe. As ext4 is > used very often (and partitions may be created small first and extended > later) this might be an issue for many users. > > One partition of mkefidisk.wks was already affected by the Y2038 > problem. By using `--mkfs-extraopts "-T default"` we tell mke2fs to use the > default usage-type, so disabling the auto-detection of some tuning > parameters that were dependent on the file system size. > > Signed-off-by: Florian Bezdeka <florian.bezdeka@siemens.com> > --- > > Changes in v2: > - Fixed a warning reported by CI system coming from a partition in > /scripts/lib/wic/canned-wks/mkefidisk.wks. One of the partitions was > already affected by the Y2038 problem. I added this and reran but its still showing some warnings unfortunately: https://autobuilder.yoctoproject.org/typhoon/#/builders/58/builds/3065 Cheers, Richard ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [OE-core][PATCH v2] wic: Warn if an ext filesystem affected by the Y2038 problem is used 2021-02-22 15:59 ` Richard Purdie @ 2021-02-22 16:31 ` florian.bezdeka 2021-02-23 15:30 ` [OE-core][PATCH v3] " florian.bezdeka 1 sibling, 0 replies; 11+ messages in thread From: florian.bezdeka @ 2021-02-22 16:31 UTC (permalink / raw) To: richard.purdie, openembedded-core On Mon, 2021-02-22 at 15:59 +0000, Richard Purdie wrote: > On Mon, 2021-02-22 at 11:42 +0000, florian.bezdeka@siemens.com wrote: > > From: Florian Bezdeka <florian.bezdeka@siemens.com> > > > > We are getting closer and closer to the year 2038 where the 32 bit > > time_t overflow will happen. While products (= embedded systems) with an > > expected life time of 15 years are still save the situation may change > > if your system has to survive the next 20 years. > > > > ext2 and ext3 file systems are always affected by the time overflow, so > > let's warn the user if these file systems are still being used. > > > > If ext4 is affected depends on the inode size chosen during file system > > creation. At least 256 bytes are necessary to be safe. As ext4 is > > used very often (and partitions may be created small first and extended > > later) this might be an issue for many users. > > > > One partition of mkefidisk.wks was already affected by the Y2038 > > problem. By using `--mkfs-extraopts "-T default"` we tell mke2fs to use the > > default usage-type, so disabling the auto-detection of some tuning > > parameters that were dependent on the file system size. > > > > Signed-off-by: Florian Bezdeka <florian.bezdeka@siemens.com> > > --- > > > > Changes in v2: > > - Fixed a warning reported by CI system coming from a partition in > > /scripts/lib/wic/canned-wks/mkefidisk.wks. One of the partitions was > > already affected by the Y2038 problem. > > I added this and reran but its still showing some warnings unfortunately: > > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fautobuilder.yoctoproject.org%2Ftyphoon%2F%23%2Fbuilders%2F58%2Fbuilds%2F3065&data=04%7C01%7Cflorian.bezdeka%40siemens.com%7C9fd84254f5d74deaec6208d8d74d5279%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637496074381666346%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=g32gFNEDvmIK2me%2BKKJEBwEIcdGMffd4TuhIMjmrZtU%3D&reserved=0 > I see... Is it possible to see `/etc/mke2fs.conf` from within the buildchroot that is actually calling `wic` somehow? > Cheers, > > Richard > ^ permalink raw reply [flat|nested] 11+ messages in thread
* [OE-core][PATCH v3] wic: Warn if an ext filesystem affected by the Y2038 problem is used 2021-02-22 15:59 ` Richard Purdie 2021-02-22 16:31 ` florian.bezdeka @ 2021-02-23 15:30 ` florian.bezdeka 2021-02-24 16:54 ` Richard Purdie 1 sibling, 1 reply; 11+ messages in thread From: florian.bezdeka @ 2021-02-23 15:30 UTC (permalink / raw) To: openembedded-core; +Cc: florian.bezdeka From: Florian Bezdeka <florian.bezdeka@siemens.com> We are getting closer and closer to the year 2038 where the 32 bit time_t overflow will happen. While products (= embedded systems) with an expected life time of 15 years are still save the situation may change if your system has to survive the next 20 years. ext2 and ext3 file systems are always affected by the time overflow, so let's warn the user if these file systems are still being used. If ext4 is affected depends on the inode size chosen during file system creation. At least 256 bytes are necessary to be safe. As ext4 is used very often (and partitions may be created small first and extended later) this might be an issue for many users. One partition of mkefidisk.wks was already affected by the Y2038 problem. By using `--mkfs-extraopts "-I 256"` we tell mke2fs to use inodes of size 256 bytes to be Y2038 safe. Signed-off-by: Florian Bezdeka <florian.bezdeka@siemens.com> --- Changes in v2: - Fixed a warning reported by CI system coming from a partition in /scripts/lib/wic/canned-wks/mkefidisk.wks. One of the partitions was already affected by the Y2038 problem. Changes in v3: - Moved to "-I 256" to become independent from the /etc/mke2fs.conf file (or even from the mke2fs compiled in defaults) scripts/lib/wic/canned-wks/mkefidisk.wks | 2 +- scripts/lib/wic/misc.py | 1 + scripts/lib/wic/partition.py | 38 ++++++++++++++++++++++++ 3 files changed, 40 insertions(+), 1 deletion(-) diff --git a/scripts/lib/wic/canned-wks/mkefidisk.wks b/scripts/lib/wic/canned-wks/mkefidisk.wks index 9f534fe184..627a03beca 100644 --- a/scripts/lib/wic/canned-wks/mkefidisk.wks +++ b/scripts/lib/wic/canned-wks/mkefidisk.wks @@ -4,7 +4,7 @@ part /boot --source bootimg-efi --sourceparams="loader=grub-efi" --ondisk sda --label msdos --active --align 1024 -part / --source rootfs --ondisk sda --fstype=ext4 --label platform --align 1024 --use-uuid +part / --source rootfs --ondisk sda --fstype=ext4 --mkfs-extraopts "-I 256" --label platform --align 1024 --use-uuid part swap --ondisk sda --size 44 --label swap1 --fstype=swap diff --git a/scripts/lib/wic/misc.py b/scripts/lib/wic/misc.py index 75b219cd3f..57c042c503 100644 --- a/scripts/lib/wic/misc.py +++ b/scripts/lib/wic/misc.py @@ -26,6 +26,7 @@ logger = logging.getLogger('wic') # executable -> recipe pairs for exec_native_cmd NATIVE_RECIPES = {"bmaptool": "bmap-tools", + "dumpe2fs": "e2fsprogs", "grub-mkimage": "grub-efi", "isohybrid": "syslinux", "mcopy": "mtools", diff --git a/scripts/lib/wic/partition.py b/scripts/lib/wic/partition.py index e574f40c47..85f9847047 100644 --- a/scripts/lib/wic/partition.py +++ b/scripts/lib/wic/partition.py @@ -298,6 +298,8 @@ class Partition(): mkfs_cmd = "fsck.%s -pvfD %s" % (self.fstype, rootfs) exec_native_cmd(mkfs_cmd, native_sysroot, pseudo=pseudo) + self.check_for_Y2038_problem(rootfs, native_sysroot) + def prepare_rootfs_btrfs(self, rootfs, cr_workdir, oe_builddir, rootfs_dir, native_sysroot, pseudo): """ @@ -388,6 +390,8 @@ class Partition(): (self.fstype, extraopts, label_str, self.fsuuid, rootfs) exec_native_cmd(mkfs_cmd, native_sysroot) + self.check_for_Y2038_problem(rootfs, native_sysroot) + def prepare_empty_partition_btrfs(self, rootfs, oe_builddir, native_sysroot): """ @@ -449,3 +453,37 @@ class Partition(): mkswap_cmd = "mkswap %s -U %s %s" % (label_str, self.fsuuid, path) exec_native_cmd(mkswap_cmd, native_sysroot) + + def check_for_Y2038_problem(self, rootfs, native_sysroot): + """ + Check if the filesystem is affected by the Y2038 problem + (Y2038 problem = 32 bit time_t overflow in January 2038) + """ + def get_err_str(part): + err = "The {} filesystem {} has no Y2038 support." + if part.mountpoint: + args = [part.fstype, "mounted at %s" % part.mountpoint] + elif part.label: + args = [part.fstype, "labeled '%s'" % part.label] + elif part.part_name: + args = [part.fstype, "in partition '%s'" % part.part_name] + else: + args = [part.fstype, "in partition %s" % part.num] + return err.format(*args) + + # ext2 and ext3 are always affected by the Y2038 problem + if self.fstype in ["ext2", "ext3"]: + logger.warn(get_err_str(self)) + return + + ret, out = exec_native_cmd("dumpe2fs %s" % rootfs, native_sysroot) + + # if ext4 is affected by the Y2038 problem depends on the inode size + for line in out.splitlines(): + if line.startswith("Inode size:"): + size = int(line.split(":")[1].strip()) + if size < 256: + logger.warn("%s Inodes (of size %d) are too small." % + (get_err_str(self), size)) + break + -- 2.29.2 ^ permalink raw reply related [flat|nested] 11+ messages in thread
* Re: [OE-core][PATCH v3] wic: Warn if an ext filesystem affected by the Y2038 problem is used 2021-02-23 15:30 ` [OE-core][PATCH v3] " florian.bezdeka @ 2021-02-24 16:54 ` Richard Purdie 2021-02-25 8:22 ` [OE-core][PATCH v4] " florian.bezdeka 0 siblings, 1 reply; 11+ messages in thread From: Richard Purdie @ 2021-02-24 16:54 UTC (permalink / raw) To: florian.bezdeka, openembedded-core On Tue, 2021-02-23 at 15:30 +0000, florian.bezdeka@siemens.com wrote: > From: Florian Bezdeka <florian.bezdeka@siemens.com> > > We are getting closer and closer to the year 2038 where the 32 bit > time_t overflow will happen. While products (= embedded systems) with an > expected life time of 15 years are still save the situation may change > if your system has to survive the next 20 years. > > ext2 and ext3 file systems are always affected by the time overflow, so > let's warn the user if these file systems are still being used. > > If ext4 is affected depends on the inode size chosen during file system > creation. At least 256 bytes are necessary to be safe. As ext4 is > used very often (and partitions may be created small first and extended > later) this might be an issue for many users. > > One partition of mkefidisk.wks was already affected by the Y2038 > problem. By using `--mkfs-extraopts "-I 256"` we tell mke2fs to use > inodes of size 256 bytes to be Y2038 safe. > > Signed-off-by: Florian Bezdeka <florian.bezdeka@siemens.com> > --- > > > Changes in v2: > - Fixed a warning reported by CI system coming from a partition in > /scripts/lib/wic/canned-wks/mkefidisk.wks. One of the partitions was > already affected by the Y2038 problem. > > Changes in v3: > - Moved to "-I 256" to become independent from the /etc/mke2fs.conf > file (or even from the mke2fs compiled in defaults) CI tests are still showing the warnings unfortunately: https://autobuilder.yoctoproject.org/typhoon/#/builders/58/builds/3079 Cheers, Richard ^ permalink raw reply [flat|nested] 11+ messages in thread
* [OE-core][PATCH v4] wic: Warn if an ext filesystem affected by the Y2038 problem is used 2021-02-24 16:54 ` Richard Purdie @ 2021-02-25 8:22 ` florian.bezdeka 0 siblings, 0 replies; 11+ messages in thread From: florian.bezdeka @ 2021-02-25 8:22 UTC (permalink / raw) To: openembedded-core, richard.purdie Cc: henning.schild, jan.kiszka, Florian Bezdeka We are getting closer and closer to the year 2038 where the 32 bit time_t overflow will happen. While products (= embedded systems) with an expected life time of 15 years are still save the situation may change if your system has to survive the next 20 years. ext2 and ext3 filesystems are always affected by the time overflow, so let's warn the user if these filesystems are still being used. If ext4 is affected depends on the inode size chosen during filesystem creation. At least 256 bytes are necessary to be safe. As ext4 is used very often (and partitions may be created small first and extended later) this might be an issue for many users. Some filesystems created during CI runs were already affected by the Y2038 problem. By using `--mkfs-extraopts "-T default"` we tell mke2fs not to auto-detect the usage type based on the filesystem size. mke2fs will use the default values for tuning parameters instead. The inode size is one of these parameters. Signed-off-by: Florian Bezdeka <florian.bezdeka@siemens.com> --- Changes in v4: - Updated the commit message to reflect current state - Fixed more affected filesystems that were reported by the CI system (directdisk-gpt.wks and directdisk.wks) - Moving back to "-T default", so disabling usage-type detection based on the filesystem size Changes in v3: - Moved to "-I 256" to become independent from the /etc/mke2fs.conf file (or even from the mke2fs compiled in defaults) Changes in v2: - Fixed a warning reported by CI system coming from a partition in /scripts/lib/wic/canned-wks/mkefidisk.wks. One of the partitions was already affected by the Y2038 problem. scripts/lib/wic/canned-wks/common.wks.inc | 2 +- scripts/lib/wic/canned-wks/directdisk-gpt.wks | 2 +- scripts/lib/wic/canned-wks/mkefidisk.wks | 2 +- scripts/lib/wic/misc.py | 1 + scripts/lib/wic/partition.py | 38 +++++++++++++++++++ 5 files changed, 42 insertions(+), 3 deletions(-) diff --git a/scripts/lib/wic/canned-wks/common.wks.inc b/scripts/lib/wic/canned-wks/common.wks.inc index 89880b417b..4fd29fa8c1 100644 --- a/scripts/lib/wic/canned-wks/common.wks.inc +++ b/scripts/lib/wic/canned-wks/common.wks.inc @@ -1,3 +1,3 @@ # This file is included into 3 canned wks files from this directory part /boot --source bootimg-pcbios --ondisk sda --label boot --active --align 1024 -part / --source rootfs --use-uuid --fstype=ext4 --label platform --align 1024 +part / --source rootfs --use-uuid --fstype=ext4 --mkfs-extraopts "-T default" --label platform --align 1024 diff --git a/scripts/lib/wic/canned-wks/directdisk-gpt.wks b/scripts/lib/wic/canned-wks/directdisk-gpt.wks index 8d7d8de6ea..cf16c0c30b 100644 --- a/scripts/lib/wic/canned-wks/directdisk-gpt.wks +++ b/scripts/lib/wic/canned-wks/directdisk-gpt.wks @@ -4,7 +4,7 @@ part /boot --source bootimg-pcbios --ondisk sda --label boot --active --align 1024 -part / --source rootfs --ondisk sda --fstype=ext4 --label platform --align 1024 --use-uuid +part / --source rootfs --ondisk sda --fstype=ext4 --mkfs-extraopts "-T default" --label platform --align 1024 --use-uuid bootloader --ptable gpt --timeout=0 --append="rootwait rootfstype=ext4 video=vesafb vga=0x318 console=tty0 console=ttyS0,115200n8" diff --git a/scripts/lib/wic/canned-wks/mkefidisk.wks b/scripts/lib/wic/canned-wks/mkefidisk.wks index 9f534fe184..d1878e23e5 100644 --- a/scripts/lib/wic/canned-wks/mkefidisk.wks +++ b/scripts/lib/wic/canned-wks/mkefidisk.wks @@ -4,7 +4,7 @@ part /boot --source bootimg-efi --sourceparams="loader=grub-efi" --ondisk sda --label msdos --active --align 1024 -part / --source rootfs --ondisk sda --fstype=ext4 --label platform --align 1024 --use-uuid +part / --source rootfs --ondisk sda --fstype=ext4 --mkfs-extraopts "-T default" --label platform --align 1024 --use-uuid part swap --ondisk sda --size 44 --label swap1 --fstype=swap diff --git a/scripts/lib/wic/misc.py b/scripts/lib/wic/misc.py index 75b219cd3f..57c042c503 100644 --- a/scripts/lib/wic/misc.py +++ b/scripts/lib/wic/misc.py @@ -26,6 +26,7 @@ logger = logging.getLogger('wic') # executable -> recipe pairs for exec_native_cmd NATIVE_RECIPES = {"bmaptool": "bmap-tools", + "dumpe2fs": "e2fsprogs", "grub-mkimage": "grub-efi", "isohybrid": "syslinux", "mcopy": "mtools", diff --git a/scripts/lib/wic/partition.py b/scripts/lib/wic/partition.py index e574f40c47..85f9847047 100644 --- a/scripts/lib/wic/partition.py +++ b/scripts/lib/wic/partition.py @@ -298,6 +298,8 @@ class Partition(): mkfs_cmd = "fsck.%s -pvfD %s" % (self.fstype, rootfs) exec_native_cmd(mkfs_cmd, native_sysroot, pseudo=pseudo) + self.check_for_Y2038_problem(rootfs, native_sysroot) + def prepare_rootfs_btrfs(self, rootfs, cr_workdir, oe_builddir, rootfs_dir, native_sysroot, pseudo): """ @@ -388,6 +390,8 @@ class Partition(): (self.fstype, extraopts, label_str, self.fsuuid, rootfs) exec_native_cmd(mkfs_cmd, native_sysroot) + self.check_for_Y2038_problem(rootfs, native_sysroot) + def prepare_empty_partition_btrfs(self, rootfs, oe_builddir, native_sysroot): """ @@ -449,3 +453,37 @@ class Partition(): mkswap_cmd = "mkswap %s -U %s %s" % (label_str, self.fsuuid, path) exec_native_cmd(mkswap_cmd, native_sysroot) + + def check_for_Y2038_problem(self, rootfs, native_sysroot): + """ + Check if the filesystem is affected by the Y2038 problem + (Y2038 problem = 32 bit time_t overflow in January 2038) + """ + def get_err_str(part): + err = "The {} filesystem {} has no Y2038 support." + if part.mountpoint: + args = [part.fstype, "mounted at %s" % part.mountpoint] + elif part.label: + args = [part.fstype, "labeled '%s'" % part.label] + elif part.part_name: + args = [part.fstype, "in partition '%s'" % part.part_name] + else: + args = [part.fstype, "in partition %s" % part.num] + return err.format(*args) + + # ext2 and ext3 are always affected by the Y2038 problem + if self.fstype in ["ext2", "ext3"]: + logger.warn(get_err_str(self)) + return + + ret, out = exec_native_cmd("dumpe2fs %s" % rootfs, native_sysroot) + + # if ext4 is affected by the Y2038 problem depends on the inode size + for line in out.splitlines(): + if line.startswith("Inode size:"): + size = int(line.split(":")[1].strip()) + if size < 256: + logger.warn("%s Inodes (of size %d) are too small." % + (get_err_str(self), size)) + break + -- 2.29.2 ^ permalink raw reply related [flat|nested] 11+ messages in thread
end of thread, other threads:[~2021-02-25 8:22 UTC | newest] Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2021-02-18 13:06 [OE-core][PATCH 0/1] wic: warn on usage of Y2038 affected file systems florian.bezdeka 2021-02-18 13:06 ` [OE-core][PATCH 1/1] wic: Warn if an ext filesystem affected by the Y2038 problem is used florian.bezdeka 2021-02-19 10:55 ` [OE-core][PATCH 0/1] wic: warn on usage of Y2038 affected file systems Richard Purdie 2021-02-19 11:19 ` florian.bezdeka 2021-02-19 11:36 ` Richard Purdie 2021-02-22 11:42 ` [OE-core][PATCH v2] wic: Warn if an ext filesystem affected by the Y2038 problem is used florian.bezdeka 2021-02-22 15:59 ` Richard Purdie 2021-02-22 16:31 ` florian.bezdeka 2021-02-23 15:30 ` [OE-core][PATCH v3] " florian.bezdeka 2021-02-24 16:54 ` Richard Purdie 2021-02-25 8:22 ` [OE-core][PATCH v4] " florian.bezdeka
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.