Andreas, thanks for such a prompt reply. On Fri, 12 Nov 2021, Andreas Dilger wrote: > On Nov 12, 2021, at 11:37, Mark Hills wrote: > > > > Surprised to hit a limit when handling a modest Maildir case; does > > this reflect a bug? > > > > rsync'ing to a new mail server, after fewer than 100,000 files there > > are intermittent failures: > > This is probably because you are using 1KB blocksize instead of 4KB, > which reduces the size of each tree level by the cube of the ratio, so > 64x. I guess that was selected because of very small files in the > maildir? Interesting! The 1Kb block size was not explicitly chosen. There was no plan other than using the defaults. However I did forget that this is a VM installed from a base image. The root cause is likely to be that the /home partition has been enlarged from a small size to 32Gb. Is block size the only factor? If so, a patch like below (untested) could make it clear it's relevant, and saved the question in this case. [...] > If you have a relatively recent kernel, you can enable the "large_dir" > feature to allow 3-level htree, which would be enough for another factor > of 1024/8 = 128 more entries than now (~12M). The system is not yet in use, so I think it's better we reformat here, and get a block size chosen by the experts :) These days I think VMs make it more common to enlarge a filesystem from a small size. We could have picked this up earlier with a warning from resize2fs; eg. if the block size will no longer match the one that would be chosen by default. That would pick it up before anyone puts 1Kb block size into production. Thanks for identifying the issue. -- Mark From 8604c50be77a4bc56a91099598c409d5a3c1fdbe Mon Sep 17 00:00:00 2001 From: Mark Hills Date: Sat, 13 Nov 2021 11:46:50 +0000 Subject: [PATCH] Block size has an effect on the index size --- fs/ext4/namei.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/fs/ext4/namei.c b/fs/ext4/namei.c index f3bbcd4efb56..8965bed4d7ff 100644 --- a/fs/ext4/namei.c +++ b/fs/ext4/namei.c @@ -2454,8 +2454,9 @@ static int ext4_dx_add_entry(handle_t *handle, struct ext4_filename *fname, } if (add_level && levels == ext4_dir_htree_level(sb)) { ext4_warning(sb, "Directory (ino: %lu) index full, " - "reach max htree level :%d", - dir->i_ino, levels); + "reach max htree level :%d" + "with block size %lu", + dir->i_ino, levels, sb->s_blocksize); if (ext4_dir_htree_level(sb) < EXT4_HTREE_LEVEL) { ext4_warning(sb, "Large directory feature is " "not enabled on this " -- 2.33.1