* [PATCH 0/9 v4] e2fsprogs: Support for orphan file feature
@ 2021-07-12 15:43 Jan Kara
2021-07-12 15:43 ` [PATCH 1/9] ext2fs: Drop HAS_SNAPSHOT feature Jan Kara
` (8 more replies)
0 siblings, 9 replies; 22+ messages in thread
From: Jan Kara @ 2021-07-12 15:43 UTC (permalink / raw)
To: Ted Tso; +Cc: linux-ext4, Jan Kara
Hello,
This is the fourth version of support for orphan file feature in e2fsprogs.
The first patch removes feature bit for a feature which never happened and
probably isn't living anywhere (if it still lives I can allocate a different
bit...). Other two patches are fixes I've spotted while debugging this series.
I'm not aware of any outstanding issue with the series, xfstests pass fine with
it.
Changes since v3:
* updated to compute checksum also from physical block
Changes since v2:
* rebased onto current master branch
* fixed various bugs I've spotted during testing
* added support for debugfs, dumpe2fs, e2image
* changed code to use dynamically allocated inode number instead of fixed one
Honza
^ permalink raw reply [flat|nested] 22+ messages in thread
* [PATCH 1/9] ext2fs: Drop HAS_SNAPSHOT feature
2021-07-12 15:43 [PATCH 0/9 v4] e2fsprogs: Support for orphan file feature Jan Kara
@ 2021-07-12 15:43 ` Jan Kara
2021-07-12 18:20 ` Andreas Dilger
2021-07-12 15:43 ` [PATCH 2/9] quota: Do not account space used by project quota file to quota Jan Kara
` (7 subsequent siblings)
8 siblings, 1 reply; 22+ messages in thread
From: Jan Kara @ 2021-07-12 15:43 UTC (permalink / raw)
To: Ted Tso; +Cc: linux-ext4, Jan Kara
It has never been implemented and is dead for quite some time and
unused AFAICT.
Signed-off-by: Jan Kara <jack@suse.cz>
---
lib/ext2fs/ext2_fs.h | 2 --
1 file changed, 2 deletions(-)
diff --git a/lib/ext2fs/ext2_fs.h b/lib/ext2fs/ext2_fs.h
index e92a045205a9..6f1d5db4b482 100644
--- a/lib/ext2fs/ext2_fs.h
+++ b/lib/ext2fs/ext2_fs.h
@@ -825,7 +825,6 @@ struct ext2_super_block {
#define EXT4_FEATURE_RO_COMPAT_GDT_CSUM 0x0010
#define EXT4_FEATURE_RO_COMPAT_DIR_NLINK 0x0020
#define EXT4_FEATURE_RO_COMPAT_EXTRA_ISIZE 0x0040
-#define EXT4_FEATURE_RO_COMPAT_HAS_SNAPSHOT 0x0080
#define EXT4_FEATURE_RO_COMPAT_QUOTA 0x0100
#define EXT4_FEATURE_RO_COMPAT_BIGALLOC 0x0200
/*
@@ -926,7 +925,6 @@ EXT4_FEATURE_RO_COMPAT_FUNCS(huge_file, 4, HUGE_FILE)
EXT4_FEATURE_RO_COMPAT_FUNCS(gdt_csum, 4, GDT_CSUM)
EXT4_FEATURE_RO_COMPAT_FUNCS(dir_nlink, 4, DIR_NLINK)
EXT4_FEATURE_RO_COMPAT_FUNCS(extra_isize, 4, EXTRA_ISIZE)
-EXT4_FEATURE_RO_COMPAT_FUNCS(has_snapshot, 4, HAS_SNAPSHOT)
EXT4_FEATURE_RO_COMPAT_FUNCS(quota, 4, QUOTA)
EXT4_FEATURE_RO_COMPAT_FUNCS(bigalloc, 4, BIGALLOC)
EXT4_FEATURE_RO_COMPAT_FUNCS(metadata_csum, 4, METADATA_CSUM)
--
2.26.2
^ permalink raw reply related [flat|nested] 22+ messages in thread
* [PATCH 2/9] quota: Do not account space used by project quota file to quota
2021-07-12 15:43 [PATCH 0/9 v4] e2fsprogs: Support for orphan file feature Jan Kara
2021-07-12 15:43 ` [PATCH 1/9] ext2fs: Drop HAS_SNAPSHOT feature Jan Kara
@ 2021-07-12 15:43 ` Jan Kara
2021-07-12 15:43 ` [PATCH 3/9] e2image: Dump quota files Jan Kara
` (6 subsequent siblings)
8 siblings, 0 replies; 22+ messages in thread
From: Jan Kara @ 2021-07-12 15:43 UTC (permalink / raw)
To: Ted Tso; +Cc: linux-ext4, Jan Kara
Project quota files have high inode numbers but are not accounted in
quota usage. Do not track them when computing quota usage.
Signed-off-by: Jan Kara <jack@suse.cz>
---
lib/support/mkquota.c | 8 +++++---
1 file changed, 5 insertions(+), 3 deletions(-)
diff --git a/lib/support/mkquota.c b/lib/support/mkquota.c
index fbc3833aee98..21a5c34d6921 100644
--- a/lib/support/mkquota.c
+++ b/lib/support/mkquota.c
@@ -500,9 +500,11 @@ errcode_t quota_compute_usage(quota_ctx_t qctx)
}
if (ino == 0)
break;
- if (inode->i_links_count &&
- (ino == EXT2_ROOT_INO ||
- ino >= EXT2_FIRST_INODE(fs->super))) {
+ if (!inode->i_links_count)
+ continue;
+ if (ino == EXT2_ROOT_INO ||
+ (ino >= EXT2_FIRST_INODE(fs->super) &&
+ ino != quota_type2inum(PRJQUOTA, fs->super))) {
space = ext2fs_get_stat_i_blocks(fs,
EXT2_INODE(inode)) << 9;
quota_data_add(qctx, inode, ino, space);
--
2.26.2
^ permalink raw reply related [flat|nested] 22+ messages in thread
* [PATCH 3/9] e2image: Dump quota files
2021-07-12 15:43 [PATCH 0/9 v4] e2fsprogs: Support for orphan file feature Jan Kara
2021-07-12 15:43 ` [PATCH 1/9] ext2fs: Drop HAS_SNAPSHOT feature Jan Kara
2021-07-12 15:43 ` [PATCH 2/9] quota: Do not account space used by project quota file to quota Jan Kara
@ 2021-07-12 15:43 ` Jan Kara
2021-07-12 15:43 ` [PATCH 4/9] libext2fs: Support for orphan file feature Jan Kara
` (5 subsequent siblings)
8 siblings, 0 replies; 22+ messages in thread
From: Jan Kara @ 2021-07-12 15:43 UTC (permalink / raw)
To: Ted Tso; +Cc: linux-ext4, Jan Kara
Dump quota files to resulting filesystem image. They are fs metadata.
Signed-off-by: Jan Kara <jack@suse.cz>
---
misc/e2image.c | 9 ++++++---
1 file changed, 6 insertions(+), 3 deletions(-)
diff --git a/misc/e2image.c b/misc/e2image.c
index 90a34bebc36d..ac00827e4628 100644
--- a/misc/e2image.c
+++ b/misc/e2image.c
@@ -52,6 +52,7 @@ extern int optind;
#include "support/nls-enable.h"
#include "support/plausible.h"
+#include "support/quotaio.h"
#include "../version.h"
#define QCOW_OFLAG_COPIED (1ULL << 63)
@@ -1364,9 +1365,11 @@ static void write_raw_image_file(ext2_filsys fs, int fd, int type, int flags,
pb.ino = ino;
pb.is_dir = LINUX_S_ISDIR(inode.i_mode);
if (LINUX_S_ISDIR(inode.i_mode) ||
- (LINUX_S_ISLNK(inode.i_mode) &&
- ext2fs_inode_has_valid_blocks2(fs, &inode)) ||
- ino == fs->super->s_journal_inum) {
+ LINUX_S_ISLNK(inode.i_mode) ||
+ ino == fs->super->s_journal_inum ||
+ ino == quota_type2inum(USRQUOTA, fs->super) ||
+ ino == quota_type2inum(GRPQUOTA, fs->super) ||
+ ino == quota_type2inum(PRJQUOTA, fs->super)) {
retval = ext2fs_block_iterate3(fs, ino,
BLOCK_FLAG_READ_ONLY, block_buf,
process_dir_block, &pb);
--
2.26.2
^ permalink raw reply related [flat|nested] 22+ messages in thread
* [PATCH 4/9] libext2fs: Support for orphan file feature
2021-07-12 15:43 [PATCH 0/9 v4] e2fsprogs: Support for orphan file feature Jan Kara
` (2 preceding siblings ...)
2021-07-12 15:43 ` [PATCH 3/9] e2image: Dump quota files Jan Kara
@ 2021-07-12 15:43 ` Jan Kara
2021-08-03 16:57 ` Theodore Ts'o
2021-07-12 15:43 ` [PATCH 5/9] mke2fs: Add support for orphan_file feature Jan Kara
` (4 subsequent siblings)
8 siblings, 1 reply; 22+ messages in thread
From: Jan Kara @ 2021-07-12 15:43 UTC (permalink / raw)
To: Ted Tso; +Cc: linux-ext4, Jan Kara
Add support for creating and deleting orphan file and a couple of
utility functions that will be used in other tools.
Signed-off-by: Jan Kara <jack@suse.cz>
---
lib/e2p/feature.c | 4 +
lib/ext2fs/Makefile.in | 2 +
lib/ext2fs/ext2_fs.h | 16 ++-
lib/ext2fs/ext2fs.h | 36 ++++-
lib/ext2fs/orphan.c | 272 ++++++++++++++++++++++++++++++++++++
lib/ext2fs/swapfs.c | 3 +-
lib/ext2fs/tst_super_size.c | 3 +-
lib/support/mkquota.c | 3 +-
8 files changed, 331 insertions(+), 8 deletions(-)
create mode 100644 lib/ext2fs/orphan.c
diff --git a/lib/e2p/feature.c b/lib/e2p/feature.c
index 2291060214ff..29b7b1512400 100644
--- a/lib/e2p/feature.c
+++ b/lib/e2p/feature.c
@@ -49,6 +49,8 @@ static struct feature feature_list[] = {
"fast_commit" },
{ E2P_FEATURE_COMPAT, EXT4_FEATURE_COMPAT_STABLE_INODES,
"stable_inodes" },
+ { E2P_FEATURE_COMPAT, EXT4_FEATURE_COMPAT_ORPHAN_FILE,
+ "orphan_file" },
{ E2P_FEATURE_RO_INCOMPAT, EXT2_FEATURE_RO_COMPAT_SPARSE_SUPER,
"sparse_super" },
@@ -80,6 +82,8 @@ static struct feature feature_list[] = {
"shared_blocks"},
{ E2P_FEATURE_RO_INCOMPAT, EXT4_FEATURE_RO_COMPAT_VERITY,
"verity"},
+ { E2P_FEATURE_RO_INCOMPAT, EXT4_FEATURE_RO_COMPAT_ORPHAN_PRESENT,
+ "orphan_present" },
{ E2P_FEATURE_INCOMPAT, EXT2_FEATURE_INCOMPAT_COMPRESSION,
"compression" },
diff --git a/lib/ext2fs/Makefile.in b/lib/ext2fs/Makefile.in
index 5d9af86e520b..ffbfd7a7fc33 100644
--- a/lib/ext2fs/Makefile.in
+++ b/lib/ext2fs/Makefile.in
@@ -115,6 +115,7 @@ OBJS= $(DEBUGFS_LIB_OBJS) $(RESIZE_LIB_OBJS) $(E2IMAGE_LIB_OBJS) \
newdir.o \
nls_utf8.o \
openfs.o \
+ orphan.o \
progress.o \
punch.o \
qcow2.o \
@@ -198,6 +199,7 @@ SRCS= ext2_err.c \
$(srcdir)/newdir.c \
$(srcdir)/nls_utf8.c \
$(srcdir)/openfs.c \
+ $(srcdir)/orphan.c \
$(srcdir)/progress.c \
$(srcdir)/punch.c \
$(srcdir)/qcow2.c \
diff --git a/lib/ext2fs/ext2_fs.h b/lib/ext2fs/ext2_fs.h
index 6f1d5db4b482..00809e7b92be 100644
--- a/lib/ext2fs/ext2_fs.h
+++ b/lib/ext2fs/ext2_fs.h
@@ -761,7 +761,8 @@ struct ext2_super_block {
__u8 s_last_error_errcode;
/*27c*/ __le16 s_encoding; /* Filename charset encoding */
__le16 s_encoding_flags; /* Filename charset encoding flags */
- __le32 s_reserved[95]; /* Padding to the end of the block */
+ __le32 s_orphan_file_inum; /* Inode for tracking orphan inodes */
+ __le32 s_reserved[94]; /* Padding to the end of the block */
/*3fc*/ __u32 s_checksum; /* crc32c(superblock) */
};
@@ -816,7 +817,7 @@ struct ext2_super_block {
#define EXT4_FEATURE_COMPAT_SPARSE_SUPER2 0x0200
#define EXT4_FEATURE_COMPAT_FAST_COMMIT 0x0400
#define EXT4_FEATURE_COMPAT_STABLE_INODES 0x0800
-
+#define EXT4_FEATURE_COMPAT_ORPHAN_FILE 0x1000
#define EXT2_FEATURE_RO_COMPAT_SPARSE_SUPER 0x0001
#define EXT2_FEATURE_RO_COMPAT_LARGE_FILE 0x0002
@@ -825,6 +826,7 @@ struct ext2_super_block {
#define EXT4_FEATURE_RO_COMPAT_GDT_CSUM 0x0010
#define EXT4_FEATURE_RO_COMPAT_DIR_NLINK 0x0020
#define EXT4_FEATURE_RO_COMPAT_EXTRA_ISIZE 0x0040
+#define EXT4_FEATURE_RO_COMPAT_ORPHAN_PRESENT 0x0080
#define EXT4_FEATURE_RO_COMPAT_QUOTA 0x0100
#define EXT4_FEATURE_RO_COMPAT_BIGALLOC 0x0200
/*
@@ -918,6 +920,7 @@ EXT4_FEATURE_COMPAT_FUNCS(exclude_bitmap, 2, EXCLUDE_BITMAP)
EXT4_FEATURE_COMPAT_FUNCS(sparse_super2, 4, SPARSE_SUPER2)
EXT4_FEATURE_COMPAT_FUNCS(fast_commit, 4, FAST_COMMIT)
EXT4_FEATURE_COMPAT_FUNCS(stable_inodes, 4, STABLE_INODES)
+EXT4_FEATURE_COMPAT_FUNCS(orphan_file, 4, ORPHAN_FILE)
EXT4_FEATURE_RO_COMPAT_FUNCS(sparse_super, 2, SPARSE_SUPER)
EXT4_FEATURE_RO_COMPAT_FUNCS(large_file, 2, LARGE_FILE)
@@ -933,6 +936,7 @@ EXT4_FEATURE_RO_COMPAT_FUNCS(readonly, 4, READONLY)
EXT4_FEATURE_RO_COMPAT_FUNCS(project, 4, PROJECT)
EXT4_FEATURE_RO_COMPAT_FUNCS(shared_blocks, 4, SHARED_BLOCKS)
EXT4_FEATURE_RO_COMPAT_FUNCS(verity, 4, VERITY)
+EXT4_FEATURE_RO_COMPAT_FUNCS(orphan_present, 4, ORPHAN_PRESENT)
EXT4_FEATURE_INCOMPAT_FUNCS(compression, 2, COMPRESSION)
EXT4_FEATURE_INCOMPAT_FUNCS(filetype, 2, FILETYPE)
@@ -1100,6 +1104,14 @@ static inline unsigned int ext2fs_dir_rec_len(__u8 name_len,
return rec_len;
}
+#define EXT4_ORPHAN_BLOCK_MAGIC 0x0b10ca04
+
+/* Structure at the tail of orphan block */
+struct ext4_orphan_block_tail {
+ __u32 ob_magic;
+ __u32 ob_checksum;
+};
+
/*
* Constants for ext4's extended time encoding
*/
diff --git a/lib/ext2fs/ext2fs.h b/lib/ext2fs/ext2fs.h
index df150f0003f2..b5648004965a 100644
--- a/lib/ext2fs/ext2fs.h
+++ b/lib/ext2fs/ext2fs.h
@@ -620,7 +620,8 @@ typedef struct ext2_icount *ext2_icount_t;
EXT2_FEATURE_COMPAT_EXT_ATTR|\
EXT4_FEATURE_COMPAT_SPARSE_SUPER2|\
EXT4_FEATURE_COMPAT_FAST_COMMIT|\
- EXT4_FEATURE_COMPAT_STABLE_INODES)
+ EXT4_FEATURE_COMPAT_STABLE_INODES|\
+ EXT4_FEATURE_COMPAT_ORPHAN_FILE)
#ifdef CONFIG_MMP
#define EXT4_LIB_INCOMPAT_MMP EXT4_FEATURE_INCOMPAT_MMP
@@ -655,7 +656,8 @@ typedef struct ext2_icount *ext2_icount_t;
EXT4_FEATURE_RO_COMPAT_READONLY |\
EXT4_FEATURE_RO_COMPAT_PROJECT |\
EXT4_FEATURE_RO_COMPAT_SHARED_BLOCKS |\
- EXT4_FEATURE_RO_COMPAT_VERITY)
+ EXT4_FEATURE_RO_COMPAT_VERITY |\
+ EXT4_FEATURE_RO_COMPAT_ORPHAN_PRESENT)
/*
* These features are only allowed if EXT2_FLAG_SOFTSUPP_FEATURES is passed
@@ -1687,6 +1689,19 @@ errcode_t ext2fs_get_data_io(ext2_filsys fs, io_channel *old_io);
errcode_t ext2fs_set_data_io(ext2_filsys fs, io_channel new_io);
errcode_t ext2fs_rewrite_to_io(ext2_filsys fs, io_channel new_io);
+/* orphan.c */
+extern errcode_t ext2fs_create_orphan_file(ext2_filsys fs, blk_t num_blocks);
+extern errcode_t ext2fs_truncate_orphan_file(ext2_filsys fs);
+extern e2_blkcnt_t ext2fs_default_orphan_file_blocks(ext2_filsys fs);
+extern __u32 ext2fs_do_orphan_file_block_csum(ext2_filsys fs, ext2_ino_t ino,
+ __u32 gen, blk64_t blk,
+ char *buf);
+extern errcode_t ext2fs_orphan_file_block_csum_set(ext2_filsys fs,
+ ext2_ino_t ino, blk64_t blk,
+ char *buf);
+extern int ext2fs_orphan_file_block_csum_verify(ext2_filsys fs, ext2_ino_t ino,
+ blk64_t blk, char *buf);
+
/* get_pathname.c */
extern errcode_t ext2fs_get_pathname(ext2_filsys fs, ext2_ino_t dir, ext2_ino_t ino,
char **name);
@@ -1840,7 +1855,9 @@ extern int ext2fs_dirent_file_type(const struct ext2_dir_entry *entry);
extern void ext2fs_dirent_set_file_type(struct ext2_dir_entry *entry, int type);
extern struct ext2_inode *ext2fs_inode(struct ext2_inode_large * large_inode);
extern const struct ext2_inode *ext2fs_const_inode(const struct ext2_inode_large * large_inode);
-
+extern int ext2fs_inodes_per_orphan_block(ext2_filsys fs);
+extern struct ext4_orphan_block_tail *ext2fs_orphan_block_tail(ext2_filsys fs,
+ char *buf);
#endif
/*
@@ -2150,6 +2167,19 @@ ext2fs_const_inode(const struct ext2_inode_large * large_inode)
return (const struct ext2_inode *) large_inode;
}
+_INLINE_ int ext2fs_inodes_per_orphan_block(ext2_filsys fs)
+{
+ return (fs->blocksize - sizeof(struct ext4_orphan_block_tail)) /
+ sizeof(__u32);
+}
+
+_INLINE_ struct ext4_orphan_block_tail *
+ext2fs_orphan_block_tail(ext2_filsys fs, char *buf)
+{
+ return (struct ext4_orphan_block_tail *)(buf + fs->blocksize -
+ sizeof(struct ext4_orphan_block_tail));
+}
+
#undef _INLINE_
#endif
diff --git a/lib/ext2fs/orphan.c b/lib/ext2fs/orphan.c
new file mode 100644
index 000000000000..fde1e347fcc9
--- /dev/null
+++ b/lib/ext2fs/orphan.c
@@ -0,0 +1,272 @@
+/*
+ * orphan.c --- utility function to handle orphan file
+ *
+ * Copyright (C) 2015 Jan Kara.
+ *
+ * %Begin-Header%
+ * This file may be redistributed under the terms of the GNU Library
+ * General Public License, version 2.
+ * %End-Header%
+ */
+
+#include "config.h"
+#include <string.h>
+
+#include "ext2_fs.h"
+#include "ext2fsP.h"
+
+errcode_t ext2fs_truncate_orphan_file(ext2_filsys fs)
+{
+ struct ext2_inode inode;
+ errcode_t err;
+ ext2_ino_t ino = fs->super->s_orphan_file_inum;
+
+ err = ext2fs_read_inode(fs, ino, &inode);
+ if (err)
+ return err;
+
+ err = ext2fs_punch(fs, ino, &inode, NULL, 0, ~0ULL);
+ if (err)
+ return err;
+
+ fs->flags &= ~EXT2_FLAG_SUPER_ONLY;
+ memset(&inode, 0, sizeof(struct ext2_inode));
+ err = ext2fs_write_inode(fs, ino, &inode);
+
+ ext2fs_clear_feature_orphan_file(fs->super);
+ ext2fs_clear_feature_orphan_present(fs->super);
+ ext2fs_mark_super_dirty(fs);
+ /* Need to update group descriptors as well */
+ fs->flags &= ~EXT2_FLAG_SUPER_ONLY;
+
+ return err;
+}
+
+__u32 ext2fs_do_orphan_file_block_csum(ext2_filsys fs, ext2_ino_t ino,
+ __u32 gen, blk64_t blk, char *buf)
+{
+ int inodes_per_ob = ext2fs_inodes_per_orphan_block(fs);
+ __u32 crc;
+
+ ino = ext2fs_cpu_to_le32(ino);
+ gen = ext2fs_cpu_to_le32(gen);
+ blk = ext2fs_cpu_to_le64(blk);
+ crc = ext2fs_crc32c_le(fs->csum_seed, (unsigned char *)&ino,
+ sizeof(ino));
+ crc = ext2fs_crc32c_le(crc, (unsigned char *)&gen, sizeof(gen));
+ crc = ext2fs_crc32c_le(crc, (unsigned char *)&blk, sizeof(blk));
+ crc = ext2fs_crc32c_le(crc, buf, inodes_per_ob * sizeof(__u32));
+
+ return ext2fs_cpu_to_le32(crc);
+}
+
+struct mkorphan_info {
+ char *buf;
+ char *zerobuf;
+ blk_t num_blocks;
+ blk_t alloc_blocks;
+ blk64_t last_blk;
+ errcode_t err;
+ ino_t ino;
+ __u32 generation;
+};
+
+static int mkorphan_proc(ext2_filsys fs,
+ blk64_t *blocknr,
+ e2_blkcnt_t blockcnt,
+ blk64_t ref_block EXT2FS_ATTR((unused)),
+ int ref_offset EXT2FS_ATTR((unused)),
+ void *priv_data)
+{
+ struct mkorphan_info *oi = (struct mkorphan_info *)priv_data;
+ blk64_t new_blk;
+ errcode_t err;
+
+ /* Can we just continue in currently allocated cluster? */
+ if (blockcnt &&
+ EXT2FS_B2C(fs, oi->last_blk) == EXT2FS_B2C(fs, oi->last_blk + 1)) {
+ new_blk = oi->last_blk + 1;
+ } else {
+ err = ext2fs_new_block2(fs, oi->last_blk, 0, &new_blk);
+ if (err) {
+ oi->err = err;
+ return BLOCK_ABORT;
+ }
+ ext2fs_block_alloc_stats2(fs, new_blk, +1);
+ oi->alloc_blocks++;
+ }
+ if (blockcnt >= 0) {
+ if (ext2fs_has_feature_metadata_csum(fs->super)) {
+ struct ext4_orphan_block_tail *tail;
+
+ tail = ext2fs_orphan_block_tail(fs, oi->buf);
+ tail->ob_checksum = ext2fs_do_orphan_file_block_csum(fs,
+ oi->ino, oi->generation, new_blk, oi->buf);
+ }
+ err = io_channel_write_blk64(fs->io, new_blk, 1, oi->buf);
+ } else /* zerobuf is used to initialize new indirect blocks... */
+ err = io_channel_write_blk64(fs->io, new_blk, 1, oi->zerobuf);
+ if (err) {
+ oi->err = err;
+ return BLOCK_ABORT;
+ }
+ oi->last_blk = new_blk;
+ *blocknr = new_blk;
+ if (blockcnt >= 0 && --oi->num_blocks == 0)
+ return BLOCK_CHANGED | BLOCK_ABORT;
+ return BLOCK_CHANGED;
+}
+
+errcode_t ext2fs_create_orphan_file(ext2_filsys fs, blk_t num_blocks)
+{
+ struct ext2_inode inode;
+ ext2_ino_t ino = fs->super->s_orphan_file_inum;
+ errcode_t err;
+ char *buf = NULL, *zerobuf = NULL;
+ struct mkorphan_info oi;
+ struct ext4_orphan_block_tail *ob_tail;
+
+ if (!ino) {
+ err = ext2fs_new_inode(fs, EXT2_ROOT_INO, LINUX_S_IFREG | 0600,
+ 0, &ino);
+ if (err)
+ return err;
+ ext2fs_inode_alloc_stats2(fs, ino, +1, 0);
+ ext2fs_mark_ib_dirty(fs);
+ }
+
+ err = ext2fs_read_inode(fs, ino, &inode);
+ if (err)
+ return err;
+ if (EXT2_I_SIZE(&inode)) {
+ err = ext2fs_truncate_orphan_file(fs);
+ if (err)
+ return err;
+ }
+
+ memset(&inode, 0, sizeof(struct ext2_inode));
+ if (ext2fs_has_feature_extents(fs->super)) {
+ inode.i_flags |= EXT4_EXTENTS_FL;
+ err = ext2fs_write_inode(fs, ino, &inode);
+ if (err)
+ return err;
+ }
+
+ err = ext2fs_get_mem(fs->blocksize, &buf);
+ if (err)
+ return err;
+ err = ext2fs_get_mem(fs->blocksize, &zerobuf);
+ if (err)
+ goto out;
+ memset(buf, 0, fs->blocksize);
+ memset(zerobuf, 0, fs->blocksize);
+ ob_tail = ext2fs_orphan_block_tail(fs, buf);
+ ob_tail->ob_magic = ext2fs_cpu_to_le32(EXT4_ORPHAN_BLOCK_MAGIC);
+ oi.num_blocks = num_blocks;
+ oi.alloc_blocks = 0;
+ oi.last_blk = 0;
+ oi.generation = inode.i_generation;
+ oi.ino = ino;
+ oi.buf = buf;
+ oi.zerobuf = zerobuf;
+ oi.err = 0;
+ err = ext2fs_block_iterate3(fs, ino, BLOCK_FLAG_APPEND,
+ 0, mkorphan_proc, &oi);
+ if (err)
+ goto out;
+ if (oi.err) {
+ err = oi.err;
+ goto out;
+ }
+
+ /* Reread inode after blocks were allocated */
+ err = ext2fs_read_inode(fs, ino, &inode);
+ if (err)
+ goto out;
+ ext2fs_iblk_set(fs, &inode, 0);
+ inode.i_atime = inode.i_mtime =
+ inode.i_ctime = fs->now ? fs->now : time(0);
+ inode.i_links_count = 1;
+ inode.i_mode = LINUX_S_IFREG | 0600;
+ ext2fs_iblk_add_blocks(fs, &inode, oi.alloc_blocks);
+ err = ext2fs_inode_size_set(fs, &inode,
+ (unsigned long long)fs->blocksize * num_blocks);
+ if (err)
+ goto out;
+ err = ext2fs_write_new_inode(fs, ino, &inode);
+ if (err)
+ goto out;
+
+ fs->super->s_orphan_file_inum = ino;
+ ext2fs_set_feature_orphan_file(fs->super);
+ ext2fs_mark_super_dirty(fs);
+ /* Need to update group descriptors as well */
+ fs->flags &= ~EXT2_FLAG_SUPER_ONLY;
+out:
+ if (buf)
+ ext2fs_free_mem(&buf);
+ if (zerobuf)
+ ext2fs_free_mem(&zerobuf);
+ return err;
+}
+
+/*
+ * Find reasonable size for orphan file. We choose orphan file size to be
+ * between 32 and 512 filesystem blocks and not more than 1/4096 of the
+ * filesystem unless it is really small.
+ */
+e2_blkcnt_t ext2fs_default_orphan_file_blocks(ext2_filsys fs)
+{
+ __u64 num_blocks = ext2fs_blocks_count(fs->super);
+ e2_blkcnt_t blks = 512;
+
+ if (num_blocks < 128 * 1024)
+ blks = 32;
+ else if (num_blocks < 2 * 1024 * 1024)
+ blks = num_blocks / 4096;
+ return (blks + EXT2FS_CLUSTER_MASK(fs)) & ~EXT2FS_CLUSTER_MASK(fs);
+}
+
+static errcode_t ext2fs_orphan_file_block_csum(ext2_filsys fs, ext2_ino_t ino,
+ blk64_t blk, char *buf,
+ __u32 *crcp)
+{
+ struct ext2_inode inode;
+ errcode_t retval;
+
+ retval = ext2fs_read_inode(fs, ino, &inode);
+ if (retval)
+ return retval;
+ *crcp = ext2fs_do_orphan_file_block_csum(fs, ino, inode.i_generation,
+ blk, buf);
+ return 0;
+}
+
+errcode_t ext2fs_orphan_file_block_csum_set(ext2_filsys fs, ext2_ino_t ino,
+ blk64_t blk, char *buf)
+{
+ struct ext4_orphan_block_tail *tail;
+
+ if (!ext2fs_has_feature_metadata_csum(fs->super))
+ return 0;
+
+ tail = ext2fs_orphan_block_tail(fs, buf);
+ return ext2fs_orphan_file_block_csum(fs, ino, blk, buf,
+ &tail->ob_checksum);
+}
+
+int ext2fs_orphan_file_block_csum_verify(ext2_filsys fs, ext2_ino_t ino,
+ blk64_t blk, char *buf)
+{
+ struct ext4_orphan_block_tail *tail;
+ __u32 crc;
+ errcode_t retval;
+
+ if (!ext2fs_has_feature_metadata_csum(fs->super))
+ return 1;
+ retval = ext2fs_orphan_file_block_csum(fs, ino, blk, buf, &crc);
+ if (retval)
+ return 0;
+ tail = ext2fs_orphan_block_tail(fs, buf);
+ return ext2fs_le32_to_cpu(tail->ob_checksum) == crc;
+}
diff --git a/lib/ext2fs/swapfs.c b/lib/ext2fs/swapfs.c
index 1006b2d2bd52..b844e7665999 100644
--- a/lib/ext2fs/swapfs.c
+++ b/lib/ext2fs/swapfs.c
@@ -131,8 +131,9 @@ void ext2fs_swap_super(struct ext2_super_block * sb)
/* s_*_time_hi are __u8 and does not need swabbing */
sb->s_encoding = ext2fs_swab16(sb->s_encoding);
sb->s_encoding_flags = ext2fs_swab16(sb->s_encoding_flags);
+ sb->s_orphan_file_inum = ext2fs_swab32(sb->s_orphan_file_inum);
/* catch when new fields are used from s_reserved */
- EXT2FS_BUILD_BUG_ON(sizeof(sb->s_reserved) != 95 * sizeof(__le32));
+ EXT2FS_BUILD_BUG_ON(sizeof(sb->s_reserved) != 94 * sizeof(__le32));
sb->s_checksum = ext2fs_swab32(sb->s_checksum);
}
diff --git a/lib/ext2fs/tst_super_size.c b/lib/ext2fs/tst_super_size.c
index 80a5269bceb7..ad452dee8eb3 100644
--- a/lib/ext2fs/tst_super_size.c
+++ b/lib/ext2fs/tst_super_size.c
@@ -152,7 +152,8 @@ int main(int argc, char **argv)
check_field(s_last_error_errcode, 1);
check_field(s_encoding, 2);
check_field(s_encoding_flags, 2);
- check_field(s_reserved, 95 * 4);
+ check_field(s_orphan_file_inum, 4);
+ check_field(s_reserved, 94 * 4);
check_field(s_checksum, 4);
do_field("Superblock end", 0, 0, cur_offset, 1024);
#endif
diff --git a/lib/support/mkquota.c b/lib/support/mkquota.c
index 21a5c34d6921..b790b93f52c2 100644
--- a/lib/support/mkquota.c
+++ b/lib/support/mkquota.c
@@ -504,7 +504,8 @@ errcode_t quota_compute_usage(quota_ctx_t qctx)
continue;
if (ino == EXT2_ROOT_INO ||
(ino >= EXT2_FIRST_INODE(fs->super) &&
- ino != quota_type2inum(PRJQUOTA, fs->super))) {
+ ino != quota_type2inum(PRJQUOTA, fs->super) &&
+ ino != fs->super->s_orphan_file_inum)) {
space = ext2fs_get_stat_i_blocks(fs,
EXT2_INODE(inode)) << 9;
quota_data_add(qctx, inode, ino, space);
--
2.26.2
^ permalink raw reply related [flat|nested] 22+ messages in thread
* [PATCH 5/9] mke2fs: Add support for orphan_file feature
2021-07-12 15:43 [PATCH 0/9 v4] e2fsprogs: Support for orphan file feature Jan Kara
` (3 preceding siblings ...)
2021-07-12 15:43 ` [PATCH 4/9] libext2fs: Support for orphan file feature Jan Kara
@ 2021-07-12 15:43 ` Jan Kara
2021-07-12 15:43 ` [PATCH 6/9] e2fsck: Add support for handling orphan file Jan Kara
` (3 subsequent siblings)
8 siblings, 0 replies; 22+ messages in thread
From: Jan Kara @ 2021-07-12 15:43 UTC (permalink / raw)
To: Ted Tso; +Cc: linux-ext4, Jan Kara
Signed-off-by: Jan Kara <jack@suse.cz>
---
misc/mke2fs.8.in | 5 +++++
misc/mke2fs.c | 36 +++++++++++++++++++++++++++++++++++-
2 files changed, 40 insertions(+), 1 deletion(-)
diff --git a/misc/mke2fs.8.in b/misc/mke2fs.8.in
index 84248ffc9e1d..3747c93a02bf 100644
--- a/misc/mke2fs.8.in
+++ b/misc/mke2fs.8.in
@@ -403,6 +403,11 @@ filesystem to change based on the user running \fBmke2fs\fR.
Set a flag in the filesystem superblock indicating that it may be
mounted using experimental kernel code, such as the ext4dev filesystem.
.TP
+.BI orphan_file_size= size
+Set size of the file for tracking unlinked but still open inodes and inodes
+with truncate in progress. Larger file allows for better scalability, reserving
+a few blocks per cpu is ideal.
+.TP
.B discard
Attempt to discard blocks at mkfs time (discarding blocks initially is useful
on solid state devices and sparse / thin-provisioned storage). When the device
diff --git a/misc/mke2fs.c b/misc/mke2fs.c
index afbcf486bad2..8a5cf9b920e6 100644
--- a/misc/mke2fs.c
+++ b/misc/mke2fs.c
@@ -94,6 +94,7 @@ static gid_t root_gid;
int journal_size;
int journal_flags;
int journal_fc_size;
+static e2_blkcnt_t orphan_file_blocks;
static int lazy_itable_init;
static int packed_meta_blocks;
int no_copy_xattrs;
@@ -1087,6 +1088,21 @@ static void parse_extended_opts(struct ext2_super_block *param,
continue;
}
encoding_flags = arg;
+ } else if (!strcmp(token, "orphan_file_size")) {
+ if (!arg) {
+ r_usage++;
+ badopt = token;
+ continue;
+ }
+ orphan_file_blocks = parse_num_blocks2(arg,
+ fs_param.s_log_block_size);
+ if (orphan_file_blocks == 0) {
+ fprintf(stderr,
+ _("Invalid size of orphan file %s\n"),
+ arg);
+ r_usage++;
+ continue;
+ }
} else {
r_usage++;
badopt = token;
@@ -1154,7 +1170,8 @@ static __u32 ok_features[3] = {
EXT2_FEATURE_COMPAT_EXT_ATTR |
EXT4_FEATURE_COMPAT_SPARSE_SUPER2 |
EXT4_FEATURE_COMPAT_FAST_COMMIT |
- EXT4_FEATURE_COMPAT_STABLE_INODES,
+ EXT4_FEATURE_COMPAT_STABLE_INODES |
+ EXT4_FEATURE_COMPAT_ORPHAN_FILE,
/* Incompat */
EXT2_FEATURE_INCOMPAT_FILETYPE|
EXT3_FEATURE_INCOMPAT_EXTENTS|
@@ -3438,6 +3455,23 @@ no_journal:
fix_cluster_bg_counts(fs);
if (ext2fs_has_feature_quota(&fs_param))
create_quota_inodes(fs);
+ if (ext2fs_has_feature_orphan_file(&fs_param)) {
+ if (!ext2fs_has_feature_journal(&fs_param)) {
+ com_err(program_name, 0, _("cannot set orphan_file "
+ "flag without a journal."));
+ exit(1);
+ }
+ if (!orphan_file_blocks) {
+ orphan_file_blocks =
+ ext2fs_default_orphan_file_blocks(fs);
+ }
+ retval = ext2fs_create_orphan_file(fs, orphan_file_blocks);
+ if (retval) {
+ com_err(program_name, retval,
+ _("while creating orphan file"));
+ exit(1);
+ }
+ }
retval = mk_hugefiles(fs, device_name);
if (retval)
--
2.26.2
^ permalink raw reply related [flat|nested] 22+ messages in thread
* [PATCH 6/9] e2fsck: Add support for handling orphan file
2021-07-12 15:43 [PATCH 0/9 v4] e2fsprogs: Support for orphan file feature Jan Kara
` (4 preceding siblings ...)
2021-07-12 15:43 ` [PATCH 5/9] mke2fs: Add support for orphan_file feature Jan Kara
@ 2021-07-12 15:43 ` Jan Kara
2021-08-03 18:56 ` Theodore Ts'o
2021-07-12 15:43 ` [PATCH 7/9] tune2fs: Add support for orphan_file feature Jan Kara
` (2 subsequent siblings)
8 siblings, 1 reply; 22+ messages in thread
From: Jan Kara @ 2021-07-12 15:43 UTC (permalink / raw)
To: Ted Tso; +Cc: linux-ext4, Jan Kara
Signed-off-by: Jan Kara <jack@suse.cz>
---
e2fsck/e2fsck.h | 1 +
e2fsck/pass1.c | 27 ++++
e2fsck/pass4.c | 2 +-
e2fsck/problem.c | 80 +++++++++++
e2fsck/problem.h | 50 +++++++
e2fsck/super.c | 358 +++++++++++++++++++++++++++++++++++++++++------
e2fsck/unix.c | 72 +++++++++-
7 files changed, 546 insertions(+), 44 deletions(-)
diff --git a/e2fsck/e2fsck.h b/e2fsck/e2fsck.h
index 15d043ee4692..979cdfcb2740 100644
--- a/e2fsck/e2fsck.h
+++ b/e2fsck/e2fsck.h
@@ -649,6 +649,7 @@ void sigcatcher_setup(void);
void check_super_block(e2fsck_t ctx);
int check_backup_super_block(e2fsck_t ctx);
void check_resize_inode(e2fsck_t ctx);
+int check_init_orphan_file(e2fsck_t ctx);
/* util.c */
extern void *e2fsck_allocate_memory(e2fsck_t ctx, unsigned long size,
diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c
index 9d4308956773..b8d8dd109096 100644
--- a/e2fsck/pass1.c
+++ b/e2fsck/pass1.c
@@ -1778,6 +1778,32 @@ void e2fsck_pass1(e2fsck_t ctx)
inode_size, "pass1");
failed_csum = 0;
}
+ } else if (ino == fs->super->s_orphan_file_inum) {
+ ext2fs_mark_inode_bitmap2(ctx->inode_used_map, ino);
+ if (ext2fs_has_feature_orphan_file(fs->super)) {
+ if (!LINUX_S_ISREG(inode->i_mode) &&
+ fix_problem(ctx, PR_1_ORPHAN_FILE_BAD_MODE,
+ &pctx)) {
+ inode->i_mode = LINUX_S_IFREG;
+ e2fsck_write_inode(ctx, ino, inode,
+ "pass1");
+ failed_csum = 0;
+ }
+ check_blocks(ctx, &pctx, block_buf, NULL);
+ FINISH_INODE_LOOP(ctx, ino, &pctx, failed_csum);
+ continue;
+ }
+ if ((inode->i_links_count ||
+ inode->i_blocks || inode->i_block[0]) &&
+ fix_problem(ctx, PR_1_ORPHAN_FILE_NOT_CLEAR,
+ &pctx)) {
+ memset(inode, 0, inode_size);
+ ext2fs_icount_store(ctx->inode_link_info, ino,
+ 0);
+ e2fsck_write_inode_full(ctx, ino, inode,
+ inode_size, "pass1");
+ failed_csum = 0;
+ }
} else if (ino < EXT2_FIRST_INODE(fs->super)) {
problem_t problem = 0;
@@ -3484,6 +3510,7 @@ static void check_blocks(e2fsck_t ctx, struct problem_context *pctx,
}
if (ino != quota_type2inum(PRJQUOTA, fs->super) &&
+ ino != fs->super->s_orphan_file_inum &&
(ino == EXT2_ROOT_INO || ino >= EXT2_FIRST_INODE(ctx->fs->super)) &&
!(inode->i_flags & EXT4_EA_INODE_FL)) {
quota_data_add(ctx->qctx, (struct ext2_inode_large *) inode,
diff --git a/e2fsck/pass4.c b/e2fsck/pass4.c
index 8c2d2f1fca12..f41eb849e567 100644
--- a/e2fsck/pass4.c
+++ b/e2fsck/pass4.c
@@ -192,7 +192,7 @@ void e2fsck_pass4(e2fsck_t ctx)
goto errout;
}
if (i == quota_type2inum(PRJQUOTA, ctx->fs->super) ||
- i == EXT2_BAD_INO ||
+ i == fs->super->s_orphan_file_inum || i == EXT2_BAD_INO ||
(i > EXT2_ROOT_INO && i < EXT2_FIRST_INODE(fs->super)))
continue;
if (!(ext2fs_test_inode_bitmap2(ctx->inode_used_map, i)) ||
diff --git a/e2fsck/problem.c b/e2fsck/problem.c
index eb2824f31684..0935fe30fc5b 100644
--- a/e2fsck/problem.c
+++ b/e2fsck/problem.c
@@ -526,6 +526,26 @@ static struct e2fsck_problem problem_table[] = {
"not compatible. Resize @i should be disabled. "),
PROMPT_FIX, 0, 0, 0, 0 },
+ /* Orphan file contains holes */
+ { PR_0_ORPHAN_FILE_HOLE,
+ N_("Orphan file (@i %i) contains hole at @b %b. Terminating orphan file recovery.\n"),
+ PROMPT_NONE, 0 },
+
+ /* Orphan file block has wrong magic */
+ { PR_0_ORPHAN_FILE_BAD_MAGIC,
+ N_("Orphan file (@i %i) @b %b contains wrong magic. Terminating orphan file recovery.\n"),
+ PROMPT_NONE, 0 },
+
+ /* Orphan file block has wrong checksum */
+ { PR_0_ORPHAN_FILE_BAD_CHECKSUM,
+ N_("Orphan file (@i %i) @b %b contains wrong checksum. Terminating orphan file recovery.\n"),
+ PROMPT_NONE, 0 },
+
+ /* Orphan file size isn't multiple of blocks size */
+ { PR_0_ORPHAN_FILE_WRONG_SIZE,
+ N_("Orphan file (@i %i) size is not multiple of block size. Terminating orphan file recovery.\n"),
+ PROMPT_NONE, 0 },
+
/* Pass 1 errors */
/* Pass 1: Checking inodes, blocks, and sizes */
@@ -1280,6 +1300,16 @@ static struct e2fsck_problem problem_table[] = {
PROMPT_CLEAR_HTREE, PR_PREEN_OK, 0, 0, 0 },
+ /* Orphan file has bad mode */
+ { PR_1_ORPHAN_FILE_BAD_MODE,
+ N_("Orphan file @i %i is not regular file. "),
+ PROMPT_CLEAR, PR_PREEN_OK },
+
+ /* Orphan file inode is not in use, but contains data */
+ { PR_1_ORPHAN_FILE_NOT_CLEAR,
+ N_("Orphan file @i %i is not in use, but contains data. "),
+ PROMPT_CLEAR, PR_PREEN_OK },
+
/* Pass 1b errors */
/* Pass 1B: Rescan for duplicate/bad blocks */
@@ -2259,6 +2289,56 @@ static struct e2fsck_problem problem_table[] = {
N_("Error writing quota info for quota type %N: %m\n"),
PROMPT_NULL, 0, 0, 0, 0 },
+ /* Orphan file without a journal */
+ { PR_6_ORPHAN_FILE_WITHOUT_JOURNAL,
+ N_("@S has orphan file without @j.\n"),
+ PROMPT_CLEAR, PR_PREEN_OK },
+
+ /* Orphan file truncation failed */
+ { PR_6_ORPHAN_FILE_TRUNC_FAILED,
+ N_("Failed to truncate orphan file.\n"),
+ PROMPT_NONE, 0 },
+
+ /* Failed to initialize orphan file */
+ { PR_6_ORPHAN_FILE_CORRUPTED,
+ N_("Failed to initialize orphan file.\n"),
+ PROMPT_RECREATE, PR_PREEN_OK },
+
+ /* Cannot fix corrupted orphan file with invalid bitmaps */
+ { PR_6_ORPHAN_FILE_BITMAP_INVALID,
+ N_("Cannot fix corrupted orphan file with invalid bitmaps.\n"),
+ PROMPT_NONE, 0 },
+
+ /* Orphan file creation failed */
+ { PR_6_ORPHAN_FILE_CREATE_FAILED,
+ N_("Failed to truncate orphan file (@i %i).\n"),
+ PROMPT_NONE, 0 },
+
+ /* Orphan file block contains data */
+ { PR_6_ORPHAN_BLOCK_DIRTY,
+ N_("Orphan file (@i %i) @b %b is not clean.\n"),
+ PROMPT_CLEAR, PR_PREEN_OK },
+
+ /* orphan_present set but orphan file is empty */
+ { PR_6_ORPHAN_PRESENT_CLEAN_FILE,
+ N_("Flag orphan_present is set but orphan file is clean.\n"),
+ PROMPT_CLEAR, PR_PREEN_OK },
+
+ /* orphan_present set but orphan_file is not */
+ { PR_6_ORPHAN_PRESENT_NO_FILE,
+ N_("Flag orphan_present is set but flag orphan_file is not.\n"),
+ PROMPT_CLEAR, PR_PREEN_OK },
+
+ /* Orphan file size isn't multiple of blocks size */
+ { PR_6_ORPHAN_FILE_WRONG_SIZE,
+ N_("Orphan file (@i %i) size is not multiple of block size.\n"),
+ PROMPT_NONE, 0 },
+
+ /* Orphan file contains holes */
+ { PR_6_ORPHAN_FILE_HOLE,
+ N_("Orphan file (@i %i) contains hole at @b %b.\n"),
+ PROMPT_NONE, 0 },
+
{ 0 }
};
diff --git a/e2fsck/problem.h b/e2fsck/problem.h
index 24cdcf9b90f7..0611d71f9e03 100644
--- a/e2fsck/problem.h
+++ b/e2fsck/problem.h
@@ -288,6 +288,18 @@ struct problem_context {
/* Meta_bg and resize_inode are not compatible, remove resize_inode*/
#define PR_0_DISABLE_RESIZE_INODE 0x000051
+/* Orphan file contains holes */
+#define PR_0_ORPHAN_FILE_HOLE 0x000052
+
+/* Orphan file block has wrong magic */
+#define PR_0_ORPHAN_FILE_BAD_MAGIC 0x000053
+
+/* Orphan file block has wrong checksum */
+#define PR_0_ORPHAN_FILE_BAD_CHECKSUM 0x000054
+
+/* Orphan file size isn't multiple of blocks size */
+#define PR_0_ORPHAN_FILE_WRONG_SIZE 0x000055
+
/*
* Pass 1 errors
*/
@@ -717,6 +729,15 @@ struct problem_context {
#define PR_1_HTREE_CANNOT_SIPHASH 0x01008E
+/* Orphan file inode is not a regular file */
+#define PR_1_ORPHAN_FILE_BAD_MODE 0x01007F
+
+/* Orphan file inode is not in use, but contains data */
+#define PR_1_ORPHAN_FILE_NOT_CLEAR 0x010080
+
+/* Orphan file inode is not clear */
+#define PR_1_ORPHAN_INODE_NOT_CLEAR 0x01007F
+
/*
* Pass 1b errors
*/
@@ -1293,6 +1314,35 @@ struct problem_context {
/* Error updating quota information */
#define PR_6_WRITE_QUOTAS 0x060006
+/* Orphan file without a journal */
+#define PR_6_ORPHAN_FILE_WITHOUT_JOURNAL 0x060007
+
+/* Orphan file truncation failed */
+#define PR_6_ORPHAN_FILE_TRUNC_FAILED 0x060008
+
+/* Failed to initialize orphan file */
+#define PR_6_ORPHAN_FILE_CORRUPTED 0x060009
+
+/* Cannot fix corrupted orphan file with invalid bitmaps */
+#define PR_6_ORPHAN_FILE_BITMAP_INVALID 0x06000A
+
+/* Orphan file creation failed */
+#define PR_6_ORPHAN_FILE_CREATE_FAILED 0x06000B
+
+/* Orphan file block contains data */
+#define PR_6_ORPHAN_BLOCK_DIRTY 0x06000C
+
+/* orphan_present set but orphan file is empty */
+#define PR_6_ORPHAN_PRESENT_CLEAN_FILE 0x06000D
+
+/* orphan_present set but orphan_file is not */
+#define PR_6_ORPHAN_PRESENT_NO_FILE 0x06000E
+
+/* Orphan file size isn't multiple of blocks size */
+#define PR_6_ORPHAN_FILE_WRONG_SIZE 0x06000F
+
+/* Orphan file contains holes */
+#define PR_6_ORPHAN_FILE_HOLE 0x060010
/*
* Function declarations
diff --git a/e2fsck/super.c b/e2fsck/super.c
index e1c3f93572f4..cff556e790f8 100644
--- a/e2fsck/super.c
+++ b/e2fsck/super.c
@@ -313,6 +313,180 @@ static errcode_t e2fsck_write_all_quotas(e2fsck_t ctx)
return pctx.errcode;
}
+static int release_orphan_inode(e2fsck_t ctx, ext2_ino_t *ino, char *block_buf)
+{
+ ext2_filsys fs = ctx->fs;
+ struct problem_context pctx;
+ struct ext2_inode_large inode;
+ ext2_ino_t next_ino;
+
+ e2fsck_read_inode_full(ctx, *ino, EXT2_INODE(&inode),
+ sizeof(inode), "release_orphan_inode");
+ clear_problem_context(&pctx);
+ pctx.ino = *ino;
+ pctx.inode = EXT2_INODE(&inode);
+ pctx.str = inode.i_links_count ? _("Truncating") : _("Clearing");
+
+ fix_problem(ctx, PR_0_ORPHAN_CLEAR_INODE, &pctx);
+
+ next_ino = inode.i_dtime;
+ if (next_ino &&
+ ((next_ino < EXT2_FIRST_INODE(fs->super)) ||
+ (next_ino > fs->super->s_inodes_count))) {
+ pctx.ino = next_ino;
+ fix_problem(ctx, PR_0_ORPHAN_ILLEGAL_INODE, &pctx);
+ return 1;
+ }
+
+ if (release_inode_blocks(ctx, *ino, &inode, block_buf, &pctx))
+ return 1;
+
+ if (!inode.i_links_count) {
+ if (ctx->qctx)
+ quota_data_inodes(ctx->qctx, &inode, *ino, -1);
+ ext2fs_inode_alloc_stats2(fs, *ino, -1,
+ LINUX_S_ISDIR(inode.i_mode));
+ ctx->free_inodes++;
+ inode.i_dtime = ctx->now;
+ } else {
+ inode.i_dtime = 0;
+ }
+ e2fsck_write_inode_full(ctx, *ino, EXT2_INODE(&inode),
+ sizeof(inode), "delete_file");
+ *ino = next_ino;
+ return 0;
+}
+
+struct process_orphan_block_data {
+ e2fsck_t ctx;
+ char *buf;
+ char *block_buf;
+ e2_blkcnt_t blocks;
+ int abort;
+ int clear;
+ errcode_t errcode;
+ ext2_ino_t ino;
+ __u32 generation;
+};
+
+static int process_orphan_block(ext2_filsys fs,
+ blk64_t *block_nr,
+ e2_blkcnt_t blockcnt,
+ blk64_t ref_blk EXT2FS_ATTR((unused)),
+ int ref_offset EXT2FS_ATTR((unused)),
+ void *priv_data)
+{
+ struct process_orphan_block_data *pd;
+ e2fsck_t ctx;
+ struct problem_context pctx;
+ blk64_t blk = *block_nr;
+ struct ext4_orphan_block_tail *tail;
+ int j;
+ int inodes_per_ob;
+ __u32 *bdata;
+ ext2_ino_t ino;
+
+ pd = priv_data;
+ ctx = pd->ctx;
+ clear_problem_context(&pctx);
+ pctx.ino = fs->super->s_orphan_file_inum;
+ pctx.blk = blockcnt;
+
+ /* Orphan file must not have holes */
+ if (!blk) {
+ if (blockcnt == pd->blocks)
+ return BLOCK_ABORT;
+ fix_problem(ctx, PR_0_ORPHAN_FILE_HOLE, &pctx);
+return_abort:
+ pd->abort = 1;
+ return BLOCK_ABORT;
+ }
+ inodes_per_ob = ext2fs_inodes_per_orphan_block(fs);
+ pd->errcode = io_channel_read_blk64(fs->io, blk, 1, pd->buf);
+ if (pd->errcode)
+ goto return_abort;
+ tail = ext2fs_orphan_block_tail(fs, pd->buf);
+ if (ext2fs_le32_to_cpu(tail->ob_magic) !=
+ EXT4_ORPHAN_BLOCK_MAGIC) {
+ fix_problem(ctx, PR_0_ORPHAN_FILE_BAD_MAGIC, &pctx);
+ goto return_abort;
+ }
+ if (!ext2fs_orphan_file_block_csum_verify(fs,
+ fs->super->s_orphan_file_inum, blk, pd->buf)) {
+ fix_problem(ctx, PR_0_ORPHAN_FILE_BAD_CHECKSUM, &pctx);
+ goto return_abort;
+ }
+ bdata = (__u32 *)pd->buf;
+ for (j = 0; j < inodes_per_ob; j++) {
+ if (!bdata[j])
+ continue;
+ ino = ext2fs_le32_to_cpu(bdata[j]);
+ if (release_orphan_inode(ctx, &ino, pd->block_buf))
+ goto return_abort;
+ }
+ return 0;
+}
+
+static int process_orphan_file(e2fsck_t ctx, char *block_buf)
+{
+ ext2_filsys fs = ctx->fs;
+ char *orphan_buf;
+ struct process_orphan_block_data pd;
+ int ret = 0;
+ ext2_ino_t orphan_inum = fs->super->s_orphan_file_inum;
+ struct ext2_inode orphan_inode;
+ struct problem_context pctx;
+ errcode_t retval;
+
+ if (!ext2fs_has_feature_orphan_file(fs->super))
+ return 0;
+
+ clear_problem_context(&pctx);
+ pctx.ino = orphan_inum;
+
+ orphan_buf = (char *) e2fsck_allocate_memory(ctx, fs->blocksize * 4,
+ "orphan block buffer");
+ retval = ext2fs_read_inode(fs, orphan_inum, &orphan_inode);
+ if (retval < 0) {
+ com_err("process_orphan_file", retval,
+ _("while reading inode %d"), orphan_inum);
+ ret = 1;
+ goto out;
+ }
+ if (EXT2_I_SIZE(&orphan_inode) & (fs->blocksize - 1)) {
+ fix_problem(ctx, PR_0_ORPHAN_FILE_WRONG_SIZE, &pctx);
+ ret = 1;
+ goto out;
+ }
+ pd.buf = orphan_buf + 3 * fs->blocksize;
+ pd.block_buf = block_buf;
+ pd.blocks = EXT2_I_SIZE(&orphan_inode) / fs->blocksize;
+ pd.ctx = ctx;
+ pd.abort = 0;
+ pd.errcode = 0;
+ retval = ext2fs_block_iterate3(fs, orphan_inum,
+ BLOCK_FLAG_DATA_ONLY | BLOCK_FLAG_HOLE,
+ orphan_buf, process_orphan_block, &pd);
+ if (retval) {
+ com_err("process_orphan_block", retval,
+ _("while calling ext2fs_block_iterate for inode %d"),
+ orphan_inum);
+ ret = 1;
+ goto out;
+ }
+ if (pd.abort) {
+ if (pd.errcode) {
+ com_err("process_orphan_block", pd.errcode,
+ _("while reading blocks of inode %d"),
+ orphan_inum);
+ }
+ ret = 1;
+ }
+out:
+ ext2fs_free_mem(&orphan_buf);
+ return ret;
+}
+
/*
* This function releases all of the orphan inodes. It returns 1 if
* it hit some error, and 0 on success.
@@ -325,10 +499,13 @@ static int release_orphan_inodes(e2fsck_t ctx)
struct problem_context pctx;
char *block_buf;
- if ((ino = fs->super->s_last_orphan) == 0)
+ if (fs->super->s_last_orphan == 0 &&
+ !ext2fs_has_feature_orphan_present(fs->super))
return 0;
clear_problem_context(&pctx);
+ ino = fs->super->s_last_orphan;
+ pctx.ino = ino;
pctx.errcode = e2fsck_read_all_quotas(ctx);
if (pctx.errcode) {
fix_problem(ctx, PR_0_QUOTA_INIT_CTX, &pctx);
@@ -343,9 +520,10 @@ static int release_orphan_inodes(e2fsck_t ctx)
ext2fs_mark_super_dirty(fs);
/*
- * If the filesystem contains errors, don't run the orphan
- * list, since the orphan list can't be trusted; and we're
- * going to be running a full e2fsck run anyway...
+ * If the filesystem contains errors, don't process the orphan list
+ * or orphan file, since neither can be trusted; and we're going to
+ * be running a full e2fsck run anyway... We clear orphan file contents
+ * after filesystem is checked to avoid clearing someone else's data.
*/
if (fs->super->s_state & EXT2_ERROR_FS) {
if (ctx->qctx)
@@ -353,10 +531,8 @@ static int release_orphan_inodes(e2fsck_t ctx)
return 0;
}
- if ((ino < EXT2_FIRST_INODE(fs->super)) ||
- (ino > fs->super->s_inodes_count)) {
- clear_problem_context(&pctx);
- pctx.ino = ino;
+ if (ino && ((ino < EXT2_FIRST_INODE(fs->super)) ||
+ (ino > fs->super->s_inodes_count))) {
fix_problem(ctx, PR_0_ORPHAN_ILLEGAL_HEAD_INODE, &pctx);
goto err_qctx;
}
@@ -365,43 +541,19 @@ static int release_orphan_inodes(e2fsck_t ctx)
"block iterate buffer");
e2fsck_read_bitmaps(ctx);
+ /* First process orphan list */
while (ino) {
- e2fsck_read_inode_full(ctx, ino, EXT2_INODE(&inode),
- sizeof(inode), "release_orphan_inodes");
- clear_problem_context(&pctx);
- pctx.ino = ino;
- pctx.inode = EXT2_INODE(&inode);
- pctx.str = inode.i_links_count ? _("Truncating") :
- _("Clearing");
-
- fix_problem(ctx, PR_0_ORPHAN_CLEAR_INODE, &pctx);
-
- next_ino = inode.i_dtime;
- if (next_ino &&
- ((next_ino < EXT2_FIRST_INODE(fs->super)) ||
- (next_ino > fs->super->s_inodes_count))) {
- pctx.ino = next_ino;
- fix_problem(ctx, PR_0_ORPHAN_ILLEGAL_INODE, &pctx);
+ if (release_orphan_inode(ctx, &ino, block_buf))
goto err_buf;
- }
+ }
- if (release_inode_blocks(ctx, ino, &inode, block_buf, &pctx))
- goto err_buf;
+ /* Next process orphan file */
+ if (ext2fs_has_feature_orphan_present(fs->super) &&
+ !ext2fs_has_feature_orphan_file(fs->super))
+ goto err_buf;
+ if (process_orphan_file(ctx, block_buf))
+ goto err_buf;
- if (!inode.i_links_count) {
- if (ctx->qctx)
- quota_data_inodes(ctx->qctx, &inode, ino, -1);
- ext2fs_inode_alloc_stats2(fs, ino, -1,
- LINUX_S_ISDIR(inode.i_mode));
- ctx->free_inodes++;
- inode.i_dtime = ctx->now;
- } else {
- inode.i_dtime = 0;
- }
- e2fsck_write_inode_full(ctx, ino, EXT2_INODE(&inode),
- sizeof(inode), "delete_file");
- ino = next_ino;
- }
ext2fs_free_mem(&block_buf);
pctx.errcode = e2fsck_write_all_quotas(ctx);
if (pctx.errcode)
@@ -416,6 +568,130 @@ err:
return 1;
}
+static int reinit_orphan_block(ext2_filsys fs,
+ blk64_t *block_nr,
+ e2_blkcnt_t blockcnt,
+ blk64_t ref_blk EXT2FS_ATTR((unused)),
+ int ref_offset EXT2FS_ATTR((unused)),
+ void *priv_data)
+{
+ struct process_orphan_block_data *pd;
+ e2fsck_t ctx;
+ blk64_t blk = *block_nr;
+ struct problem_context pctx;
+ struct ext4_orphan_block_tail *tail;
+
+ pd = priv_data;
+ ctx = pd->ctx;
+ tail = ext2fs_orphan_block_tail(fs, pd->buf);
+
+ /* Orphan file must not have holes */
+ if (!blk) {
+ if (blockcnt == pd->blocks)
+ return BLOCK_ABORT;
+
+ clear_problem_context(&pctx);
+ pctx.ino = fs->super->s_orphan_file_inum;
+ pctx.blk = blockcnt;
+ fix_problem(ctx, PR_6_ORPHAN_FILE_HOLE, &pctx);
+return_abort:
+ pd->abort = 1;
+ return BLOCK_ABORT;
+ }
+ /*
+ * Update checksum to match expected buffer contents with appropriate
+ * block number.
+ */
+ tail->ob_checksum = ext2fs_do_orphan_file_block_csum(fs, pd->ino,
+ pd->generation, blk, pd->buf);
+ if (!pd->clear) {
+ pd->errcode = io_channel_read_blk64(fs->io, blk, 1,
+ pd->block_buf);
+ /* Block is already cleanly initialized? */
+ if (!memcmp(pd->block_buf, pd->buf, fs->blocksize))
+ return 0;
+
+ clear_problem_context(&pctx);
+ pctx.ino = fs->super->s_orphan_file_inum;
+ pctx.blk = blockcnt;
+ if (!fix_problem(ctx, PR_6_ORPHAN_BLOCK_DIRTY, &pctx))
+ goto return_abort;
+ pd->clear = 1;
+ }
+ pd->errcode = io_channel_write_blk64(fs->io, blk, 1, pd->buf);
+ if (pd->errcode)
+ goto return_abort;
+ return 0;
+}
+
+/*
+ * Check and clear orphan file. We just return non-zero if we hit some
+ * inconsistency. Caller will truncate & recreate new orphan file.
+ */
+int check_init_orphan_file(e2fsck_t ctx)
+{
+ ext2_filsys fs = ctx->fs;
+ char *orphan_buf;
+ struct process_orphan_block_data pd;
+ struct ext4_orphan_block_tail *tail;
+ ext2_ino_t orphan_inum = fs->super->s_orphan_file_inum;
+ struct ext2_inode orphan_inode;
+ int ret = 0;
+ errcode_t retval;
+
+ orphan_buf = (char *) e2fsck_allocate_memory(ctx, fs->blocksize * 5,
+ "orphan block buffer");
+ e2fsck_read_inode(ctx, orphan_inum, &orphan_inode, "orphan inode");
+ if (EXT2_I_SIZE(&orphan_inode) & (fs->blocksize - 1)) {
+ struct problem_context pctx;
+
+ clear_problem_context(&pctx);
+ pctx.ino = orphan_inum;
+ fix_problem(ctx, PR_6_ORPHAN_FILE_WRONG_SIZE, &pctx);
+ ret = 1;
+ goto out;
+ }
+ pd.buf = orphan_buf + 3 * fs->blocksize;
+ pd.block_buf = orphan_buf + 4 * fs->blocksize;
+ pd.blocks = EXT2_I_SIZE(&orphan_inode) / fs->blocksize;
+ pd.ctx = ctx;
+ pd.abort = 0;
+ pd.clear = 0;
+ pd.errcode = 0;
+ pd.ino = orphan_inum;
+ pd.generation = orphan_inode.i_generation;
+ /* Initialize buffer to write */
+ memset(pd.buf, 0, fs->blocksize);
+ tail = ext2fs_orphan_block_tail(fs, pd.buf);
+ tail->ob_magic = ext2fs_cpu_to_le32(EXT4_ORPHAN_BLOCK_MAGIC);
+
+ retval = ext2fs_block_iterate3(fs, orphan_inum,
+ BLOCK_FLAG_DATA_ONLY | BLOCK_FLAG_HOLE,
+ orphan_buf, reinit_orphan_block, &pd);
+ if (retval) {
+ com_err("reinit_orphan_block", retval,
+ _("while calling ext2fs_block_iterate for inode %d"),
+ orphan_inum);
+ ret = 1;
+ goto out;
+ }
+ if (pd.abort) {
+ if (pd.errcode) {
+ com_err("process_orphan_block", pd.errcode,
+ _("while reading blocks of inode %d"),
+ orphan_inum);
+ }
+ ret = 1;
+ }
+
+ /* We had to clear some blocks. Report it up. */
+ if (ret == 0 && pd.clear)
+ ret = 2;
+out:
+ ext2fs_free_mem(&orphan_buf);
+ return ret;
+}
+
/*
* Check the resize inode to make sure it is sane. We check both for
* the case where on-line resizing is not enabled (in which case the
diff --git a/e2fsck/unix.c b/e2fsck/unix.c
index c5f9e4415f8f..bf9b0bf2ecb8 100644
--- a/e2fsck/unix.c
+++ b/e2fsck/unix.c
@@ -1945,15 +1945,82 @@ print_unsupp_features:
_("\n*** journal has been regenerated ***\n"));
}
}
-no_journal:
+no_journal:
if (run_result & E2F_FLAG_ABORT) {
fatal_error(ctx, _("aborted"));
} else if (run_result & E2F_FLAG_CANCEL) {
log_out(ctx, _("%s: e2fsck canceled.\n"), ctx->device_name ?
ctx->device_name : ctx->filesystem_name);
exit_value |= FSCK_CANCELED;
- } else if (ctx->qctx && !ctx->invalid_bitmaps) {
+ goto cleanup;
+ }
+
+ if (ext2fs_has_feature_orphan_file(fs->super)) {
+ int ret;
+
+ /* No point in orphan file without a journal... */
+ if (!ext2fs_has_feature_journal(fs->super) &&
+ fix_problem(ctx, PR_6_ORPHAN_FILE_WITHOUT_JOURNAL, &pctx)) {
+ retval = ext2fs_truncate_orphan_file(fs);
+ if (retval) {
+ /* Huh, failed to delete file */
+ fix_problem(ctx, PR_6_ORPHAN_FILE_TRUNC_FAILED,
+ &pctx);
+ goto check_quotas;
+ }
+ ext2fs_clear_feature_orphan_file(fs->super);
+ ext2fs_mark_super_dirty(fs);
+ goto check_quotas;
+ }
+ ret = check_init_orphan_file(ctx);
+ if (ret == 2 ||
+ (ret == 0 && ext2fs_has_feature_orphan_present(fs->super) &&
+ fix_problem(ctx, PR_6_ORPHAN_PRESENT_CLEAN_FILE, &pctx))) {
+ ext2fs_clear_feature_orphan_present(fs->super);
+ ext2fs_mark_super_dirty(fs);
+ } else if (ret == 1 &&
+ fix_problem(ctx, PR_6_ORPHAN_FILE_CORRUPTED, &pctx)) {
+ int orphan_file_blocks;
+
+ if (ctx->invalid_bitmaps) {
+ fix_problem(ctx,
+ PR_6_ORPHAN_FILE_BITMAP_INVALID,
+ &pctx);
+ goto check_quotas;
+ }
+
+ retval = ext2fs_truncate_orphan_file(fs);
+ if (retval) {
+ /* Huh, failed to truncate file */
+ fix_problem(ctx, PR_6_ORPHAN_FILE_TRUNC_FAILED,
+ &pctx);
+ goto check_quotas;
+ }
+
+ orphan_file_blocks =
+ ext2fs_default_orphan_file_blocks(fs);
+ log_out(ctx, _("Creating orphan file (%d blocks): "),
+ orphan_file_blocks);
+ fflush(stdout);
+ retval = ext2fs_create_orphan_file(fs,
+ orphan_file_blocks);
+ if (retval) {
+ log_out(ctx, "%s: while trying to create "
+ "orphan file\n", error_message(retval));
+ fix_problem(ctx, PR_6_ORPHAN_FILE_CREATE_FAILED,
+ &pctx);
+ goto check_quotas;
+ }
+ log_out(ctx, "%s", _(" Done.\n"));
+ }
+ } else if (ext2fs_has_feature_orphan_present(fs->super) &&
+ fix_problem(ctx, PR_6_ORPHAN_PRESENT_NO_FILE, &pctx)) {
+ ext2fs_clear_feature_orphan_present(fs->super);
+ ext2fs_mark_super_dirty(fs);
+ }
+check_quotas:
+ if (ctx->qctx && !ctx->invalid_bitmaps) {
int needs_writeout;
for (qtype = 0; qtype < MAXQUOTAS; qtype++) {
@@ -1988,6 +2055,7 @@ no_journal:
goto restart;
}
+cleanup:
#ifdef MTRACE
mtrace_print("Cleanup");
#endif
--
2.26.2
^ permalink raw reply related [flat|nested] 22+ messages in thread
* [PATCH 7/9] tune2fs: Add support for orphan_file feature
2021-07-12 15:43 [PATCH 0/9 v4] e2fsprogs: Support for orphan file feature Jan Kara
` (5 preceding siblings ...)
2021-07-12 15:43 ` [PATCH 6/9] e2fsck: Add support for handling orphan file Jan Kara
@ 2021-07-12 15:43 ` Jan Kara
2021-07-12 15:43 ` [PATCH 8/9] mke2fs: Add orphan_file feature into mke2fs.conf Jan Kara
2021-07-12 15:43 ` [PATCH 9/9] dumpe2fs, debugfs, e2image: Add support for orphan file Jan Kara
8 siblings, 0 replies; 22+ messages in thread
From: Jan Kara @ 2021-07-12 15:43 UTC (permalink / raw)
To: Ted Tso; +Cc: linux-ext4, Jan Kara
Signed-off-by: Jan Kara <jack@suse.cz>
---
misc/tune2fs.8.in | 5 +++
misc/tune2fs.c | 89 +++++++++++++++++++++++++++++++++++++++++++++--
2 files changed, 92 insertions(+), 2 deletions(-)
diff --git a/misc/tune2fs.8.in b/misc/tune2fs.8.in
index b963f30edef3..849f94b68d6e 100644
--- a/misc/tune2fs.8.in
+++ b/misc/tune2fs.8.in
@@ -257,6 +257,11 @@ program.
This superblock setting is only honored in 2.6.35+ kernels;
and not at all by the ext2 and ext3 file system drivers.
.TP
+.BI orphan_file_size= size
+Set size of the file for tracking unlinked but still open inodes and inodes
+with truncate in progress. Larger file allows for better scalability, reserving
+a few blocks per cpu is ideal.
+.TP
.B force_fsck
Set a flag in the filesystem superblock indicating that errors have been found.
This will force fsck to run at the next mount.
diff --git a/misc/tune2fs.c b/misc/tune2fs.c
index f739f16cd62b..4d4cf5a13384 100644
--- a/misc/tune2fs.c
+++ b/misc/tune2fs.c
@@ -106,6 +106,7 @@ int enabling_casefold;
int journal_size, journal_fc_size, journal_flags;
char *journal_device;
static blk64_t journal_location = ~0LL;
+static e2_blkcnt_t orphan_file_blocks;
static struct list_head blk_move_list;
@@ -152,7 +153,8 @@ static __u32 ok_features[3] = {
EXT3_FEATURE_COMPAT_HAS_JOURNAL |
EXT2_FEATURE_COMPAT_DIR_INDEX |
EXT4_FEATURE_COMPAT_FAST_COMMIT |
- EXT4_FEATURE_COMPAT_STABLE_INODES,
+ EXT4_FEATURE_COMPAT_STABLE_INODES |
+ EXT4_FEATURE_COMPAT_ORPHAN_FILE,
/* Incompat */
EXT2_FEATURE_INCOMPAT_FILETYPE |
EXT3_FEATURE_INCOMPAT_EXTENTS |
@@ -183,7 +185,8 @@ static __u32 clear_ok_features[3] = {
EXT3_FEATURE_COMPAT_HAS_JOURNAL |
EXT2_FEATURE_COMPAT_RESIZE_INODE |
EXT2_FEATURE_COMPAT_DIR_INDEX |
- EXT4_FEATURE_COMPAT_FAST_COMMIT,
+ EXT4_FEATURE_COMPAT_FAST_COMMIT |
+ EXT4_FEATURE_COMPAT_ORPHAN_FILE,
/* Incompat */
EXT2_FEATURE_INCOMPAT_FILETYPE |
EXT4_FEATURE_INCOMPAT_FLEX_BG |
@@ -1143,6 +1146,55 @@ static int update_feature_set(ext2_filsys fs, char *features)
}
}
+ if (FEATURE_OFF(E2P_FEATURE_COMPAT, EXT4_FEATURE_COMPAT_ORPHAN_FILE)) {
+ ext2_ino_t ino;
+
+ if (mount_flags & EXT2_MF_MOUNTED) {
+ fputs(_("The orphan_file feature may only be cleared "
+ "when the filesystem is unmounted.\n"), stderr);
+ return 1;
+ }
+ if (ext2fs_has_feature_orphan_present(sb) && f_flag < 2) {
+ fputs(_("The orphan_present flag is set. Please run "
+ "e2fsck before clearing orphan_file flag.\n"),
+ stderr);
+ return 1;
+ }
+ err = ext2fs_read_bitmaps(fs);
+ if (err) {
+ com_err(program_name, err, "%s",
+ _("while loading bitmaps"));
+ return 1;
+ }
+ err = ext2fs_truncate_orphan_file(fs);
+ if (err) {
+ com_err(program_name, err,
+ _("\n\twhile trying to delete orphan file\n"));
+ return 1;
+ }
+ ino = sb->s_orphan_file_inum;
+ sb->s_orphan_file_inum = 0;
+ ext2fs_inode_alloc_stats2(fs, ino, -1, 0);
+ ext2fs_clear_feature_orphan_file(sb);
+ ext2fs_clear_feature_orphan_present(sb);
+ ext2fs_mark_super_dirty(fs);
+ }
+
+ if (FEATURE_ON(E2P_FEATURE_COMPAT, EXT4_FEATURE_COMPAT_ORPHAN_FILE)) {
+ if (!ext2fs_has_feature_journal(sb)) {
+ fputs(_("orphan_file flag can be set only for "
+ "filesystems with journal.\n"), stderr);
+ return 1;
+ }
+ /*
+ * If adding an orphan file, let the create orphan file
+ * code below handle setting the flag and creating it.
+ * We supply a default size if necessary.
+ */
+ orphan_file_blocks = ext2fs_default_orphan_file_blocks(fs);
+ ext2fs_set_feature_orphan_file(sb);
+ }
+
if (FEATURE_ON(E2P_FEATURE_RO_INCOMPAT,
EXT2_FEATURE_RO_COMPAT_SPARSE_SUPER)) {
if (ext2fs_has_feature_meta_bg(sb)) {
@@ -2262,6 +2314,21 @@ static int parse_extended_opts(ext2_filsys fs, const char *opts)
continue;
}
encoding_flags = arg;
+ } else if (!strcmp(token, "orphan_file_size")) {
+ if (!arg) {
+ r_usage++;
+ continue;
+ }
+ orphan_file_blocks = parse_num_blocks2(arg,
+ fs->super->s_log_block_size);
+
+ if (orphan_file_blocks < 1) {
+ fprintf(stderr,
+ _("Invalid size of orphan file %s\n"),
+ arg);
+ r_usage++;
+ continue;
+ }
} else
r_usage++;
}
@@ -3247,6 +3314,24 @@ _("Warning: The journal is dirty. You may wish to replay the journal like:\n\n"
if (rc)
goto closefs;
}
+ if (orphan_file_blocks) {
+ errcode_t err;
+
+ err = ext2fs_read_bitmaps(fs);
+ if (err) {
+ com_err(program_name, err, "%s",
+ _("while loading bitmaps"));
+ rc = 1;
+ goto closefs;
+ }
+ err = ext2fs_create_orphan_file(fs, orphan_file_blocks);
+ if (err) {
+ com_err(program_name, err, "%s",
+ _("while creating orphan file"));
+ rc = 1;
+ goto closefs;
+ }
+ }
if (Q_flag) {
if (mount_flags & EXT2_MF_MOUNTED) {
--
2.26.2
^ permalink raw reply related [flat|nested] 22+ messages in thread
* [PATCH 8/9] mke2fs: Add orphan_file feature into mke2fs.conf
2021-07-12 15:43 [PATCH 0/9 v4] e2fsprogs: Support for orphan file feature Jan Kara
` (6 preceding siblings ...)
2021-07-12 15:43 ` [PATCH 7/9] tune2fs: Add support for orphan_file feature Jan Kara
@ 2021-07-12 15:43 ` Jan Kara
2021-08-04 19:04 ` Theodore Ts'o
2021-07-12 15:43 ` [PATCH 9/9] dumpe2fs, debugfs, e2image: Add support for orphan file Jan Kara
8 siblings, 1 reply; 22+ messages in thread
From: Jan Kara @ 2021-07-12 15:43 UTC (permalink / raw)
To: Ted Tso; +Cc: linux-ext4, Jan Kara
Enable orphan_file feature by default in larger filesystems. Since the
feature is COMPAT, older kernels will just ignore it and happily work
with the filesystem as well.
Signed-off-by: Jan Kara <jack@suse.cz>
---
misc/mke2fs.conf.in | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/misc/mke2fs.conf.in b/misc/mke2fs.conf.in
index 01e35cf83150..d97d8d643d1d 100644
--- a/misc/mke2fs.conf.in
+++ b/misc/mke2fs.conf.in
@@ -11,15 +11,17 @@
features = has_journal
}
ext4 = {
- features = has_journal,extent,huge_file,flex_bg,metadata_csum,64bit,dir_nlink,extra_isize
+ features = has_journal,extent,huge_file,flex_bg,metadata_csum,64bit,dir_nlink,extra_isize,orphan_file
inode_size = 256
}
small = {
+ default_features = ^orphan_file
blocksize = 1024
inode_size = 128
inode_ratio = 4096
}
floppy = {
+ default_features = ^orphan_file
blocksize = 1024
inode_size = 128
inode_ratio = 8192
--
2.26.2
^ permalink raw reply related [flat|nested] 22+ messages in thread
* [PATCH 9/9] dumpe2fs, debugfs, e2image: Add support for orphan file
2021-07-12 15:43 [PATCH 0/9 v4] e2fsprogs: Support for orphan file feature Jan Kara
` (7 preceding siblings ...)
2021-07-12 15:43 ` [PATCH 8/9] mke2fs: Add orphan_file feature into mke2fs.conf Jan Kara
@ 2021-07-12 15:43 ` Jan Kara
8 siblings, 0 replies; 22+ messages in thread
From: Jan Kara @ 2021-07-12 15:43 UTC (permalink / raw)
To: Ted Tso; +Cc: linux-ext4, Jan Kara
Print inode number of orphan file in outputs, dump e2image file to
filesystem image.
Signed-off-by: Jan Kara <jack@suse.cz>
---
debugfs/set_fields.c | 1 +
lib/e2p/ls.c | 3 +++
misc/e2image.c | 3 ++-
3 files changed, 6 insertions(+), 1 deletion(-)
diff --git a/debugfs/set_fields.c b/debugfs/set_fields.c
index b00157940774..f916deab8cea 100644
--- a/debugfs/set_fields.c
+++ b/debugfs/set_fields.c
@@ -183,6 +183,7 @@ static struct field_set_info super_fields[] = {
{ "lpf_ino", &set_sb.s_lpf_ino, NULL, 4, parse_uint },
{ "checksum_seed", &set_sb.s_checksum_seed, NULL, 4, parse_uint },
{ "encoding", &set_sb.s_encoding, NULL, 2, parse_encoding },
+ { "orphan_file_inum", &set_sb.s_orphan_file_inum, NULL, 4, parse_uint },
{ 0, 0, 0, 0 }
};
diff --git a/lib/e2p/ls.c b/lib/e2p/ls.c
index 176bee0fd19f..1762bc44cac4 100644
--- a/lib/e2p/ls.c
+++ b/lib/e2p/ls.c
@@ -482,6 +482,9 @@ void list_super2(struct ext2_super_block * sb, FILE *f)
if (ext2fs_has_feature_casefold(sb))
fprintf(f, "Character encoding: %s\n",
e2p_encoding2str(sb->s_encoding));
+ if (ext2fs_has_feature_orphan_file(sb))
+ fprintf(f, "Orphan file inode: %u\n",
+ sb->s_orphan_file_inum);
}
void list_super (struct ext2_super_block * s)
diff --git a/misc/e2image.c b/misc/e2image.c
index ac00827e4628..a9c64506d7cc 100644
--- a/misc/e2image.c
+++ b/misc/e2image.c
@@ -1369,7 +1369,8 @@ static void write_raw_image_file(ext2_filsys fs, int fd, int type, int flags,
ino == fs->super->s_journal_inum ||
ino == quota_type2inum(USRQUOTA, fs->super) ||
ino == quota_type2inum(GRPQUOTA, fs->super) ||
- ino == quota_type2inum(PRJQUOTA, fs->super)) {
+ ino == quota_type2inum(PRJQUOTA, fs->super) ||
+ ino == fs->super->s_orphan_file_inum) {
retval = ext2fs_block_iterate3(fs, ino,
BLOCK_FLAG_READ_ONLY, block_buf,
process_dir_block, &pb);
--
2.26.2
^ permalink raw reply related [flat|nested] 22+ messages in thread
* Re: [PATCH 1/9] ext2fs: Drop HAS_SNAPSHOT feature
2021-07-12 15:43 ` [PATCH 1/9] ext2fs: Drop HAS_SNAPSHOT feature Jan Kara
@ 2021-07-12 18:20 ` Andreas Dilger
2021-07-13 8:11 ` Jan Kara
0 siblings, 1 reply; 22+ messages in thread
From: Andreas Dilger @ 2021-07-12 18:20 UTC (permalink / raw)
To: Jan Kara; +Cc: Ted Tso, linux-ext4
NAK.
We are working on a snapshot implementation for ext4, it is
just taking a lot longer that I thought it would to complete.
There isn't any shortage of these feature bits, so no reason to re-use them.
Cheers, Andreas
> On Jul 12, 2021, at 09:43, Jan Kara <jack@suse.cz> wrote:
>
> It has never been implemented and is dead for quite some time and
> unused AFAICT.
>
> Signed-off-by: Jan Kara <jack@suse.cz>
> ---
> lib/ext2fs/ext2_fs.h | 2 --
> 1 file changed, 2 deletions(-)
>
> diff --git a/lib/ext2fs/ext2_fs.h b/lib/ext2fs/ext2_fs.h
> index e92a045205a9..6f1d5db4b482 100644
> --- a/lib/ext2fs/ext2_fs.h
> +++ b/lib/ext2fs/ext2_fs.h
> @@ -825,7 +825,6 @@ struct ext2_super_block {
> #define EXT4_FEATURE_RO_COMPAT_GDT_CSUM 0x0010
> #define EXT4_FEATURE_RO_COMPAT_DIR_NLINK 0x0020
> #define EXT4_FEATURE_RO_COMPAT_EXTRA_ISIZE 0x0040
> -#define EXT4_FEATURE_RO_COMPAT_HAS_SNAPSHOT 0x0080
> #define EXT4_FEATURE_RO_COMPAT_QUOTA 0x0100
> #define EXT4_FEATURE_RO_COMPAT_BIGALLOC 0x0200
> /*
> @@ -926,7 +925,6 @@ EXT4_FEATURE_RO_COMPAT_FUNCS(huge_file, 4, HUGE_FILE)
> EXT4_FEATURE_RO_COMPAT_FUNCS(gdt_csum, 4, GDT_CSUM)
> EXT4_FEATURE_RO_COMPAT_FUNCS(dir_nlink, 4, DIR_NLINK)
> EXT4_FEATURE_RO_COMPAT_FUNCS(extra_isize, 4, EXTRA_ISIZE)
> -EXT4_FEATURE_RO_COMPAT_FUNCS(has_snapshot, 4, HAS_SNAPSHOT)
> EXT4_FEATURE_RO_COMPAT_FUNCS(quota, 4, QUOTA)
> EXT4_FEATURE_RO_COMPAT_FUNCS(bigalloc, 4, BIGALLOC)
> EXT4_FEATURE_RO_COMPAT_FUNCS(metadata_csum, 4, METADATA_CSUM)
> --
> 2.26.2
>
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [PATCH 1/9] ext2fs: Drop HAS_SNAPSHOT feature
2021-07-12 18:20 ` Andreas Dilger
@ 2021-07-13 8:11 ` Jan Kara
0 siblings, 0 replies; 22+ messages in thread
From: Jan Kara @ 2021-07-13 8:11 UTC (permalink / raw)
To: Andreas Dilger; +Cc: Jan Kara, Ted Tso, linux-ext4
Hello,
On Mon 12-07-21 12:20:03, Andreas Dilger wrote:
> NAK.
>
> We are working on a snapshot implementation for ext4, it is
> just taking a lot longer that I thought it would to complete.
>
> There isn't any shortage of these feature bits, so no reason to re-use them.
Sure. I thought this feature bit was a leftover from experimental ext4
snapshot feature Amir Goldstein was playing with some 10-15 years ago and
which never completed. If there's somebody working on this, I'll allocate a
different bit.
Honza
>
> Cheers, Andreas
>
> > On Jul 12, 2021, at 09:43, Jan Kara <jack@suse.cz> wrote:
> >
> > It has never been implemented and is dead for quite some time and
> > unused AFAICT.
> >
> > Signed-off-by: Jan Kara <jack@suse.cz>
> > ---
> > lib/ext2fs/ext2_fs.h | 2 --
> > 1 file changed, 2 deletions(-)
> >
> > diff --git a/lib/ext2fs/ext2_fs.h b/lib/ext2fs/ext2_fs.h
> > index e92a045205a9..6f1d5db4b482 100644
> > --- a/lib/ext2fs/ext2_fs.h
> > +++ b/lib/ext2fs/ext2_fs.h
> > @@ -825,7 +825,6 @@ struct ext2_super_block {
> > #define EXT4_FEATURE_RO_COMPAT_GDT_CSUM 0x0010
> > #define EXT4_FEATURE_RO_COMPAT_DIR_NLINK 0x0020
> > #define EXT4_FEATURE_RO_COMPAT_EXTRA_ISIZE 0x0040
> > -#define EXT4_FEATURE_RO_COMPAT_HAS_SNAPSHOT 0x0080
> > #define EXT4_FEATURE_RO_COMPAT_QUOTA 0x0100
> > #define EXT4_FEATURE_RO_COMPAT_BIGALLOC 0x0200
> > /*
> > @@ -926,7 +925,6 @@ EXT4_FEATURE_RO_COMPAT_FUNCS(huge_file, 4, HUGE_FILE)
> > EXT4_FEATURE_RO_COMPAT_FUNCS(gdt_csum, 4, GDT_CSUM)
> > EXT4_FEATURE_RO_COMPAT_FUNCS(dir_nlink, 4, DIR_NLINK)
> > EXT4_FEATURE_RO_COMPAT_FUNCS(extra_isize, 4, EXTRA_ISIZE)
> > -EXT4_FEATURE_RO_COMPAT_FUNCS(has_snapshot, 4, HAS_SNAPSHOT)
> > EXT4_FEATURE_RO_COMPAT_FUNCS(quota, 4, QUOTA)
> > EXT4_FEATURE_RO_COMPAT_FUNCS(bigalloc, 4, BIGALLOC)
> > EXT4_FEATURE_RO_COMPAT_FUNCS(metadata_csum, 4, METADATA_CSUM)
> > --
> > 2.26.2
> >
--
Jan Kara <jack@suse.com>
SUSE Labs, CR
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [PATCH 4/9] libext2fs: Support for orphan file feature
2021-07-12 15:43 ` [PATCH 4/9] libext2fs: Support for orphan file feature Jan Kara
@ 2021-08-03 16:57 ` Theodore Ts'o
2021-08-03 17:40 ` Theodore Ts'o
2021-08-04 9:25 ` Jan Kara
0 siblings, 2 replies; 22+ messages in thread
From: Theodore Ts'o @ 2021-08-03 16:57 UTC (permalink / raw)
To: Jan Kara; +Cc: linux-ext4
On Mon, Jul 12, 2021 at 05:43:10PM +0200, Jan Kara wrote:
> @@ -825,6 +826,7 @@ struct ext2_super_block {
> #define EXT4_FEATURE_RO_COMPAT_GDT_CSUM 0x0010
> #define EXT4_FEATURE_RO_COMPAT_DIR_NLINK 0x0020
> #define EXT4_FEATURE_RO_COMPAT_EXTRA_ISIZE 0x0040
> +#define EXT4_FEATURE_RO_COMPAT_ORPHAN_PRESENT 0x0080
> #define EXT4_FEATURE_RO_COMPAT_QUOTA 0x0100
> #define EXT4_FEATURE_RO_COMPAT_BIGALLOC 0x0200
(This isn't a full review of the patch, but just a quick feedback of
what I've noticed so far.)
Since Andreas has requested that we not get rid of the
RO_COMPAT_SNAPSHOT, I'm using 0x0400 for
EXT4_FEATURE_RO_COMPAT_ORPHAN_PRESENT in my testing.
I also noted a number of new GCC warnings when running "make gcc-wall"
on lib/ext2fs after applying this commit.
- Ted
/usr/projects/e2fsprogs/e2fsprogs/lib/ext2fs/orphan.c: In function ‘ext2fs_do_orphan_file_block_csum’:
/usr/projects/e2fsprogs/e2fsprogs/lib/ext2fs/orphan.c:58:30: warning: pointer targets in passing argument 2 of ‘ext2fs_crc32c_le’ differ in signedness [-Wpointer-sign]
58 | crc = ext2fs_crc32c_le(crc, buf, inodes_per_ob * sizeof(__u32));
| ^~~
| |
| char *
In file included from /usr/projects/e2fsprogs/e2fsprogs/lib/ext2fs/ext2fsP.h:16,
from /usr/projects/e2fsprogs/e2fsprogs/lib/ext2fs/orphan.c:16:
/usr/projects/e2fsprogs/e2fsprogs/lib/ext2fs/ext2fs.h:1075:63: note: expected ‘const unsigned char *’ but argument is of type ‘char *’
1075 | extern __u32 ext2fs_crc32c_le(__u32 crc, unsigned char const *p, size_t len);
| ~~~~~~~~~~~~~~~~~~~~~^
/usr/projects/e2fsprogs/e2fsprogs/lib/ext2fs/orphan.c: In function ‘ext2fs_do_orphan_file_block_csum’:
/usr/projects/e2fsprogs/e2fsprogs/lib/ext2fs/orphan.c:58:30: warning: pointer targets in passing argument 2 of ‘ext2fs_crc32c_le’ differ in signedness [-Wpointer-sign]
58 | crc = ext2fs_crc32c_le(crc, buf, inodes_per_ob * sizeof(__u32));
| ^~~
| |
| char *
In file included from /usr/projects/e2fsprogs/e2fsprogs/lib/ext2fs/ext2fsP.h:16,
from /usr/projects/e2fsprogs/e2fsprogs/lib/ext2fs/orphan.c:16:
/usr/projects/e2fsprogs/e2fsprogs/lib/ext2fs/ext2fs.h:1075:63: note: expected ‘const unsigned char *’ but argument is of type ‘char *’
1075 | extern __u32 ext2fs_crc32c_le(__u32 crc, unsigned char const *p, size_t len);
| ~~~~~~~~~~~~~~~~~~~~~^
make[1]: Leaving directory '/build/e2fsprogs/lib/ext2fs'
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [PATCH 4/9] libext2fs: Support for orphan file feature
2021-08-03 16:57 ` Theodore Ts'o
@ 2021-08-03 17:40 ` Theodore Ts'o
2021-08-04 9:25 ` Jan Kara
1 sibling, 0 replies; 22+ messages in thread
From: Theodore Ts'o @ 2021-08-03 17:40 UTC (permalink / raw)
To: Jan Kara; +Cc: linux-ext4
On Tue, Aug 03, 2021 at 12:57:36PM -0400, Theodore Ts'o wrote:
> On Mon, Jul 12, 2021 at 05:43:10PM +0200, Jan Kara wrote:
> > @@ -825,6 +826,7 @@ struct ext2_super_block {
> > #define EXT4_FEATURE_RO_COMPAT_GDT_CSUM 0x0010
> > #define EXT4_FEATURE_RO_COMPAT_DIR_NLINK 0x0020
> > #define EXT4_FEATURE_RO_COMPAT_EXTRA_ISIZE 0x0040
> > +#define EXT4_FEATURE_RO_COMPAT_ORPHAN_PRESENT 0x0080
> > #define EXT4_FEATURE_RO_COMPAT_QUOTA 0x0100
> > #define EXT4_FEATURE_RO_COMPAT_BIGALLOC 0x0200
>
> (This isn't a full review of the patch, but just a quick feedback of
> what I've noticed so far.)
>
> Since Andreas has requested that we not get rid of the
> RO_COMPAT_SNAPSHOT, I'm using 0x0400 for
> EXT4_FEATURE_RO_COMPAT_ORPHAN_PRESENT in my testing.
Correction, this should have been 0x10000.
- Ted
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [PATCH 6/9] e2fsck: Add support for handling orphan file
2021-07-12 15:43 ` [PATCH 6/9] e2fsck: Add support for handling orphan file Jan Kara
@ 2021-08-03 18:56 ` Theodore Ts'o
0 siblings, 0 replies; 22+ messages in thread
From: Theodore Ts'o @ 2021-08-03 18:56 UTC (permalink / raw)
To: Jan Kara; +Cc: linux-ext4
On Mon, Jul 12, 2021 at 05:43:12PM +0200, Jan Kara wrote:
> diff --git a/e2fsck/problem.h b/e2fsck/problem.h
> index 24cdcf9b90f7..0611d71f9e03 100644
> --- a/e2fsck/problem.h
> +++ b/e2fsck/problem.h
> @@ -717,6 +729,15 @@ struct problem_context {
> #define PR_1_HTREE_CANNOT_SIPHASH 0x01008E
>
>
> +/* Orphan file inode is not a regular file */
> +#define PR_1_ORPHAN_FILE_BAD_MODE 0x01007F
> +
> +/* Orphan file inode is not in use, but contains data */
> +#define PR_1_ORPHAN_FILE_NOT_CLEAR 0x010080
> +
> +/* Orphan file inode is not clear */
> +#define PR_1_ORPHAN_INODE_NOT_CLEAR 0x01007F
> +
The problem codes for PR_1_ORPHAN_FILE_BAD_MODE,
PR_1_ORPHAN_FILE_NOT_CLEAR, and PR_1_ORPHAN_INODE_NOT_CLEAR overlap
with pre-existing problem codes. This was picked up by running "make
check", either at the top-level or in the e2fsck subdirectory. I've
fixed this up in my repo.
I've pushed out the slightly massaged e2fsprogs parallel orphan list
patches on the "pu" (proposed updates, a terminology that Junio uses
for the git repo) branch, for folks who want to experiment with the
parallel orphan list patches.
- Ted
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [PATCH 4/9] libext2fs: Support for orphan file feature
2021-08-03 16:57 ` Theodore Ts'o
2021-08-03 17:40 ` Theodore Ts'o
@ 2021-08-04 9:25 ` Jan Kara
2021-08-04 10:13 ` Jan Kara
1 sibling, 1 reply; 22+ messages in thread
From: Jan Kara @ 2021-08-04 9:25 UTC (permalink / raw)
To: Theodore Ts'o; +Cc: Jan Kara, linux-ext4
On Tue 03-08-21 12:57:36, Theodore Ts'o wrote:
> On Mon, Jul 12, 2021 at 05:43:10PM +0200, Jan Kara wrote:
> > @@ -825,6 +826,7 @@ struct ext2_super_block {
> > #define EXT4_FEATURE_RO_COMPAT_GDT_CSUM 0x0010
> > #define EXT4_FEATURE_RO_COMPAT_DIR_NLINK 0x0020
> > #define EXT4_FEATURE_RO_COMPAT_EXTRA_ISIZE 0x0040
> > +#define EXT4_FEATURE_RO_COMPAT_ORPHAN_PRESENT 0x0080
> > #define EXT4_FEATURE_RO_COMPAT_QUOTA 0x0100
> > #define EXT4_FEATURE_RO_COMPAT_BIGALLOC 0x0200
>
> (This isn't a full review of the patch, but just a quick feedback of
> what I've noticed so far.)
>
> Since Andreas has requested that we not get rid of the
> RO_COMPAT_SNAPSHOT, I'm using 0x0400 for
> EXT4_FEATURE_RO_COMPAT_ORPHAN_PRESENT in my testing.
Yeah, I'm sorry. Somehow older version of the patch escaped to this posting
(I've checked and this was the only difference between what I have in git
and what I have posted).
> I also noted a number of new GCC warnings when running "make gcc-wall"
> on lib/ext2fs after applying this commit.
I'll check these fix them up and repost. Thanks for noticing.
Honza
--
Jan Kara <jack@suse.com>
SUSE Labs, CR
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [PATCH 4/9] libext2fs: Support for orphan file feature
2021-08-04 9:25 ` Jan Kara
@ 2021-08-04 10:13 ` Jan Kara
0 siblings, 0 replies; 22+ messages in thread
From: Jan Kara @ 2021-08-04 10:13 UTC (permalink / raw)
To: Theodore Ts'o; +Cc: Jan Kara, linux-ext4
On Wed 04-08-21 11:25:37, Jan Kara wrote:
> On Tue 03-08-21 12:57:36, Theodore Ts'o wrote:
> > On Mon, Jul 12, 2021 at 05:43:10PM +0200, Jan Kara wrote:
> > > @@ -825,6 +826,7 @@ struct ext2_super_block {
> > > #define EXT4_FEATURE_RO_COMPAT_GDT_CSUM 0x0010
> > > #define EXT4_FEATURE_RO_COMPAT_DIR_NLINK 0x0020
> > > #define EXT4_FEATURE_RO_COMPAT_EXTRA_ISIZE 0x0040
> > > +#define EXT4_FEATURE_RO_COMPAT_ORPHAN_PRESENT 0x0080
> > > #define EXT4_FEATURE_RO_COMPAT_QUOTA 0x0100
> > > #define EXT4_FEATURE_RO_COMPAT_BIGALLOC 0x0200
> >
> > (This isn't a full review of the patch, but just a quick feedback of
> > what I've noticed so far.)
> >
> > Since Andreas has requested that we not get rid of the
> > RO_COMPAT_SNAPSHOT, I'm using 0x0400 for
> > EXT4_FEATURE_RO_COMPAT_ORPHAN_PRESENT in my testing.
>
> Yeah, I'm sorry. Somehow older version of the patch escaped to this posting
> (I've checked and this was the only difference between what I have in git
> and what I have posted).
>
> > I also noted a number of new GCC warnings when running "make gcc-wall"
> > on lib/ext2fs after applying this commit.
>
> I'll check these fix them up and repost. Thanks for noticing.
OK, I have all the problems fixed up in my local branch. Since they were
all rather local issues and you have something working in your tree I guess
I'll wait with reposting the series if you have some other review comments.
But feel free to speak up if you want the current state of the series
posted.
Honza
--
Jan Kara <jack@suse.com>
SUSE Labs, CR
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [PATCH 8/9] mke2fs: Add orphan_file feature into mke2fs.conf
2021-07-12 15:43 ` [PATCH 8/9] mke2fs: Add orphan_file feature into mke2fs.conf Jan Kara
@ 2021-08-04 19:04 ` Theodore Ts'o
2021-08-05 15:00 ` Jan Kara
0 siblings, 1 reply; 22+ messages in thread
From: Theodore Ts'o @ 2021-08-04 19:04 UTC (permalink / raw)
To: Jan Kara; +Cc: linux-ext4
On Mon, Jul 12, 2021 at 05:43:14PM +0200, Jan Kara wrote:
> Enable orphan_file feature by default in larger filesystems. Since the
> feature is COMPAT, older kernels will just ignore it and happily work
> with the filesystem as well.
>
> Signed-off-by: Jan Kara <jack@suse.cz>
We'll need to decide whether we want to enable this by default, at
least initially. The general practice has been to not enable new
kernel functionality right away by default. It's true that older
kernels will ignore the feature if they aren't orphan_file aware;
however, we if have a file system which is created with orphan_file
eanbled, but that file system with the orphan_file feature is made
available to a system which is running an orphan_file-aware kernel,
but the distro hadn't picked up the a version e2fsprogs which is
orphan_file-aware. This might happen if the file system was created
on one system, and then it gets connected to an system w/o a new
version of e2fsprogs (e.g. via fibre channel, iscsi, AWS EBS, GCE PD,
etc), then could be a surprise to the user. So that's something for
us to discuss.
In the shorter term, there's another problem I've notied, which is if
we add this to mke2fs.conf, and the user runs:
mke2fs -t ext4 -O ^has_journal foo.img 1G
mke2fs will fail mid-way through the mkfs process with the error
message, "mke2fs: cannot set orphan_file flag without a journal".
This represents a regression, and if we don't want to drop orphan_file
from the default feature set in mke2fs.conf, I think we'll need to
check for the case where the file system doesn't have a journal, and
only fail when the user has explicitly requested orphan_file on the
command line. But if orphan_file is a default as defined in
mke2fs.conf, and the journal is not present for whatever reaseon, we
need to silently disable the orphan_file feature.
(Also note that to avoid user confusion, we should refer to
orphan_file as a "feature" instead of a "flag". Even for things like
"orphan_present" or "recovery_needed" it is probably clearer to call
them features, simply because it makes it easier for system
adminsitrators and developers to be able to find the "flag" location.)
Cheers,
- Ted
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [PATCH 8/9] mke2fs: Add orphan_file feature into mke2fs.conf
2021-08-04 19:04 ` Theodore Ts'o
@ 2021-08-05 15:00 ` Jan Kara
0 siblings, 0 replies; 22+ messages in thread
From: Jan Kara @ 2021-08-05 15:00 UTC (permalink / raw)
To: Theodore Ts'o; +Cc: Jan Kara, linux-ext4
On Wed 04-08-21 15:04:10, Theodore Ts'o wrote:
> On Mon, Jul 12, 2021 at 05:43:14PM +0200, Jan Kara wrote:
> > Enable orphan_file feature by default in larger filesystems. Since the
> > feature is COMPAT, older kernels will just ignore it and happily work
> > with the filesystem as well.
> >
> > Signed-off-by: Jan Kara <jack@suse.cz>
>
> We'll need to decide whether we want to enable this by default, at
> least initially. The general practice has been to not enable new
> kernel functionality right away by default. It's true that older
> kernels will ignore the feature if they aren't orphan_file aware;
> however, we if have a file system which is created with orphan_file
> eanbled, but that file system with the orphan_file feature is made
> available to a system which is running an orphan_file-aware kernel,
> but the distro hadn't picked up the a version e2fsprogs which is
> orphan_file-aware. This might happen if the file system was created
> on one system, and then it gets connected to an system w/o a new
> version of e2fsprogs (e.g. via fibre channel, iscsi, AWS EBS, GCE PD,
> etc), then could be a surprise to the user. So that's something for
> us to discuss.
Yes, I think you are right that enabling the functionality by default right
from the start may be too aggressive. I'm fine with postponing this for
some time.
> In the shorter term, there's another problem I've notied, which is if
> we add this to mke2fs.conf, and the user runs:
>
> mke2fs -t ext4 -O ^has_journal foo.img 1G
>
> mke2fs will fail mid-way through the mkfs process with the error
> message, "mke2fs: cannot set orphan_file flag without a journal".
> This represents a regression, and if we don't want to drop orphan_file
> from the default feature set in mke2fs.conf, I think we'll need to
> check for the case where the file system doesn't have a journal, and
> only fail when the user has explicitly requested orphan_file on the
> command line. But if orphan_file is a default as defined in
> mke2fs.conf, and the journal is not present for whatever reaseon, we
> need to silently disable the orphan_file feature.
Good catch. I'll fix this.
> (Also note that to avoid user confusion, we should refer to
> orphan_file as a "feature" instead of a "flag". Even for things like
> "orphan_present" or "recovery_needed" it is probably clearer to call
> them features, simply because it makes it easier for system
> adminsitrators and developers to be able to find the "flag" location.)
Good point. I've fixed up couple of occurences of 'flag' I've found in the
messages.
Honza
--
Jan Kara <jack@suse.com>
SUSE Labs, CR
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [PATCH 4/9] libext2fs: Support for orphan file feature
2021-08-03 17:36 ` Theodore Ts'o
@ 2021-08-03 17:57 ` Theodore Ts'o
0 siblings, 0 replies; 22+ messages in thread
From: Theodore Ts'o @ 2021-08-03 17:57 UTC (permalink / raw)
To: Jan Kara; +Cc: linux-ext4
On Tue, Aug 03, 2021 at 01:36:59PM -0400, Theodore Ts'o wrote:
> On Mon, Jul 12, 2021 at 05:43:10PM +0200, Jan Kara wrote:
> > @@ -825,6 +826,7 @@ struct ext2_super_block {
> > #define EXT4_FEATURE_RO_COMPAT_GDT_CSUM 0x0010
> > #define EXT4_FEATURE_RO_COMPAT_DIR_NLINK 0x0020
> > #define EXT4_FEATURE_RO_COMPAT_EXTRA_ISIZE 0x0040
> > +#define EXT4_FEATURE_RO_COMPAT_ORPHAN_PRESENT 0x0080
> > #define EXT4_FEATURE_RO_COMPAT_QUOTA 0x0100
> > #define EXT4_FEATURE_RO_COMPAT_BIGALLOC 0x0200
>
> (This isn't a full review of the patch, but just a quick feedback of
> what I've noticed so far.)
>
> Since Andreas has requested that we not get rid of the
> RO_COMPAT_SNAPSHOT, I'm using 0x10000 for
> EXT4_FEATURE_RO_COMPAT_ORPHAN_PRESENT in my testing.
>
> I also noted a number of new GCC warnings when running "make gcc-wall"
> on lib/ext2fs after applying this commit.
Sorry, I just realized I had used b4 to grab the v3 instead of the v4
of the patch series. Fortunately it looks like the 2nd and 3rd
patches hadn't changed between the two patch series, so I'll regroup
and try again this time with the v4 patches.
- Ted
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [PATCH 4/9] libext2fs: Support for orphan file feature
2021-06-16 10:57 ` [PATCH 4/9] libext2fs: " Jan Kara
@ 2021-08-03 17:36 ` Theodore Ts'o
2021-08-03 17:57 ` Theodore Ts'o
0 siblings, 1 reply; 22+ messages in thread
From: Theodore Ts'o @ 2021-08-03 17:36 UTC (permalink / raw)
To: Jan Kara; +Cc: linux-ext4
On Mon, Jul 12, 2021 at 05:43:10PM +0200, Jan Kara wrote:
> @@ -825,6 +826,7 @@ struct ext2_super_block {
> #define EXT4_FEATURE_RO_COMPAT_GDT_CSUM 0x0010
> #define EXT4_FEATURE_RO_COMPAT_DIR_NLINK 0x0020
> #define EXT4_FEATURE_RO_COMPAT_EXTRA_ISIZE 0x0040
> +#define EXT4_FEATURE_RO_COMPAT_ORPHAN_PRESENT 0x0080
> #define EXT4_FEATURE_RO_COMPAT_QUOTA 0x0100
> #define EXT4_FEATURE_RO_COMPAT_BIGALLOC 0x0200
(This isn't a full review of the patch, but just a quick feedback of
what I've noticed so far.)
Since Andreas has requested that we not get rid of the
RO_COMPAT_SNAPSHOT, I'm using 0x10000 for
EXT4_FEATURE_RO_COMPAT_ORPHAN_PRESENT in my testing.
I also noted a number of new GCC warnings when running "make gcc-wall"
on lib/ext2fs after applying this commit.
- Ted
/usr/projects/e2fsprogs/e2fsprogs/lib/ext2fs/orphan.c: In function ‘ext2fs_do_orphan_file_block_csum’:
/usr/projects/e2fsprogs/e2fsprogs/lib/ext2fs/orphan.c:58:30: warning: pointer targets in passing argument 2 of ‘ext2fs_crc32c_le’ differ in signedness [-Wpointer-sign]
58 | crc = ext2fs_crc32c_le(crc, buf, inodes_per_ob * sizeof(__u32));
| ^~~
| |
| char *
In file included from /usr/projects/e2fsprogs/e2fsprogs/lib/ext2fs/ext2fsP.h:16,
from /usr/projects/e2fsprogs/e2fsprogs/lib/ext2fs/orphan.c:16:
/usr/projects/e2fsprogs/e2fsprogs/lib/ext2fs/ext2fs.h:1075:63: note: expected ‘const unsigned char *’ but argument is of type ‘char *’
1075 | extern __u32 ext2fs_crc32c_le(__u32 crc, unsigned char const *p, size_t len);
| ~~~~~~~~~~~~~~~~~~~~~^
/usr/projects/e2fsprogs/e2fsprogs/lib/ext2fs/orphan.c: In function ‘ext2fs_do_orphan_file_block_csum’:
/usr/projects/e2fsprogs/e2fsprogs/lib/ext2fs/orphan.c:58:30: warning: pointer targets in passing argument 2 of ‘ext2fs_crc32c_le’ differ in signedness [-Wpointer-sign]
58 | crc = ext2fs_crc32c_le(crc, buf, inodes_per_ob * sizeof(__u32));
| ^~~
| |
| char *
In file included from /usr/projects/e2fsprogs/e2fsprogs/lib/ext2fs/ext2fsP.h:16,
from /usr/projects/e2fsprogs/e2fsprogs/lib/ext2fs/orphan.c:16:
/usr/projects/e2fsprogs/e2fsprogs/lib/ext2fs/ext2fs.h:1075:63: note: expected ‘const unsigned char *’ but argument is of type ‘char *’
1075 | extern __u32 ext2fs_crc32c_le(__u32 crc, unsigned char const *p, size_t len);
| ~~~~~~~~~~~~~~~~~~~~~^
make[1]: Leaving directory '/build/e2fsprogs/lib/ext2fs'
^ permalink raw reply [flat|nested] 22+ messages in thread
* [PATCH 4/9] libext2fs: Support for orphan file feature
2021-06-16 10:57 [PATCH 0/9 v3] e2fsprogs: Support for orphan file feature Jan Kara
@ 2021-06-16 10:57 ` Jan Kara
2021-08-03 17:36 ` Theodore Ts'o
0 siblings, 1 reply; 22+ messages in thread
From: Jan Kara @ 2021-06-16 10:57 UTC (permalink / raw)
To: Ted Tso; +Cc: linux-ext4, Jan Kara
Add support for creating and deleting orphan file and a couple of
utility functions that will be used in other tools.
Signed-off-by: Jan Kara <jack@suse.cz>
---
lib/e2p/feature.c | 4 +
lib/ext2fs/Makefile.in | 2 +
lib/ext2fs/ext2_fs.h | 16 ++-
lib/ext2fs/ext2fs.h | 32 ++++-
lib/ext2fs/orphan.c | 252 ++++++++++++++++++++++++++++++++++++
lib/ext2fs/swapfs.c | 3 +-
lib/ext2fs/tst_super_size.c | 3 +-
lib/support/mkquota.c | 3 +-
8 files changed, 307 insertions(+), 8 deletions(-)
create mode 100644 lib/ext2fs/orphan.c
diff --git a/lib/e2p/feature.c b/lib/e2p/feature.c
index 2291060214ff..29b7b1512400 100644
--- a/lib/e2p/feature.c
+++ b/lib/e2p/feature.c
@@ -49,6 +49,8 @@ static struct feature feature_list[] = {
"fast_commit" },
{ E2P_FEATURE_COMPAT, EXT4_FEATURE_COMPAT_STABLE_INODES,
"stable_inodes" },
+ { E2P_FEATURE_COMPAT, EXT4_FEATURE_COMPAT_ORPHAN_FILE,
+ "orphan_file" },
{ E2P_FEATURE_RO_INCOMPAT, EXT2_FEATURE_RO_COMPAT_SPARSE_SUPER,
"sparse_super" },
@@ -80,6 +82,8 @@ static struct feature feature_list[] = {
"shared_blocks"},
{ E2P_FEATURE_RO_INCOMPAT, EXT4_FEATURE_RO_COMPAT_VERITY,
"verity"},
+ { E2P_FEATURE_RO_INCOMPAT, EXT4_FEATURE_RO_COMPAT_ORPHAN_PRESENT,
+ "orphan_present" },
{ E2P_FEATURE_INCOMPAT, EXT2_FEATURE_INCOMPAT_COMPRESSION,
"compression" },
diff --git a/lib/ext2fs/Makefile.in b/lib/ext2fs/Makefile.in
index 5d9af86e520b..ffbfd7a7fc33 100644
--- a/lib/ext2fs/Makefile.in
+++ b/lib/ext2fs/Makefile.in
@@ -115,6 +115,7 @@ OBJS= $(DEBUGFS_LIB_OBJS) $(RESIZE_LIB_OBJS) $(E2IMAGE_LIB_OBJS) \
newdir.o \
nls_utf8.o \
openfs.o \
+ orphan.o \
progress.o \
punch.o \
qcow2.o \
@@ -198,6 +199,7 @@ SRCS= ext2_err.c \
$(srcdir)/newdir.c \
$(srcdir)/nls_utf8.c \
$(srcdir)/openfs.c \
+ $(srcdir)/orphan.c \
$(srcdir)/progress.c \
$(srcdir)/punch.c \
$(srcdir)/qcow2.c \
diff --git a/lib/ext2fs/ext2_fs.h b/lib/ext2fs/ext2_fs.h
index 6f1d5db4b482..00809e7b92be 100644
--- a/lib/ext2fs/ext2_fs.h
+++ b/lib/ext2fs/ext2_fs.h
@@ -761,7 +761,8 @@ struct ext2_super_block {
__u8 s_last_error_errcode;
/*27c*/ __le16 s_encoding; /* Filename charset encoding */
__le16 s_encoding_flags; /* Filename charset encoding flags */
- __le32 s_reserved[95]; /* Padding to the end of the block */
+ __le32 s_orphan_file_inum; /* Inode for tracking orphan inodes */
+ __le32 s_reserved[94]; /* Padding to the end of the block */
/*3fc*/ __u32 s_checksum; /* crc32c(superblock) */
};
@@ -816,7 +817,7 @@ struct ext2_super_block {
#define EXT4_FEATURE_COMPAT_SPARSE_SUPER2 0x0200
#define EXT4_FEATURE_COMPAT_FAST_COMMIT 0x0400
#define EXT4_FEATURE_COMPAT_STABLE_INODES 0x0800
-
+#define EXT4_FEATURE_COMPAT_ORPHAN_FILE 0x1000
#define EXT2_FEATURE_RO_COMPAT_SPARSE_SUPER 0x0001
#define EXT2_FEATURE_RO_COMPAT_LARGE_FILE 0x0002
@@ -825,6 +826,7 @@ struct ext2_super_block {
#define EXT4_FEATURE_RO_COMPAT_GDT_CSUM 0x0010
#define EXT4_FEATURE_RO_COMPAT_DIR_NLINK 0x0020
#define EXT4_FEATURE_RO_COMPAT_EXTRA_ISIZE 0x0040
+#define EXT4_FEATURE_RO_COMPAT_ORPHAN_PRESENT 0x0080
#define EXT4_FEATURE_RO_COMPAT_QUOTA 0x0100
#define EXT4_FEATURE_RO_COMPAT_BIGALLOC 0x0200
/*
@@ -918,6 +920,7 @@ EXT4_FEATURE_COMPAT_FUNCS(exclude_bitmap, 2, EXCLUDE_BITMAP)
EXT4_FEATURE_COMPAT_FUNCS(sparse_super2, 4, SPARSE_SUPER2)
EXT4_FEATURE_COMPAT_FUNCS(fast_commit, 4, FAST_COMMIT)
EXT4_FEATURE_COMPAT_FUNCS(stable_inodes, 4, STABLE_INODES)
+EXT4_FEATURE_COMPAT_FUNCS(orphan_file, 4, ORPHAN_FILE)
EXT4_FEATURE_RO_COMPAT_FUNCS(sparse_super, 2, SPARSE_SUPER)
EXT4_FEATURE_RO_COMPAT_FUNCS(large_file, 2, LARGE_FILE)
@@ -933,6 +936,7 @@ EXT4_FEATURE_RO_COMPAT_FUNCS(readonly, 4, READONLY)
EXT4_FEATURE_RO_COMPAT_FUNCS(project, 4, PROJECT)
EXT4_FEATURE_RO_COMPAT_FUNCS(shared_blocks, 4, SHARED_BLOCKS)
EXT4_FEATURE_RO_COMPAT_FUNCS(verity, 4, VERITY)
+EXT4_FEATURE_RO_COMPAT_FUNCS(orphan_present, 4, ORPHAN_PRESENT)
EXT4_FEATURE_INCOMPAT_FUNCS(compression, 2, COMPRESSION)
EXT4_FEATURE_INCOMPAT_FUNCS(filetype, 2, FILETYPE)
@@ -1100,6 +1104,14 @@ static inline unsigned int ext2fs_dir_rec_len(__u8 name_len,
return rec_len;
}
+#define EXT4_ORPHAN_BLOCK_MAGIC 0x0b10ca04
+
+/* Structure at the tail of orphan block */
+struct ext4_orphan_block_tail {
+ __u32 ob_magic;
+ __u32 ob_checksum;
+};
+
/*
* Constants for ext4's extended time encoding
*/
diff --git a/lib/ext2fs/ext2fs.h b/lib/ext2fs/ext2fs.h
index df150f0003f2..886e12793620 100644
--- a/lib/ext2fs/ext2fs.h
+++ b/lib/ext2fs/ext2fs.h
@@ -620,7 +620,8 @@ typedef struct ext2_icount *ext2_icount_t;
EXT2_FEATURE_COMPAT_EXT_ATTR|\
EXT4_FEATURE_COMPAT_SPARSE_SUPER2|\
EXT4_FEATURE_COMPAT_FAST_COMMIT|\
- EXT4_FEATURE_COMPAT_STABLE_INODES)
+ EXT4_FEATURE_COMPAT_STABLE_INODES|\
+ EXT4_FEATURE_COMPAT_ORPHAN_FILE)
#ifdef CONFIG_MMP
#define EXT4_LIB_INCOMPAT_MMP EXT4_FEATURE_INCOMPAT_MMP
@@ -655,7 +656,8 @@ typedef struct ext2_icount *ext2_icount_t;
EXT4_FEATURE_RO_COMPAT_READONLY |\
EXT4_FEATURE_RO_COMPAT_PROJECT |\
EXT4_FEATURE_RO_COMPAT_SHARED_BLOCKS |\
- EXT4_FEATURE_RO_COMPAT_VERITY)
+ EXT4_FEATURE_RO_COMPAT_VERITY |\
+ EXT4_FEATURE_RO_COMPAT_ORPHAN_PRESENT)
/*
* These features are only allowed if EXT2_FLAG_SOFTSUPP_FEATURES is passed
@@ -1687,6 +1689,15 @@ errcode_t ext2fs_get_data_io(ext2_filsys fs, io_channel *old_io);
errcode_t ext2fs_set_data_io(ext2_filsys fs, io_channel new_io);
errcode_t ext2fs_rewrite_to_io(ext2_filsys fs, io_channel new_io);
+/* orphan.c */
+extern errcode_t ext2fs_create_orphan_file(ext2_filsys fs, blk_t num_blocks);
+extern errcode_t ext2fs_truncate_orphan_file(ext2_filsys fs);
+extern e2_blkcnt_t ext2fs_default_orphan_file_blocks(ext2_filsys fs);
+extern errcode_t ext2fs_orphan_file_block_csum_set(ext2_filsys fs,
+ ext2_ino_t ino, char *buf);
+extern int ext2fs_orphan_file_block_csum_verify(ext2_filsys fs, ext2_ino_t ino,
+ char *buf);
+
/* get_pathname.c */
extern errcode_t ext2fs_get_pathname(ext2_filsys fs, ext2_ino_t dir, ext2_ino_t ino,
char **name);
@@ -1840,7 +1851,9 @@ extern int ext2fs_dirent_file_type(const struct ext2_dir_entry *entry);
extern void ext2fs_dirent_set_file_type(struct ext2_dir_entry *entry, int type);
extern struct ext2_inode *ext2fs_inode(struct ext2_inode_large * large_inode);
extern const struct ext2_inode *ext2fs_const_inode(const struct ext2_inode_large * large_inode);
-
+extern int ext2fs_inodes_per_orphan_block(ext2_filsys fs);
+extern struct ext4_orphan_block_tail *ext2fs_orphan_block_tail(ext2_filsys fs,
+ char *buf);
#endif
/*
@@ -2150,6 +2163,19 @@ ext2fs_const_inode(const struct ext2_inode_large * large_inode)
return (const struct ext2_inode *) large_inode;
}
+_INLINE_ int ext2fs_inodes_per_orphan_block(ext2_filsys fs)
+{
+ return (fs->blocksize - sizeof(struct ext4_orphan_block_tail)) /
+ sizeof(__u32);
+}
+
+_INLINE_ struct ext4_orphan_block_tail *
+ext2fs_orphan_block_tail(ext2_filsys fs, char *buf)
+{
+ return (struct ext4_orphan_block_tail *)(buf + fs->blocksize -
+ sizeof(struct ext4_orphan_block_tail));
+}
+
#undef _INLINE_
#endif
diff --git a/lib/ext2fs/orphan.c b/lib/ext2fs/orphan.c
new file mode 100644
index 000000000000..71e8674ce601
--- /dev/null
+++ b/lib/ext2fs/orphan.c
@@ -0,0 +1,252 @@
+/*
+ * orphan.c --- utility function to handle orphan file
+ *
+ * Copyright (C) 2015 Jan Kara.
+ *
+ * %Begin-Header%
+ * This file may be redistributed under the terms of the GNU Library
+ * General Public License, version 2.
+ * %End-Header%
+ */
+
+#include "config.h"
+#include <string.h>
+
+#include "ext2_fs.h"
+#include "ext2fsP.h"
+
+errcode_t ext2fs_truncate_orphan_file(ext2_filsys fs)
+{
+ struct ext2_inode inode;
+ errcode_t err;
+ ext2_ino_t ino = fs->super->s_orphan_file_inum;
+
+ err = ext2fs_read_inode(fs, ino, &inode);
+ if (err)
+ return err;
+
+ err = ext2fs_punch(fs, ino, &inode, NULL, 0, ~0ULL);
+ if (err)
+ return err;
+
+ fs->flags &= ~EXT2_FLAG_SUPER_ONLY;
+ memset(&inode, 0, sizeof(struct ext2_inode));
+ err = ext2fs_write_inode(fs, ino, &inode);
+
+ ext2fs_clear_feature_orphan_file(fs->super);
+ ext2fs_clear_feature_orphan_present(fs->super);
+ ext2fs_mark_super_dirty(fs);
+ /* Need to update group descriptors as well */
+ fs->flags &= ~EXT2_FLAG_SUPER_ONLY;
+
+ return err;
+}
+
+struct mkorphan_info {
+ char *buf;
+ char *zerobuf;
+ blk_t num_blocks;
+ blk_t alloc_blocks;
+ blk64_t last_blk;
+ errcode_t err;
+};
+
+static int mkorphan_proc(ext2_filsys fs,
+ blk64_t *blocknr,
+ e2_blkcnt_t blockcnt,
+ blk64_t ref_block EXT2FS_ATTR((unused)),
+ int ref_offset EXT2FS_ATTR((unused)),
+ void *priv_data)
+{
+ struct mkorphan_info *oi = (struct mkorphan_info *)priv_data;
+ blk64_t new_blk;
+ errcode_t err;
+
+ /* Can we just continue in currently allocated cluster? */
+ if (blockcnt &&
+ EXT2FS_B2C(fs, oi->last_blk) == EXT2FS_B2C(fs, oi->last_blk + 1)) {
+ new_blk = oi->last_blk + 1;
+ } else {
+ err = ext2fs_new_block2(fs, oi->last_blk, 0, &new_blk);
+ if (err) {
+ oi->err = err;
+ return BLOCK_ABORT;
+ }
+ ext2fs_block_alloc_stats2(fs, new_blk, +1);
+ oi->alloc_blocks++;
+ }
+ if (blockcnt >= 0)
+ err = io_channel_write_blk64(fs->io, new_blk, 1, oi->buf);
+ else /* zerobuf is used to initialize new indirect blocks... */
+ err = io_channel_write_blk64(fs->io, new_blk, 1, oi->zerobuf);
+ if (err) {
+ oi->err = err;
+ return BLOCK_ABORT;
+ }
+ oi->last_blk = new_blk;
+ *blocknr = new_blk;
+ if (blockcnt >= 0 && --oi->num_blocks == 0)
+ return BLOCK_CHANGED | BLOCK_ABORT;
+ return BLOCK_CHANGED;
+}
+
+errcode_t ext2fs_create_orphan_file(ext2_filsys fs, blk_t num_blocks)
+{
+ struct ext2_inode inode;
+ ext2_ino_t ino = fs->super->s_orphan_file_inum;
+ errcode_t err;
+ char *buf = NULL, *zerobuf = NULL;
+ struct mkorphan_info oi;
+ struct ext4_orphan_block_tail *ob_tail;
+
+ if (!ino) {
+ err = ext2fs_new_inode(fs, EXT2_ROOT_INO, LINUX_S_IFREG | 0600,
+ 0, &ino);
+ if (err)
+ return err;
+ ext2fs_inode_alloc_stats2(fs, ino, +1, 0);
+ ext2fs_mark_ib_dirty(fs);
+ }
+
+ err = ext2fs_read_inode(fs, ino, &inode);
+ if (err)
+ return err;
+ if (EXT2_I_SIZE(&inode)) {
+ err = ext2fs_truncate_orphan_file(fs);
+ if (err)
+ return err;
+ }
+
+ memset(&inode, 0, sizeof(struct ext2_inode));
+ if (ext2fs_has_feature_extents(fs->super)) {
+ inode.i_flags |= EXT4_EXTENTS_FL;
+ err = ext2fs_write_inode(fs, ino, &inode);
+ if (err)
+ return err;
+ }
+
+ err = ext2fs_get_mem(fs->blocksize, &buf);
+ if (err)
+ return err;
+ err = ext2fs_get_mem(fs->blocksize, &zerobuf);
+ if (err)
+ goto out;
+ memset(buf, 0, fs->blocksize);
+ memset(zerobuf, 0, fs->blocksize);
+ ob_tail = ext2fs_orphan_block_tail(fs, buf);
+ ob_tail->ob_magic = ext2fs_cpu_to_le32(EXT4_ORPHAN_BLOCK_MAGIC);
+ err = ext2fs_orphan_file_block_csum_set(fs, ino, buf);
+ if (err)
+ goto out;
+ oi.num_blocks = num_blocks;
+ oi.alloc_blocks = 0;
+ oi.last_blk = 0;
+ oi.buf = buf;
+ oi.zerobuf = zerobuf;
+ oi.err = 0;
+ err = ext2fs_block_iterate3(fs, ino, BLOCK_FLAG_APPEND,
+ 0, mkorphan_proc, &oi);
+ if (err)
+ goto out;
+ if (oi.err) {
+ err = oi.err;
+ goto out;
+ }
+
+ /* Reread inode after blocks were allocated */
+ err = ext2fs_read_inode(fs, ino, &inode);
+ if (err)
+ goto out;
+ ext2fs_iblk_set(fs, &inode, 0);
+ inode.i_atime = inode.i_mtime =
+ inode.i_ctime = fs->now ? fs->now : time(0);
+ inode.i_links_count = 1;
+ inode.i_mode = LINUX_S_IFREG | 0600;
+ ext2fs_iblk_add_blocks(fs, &inode, oi.alloc_blocks);
+ err = ext2fs_inode_size_set(fs, &inode,
+ (unsigned long long)fs->blocksize * num_blocks);
+ if (err)
+ goto out;
+ err = ext2fs_write_new_inode(fs, ino, &inode);
+ if (err)
+ goto out;
+
+ fs->super->s_orphan_file_inum = ino;
+ ext2fs_set_feature_orphan_file(fs->super);
+ ext2fs_mark_super_dirty(fs);
+ /* Need to update group descriptors as well */
+ fs->flags &= ~EXT2_FLAG_SUPER_ONLY;
+out:
+ if (buf)
+ ext2fs_free_mem(&buf);
+ if (zerobuf)
+ ext2fs_free_mem(&zerobuf);
+ return err;
+}
+
+/*
+ * Find reasonable size for orphan file. We choose orphan file size to be
+ * between 32 and 512 filesystem blocks and not more than 1/4096 of the
+ * filesystem unless it is really small.
+ */
+e2_blkcnt_t ext2fs_default_orphan_file_blocks(ext2_filsys fs)
+{
+ __u64 num_blocks = ext2fs_blocks_count(fs->super);
+ e2_blkcnt_t blks = 512;
+
+ if (num_blocks < 128 * 1024)
+ blks = 32;
+ else if (num_blocks < 2 * 1024 * 1024)
+ blks = num_blocks / 4096;
+ return (blks + EXT2FS_CLUSTER_MASK(fs)) & ~EXT2FS_CLUSTER_MASK(fs);
+}
+
+static errcode_t ext2fs_orphan_file_block_csum(ext2_filsys fs, ext2_ino_t ino,
+ char *buf, __u32 *crcp)
+{
+ int inodes_per_ob = ext2fs_inodes_per_orphan_block(fs);
+ __u32 gen, crc;
+ struct ext2_inode inode;
+ errcode_t retval;
+
+ retval = ext2fs_read_inode(fs, ino, &inode);
+ if (retval)
+ return retval;
+ ino = ext2fs_cpu_to_le32(ino);
+ gen = ext2fs_cpu_to_le32(inode.i_generation);
+ crc = ext2fs_crc32c_le(fs->csum_seed, (unsigned char *)&ino,
+ sizeof(ino));
+ crc = ext2fs_crc32c_le(crc, (unsigned char *)&gen, sizeof(gen));
+ crc = ext2fs_crc32c_le(crc, buf, inodes_per_ob * sizeof(__u32));
+ *crcp = ext2fs_cpu_to_le32(crc);
+
+ return 0;
+}
+
+errcode_t ext2fs_orphan_file_block_csum_set(ext2_filsys fs, ext2_ino_t ino,
+ char *buf)
+{
+ struct ext4_orphan_block_tail *tail;
+
+ if (!ext2fs_has_feature_metadata_csum(fs->super))
+ return 0;
+
+ tail = ext2fs_orphan_block_tail(fs, buf);
+ return ext2fs_orphan_file_block_csum(fs, ino, buf, &tail->ob_checksum);
+}
+
+int ext2fs_orphan_file_block_csum_verify(ext2_filsys fs, ext2_ino_t ino,
+ char *buf)
+{
+ struct ext4_orphan_block_tail *tail;
+ __u32 crc;
+ errcode_t retval;
+
+ if (!ext2fs_has_feature_metadata_csum(fs->super))
+ return 1;
+ retval = ext2fs_orphan_file_block_csum(fs, ino, buf, &crc);
+ if (retval)
+ return 0;
+ tail = ext2fs_orphan_block_tail(fs, buf);
+ return ext2fs_le32_to_cpu(tail->ob_checksum) == crc;
+}
diff --git a/lib/ext2fs/swapfs.c b/lib/ext2fs/swapfs.c
index 1006b2d2bd52..b844e7665999 100644
--- a/lib/ext2fs/swapfs.c
+++ b/lib/ext2fs/swapfs.c
@@ -131,8 +131,9 @@ void ext2fs_swap_super(struct ext2_super_block * sb)
/* s_*_time_hi are __u8 and does not need swabbing */
sb->s_encoding = ext2fs_swab16(sb->s_encoding);
sb->s_encoding_flags = ext2fs_swab16(sb->s_encoding_flags);
+ sb->s_orphan_file_inum = ext2fs_swab32(sb->s_orphan_file_inum);
/* catch when new fields are used from s_reserved */
- EXT2FS_BUILD_BUG_ON(sizeof(sb->s_reserved) != 95 * sizeof(__le32));
+ EXT2FS_BUILD_BUG_ON(sizeof(sb->s_reserved) != 94 * sizeof(__le32));
sb->s_checksum = ext2fs_swab32(sb->s_checksum);
}
diff --git a/lib/ext2fs/tst_super_size.c b/lib/ext2fs/tst_super_size.c
index 80a5269bceb7..ad452dee8eb3 100644
--- a/lib/ext2fs/tst_super_size.c
+++ b/lib/ext2fs/tst_super_size.c
@@ -152,7 +152,8 @@ int main(int argc, char **argv)
check_field(s_last_error_errcode, 1);
check_field(s_encoding, 2);
check_field(s_encoding_flags, 2);
- check_field(s_reserved, 95 * 4);
+ check_field(s_orphan_file_inum, 4);
+ check_field(s_reserved, 94 * 4);
check_field(s_checksum, 4);
do_field("Superblock end", 0, 0, cur_offset, 1024);
#endif
diff --git a/lib/support/mkquota.c b/lib/support/mkquota.c
index 21a5c34d6921..b790b93f52c2 100644
--- a/lib/support/mkquota.c
+++ b/lib/support/mkquota.c
@@ -504,7 +504,8 @@ errcode_t quota_compute_usage(quota_ctx_t qctx)
continue;
if (ino == EXT2_ROOT_INO ||
(ino >= EXT2_FIRST_INODE(fs->super) &&
- ino != quota_type2inum(PRJQUOTA, fs->super))) {
+ ino != quota_type2inum(PRJQUOTA, fs->super) &&
+ ino != fs->super->s_orphan_file_inum)) {
space = ext2fs_get_stat_i_blocks(fs,
EXT2_INODE(inode)) << 9;
quota_data_add(qctx, inode, ino, space);
--
2.26.2
^ permalink raw reply related [flat|nested] 22+ messages in thread
end of thread, other threads:[~2021-08-05 15:00 UTC | newest]
Thread overview: 22+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-07-12 15:43 [PATCH 0/9 v4] e2fsprogs: Support for orphan file feature Jan Kara
2021-07-12 15:43 ` [PATCH 1/9] ext2fs: Drop HAS_SNAPSHOT feature Jan Kara
2021-07-12 18:20 ` Andreas Dilger
2021-07-13 8:11 ` Jan Kara
2021-07-12 15:43 ` [PATCH 2/9] quota: Do not account space used by project quota file to quota Jan Kara
2021-07-12 15:43 ` [PATCH 3/9] e2image: Dump quota files Jan Kara
2021-07-12 15:43 ` [PATCH 4/9] libext2fs: Support for orphan file feature Jan Kara
2021-08-03 16:57 ` Theodore Ts'o
2021-08-03 17:40 ` Theodore Ts'o
2021-08-04 9:25 ` Jan Kara
2021-08-04 10:13 ` Jan Kara
2021-07-12 15:43 ` [PATCH 5/9] mke2fs: Add support for orphan_file feature Jan Kara
2021-07-12 15:43 ` [PATCH 6/9] e2fsck: Add support for handling orphan file Jan Kara
2021-08-03 18:56 ` Theodore Ts'o
2021-07-12 15:43 ` [PATCH 7/9] tune2fs: Add support for orphan_file feature Jan Kara
2021-07-12 15:43 ` [PATCH 8/9] mke2fs: Add orphan_file feature into mke2fs.conf Jan Kara
2021-08-04 19:04 ` Theodore Ts'o
2021-08-05 15:00 ` Jan Kara
2021-07-12 15:43 ` [PATCH 9/9] dumpe2fs, debugfs, e2image: Add support for orphan file Jan Kara
-- strict thread matches above, loose matches on Subject: below --
2021-06-16 10:57 [PATCH 0/9 v3] e2fsprogs: Support for orphan file feature Jan Kara
2021-06-16 10:57 ` [PATCH 4/9] libext2fs: " Jan Kara
2021-08-03 17:36 ` Theodore Ts'o
2021-08-03 17:57 ` Theodore Ts'o
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.