All of lore.kernel.org
 help / color / mirror / Atom feed
* [RFC PATCH 0/5] f2fs-tools: introduce lost+found feature
@ 2018-02-06  4:31 Sheng Yong
  2018-02-06  4:31 ` [RFC PATCH 1/5] mkfs.f2fs: introduce mkfs parameters in f2fs_configuration Sheng Yong
                   ` (4 more replies)
  0 siblings, 5 replies; 18+ messages in thread
From: Sheng Yong @ 2018-02-06  4:31 UTC (permalink / raw)
  To: jaegeuk, yuchao0; +Cc: heyunlei, miaoxie, shengyong1, linux-f2fs-devel

Commit 390fe587b1 ("fsck.f2fs: support restore lost files into
./lost_found/") restores unreachable files, which have no parent directory
or their parent directories are removed by fsck, to ./lost_found directory.

However, it has several limitations:
  1. ./lost_found dir does not locate inside f2fs, thus it may needs
     another partition to save ./lost_found.
  2. encrypted files are not supported to be restored.
  3. some xattrs may be lost, it depends on whether dump_node() enables
     dumping such xattrs.
  4. not support restoring files on Android devices (the same reason as
     #1).

In order to address the above limitations, the lost+found feature is
introduced. This is an ext4-style lost+found. It tries to restore
unreachable files into /lost+found directory inside f2fs.

If lost+found feature is switched on when mkfs, /lost+found directory is
created by mkfs. It can also be created by mkdir manually. It should be
located right under root directory.

If fsck detects unreachable nids, it scans all these nids, and skips non-
inode or directory inode nodes. For the left nids, they are all file inodes.
Fsck then checks these inodes and update corresponding counters and bitmaps.

From now on, the filesystem is in a "consistent" state, fsck can allocate
blocks safely and start reconnecting files to lost+found. The reconnection
adds new dentry in lost+found and update metadata (i_name, i_pino) of file
inode. If reconnection fails, fsck will clear the counters and bitmaps.
Files reconnected to lost+found will be renamed after its ino number.

However, because of lost+found directory, f2fs should avoid to encrypting
root directory. So f2fs should also check if lost+found feature is enabled
or not.

I create a testcase which injects faults to generate unreachable nids.

$ mkfs.f2fs -O lost+found -O encrypt test.img
$ tree /data
/data
├── dir
│   ├── foo
│   └── subdir
│       └── bar
├── encrypt-dir
│   ├── foo
│   └── subdir
│       └── bar
└── lost+found

Inject nat_entry->blk_addr of dir/encrypt-dir to 0.

$ fsck.f2fs -f test.img
Info: Force to fix corruption
Info: Segments per section = 1
Info: Sections per zone = 1
Info: sector size = 512
Info: total sectors = 204800 (100 MB)
Info: MKFS version
  "Linux version 4.10.0-42-generic (buildd@lgw01-amd64-012) (gcc version 6.3.0 20170406 (Ubuntu 6.3.0-12ubuntu2) ) #46-Ubuntu SMP Mon Dec 4 14:38:01 UTC 2017"
Info: FSCK version
  from "Linux version 4.10.0-42-generic (buildd@lgw01-amd64-012) (gcc version 6.3.0 20170406 (Ubuntu 6.3.0-12ubuntu2) ) #46-Ubuntu SMP Mon Dec 4 14:38:01 UTC 2017"
    to "Linux version 4.10.0-42-generic (buildd@lgw01-amd64-012) (gcc version 6.3.0 20170406 (Ubuntu 6.3.0-12ubuntu2) ) #46-Ubuntu SMP Mon Dec 4 14:38:01 UTC 2017"
Info: superblock features = 201 :  encrypt lost+found
Info: superblock encrypt level = 0, salt = 00000000000000000000000000000000
Info: total FS sectors = 204800 (100 MB)
Info: CKPT version = 2995f9d3
Info: Found valid nat_bits in checkpoint
Info: checkpoint state = 1c5 :  trimmed nat_bits crc compacted_summary unmount
[ASSERT] (sanity_check_nid: 386)  --> nid[0x5] ino is 0
[FIX] (__chk_dentries:1456)  --> Unlink [0x5] - dir len[0x3], type[0x2]
[ASSERT] (sanity_check_nid: 386)  --> nid[0xd] ino is 0
[FIX] (__chk_dentries:1456)  --> Unlink [0xd] - encrypt-dir len[0xb], type[0x2]
[ASSERT] (fsck_chk_inode_blk: 895)  --> ino: 0x3 i_links: 5, real links: 3
[FIX] (fsck_chk_inode_blk: 900)  --> Dir: 0x3 i_links= 0x5 -> 0x3

NID[0x6] is unreachable
NID[0x7] is unreachable
NID[0x8] is unreachable
NID[0x9] is unreachable
NID[0xa] is unreachable
NID[0xb] is unreachable
NID[0xc] is unreachable
NID[0xe] is unreachable
NID[0xf] is unreachable
NID[0x10] is unreachable

[FSCK] Reconnect 4 files to lost+found
[FSCK] Unreachable nat entries                        [Fail] [0xa]
[FSCK] SIT valid block bitmap checking                [Fail]
[FSCK] Hard link checking for regular file            [Ok..] [0x0]
[FSCK] valid_block_count matching with CP             [Fail] [0xc56]
[FSCK] valid_node_count matcing with CP (de lookup)   [Fail] [0xa]
[FSCK] valid_node_count matcing with CP (nat lookup)  [Fail] [0xc]
[FSCK] valid_inode_count matched with CP              [Fail] [0x6]
[FSCK] free segment_count matched with CP             [Ok..] [0x1e]
[FSCK] next block offset is free                      [Ok..]
[FSCK] fixing SIT types
[FSCK] other corrupted bugs                           [Fail]
[FIX] (nullify_nat_entry:2064)  --> Remove nid [0xb] in NAT
[FIX] (nullify_nat_entry:2064)  --> Remove nid [0xf] in NAT
Info: Write valid nat_bits in checkpoint

Done.

$ tree /data
/data/
└── lost+found
    ├── 12
    ├── 14
    ├── 16
    └── 6

Any comments and tests are appreciated.

Thanks,
Sheng

Sheng Yong (5):
  mkfs.f2fs: introduce mkfs parameters in f2fs_configuration
  f2fs-tools: init f2fs_configuration wit as 0
  fsck.f2fs: integrate sanity_check_inode to __check_inode_mode
  mkfs.f2fs: create lost+found directory
  fsck.f2fs: reconnect unreachable files to lost+found

 fsck/dir.c              |  19 ++-
 fsck/fsck.c             | 402 +++++++++++++++++++++++++++++++++++++++++++++---
 fsck/fsck.h             |   3 +
 fsck/mount.c            |   5 +
 include/f2fs_fs.h       |  11 ++
 lib/libf2fs.c           |  18 +--
 mkfs/f2fs_format.c      | 251 ++++++++++++++++++++++++++----
 mkfs/f2fs_format_main.c |   2 +
 8 files changed, 639 insertions(+), 72 deletions(-)

-- 
2.11.0


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

^ permalink raw reply	[flat|nested] 18+ messages in thread

* [RFC PATCH 1/5] mkfs.f2fs: introduce mkfs parameters in f2fs_configuration
  2018-02-06  4:31 [RFC PATCH 0/5] f2fs-tools: introduce lost+found feature Sheng Yong
@ 2018-02-06  4:31 ` Sheng Yong
  2018-02-08 13:30   ` Chao Yu
  2018-02-06  4:31 ` [RFC PATCH 2/5] f2fs-tools: init f2fs_configuration as 0 Sheng Yong
                   ` (3 subsequent siblings)
  4 siblings, 1 reply; 18+ messages in thread
From: Sheng Yong @ 2018-02-06  4:31 UTC (permalink / raw)
  To: jaegeuk, yuchao0; +Cc: heyunlei, miaoxie, shengyong1, linux-f2fs-devel

Introduce new parameters in f2fs_configuration for mkfs:
  * next_free_nid: save the next free nid
  * quota_inum: save how many blocks are used for quota inodes
  * quota_dnum: save how many blocks are used for quota data

Use these parameters to avoid duplicated calculation of these values.

Signed-off-by: Sheng Yong <shengyong1@huawei.com>
---
 include/f2fs_fs.h  |  5 +++++
 mkfs/f2fs_format.c | 46 +++++++++++++++++-----------------------------
 2 files changed, 22 insertions(+), 29 deletions(-)

diff --git a/include/f2fs_fs.h b/include/f2fs_fs.h
index 548a3e8..ca4522d 100644
--- a/include/f2fs_fs.h
+++ b/include/f2fs_fs.h
@@ -365,6 +365,11 @@ struct f2fs_configuration {
 	int large_nat_bitmap;
 	__le32 feature;			/* defined features */
 
+	/* mkfs parameters */
+	u_int32_t next_free_nid;
+	u_int32_t quota_inum;
+	u_int32_t quota_dnum;
+
 	/* defragmentation parameters */
 	int defrag_shrink;
 	u_int64_t defrag_start;
diff --git a/mkfs/f2fs_format.c b/mkfs/f2fs_format.c
index 4fb429e..bda3c9d 100644
--- a/mkfs/f2fs_format.c
+++ b/mkfs/f2fs_format.c
@@ -159,7 +159,6 @@ static int f2fs_prepare_super_block(void)
 	u_int32_t sit_bitmap_size, max_sit_bitmap_size;
 	u_int32_t max_nat_bitmap_size, max_nat_segments;
 	u_int32_t total_zones;
-	u_int32_t next_ino;
 	enum quota_type qtype;
 	int i;
 
@@ -411,7 +410,7 @@ static int f2fs_prepare_super_block(void)
 	set_sb(node_ino, 1);
 	set_sb(meta_ino, 2);
 	set_sb(root_ino, 3);
-	next_ino = 4;
+	c.next_free_nid = 4;
 
 	if (c.feature & cpu_to_le32(F2FS_FEATURE_QUOTA_INO)) {
 		quotatype_bits = QUOTA_USR_BIT | QUOTA_GRP_BIT;
@@ -422,9 +421,9 @@ static int f2fs_prepare_super_block(void)
 	for (qtype = 0; qtype < F2FS_MAX_QUOTAS; qtype++) {
 		if (!((1 << qtype) & quotatype_bits))
 			continue;
-		sb->qf_ino[qtype] = cpu_to_le32(next_ino++);
+		sb->qf_ino[qtype] = cpu_to_le32(c.next_free_nid++);
 		MSG(0, "Info: add quota type = %u => %u\n",
-					qtype, next_ino - 1);
+					qtype, c.next_free_nid - 1);
 	}
 
 	if (total_zones <= 6) {
@@ -558,7 +557,6 @@ static int f2fs_write_check_point_pack(void)
 	char *sum_compact, *sum_compact_p;
 	struct f2fs_summary *sum_entry;
 	enum quota_type qtype;
-	u_int32_t quota_inum, quota_dnum;
 	int off;
 	int ret = -1;
 
@@ -610,16 +608,9 @@ static int f2fs_write_check_point_pack(void)
 		set_cp(cur_data_segno[i], 0xffffffff);
 	}
 
-	quota_inum = quota_dnum = 0;
-	for (qtype = 0; qtype < F2FS_MAX_QUOTAS; qtype++)
-		if (sb->qf_ino[qtype]) {
-			quota_inum++;
-			quota_dnum += QUOTA_DATA(qtype);
-		}
-
-	set_cp(cur_node_blkoff[0], 1 + quota_inum);
-	set_cp(cur_data_blkoff[0], 1 + quota_dnum);
-	set_cp(valid_block_count, 2 + quota_inum + quota_dnum);
+	set_cp(cur_node_blkoff[0], 1 + c.quota_inum);
+	set_cp(cur_data_blkoff[0], 1 + c.quota_dnum);
+	set_cp(valid_block_count, 2 + c.quota_inum + c.quota_dnum);
 	set_cp(rsvd_segment_count, c.reserved_segments);
 	set_cp(overprov_segment_count, (get_sb(segment_count_main) -
 			get_cp(rsvd_segment_count)) *
@@ -651,9 +642,9 @@ static int f2fs_write_check_point_pack(void)
 
 	set_cp(ckpt_flags, flags);
 	set_cp(cp_pack_start_sum, 1 + get_sb(cp_payload));
-	set_cp(valid_node_count, 1 + quota_inum);
-	set_cp(valid_inode_count, 1 + quota_inum);
-	set_cp(next_free_nid, get_sb(root_ino) + 1 + quota_inum);
+	set_cp(valid_node_count, 1 + c.quota_inum);
+	set_cp(valid_inode_count, 1 + c.quota_inum);
+	set_cp(next_free_nid, c.next_free_nid);
 	set_cp(sit_ver_bitmap_bytesize, ((get_sb(segment_count_sit) / 2) <<
 			get_sb(log_blocks_per_seg)) / 8);
 
@@ -711,7 +702,7 @@ static int f2fs_write_check_point_pack(void)
 	SET_SUM_TYPE((&sum->footer), SUM_TYPE_DATA);
 
 	journal = &sum->journal;
-	journal->n_nats = cpu_to_le16(1 + quota_inum);
+	journal->n_nats = cpu_to_le16(1 + c.quota_inum);
 	journal->nat_j.entries[0].nid = sb->root_ino;
 	journal->nat_j.entries[0].ne.version = 0;
 	journal->nat_j.entries[0].ne.ino = sb->root_ino;
@@ -741,9 +732,9 @@ static int f2fs_write_check_point_pack(void)
 	journal->sit_j.entries[0].segno = cp->cur_node_segno[0];
 	journal->sit_j.entries[0].se.vblocks =
 				cpu_to_le16((CURSEG_HOT_NODE << 10) |
-						(1 + quota_inum));
+						(1 + c.quota_inum));
 	f2fs_set_bit(0, (char *)journal->sit_j.entries[0].se.valid_map);
-	for (i = 1; i <= quota_inum; i++)
+	for (i = 1; i <= c.quota_inum; i++)
 		f2fs_set_bit(i, (char *)journal->sit_j.entries[0].se.valid_map);
 	journal->sit_j.entries[1].segno = cp->cur_node_segno[1];
 	journal->sit_j.entries[1].se.vblocks =
@@ -756,9 +747,9 @@ static int f2fs_write_check_point_pack(void)
 	journal->sit_j.entries[3].segno = cp->cur_data_segno[0];
 	journal->sit_j.entries[3].se.vblocks =
 				cpu_to_le16((CURSEG_HOT_DATA << 10) |
-						(1 + quota_dnum));
+						(1 + c.quota_dnum));
 	f2fs_set_bit(0, (char *)journal->sit_j.entries[3].se.valid_map);
-	for (i = 1; i <= quota_dnum; i++)
+	for (i = 1; i <= c.quota_dnum; i++)
 		f2fs_set_bit(i, (char *)journal->sit_j.entries[3].se.valid_map);
 
 	journal->sit_j.entries[4].segno = cp->cur_data_segno[1];
@@ -958,15 +949,10 @@ static int discard_obsolete_dnode(struct f2fs_node *raw_node, u_int64_t offset)
 			get_sb(log_blocks_per_seg)) + get_sb(main_blkaddr);
 	u_int64_t start_inode_pos = get_sb(main_blkaddr);
 	u_int64_t last_inode_pos;
-	enum quota_type qtype;
-	u_int32_t quota_inum = 0;
-
-	for (qtype = 0; qtype < F2FS_MAX_QUOTAS; qtype++)
-		if (sb->qf_ino[qtype]) quota_inum++;
 
 	/* only root inode was written before truncating dnodes */
 	last_inode_pos = start_inode_pos +
-		c.cur_seg[CURSEG_HOT_NODE] * c.blks_per_seg + quota_inum;
+		c.cur_seg[CURSEG_HOT_NODE] * c.blks_per_seg + c.quota_inum;
 
 	if (c.zoned_mode)
 		return 0;
@@ -1154,6 +1140,7 @@ static int f2fs_write_default_quota(int qtype, unsigned int blkaddr,
 	DBG(1, "\tWriting quota data, at offset %08x, %08x\n",
 					blkaddr, blkaddr + 1);
 	free(filebuf);
+	c.quota_dnum += QUOTA_DATA(qtype);
 	return 0;
 }
 
@@ -1255,6 +1242,7 @@ static int f2fs_write_qf_inode(int qtype)
 	}
 
 	free(raw_node);
+	c.quota_inum++;
 	return 0;
 }
 
-- 
2.11.0


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot

^ permalink raw reply related	[flat|nested] 18+ messages in thread

* [RFC PATCH 2/5] f2fs-tools: init f2fs_configuration as 0
  2018-02-06  4:31 [RFC PATCH 0/5] f2fs-tools: introduce lost+found feature Sheng Yong
  2018-02-06  4:31 ` [RFC PATCH 1/5] mkfs.f2fs: introduce mkfs parameters in f2fs_configuration Sheng Yong
@ 2018-02-06  4:31 ` Sheng Yong
  2018-02-08 13:32   ` Chao Yu
  2018-02-06  4:31 ` [RFC PATCH 3/5] fsck.f2fs: integrate sanity_check_inode to __check_inode_mode Sheng Yong
                   ` (2 subsequent siblings)
  4 siblings, 1 reply; 18+ messages in thread
From: Sheng Yong @ 2018-02-06  4:31 UTC (permalink / raw)
  To: jaegeuk, yuchao0; +Cc: heyunlei, miaoxie, shengyong1, linux-f2fs-devel

Signed-off-by: Sheng Yong <shengyong1@huawei.com>
---
 lib/libf2fs.c | 18 ++----------------
 1 file changed, 2 insertions(+), 16 deletions(-)

diff --git a/lib/libf2fs.c b/lib/libf2fs.c
index e8b1842..0c684d5 100644
--- a/lib/libf2fs.c
+++ b/lib/libf2fs.c
@@ -586,24 +586,17 @@ void f2fs_init_configuration(void)
 {
 	int i;
 
+	memset(&c, 0, sizeof(struct f2fs_configuration));
 	c.ndevs = 1;
-	c.total_sectors = 0;
-	c.sector_size = 0;
 	c.sectors_per_blk = DEFAULT_SECTORS_PER_BLOCK;
 	c.blks_per_seg = DEFAULT_BLOCKS_PER_SEGMENT;
 	c.rootdev_name = get_rootdev();
 	c.wanted_total_sectors = -1;
-	c.zoned_mode = 0;
-	c.zoned_model = 0;
-	c.zone_blocks = 0;
-#ifdef WITH_ANDROID
-	c.preserve_limits = 0;
-#else
+#ifndef WITH_ANDROID
 	c.preserve_limits = 1;
 #endif
 
 	for (i = 0; i < MAX_DEVICES; i++) {
-		memset(&c.devices[i], 0, sizeof(struct device_info));
 		c.devices[i].fd = -1;
 		c.devices[i].sector_size = DEFAULT_SECTOR_SIZE;
 		c.devices[i].end_blkaddr = -1;
@@ -611,19 +604,12 @@ void f2fs_init_configuration(void)
 	}
 
 	/* calculated by overprovision ratio */
-	c.reserved_segments = 0;
-	c.overprovision = 0;
 	c.segs_per_sec = 1;
 	c.secs_per_zone = 1;
 	c.segs_per_zone = 1;
-	c.heap = 0;
 	c.vol_label = "";
 	c.trim = 1;
-	c.trimmed = 0;
-	c.ro = 0;
 	c.kd = -1;
-	c.dry_run = 0;
-	c.large_nat_bitmap = 0;
 	c.fixed_time = -1;
 }
 
-- 
2.11.0


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot

^ permalink raw reply related	[flat|nested] 18+ messages in thread

* [RFC PATCH 3/5] fsck.f2fs: integrate sanity_check_inode to __check_inode_mode
  2018-02-06  4:31 [RFC PATCH 0/5] f2fs-tools: introduce lost+found feature Sheng Yong
  2018-02-06  4:31 ` [RFC PATCH 1/5] mkfs.f2fs: introduce mkfs parameters in f2fs_configuration Sheng Yong
  2018-02-06  4:31 ` [RFC PATCH 2/5] f2fs-tools: init f2fs_configuration as 0 Sheng Yong
@ 2018-02-06  4:31 ` Sheng Yong
  2018-02-08 13:44   ` Chao Yu
  2018-02-06  4:31 ` [RFC PATCH 4/5] mkfs.f2fs: create lost+found directory Sheng Yong
  2018-02-06  4:31 ` [RFC PATCH 5/5] fsck.f2fs: reconnect unreachable files to lost+found Sheng Yong
  4 siblings, 1 reply; 18+ messages in thread
From: Sheng Yong @ 2018-02-06  4:31 UTC (permalink / raw)
  To: jaegeuk, yuchao0; +Cc: heyunlei, miaoxie, shengyong1, linux-f2fs-devel

In sanity_check_nid, __check_inode_mode will check i_mode value of an
inode. So integrate sanity_check_inode to __check_inode_mode to clean
up the code.

Signed-off-by: Sheng Yong <shengyong1@huawei.com>
---
 fsck/fsck.c | 26 +++++---------------------
 1 file changed, 5 insertions(+), 21 deletions(-)

diff --git a/fsck/fsck.c b/fsck/fsck.c
index 29823a1..fcaab14 100644
--- a/fsck/fsck.c
+++ b/fsck/fsck.c
@@ -334,6 +334,11 @@ static int __check_inode_mode(u32 nid, enum FILE_TYPE ftype, u32 mode)
 {
 	if (ftype >= F2FS_FT_MAX)
 		return 0;
+	/* f2fs_iget will return -EIO if mode is not valid file type */
+	if (!S_ISLNK(mode) && !S_ISREG(mode) && !S_ISDIR(mode) &&
+	    !S_ISCHR(mode) && !S_ISBLK(mode) && !S_ISFIFO(mode) &&
+	    !S_ISSOCK(mode))
+		goto err;
 	if (S_ISLNK(mode) && ftype != F2FS_FT_SYMLINK)
 		goto err;
 	if (S_ISREG(mode) && ftype != F2FS_FT_REG_FILE)
@@ -465,25 +470,6 @@ static int sanity_check_nid(struct f2fs_sb_info *sbi, u32 nid,
 	return 0;
 }
 
-static int sanity_check_inode(struct f2fs_sb_info *sbi, struct f2fs_node *node)
-{
-	struct f2fs_fsck *fsck = F2FS_FSCK(sbi);
-	struct f2fs_inode *fi = &node->i;
-
-	if (!(le16_to_cpu(fi->i_mode) & S_IFMT)) {
-		ASSERT_MSG("i_mode is not valid. [0x%x]", le16_to_cpu(fi->i_mode));
-		goto remove_node;
-	}
-
-	return 0;
-
-remove_node:
-	f2fs_set_bit(le32_to_cpu(node->footer.ino), fsck->nat_area_bitmap);
-	fsck->chk.valid_blk_cnt--;
-	fsck->chk.valid_node_cnt--;
-	return -EINVAL;
-}
-
 static int fsck_chk_xattr_blk(struct f2fs_sb_info *sbi, u32 ino,
 					u32 x_nid, u32 *blk_cnt)
 {
@@ -528,8 +514,6 @@ int fsck_chk_node_blk(struct f2fs_sb_info *sbi, struct f2fs_inode *inode,
 	if (ntype == TYPE_INODE) {
 		struct f2fs_fsck *fsck = F2FS_FSCK(sbi);
 
-		if (sanity_check_inode(sbi, node_blk))
-			goto err;
 		fsck_chk_inode_blk(sbi, nid, ftype, node_blk, blk_cnt, &ni, child);
 		quota_add_inode_usage(fsck->qctx, nid, &node_blk->i);
 	} else {
-- 
2.11.0


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot

^ permalink raw reply related	[flat|nested] 18+ messages in thread

* [RFC PATCH 4/5] mkfs.f2fs: create lost+found directory
  2018-02-06  4:31 [RFC PATCH 0/5] f2fs-tools: introduce lost+found feature Sheng Yong
                   ` (2 preceding siblings ...)
  2018-02-06  4:31 ` [RFC PATCH 3/5] fsck.f2fs: integrate sanity_check_inode to __check_inode_mode Sheng Yong
@ 2018-02-06  4:31 ` Sheng Yong
  2018-02-08 15:08   ` Chao Yu
  2018-02-06  4:31 ` [RFC PATCH 5/5] fsck.f2fs: reconnect unreachable files to lost+found Sheng Yong
  4 siblings, 1 reply; 18+ messages in thread
From: Sheng Yong @ 2018-02-06  4:31 UTC (permalink / raw)
  To: jaegeuk, yuchao0; +Cc: heyunlei, miaoxie, shengyong1, linux-f2fs-devel

This patch introduces a new feature F2FS_FEATURE_LOST_FOUND. It can be
switched on by indicating a new option `lost+found' with -O. If
F2FS_FEATUER_LOST_FOUND is enabled, an empty directory lost+found is
created during mkfs.

This is a preparation for fsck. During fsck, the directory is used to
save unreachable files, which have no parent directory or their parent
directory is removed by fsck. Encrypted files are also allowed to be
saved here.

Signed-off-by: Sheng Yong <shengyong1@huawei.com>
---
 fsck/mount.c            |   3 +
 include/f2fs_fs.h       |   6 ++
 mkfs/f2fs_format.c      | 223 +++++++++++++++++++++++++++++++++++++++++++++---
 mkfs/f2fs_format_main.c |   2 +
 4 files changed, 222 insertions(+), 12 deletions(-)

diff --git a/fsck/mount.c b/fsck/mount.c
index 46cb571..df53c48 100644
--- a/fsck/mount.c
+++ b/fsck/mount.c
@@ -460,6 +460,9 @@ void print_sb_state(struct f2fs_super_block *sb)
 	if (f & cpu_to_le32(F2FS_FEATURE_INODE_CRTIME)) {
 		MSG(0, "%s", " inode_crtime");
 	}
+	if (f & cpu_to_le32(F2FS_FEATURE_LOST_FOUND)) {
+		MSG(0, "%s", " lost+found");
+	}
 	MSG(0, "\n");
 	MSG(0, "Info: superblock encrypt level = %d, salt = ",
 					sb->encryption_level);
diff --git a/include/f2fs_fs.h b/include/f2fs_fs.h
index ca4522d..093c402 100644
--- a/include/f2fs_fs.h
+++ b/include/f2fs_fs.h
@@ -291,6 +291,8 @@ static inline uint64_t bswap_64(uint64_t val)
 
 #define VERSION_LEN	256
 
+#define LPF "lost+found"
+
 enum f2fs_config_func {
 	MKFS,
 	FSCK,
@@ -369,6 +371,9 @@ struct f2fs_configuration {
 	u_int32_t next_free_nid;
 	u_int32_t quota_inum;
 	u_int32_t quota_dnum;
+	u_int32_t lpf_inum;
+	u_int32_t lpf_dnum;
+	u_int32_t lpf_ino;
 
 	/* defragmentation parameters */
 	int defrag_shrink;
@@ -557,6 +562,7 @@ enum {
 #define F2FS_FEATURE_FLEXIBLE_INLINE_XATTR	0x0040
 #define F2FS_FEATURE_QUOTA_INO		0x0080
 #define F2FS_FEATURE_INODE_CRTIME	0x0100
+#define F2FS_FEATURE_LOST_FOUND		0x0200
 
 #define MAX_VOLUME_NAME		512
 
diff --git a/mkfs/f2fs_format.c b/mkfs/f2fs_format.c
index bda3c9d..33dd98c 100644
--- a/mkfs/f2fs_format.c
+++ b/mkfs/f2fs_format.c
@@ -426,6 +426,9 @@ static int f2fs_prepare_super_block(void)
 					qtype, c.next_free_nid - 1);
 	}
 
+	if (c.feature & cpu_to_le32(F2FS_FEATURE_LOST_FOUND))
+		c.lpf_ino = c.next_free_nid++;
+
 	if (total_zones <= 6) {
 		MSG(1, "\tError: %d zones: Need more zones "
 			"by shrinking zone size\n", total_zones);
@@ -608,9 +611,10 @@ static int f2fs_write_check_point_pack(void)
 		set_cp(cur_data_segno[i], 0xffffffff);
 	}
 
-	set_cp(cur_node_blkoff[0], 1 + c.quota_inum);
-	set_cp(cur_data_blkoff[0], 1 + c.quota_dnum);
-	set_cp(valid_block_count, 2 + c.quota_inum + c.quota_dnum);
+	set_cp(cur_node_blkoff[0], 1 + c.quota_inum + c.lpf_inum);
+	set_cp(cur_data_blkoff[0], 1 + c.quota_dnum + c.lpf_dnum);
+	set_cp(valid_block_count, 2 + c.quota_inum + c.quota_dnum +
+			c.lpf_inum + c.lpf_dnum);
 	set_cp(rsvd_segment_count, c.reserved_segments);
 	set_cp(overprov_segment_count, (get_sb(segment_count_main) -
 			get_cp(rsvd_segment_count)) *
@@ -642,8 +646,8 @@ static int f2fs_write_check_point_pack(void)
 
 	set_cp(ckpt_flags, flags);
 	set_cp(cp_pack_start_sum, 1 + get_sb(cp_payload));
-	set_cp(valid_node_count, 1 + c.quota_inum);
-	set_cp(valid_inode_count, 1 + c.quota_inum);
+	set_cp(valid_node_count, 1 + c.quota_inum + c.lpf_inum);
+	set_cp(valid_inode_count, 1 + c.quota_inum + c.lpf_inum);
 	set_cp(next_free_nid, c.next_free_nid);
 	set_cp(sit_ver_bitmap_bytesize, ((get_sb(segment_count_sit) / 2) <<
 			get_sb(log_blocks_per_seg)) / 8);
@@ -702,7 +706,7 @@ static int f2fs_write_check_point_pack(void)
 	SET_SUM_TYPE((&sum->footer), SUM_TYPE_DATA);
 
 	journal = &sum->journal;
-	journal->n_nats = cpu_to_le16(1 + c.quota_inum);
+	journal->n_nats = cpu_to_le16(1 + c.quota_inum + c.lpf_inum);
 	journal->nat_j.entries[0].nid = sb->root_ino;
 	journal->nat_j.entries[0].ne.version = 0;
 	journal->nat_j.entries[0].ne.ino = sb->root_ino;
@@ -723,6 +727,16 @@ static int f2fs_write_check_point_pack(void)
 		i++;
 	}
 
+	if (c.lpf_inum) {
+		journal->nat_j.entries[i].nid = cpu_to_le32(c.lpf_ino);
+		journal->nat_j.entries[i].ne.version = 0;
+		journal->nat_j.entries[i].ne.ino = cpu_to_le32(c.lpf_ino);
+		journal->nat_j.entries[i].ne.block_addr = cpu_to_le32(
+				get_sb(main_blkaddr) +
+				get_cp(cur_node_segno[0]) *
+				c.blks_per_seg + i);
+	}
+
 	memcpy(sum_compact_p, &journal->n_nats, SUM_JOURNAL_SIZE);
 	sum_compact_p += SUM_JOURNAL_SIZE;
 
@@ -732,10 +746,13 @@ static int f2fs_write_check_point_pack(void)
 	journal->sit_j.entries[0].segno = cp->cur_node_segno[0];
 	journal->sit_j.entries[0].se.vblocks =
 				cpu_to_le16((CURSEG_HOT_NODE << 10) |
-						(1 + c.quota_inum));
+						(1 + c.quota_inum + c.lpf_inum));
 	f2fs_set_bit(0, (char *)journal->sit_j.entries[0].se.valid_map);
 	for (i = 1; i <= c.quota_inum; i++)
 		f2fs_set_bit(i, (char *)journal->sit_j.entries[0].se.valid_map);
+	if (c.lpf_inum)
+		f2fs_set_bit(i, (char *)journal->sit_j.entries[0].se.valid_map);
+
 	journal->sit_j.entries[1].segno = cp->cur_node_segno[1];
 	journal->sit_j.entries[1].se.vblocks =
 				cpu_to_le16((CURSEG_WARM_NODE << 10));
@@ -747,10 +764,12 @@ static int f2fs_write_check_point_pack(void)
 	journal->sit_j.entries[3].segno = cp->cur_data_segno[0];
 	journal->sit_j.entries[3].se.vblocks =
 				cpu_to_le16((CURSEG_HOT_DATA << 10) |
-						(1 + c.quota_dnum));
+						(1 + c.quota_dnum + c.lpf_dnum));
 	f2fs_set_bit(0, (char *)journal->sit_j.entries[3].se.valid_map);
 	for (i = 1; i <= c.quota_dnum; i++)
 		f2fs_set_bit(i, (char *)journal->sit_j.entries[3].se.valid_map);
+	if (c.lpf_dnum)
+		f2fs_set_bit(i, (char *)journal->sit_j.entries[3].se.valid_map);
 
 	journal->sit_j.entries[4].segno = cp->cur_data_segno[1];
 	journal->sit_j.entries[4].se.vblocks =
@@ -780,6 +799,11 @@ static int f2fs_write_check_point_pack(void)
 		off += QUOTA_DATA(qtype);
 	}
 
+	if (c.lpf_dnum) {
+		(sum_entry + off)->nid = cpu_to_le32(c.lpf_ino);
+		(sum_entry + off)->ofs_in_node = 0;
+	}
+
 	/* warm data summary, nothing to do */
 	/* cold data summary, nothing to do */
 
@@ -804,6 +828,11 @@ static int f2fs_write_check_point_pack(void)
 		sum->entries[1 + i].ofs_in_node = 0;
 		i++;
 	}
+	if (c.lpf_inum) {
+		i++;
+		sum->entries[i].nid = cpu_to_le32(c.lpf_ino);
+		sum->entries[i].ofs_in_node = 0;
+	}
 
 	cp_seg_blk++;
 	DBG(1, "\tWriting Segment summary for HOT_NODE, at offset 0x%08"PRIx64"\n",
@@ -952,7 +981,7 @@ static int discard_obsolete_dnode(struct f2fs_node *raw_node, u_int64_t offset)
 
 	/* only root inode was written before truncating dnodes */
 	last_inode_pos = start_inode_pos +
-		c.cur_seg[CURSEG_HOT_NODE] * c.blks_per_seg + c.quota_inum;
+		c.cur_seg[CURSEG_HOT_NODE] * c.blks_per_seg + c.quota_inum + c.lpf_inum;
 
 	if (c.zoned_mode)
 		return 0;
@@ -1004,7 +1033,10 @@ static int f2fs_write_root_inode(void)
 			c.blks_per_seg + 1);
 
 	raw_node->i.i_mode = cpu_to_le16(0x41ed);
-	raw_node->i.i_links = cpu_to_le32(2);
+	if (c.lpf_ino)
+		raw_node->i.i_links = cpu_to_le32(3);
+	else
+		raw_node->i.i_links = cpu_to_le32(2);
 	raw_node->i.i_uid = cpu_to_le32(getuid());
 	raw_node->i.i_gid = cpu_to_le32(getgid());
 
@@ -1121,10 +1153,16 @@ static int f2fs_write_default_quota(int qtype, unsigned int blkaddr,
 	dqblk.dqb_pad = cpu_to_le32(0);
 	dqblk.dqb_ihardlimit = cpu_to_le64(0);
 	dqblk.dqb_isoftlimit = cpu_to_le64(0);
-	dqblk.dqb_curinodes = cpu_to_le64(1);
+	if (c.lpf_ino)
+		dqblk.dqb_curinodes = cpu_to_le64(2);
+	else
+		dqblk.dqb_curinodes = cpu_to_le64(1);
 	dqblk.dqb_bhardlimit = cpu_to_le64(0);
 	dqblk.dqb_bsoftlimit = cpu_to_le64(0);
-	dqblk.dqb_curspace = cpu_to_le64(4096);
+	if (c.lpf_ino)
+		dqblk.dqb_curspace = cpu_to_le64(8192);
+	else
+		dqblk.dqb_curspace = cpu_to_le64(4096);
 	dqblk.dqb_btime = cpu_to_le64(0);
 	dqblk.dqb_itime = cpu_to_le64(0);
 
@@ -1299,6 +1337,137 @@ static int f2fs_update_nat_root(void)
 	return 0;
 }
 
+static u_int64_t f2fs_add_default_dentry_lpf(void)
+{
+	struct f2fs_dentry_block *dent_blk;
+	u_int64_t data_blk_offset = 0;
+
+	dent_blk = calloc(F2FS_BLKSIZE, 1);
+	if (dent_blk == NULL) {
+		MSG(1, "\tError: Calloc Failed for dent_blk!!!\n");
+		return -1;
+	}
+
+	dent_blk->dentry[0].hash_code = 0;
+	dent_blk->dentry[0].ino = cpu_to_le32(c.lpf_ino);
+	dent_blk->dentry[0].name_len = cpu_to_le16(1);
+	dent_blk->dentry[0].file_type = F2FS_FT_DIR;
+	memcpy(dent_blk->filename[0], ".", 1);
+
+	dent_blk->dentry[1].hash_code = 0;
+	dent_blk->dentry[1].ino = sb->root_ino;
+	dent_blk->dentry[1].name_len = cpu_to_le16(2);
+	dent_blk->dentry[1].file_type = F2FS_FT_DIR;
+	memcpy(dent_blk->filename[1], "..", 2);
+
+	test_and_set_bit_le(0, dent_blk->dentry_bitmap);
+	test_and_set_bit_le(1, dent_blk->dentry_bitmap);
+
+	data_blk_offset = get_sb(main_blkaddr);
+	data_blk_offset += c.cur_seg[CURSEG_HOT_DATA] * c.blks_per_seg +
+		1 + c.quota_dnum;
+
+	DBG(1, "\tWriting default dentry lost+found, at offset 0x%08"PRIx64"\n",
+			data_blk_offset);
+	if (dev_write_block(dent_blk, data_blk_offset)) {
+		MSG(1, "\tError While writing the dentry_blk to disk!!!\n");
+		free(dent_blk);
+		return -1;
+	}
+
+	free(dent_blk);
+	c.lpf_dnum++;
+	return data_blk_offset;
+}
+
+static int f2fs_write_lpf_inode(void)
+{
+	struct f2fs_node *raw_node;
+	u_int64_t blk_size_bytes, data_blk_nor;
+	u_int64_t main_area_node_seg_blk_offset;
+	int err = 0;
+
+	ASSERT(c.lpf_ino);
+
+	raw_node = calloc(F2FS_BLKSIZE, 1);
+	if (raw_node == NULL) {
+		MSG(1, "\tError: Calloc Failed for raw_node!!!\n");
+		return -1;
+	}
+
+	raw_node->footer.nid = cpu_to_le32(c.lpf_ino);
+	raw_node->footer.ino = raw_node->footer.nid;
+	raw_node->footer.cp_ver = cpu_to_le64(1);
+	raw_node->footer.next_blkaddr = cpu_to_le32(
+			get_sb(main_blkaddr) +
+			c.cur_seg[CURSEG_HOT_NODE] * c.blks_per_seg +
+			1 + c.quota_inum + 1);
+
+	raw_node->i.i_mode = cpu_to_le16(0x41c0); /* 0700 */
+	raw_node->i.i_links = cpu_to_le32(2);
+	raw_node->i.i_uid = cpu_to_le32(getuid());
+	raw_node->i.i_gid = cpu_to_le32(getgid());
+
+	blk_size_bytes = 1 << get_sb(log_blocksize);
+	raw_node->i.i_size = cpu_to_le64(1 * blk_size_bytes); /* dentry */
+	raw_node->i.i_blocks = cpu_to_le64(2);
+
+	raw_node->i.i_atime = cpu_to_le32(time(NULL));
+	raw_node->i.i_atime_nsec = 0;
+	raw_node->i.i_ctime = cpu_to_le32(time(NULL));
+	raw_node->i.i_ctime_nsec = 0;
+	raw_node->i.i_mtime = cpu_to_le32(time(NULL));
+	raw_node->i.i_mtime_nsec = 0;
+	raw_node->i.i_generation = 0;
+	raw_node->i.i_xattr_nid = 0;
+	raw_node->i.i_flags = 0;
+	raw_node->i.i_pino = le32_to_cpu(sb->root_ino);
+	raw_node->i.i_namelen = le32_to_cpu(strlen(LPF));
+	memcpy(raw_node->i.i_name, LPF, strlen(LPF));
+	raw_node->i.i_current_depth = cpu_to_le32(1);
+	raw_node->i.i_dir_level = DEF_DIR_LEVEL;
+
+	if (c.feature & cpu_to_le32(F2FS_FEATURE_EXTRA_ATTR)) {
+		raw_node->i.i_inline = F2FS_EXTRA_ATTR;
+		raw_node->i.i_extra_isize =
+			cpu_to_le16(F2FS_TOTAL_EXTRA_ATTR_SIZE);
+	}
+
+	if (c.feature & cpu_to_le32(F2FS_FEATURE_PRJQUOTA))
+		raw_node->i.i_projid = cpu_to_le32(F2FS_DEF_PROJID);
+
+	data_blk_nor = f2fs_add_default_dentry_lpf();
+	if (data_blk_nor < 0) {
+		MSG(1, "\tError: Failed to add default dentries for lost+found!!!\n");
+		err = -1;
+		goto exit;
+	}
+	raw_node->i.i_addr[get_extra_isize(raw_node)] = cpu_to_le32(data_blk_nor);
+
+	if (c.feature & cpu_to_le32(F2FS_FEATURE_INODE_CHKSUM))
+		raw_node->i.i_inode_checksum =
+			cpu_to_le32(f2fs_inode_chksum(raw_node));
+
+	main_area_node_seg_blk_offset = get_sb(main_blkaddr);
+	main_area_node_seg_blk_offset += c.cur_seg[CURSEG_HOT_NODE] *
+		c.blks_per_seg + c.quota_inum + 1;
+
+	DBG(1, "\tWriting lost+found inode (hot node), %x %x %x at offset 0x%08"PRIu64"\n",
+			get_sb(main_blkaddr),
+			c.cur_seg[CURSEG_HOT_NODE],
+			c.blks_per_seg, main_area_node_seg_blk_offset);
+	if (dev_write_block(raw_node, main_area_node_seg_blk_offset)) {
+		MSG(1, "\tError: While writing the raw_node to disk!!!\n");
+		err = -1;
+		goto exit;
+	}
+
+	c.lpf_inum++;
+exit:
+	free(raw_node);
+	return err;
+}
+
 static int f2fs_add_default_dentry_root(void)
 {
 	struct f2fs_dentry_block *dent_blk = NULL;
@@ -1326,6 +1495,27 @@ static int f2fs_add_default_dentry_root(void)
 	test_and_set_bit_le(0, dent_blk->dentry_bitmap);
 	test_and_set_bit_le(1, dent_blk->dentry_bitmap);
 
+	if (c.lpf_ino) {
+		int len = strlen(LPF);
+		f2fs_hash_t hash = f2fs_dentry_hash((unsigned char *)LPF, len);
+
+		dent_blk->dentry[2].hash_code = cpu_to_le32(hash);
+		dent_blk->dentry[2].ino = cpu_to_le32(c.lpf_ino);
+		dent_blk->dentry[2].name_len = strlen(LPF);
+		dent_blk->dentry[2].file_type = F2FS_FT_DIR;
+		memcpy(dent_blk->filename[2], LPF, F2FS_SLOT_LEN);
+
+		dent_blk->dentry[3].hash_code = cpu_to_le32(hash);
+		dent_blk->dentry[3].ino = cpu_to_le32(c.lpf_ino);
+		dent_blk->dentry[3].name_len = strlen(LPF);
+		dent_blk->dentry[3].file_type = F2FS_FT_DIR;
+		memcpy(dent_blk->filename[3], LPF + F2FS_SLOT_LEN,
+				len - F2FS_SLOT_LEN);
+
+		test_and_set_bit_le(2, dent_blk->dentry_bitmap);
+		test_and_set_bit_le(3, dent_blk->dentry_bitmap);
+	}
+
 	data_blk_offset = get_sb(main_blkaddr);
 	data_blk_offset += c.cur_seg[CURSEG_HOT_DATA] *
 				c.blks_per_seg;
@@ -1363,6 +1553,14 @@ static int f2fs_create_root_dir(void)
 		}
 	}
 
+	if (c.feature & cpu_to_le32(F2FS_FEATURE_LOST_FOUND)) {
+		err = f2fs_write_lpf_inode();
+		if (err < 0) {
+			MSG(1, "\tError: Failed to write lost+found inode!!!\n");
+			goto exit;
+		}
+	}
+
 	err = f2fs_update_nat_root();
 	if (err < 0) {
 		MSG(1, "\tError: Failed to update NAT for root!!!\n");
@@ -1428,6 +1626,7 @@ int f2fs_format_device(void)
 		MSG(0, "\tError: Failed to write the Super Block!!!\n");
 		goto exit;
 	}
+
 exit:
 	if (err)
 		MSG(0, "\tError: Could not format the device!!!\n");
diff --git a/mkfs/f2fs_format_main.c b/mkfs/f2fs_format_main.c
index 120eeba..6382e71 100644
--- a/mkfs/f2fs_format_main.c
+++ b/mkfs/f2fs_format_main.c
@@ -95,6 +95,8 @@ static void parse_feature(const char *features)
 		c.feature |= cpu_to_le32(F2FS_FEATURE_QUOTA_INO);
 	} else if (!strcmp(features, "inode_crtime")) {
 		c.feature |= cpu_to_le32(F2FS_FEATURE_INODE_CRTIME);
+	} else if (!strcmp(features, "lost+found")) {
+		c.feature |= cpu_to_le32(F2FS_FEATURE_LOST_FOUND);
 	} else {
 		MSG(0, "Error: Wrong features\n");
 		mkfs_usage();
-- 
2.11.0


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot

^ permalink raw reply related	[flat|nested] 18+ messages in thread

* [RFC PATCH 5/5] fsck.f2fs: reconnect unreachable files to lost+found
  2018-02-06  4:31 [RFC PATCH 0/5] f2fs-tools: introduce lost+found feature Sheng Yong
                   ` (3 preceding siblings ...)
  2018-02-06  4:31 ` [RFC PATCH 4/5] mkfs.f2fs: create lost+found directory Sheng Yong
@ 2018-02-06  4:31 ` Sheng Yong
  2018-02-07 10:01   ` Sheng Yong
  2018-02-13 14:22   ` Chao Yu
  4 siblings, 2 replies; 18+ messages in thread
From: Sheng Yong @ 2018-02-06  4:31 UTC (permalink / raw)
  To: jaegeuk, yuchao0; +Cc: heyunlei, miaoxie, shengyong1, linux-f2fs-devel

This patch introduces lost+found feature to fsck. If a file is found
unreachable by fsck. Fsck tries to reconnect the file to lost+found
directory:
  1. Scan all unreachable file inodes, ignore non-inodes ones and
     directories.
  2. Check them and fix incorrupted data to make sure filesystem
     metadata (mainly counters and main/nat bitmap) are all consistent.
  3. Reconnect these files to lost+found. If lost+found does not exist,
     create it first. During reconnecting, expand lost+found's dentry
     block automatically. Reconnected files are renamed after its ino
     number.
  4. If reconnect fails drop the node and restore filesystem metadata.

Signed-off-by: Sheng Yong <shengyong1@huawei.com>
---
 fsck/dir.c   |  19 ++-
 fsck/fsck.c  | 376 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-
 fsck/fsck.h  |   3 +
 fsck/mount.c |   2 +
 4 files changed, 397 insertions(+), 3 deletions(-)

diff --git a/fsck/dir.c b/fsck/dir.c
index b2ea18f..567a4e9 100644
--- a/fsck/dir.c
+++ b/fsck/dir.c
@@ -176,6 +176,23 @@ static int f2fs_find_entry(struct f2fs_sb_info *sbi,
 	return 0;
 }
 
+/* return ino if file exists, otherwise return 0 */
+nid_t f2fs_lookup(struct f2fs_sb_info *sbi, struct f2fs_node *dir,
+				u8 *name, int len)
+{
+	int err;
+	struct dentry de = {
+		.name = name,
+		.len = len,
+	};
+
+	err = f2fs_find_entry(sbi, dir, &de);
+	if (err == 1)
+		return de.ino;
+	else
+		return 0;
+}
+
 static void f2fs_update_dentry(nid_t ino, int file_type,
 		struct f2fs_dentry_ptr *d,
 		const unsigned char *name, int len, f2fs_hash_t name_hash,
@@ -199,7 +216,7 @@ static void f2fs_update_dentry(nid_t ino, int file_type,
 /*
  * f2fs_add_link - Add a new file(dir) to parent dir.
  */
-static int f2fs_add_link(struct f2fs_sb_info *sbi, struct f2fs_node *parent,
+int f2fs_add_link(struct f2fs_sb_info *sbi, struct f2fs_node *parent,
 			const unsigned char *name, int name_len, nid_t ino,
 			int file_type, block_t p_blkaddr, int inc_link)
 {
diff --git a/fsck/fsck.c b/fsck/fsck.c
index fcaab14..81e1145 100644
--- a/fsck/fsck.c
+++ b/fsck/fsck.c
@@ -10,6 +10,7 @@
  */
 #include "fsck.h"
 #include "quotaio.h"
+#include <time.h>
 
 char *tree_mark;
 uint32_t tree_mark_size = 256;
@@ -43,6 +44,14 @@ static inline int f2fs_test_main_bitmap(struct f2fs_sb_info *sbi, u32 blk)
 						fsck->main_area_bitmap);
 }
 
+static inline int f2fs_clear_main_bitmap(struct f2fs_sb_info *sbi, u32 blk)
+{
+	struct f2fs_fsck *fsck = F2FS_FSCK(sbi);
+
+	return f2fs_clear_bit(BLKOFF_FROM_MAIN(sbi, blk),
+						fsck->main_area_bitmap);
+}
+
 static inline int f2fs_test_sit_bitmap(struct f2fs_sb_info *sbi, u32 blk)
 {
 	struct f2fs_fsck *fsck = F2FS_FSCK(sbi);
@@ -448,9 +457,11 @@ static int sanity_check_nid(struct f2fs_sb_info *sbi, u32 nid,
 
 	/* workaround to fix later */
 	if (ftype != F2FS_FT_ORPHAN ||
-			f2fs_test_bit(nid, fsck->nat_area_bitmap) != 0)
+			f2fs_test_bit(nid, fsck->nat_area_bitmap) != 0) {
 		f2fs_clear_bit(nid, fsck->nat_area_bitmap);
-	else
+		/* avoid reusing nid when reconnecting files */
+		f2fs_set_bit(nid, NM_I(sbi)->nid_bitmap);
+	} else
 		ASSERT_MSG("orphan or xattr nid is duplicated [0x%x]\n",
 				nid);
 
@@ -2041,6 +2052,362 @@ int check_sit_types(struct f2fs_sb_info *sbi)
 	return err;
 }
 
+static struct f2fs_node *fsck_get_lpf(struct f2fs_sb_info *sbi)
+{
+	struct f2fs_node *node;
+	struct node_info ni;
+	nid_t lpf_ino;
+	int err;
+
+	/* read root inode first */
+	node = calloc(F2FS_BLKSIZE, 1);
+	ASSERT(node);
+	get_node_info(sbi, F2FS_ROOT_INO(sbi), &ni);
+	err = dev_read_block(node, ni.blk_addr);
+	ASSERT(err >= 0);
+
+	/* lookup lost+found in root directory */
+	lpf_ino = f2fs_lookup(sbi, node, (u8 *) LPF, strlen(LPF));
+	if (lpf_ino) { /* found */
+		get_node_info(sbi, lpf_ino, &ni);
+		err = dev_read_block(node, ni.blk_addr);
+		ASSERT(err >= 0);
+		DBG(1, "Found lost+found 0x%x at blkaddr [0x%x]\n",
+		    lpf_ino, ni.blk_addr);
+		if (!S_ISDIR(le16_to_cpu(node->i.i_mode))) {
+			ASSERT_MSG("lost+found is not directory [0%o]\n",
+				   le16_to_cpu(node->i.i_mode));
+			/* FIXME: give up? */
+			goto out;
+		}
+	} else { /* not found, create it */
+		struct dentry de;
+
+		memset(&de, 0, sizeof(de));
+		de.name = (u8 *) LPF;
+		de.len = strlen(LPF);
+		de.mode = 0x41c0;
+		de.pino = F2FS_ROOT_INO(sbi),
+		de.file_type = F2FS_FT_DIR,
+		de.uid = getuid();
+		de.gid = getgid();
+		de.mtime = time(NULL);
+
+		err = f2fs_mkdir(sbi, &de);
+		if (err) {
+			ASSERT_MSG("Failed create lost+found");
+			goto out;
+		}
+
+		get_node_info(sbi, de.ino, &ni);
+		err = dev_read_block(node, ni.blk_addr);
+		ASSERT(err >= 0);
+		DBG(1, "Create lost+found 0x%x at blkaddr [0x%x]\n",
+		    de.ino, ni.blk_addr);
+	}
+
+	c.lpf_ino = le32_to_cpu(node->footer.ino);
+	return node;
+out:
+	free(node);
+	return NULL;
+}
+
+static int fsck_do_reconnect_file(struct f2fs_sb_info *sbi,
+				  struct f2fs_node *lpf,
+				  struct f2fs_node *fnode)
+{
+	char name[80];
+	size_t namelen;
+	nid_t ino = le32_to_cpu(fnode->footer.ino);
+	struct node_info ni;
+	int ret;
+
+	namelen = snprintf(name, 80, "%u", ino);
+	if (namelen >= 80)
+		/* ignore terminating '\0', should never happen */
+		namelen = 79;
+
+	if (f2fs_lookup(sbi, lpf, (u8 *) name, strlen(LPF))) {
+		ASSERT_MSG("Name %s already exist in lost+found", name);
+		return -EEXIST;
+	}
+
+	get_node_info(sbi, le32_to_cpu(lpf->footer.ino), &ni);
+	ret = f2fs_add_link(sbi, lpf, (unsigned char *)name, namelen,
+			    ino, F2FS_FT_REG_FILE, ni.blk_addr, 0);
+	if (ret) {
+		ASSERT_MSG("Failed to add inode [0x%x] to lost+found", ino);
+		return -EINVAL;
+	}
+
+	/* update fnode */
+	memcpy(fnode->i.i_name, name, namelen);
+	fnode->i.i_namelen = cpu_to_le32(namelen);
+	fnode->i.i_pino = c.lpf_ino;
+	get_node_info(sbi, le32_to_cpu(fnode->footer.ino), &ni);
+	ret = dev_write_block(fnode, ni.blk_addr);
+	ASSERT(ret >= 0);
+
+	DBG(1, "Reconnect inode [0x%x] to lost+found\n", ino);
+	return 0;
+}
+
+static void fsck_failed_reconnect_file_dnode(struct f2fs_sb_info *sbi,
+					     nid_t nid)
+{
+	struct f2fs_fsck *fsck = F2FS_FSCK(sbi);
+	struct f2fs_node *node;
+	struct node_info ni;
+	u32 addr;
+	int i, err;
+
+	node = calloc(F2FS_BLKSIZE, 1);
+	ASSERT(node);
+
+	get_node_info(sbi, nid, &ni);
+	err = dev_read_block(node, ni.blk_addr);
+	ASSERT(err >= 0);
+
+	fsck->chk.valid_node_cnt--;
+	fsck->chk.valid_blk_cnt--;
+	f2fs_clear_main_bitmap(sbi, ni.blk_addr);
+
+	for (i = 0; i < ADDRS_PER_BLOCK; i++) {
+		addr = le32_to_cpu(node->dn.addr[i]);
+		if (!addr)
+			continue;
+		fsck->chk.valid_blk_cnt--;
+		if (addr == NEW_ADDR)
+			continue;
+		f2fs_clear_main_bitmap(sbi, addr);
+	}
+}
+
+static void fsck_failed_reconnect_file_idnode(struct f2fs_sb_info *sbi,
+					      nid_t nid)
+{
+	struct f2fs_fsck *fsck = F2FS_FSCK(sbi);
+	struct f2fs_node *node;
+	struct node_info ni;
+	nid_t tmp;
+	int i, err;
+
+	node = calloc(F2FS_BLKSIZE, 1);
+	ASSERT(node);
+
+	get_node_info(sbi, nid, &ni);
+	err = dev_read_block(node, ni.blk_addr);
+	ASSERT(err >= 0);
+
+	fsck->chk.valid_node_cnt--;
+	fsck->chk.valid_blk_cnt--;
+	f2fs_clear_main_bitmap(sbi, ni.blk_addr);
+
+	for (i = 0; i < NIDS_PER_BLOCK; i++) {
+		tmp = le32_to_cpu(node->in.nid[i]);
+		if (!tmp)
+			continue;
+		fsck_failed_reconnect_file_dnode(sbi, tmp);
+	}
+}
+
+static void fsck_failed_reconnect_file_didnode(struct f2fs_sb_info *sbi,
+					       nid_t nid)
+{
+	struct f2fs_fsck *fsck = F2FS_FSCK(sbi);
+	struct f2fs_node *node;
+	struct node_info ni;
+	nid_t tmp;
+	int i, err;
+
+	node = calloc(F2FS_BLKSIZE, 1);
+	ASSERT(node);
+
+	get_node_info(sbi, nid, &ni);
+	err = dev_read_block(node, ni.blk_addr);
+	ASSERT(err >= 0);
+
+	fsck->chk.valid_node_cnt--;
+	fsck->chk.valid_blk_cnt--;
+	f2fs_clear_main_bitmap(sbi, ni.blk_addr);
+
+	for (i = 0; i < NIDS_PER_BLOCK; i++) {
+		tmp = le32_to_cpu(node->in.nid[i]);
+		if (!tmp)
+			continue;
+		fsck_failed_reconnect_file_idnode(sbi, tmp);
+	}
+}
+
+/*
+ * Counters and main_area_bitmap are already changed during checking
+ * inode block, so clear them. There is no need to clear new blocks
+ * allocted to lost+found.
+ */
+static void fsck_failed_reconnect_file(struct f2fs_sb_info *sbi, nid_t ino)
+{
+	struct f2fs_fsck *fsck = F2FS_FSCK(sbi);
+	struct f2fs_node *node;
+	struct node_info ni;
+	nid_t nid;
+	int ofs, i, err;
+
+	node = calloc(F2FS_BLKSIZE, 1);
+	ASSERT(node);
+
+	get_node_info(sbi, ino, &ni);
+	err = dev_read_block(node, ni.blk_addr);
+	ASSERT(err >= 0);
+
+	/* clear inode counters */
+	fsck->chk.valid_inode_cnt--;
+	fsck->chk.valid_node_cnt--;
+	fsck->chk.valid_blk_cnt--;
+	f2fs_clear_main_bitmap(sbi, ni.blk_addr);
+
+	/* clear xnid counters */
+	if (node->i.i_xattr_nid) {
+		nid = le32_to_cpu(node->i.i_xattr_nid);
+		fsck->chk.valid_node_cnt--;
+		fsck->chk.valid_blk_cnt--;
+		get_node_info(sbi, nid, &ni);
+		f2fs_clear_main_bitmap(sbi, ni.blk_addr);
+	}
+
+	/* clear data counters */
+	if(!(node->i.i_inline & F2FS_INLINE_DATA)) {
+		ofs = get_extra_isize(node);
+		for (i = 0; i < ADDRS_PER_INODE(&node->i); i++) {
+			block_t addr = le32_to_cpu(node->i.i_addr[ofs + i]);
+			if (!addr)
+				continue;
+			fsck->chk.valid_blk_cnt--;
+			if (addr == NEW_ADDR)
+				continue;
+			f2fs_clear_main_bitmap(sbi, addr);
+		}
+	}
+
+	for (i = 0; i < 5; i++) {
+		nid = le32_to_cpu(node->i.i_nid[i]);
+		if (!nid)
+			continue;
+
+		switch (i) {
+		case 0: /* direct node */
+		case 1:
+			fsck_failed_reconnect_file_dnode(sbi, nid);
+			break;
+		case 2: /* indirect node */
+		case 3:
+			fsck_failed_reconnect_file_idnode(sbi, nid);
+			break;
+		case 4: /* double indirect node */
+			fsck_failed_reconnect_file_didnode(sbi, nid);
+			break;
+		}
+	}
+}
+
+/*
+ * Scan unreachable nids and find only regular file inodes. If these files
+ * are not corrupted, reconnect them to lost+found.
+ *
+ * Since all unreachable nodes are already checked, we can allocate new
+ * blocks safely.
+ *
+ * This function returns the number of files been reconnected.
+ */
+static int fsck_reconnect_file(struct f2fs_sb_info *sbi)
+{
+	struct f2fs_fsck *fsck = F2FS_FSCK(sbi);
+	struct f2fs_node *lpf_node, *node;
+	struct node_info ni;
+	char *reconnect_bitmap;
+	u32 blk_cnt;
+	nid_t nid;
+	int err, cnt = 0;
+
+	node = calloc(F2FS_BLKSIZE, 1);
+	ASSERT(node);
+
+	reconnect_bitmap = calloc(fsck->nat_area_bitmap_sz, 1);
+	ASSERT(reconnect_bitmap);
+
+	for (nid = 0; nid < fsck->nr_nat_entries; nid++) {
+		if (f2fs_test_bit(nid, fsck->nat_area_bitmap)) {
+			if (is_qf_ino(F2FS_RAW_SUPER(sbi), nid)) {
+				DBG(1, "Not support quota inode [0x%x]\n",
+				    nid);
+				continue;
+			}
+
+			get_node_info(sbi, nid, &ni);
+			err = dev_read_block(node, ni.blk_addr);
+			ASSERT(err >= 0);
+
+			/* reconnection will restore these nodes if needed */
+			if (node->footer.ino != node->footer.nid) {
+				DBG(1, "Not support non-inode node [0x%x]\n",
+				    nid);
+				continue;
+			}
+
+			if (S_ISDIR(le16_to_cpu(node->i.i_mode))) {
+				DBG(1, "Not support directory inode [0x%x]\n",
+				    nid);
+				continue;
+			}
+
+			if (sanity_check_nid(sbi, nid, node,
+					     F2FS_FT_REG_FILE, TYPE_INODE,
+					     &ni)) {
+				ASSERT_MSG("Invalid nid [0x%x]\n", nid);
+				continue;
+			}
+
+			DBG(1, "Check inode 0x%x\n", nid);
+			blk_cnt = 1;
+			fsck_chk_inode_blk(sbi, nid, F2FS_FT_REG_FILE,
+					   node, &blk_cnt, &ni, NULL);
+
+			f2fs_set_bit(nid, reconnect_bitmap);
+		}
+	}
+
+	lpf_node = fsck_get_lpf(sbi);
+	if (!lpf_node)
+		goto out;
+
+	for (nid = 0; nid < fsck->nr_nat_entries; nid++) {
+		if (f2fs_test_bit(nid, reconnect_bitmap)) {
+			get_node_info(sbi, nid, &ni);
+			err = dev_read_block(node, ni.blk_addr);
+			ASSERT(err >= 0);
+
+			if (fsck_do_reconnect_file(sbi, lpf_node, node)) {
+				DBG(1, "Failed to reconnect inode [0x%x]\n",
+				    nid);
+				fsck_failed_reconnect_file(sbi, nid);
+				continue;
+			}
+
+			/* FIXME: need update quota? */
+			quota_add_inode_usage(fsck->qctx, nid, &node->i);
+
+			DBG(1, "Reconnected inode [0x%x] to lost+found\n", nid);
+			cnt++;
+		}
+	}
+
+out:
+	free(node);
+	free(lpf_node);
+	free(reconnect_bitmap);
+	return cnt;
+}
+
 int fsck_verify(struct f2fs_sb_info *sbi)
 {
 	unsigned int i = 0;
@@ -2059,6 +2426,11 @@ int fsck_verify(struct f2fs_sb_info *sbi)
 		}
 	}
 
+	if (nr_unref_nid && c.feature & cpu_to_le32(F2FS_FEATURE_LOST_FOUND)) {
+		i = fsck_reconnect_file(sbi);
+		printf("\n[FSCK] Reconnect %u files to lost+found\n", i);
+	}
+
 	if (fsck->hard_link_list_head != NULL) {
 		node = fsck->hard_link_list_head;
 		while (node) {
diff --git a/fsck/fsck.h b/fsck/fsck.h
index 648c2db..8e133fa 100644
--- a/fsck/fsck.h
+++ b/fsck/fsck.h
@@ -243,6 +243,9 @@ int f2fs_mkdir(struct f2fs_sb_info *, struct dentry *);
 int f2fs_symlink(struct f2fs_sb_info *, struct dentry *);
 int inode_set_selinux(struct f2fs_sb_info *, u32, const char *);
 int f2fs_find_path(struct f2fs_sb_info *, char *, nid_t *);
+nid_t f2fs_lookup(struct f2fs_sb_info *, struct f2fs_node *, u8 *, int);
+int f2fs_add_link(struct f2fs_sb_info *, struct f2fs_node *,
+		const unsigned char *, int, nid_t, int, block_t, int);
 
 /* xattr.c */
 void *read_all_xattrs(struct f2fs_sb_info *, struct f2fs_node *);
diff --git a/fsck/mount.c b/fsck/mount.c
index df53c48..9775e9e 100644
--- a/fsck/mount.c
+++ b/fsck/mount.c
@@ -1587,6 +1587,8 @@ void update_nat_blkaddr(struct f2fs_sb_info *sbi, nid_t ino,
 	if (ino)
 		nat_block->entries[entry_off].ino = cpu_to_le32(ino);
 	nat_block->entries[entry_off].block_addr = cpu_to_le32(newaddr);
+	if (c.func == FSCK)
+		F2FS_FSCK(sbi)->entries[nid] = nat_block->entries[entry_off];
 
 	ret = dev_write_block(nat_block, block_addr);
 	ASSERT(ret >= 0);
-- 
2.11.0


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot

^ permalink raw reply related	[flat|nested] 18+ messages in thread

* Re: [RFC PATCH 5/5] fsck.f2fs: reconnect unreachable files to lost+found
  2018-02-06  4:31 ` [RFC PATCH 5/5] fsck.f2fs: reconnect unreachable files to lost+found Sheng Yong
@ 2018-02-07 10:01   ` Sheng Yong
  2018-02-07 10:04     ` Sheng Yong
  2018-02-13 14:22   ` Chao Yu
  1 sibling, 1 reply; 18+ messages in thread
From: Sheng Yong @ 2018-02-07 10:01 UTC (permalink / raw)
  To: jaegeuk, yuchao0; +Cc: heyunlei, miaoxie, linux-f2fs-devel


On 2018/2/6 12:31, Sheng Yong wrote:
> This patch introduces lost+found feature to fsck. If a file is found
> unreachable by fsck. Fsck tries to reconnect the file to lost+found
> directory:
>    1. Scan all unreachable file inodes, ignore non-inodes ones and
>       directories.
>    2. Check them and fix incorrupted data to make sure filesystem
>       metadata (mainly counters and main/nat bitmap) are all consistent.
>    3. Reconnect these files to lost+found. If lost+found does not exist,
>       create it first. During reconnecting, expand lost+found's dentry
>       block automatically. Reconnected files are renamed after its ino
>       number.
>    4. If reconnect fails drop the node and restore filesystem metadata.
> 
> Signed-off-by: Sheng Yong <shengyong1@huawei.com>
> ---
>   fsck/dir.c   |  19 ++-
>   fsck/fsck.c  | 376 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-
>   fsck/fsck.h  |   3 +
>   fsck/mount.c |   2 +
>   4 files changed, 397 insertions(+), 3 deletions(-)
> 
> diff --git a/fsck/dir.c b/fsck/dir.c
> index b2ea18f..567a4e9 100644
> --- a/fsck/dir.c
> +++ b/fsck/dir.c
> @@ -176,6 +176,23 @@ static int f2fs_find_entry(struct f2fs_sb_info *sbi,
>   	return 0;
>   }
>   
> +/* return ino if file exists, otherwise return 0 */
> +nid_t f2fs_lookup(struct f2fs_sb_info *sbi, struct f2fs_node *dir,
> +				u8 *name, int len)
> +{
> +	int err;
> +	struct dentry de = {
> +		.name = name,
> +		.len = len,
> +	};
> +
> +	err = f2fs_find_entry(sbi, dir, &de);
> +	if (err == 1)
> +		return de.ino;
> +	else
> +		return 0;
> +}
> +
>   static void f2fs_update_dentry(nid_t ino, int file_type,
>   		struct f2fs_dentry_ptr *d,
>   		const unsigned char *name, int len, f2fs_hash_t name_hash,
> @@ -199,7 +216,7 @@ static void f2fs_update_dentry(nid_t ino, int file_type,
>   /*
>    * f2fs_add_link - Add a new file(dir) to parent dir.
>    */
> -static int f2fs_add_link(struct f2fs_sb_info *sbi, struct f2fs_node *parent,
> +int f2fs_add_link(struct f2fs_sb_info *sbi, struct f2fs_node *parent,
>   			const unsigned char *name, int name_len, nid_t ino,
>   			int file_type, block_t p_blkaddr, int inc_link)
>   {
> diff --git a/fsck/fsck.c b/fsck/fsck.c
> index fcaab14..81e1145 100644
> --- a/fsck/fsck.c
> +++ b/fsck/fsck.c
> @@ -10,6 +10,7 @@
>    */
>   #include "fsck.h"
>   #include "quotaio.h"
> +#include <time.h>
>   
>   char *tree_mark;
>   uint32_t tree_mark_size = 256;
> @@ -43,6 +44,14 @@ static inline int f2fs_test_main_bitmap(struct f2fs_sb_info *sbi, u32 blk)
>   						fsck->main_area_bitmap);
>   }
>   
> +static inline int f2fs_clear_main_bitmap(struct f2fs_sb_info *sbi, u32 blk)
> +{
> +	struct f2fs_fsck *fsck = F2FS_FSCK(sbi);
> +
> +	return f2fs_clear_bit(BLKOFF_FROM_MAIN(sbi, blk),
> +						fsck->main_area_bitmap);
> +}
> +
>   static inline int f2fs_test_sit_bitmap(struct f2fs_sb_info *sbi, u32 blk)
>   {
>   	struct f2fs_fsck *fsck = F2FS_FSCK(sbi);
> @@ -448,9 +457,11 @@ static int sanity_check_nid(struct f2fs_sb_info *sbi, u32 nid,
>   
>   	/* workaround to fix later */
>   	if (ftype != F2FS_FT_ORPHAN ||
> -			f2fs_test_bit(nid, fsck->nat_area_bitmap) != 0)
> +			f2fs_test_bit(nid, fsck->nat_area_bitmap) != 0) {
>   		f2fs_clear_bit(nid, fsck->nat_area_bitmap);
> -	else
> +		/* avoid reusing nid when reconnecting files */
> +		f2fs_set_bit(nid, NM_I(sbi)->nid_bitmap);
> +	} else
>   		ASSERT_MSG("orphan or xattr nid is duplicated [0x%x]\n",
>   				nid);
>   
> @@ -2041,6 +2052,362 @@ int check_sit_types(struct f2fs_sb_info *sbi)
>   	return err;
>   }
>   
> +static struct f2fs_node *fsck_get_lpf(struct f2fs_sb_info *sbi)
> +{
> +	struct f2fs_node *node;
> +	struct node_info ni;
> +	nid_t lpf_ino;
> +	int err;
> +
> +	/* read root inode first */
> +	node = calloc(F2FS_BLKSIZE, 1);
> +	ASSERT(node);
> +	get_node_info(sbi, F2FS_ROOT_INO(sbi), &ni);
> +	err = dev_read_block(node, ni.blk_addr);
> +	ASSERT(err >= 0);
> +
> +	/* lookup lost+found in root directory */
> +	lpf_ino = f2fs_lookup(sbi, node, (u8 *) LPF, strlen(LPF));

The 4th parameter should be namelen but not strlen(LPF).

> +	if (lpf_ino) { /* found */
> +		get_node_info(sbi, lpf_ino, &ni);
> +		err = dev_read_block(node, ni.blk_addr);
> +		ASSERT(err >= 0);
> +		DBG(1, "Found lost+found 0x%x at blkaddr [0x%x]\n",
> +		    lpf_ino, ni.blk_addr);
> +		if (!S_ISDIR(le16_to_cpu(node->i.i_mode))) {
> +			ASSERT_MSG("lost+found is not directory [0%o]\n",
> +				   le16_to_cpu(node->i.i_mode));
> +			/* FIXME: give up? */
> +			goto out;
> +		}
> +	} else { /* not found, create it */
> +		struct dentry de;
> +
> +		memset(&de, 0, sizeof(de));
> +		de.name = (u8 *) LPF;
> +		de.len = strlen(LPF);
> +		de.mode = 0x41c0;
> +		de.pino = F2FS_ROOT_INO(sbi),
> +		de.file_type = F2FS_FT_DIR,
> +		de.uid = getuid();
> +		de.gid = getgid();
> +		de.mtime = time(NULL);
> +
> +		err = f2fs_mkdir(sbi, &de);
> +		if (err) {
> +			ASSERT_MSG("Failed create lost+found");
> +			goto out;
> +		}
> +
> +		get_node_info(sbi, de.ino, &ni);
> +		err = dev_read_block(node, ni.blk_addr);
> +		ASSERT(err >= 0);
> +		DBG(1, "Create lost+found 0x%x at blkaddr [0x%x]\n",
> +		    de.ino, ni.blk_addr);
> +	}
> +
> +	c.lpf_ino = le32_to_cpu(node->footer.ino);
> +	return node;
> +out:
> +	free(node);
> +	return NULL;
> +}
> +
> +static int fsck_do_reconnect_file(struct f2fs_sb_info *sbi,
> +				  struct f2fs_node *lpf,
> +				  struct f2fs_node *fnode)
> +{
> +	char name[80];
> +	size_t namelen;
> +	nid_t ino = le32_to_cpu(fnode->footer.ino);
> +	struct node_info ni;
> +	int ret;
> +
> +	namelen = snprintf(name, 80, "%u", ino);
> +	if (namelen >= 80)
> +		/* ignore terminating '\0', should never happen */
> +		namelen = 79;
> +
> +	if (f2fs_lookup(sbi, lpf, (u8 *) name, strlen(LPF))) {
> +		ASSERT_MSG("Name %s already exist in lost+found", name);
> +		return -EEXIST;
> +	}
> +
> +	get_node_info(sbi, le32_to_cpu(lpf->footer.ino), &ni);
> +	ret = f2fs_add_link(sbi, lpf, (unsigned char *)name, namelen,
> +			    ino, F2FS_FT_REG_FILE, ni.blk_addr, 0);
> +	if (ret) {
> +		ASSERT_MSG("Failed to add inode [0x%x] to lost+found", ino);
> +		return -EINVAL;
> +	}
> +
> +	/* update fnode */
> +	memcpy(fnode->i.i_name, name, namelen);
> +	fnode->i.i_namelen = cpu_to_le32(namelen);
> +	fnode->i.i_pino = c.lpf_ino;
> +	get_node_info(sbi, le32_to_cpu(fnode->footer.ino), &ni);
> +	ret = dev_write_block(fnode, ni.blk_addr);
> +	ASSERT(ret >= 0);
> +
> +	DBG(1, "Reconnect inode [0x%x] to lost+found\n", ino);
> +	return 0;
> +}
> +
> +static void fsck_failed_reconnect_file_dnode(struct f2fs_sb_info *sbi,
> +					     nid_t nid)
> +{
> +	struct f2fs_fsck *fsck = F2FS_FSCK(sbi);
> +	struct f2fs_node *node;
> +	struct node_info ni;
> +	u32 addr;
> +	int i, err;
> +
> +	node = calloc(F2FS_BLKSIZE, 1);
> +	ASSERT(node);
> +
> +	get_node_info(sbi, nid, &ni);
> +	err = dev_read_block(node, ni.blk_addr);
> +	ASSERT(err >= 0);
> +
> +	fsck->chk.valid_node_cnt--;
> +	fsck->chk.valid_blk_cnt--;
> +	f2fs_clear_main_bitmap(sbi, ni.blk_addr);
> +
> +	for (i = 0; i < ADDRS_PER_BLOCK; i++) {
> +		addr = le32_to_cpu(node->dn.addr[i]);
> +		if (!addr)
> +			continue;
> +		fsck->chk.valid_blk_cnt--;
> +		if (addr == NEW_ADDR)
> +			continue;
> +		f2fs_clear_main_bitmap(sbi, addr);
> +	}
> +}
> +
> +static void fsck_failed_reconnect_file_idnode(struct f2fs_sb_info *sbi,
> +					      nid_t nid)
> +{
> +	struct f2fs_fsck *fsck = F2FS_FSCK(sbi);
> +	struct f2fs_node *node;
> +	struct node_info ni;
> +	nid_t tmp;
> +	int i, err;
> +
> +	node = calloc(F2FS_BLKSIZE, 1);
> +	ASSERT(node);
> +
> +	get_node_info(sbi, nid, &ni);
> +	err = dev_read_block(node, ni.blk_addr);
> +	ASSERT(err >= 0);
> +
> +	fsck->chk.valid_node_cnt--;
> +	fsck->chk.valid_blk_cnt--;
> +	f2fs_clear_main_bitmap(sbi, ni.blk_addr);
> +
> +	for (i = 0; i < NIDS_PER_BLOCK; i++) {
> +		tmp = le32_to_cpu(node->in.nid[i]);
> +		if (!tmp)
> +			continue;
> +		fsck_failed_reconnect_file_dnode(sbi, tmp);
> +	}
> +}
> +
> +static void fsck_failed_reconnect_file_didnode(struct f2fs_sb_info *sbi,
> +					       nid_t nid)
> +{
> +	struct f2fs_fsck *fsck = F2FS_FSCK(sbi);
> +	struct f2fs_node *node;
> +	struct node_info ni;
> +	nid_t tmp;
> +	int i, err;
> +
> +	node = calloc(F2FS_BLKSIZE, 1);
> +	ASSERT(node);
> +
> +	get_node_info(sbi, nid, &ni);
> +	err = dev_read_block(node, ni.blk_addr);
> +	ASSERT(err >= 0);
> +
> +	fsck->chk.valid_node_cnt--;
> +	fsck->chk.valid_blk_cnt--;
> +	f2fs_clear_main_bitmap(sbi, ni.blk_addr);
> +
> +	for (i = 0; i < NIDS_PER_BLOCK; i++) {
> +		tmp = le32_to_cpu(node->in.nid[i]);
> +		if (!tmp)
> +			continue;
> +		fsck_failed_reconnect_file_idnode(sbi, tmp);
> +	}
> +}
> +
> +/*
> + * Counters and main_area_bitmap are already changed during checking
> + * inode block, so clear them. There is no need to clear new blocks
> + * allocted to lost+found.
> + */
> +static void fsck_failed_reconnect_file(struct f2fs_sb_info *sbi, nid_t ino)
> +{
> +	struct f2fs_fsck *fsck = F2FS_FSCK(sbi);
> +	struct f2fs_node *node;
> +	struct node_info ni;
> +	nid_t nid;
> +	int ofs, i, err;
> +
> +	node = calloc(F2FS_BLKSIZE, 1);
> +	ASSERT(node);
> +
> +	get_node_info(sbi, ino, &ni);
> +	err = dev_read_block(node, ni.blk_addr);
> +	ASSERT(err >= 0);
> +
> +	/* clear inode counters */
> +	fsck->chk.valid_inode_cnt--;
> +	fsck->chk.valid_node_cnt--;
> +	fsck->chk.valid_blk_cnt--;
> +	f2fs_clear_main_bitmap(sbi, ni.blk_addr);
> +
> +	/* clear xnid counters */
> +	if (node->i.i_xattr_nid) {
> +		nid = le32_to_cpu(node->i.i_xattr_nid);
> +		fsck->chk.valid_node_cnt--;
> +		fsck->chk.valid_blk_cnt--;
> +		get_node_info(sbi, nid, &ni);
> +		f2fs_clear_main_bitmap(sbi, ni.blk_addr);
> +	}
> +
> +	/* clear data counters */
> +	if(!(node->i.i_inline & F2FS_INLINE_DATA)) {
> +		ofs = get_extra_isize(node);
> +		for (i = 0; i < ADDRS_PER_INODE(&node->i); i++) {
> +			block_t addr = le32_to_cpu(node->i.i_addr[ofs + i]);
> +			if (!addr)
> +				continue;
> +			fsck->chk.valid_blk_cnt--;
> +			if (addr == NEW_ADDR)
> +				continue;
> +			f2fs_clear_main_bitmap(sbi, addr);
> +		}
> +	}
> +
> +	for (i = 0; i < 5; i++) {
> +		nid = le32_to_cpu(node->i.i_nid[i]);
> +		if (!nid)
> +			continue;
> +
> +		switch (i) {
> +		case 0: /* direct node */
> +		case 1:
> +			fsck_failed_reconnect_file_dnode(sbi, nid);
> +			break;
> +		case 2: /* indirect node */
> +		case 3:
> +			fsck_failed_reconnect_file_idnode(sbi, nid);
> +			break;
> +		case 4: /* double indirect node */
> +			fsck_failed_reconnect_file_didnode(sbi, nid);
> +			break;
> +		}
> +	}
> +}
> +
> +/*
> + * Scan unreachable nids and find only regular file inodes. If these files
> + * are not corrupted, reconnect them to lost+found.
> + *
> + * Since all unreachable nodes are already checked, we can allocate new
> + * blocks safely.
> + *
> + * This function returns the number of files been reconnected.
> + */
> +static int fsck_reconnect_file(struct f2fs_sb_info *sbi)
> +{
> +	struct f2fs_fsck *fsck = F2FS_FSCK(sbi);
> +	struct f2fs_node *lpf_node, *node;
> +	struct node_info ni;
> +	char *reconnect_bitmap;
> +	u32 blk_cnt;
> +	nid_t nid;
> +	int err, cnt = 0;
> +
> +	node = calloc(F2FS_BLKSIZE, 1);
> +	ASSERT(node);
> +
> +	reconnect_bitmap = calloc(fsck->nat_area_bitmap_sz, 1);
> +	ASSERT(reconnect_bitmap);
> +
> +	for (nid = 0; nid < fsck->nr_nat_entries; nid++) {
> +		if (f2fs_test_bit(nid, fsck->nat_area_bitmap)) {
> +			if (is_qf_ino(F2FS_RAW_SUPER(sbi), nid)) {
> +				DBG(1, "Not support quota inode [0x%x]\n",
> +				    nid);
> +				continue;
> +			}
> +
> +			get_node_info(sbi, nid, &ni);
> +			err = dev_read_block(node, ni.blk_addr);
> +			ASSERT(err >= 0);
> +
> +			/* reconnection will restore these nodes if needed */
> +			if (node->footer.ino != node->footer.nid) {
> +				DBG(1, "Not support non-inode node [0x%x]\n",
> +				    nid);
> +				continue;
> +			}
> +
> +			if (S_ISDIR(le16_to_cpu(node->i.i_mode))) {
> +				DBG(1, "Not support directory inode [0x%x]\n",
> +				    nid);
> +				continue;
> +			}
> +
> +			if (sanity_check_nid(sbi, nid, node,
> +					     F2FS_FT_REG_FILE, TYPE_INODE,
> +					     &ni)) {
> +				ASSERT_MSG("Invalid nid [0x%x]\n", nid);
> +				continue;
> +			}
> +
> +			DBG(1, "Check inode 0x%x\n", nid);
> +			blk_cnt = 1;
> +			fsck_chk_inode_blk(sbi, nid, F2FS_FT_REG_FILE,
> +					   node, &blk_cnt, &ni, NULL);
> +
> +			f2fs_set_bit(nid, reconnect_bitmap);
> +		}
> +	}
> +
> +	lpf_node = fsck_get_lpf(sbi);
> +	if (!lpf_node)
> +		goto out;
> +
> +	for (nid = 0; nid < fsck->nr_nat_entries; nid++) {
> +		if (f2fs_test_bit(nid, reconnect_bitmap)) {
> +			get_node_info(sbi, nid, &ni);
> +			err = dev_read_block(node, ni.blk_addr);
> +			ASSERT(err >= 0);
> +
> +			if (fsck_do_reconnect_file(sbi, lpf_node, node)) {
> +				DBG(1, "Failed to reconnect inode [0x%x]\n",
> +				    nid);
> +				fsck_failed_reconnect_file(sbi, nid);
> +				continue;
> +			}
> +
> +			/* FIXME: need update quota? */
> +			quota_add_inode_usage(fsck->qctx, nid, &node->i);
> +
> +			DBG(1, "Reconnected inode [0x%x] to lost+found\n", nid);
> +			cnt++;
> +		}
> +	}
> +
> +out:
> +	free(node);
> +	free(lpf_node);
> +	free(reconnect_bitmap);
> +	return cnt;
> +}
> +
>   int fsck_verify(struct f2fs_sb_info *sbi)
>   {
>   	unsigned int i = 0;
> @@ -2059,6 +2426,11 @@ int fsck_verify(struct f2fs_sb_info *sbi)
>   		}
>   	}
>   
> +	if (nr_unref_nid && c.feature & cpu_to_le32(F2FS_FEATURE_LOST_FOUND)) {
> +		i = fsck_reconnect_file(sbi);

nr_unref_nid should be updated here.   nr_unref_nid -= i;

> +		printf("\n[FSCK] Reconnect %u files to lost+found\n", i);
> +	}
> +
>   	if (fsck->hard_link_list_head != NULL) {
>   		node = fsck->hard_link_list_head;
>   		while (node) {
> diff --git a/fsck/fsck.h b/fsck/fsck.h
> index 648c2db..8e133fa 100644
> --- a/fsck/fsck.h
> +++ b/fsck/fsck.h
> @@ -243,6 +243,9 @@ int f2fs_mkdir(struct f2fs_sb_info *, struct dentry *);
>   int f2fs_symlink(struct f2fs_sb_info *, struct dentry *);
>   int inode_set_selinux(struct f2fs_sb_info *, u32, const char *);
>   int f2fs_find_path(struct f2fs_sb_info *, char *, nid_t *);
> +nid_t f2fs_lookup(struct f2fs_sb_info *, struct f2fs_node *, u8 *, int);
> +int f2fs_add_link(struct f2fs_sb_info *, struct f2fs_node *,
> +		const unsigned char *, int, nid_t, int, block_t, int);
>   
>   /* xattr.c */
>   void *read_all_xattrs(struct f2fs_sb_info *, struct f2fs_node *);
> diff --git a/fsck/mount.c b/fsck/mount.c
> index df53c48..9775e9e 100644
> --- a/fsck/mount.c
> +++ b/fsck/mount.c
> @@ -1587,6 +1587,8 @@ void update_nat_blkaddr(struct f2fs_sb_info *sbi, nid_t ino,
>   	if (ino)
>   		nat_block->entries[entry_off].ino = cpu_to_le32(ino);
>   	nat_block->entries[entry_off].block_addr = cpu_to_le32(newaddr);
> +	if (c.func == FSCK)
> +		F2FS_FSCK(sbi)->entries[nid] = nat_block->entries[entry_off];
>   
>   	ret = dev_write_block(nat_block, block_addr);
>   	ASSERT(ret >= 0);
> 


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot

^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: [RFC PATCH 5/5] fsck.f2fs: reconnect unreachable files to lost+found
  2018-02-07 10:01   ` Sheng Yong
@ 2018-02-07 10:04     ` Sheng Yong
  0 siblings, 0 replies; 18+ messages in thread
From: Sheng Yong @ 2018-02-07 10:04 UTC (permalink / raw)
  To: jaegeuk, yuchao0; +Cc: heyunlei, miaoxie, linux-f2fs-devel



On 2018/2/7 18:01, Sheng Yong wrote:
> 
[...]
>> +
>> +    /* lookup lost+found in root directory */
>> +    lpf_ino = f2fs_lookup(sbi, node, (u8 *) LPF, strlen(LPF));
> 
> The 4th parameter should be namelen but not strlen(LPF).
Sorry for the noise. The comment here is wrong :(
> 
[...]
>> +
>> +static int fsck_do_reconnect_file(struct f2fs_sb_info *sbi,
>> +                  struct f2fs_node *lpf,
>> +                  struct f2fs_node *fnode)
>> +{
>> +    char name[80];
>> +    size_t namelen;
>> +    nid_t ino = le32_to_cpu(fnode->footer.ino);
>> +    struct node_info ni;
>> +    int ret;
>> +
>> +    namelen = snprintf(name, 80, "%u", ino);
>> +    if (namelen >= 80)
>> +        /* ignore terminating '\0', should never happen */
>> +        namelen = 79;
>> +
>> +    if (f2fs_lookup(sbi, lpf, (u8 *) name, strlen(LPF))) {

The 4th parameter should be namelen but not strlen(LPF).

>> +        ASSERT_MSG("Name %s already exist in lost+found", name);
>> +        return -EEXIST;
>> +    }
>> +
[...]
>>


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot

^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: [RFC PATCH 1/5] mkfs.f2fs: introduce mkfs parameters in f2fs_configuration
  2018-02-06  4:31 ` [RFC PATCH 1/5] mkfs.f2fs: introduce mkfs parameters in f2fs_configuration Sheng Yong
@ 2018-02-08 13:30   ` Chao Yu
  2018-02-09  3:21     ` Sheng Yong
  0 siblings, 1 reply; 18+ messages in thread
From: Chao Yu @ 2018-02-08 13:30 UTC (permalink / raw)
  To: Sheng Yong, jaegeuk, yuchao0; +Cc: miaoxie, heyunlei, linux-f2fs-devel

On 2018/2/6 12:31, Sheng Yong wrote:
>  	/* only root inode was written before truncating dnodes */
>  	last_inode_pos = start_inode_pos +
> -		c.cur_seg[CURSEG_HOT_NODE] * c.blks_per_seg + quota_inum;
> +		c.cur_seg[CURSEG_HOT_NODE] * c.blks_per_seg + c.quota_inum;

- f2fs_create_root_dir
 - f2fs_write_root_inode
  - discard_obsolete_dnode
    access c.quota_inum
 - f2fs_write_qf_inode
   update c.quota_inum

Should c.quota_inum be initialized more early?

Thanks,

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot

^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: [RFC PATCH 2/5] f2fs-tools: init f2fs_configuration as 0
  2018-02-06  4:31 ` [RFC PATCH 2/5] f2fs-tools: init f2fs_configuration as 0 Sheng Yong
@ 2018-02-08 13:32   ` Chao Yu
  2018-02-10  2:49     ` Jaegeuk Kim
  0 siblings, 1 reply; 18+ messages in thread
From: Chao Yu @ 2018-02-08 13:32 UTC (permalink / raw)
  To: Sheng Yong, jaegeuk, yuchao0; +Cc: miaoxie, heyunlei, linux-f2fs-devel

On 2018/2/6 12:31, Sheng Yong wrote:
> Signed-off-by: Sheng Yong <shengyong1@huawei.com>

Reviewed-by: Chao Yu <yuchao0@huawei.com>

Thanks,

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot

^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: [RFC PATCH 3/5] fsck.f2fs: integrate sanity_check_inode to __check_inode_mode
  2018-02-06  4:31 ` [RFC PATCH 3/5] fsck.f2fs: integrate sanity_check_inode to __check_inode_mode Sheng Yong
@ 2018-02-08 13:44   ` Chao Yu
  0 siblings, 0 replies; 18+ messages in thread
From: Chao Yu @ 2018-02-08 13:44 UTC (permalink / raw)
  To: Sheng Yong, jaegeuk, yuchao0; +Cc: miaoxie, heyunlei, linux-f2fs-devel

On 2018/2/6 12:31, Sheng Yong wrote:
> In sanity_check_nid, __check_inode_mode will check i_mode value of an
> inode. So integrate sanity_check_inode to __check_inode_mode to clean
> up the code.
> 
> Signed-off-by: Sheng Yong <shengyong1@huawei.com>
> ---
>  fsck/fsck.c | 26 +++++---------------------
>  1 file changed, 5 insertions(+), 21 deletions(-)
> 
> diff --git a/fsck/fsck.c b/fsck/fsck.c
> index 29823a1..fcaab14 100644
> --- a/fsck/fsck.c
> +++ b/fsck/fsck.c
> @@ -334,6 +334,11 @@ static int __check_inode_mode(u32 nid, enum FILE_TYPE ftype, u32 mode)
>  {
>  	if (ftype >= F2FS_FT_MAX)
>  		return 0;
> +	/* f2fs_iget will return -EIO if mode is not valid file type */
> +	if (!S_ISLNK(mode) && !S_ISREG(mode) && !S_ISDIR(mode) &&
> +	    !S_ISCHR(mode) && !S_ISBLK(mode) && !S_ISFIFO(mode) &&
> +	    !S_ISSOCK(mode))

Need an extra error message for this case instead of below one.

Thanks,

> +		goto err;
>  	if (S_ISLNK(mode) && ftype != F2FS_FT_SYMLINK)
>  		goto err;
>  	if (S_ISREG(mode) && ftype != F2FS_FT_REG_FILE)
> @@ -465,25 +470,6 @@ static int sanity_check_nid(struct f2fs_sb_info *sbi, u32 nid,
>  	return 0;
>  }
>  
> -static int sanity_check_inode(struct f2fs_sb_info *sbi, struct f2fs_node *node)
> -{
> -	struct f2fs_fsck *fsck = F2FS_FSCK(sbi);
> -	struct f2fs_inode *fi = &node->i;
> -
> -	if (!(le16_to_cpu(fi->i_mode) & S_IFMT)) {
> -		ASSERT_MSG("i_mode is not valid. [0x%x]", le16_to_cpu(fi->i_mode));
> -		goto remove_node;
> -	}
> -
> -	return 0;
> -
> -remove_node:
> -	f2fs_set_bit(le32_to_cpu(node->footer.ino), fsck->nat_area_bitmap);
> -	fsck->chk.valid_blk_cnt--;
> -	fsck->chk.valid_node_cnt--;
> -	return -EINVAL;
> -}
> -
>  static int fsck_chk_xattr_blk(struct f2fs_sb_info *sbi, u32 ino,
>  					u32 x_nid, u32 *blk_cnt)
>  {
> @@ -528,8 +514,6 @@ int fsck_chk_node_blk(struct f2fs_sb_info *sbi, struct f2fs_inode *inode,
>  	if (ntype == TYPE_INODE) {
>  		struct f2fs_fsck *fsck = F2FS_FSCK(sbi);
>  
> -		if (sanity_check_inode(sbi, node_blk))
> -			goto err;
>  		fsck_chk_inode_blk(sbi, nid, ftype, node_blk, blk_cnt, &ni, child);
>  		quota_add_inode_usage(fsck->qctx, nid, &node_blk->i);
>  	} else {
> 

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot

^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: [RFC PATCH 4/5] mkfs.f2fs: create lost+found directory
  2018-02-06  4:31 ` [RFC PATCH 4/5] mkfs.f2fs: create lost+found directory Sheng Yong
@ 2018-02-08 15:08   ` Chao Yu
  2018-02-09  3:21     ` Sheng Yong
  0 siblings, 1 reply; 18+ messages in thread
From: Chao Yu @ 2018-02-08 15:08 UTC (permalink / raw)
  To: Sheng Yong, jaegeuk, yuchao0; +Cc: miaoxie, heyunlei, linux-f2fs-devel

On 2018/2/6 12:31, Sheng Yong wrote:
> This patch introduces a new feature F2FS_FEATURE_LOST_FOUND. It can be
> switched on by indicating a new option `lost+found' with -O. If

Not sure, do we need to change this option to 'lost_found' to follow other
generic -O options?

> F2FS_FEATUER_LOST_FOUND is enabled, an empty directory lost+found is
> created during mkfs.
> 
> This is a preparation for fsck. During fsck, the directory is used to
> save unreachable files, which have no parent directory or their parent
> directory is removed by fsck. Encrypted files are also allowed to be
> saved here.
> 
> Signed-off-by: Sheng Yong <shengyong1@huawei.com>
> ---
>  fsck/mount.c            |   3 +
>  include/f2fs_fs.h       |   6 ++
>  mkfs/f2fs_format.c      | 223 +++++++++++++++++++++++++++++++++++++++++++++---
>  mkfs/f2fs_format_main.c |   2 +
>  4 files changed, 222 insertions(+), 12 deletions(-)
> 
> diff --git a/fsck/mount.c b/fsck/mount.c
> index 46cb571..df53c48 100644
> --- a/fsck/mount.c
> +++ b/fsck/mount.c
> @@ -460,6 +460,9 @@ void print_sb_state(struct f2fs_super_block *sb)
>  	if (f & cpu_to_le32(F2FS_FEATURE_INODE_CRTIME)) {
>  		MSG(0, "%s", " inode_crtime");
>  	}
> +	if (f & cpu_to_le32(F2FS_FEATURE_LOST_FOUND)) {
> +		MSG(0, "%s", " lost+found");
> +	}
>  	MSG(0, "\n");
>  	MSG(0, "Info: superblock encrypt level = %d, salt = ",
>  					sb->encryption_level);
> diff --git a/include/f2fs_fs.h b/include/f2fs_fs.h
> index ca4522d..093c402 100644
> --- a/include/f2fs_fs.h
> +++ b/include/f2fs_fs.h
> @@ -291,6 +291,8 @@ static inline uint64_t bswap_64(uint64_t val)
>  
>  #define VERSION_LEN	256
>  
> +#define LPF "lost+found"
> +
>  enum f2fs_config_func {
>  	MKFS,
>  	FSCK,
> @@ -369,6 +371,9 @@ struct f2fs_configuration {
>  	u_int32_t next_free_nid;
>  	u_int32_t quota_inum;
>  	u_int32_t quota_dnum;
> +	u_int32_t lpf_inum;
> +	u_int32_t lpf_dnum;
> +	u_int32_t lpf_ino;
>  
>  	/* defragmentation parameters */
>  	int defrag_shrink;
> @@ -557,6 +562,7 @@ enum {
>  #define F2FS_FEATURE_FLEXIBLE_INLINE_XATTR	0x0040
>  #define F2FS_FEATURE_QUOTA_INO		0x0080
>  #define F2FS_FEATURE_INODE_CRTIME	0x0100
> +#define F2FS_FEATURE_LOST_FOUND		0x0200
>  
>  #define MAX_VOLUME_NAME		512
>  
> diff --git a/mkfs/f2fs_format.c b/mkfs/f2fs_format.c
> index bda3c9d..33dd98c 100644
> --- a/mkfs/f2fs_format.c
> +++ b/mkfs/f2fs_format.c
> @@ -426,6 +426,9 @@ static int f2fs_prepare_super_block(void)
>  					qtype, c.next_free_nid - 1);
>  	}
>  
> +	if (c.feature & cpu_to_le32(F2FS_FEATURE_LOST_FOUND))
> +		c.lpf_ino = c.next_free_nid++;
> +
>  	if (total_zones <= 6) {
>  		MSG(1, "\tError: %d zones: Need more zones "
>  			"by shrinking zone size\n", total_zones);
> @@ -608,9 +611,10 @@ static int f2fs_write_check_point_pack(void)
>  		set_cp(cur_data_segno[i], 0xffffffff);
>  	}
>  
> -	set_cp(cur_node_blkoff[0], 1 + c.quota_inum);
> -	set_cp(cur_data_blkoff[0], 1 + c.quota_dnum);
> -	set_cp(valid_block_count, 2 + c.quota_inum + c.quota_dnum);
> +	set_cp(cur_node_blkoff[0], 1 + c.quota_inum + c.lpf_inum);
> +	set_cp(cur_data_blkoff[0], 1 + c.quota_dnum + c.lpf_dnum);
> +	set_cp(valid_block_count, 2 + c.quota_inum + c.quota_dnum +
> +			c.lpf_inum + c.lpf_dnum);
>  	set_cp(rsvd_segment_count, c.reserved_segments);
>  	set_cp(overprov_segment_count, (get_sb(segment_count_main) -
>  			get_cp(rsvd_segment_count)) *
> @@ -642,8 +646,8 @@ static int f2fs_write_check_point_pack(void)
>  
>  	set_cp(ckpt_flags, flags);
>  	set_cp(cp_pack_start_sum, 1 + get_sb(cp_payload));
> -	set_cp(valid_node_count, 1 + c.quota_inum);
> -	set_cp(valid_inode_count, 1 + c.quota_inum);
> +	set_cp(valid_node_count, 1 + c.quota_inum + c.lpf_inum);
> +	set_cp(valid_inode_count, 1 + c.quota_inum + c.lpf_inum);
>  	set_cp(next_free_nid, c.next_free_nid);
>  	set_cp(sit_ver_bitmap_bytesize, ((get_sb(segment_count_sit) / 2) <<
>  			get_sb(log_blocks_per_seg)) / 8);
> @@ -702,7 +706,7 @@ static int f2fs_write_check_point_pack(void)
>  	SET_SUM_TYPE((&sum->footer), SUM_TYPE_DATA);
>  
>  	journal = &sum->journal;
> -	journal->n_nats = cpu_to_le16(1 + c.quota_inum);
> +	journal->n_nats = cpu_to_le16(1 + c.quota_inum + c.lpf_inum);
>  	journal->nat_j.entries[0].nid = sb->root_ino;
>  	journal->nat_j.entries[0].ne.version = 0;
>  	journal->nat_j.entries[0].ne.ino = sb->root_ino;
> @@ -723,6 +727,16 @@ static int f2fs_write_check_point_pack(void)
>  		i++;
>  	}
>  
> +	if (c.lpf_inum) {
> +		journal->nat_j.entries[i].nid = cpu_to_le32(c.lpf_ino);
> +		journal->nat_j.entries[i].ne.version = 0;
> +		journal->nat_j.entries[i].ne.ino = cpu_to_le32(c.lpf_ino);
> +		journal->nat_j.entries[i].ne.block_addr = cpu_to_le32(
> +				get_sb(main_blkaddr) +
> +				get_cp(cur_node_segno[0]) *
> +				c.blks_per_seg + i);
> +	}
> +
>  	memcpy(sum_compact_p, &journal->n_nats, SUM_JOURNAL_SIZE);
>  	sum_compact_p += SUM_JOURNAL_SIZE;
>  
> @@ -732,10 +746,13 @@ static int f2fs_write_check_point_pack(void)
>  	journal->sit_j.entries[0].segno = cp->cur_node_segno[0];
>  	journal->sit_j.entries[0].se.vblocks =
>  				cpu_to_le16((CURSEG_HOT_NODE << 10) |
> -						(1 + c.quota_inum));
> +						(1 + c.quota_inum + c.lpf_inum));
>  	f2fs_set_bit(0, (char *)journal->sit_j.entries[0].se.valid_map);
>  	for (i = 1; i <= c.quota_inum; i++)
>  		f2fs_set_bit(i, (char *)journal->sit_j.entries[0].se.valid_map);
> +	if (c.lpf_inum)
> +		f2fs_set_bit(i, (char *)journal->sit_j.entries[0].se.valid_map);
> +
>  	journal->sit_j.entries[1].segno = cp->cur_node_segno[1];
>  	journal->sit_j.entries[1].se.vblocks =
>  				cpu_to_le16((CURSEG_WARM_NODE << 10));
> @@ -747,10 +764,12 @@ static int f2fs_write_check_point_pack(void)
>  	journal->sit_j.entries[3].segno = cp->cur_data_segno[0];
>  	journal->sit_j.entries[3].se.vblocks =
>  				cpu_to_le16((CURSEG_HOT_DATA << 10) |
> -						(1 + c.quota_dnum));
> +						(1 + c.quota_dnum + c.lpf_dnum));
>  	f2fs_set_bit(0, (char *)journal->sit_j.entries[3].se.valid_map);
>  	for (i = 1; i <= c.quota_dnum; i++)
>  		f2fs_set_bit(i, (char *)journal->sit_j.entries[3].se.valid_map);
> +	if (c.lpf_dnum)
> +		f2fs_set_bit(i, (char *)journal->sit_j.entries[3].se.valid_map);
>  
>  	journal->sit_j.entries[4].segno = cp->cur_data_segno[1];
>  	journal->sit_j.entries[4].se.vblocks =
> @@ -780,6 +799,11 @@ static int f2fs_write_check_point_pack(void)
>  		off += QUOTA_DATA(qtype);
>  	}
>  
> +	if (c.lpf_dnum) {
> +		(sum_entry + off)->nid = cpu_to_le32(c.lpf_ino);
> +		(sum_entry + off)->ofs_in_node = 0;
> +	}
> +
>  	/* warm data summary, nothing to do */
>  	/* cold data summary, nothing to do */
>  
> @@ -804,6 +828,11 @@ static int f2fs_write_check_point_pack(void)
>  		sum->entries[1 + i].ofs_in_node = 0;
>  		i++;
>  	}
> +	if (c.lpf_inum) {
> +		i++;
> +		sum->entries[i].nid = cpu_to_le32(c.lpf_ino);
> +		sum->entries[i].ofs_in_node = 0;
> +	}
>  
>  	cp_seg_blk++;
>  	DBG(1, "\tWriting Segment summary for HOT_NODE, at offset 0x%08"PRIx64"\n",
> @@ -952,7 +981,7 @@ static int discard_obsolete_dnode(struct f2fs_node *raw_node, u_int64_t offset)
>  
>  	/* only root inode was written before truncating dnodes */
>  	last_inode_pos = start_inode_pos +
> -		c.cur_seg[CURSEG_HOT_NODE] * c.blks_per_seg + c.quota_inum;
> +		c.cur_seg[CURSEG_HOT_NODE] * c.blks_per_seg + c.quota_inum + c.lpf_inum;
>  
>  	if (c.zoned_mode)
>  		return 0;
> @@ -1004,7 +1033,10 @@ static int f2fs_write_root_inode(void)
>  			c.blks_per_seg + 1);
>  
>  	raw_node->i.i_mode = cpu_to_le16(0x41ed);
> -	raw_node->i.i_links = cpu_to_le32(2);
> +	if (c.lpf_ino)
> +		raw_node->i.i_links = cpu_to_le32(3);
> +	else
> +		raw_node->i.i_links = cpu_to_le32(2);
>  	raw_node->i.i_uid = cpu_to_le32(getuid());
>  	raw_node->i.i_gid = cpu_to_le32(getgid());
>  
> @@ -1121,10 +1153,16 @@ static int f2fs_write_default_quota(int qtype, unsigned int blkaddr,
>  	dqblk.dqb_pad = cpu_to_le32(0);
>  	dqblk.dqb_ihardlimit = cpu_to_le64(0);
>  	dqblk.dqb_isoftlimit = cpu_to_le64(0);
> -	dqblk.dqb_curinodes = cpu_to_le64(1);
> +	if (c.lpf_ino)
> +		dqblk.dqb_curinodes = cpu_to_le64(2);
> +	else
> +		dqblk.dqb_curinodes = cpu_to_le64(1);
>  	dqblk.dqb_bhardlimit = cpu_to_le64(0);
>  	dqblk.dqb_bsoftlimit = cpu_to_le64(0);
> -	dqblk.dqb_curspace = cpu_to_le64(4096);
> +	if (c.lpf_ino)
> +		dqblk.dqb_curspace = cpu_to_le64(8192);
> +	else
> +		dqblk.dqb_curspace = cpu_to_le64(4096);
>  	dqblk.dqb_btime = cpu_to_le64(0);
>  	dqblk.dqb_itime = cpu_to_le64(0);
>  
> @@ -1299,6 +1337,137 @@ static int f2fs_update_nat_root(void)
>  	return 0;
>  }
>  
> +static u_int64_t f2fs_add_default_dentry_lpf(void)
> +{
> +	struct f2fs_dentry_block *dent_blk;
> +	u_int64_t data_blk_offset = 0;
> +
> +	dent_blk = calloc(F2FS_BLKSIZE, 1);
> +	if (dent_blk == NULL) {
> +		MSG(1, "\tError: Calloc Failed for dent_blk!!!\n");
> +		return -1;
> +	}
> +
> +	dent_blk->dentry[0].hash_code = 0;
> +	dent_blk->dentry[0].ino = cpu_to_le32(c.lpf_ino);
> +	dent_blk->dentry[0].name_len = cpu_to_le16(1);
> +	dent_blk->dentry[0].file_type = F2FS_FT_DIR;
> +	memcpy(dent_blk->filename[0], ".", 1);
> +
> +	dent_blk->dentry[1].hash_code = 0;
> +	dent_blk->dentry[1].ino = sb->root_ino;
> +	dent_blk->dentry[1].name_len = cpu_to_le16(2);
> +	dent_blk->dentry[1].file_type = F2FS_FT_DIR;
> +	memcpy(dent_blk->filename[1], "..", 2);
> +
> +	test_and_set_bit_le(0, dent_blk->dentry_bitmap);
> +	test_and_set_bit_le(1, dent_blk->dentry_bitmap);
> +
> +	data_blk_offset = get_sb(main_blkaddr);
> +	data_blk_offset += c.cur_seg[CURSEG_HOT_DATA] * c.blks_per_seg +
> +		1 + c.quota_dnum;
> +
> +	DBG(1, "\tWriting default dentry lost+found, at offset 0x%08"PRIx64"\n",
> +			data_blk_offset);
> +	if (dev_write_block(dent_blk, data_blk_offset)) {
> +		MSG(1, "\tError While writing the dentry_blk to disk!!!\n");
> +		free(dent_blk);
> +		return -1;
> +	}
> +
> +	free(dent_blk);
> +	c.lpf_dnum++;
> +	return data_blk_offset;
> +}
> +
> +static int f2fs_write_lpf_inode(void)
> +{
> +	struct f2fs_node *raw_node;
> +	u_int64_t blk_size_bytes, data_blk_nor;
> +	u_int64_t main_area_node_seg_blk_offset;
> +	int err = 0;
> +
> +	ASSERT(c.lpf_ino);
> +
> +	raw_node = calloc(F2FS_BLKSIZE, 1);
> +	if (raw_node == NULL) {
> +		MSG(1, "\tError: Calloc Failed for raw_node!!!\n");
> +		return -1;
> +	}
> +
> +	raw_node->footer.nid = cpu_to_le32(c.lpf_ino);
> +	raw_node->footer.ino = raw_node->footer.nid;
> +	raw_node->footer.cp_ver = cpu_to_le64(1);
> +	raw_node->footer.next_blkaddr = cpu_to_le32(
> +			get_sb(main_blkaddr) +
> +			c.cur_seg[CURSEG_HOT_NODE] * c.blks_per_seg +
> +			1 + c.quota_inum + 1);
> +
> +	raw_node->i.i_mode = cpu_to_le16(0x41c0); /* 0700 */
> +	raw_node->i.i_links = cpu_to_le32(2);
> +	raw_node->i.i_uid = cpu_to_le32(getuid());
> +	raw_node->i.i_gid = cpu_to_le32(getgid());
> +
> +	blk_size_bytes = 1 << get_sb(log_blocksize);
> +	raw_node->i.i_size = cpu_to_le64(1 * blk_size_bytes); /* dentry */
> +	raw_node->i.i_blocks = cpu_to_le64(2);
> +
> +	raw_node->i.i_atime = cpu_to_le32(time(NULL));
> +	raw_node->i.i_atime_nsec = 0;
> +	raw_node->i.i_ctime = cpu_to_le32(time(NULL));
> +	raw_node->i.i_ctime_nsec = 0;
> +	raw_node->i.i_mtime = cpu_to_le32(time(NULL));
> +	raw_node->i.i_mtime_nsec = 0;
> +	raw_node->i.i_generation = 0;
> +	raw_node->i.i_xattr_nid = 0;
> +	raw_node->i.i_flags = 0;
> +	raw_node->i.i_pino = le32_to_cpu(sb->root_ino);
> +	raw_node->i.i_namelen = le32_to_cpu(strlen(LPF));
> +	memcpy(raw_node->i.i_name, LPF, strlen(LPF));
> +	raw_node->i.i_current_depth = cpu_to_le32(1);
> +	raw_node->i.i_dir_level = DEF_DIR_LEVEL;
> +
> +	if (c.feature & cpu_to_le32(F2FS_FEATURE_EXTRA_ATTR)) {
> +		raw_node->i.i_inline = F2FS_EXTRA_ATTR;
> +		raw_node->i.i_extra_isize =
> +			cpu_to_le16(F2FS_TOTAL_EXTRA_ATTR_SIZE);
> +	}
> +
> +	if (c.feature & cpu_to_le32(F2FS_FEATURE_PRJQUOTA))
> +		raw_node->i.i_projid = cpu_to_le32(F2FS_DEF_PROJID);

Need to handle inode_crtime case?

> +
> +	data_blk_nor = f2fs_add_default_dentry_lpf();
> +	if (data_blk_nor < 0) {
> +		MSG(1, "\tError: Failed to add default dentries for lost+found!!!\n");
> +		err = -1;
> +		goto exit;
> +	}
> +	raw_node->i.i_addr[get_extra_isize(raw_node)] = cpu_to_le32(data_blk_nor);
> +
> +	if (c.feature & cpu_to_le32(F2FS_FEATURE_INODE_CHKSUM))
> +		raw_node->i.i_inode_checksum =
> +			cpu_to_le32(f2fs_inode_chksum(raw_node));
> +
> +	main_area_node_seg_blk_offset = get_sb(main_blkaddr);
> +	main_area_node_seg_blk_offset += c.cur_seg[CURSEG_HOT_NODE] *
> +		c.blks_per_seg + c.quota_inum + 1;
> +
> +	DBG(1, "\tWriting lost+found inode (hot node), %x %x %x at offset 0x%08"PRIu64"\n",
> +			get_sb(main_blkaddr),
> +			c.cur_seg[CURSEG_HOT_NODE],
> +			c.blks_per_seg, main_area_node_seg_blk_offset);
> +	if (dev_write_block(raw_node, main_area_node_seg_blk_offset)) {
> +		MSG(1, "\tError: While writing the raw_node to disk!!!\n");
> +		err = -1;
> +		goto exit;
> +	}
> +
> +	c.lpf_inum++;
> +exit:
> +	free(raw_node);
> +	return err;
> +}
> +
>  static int f2fs_add_default_dentry_root(void)
>  {
>  	struct f2fs_dentry_block *dent_blk = NULL;
> @@ -1326,6 +1495,27 @@ static int f2fs_add_default_dentry_root(void)
>  	test_and_set_bit_le(0, dent_blk->dentry_bitmap);
>  	test_and_set_bit_le(1, dent_blk->dentry_bitmap);
>  
> +	if (c.lpf_ino) {
> +		int len = strlen(LPF);
> +		f2fs_hash_t hash = f2fs_dentry_hash((unsigned char *)LPF, len);
> +
> +		dent_blk->dentry[2].hash_code = cpu_to_le32(hash);
> +		dent_blk->dentry[2].ino = cpu_to_le32(c.lpf_ino);
> +		dent_blk->dentry[2].name_len = strlen(LPF);

cpu_to_le16(strlen(LPF))

> +		dent_blk->dentry[2].file_type = F2FS_FT_DIR;
> +		memcpy(dent_blk->filename[2], LPF, F2FS_SLOT_LEN);
> +
> +		dent_blk->dentry[3].hash_code = cpu_to_le32(hash);
> +		dent_blk->dentry[3].ino = cpu_to_le32(c.lpf_ino);
> +		dent_blk->dentry[3].name_len = strlen(LPF);
> +		dent_blk->dentry[3].file_type = F2FS_FT_DIR;

Do not need to update this whole slot?

> +		memcpy(dent_blk->filename[3], LPF + F2FS_SLOT_LEN,
> +				len - F2FS_SLOT_LEN);
> +
> +		test_and_set_bit_le(2, dent_blk->dentry_bitmap);
> +		test_and_set_bit_le(3, dent_blk->dentry_bitmap);
> +	}
> +
>  	data_blk_offset = get_sb(main_blkaddr);
>  	data_blk_offset += c.cur_seg[CURSEG_HOT_DATA] *
>  				c.blks_per_seg;
> @@ -1363,6 +1553,14 @@ static int f2fs_create_root_dir(void)
>  		}
>  	}
>  
> +	if (c.feature & cpu_to_le32(F2FS_FEATURE_LOST_FOUND)) {
> +		err = f2fs_write_lpf_inode();
> +		if (err < 0) {
> +			MSG(1, "\tError: Failed to write lost+found inode!!!\n");
> +			goto exit;
> +		}
> +	}
> +
>  	err = f2fs_update_nat_root();
>  	if (err < 0) {
>  		MSG(1, "\tError: Failed to update NAT for root!!!\n");
> @@ -1428,6 +1626,7 @@ int f2fs_format_device(void)
>  		MSG(0, "\tError: Failed to write the Super Block!!!\n");
>  		goto exit;
>  	}
> +

Meaningless blank?

Thanks,

>  exit:
>  	if (err)
>  		MSG(0, "\tError: Could not format the device!!!\n");
> diff --git a/mkfs/f2fs_format_main.c b/mkfs/f2fs_format_main.c
> index 120eeba..6382e71 100644
> --- a/mkfs/f2fs_format_main.c
> +++ b/mkfs/f2fs_format_main.c
> @@ -95,6 +95,8 @@ static void parse_feature(const char *features)
>  		c.feature |= cpu_to_le32(F2FS_FEATURE_QUOTA_INO);
>  	} else if (!strcmp(features, "inode_crtime")) {
>  		c.feature |= cpu_to_le32(F2FS_FEATURE_INODE_CRTIME);
> +	} else if (!strcmp(features, "lost+found")) {
> +		c.feature |= cpu_to_le32(F2FS_FEATURE_LOST_FOUND);
>  	} else {
>  		MSG(0, "Error: Wrong features\n");
>  		mkfs_usage();
> 

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot

^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: [RFC PATCH 1/5] mkfs.f2fs: introduce mkfs parameters in f2fs_configuration
  2018-02-08 13:30   ` Chao Yu
@ 2018-02-09  3:21     ` Sheng Yong
  2018-02-09 12:59       ` Chao Yu
  0 siblings, 1 reply; 18+ messages in thread
From: Sheng Yong @ 2018-02-09  3:21 UTC (permalink / raw)
  To: Chao Yu, jaegeuk, yuchao0, hyojun; +Cc: miaoxie, heyunlei, linux-f2fs-devel

Hi, Chao

Add Hyojun.

On 2018/2/8 21:30, Chao Yu wrote:
> On 2018/2/6 12:31, Sheng Yong wrote:
>>   	/* only root inode was written before truncating dnodes */
>>   	last_inode_pos = start_inode_pos +
>> -		c.cur_seg[CURSEG_HOT_NODE] * c.blks_per_seg + quota_inum;
>> +		c.cur_seg[CURSEG_HOT_NODE] * c.blks_per_seg + c.quota_inum;
> 
> - f2fs_create_root_dir
>   - f2fs_write_root_inode
>    - discard_obsolete_dnode
>      access c.quota_inum
>   - f2fs_write_qf_inode
>     update c.quota_inum
> 
> Should c.quota_inum be initialized more early?

I think we should only count quota inodes if thye are already initialized.
Otherwise, if quota_ino is not enabled, there is only one inode (root) in
CURSEG_HOT_NODE, and two blocks next to root inode will not be discarded.

I'm a bit confused about the `offset' scale in discard_obsolete_dnode.
It seems `if (offset >= start_inode_pos || offset <= last_inode_pos)'
will always be true?

Hi, Hyojun, could you please also have a look at this, thanks :)

diff --git a/mkfs/f2fs_format.c b/mkfs/f2fs_format.c
index 0fc8b30..6c9ae22 100644
--- a/mkfs/f2fs_format.c
+++ b/mkfs/f2fs_format.c
@@ -963,7 +963,7 @@ static int discard_obsolete_dnode(struct f2fs_node *raw_node, u_int64_t offset)
                 }
                 offset = next_blkaddr;
                 /* should avoid recursive chain due to stale data */
-               if (offset >= start_inode_pos || offset <= last_inode_pos)
+               if (offset <= last_inode_pos)
                         break;
         } while (1);

thanks,
Sheng
> 
> Thanks,
> 
> .
> 


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot

^ permalink raw reply related	[flat|nested] 18+ messages in thread

* Re: [RFC PATCH 4/5] mkfs.f2fs: create lost+found directory
  2018-02-08 15:08   ` Chao Yu
@ 2018-02-09  3:21     ` Sheng Yong
  2018-02-09 13:13       ` Chao Yu
  0 siblings, 1 reply; 18+ messages in thread
From: Sheng Yong @ 2018-02-09  3:21 UTC (permalink / raw)
  To: Chao Yu, jaegeuk, yuchao0; +Cc: miaoxie, heyunlei, linux-f2fs-devel

Hi, Chao

On 2018/2/8 23:08, Chao Yu wrote:
> On 2018/2/6 12:31, Sheng Yong wrote:
>> This patch introduces a new feature F2FS_FEATURE_LOST_FOUND. It can be
>> switched on by indicating a new option `lost+found' with -O. If
> 
> Not sure, do we need to change this option to 'lost_found' to follow other
> generic -O options?

I'm also not sure about this, since there is already a `lost_found' in fsck :(

> 
>> F2FS_FEATUER_LOST_FOUND is enabled, an empty directory lost+found is
>> created during mkfs.
>>
>> This is a preparation for fsck. During fsck, the directory is used to
>> save unreachable files, which have no parent directory or their parent
>> directory is removed by fsck. Encrypted files are also allowed to be
>> saved here.
>>
[...]
>> +static int f2fs_write_lpf_inode(void)
>> +{
>> +	struct f2fs_node *raw_node;
>> +	u_int64_t blk_size_bytes, data_blk_nor;
>> +	u_int64_t main_area_node_seg_blk_offset;
>> +	int err = 0;
>> +
>> +	ASSERT(c.lpf_ino);
>> +
>> +	raw_node = calloc(F2FS_BLKSIZE, 1);
>> +	if (raw_node == NULL) {
>> +		MSG(1, "\tError: Calloc Failed for raw_node!!!\n");
>> +		return -1;
>> +	}
>> +
>> +	raw_node->footer.nid = cpu_to_le32(c.lpf_ino);
>> +	raw_node->footer.ino = raw_node->footer.nid;
>> +	raw_node->footer.cp_ver = cpu_to_le64(1);
>> +	raw_node->footer.next_blkaddr = cpu_to_le32(
>> +			get_sb(main_blkaddr) +
>> +			c.cur_seg[CURSEG_HOT_NODE] * c.blks_per_seg +
>> +			1 + c.quota_inum + 1);
>> +
>> +	raw_node->i.i_mode = cpu_to_le16(0x41c0); /* 0700 */
>> +	raw_node->i.i_links = cpu_to_le32(2);
>> +	raw_node->i.i_uid = cpu_to_le32(getuid());
>> +	raw_node->i.i_gid = cpu_to_le32(getgid());
>> +
>> +	blk_size_bytes = 1 << get_sb(log_blocksize);
>> +	raw_node->i.i_size = cpu_to_le64(1 * blk_size_bytes); /* dentry */
>> +	raw_node->i.i_blocks = cpu_to_le64(2);
>> +
>> +	raw_node->i.i_atime = cpu_to_le32(time(NULL));
>> +	raw_node->i.i_atime_nsec = 0;
>> +	raw_node->i.i_ctime = cpu_to_le32(time(NULL));
>> +	raw_node->i.i_ctime_nsec = 0;
>> +	raw_node->i.i_mtime = cpu_to_le32(time(NULL));
>> +	raw_node->i.i_mtime_nsec = 0;
>> +	raw_node->i.i_generation = 0;
>> +	raw_node->i.i_xattr_nid = 0;
>> +	raw_node->i.i_flags = 0;
>> +	raw_node->i.i_pino = le32_to_cpu(sb->root_ino);
>> +	raw_node->i.i_namelen = le32_to_cpu(strlen(LPF));
>> +	memcpy(raw_node->i.i_name, LPF, strlen(LPF));
>> +	raw_node->i.i_current_depth = cpu_to_le32(1);
>> +	raw_node->i.i_dir_level = DEF_DIR_LEVEL;
>> +
>> +	if (c.feature & cpu_to_le32(F2FS_FEATURE_EXTRA_ATTR)) {
>> +		raw_node->i.i_inline = F2FS_EXTRA_ATTR;
>> +		raw_node->i.i_extra_isize =
>> +			cpu_to_le16(F2FS_TOTAL_EXTRA_ATTR_SIZE);
>> +	}
>> +
>> +	if (c.feature & cpu_to_le32(F2FS_FEATURE_PRJQUOTA))
>> +		raw_node->i.i_projid = cpu_to_le32(F2FS_DEF_PROJID);
> 
> Need to handle inode_crtime case?

Right, thanks.
> 
>> +
>> +	data_blk_nor = f2fs_add_default_dentry_lpf();
>> +	if (data_blk_nor < 0) {
>> +		MSG(1, "\tError: Failed to add default dentries for lost+found!!!\n");
>> +		err = -1;
>> +		goto exit;
>> +	}
>> +	raw_node->i.i_addr[get_extra_isize(raw_node)] = cpu_to_le32(data_blk_nor);
>> +
>> +	if (c.feature & cpu_to_le32(F2FS_FEATURE_INODE_CHKSUM))
>> +		raw_node->i.i_inode_checksum =
>> +			cpu_to_le32(f2fs_inode_chksum(raw_node));
>> +
>> +	main_area_node_seg_blk_offset = get_sb(main_blkaddr);
>> +	main_area_node_seg_blk_offset += c.cur_seg[CURSEG_HOT_NODE] *
>> +		c.blks_per_seg + c.quota_inum + 1;
>> +
>> +	DBG(1, "\tWriting lost+found inode (hot node), %x %x %x at offset 0x%08"PRIu64"\n",
>> +			get_sb(main_blkaddr),
>> +			c.cur_seg[CURSEG_HOT_NODE],
>> +			c.blks_per_seg, main_area_node_seg_blk_offset);
>> +	if (dev_write_block(raw_node, main_area_node_seg_blk_offset)) {
>> +		MSG(1, "\tError: While writing the raw_node to disk!!!\n");
>> +		err = -1;
>> +		goto exit;
>> +	}
>> +
>> +	c.lpf_inum++;
>> +exit:
>> +	free(raw_node);
>> +	return err;
>> +}
>> +
>>   static int f2fs_add_default_dentry_root(void)
>>   {
>>   	struct f2fs_dentry_block *dent_blk = NULL;
>> @@ -1326,6 +1495,27 @@ static int f2fs_add_default_dentry_root(void)
>>   	test_and_set_bit_le(0, dent_blk->dentry_bitmap);
>>   	test_and_set_bit_le(1, dent_blk->dentry_bitmap);
>>   
>> +	if (c.lpf_ino) {
>> +		int len = strlen(LPF);
>> +		f2fs_hash_t hash = f2fs_dentry_hash((unsigned char *)LPF, len);
>> +
>> +		dent_blk->dentry[2].hash_code = cpu_to_le32(hash);
>> +		dent_blk->dentry[2].ino = cpu_to_le32(c.lpf_ino);
>> +		dent_blk->dentry[2].name_len = strlen(LPF);
> 
> cpu_to_le16(strlen(LPF))

Will re-check all endianness problems :)
> 
>> +		dent_blk->dentry[2].file_type = F2FS_FT_DIR;
>> +		memcpy(dent_blk->filename[2], LPF, F2FS_SLOT_LEN);
>> +
>> +		dent_blk->dentry[3].hash_code = cpu_to_le32(hash);
>> +		dent_blk->dentry[3].ino = cpu_to_le32(c.lpf_ino);
>> +		dent_blk->dentry[3].name_len = strlen(LPF);
>> +		dent_blk->dentry[3].file_type = F2FS_FT_DIR;
> 
> Do not need to update this whole slot?

What do you mean by updating the whole slot? Two slots used by "lost+found"
are filled.

thanks,
Sheng
> 
>> +		memcpy(dent_blk->filename[3], LPF + F2FS_SLOT_LEN,
>> +				len - F2FS_SLOT_LEN);
>> +
>> +		test_and_set_bit_le(2, dent_blk->dentry_bitmap);
>> +		test_and_set_bit_le(3, dent_blk->dentry_bitmap);
>> +	}
>> +
>>   	data_blk_offset = get_sb(main_blkaddr);
>>   	data_blk_offset += c.cur_seg[CURSEG_HOT_DATA] *
>>   				c.blks_per_seg;
>> @@ -1363,6 +1553,14 @@ static int f2fs_create_root_dir(void)
>>   		}
>>   	}
>>   
>> +	if (c.feature & cpu_to_le32(F2FS_FEATURE_LOST_FOUND)) {
>> +		err = f2fs_write_lpf_inode();
>> +		if (err < 0) {
>> +			MSG(1, "\tError: Failed to write lost+found inode!!!\n");
>> +			goto exit;
>> +		}
>> +	}
>> +
>>   	err = f2fs_update_nat_root();
>>   	if (err < 0) {
>>   		MSG(1, "\tError: Failed to update NAT for root!!!\n");
>> @@ -1428,6 +1626,7 @@ int f2fs_format_device(void)
>>   		MSG(0, "\tError: Failed to write the Super Block!!!\n");
>>   		goto exit;
>>   	}
>> +
> 
> Meaningless blank?
> 
> Thanks,
> 
>>   exit:
>>   	if (err)
>>   		MSG(0, "\tError: Could not format the device!!!\n");
>> diff --git a/mkfs/f2fs_format_main.c b/mkfs/f2fs_format_main.c
>> index 120eeba..6382e71 100644
>> --- a/mkfs/f2fs_format_main.c
>> +++ b/mkfs/f2fs_format_main.c
>> @@ -95,6 +95,8 @@ static void parse_feature(const char *features)
>>   		c.feature |= cpu_to_le32(F2FS_FEATURE_QUOTA_INO);
>>   	} else if (!strcmp(features, "inode_crtime")) {
>>   		c.feature |= cpu_to_le32(F2FS_FEATURE_INODE_CRTIME);
>> +	} else if (!strcmp(features, "lost+found")) {
>> +		c.feature |= cpu_to_le32(F2FS_FEATURE_LOST_FOUND);
>>   	} else {
>>   		MSG(0, "Error: Wrong features\n");
>>   		mkfs_usage();
>>
> 
> .
> 


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot

^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: [RFC PATCH 1/5] mkfs.f2fs: introduce mkfs parameters in f2fs_configuration
  2018-02-09  3:21     ` Sheng Yong
@ 2018-02-09 12:59       ` Chao Yu
  0 siblings, 0 replies; 18+ messages in thread
From: Chao Yu @ 2018-02-09 12:59 UTC (permalink / raw)
  To: Sheng Yong, jaegeuk, yuchao0, hyojun; +Cc: miaoxie, heyunlei, linux-f2fs-devel

Hi Sheng Yong,

On 2018/2/9 11:21, Sheng Yong wrote:
> Hi, Chao
> 
> Add Hyojun.
> 
> On 2018/2/8 21:30, Chao Yu wrote:
>> On 2018/2/6 12:31, Sheng Yong wrote:
>>>       /* only root inode was written before truncating dnodes */
>>>       last_inode_pos = start_inode_pos +
>>> -        c.cur_seg[CURSEG_HOT_NODE] * c.blks_per_seg + quota_inum;
>>> +        c.cur_seg[CURSEG_HOT_NODE] * c.blks_per_seg + c.quota_inum;
>>
>> - f2fs_create_root_dir
>>   - f2fs_write_root_inode
>>    - discard_obsolete_dnode
>>      access c.quota_inum
>>   - f2fs_write_qf_inode
>>     update c.quota_inum
>>
>> Should c.quota_inum be initialized more early?
> 
> I think we should only count quota inodes if thye are already initialized.
> Otherwise, if quota_ino is not enabled, there is only one inode (root) in
> CURSEG_HOT_NODE, and two blocks next to root inode will not be discarded.

IMO, we need to move discard_obsolete_dnode in to f2fs_create_root_dir as below:

- f2fs_create_root_dir
 - f2fs_write_root_inode
 - f2fs_write_qf_inode
 - discard_obsolete_dnode

So the last obsolete position can be calculated correctly.

Thanks,

> 
> I'm a bit confused about the `offset' scale in discard_obsolete_dnode.
> It seems `if (offset >= start_inode_pos || offset <= last_inode_pos)'
> will always be true?
> 
> Hi, Hyojun, could you please also have a look at this, thanks :)
> 
> diff --git a/mkfs/f2fs_format.c b/mkfs/f2fs_format.c
> index 0fc8b30..6c9ae22 100644
> --- a/mkfs/f2fs_format.c
> +++ b/mkfs/f2fs_format.c
> @@ -963,7 +963,7 @@ static int discard_obsolete_dnode(struct f2fs_node *raw_node, u_int64_t offset)
>                 }
>                 offset = next_blkaddr;
>                 /* should avoid recursive chain due to stale data */
> -               if (offset >= start_inode_pos || offset <= last_inode_pos)
> +               if (offset <= last_inode_pos)
>                         break;
>         } while (1);
> 
> thanks,
> Sheng
>>
>> Thanks,
>>
>> .
>>
> 

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: [RFC PATCH 4/5] mkfs.f2fs: create lost+found directory
  2018-02-09  3:21     ` Sheng Yong
@ 2018-02-09 13:13       ` Chao Yu
  0 siblings, 0 replies; 18+ messages in thread
From: Chao Yu @ 2018-02-09 13:13 UTC (permalink / raw)
  To: Sheng Yong, jaegeuk, yuchao0; +Cc: heyunlei, miaoxie, linux-f2fs-devel

Hi Sheng Yong,

On 2018/2/9 11:21, Sheng Yong wrote:
> Hi, Chao
> 
> On 2018/2/8 23:08, Chao Yu wrote:
>> On 2018/2/6 12:31, Sheng Yong wrote:
>>> This patch introduces a new feature F2FS_FEATURE_LOST_FOUND. It can be
>>> switched on by indicating a new option `lost+found' with -O. If
>>
>> Not sure, do we need to change this option to 'lost_found' to follow other
>> generic -O options?
> 
> I'm also not sure about this, since there is already a `lost_found' in fsck :(

Change 'lost_found' to 'lost+found' in fsck to unify the output path?

>> Do not need to update this whole slot?
> 
> What do you mean by updating the whole slot? Two slots used by "lost+found"
> are filled.

'lost+found' takes two slots, but we need only fill all contents in first slot
and filename & bitmap of these two slots, as in both kernel/userspace sides,
we will skip updating & travelling second and later slots during add_link & lookup.
Let's keep consistent with the behavior.

Thanks,

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot

^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: [RFC PATCH 2/5] f2fs-tools: init f2fs_configuration as 0
  2018-02-08 13:32   ` Chao Yu
@ 2018-02-10  2:49     ` Jaegeuk Kim
  0 siblings, 0 replies; 18+ messages in thread
From: Jaegeuk Kim @ 2018-02-10  2:49 UTC (permalink / raw)
  To: Chao Yu; +Cc: miaoxie, heyunlei, linux-f2fs-devel

On 02/08, Chao Yu wrote:
> On 2018/2/6 12:31, Sheng Yong wrote:
> > Signed-off-by: Sheng Yong <shengyong1@huawei.com>
> 
> Reviewed-by: Chao Yu <yuchao0@huawei.com>

Sheng Yong,

I only merged this patch in this set. ;)

> 
> Thanks,

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot

^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: [RFC PATCH 5/5] fsck.f2fs: reconnect unreachable files to lost+found
  2018-02-06  4:31 ` [RFC PATCH 5/5] fsck.f2fs: reconnect unreachable files to lost+found Sheng Yong
  2018-02-07 10:01   ` Sheng Yong
@ 2018-02-13 14:22   ` Chao Yu
  1 sibling, 0 replies; 18+ messages in thread
From: Chao Yu @ 2018-02-13 14:22 UTC (permalink / raw)
  To: Sheng Yong, jaegeuk, yuchao0; +Cc: miaoxie, heyunlei, linux-f2fs-devel

On 2018/2/6 12:31, Sheng Yong wrote:
> This patch introduces lost+found feature to fsck. If a file is found
> unreachable by fsck. Fsck tries to reconnect the file to lost+found
> directory:
>   1. Scan all unreachable file inodes, ignore non-inodes ones and
>      directories.
>   2. Check them and fix incorrupted data to make sure filesystem
>      metadata (mainly counters and main/nat bitmap) are all consistent.
>   3. Reconnect these files to lost+found. If lost+found does not exist,
>      create it first. During reconnecting, expand lost+found's dentry
>      block automatically. Reconnected files are renamed after its ino
>      number.
>   4. If reconnect fails drop the node and restore filesystem metadata.
> 
> Signed-off-by: Sheng Yong <shengyong1@huawei.com>
> ---
>  fsck/dir.c   |  19 ++-
>  fsck/fsck.c  | 376 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-
>  fsck/fsck.h  |   3 +
>  fsck/mount.c |   2 +
>  4 files changed, 397 insertions(+), 3 deletions(-)
> 
> diff --git a/fsck/dir.c b/fsck/dir.c
> index b2ea18f..567a4e9 100644
> --- a/fsck/dir.c
> +++ b/fsck/dir.c
> @@ -176,6 +176,23 @@ static int f2fs_find_entry(struct f2fs_sb_info *sbi,
>  	return 0;
>  }
>  
> +/* return ino if file exists, otherwise return 0 */
> +nid_t f2fs_lookup(struct f2fs_sb_info *sbi, struct f2fs_node *dir,
> +				u8 *name, int len)
> +{
> +	int err;
> +	struct dentry de = {
> +		.name = name,
> +		.len = len,
> +	};
> +
> +	err = f2fs_find_entry(sbi, dir, &de);
> +	if (err == 1)
> +		return de.ino;
> +	else
> +		return 0;
> +}
> +
>  static void f2fs_update_dentry(nid_t ino, int file_type,
>  		struct f2fs_dentry_ptr *d,
>  		const unsigned char *name, int len, f2fs_hash_t name_hash,
> @@ -199,7 +216,7 @@ static void f2fs_update_dentry(nid_t ino, int file_type,
>  /*
>   * f2fs_add_link - Add a new file(dir) to parent dir.
>   */
> -static int f2fs_add_link(struct f2fs_sb_info *sbi, struct f2fs_node *parent,
> +int f2fs_add_link(struct f2fs_sb_info *sbi, struct f2fs_node *parent,
>  			const unsigned char *name, int name_len, nid_t ino,
>  			int file_type, block_t p_blkaddr, int inc_link)
>  {
> diff --git a/fsck/fsck.c b/fsck/fsck.c
> index fcaab14..81e1145 100644
> --- a/fsck/fsck.c
> +++ b/fsck/fsck.c
> @@ -10,6 +10,7 @@
>   */
>  #include "fsck.h"
>  #include "quotaio.h"
> +#include <time.h>
>  
>  char *tree_mark;
>  uint32_t tree_mark_size = 256;
> @@ -43,6 +44,14 @@ static inline int f2fs_test_main_bitmap(struct f2fs_sb_info *sbi, u32 blk)
>  						fsck->main_area_bitmap);
>  }
>  
> +static inline int f2fs_clear_main_bitmap(struct f2fs_sb_info *sbi, u32 blk)
> +{
> +	struct f2fs_fsck *fsck = F2FS_FSCK(sbi);
> +
> +	return f2fs_clear_bit(BLKOFF_FROM_MAIN(sbi, blk),
> +						fsck->main_area_bitmap);
> +}
> +
>  static inline int f2fs_test_sit_bitmap(struct f2fs_sb_info *sbi, u32 blk)
>  {
>  	struct f2fs_fsck *fsck = F2FS_FSCK(sbi);
> @@ -448,9 +457,11 @@ static int sanity_check_nid(struct f2fs_sb_info *sbi, u32 nid,
>  
>  	/* workaround to fix later */
>  	if (ftype != F2FS_FT_ORPHAN ||
> -			f2fs_test_bit(nid, fsck->nat_area_bitmap) != 0)
> +			f2fs_test_bit(nid, fsck->nat_area_bitmap) != 0) {
>  		f2fs_clear_bit(nid, fsck->nat_area_bitmap);
> -	else
> +		/* avoid reusing nid when reconnecting files */
> +		f2fs_set_bit(nid, NM_I(sbi)->nid_bitmap);
> +	} else
>  		ASSERT_MSG("orphan or xattr nid is duplicated [0x%x]\n",
>  				nid);
>  
> @@ -2041,6 +2052,362 @@ int check_sit_types(struct f2fs_sb_info *sbi)
>  	return err;
>  }
>  
> +static struct f2fs_node *fsck_get_lpf(struct f2fs_sb_info *sbi)
> +{
> +	struct f2fs_node *node;
> +	struct node_info ni;
> +	nid_t lpf_ino;
> +	int err;
> +
> +	/* read root inode first */
> +	node = calloc(F2FS_BLKSIZE, 1);
> +	ASSERT(node);
> +	get_node_info(sbi, F2FS_ROOT_INO(sbi), &ni);
> +	err = dev_read_block(node, ni.blk_addr);
> +	ASSERT(err >= 0);
> +
> +	/* lookup lost+found in root directory */
> +	lpf_ino = f2fs_lookup(sbi, node, (u8 *) LPF, strlen(LPF));
> +	if (lpf_ino) { /* found */
> +		get_node_info(sbi, lpf_ino, &ni);
> +		err = dev_read_block(node, ni.blk_addr);
> +		ASSERT(err >= 0);
> +		DBG(1, "Found lost+found 0x%x at blkaddr [0x%x]\n",
> +		    lpf_ino, ni.blk_addr);
> +		if (!S_ISDIR(le16_to_cpu(node->i.i_mode))) {
> +			ASSERT_MSG("lost+found is not directory [0%o]\n",
> +				   le16_to_cpu(node->i.i_mode));
> +			/* FIXME: give up? */
> +			goto out;
> +		}
> +	} else { /* not found, create it */
> +		struct dentry de;
> +
> +		memset(&de, 0, sizeof(de));
> +		de.name = (u8 *) LPF;
> +		de.len = strlen(LPF);
> +		de.mode = 0x41c0;
> +		de.pino = F2FS_ROOT_INO(sbi),
> +		de.file_type = F2FS_FT_DIR,
> +		de.uid = getuid();
> +		de.gid = getgid();
> +		de.mtime = time(NULL);
> +
> +		err = f2fs_mkdir(sbi, &de);
> +		if (err) {
> +			ASSERT_MSG("Failed create lost+found");
> +			goto out;
> +		}
> +
> +		get_node_info(sbi, de.ino, &ni);
> +		err = dev_read_block(node, ni.blk_addr);
> +		ASSERT(err >= 0);
> +		DBG(1, "Create lost+found 0x%x at blkaddr [0x%x]\n",
> +		    de.ino, ni.blk_addr);
> +	}
> +
> +	c.lpf_ino = le32_to_cpu(node->footer.ino);
> +	return node;
> +out:
> +	free(node);
> +	return NULL;
> +}
> +
> +static int fsck_do_reconnect_file(struct f2fs_sb_info *sbi,
> +				  struct f2fs_node *lpf,
> +				  struct f2fs_node *fnode)
> +{
> +	char name[80];
> +	size_t namelen;
> +	nid_t ino = le32_to_cpu(fnode->footer.ino);
> +	struct node_info ni;
> +	int ret;
> +
> +	namelen = snprintf(name, 80, "%u", ino);
> +	if (namelen >= 80)
> +		/* ignore terminating '\0', should never happen */
> +		namelen = 79;
> +
> +	if (f2fs_lookup(sbi, lpf, (u8 *) name, strlen(LPF))) {
> +		ASSERT_MSG("Name %s already exist in lost+found", name);
> +		return -EEXIST;
> +	}
> +
> +	get_node_info(sbi, le32_to_cpu(lpf->footer.ino), &ni);
> +	ret = f2fs_add_link(sbi, lpf, (unsigned char *)name, namelen,
> +			    ino, F2FS_FT_REG_FILE, ni.blk_addr, 0);
> +	if (ret) {
> +		ASSERT_MSG("Failed to add inode [0x%x] to lost+found", ino);
> +		return -EINVAL;
> +	}
> +
> +	/* update fnode */
> +	memcpy(fnode->i.i_name, name, namelen);
> +	fnode->i.i_namelen = cpu_to_le32(namelen);
> +	fnode->i.i_pino = c.lpf_ino;
> +	get_node_info(sbi, le32_to_cpu(fnode->footer.ino), &ni);

We don't need update those stuff, right? As all those fields are just related
to recovery.

> +	ret = dev_write_block(fnode, ni.blk_addr);
> +	ASSERT(ret >= 0);
> +
> +	DBG(1, "Reconnect inode [0x%x] to lost+found\n", ino);
> +	return 0;
> +}
> +
> +static void fsck_failed_reconnect_file_dnode(struct f2fs_sb_info *sbi,
> +					     nid_t nid)
> +{
> +	struct f2fs_fsck *fsck = F2FS_FSCK(sbi);
> +	struct f2fs_node *node;
> +	struct node_info ni;
> +	u32 addr;
> +	int i, err;
> +
> +	node = calloc(F2FS_BLKSIZE, 1);
> +	ASSERT(node);
> +
> +	get_node_info(sbi, nid, &ni);
> +	err = dev_read_block(node, ni.blk_addr);
> +	ASSERT(err >= 0);
> +
> +	fsck->chk.valid_node_cnt--;
> +	fsck->chk.valid_blk_cnt--;
> +	f2fs_clear_main_bitmap(sbi, ni.blk_addr);
> +
> +	for (i = 0; i < ADDRS_PER_BLOCK; i++) {
> +		addr = le32_to_cpu(node->dn.addr[i]);
> +		if (!addr)
> +			continue;
> +		fsck->chk.valid_blk_cnt--;
> +		if (addr == NEW_ADDR)
> +			continue;
> +		f2fs_clear_main_bitmap(sbi, addr);
> +	}

free(node);

> +}
> +
> +static void fsck_failed_reconnect_file_idnode(struct f2fs_sb_info *sbi,
> +					      nid_t nid)
> +{
> +	struct f2fs_fsck *fsck = F2FS_FSCK(sbi);
> +	struct f2fs_node *node;
> +	struct node_info ni;
> +	nid_t tmp;
> +	int i, err;
> +
> +	node = calloc(F2FS_BLKSIZE, 1);
> +	ASSERT(node);
> +
> +	get_node_info(sbi, nid, &ni);
> +	err = dev_read_block(node, ni.blk_addr);
> +	ASSERT(err >= 0);
> +
> +	fsck->chk.valid_node_cnt--;
> +	fsck->chk.valid_blk_cnt--;
> +	f2fs_clear_main_bitmap(sbi, ni.blk_addr);
> +
> +	for (i = 0; i < NIDS_PER_BLOCK; i++) {
> +		tmp = le32_to_cpu(node->in.nid[i]);
> +		if (!tmp)
> +			continue;
> +		fsck_failed_reconnect_file_dnode(sbi, tmp);
> +	}

free(node);

> +}
> +
> +static void fsck_failed_reconnect_file_didnode(struct f2fs_sb_info *sbi,
> +					       nid_t nid)
> +{
> +	struct f2fs_fsck *fsck = F2FS_FSCK(sbi);
> +	struct f2fs_node *node;
> +	struct node_info ni;
> +	nid_t tmp;
> +	int i, err;
> +
> +	node = calloc(F2FS_BLKSIZE, 1);
> +	ASSERT(node);
> +
> +	get_node_info(sbi, nid, &ni);
> +	err = dev_read_block(node, ni.blk_addr);
> +	ASSERT(err >= 0);
> +
> +	fsck->chk.valid_node_cnt--;
> +	fsck->chk.valid_blk_cnt--;
> +	f2fs_clear_main_bitmap(sbi, ni.blk_addr);
> +
> +	for (i = 0; i < NIDS_PER_BLOCK; i++) {
> +		tmp = le32_to_cpu(node->in.nid[i]);
> +		if (!tmp)
> +			continue;
> +		fsck_failed_reconnect_file_idnode(sbi, tmp);
> +	}

free(node);

> +}
> +
> +/*
> + * Counters and main_area_bitmap are already changed during checking
> + * inode block, so clear them. There is no need to clear new blocks
> + * allocted to lost+found.
> + */
> +static void fsck_failed_reconnect_file(struct f2fs_sb_info *sbi, nid_t ino)
> +{
> +	struct f2fs_fsck *fsck = F2FS_FSCK(sbi);
> +	struct f2fs_node *node;
> +	struct node_info ni;
> +	nid_t nid;
> +	int ofs, i, err;
> +
> +	node = calloc(F2FS_BLKSIZE, 1);
> +	ASSERT(node);
> +
> +	get_node_info(sbi, ino, &ni);
> +	err = dev_read_block(node, ni.blk_addr);
> +	ASSERT(err >= 0);
> +
> +	/* clear inode counters */
> +	fsck->chk.valid_inode_cnt--;
> +	fsck->chk.valid_node_cnt--;
> +	fsck->chk.valid_blk_cnt--;
> +	f2fs_clear_main_bitmap(sbi, ni.blk_addr);

free(node);

> +
> +	/* clear xnid counters */
> +	if (node->i.i_xattr_nid) {
> +		nid = le32_to_cpu(node->i.i_xattr_nid);
> +		fsck->chk.valid_node_cnt--;
> +		fsck->chk.valid_blk_cnt--;
> +		get_node_info(sbi, nid, &ni);
> +		f2fs_clear_main_bitmap(sbi, ni.blk_addr);
> +	}
> +
> +	/* clear data counters */
> +	if(!(node->i.i_inline & F2FS_INLINE_DATA)) {
> +		ofs = get_extra_isize(node);
> +		for (i = 0; i < ADDRS_PER_INODE(&node->i); i++) {
> +			block_t addr = le32_to_cpu(node->i.i_addr[ofs + i]);
> +			if (!addr)
> +				continue;
> +			fsck->chk.valid_blk_cnt--;
> +			if (addr == NEW_ADDR)
> +				continue;
> +			f2fs_clear_main_bitmap(sbi, addr);
> +		}
> +	}
> +
> +	for (i = 0; i < 5; i++) {
> +		nid = le32_to_cpu(node->i.i_nid[i]);
> +		if (!nid)
> +			continue;
> +
> +		switch (i) {
> +		case 0: /* direct node */
> +		case 1:
> +			fsck_failed_reconnect_file_dnode(sbi, nid);
> +			break;
> +		case 2: /* indirect node */
> +		case 3:
> +			fsck_failed_reconnect_file_idnode(sbi, nid);
> +			break;
> +		case 4: /* double indirect node */
> +			fsck_failed_reconnect_file_didnode(sbi, nid);
> +			break;
> +		}
> +	}
> +}
> +
> +/*
> + * Scan unreachable nids and find only regular file inodes. If these files
> + * are not corrupted, reconnect them to lost+found.
> + *
> + * Since all unreachable nodes are already checked, we can allocate new
> + * blocks safely.
> + *
> + * This function returns the number of files been reconnected.
> + */
> +static int fsck_reconnect_file(struct f2fs_sb_info *sbi)
> +{
> +	struct f2fs_fsck *fsck = F2FS_FSCK(sbi);
> +	struct f2fs_node *lpf_node, *node;
> +	struct node_info ni;
> +	char *reconnect_bitmap;
> +	u32 blk_cnt;
> +	nid_t nid;
> +	int err, cnt = 0;
> +
> +	node = calloc(F2FS_BLKSIZE, 1);
> +	ASSERT(node);
> +
> +	reconnect_bitmap = calloc(fsck->nat_area_bitmap_sz, 1);
> +	ASSERT(reconnect_bitmap);
> +
> +	for (nid = 0; nid < fsck->nr_nat_entries; nid++) {
> +		if (f2fs_test_bit(nid, fsck->nat_area_bitmap)) {
> +			if (is_qf_ino(F2FS_RAW_SUPER(sbi), nid)) {
> +				DBG(1, "Not support quota inode [0x%x]\n",
> +				    nid);
> +				continue;
> +			}
> +
> +			get_node_info(sbi, nid, &ni);
> +			err = dev_read_block(node, ni.blk_addr);
> +			ASSERT(err >= 0);
> +
> +			/* reconnection will restore these nodes if needed */
> +			if (node->footer.ino != node->footer.nid) {
> +				DBG(1, "Not support non-inode node [0x%x]\n",
> +				    nid);
> +				continue;
> +			}
> +
> +			if (S_ISDIR(le16_to_cpu(node->i.i_mode))) {
> +				DBG(1, "Not support directory inode [0x%x]\n",
> +				    nid);
> +				continue;
> +			}
> +
> +			if (sanity_check_nid(sbi, nid, node,
> +					     F2FS_FT_REG_FILE, TYPE_INODE,

As we have lost dirent which contains inode type, so we have no way to check
file type in between f2fs_dir_entry.file_type and inode.i_mode in
sanity_check_nid, here how about just passing type converted from inode.i_mode?

Need to check all places using F2FS_FT_REG_FILE.

Thanks,

> +					     &ni)) {
> +				ASSERT_MSG("Invalid nid [0x%x]\n", nid);
> +				continue;
> +			}
> +
> +			DBG(1, "Check inode 0x%x\n", nid);
> +			blk_cnt = 1;
> +			fsck_chk_inode_blk(sbi, nid, F2FS_FT_REG_FILE,
> +					   node, &blk_cnt, &ni, NULL);
> +
> +			f2fs_set_bit(nid, reconnect_bitmap);
> +		}
> +	}
> +
> +	lpf_node = fsck_get_lpf(sbi);
> +	if (!lpf_node)
> +		goto out;
> +
> +	for (nid = 0; nid < fsck->nr_nat_entries; nid++) {
> +		if (f2fs_test_bit(nid, reconnect_bitmap)) {
> +			get_node_info(sbi, nid, &ni);
> +			err = dev_read_block(node, ni.blk_addr);
> +			ASSERT(err >= 0);
> +
> +			if (fsck_do_reconnect_file(sbi, lpf_node, node)) {
> +				DBG(1, "Failed to reconnect inode [0x%x]\n",
> +				    nid);
> +				fsck_failed_reconnect_file(sbi, nid);
> +				continue;
> +			}
> +
> +			/* FIXME: need update quota? */
> +			quota_add_inode_usage(fsck->qctx, nid, &node->i);
> +
> +			DBG(1, "Reconnected inode [0x%x] to lost+found\n", nid);
> +			cnt++;
> +		}
> +	}
> +
> +out:
> +	free(node);
> +	free(lpf_node);
> +	free(reconnect_bitmap);
> +	return cnt;
> +}
> +
>  int fsck_verify(struct f2fs_sb_info *sbi)
>  {
>  	unsigned int i = 0;
> @@ -2059,6 +2426,11 @@ int fsck_verify(struct f2fs_sb_info *sbi)
>  		}
>  	}
>  
> +	if (nr_unref_nid && c.feature & cpu_to_le32(F2FS_FEATURE_LOST_FOUND)) {
> +		i = fsck_reconnect_file(sbi);
> +		printf("\n[FSCK] Reconnect %u files to lost+found\n", i);
> +	}
> +
>  	if (fsck->hard_link_list_head != NULL) {
>  		node = fsck->hard_link_list_head;
>  		while (node) {
> diff --git a/fsck/fsck.h b/fsck/fsck.h
> index 648c2db..8e133fa 100644
> --- a/fsck/fsck.h
> +++ b/fsck/fsck.h
> @@ -243,6 +243,9 @@ int f2fs_mkdir(struct f2fs_sb_info *, struct dentry *);
>  int f2fs_symlink(struct f2fs_sb_info *, struct dentry *);
>  int inode_set_selinux(struct f2fs_sb_info *, u32, const char *);
>  int f2fs_find_path(struct f2fs_sb_info *, char *, nid_t *);
> +nid_t f2fs_lookup(struct f2fs_sb_info *, struct f2fs_node *, u8 *, int);
> +int f2fs_add_link(struct f2fs_sb_info *, struct f2fs_node *,
> +		const unsigned char *, int, nid_t, int, block_t, int);
>  
>  /* xattr.c */
>  void *read_all_xattrs(struct f2fs_sb_info *, struct f2fs_node *);
> diff --git a/fsck/mount.c b/fsck/mount.c
> index df53c48..9775e9e 100644
> --- a/fsck/mount.c
> +++ b/fsck/mount.c
> @@ -1587,6 +1587,8 @@ void update_nat_blkaddr(struct f2fs_sb_info *sbi, nid_t ino,
>  	if (ino)
>  		nat_block->entries[entry_off].ino = cpu_to_le32(ino);
>  	nat_block->entries[entry_off].block_addr = cpu_to_le32(newaddr);
> +	if (c.func == FSCK)
> +		F2FS_FSCK(sbi)->entries[nid] = nat_block->entries[entry_off];
>  
>  	ret = dev_write_block(nat_block, block_addr);
>  	ASSERT(ret >= 0);
> 

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot

^ permalink raw reply	[flat|nested] 18+ messages in thread

end of thread, other threads:[~2018-02-13 14:22 UTC | newest]

Thread overview: 18+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-02-06  4:31 [RFC PATCH 0/5] f2fs-tools: introduce lost+found feature Sheng Yong
2018-02-06  4:31 ` [RFC PATCH 1/5] mkfs.f2fs: introduce mkfs parameters in f2fs_configuration Sheng Yong
2018-02-08 13:30   ` Chao Yu
2018-02-09  3:21     ` Sheng Yong
2018-02-09 12:59       ` Chao Yu
2018-02-06  4:31 ` [RFC PATCH 2/5] f2fs-tools: init f2fs_configuration as 0 Sheng Yong
2018-02-08 13:32   ` Chao Yu
2018-02-10  2:49     ` Jaegeuk Kim
2018-02-06  4:31 ` [RFC PATCH 3/5] fsck.f2fs: integrate sanity_check_inode to __check_inode_mode Sheng Yong
2018-02-08 13:44   ` Chao Yu
2018-02-06  4:31 ` [RFC PATCH 4/5] mkfs.f2fs: create lost+found directory Sheng Yong
2018-02-08 15:08   ` Chao Yu
2018-02-09  3:21     ` Sheng Yong
2018-02-09 13:13       ` Chao Yu
2018-02-06  4:31 ` [RFC PATCH 5/5] fsck.f2fs: reconnect unreachable files to lost+found Sheng Yong
2018-02-07 10:01   ` Sheng Yong
2018-02-07 10:04     ` Sheng Yong
2018-02-13 14:22   ` Chao Yu

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.