[Cluster-devel] [PATCH 00/56] fsck.gfs2: Support for checking gfs1 file systems

[Cluster-devel] [PATCH 00/56] fsck.gfs2: Support for checking gfs1 file systems

[Bob Peterson]

Hi,

I previously posted a series of 44 patches to fsck.gfs2 that
fix numerous bugs and allow fsck.gfs2 to operate on GFS1 file
systems.  Since I posted those patches, I received feedback
and changed the patches accordingly.  I've also done extensive
testing and discovered and fixed several more bugs.
There are now 56 patches in all.  As Andy Price suggested,
I renamed their descriptions to make them shorter so
they're not cut off.

My latest version has passed all my tests and I'm ready to
start pushing them to the upstream gfs2-utils git repository.
But before I do, I wanted to post the corrected patches and
the new ones here.  I'm not reposting the patches that got
ACKed.  Here is the complete list of patches:

 1/56 fsck.gfs2: Make functions use sdp rather than sbp
 2/56 fsck.gfs2: Change "if(" to "if ("
 3/56 libgfs1: Add gfs1 variable to superblock structure
 4/56 libgfs2: Make check_sb and read_sb operate on gfs1
 5/56 libgfs2: move gfs1 structures to libgfs2
 6/56 fsck.gfs2: Check for blocks wrongly inside resource groups
 7/56 fsck.gfs2: Rename check_leaf to check_ealeaf_block
 8/56 fsck.gfs2: fsck.gfs2: Delete vestigial buffer_head in check_leaf
 9/56 fsck.gfs2: fsck.gfs2: Rename nlink functions to be intuitive
10/56 fsck.gfs2: fsck.gfs2: Sync di_nlink adding links for lost+found

11/56 fsck.gfs2: fsck.gfs2: Make dir entry count 32 bits
12/56 fsck.gfs2: get rid of triple negative logic
13/56 dirent_repair needs to mark the buffer as modified
14/56 fsck.gfs2: fsck.gfs2: Ask to reclaim unlinked meta per-rgrp only
15/56 fsck.gfs2: fsck.gfs2: Refactor add_dotdot function in lost+found
16/56 libgfs2: libgfs2: Use __FUNCTION__ rather than __FILE__
17/56 fsck.gfs2: fsck.gfs2: Don't stop invalidating blocks on invalid
18/56 fsck.gfs2: fsck.gfs2: Find and clear duplicate leaf blocks refs
19/56 fsck.gfs2: fsck.gfs2: Move check_num_ptrs from metawalk to pass1
20/56 fsck.gfs2: fsck.gfs2: Duplicate ref processing for leaf blocks

21/56 fsck.gfs2: fsck.gfs2: split check_leaf_blks to be more readable
22/56 fsck.gfs2: Shorten output
23/56 fsck.gfs2: Make output messages more sensible
24/56 fsck.gfs pass2: Refactor function set_dotdor_dir
25/56 fsck.gfs2 pass2: Delete extended attributes with inode
26/56 fsck.gfs2 pass2: Don't delete invalid inode metadata
27/56 fsck.gfs2 pass3: Refactor mark_and_return_parent
28/56 fsck.gfs2: misc cosmetic changes
29/56 fsck.gfs2: Don't use old_leaf if it was a duplicate
30/56 fsck.gfs2: Add find_remove_dup, free_block_if_notdup

31/56 fsck.gfs2: don't free prev rgrp list repairing rgrps
32/56 libgfs2: eliminate gfs1_readi in favor of gfs2_readi
33/56 libgfs2: Mark buffer modified adding a new GFS1 block
34/56 libgfs2: Use dinode buffer to map gfs1 dinode blocks
35/56 libgfs2: move block_map functions to fsck.gfs2
36/56 libgfs2: eliminate gfs1_rindex_read
37/56 libgfs2: combine ri_update and gfs1_ri_update
38/56 libgfs2: combine gfs_inode_read and gfs_inode_get
39/56 libgfs2: move gfs1 functions from edit to libgfs2
40/56 gfs2_edit savemeta: save_inode_data backward for gfs1

41/56 libgfs2: expand capabilities to operate on gfs1
42/56 fsck.gfs2: Combine block and char device inode types
43/56 fsck.gfs2: four-step duplicate elimination process
44/56 fsck.gfs2: Add ability to check gfs1 file systems
45/56 fsck.gfs2: Remove bad inodes from duplicate tree
46/56 fsck.gfs2: Handle duplicate reference to dinode blocks
47/56 fsck.gfs2: Bad extended attributes not deleted  properly
48/56 libgfs2: Make rebuild functions not re-read ip
49/56 fsck.gfs2: Shorten debug output
50/56 fsck.gfs2: Increment link count reporting wrong dinode

51/56 fsck.gfs2: system dinodes take priority over user dinodes
52/56 fsck.gfs2: Recognize partially gfs2-converted dinodes
53/56 fsck.gfs2: Print step 2 duplicate debug msg first
54/56 fsck.gfs2: pass1c counts percentage backward
55/56 fsck.gfs2: Speed up rangecheck functions
56/56 libgfs2: Make in-core rgrps use rbtree

In addition, I just put my experimental new fsck.gfs2,
which contains all these patches, on my people page at this
location:

http://people.redhat.com/rpeterso/Experimental/RHEL6.x/fsck.gfs2

It is experimental, so make backups and try it AT YOUR OWN RISK.
However, it is also better tested than all of its predecessors.

Regards,

Bob Peterson
Red Hat File Systems

[Cluster-devel] [PATCH 00/56] fsck.gfs2: Support for checking gfs1 file systems

[Andrew Price]

Hi Bob,

On 25/08/11 17:49, Bob Peterson wrote:
> 32/56 libgfs2: eliminate gfs1_readi in favor of gfs2_readi

Could you send the latest version of patch 32? Using the original one 
I'm getting:

Applying: libgfs2: eliminate gfs1_readi in favor of gfs2_readi
error: patch failed: gfs2/edit/extended.c:516
error: gfs2/edit/extended.c: patch does not apply
error: patch failed: gfs2/libgfs2/super.c:169
error: gfs2/libgfs2/super.c: patch does not apply

Cheers,

Andy

[Cluster-devel] [PATCH 00/56] fsck.gfs2: Support for checking gfs1 file systems

[Bob Peterson]

----- Original Message -----
| Hi Bob,
| 
| On 25/08/11 17:49, Bob Peterson wrote:
| > 32/56 libgfs2: eliminate gfs1_readi in favor of gfs2_readi
| 
| Could you send the latest version of patch 32? Using the original one
| I'm getting:
| 
| Applying: libgfs2: eliminate gfs1_readi in favor of gfs2_readi
| error: patch failed: gfs2/edit/extended.c:516
| error: gfs2/edit/extended.c: patch does not apply
| error: patch failed: gfs2/libgfs2/super.c:169
| error: gfs2/libgfs2/super.c: patch does not apply
| 
| Cheers,
| 
| Andy

Hi Andy,

Here it is:

Bob
--
From 03f0ed42c95b10f02bfe52ae08cb7c18402ce662 Mon Sep 17 00:00:00 2001
From: Bob Peterson <rpeterso@redhat.com>
Date: Wed, 10 Aug 2011 10:21:00 -0500
Subject: [PATCH 32/56] libgfs2: eliminate gfs1_readi in favor of gfs2_readi

This patch eliminates function gfs1_readi because it's nearly
identical to gfs2_readi.  The gfs1-specific bits have been made
generic in gfs2_readi based on the sdp->gfs1 setting.

rhbz#675723
---
 gfs2/edit/extended.c   |    8 +----
 gfs2/libgfs2/fs_ops.c  |   21 ++++++++++----
 gfs2/libgfs2/gfs1.c    |   68 +-----------------------------------------------
 gfs2/libgfs2/libgfs2.h |    2 -
 gfs2/libgfs2/super.c   |    5 ---
 5 files changed, 18 insertions(+), 86 deletions(-)

diff --git a/gfs2/edit/extended.c b/gfs2/edit/extended.c
index 792b724..071f589 100644
--- a/gfs2/edit/extended.c
+++ b/gfs2/edit/extended.c
@@ -516,12 +516,8 @@ static int parse_rindex(struct gfs2_inode *dip, int print_rindex)
 
 		roff = print_entry_ndx * sizeof(struct gfs2_rindex);
 
-		if (sbd.gfs1)
-			error = gfs1_readi(dip, (void *)&rbuf, roff,
-					   sizeof(struct gfs2_rindex));
-		else
-			error = gfs2_readi(dip, (void *)&rbuf, roff,
-					   sizeof(struct gfs2_rindex));
+		error = gfs2_readi(dip, (void *)&rbuf, roff,
+				   sizeof(struct gfs2_rindex));
 		if (!error) /* end of file */
 			break;
 		gfs2_rindex_in(&ri, rbuf);
diff --git a/gfs2/libgfs2/fs_ops.c b/gfs2/libgfs2/fs_ops.c
index 60ee467..4c8d31e 100644
--- a/gfs2/libgfs2/fs_ops.c
+++ b/gfs2/libgfs2/fs_ops.c
@@ -510,6 +510,7 @@ int gfs2_readi(struct gfs2_inode *ip, void *buf,
 	unsigned int amount;
 	int not_new = 0;
 	int isdir = !!(S_ISDIR(ip->i_di.di_mode));
+	int journaled = ip->i_di.di_flags & GFS2_DIF_JDATA;
 	int copied = 0;
 
 	if (offset >= ip->i_di.di_size)
@@ -521,7 +522,7 @@ int gfs2_readi(struct gfs2_inode *ip, void *buf,
 	if (!size)
 		return 0;
 
-	if (isdir) {
+	if ((sdp->gfs1 && journaled) || (!sdp->gfs1 && isdir)) {
 		lblock = offset;
 		o = lblock % sdp->sd_jbsize;
 		lblock /= sdp->sd_jbsize;
@@ -532,7 +533,7 @@ int gfs2_readi(struct gfs2_inode *ip, void *buf,
 
 	if (inode_is_stuffed(ip))
 		o += sizeof(struct gfs2_dinode);
-	else if (isdir)
+	else if ((sdp->gfs1 && journaled) || (!sdp->gfs1 && isdir))
 		o += sizeof(struct gfs2_meta_header);
 
 	while (copied < size) {
@@ -540,9 +541,14 @@ int gfs2_readi(struct gfs2_inode *ip, void *buf,
 		if (amount > sdp->bsize - o)
 			amount = sdp->bsize - o;
 
-		if (!extlen)
-			block_map(ip, lblock, &not_new, &dblock, &extlen,
-				  FALSE);
+		if (!extlen) {
+			if (sdp->gfs1)
+				gfs1_block_map(ip, lblock, &not_new, &dblock,
+					       &extlen, FALSE);
+			else
+				block_map(ip, lblock, &not_new, &dblock,
+					  &extlen, FALSE);
+		}
 
 		if (dblock) {
 			if (dblock == ip->i_di.di_num.no_addr)
@@ -561,7 +567,10 @@ int gfs2_readi(struct gfs2_inode *ip, void *buf,
 		copied += amount;
 		lblock++;
 
-		o = (isdir) ? sizeof(struct gfs2_meta_header) : 0;
+		if (sdp->gfs1)
+			o = (journaled) ? sizeof(struct gfs2_meta_header) : 0;
+		else
+			o = (isdir) ? sizeof(struct gfs2_meta_header) : 0;
 	}
 
 	return copied;
diff --git a/gfs2/libgfs2/gfs1.c b/gfs2/libgfs2/gfs1.c
index 66e00ff..dc29006 100644
--- a/gfs2/libgfs2/gfs1.c
+++ b/gfs2/libgfs2/gfs1.c
@@ -156,72 +156,6 @@ void gfs1_block_map(struct gfs2_inode *ip, uint64_t lblock, int *new,
 	free(mp);
 }
 
-int gfs1_readi(struct gfs2_inode *ip, void *bufin,
-	       uint64_t offset, unsigned int size)
-{
-	struct gfs2_sbd *sdp = ip->i_sbd;
-	struct gfs2_buffer_head *bh;
-	uint64_t lblock, dblock = 0;
-	uint32_t extlen = 0;
-	unsigned int amount;
-	int not_new = 0;
-	int journaled = fs_is_jdata(ip);
-	int copied = 0;
-	char *buf = bufin;
-
-	if (offset >= ip->i_di.di_size)
-		return 0;
-
-	if ((offset + size) > ip->i_di.di_size)
-		size = ip->i_di.di_size - offset;
-
-	if (!size)
-		return 0;
-
-	if (journaled) {
-		lblock = offset / sdp->sd_jbsize;
-		offset %= sdp->sd_jbsize;
-	} else {
-		lblock = offset >> sdp->sd_sb.sb_bsize_shift;
-		offset &= sdp->sd_sb.sb_bsize - 1;
-	}
-
-	if (!ip->i_di.di_height) /* stuffed */
-		offset += sizeof(struct gfs_dinode);
-	else if (journaled)
-		offset += sizeof(struct gfs2_meta_header);
-
-	while (copied < size) {
-		amount = size - copied;
-		if (amount > sdp->bsize - offset)
-			amount = sdp->bsize - offset;
-
-		if (!extlen)
-			gfs1_block_map(ip, lblock, &not_new, &dblock,
-				       &extlen, FALSE);
-
-		if (dblock) {
-			bh = bread(sdp, dblock);
-			dblock++;
-			extlen--;
-		} else
-			bh = NULL;
-
-
-		if (bh) {
-			memcpy(buf+copied, bh->b_data + offset, amount);
-			brelse(bh);
-		} else
-			memset(buf+copied, 0, amount);
-		copied += amount;
-		lblock++;
-
-		offset = (journaled) ? sizeof(struct gfs2_meta_header) : 0;
-	}
-
-	return copied;
-}
-
 /**
  * gfs1_rindex_read - read in the rg index file
  *                  Stolen from libgfs2/super.c, but modified to handle gfs1.
@@ -246,7 +180,7 @@ int gfs1_rindex_read(struct gfs2_sbd *sdp, int fd, int *count1)
 		if (fd > 0)
 			error = read(fd, &buf, sizeof(struct gfs2_rindex));
 		else
-			error = gfs1_readi(sdp->md.riinode, (char *)&buf,
+			error = gfs2_readi(sdp->md.riinode, (char *)&buf,
 					   (rg * sizeof(struct gfs2_rindex)),
 					   sizeof(struct gfs2_rindex));
 		if (!error)
diff --git a/gfs2/libgfs2/libgfs2.h b/gfs2/libgfs2/libgfs2.h
index f8f55b1..0ac5f3d 100644
--- a/gfs2/libgfs2/libgfs2.h
+++ b/gfs2/libgfs2/libgfs2.h
@@ -669,8 +669,6 @@ extern void gfs1_lookup_block(struct gfs2_inode *ip,
 			      int create, int *new, uint64_t *block);
 extern void gfs1_block_map(struct gfs2_inode *ip, uint64_t lblock, int *new,
 			   uint64_t *dblock, uint32_t *extlen, int prealloc);
-extern int gfs1_readi(struct gfs2_inode *ip, void *buf, uint64_t offset,
-		      unsigned int size);
 extern int gfs1_rindex_read(struct gfs2_sbd *sdp, int fd, int *count1);
 extern int gfs1_ri_update(struct gfs2_sbd *sdp, int fd, int *rgcount, int quiet);
 extern struct gfs2_inode *gfs_inode_get(struct gfs2_sbd *sdp,
diff --git a/gfs2/libgfs2/super.c b/gfs2/libgfs2/super.c
index 9e8d079..693e547 100644
--- a/gfs2/libgfs2/super.c
+++ b/gfs2/libgfs2/super.c
@@ -160,11 +160,6 @@ int rindex_read(struct gfs2_sbd *sdp, int fd, int *count1, int *sane)
 	for (rg = 0; ; rg++) {
 		if (fd > 0)
 			error = read(fd, &buf, sizeof(struct gfs2_rindex));
-		else if (sdp->gfs1)
-			error = gfs1_readi(sdp->md.riinode,
-					   (char *)&buf.bufgfs1,
-					   rg * sizeof(struct gfs2_rindex),
-					   sizeof(struct gfs2_rindex));
 		else
 			error = gfs2_readi(sdp->md.riinode,
 					   (char *)&buf.bufgfs2,
-- 
1.7.4.4