[PATCH 0/3] do not use s_dirt in JFFS2

[PATCH 0/3] do not use s_dirt in JFFS2

[Artem Bityutskiy]

This patch-set makes JFFS2 file-system stop using the VFS '->write_supers()'
call-back because I plan to remove it once all users are gone.

The final goal is to get rid of the 'sync_supers()' kernel thread. This kernel
thread wakes up every 5 seconds (by default) and calls '->write_super()' for
all mounted file-systems. And the bad thing is that this is done even if all
the superblocks are clean. Moreover, some file-systems do not even need this
end they do not register the '->write_super()' method at all (e.g., btrfs).

So 'sync_supers()' most often just generates useless wake-ups and wastes power.
I am trying to make all file-systems independent of '->write_super()' and plan
to remove 'sync_supers()' and '->write_super' completely once there are no more
users.

The '->write_supers()' method is mostly used by baroque file-systems like hfs,
udf, etc. Modern file-systems like btrfs and xfs do not use it. This justifies
removing this stuff from VFS completely and make every FS self-manage own
superblock.

Note: in the past I was trying to upstream patches which optimized 'sync_super()',
but Al Viro wanted me to kill it completely instead, which I am trying to do
now, see http://lkml.org/lkml/2010/7/22/96

======
Overall status:

1. ext4: patches submitted, waiting for reply from Ted Ts'o:
   https://lkml.org/lkml/2012/4/2/111
   Ted keeps silence so far WRT the fate of this patch-set.
2. ext2: patches are in the ext2 tree maintained by Jan Kara:
   git://git.kernel.org/pub/scm/linux/kernel/git/jack/linux-fs.git for_next
3. Version 3 of FAT FS changes were sent to Andrew and Hirofumi:
   https://lkml.org/lkml/2012/5/4/183
4. JFFS2 patches are sent being sent now.

TODO: affs, exofs, hfs, hfsplus, reiserfs, sysv, udf, ufs
======

 fs/fat/fat.h             |    1 +
 fs/fat/fatent.c          |   22 +++++++++++++-----
 fs/fat/inode.c           |   54 ++++++++++++++++++++-------------------------
 include/linux/msdos_fs.h |    3 +-
 4 files changed, 43 insertions(+), 37 deletions(-)

Thanks,
Artem.

[PATCH 1/2] jffs2: introduce fake wbuf inode

[Artem Bityutskiy]

From: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>

This patch is just a preparation for further changes. It introduces a special
inode ('wbuf_inode') in JFFS2 file-system which we'll later use for delayed
write-out of the write-buffer.

We do not allocate JFFS2 data structures like 'struct jffs2_inode_info' for
this inode because we won't need them. We allocate the next available inode
number for 'wbuf_inode' after the JFFS2 media is scanned. Of course, wbuf_inode
won't ever be written to the flash media.

Signed-off-by: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>
---
 fs/jffs2/fs.c          |   13 ++++++++++---
 fs/jffs2/jffs2_fs_sb.h |    1 +
 fs/jffs2/os-linux.h    |    4 ++++
 fs/jffs2/super.c       |    1 +
 fs/jffs2/wbuf.c        |   24 ++++++++++++++++++++++++
 5 files changed, 40 insertions(+), 3 deletions(-)

diff --git a/fs/jffs2/fs.c b/fs/jffs2/fs.c
index bb6f993..94016a9 100644
--- a/fs/jffs2/fs.c
+++ b/fs/jffs2/fs.c
@@ -241,7 +241,8 @@ void jffs2_evict_inode (struct inode *inode)
 		  __func__, inode->i_ino, inode->i_mode);
 	truncate_inode_pages(&inode->i_data, 0);
 	end_writeback(inode);
-	jffs2_do_clear_inode(c, f);
+	if (inode != c->wbuf_inode)
+		jffs2_do_clear_inode(c, f);
 }
 
 struct inode *jffs2_iget(struct super_block *sb, unsigned long ino)
@@ -562,12 +563,16 @@ int jffs2_do_fill_super(struct super_block *sb, void *data, int silent)
 	if ((ret = jffs2_do_mount_fs(c)))
 		goto out_inohash;
 
+	ret = jffs2_setup_wbuf_inode(c);
+	if (ret)
+		goto out_root;
+
 	jffs2_dbg(1, "%s(): Getting root inode\n", __func__);
 	root_i = jffs2_iget(sb, 1);
 	if (IS_ERR(root_i)) {
 		jffs2_dbg(1, "get root inode failed\n");
 		ret = PTR_ERR(root_i);
-		goto out_root;
+		goto out_wbuf_inode;
 	}
 
 	ret = -ENOMEM;
@@ -575,7 +580,7 @@ int jffs2_do_fill_super(struct super_block *sb, void *data, int silent)
 	jffs2_dbg(1, "%s(): d_make_root()\n", __func__);
 	sb->s_root = d_make_root(root_i);
 	if (!sb->s_root)
-		goto out_root;
+		goto out_wbuf_inode;
 
 	sb->s_maxbytes = 0xFFFFFFFF;
 	sb->s_blocksize = PAGE_CACHE_SIZE;
@@ -585,6 +590,8 @@ int jffs2_do_fill_super(struct super_block *sb, void *data, int silent)
 		jffs2_start_garbage_collect_thread(c);
 	return 0;
 
+out_wbuf_inode:
+	jffs2_free_wbuf_inode(c);
 out_root:
 	jffs2_free_ino_caches(c);
 	jffs2_free_raw_node_refs(c);
diff --git a/fs/jffs2/jffs2_fs_sb.h b/fs/jffs2/jffs2_fs_sb.h
index 55a0c1d..11d43f2 100644
--- a/fs/jffs2/jffs2_fs_sb.h
+++ b/fs/jffs2/jffs2_fs_sb.h
@@ -125,6 +125,7 @@ struct jffs2_sb_info {
 	uint32_t wbuf_len;
 	struct jffs2_inodirty *wbuf_inodes;
 	struct rw_semaphore wbuf_sem;	/* Protects the write buffer */
+	struct inode *wbuf_inode; /* Fake inode for delayed wbuf write-out */
 
 	unsigned char *oobbuf;
 	int oobavail; /* How many bytes are available for JFFS2 in OOB */
diff --git a/fs/jffs2/os-linux.h b/fs/jffs2/os-linux.h
index 1cd3aec..6f28cc5 100644
--- a/fs/jffs2/os-linux.h
+++ b/fs/jffs2/os-linux.h
@@ -82,6 +82,8 @@ static inline void jffs2_init_inode_info(struct jffs2_inode_info *f)
 #define jffs2_write_nand_badblock(c,jeb,bad_offset) (1)
 #define jffs2_nand_flash_setup(c) (0)
 #define jffs2_nand_flash_cleanup(c) do {} while(0)
+#define jffs2_setup_wbuf_inode(c) 0
+#define jffs2_free_wbuf_inode(c) do {} while(0)
 #define jffs2_wbuf_dirty(c) (0)
 #define jffs2_flash_writev(a,b,c,d,e,f) jffs2_flash_direct_writev(a,b,c,d,e)
 #define jffs2_wbuf_timeout NULL
@@ -124,6 +126,8 @@ int jffs2_flush_wbuf_gc(struct jffs2_sb_info *c, uint32_t ino);
 int jffs2_flush_wbuf_pad(struct jffs2_sb_info *c);
 int jffs2_nand_flash_setup(struct jffs2_sb_info *c);
 void jffs2_nand_flash_cleanup(struct jffs2_sb_info *c);
+int jffs2_setup_wbuf_inode(struct jffs2_sb_info *c);
+void jffs2_free_wbuf_inode(struct jffs2_sb_info *c);
 
 #define jffs2_dataflash(c) (c->mtd->type == MTD_DATAFLASH)
 int jffs2_dataflash_setup(struct jffs2_sb_info *c);
diff --git a/fs/jffs2/super.c b/fs/jffs2/super.c
index f9916f3..a180409 100644
--- a/fs/jffs2/super.c
+++ b/fs/jffs2/super.c
@@ -326,6 +326,7 @@ static void jffs2_put_super (struct super_block *sb)
 	jffs2_flush_wbuf_pad(c);
 	mutex_unlock(&c->alloc_sem);
 
+	jffs2_free_wbuf_inode(c);
 	jffs2_sum_exit(c);
 
 	jffs2_free_ino_caches(c);
diff --git a/fs/jffs2/wbuf.c b/fs/jffs2/wbuf.c
index 74d9be1..65373cf 100644
--- a/fs/jffs2/wbuf.c
+++ b/fs/jffs2/wbuf.c
@@ -1315,3 +1315,27 @@ int jffs2_ubivol_setup(struct jffs2_sb_info *c) {
 void jffs2_ubivol_cleanup(struct jffs2_sb_info *c) {
 	kfree(c->wbuf);
 }
+
+/*
+ * Allocate a fake inode and which is used for delayed write-out of the
+ * write-buffer: we mark it as dirty when the write-buffer is not empty we will
+ * synchronize the write-buffer in 'jffs2_write_inode'.
+ */
+int jffs2_setup_wbuf_inode(struct jffs2_sb_info *c)
+{
+	struct super_block *sb = OFNI_BS_2SFFJ(c);
+
+	c->wbuf_inode = new_inode(sb);
+	if (!c->wbuf_inode)
+		return -ENOMEM;
+
+	c->wbuf_inode->i_ino = ++c->highest_ino;
+	insert_inode_hash(c->wbuf_inode);
+
+	return 0;
+}
+
+void jffs2_free_wbuf_inode(struct jffs2_sb_info *c)
+{
+	iput(c->wbuf_inode);
+}
-- 
1.7.7.6

[PATCH 2/2] jffs2: get rid of jffs2_sync_super

[Artem Bityutskiy]

From: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>

Currently JFFS2 file-system maps the VFS "superblock" abstraction to the
write-buffer. Namely, it uses VFS services to synchronize the write-buffer
periodically.

The whole "superblock write-out" VFS infrastructure is served by the
'sync_supers()' kernel thread, which wakes up every 5 (by default) seconds and
writes out all dirty superblock using the '->write_super()' call-back. But the
problem with this thread is that it wastes power by waking up the system every
5 seconds no matter what. So we want to kill it completely and thus, we need to
make file-systems to stop using the '->write_super' VFS service, and then
remove it together with the kernel thread.

This patch switches the JFFS2 write-buffer management from
'->write_super()'/'->s_dirt' to 'wbuf_inode'/'->write_inode'. Instead of
setting the 's_dirt' flag, we just mark the special 'wbuf_inode' inode as
dirty and let VFS invoke the '->write_inode' call-back when needed, where we
synchronize the write-buffer.

Signed-off-by: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>
---
 fs/jffs2/fs.c       |   18 ++++++++++++++++++
 fs/jffs2/os-linux.h |    3 ++-
 fs/jffs2/super.c    |   22 +---------------------
 3 files changed, 21 insertions(+), 22 deletions(-)

diff --git a/fs/jffs2/fs.c b/fs/jffs2/fs.c
index 94016a9..c6aacbb 100644
--- a/fs/jffs2/fs.c
+++ b/fs/jffs2/fs.c
@@ -228,6 +228,24 @@ int jffs2_statfs(struct dentry *dentry, struct kstatfs *buf)
 	return 0;
 }
 
+int jffs2_write_inode(struct inode *inode, struct writeback_control *wbc)
+{
+	struct jffs2_sb_info *c = JFFS2_SB_INFO(inode->i_sb);
+
+	/*
+	 * JFFS2 is synchronous file-system, and the only the fake write-buffer
+	 * inode is allowed to be writen-out asynchronously - we use this to do
+	 * delayed write-buffer synchronization.
+	 */
+	BUG_ON(inode != c->wbuf_inode);
+
+	if (!(inode->i_sb->s_flags & MS_RDONLY)) {
+		jffs2_dbg(1, "%s()\n", __func__);
+		jffs2_flush_wbuf_gc(c, 0);
+	}
+
+	return 0;
+}
 
 void jffs2_evict_inode (struct inode *inode)
 {
diff --git a/fs/jffs2/os-linux.h b/fs/jffs2/os-linux.h
index 6f28cc5..ab97e88 100644
--- a/fs/jffs2/os-linux.h
+++ b/fs/jffs2/os-linux.h
@@ -144,7 +144,7 @@ void jffs2_nor_wbuf_flash_cleanup(struct jffs2_sb_info *c);
 
 static inline void jffs2_dirty_trigger(struct jffs2_sb_info *c)
 {
-	OFNI_BS_2SFFJ(c)->s_dirt = 1;
+	__mark_inode_dirty(c->wbuf_inode, I_DIRTY_SYNC);
 }
 
 /* background.c */
@@ -173,6 +173,7 @@ extern const struct inode_operations jffs2_symlink_inode_operations;
 int jffs2_setattr (struct dentry *, struct iattr *);
 int jffs2_do_setattr (struct inode *, struct iattr *);
 struct inode *jffs2_iget(struct super_block *, unsigned long);
+int jffs2_write_inode(struct inode *inode, struct writeback_control *wbc);
 void jffs2_evict_inode (struct inode *);
 void jffs2_dirty_inode(struct inode *inode, int flags);
 struct inode *jffs2_new_inode (struct inode *dir_i, umode_t mode,
diff --git a/fs/jffs2/super.c b/fs/jffs2/super.c
index a180409..0b4e632 100644
--- a/fs/jffs2/super.c
+++ b/fs/jffs2/super.c
@@ -63,21 +63,6 @@ static void jffs2_i_init_once(void *foo)
 	inode_init_once(&f->vfs_inode);
 }
 
-static void jffs2_write_super(struct super_block *sb)
-{
-	struct jffs2_sb_info *c = JFFS2_SB_INFO(sb);
-
-	lock_super(sb);
-	sb->s_dirt = 0;
-
-	if (!(sb->s_flags & MS_RDONLY)) {
-		jffs2_dbg(1, "%s()\n", __func__);
-		jffs2_flush_wbuf_gc(c, 0);
-	}
-
-	unlock_super(sb);
-}
-
 static const char *jffs2_compr_name(unsigned int compr)
 {
 	switch (compr) {
@@ -113,8 +98,6 @@ static int jffs2_sync_fs(struct super_block *sb, int wait)
 {
 	struct jffs2_sb_info *c = JFFS2_SB_INFO(sb);
 
-	jffs2_write_super(sb);
-
 	mutex_lock(&c->alloc_sem);
 	jffs2_flush_wbuf_pad(c);
 	mutex_unlock(&c->alloc_sem);
@@ -249,9 +232,9 @@ static int jffs2_remount_fs(struct super_block *sb, int *flags, char *data)
 static const struct super_operations jffs2_super_operations =
 {
 	.alloc_inode =	jffs2_alloc_inode,
+	.write_inode =  jffs2_write_inode,
 	.destroy_inode =jffs2_destroy_inode,
 	.put_super =	jffs2_put_super,
-	.write_super =	jffs2_write_super,
 	.statfs =	jffs2_statfs,
 	.remount_fs =	jffs2_remount_fs,
 	.evict_inode =	jffs2_evict_inode,
@@ -319,9 +302,6 @@ static void jffs2_put_super (struct super_block *sb)
 
 	jffs2_dbg(2, "%s()\n", __func__);
 
-	if (sb->s_dirt)
-		jffs2_write_super(sb);
-
 	mutex_lock(&c->alloc_sem);
 	jffs2_flush_wbuf_pad(c);
 	mutex_unlock(&c->alloc_sem);
-- 
1.7.7.6

Re: [PATCH 0/3] do not use s_dirt in JFFS2

[Artem Bityutskiy]

[-- Attachment #1: Type: text/plain, Size: 320 bytes --]

On Fri, 2012-05-04 at 20:27 +0300, Artem Bityutskiy wrote:
> This patch-set makes JFFS2 file-system stop using the VFS '->write_supers()'
> call-back because I plan to remove it once all users are gone.

I'd really appreciate if someone on the MTD list tested these patches.

-- 
Best Regards,
Artem Bityutskiy

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [PATCH 0/3] do not use s_dirt in JFFS2

[Brian Norris]

On Fri, May 4, 2012 at 10:41 AM, Artem Bityutskiy <dedekind1@gmail.com> wrote:
> On Fri, 2012-05-04 at 20:27 +0300, Artem Bityutskiy wrote:
>> This patch-set makes JFFS2 file-system stop using the VFS '->write_supers()'
>> call-back because I plan to remove it once all users are gone.
>
> I'd really appreciate if someone on the MTD list tested these patches.

Am I crazy, or did you send the cover letter without the patches?

Brian

Re: [PATCH 0/3] do not use s_dirt in JFFS2

[Brian Norris]

On Fri, May 4, 2012 at 5:55 PM, Brian Norris
<computersforpeace@gmail.com> wrote:
> On Fri, May 4, 2012 at 10:41 AM, Artem Bityutskiy <dedekind1@gmail.com> wrote:
>> On Fri, 2012-05-04 at 20:27 +0300, Artem Bityutskiy wrote:
>>> This patch-set makes JFFS2 file-system stop using the VFS '->write_supers()'
>>> call-back because I plan to remove it once all users are gone.
>>
>> I'd really appreciate if someone on the MTD list tested these patches.
>
> Am I crazy, or did you send the cover letter without the patches?

Sorry, I am at least a little crazy: I actually see two patches
(labelled 1/2 and 2/2) while the cover is labelled 0/3, with a
diffstat from FAT changes...

Brian

Re: [PATCH 0/3] do not use s_dirt in JFFS2

[Artem Bityutskiy]

[-- Attachment #1: Type: text/plain, Size: 677 bytes --]

On Fri, 2012-05-04 at 17:55 -0700, Brian Norris wrote:
> On Fri, May 4, 2012 at 10:41 AM, Artem Bityutskiy <dedekind1@gmail.com> wrote:
> > On Fri, 2012-05-04 at 20:27 +0300, Artem Bityutskiy wrote:
> >> This patch-set makes JFFS2 file-system stop using the VFS '->write_supers()'
> >> call-back because I plan to remove it once all users are gone.
> >
> > I'd really appreciate if someone on the MTD list tested these patches.
> 
> Am I crazy, or did you send the cover letter without the patches?

Yeah, sorry, I copied the cover letter from FAT FS patches and did not
clean-it up well enough. This set has only 2 patches.

-- 
Best Regards,
Artem Bityutskiy

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 836 bytes --]