[PATCH 0/3] Remove ext3 filesystem driver

[PATCH 0/3] Remove ext3 filesystem driver

[Jan Kara]

  Hello,

  so I have created this patch set which removes ext3 driver (and some related support
code) from the kernel. See changelog of patch 2/3 for more details. If noone objects,
I will queue the series in my tree for the next merge window.

Jens, are you OK with me merging patch 3/3 or do you want me to send it to you once
the first two patches get merged?

Ted, this also touches ext4 driver so that it always handles ext3 filesystems so ack
from you would be nice.

									Honza

[PATCH 1/3] doc: Update doc about journalling layer

[Jan Kara]

Documentation of journalling layer in
Documentation/DocBook/filesystems.tmpl speaks about JBD layer. Since
that is going away, update the documentation to speak about JBD2. Also
update the parts that have changed since someone last touched the
document and remove some parts which are just misleading and outdated.

Signed-off-by: Jan Kara <jack@suse.com>
---
 Documentation/DocBook/filesystems.tmpl | 178 +++++++++++++--------------------
 1 file changed, 67 insertions(+), 111 deletions(-)

diff --git a/Documentation/DocBook/filesystems.tmpl b/Documentation/DocBook/filesystems.tmpl
index bcdfdb9a9277..6006b6358c86 100644
--- a/Documentation/DocBook/filesystems.tmpl
+++ b/Documentation/DocBook/filesystems.tmpl
@@ -146,36 +146,30 @@
 The journalling layer is  easy to use. You need to
 first of all create a journal_t data structure. There are
 two calls to do this dependent on how you decide to allocate the physical
-media on which the journal resides. The journal_init_inode() call
-is for journals stored in filesystem inodes, or the journal_init_dev()
-call can be use for journal stored on a raw device (in a continuous range
+media on which the journal resides. The jbd2_journal_init_inode() call
+is for journals stored in filesystem inodes, or the jbd2_journal_init_dev()
+call can be used for journal stored on a raw device (in a continuous range
 of blocks). A journal_t is a typedef for a struct pointer, so when
-you are finally finished make sure you call journal_destroy() on it
+you are finally finished make sure you call jbd2_journal_destroy() on it
 to free up any used kernel memory.
 </para>
 
 <para>
 Once you have got your journal_t object you need to 'mount' or load the journal
-file, unless of course you haven't initialised it yet - in which case you
-need to call journal_create().
+file. The journalling layer expects the space for the journal was already
+allocated and initialized properly by the userspace tools.  When loading the
+journal you must call jbd2_journal_load() to process journal contents.  If the
+client file system detects the journal contents does not need to be processed
+(or even need not have valid contents), it may call jbd2_journal_wipe() to
+clear the journal contents before calling jbd2_journal_load().
 </para>
 
 <para>
-Most of the time however your journal file will already have been created, but
-before you load it you must call journal_wipe() to empty the journal file.
-Hang on, you say , what if the filesystem wasn't cleanly umount()'d . Well, it is the
-job of the client file system to detect this and skip the call to journal_wipe().
-</para>
-
-<para>
-In either case the next call should be to journal_load() which prepares the
-journal file for use. Note that journal_wipe(..,0) calls journal_skip_recovery()
-for you if it detects any outstanding transactions in the journal and similarly
-journal_load() will call journal_recover() if necessary.
-I would advise reading fs/ext3/super.c for examples on this stage.
-[RGG: Why is the journal_wipe() call necessary - doesn't this needlessly
-complicate the API. Or isn't a good idea for the journal layer to hide
-dirty mounts from the client fs]
+Note that jbd2_journal_wipe(..,0) calls jbd2_journal_skip_recovery() for you if
+it detects any outstanding transactions in the journal and similarly
+jbd2_journal_load() will call jbd2_journal_recover() if necessary.  I would
+advise reading ext4_load_journal() in fs/ext4/super.c for examples on this
+stage.
 </para>
 
 <para>
@@ -189,41 +183,41 @@ You still need to actually journal your filesystem changes, this
 is done by wrapping them into transactions. Additionally you
 also need to wrap the modification of each of the buffers
 with calls to the journal layer, so it knows what the modifications
-you are actually making are. To do this use  journal_start() which
+you are actually making are. To do this use jbd2_journal_start() which
 returns a transaction handle.
 </para>
 
 <para>
-journal_start()
-and its counterpart journal_stop(), which indicates the end of a transaction
-are nestable calls, so you can reenter a transaction if necessary,
-but remember you must call journal_stop() the same number of times as
-journal_start() before the transaction is completed (or more accurately
-leaves the update phase). Ext3/VFS makes use of this feature to simplify
-quota support.
+jbd2_journal_start()
+and its counterpart jbd2_journal_stop(), which indicates the end of a
+transaction are nestable calls, so you can reenter a transaction if necessary,
+but remember you must call jbd2_journal_stop() the same number of times as
+jbd2_journal_start() before the transaction is completed (or more accurately
+leaves the update phase). Ext4/VFS makes use of this feature to simplify
+handling of inode dirtying, quota support, etc.
 </para>
 
 <para>
 Inside each transaction you need to wrap the modifications to the
 individual buffers (blocks). Before you start to modify a buffer you
-need to call journal_get_{create,write,undo}_access() as appropriate,
+need to call jbd2_journal_get_{create,write,undo}_access() as appropriate,
 this allows the journalling layer to copy the unmodified data if it
 needs to. After all the buffer may be part of a previously uncommitted
 transaction.
 At this point you are at last ready to modify a buffer, and once
-you are have done so you need to call journal_dirty_{meta,}data().
+you are have done so you need to call jbd2_journal_dirty_{meta,}data().
 Or if you've asked for access to a buffer you now know is now longer
-required to be pushed back on the device you can call journal_forget()
+required to be pushed back on the device you can call jbd2_journal_forget()
 in much the same way as you might have used bforget() in the past.
 </para>
 
 <para>
-A journal_flush() may be called at any time to commit and checkpoint
+A jbd2_journal_flush() may be called at any time to commit and checkpoint
 all your transactions.
 </para>
 
 <para>
-Then at umount time , in your put_super() you can then call journal_destroy()
+Then at umount time , in your put_super() you can then call jbd2_journal_destroy()
 to clean up your in-core journal object.
 </para>
 
@@ -231,53 +225,68 @@ to clean up your in-core journal object.
 Unfortunately there a couple of ways the journal layer can cause a deadlock.
 The first thing to note is that each task can only have
 a single outstanding transaction at any one time, remember nothing
-commits until the outermost journal_stop(). This means
+commits until the outermost jbd2_journal_stop(). This means
 you must complete the transaction at the end of each file/inode/address
 etc. operation you perform, so that the journalling system isn't re-entered
 on another journal. Since transactions can't be nested/batched
 across differing journals, and another filesystem other than
-yours (say ext3) may be modified in a later syscall.
+yours (say ext4) may be modified in a later syscall.
 </para>
 
 <para>
-The second case to bear in mind is that journal_start() can
+The second case to bear in mind is that jbd2_journal_start() can
 block if there isn't enough space in the journal for your transaction
 (based on the passed nblocks param) - when it blocks it merely(!) needs to
 wait for transactions to complete and be committed from other tasks,
-so essentially we are waiting for journal_stop(). So to avoid
-deadlocks you must treat journal_start/stop() as if they
+so essentially we are waiting for jbd2_journal_stop(). So to avoid
+deadlocks you must treat jbd2_journal_start/stop() as if they
 were semaphores and include them in your semaphore ordering rules to prevent
-deadlocks. Note that journal_extend() has similar blocking behaviour to
-journal_start() so you can deadlock here just as easily as on journal_start().
+deadlocks. Note that jbd2_journal_extend() has similar blocking behaviour to
+jbd2_journal_start() so you can deadlock here just as easily as on
+jbd2_journal_start().
 </para>
 
 <para>
 Try to reserve the right number of blocks the first time. ;-). This will
 be the maximum number of blocks you are going to touch in this transaction.
-I advise having a look at at least ext3_jbd.h to see the basis on which
-ext3 uses to make these decisions.
+I advise having a look at at least ext4_jbd.h to see the basis on which
+ext4 uses to make these decisions.
 </para>
 
 <para>
 Another wriggle to watch out for is your on-disk block allocation strategy.
-why? Because, if you undo a delete, you need to ensure you haven't reused any
-of the freed blocks in a later transaction. One simple way of doing this
-is make sure any blocks you allocate only have checkpointed transactions
-listed against them. Ext3 does this in ext3_test_allocatable().
+Why? Because, if you do a delete, you need to ensure you haven't reused any
+of the freed blocks until the transaction freeing these blocks commits. If you
+reused these blocks and crash happens, there is no way to restore the contents
+of the reallocated blocks at the end of the last fully committed transaction.
+
+One simple way of doing this is to mark blocks as free in internal in-memory
+block allocation structures only after the transaction freeing them commits.
+Ext4 uses journal commit callback for this purpose.
+</para>
+
+<para>
+With journal commit callbacks you can ask the journalling layer to call a
+callback function when the transaction is finally committed to disk, so that
+you can do some of your own management. You ask the journalling layer for
+calling the callback by simply setting journal->j_commit_callback function
+pointer and that function is called after each transaction commit. You can also
+use transaction->t_private_list for attaching entries to a transaction that
+need processing when the transaction commits.
 </para>
 
 <para>
-Lock is also providing through journal_{un,}lock_updates(),
-ext3 uses this when it wants a window with a clean and stable fs for a moment.
-eg.
+JBD2 also provides a way to block all transaction updates via
+jbd2_journal_{un,}lock_updates(). Ext4 uses this when it wants a window with a
+clean and stable fs for a moment.  E.g.
 </para>
 
 <programlisting>
 
-	journal_lock_updates() //stop new stuff happening..
-	journal_flush()        // checkpoint everything.
+	jbd2_journal_lock_updates() //stop new stuff happening..
+	jbd2_journal_flush()        // checkpoint everything.
 	..do stuff on stable fs
-	journal_unlock_updates() // carry on with filesystem use.
+	jbd2_journal_unlock_updates() // carry on with filesystem use.
 </programlisting>
 
 <para>
@@ -286,29 +295,6 @@ if you allow unprivileged userspace to trigger codepaths containing these
 calls.
 </para>
 
-<para>
-A new feature of jbd since 2.5.25 is commit callbacks with the new
-journal_callback_set() function you can now ask the journalling layer
-to call you back when the transaction is finally committed to disk, so that
-you can do some of your own management. The key to this is the journal_callback
-struct, this maintains the internal callback information but you can
-extend it like this:-
-</para>
-<programlisting>
-	struct  myfs_callback_s {
-		//Data structure element required by jbd..
-		struct journal_callback for_jbd;
-		// Stuff for myfs allocated together.
-		myfs_inode*    i_commited;
-
-	}
-</programlisting>
-
-<para>
-this would be useful if you needed to know when data was committed to a
-particular inode.
-</para>
-
     </sect2>
 
     <sect2 id="jbd_summary">
@@ -319,36 +305,6 @@ being each mount, each modification (transaction) and each changed buffer
 to tell the journalling layer about them.
 </para>
 
-<para>
-Here is a some pseudo code to give you an idea of how it works, as
-an example.
-</para>
-
-<programlisting>
-  journal_t* my_jnrl = journal_create();
-  journal_init_{dev,inode}(jnrl,...)
-  if (clean) journal_wipe();
-  journal_load();
-
-   foreach(transaction) { /*transactions must be
-                            completed before
-                            a syscall returns to
-                            userspace*/
-
-          handle_t * xct=journal_start(my_jnrl);
-          foreach(bh) {
-                journal_get_{create,write,undo}_access(xact,bh);
-                if ( myfs_modify(bh) ) { /* returns true
-                                        if makes changes */
-                           journal_dirty_{meta,}data(xact,bh);
-                } else {
-                           journal_forget(bh);
-                }
-          }
-          journal_stop(xct);
-   }
-   journal_destroy(my_jrnl);
-</programlisting>
     </sect2>
 
     </sect1>
@@ -357,13 +313,13 @@ an example.
      <title>Data Types</title>
      <para>
 	The journalling layer uses typedefs to 'hide' the concrete definitions
-	of the structures used. As a client of the JBD layer you can
+	of the structures used. As a client of the JBD2 layer you can
 	just rely on the using the pointer as a magic cookie  of some sort.
 
 	Obviously the hiding is not enforced as this is 'C'.
      </para>
 	<sect2 id="structures"><title>Structures</title>
-!Iinclude/linux/jbd.h
+!Iinclude/linux/jbd2.h
 	</sect2>
     </sect1>
 
@@ -375,11 +331,11 @@ an example.
 	manage transactions
      </para>
 	<sect2 id="journal_level"><title>Journal Level</title>
-!Efs/jbd/journal.c
-!Ifs/jbd/recovery.c
+!Efs/jbd2/journal.c
+!Ifs/jbd2/recovery.c
 	</sect2>
 	<sect2 id="transaction_level"><title>Transasction Level</title>
-!Efs/jbd/transaction.c
+!Efs/jbd2/transaction.c
 	</sect2>
     </sect1>
     <sect1 id="see_also">
-- 
2.1.4

[PATCH 3/3] block: Remove forced page bouncing under IO

[Jan Kara]

From: Jan Kara <jack@suse.cz>

JBD layer wrote back data buffers without setting PageWriteback bit.
Thus standard mechanism for guaranteeing stable pages under IO did not
work. Since JBD is gone now and there is no other user of the
functionality, just remove it.

CC: Jens Axboe <axboe@kernel.dk>
Signed-off-by: Jan Kara <jack@suse.cz>
---
 block/bounce.c            | 31 ++++---------------------------
 include/linux/blk_types.h |  5 ++---
 2 files changed, 6 insertions(+), 30 deletions(-)

diff --git a/block/bounce.c b/block/bounce.c
index b17311227c12..31cad13a0c9d 100644
--- a/block/bounce.c
+++ b/block/bounce.c
@@ -176,26 +176,8 @@ static void bounce_end_io_read_isa(struct bio *bio, int err)
 	__bounce_end_io_read(bio, isa_page_pool, err);
 }
 
-#ifdef CONFIG_NEED_BOUNCE_POOL
-static int must_snapshot_stable_pages(struct request_queue *q, struct bio *bio)
-{
-	if (bio_data_dir(bio) != WRITE)
-		return 0;
-
-	if (!bdi_cap_stable_pages_required(&q->backing_dev_info))
-		return 0;
-
-	return test_bit(BIO_SNAP_STABLE, &bio->bi_flags);
-}
-#else
-static int must_snapshot_stable_pages(struct request_queue *q, struct bio *bio)
-{
-	return 0;
-}
-#endif /* CONFIG_NEED_BOUNCE_POOL */
-
 static void __blk_queue_bounce(struct request_queue *q, struct bio **bio_orig,
-			       mempool_t *pool, int force)
+			       mempool_t *pool)
 {
 	struct bio *bio;
 	int rw = bio_data_dir(*bio_orig);
@@ -203,8 +185,6 @@ static void __blk_queue_bounce(struct request_queue *q, struct bio **bio_orig,
 	struct bvec_iter iter;
 	unsigned i;
 
-	if (force)
-		goto bounce;
 	bio_for_each_segment(from, *bio_orig, iter)
 		if (page_to_pfn(from.bv_page) > queue_bounce_pfn(q))
 			goto bounce;
@@ -216,7 +196,7 @@ bounce:
 	bio_for_each_segment_all(to, bio, i) {
 		struct page *page = to->bv_page;
 
-		if (page_to_pfn(page) <= queue_bounce_pfn(q) && !force)
+		if (page_to_pfn(page) <= queue_bounce_pfn(q))
 			continue;
 
 		to->bv_page = mempool_alloc(pool, q->bounce_gfp);
@@ -254,7 +234,6 @@ bounce:
 
 void blk_queue_bounce(struct request_queue *q, struct bio **bio_orig)
 {
-	int must_bounce;
 	mempool_t *pool;
 
 	/*
@@ -263,15 +242,13 @@ void blk_queue_bounce(struct request_queue *q, struct bio **bio_orig)
 	if (!bio_has_data(*bio_orig))
 		return;
 
-	must_bounce = must_snapshot_stable_pages(q, *bio_orig);
-
 	/*
 	 * for non-isa bounce case, just check if the bounce pfn is equal
 	 * to or bigger than the highest pfn in the system -- in that case,
 	 * don't waste time iterating over bio segments
 	 */
 	if (!(q->bounce_gfp & GFP_DMA)) {
-		if (queue_bounce_pfn(q) >= blk_max_pfn && !must_bounce)
+		if (queue_bounce_pfn(q) >= blk_max_pfn)
 			return;
 		pool = page_pool;
 	} else {
@@ -282,7 +259,7 @@ void blk_queue_bounce(struct request_queue *q, struct bio **bio_orig)
 	/*
 	 * slow path
 	 */
-	__blk_queue_bounce(q, bio_orig, pool, must_bounce);
+	__blk_queue_bounce(q, bio_orig, pool);
 }
 
 EXPORT_SYMBOL(blk_queue_bounce);
diff --git a/include/linux/blk_types.h b/include/linux/blk_types.h
index 7303b3405520..89fd49184b48 100644
--- a/include/linux/blk_types.h
+++ b/include/linux/blk_types.h
@@ -118,9 +118,8 @@ struct bio {
 #define BIO_USER_MAPPED 4	/* contains user pages */
 #define BIO_NULL_MAPPED 5	/* contains invalid user pages */
 #define BIO_QUIET	6	/* Make BIO Quiet */
-#define BIO_SNAP_STABLE	7	/* bio data must be snapshotted during write */
-#define BIO_CHAIN	8	/* chained bio, ->bi_remaining in effect */
-#define BIO_REFFED	9	/* bio has elevated ->bi_cnt */
+#define BIO_CHAIN	7	/* chained bio, ->bi_remaining in effect */
+#define BIO_REFFED	8	/* bio has elevated ->bi_cnt */
 
 /*
  * Flags starting here get preserved by bio_reset() - this includes
-- 
2.1.4

Re: [PATCH 0/3] Remove ext3 filesystem driver

[Nikolay Borisov]

On 07/15/2015 01:26 PM, Jan Kara wrote:
>   Hello,
> 
>   so I have created this patch set which removes ext3 driver (and some related support
> code) from the kernel. See changelog of patch 2/3 for more details. If noone objects,
> I will queue the series in my tree for the next merge window.

Did you accidentally miss patch 2/3 ?

> 
> Jens, are you OK with me merging patch 3/3 or do you want me to send it to you once
> the first two patches get merged?
> 
> Ted, this also touches ext4 driver so that it always handles ext3 filesystems so ack
> from you would be nice.
> 
> 									Honza
> --
> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

Re: [PATCH 0/3] Remove ext3 filesystem driver

[Jan Kara]

On Wed 15-07-15 14:02:56, Nikolay Borisov wrote:
> 
> 
> On 07/15/2015 01:26 PM, Jan Kara wrote:
> >   Hello,
> > 
> >   so I have created this patch set which removes ext3 driver (and some related support
> > code) from the kernel. See changelog of patch 2/3 for more details. If noone objects,
> > I will queue the series in my tree for the next merge window.
> 
> Did you accidentally miss patch 2/3 ?

No, I did not but apparently it was too big for the list. Interested people
can have a look at my tree:
git.kernel.org/pub/scm/linux/kernel/git/jack/linux-fs.git releasepage

The particular commit is:

https://git.kernel.org/cgit/linux/kernel/git/jack/linux-fs.git/commit/?h=releasepage&id=32c4b776ad2774aa0e8fdd8ad35f848b15b84f38

Thanks for the notice.

								Honza
> > 
> > Jens, are you OK with me merging patch 3/3 or do you want me to send it to you once
> > the first two patches get merged?
> > 
> > Ted, this also touches ext4 driver so that it always handles ext3 filesystems so ack
> > from you would be nice.
> > 
> > 									Honza
> > --
> > To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> > 
> 
-- 
Jan Kara <jack@suse.cz>
SUSE Labs, CR

Re: [PATCH 0/3] Remove ext3 filesystem driver

[Theodore Ts'o]

On Wed, Jul 15, 2015 at 12:26:24PM +0200, Jan Kara wrote:
> 
>   so I have created this patch set which removes ext3 driver (and some related support
> code) from the kernel. See changelog of patch 2/3 for more details. If noone objects,
> I will queue the series in my tree for the next merge window.
> 
> Jens, are you OK with me merging patch 3/3 or do you want me to send it to you once
> the first two patches get merged?
> 
> Ted, this also touches ext4 driver so that it always handles ext3 filesystems so ack
> from you would be nice.

Got tired of maintaining ext3, eh?   :-)

As a minor nit, I wonder if it's worth while to put in some hackery in
fs/Kconfig so that if someone only has CONFIG_FS_EXT3 defined, but not
CONFIG_FS_EXT4 defined, that it will automatically enable
CONFIG_FS_EXT4 instead?  Otherwise someone upgrading their kernel
might accidentally end up getting a non-functional kernel.

i.e., something like this? (not tested!)

config FS_EXT3
       def_bool n
       select FS_EXT4


In any case, since the 2/3 patch didn't hit the list, here's my:

Acked-by: Theodore Ts'o <tytso@mit.edu>

For that patch.

    	 						- Ted

Re: [PATCH 0/3] Remove ext3 filesystem driver

[Jens Axboe]

On 07/15/2015 04:26 AM, Jan Kara wrote:
>    Hello,
>
>    so I have created this patch set which removes ext3 driver (and some related support
> code) from the kernel. See changelog of patch 2/3 for more details. If noone objects,
> I will queue the series in my tree for the next merge window.
>
> Jens, are you OK with me merging patch 3/3 or do you want me to send it to you once
> the first two patches get merged?

You can merge that one, will be easier than splitting. Add my acked-by 
to the patch.


-- 
Jens Axboe

Re: [PATCH 0/3] Remove ext3 filesystem driver

[Joe Perches]

On Wed, 2015-07-15 at 12:26 +0200, Jan Kara wrote:
> I have created this patch set which removes ext3 driver (and some related support
> code) from the kernel. See changelog of patch 2/3 for more details.

It'd be nice if you regenerate 2/3 using
git format-patch -M and resend it.

Re: [PATCH 0/3] Remove ext3 filesystem driver

[Randy Dunlap]

On 07/15/15 07:18, Theodore Ts'o wrote:
> On Wed, Jul 15, 2015 at 12:26:24PM +0200, Jan Kara wrote:
>>
>>   so I have created this patch set which removes ext3 driver (and some related support
>> code) from the kernel. See changelog of patch 2/3 for more details. If noone objects,
>> I will queue the series in my tree for the next merge window.
>>
>> Jens, are you OK with me merging patch 3/3 or do you want me to send it to you once
>> the first two patches get merged?
>>
>> Ted, this also touches ext4 driver so that it always handles ext3 filesystems so ack
>> from you would be nice.
> 
> Got tired of maintaining ext3, eh?   :-)
> 
> As a minor nit, I wonder if it's worth while to put in some hackery in
> fs/Kconfig so that if someone only has CONFIG_FS_EXT3 defined, but not
> CONFIG_FS_EXT4 defined, that it will automatically enable
> CONFIG_FS_EXT4 instead?  Otherwise someone upgrading their kernel
> might accidentally end up getting a non-functional kernel.
> 
> i.e., something like this? (not tested!)
> 
> config FS_EXT3
>        def_bool n
>        select FS_EXT4


Yes, if someone is going to force us to use ext4, some automation would
be helpful.

Thanks,
-- 
~Randy

Re: [PATCH 0/3] Remove ext3 filesystem driver

[Randy Dunlap]

On 07/15/15 08:05, Randy Dunlap wrote:
> On 07/15/15 07:18, Theodore Ts'o wrote:
>> On Wed, Jul 15, 2015 at 12:26:24PM +0200, Jan Kara wrote:
>>>
>>>   so I have created this patch set which removes ext3 driver (and some related support
>>> code) from the kernel. See changelog of patch 2/3 for more details. If noone objects,
>>> I will queue the series in my tree for the next merge window.
>>>
>>> Jens, are you OK with me merging patch 3/3 or do you want me to send it to you once
>>> the first two patches get merged?
>>>
>>> Ted, this also touches ext4 driver so that it always handles ext3 filesystems so ack
>>> from you would be nice.
>>
>> Got tired of maintaining ext3, eh?   :-)
>>
>> As a minor nit, I wonder if it's worth while to put in some hackery in
>> fs/Kconfig so that if someone only has CONFIG_FS_EXT3 defined, but not
>> CONFIG_FS_EXT4 defined, that it will automatically enable
>> CONFIG_FS_EXT4 instead?  Otherwise someone upgrading their kernel
>> might accidentally end up getting a non-functional kernel.
>>
>> i.e., something like this? (not tested!)
>>
>> config FS_EXT3
>>        def_bool n
>>        select FS_EXT4
> 
> 
> Yes, if someone is going to force us to use ext4, some automation would
> be helpful.
> 
> Thanks,
> 

Are there any other changes required, e.g., to /etc/fstab ?

btw, this looks like an April 1 joke to me.

-- 
~Randy

Re: [PATCH 0/3] Remove ext3 filesystem driver

[Eric Sandeen]

On 7/15/15 10:09 AM, Randy Dunlap wrote:
> On 07/15/15 08:05, Randy Dunlap wrote:
>> On 07/15/15 07:18, Theodore Ts'o wrote:
>>> On Wed, Jul 15, 2015 at 12:26:24PM +0200, Jan Kara wrote:
>>>>
>>>>   so I have created this patch set which removes ext3 driver (and some related support
>>>> code) from the kernel. See changelog of patch 2/3 for more details. If noone objects,
>>>> I will queue the series in my tree for the next merge window.
>>>>
>>>> Jens, are you OK with me merging patch 3/3 or do you want me to send it to you once
>>>> the first two patches get merged?
>>>>
>>>> Ted, this also touches ext4 driver so that it always handles ext3 filesystems so ack
>>>> from you would be nice.
>>>
>>> Got tired of maintaining ext3, eh?   :-)
>>>
>>> As a minor nit, I wonder if it's worth while to put in some hackery in
>>> fs/Kconfig so that if someone only has CONFIG_FS_EXT3 defined, but not
>>> CONFIG_FS_EXT4 defined, that it will automatically enable
>>> CONFIG_FS_EXT4 instead?  Otherwise someone upgrading their kernel
>>> might accidentally end up getting a non-functional kernel.
>>>
>>> i.e., something like this? (not tested!)
>>>
>>> config FS_EXT3
>>>        def_bool n
>>>        select FS_EXT4
>>
>>
>> Yes, if someone is going to force us to use ext4, some automation would
>> be helpful.
>>
>> Thanks,
>>
> 
> Are there any other changes required, e.g., to /etc/fstab ?
> 
> btw, this looks like an April 1 joke to me.

no fstab changes required.  The ext4 driver has been able to register
itself as ext[23] for quite some time now, so it's transparent.

Many/most distros have been using the ext4.ko driver for ext3 & ext2 for
years.  You may already be using that on some boxes, and not even know it.
;)

-Eric

Re: [PATCH 0/3] Remove ext3 filesystem driver

[Randy Dunlap]

On 07/15/15 08:11, Eric Sandeen wrote:
> On 7/15/15 10:09 AM, Randy Dunlap wrote:
>> On 07/15/15 08:05, Randy Dunlap wrote:
>>> On 07/15/15 07:18, Theodore Ts'o wrote:
>>>> On Wed, Jul 15, 2015 at 12:26:24PM +0200, Jan Kara wrote:
>>>>>
>>>>>   so I have created this patch set which removes ext3 driver (and some related support
>>>>> code) from the kernel. See changelog of patch 2/3 for more details. If noone objects,
>>>>> I will queue the series in my tree for the next merge window.
>>>>>
>>>>> Jens, are you OK with me merging patch 3/3 or do you want me to send it to you once
>>>>> the first two patches get merged?
>>>>>
>>>>> Ted, this also touches ext4 driver so that it always handles ext3 filesystems so ack
>>>>> from you would be nice.
>>>>
>>>> Got tired of maintaining ext3, eh?   :-)
>>>>
>>>> As a minor nit, I wonder if it's worth while to put in some hackery in
>>>> fs/Kconfig so that if someone only has CONFIG_FS_EXT3 defined, but not
>>>> CONFIG_FS_EXT4 defined, that it will automatically enable
>>>> CONFIG_FS_EXT4 instead?  Otherwise someone upgrading their kernel
>>>> might accidentally end up getting a non-functional kernel.
>>>>
>>>> i.e., something like this? (not tested!)
>>>>
>>>> config FS_EXT3
>>>>        def_bool n
>>>>        select FS_EXT4
>>>
>>>
>>> Yes, if someone is going to force us to use ext4, some automation would
>>> be helpful.
>>>
>>> Thanks,
>>>
>>
>> Are there any other changes required, e.g., to /etc/fstab ?
>>
>> btw, this looks like an April 1 joke to me.
> 
> no fstab changes required.  The ext4 driver has been able to register
> itself as ext[23] for quite some time now, so it's transparent.
> 
> Many/most distros have been using the ext4.ko driver for ext3 & ext2 for
> years.  You may already be using that on some boxes, and not even know it.
> ;)

Aha.  Right you are.  Thanks.  :)


-- 
~Randy

Re: [PATCH 0/3] Remove ext3 filesystem driver

[Austin S Hemmelgarn]

[-- Attachment #1: Type: text/plain, Size: 2374 bytes --]

On 2015-07-15 11:11, Eric Sandeen wrote:
> On 7/15/15 10:09 AM, Randy Dunlap wrote:
>> On 07/15/15 08:05, Randy Dunlap wrote:
>>> On 07/15/15 07:18, Theodore Ts'o wrote:
>>>> On Wed, Jul 15, 2015 at 12:26:24PM +0200, Jan Kara wrote:
>>>>>
>>>>>    so I have created this patch set which removes ext3 driver (and some related support
>>>>> code) from the kernel. See changelog of patch 2/3 for more details. If noone objects,
>>>>> I will queue the series in my tree for the next merge window.
>>>>>
>>>>> Jens, are you OK with me merging patch 3/3 or do you want me to send it to you once
>>>>> the first two patches get merged?
>>>>>
>>>>> Ted, this also touches ext4 driver so that it always handles ext3 filesystems so ack
>>>>> from you would be nice.
>>>>
>>>> Got tired of maintaining ext3, eh?   :-)
>>>>
>>>> As a minor nit, I wonder if it's worth while to put in some hackery in
>>>> fs/Kconfig so that if someone only has CONFIG_FS_EXT3 defined, but not
>>>> CONFIG_FS_EXT4 defined, that it will automatically enable
>>>> CONFIG_FS_EXT4 instead?  Otherwise someone upgrading their kernel
>>>> might accidentally end up getting a non-functional kernel.
>>>>
>>>> i.e., something like this? (not tested!)
>>>>
>>>> config FS_EXT3
>>>>         def_bool n
>>>>         select FS_EXT4
>>>
>>>
>>> Yes, if someone is going to force us to use ext4, some automation would
>>> be helpful.
>>>
>>> Thanks,
>>>
>>
>> Are there any other changes required, e.g., to /etc/fstab ?
>>
>> btw, this looks like an April 1 joke to me.
>
> no fstab changes required.  The ext4 driver has been able to register
> itself as ext[23] for quite some time now, so it's transparent.
>
> Many/most distros have been using the ext4.ko driver for ext3 & ext2 for
> years.  You may already be using that on some boxes, and not even know it.
> ;)
Yeah, 99% of the systems I've seen in the past 2 years have been using 
either the ext4 driver, or using exclusively ext2 (mostly embedded 
systems, and they usually choose ext2 because it's _very_ lightweight 
compared to ext[34] and the other alternatives).

There really isn't much of a clear distinction between ext3 and ext4 (at 
least from an end user standpoint), other than the fact that there are 
some options that only the ext4 driver understands (like extent based 
allocation).


[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 2967 bytes --]

Re: [PATCH 2/3] fs: Remove ext3 filesystem driver

[Andrew Morton]

On Wed, 15 Jul 2015 12:26:26 +0200 Jan Kara <jack@suse.com> wrote:

> From: Jan Kara <jack@suse.cz>
> 
> The functionality of ext3 is fully supported by ext4 driver. Major
> distributions (SUSE, RedHat) already use ext4 driver to handle ext3
> filesystems for quite some time. There is some ugliness in mm resulting
> from jbd cleaning buffers in a dirty page without cleaning page dirty
> bit and also support for buffer bouncing in the block layer when stable
> pages are required is there only because of jbd. So let's remove the
> ext3 driver.

Does this imply that ext4 doesn't do the
secretly-clean-the-page-via-buffers thing?  If so, how?

The comment in shrink_page_list() says the blockdev mapping will do
this as well, although I can't imagine how - there's no means of
getting to those buffer_heads except via the page.  So maybe the "even
if the page is PageDirty()" is no longer true.  It was added by:

commit 493f4988d640a73337df91f2c63e94c78ecd5e97
Author: Andrew Morton <akpm@zip.com.au>
Date:   Mon Jun 17 20:20:53 2002 -0700

    [PATCH] allow GFP_NOFS allocators to perform swapcache writeout
    
    One weakness which was introduced when the buffer LRU went away was
    that GFP_NOFS allocations became equivalent to GFP_NOIO.  Because all
    writeback goes via writepage/writepages, which requires entry into the
    filesystem.
    
    However now that swapout no longer calls bmap(), we can honour
    GFP_NOFS's intent for swapcache pages.  So if the allocation request
    specifies __GFP_IO and !__GFP_FS, we can wait on swapcache pages and we
    can perform swapcache writeout.
    
    This should strengthen the VM somewhat.

I wonder what I was thinking.



Also, what's the status of ext4's data=journal?  It's the hardest ext3
mode for the rest of the kernel to support and I suspect hardly anyone
uses it.

> 46 files changed, 54 insertions(+), 28109 deletions(-)

Heroic. 

Re: [PATCH 2/3] fs: Remove ext3 filesystem driver

[Austin S Hemmelgarn]

[-- Attachment #1: Type: text/plain, Size: 439 bytes --]

On 2015-07-15 12:58, Andrew Morton wrote:
> Also, what's the status of ext4's data=journal?  It's the hardest ext3
> mode for the rest of the kernel to support and I suspect hardly anyone
> uses it.
I use it, as do some other people I know, but only for stuff that pretty 
much needs to be 100% guaranteed to be consistent after a power failure. 
  It is _very_ slow, but it does work correctly based on my recent 
experience.



[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 2967 bytes --]

Re: [PATCH 0/3] Remove ext3 filesystem driver

[Theodore Ts'o]

On Wed, Jul 15, 2015 at 11:20:55AM -0400, Austin S Hemmelgarn wrote:
> 
> There really isn't much of a clear distinction between ext3 and ext4 (at
> least from an end user standpoint), other than the fact that there are some
> options that only the ext4 driver understands (like extent based
> allocation).

Yeah, the main reason why we did the ext3 -> ext4 fork was that adding
64-bit numbers required major surgery, and we didn't want to break a
lot of production users who were using ext3.  But from a file system
format perspective, ext2, ext3, and ext4 are the same logical file
system.  There are just multiple different implementations, which all
support slightly different sets of file system features:

   * Linux's ext2
   * Linux's ext3
   * Linux's ext4
   * Hurd's ext2
   * *BSD's ext2
   * Grub's ext2/3/4

The last three implementations are in fact independent ones created
from scratch.   :-)

Fortunately we use the same file system support code, e2fsprogs, for
all of them, which is good since it has a very extensive set of
regression test sets for our fsck program, and we've continued adding
to it as we add new file system features.

					- Ted

Re: [PATCH 0/3] Remove ext3 filesystem driver

[Jan Kara]

On Wed 15-07-15 08:01:05, Joe Perches wrote:
> On Wed, 2015-07-15 at 12:26 +0200, Jan Kara wrote:
> > I have created this patch set which removes ext3 driver (and some related support
> > code) from the kernel. See changelog of patch 2/3 for more details.
> 
> It'd be nice if you regenerate 2/3 using
> git format-patch -M and resend it.

Good idea, thanks. Will do with the next submission.

								Honza
-- 
Jan Kara <jack@suse.cz>
SUSE Labs, CR

Re: [PATCH 0/3] Remove ext3 filesystem driver

[Jan Kara]

On Wed 15-07-15 10:18:36, Ted Tso wrote:
> On Wed, Jul 15, 2015 at 12:26:24PM +0200, Jan Kara wrote:
> > 
> >   so I have created this patch set which removes ext3 driver (and some related support
> > code) from the kernel. See changelog of patch 2/3 for more details. If noone objects,
> > I will queue the series in my tree for the next merge window.
> > 
> > Jens, are you OK with me merging patch 3/3 or do you want me to send it to you once
> > the first two patches get merged?
> > 
> > Ted, this also touches ext4 driver so that it always handles ext3 filesystems so ack
> > from you would be nice.
> 
> Got tired of maintaining ext3, eh?   :-)

Not really. Maintaining ext3 has been pretty low-effort in last years. But
the releasepage trouble has made me reconsider whether it's worth keeping
ext3 codebase and I just came to conclusion that it may be the right time
to drop it.

> As a minor nit, I wonder if it's worth while to put in some hackery in
> fs/Kconfig so that if someone only has CONFIG_FS_EXT3 defined, but not
> CONFIG_FS_EXT4 defined, that it will automatically enable
> CONFIG_FS_EXT4 instead?  Otherwise someone upgrading their kernel
> might accidentally end up getting a non-functional kernel.
> 
> i.e., something like this? (not tested!)
> 
> config FS_EXT3
>        def_bool n
>        select FS_EXT4

Good point. Will add something like this, although the config will have to
have some prompt for this to work...

> In any case, since the 2/3 patch didn't hit the list, here's my:
> 
> Acked-by: Theodore Ts'o <tytso@mit.edu>

Thanks.

								Honza
-- 
Jan Kara <jack@suse.cz>
SUSE Labs, CR

Re: [PATCH 2/3] fs: Remove ext3 filesystem driver

[Jan Kara]

On Wed 15-07-15 09:58:22, Andrew Morton wrote:
> On Wed, 15 Jul 2015 12:26:26 +0200 Jan Kara <jack@suse.com> wrote:
> 
> > From: Jan Kara <jack@suse.cz>
> > 
> > The functionality of ext3 is fully supported by ext4 driver. Major
> > distributions (SUSE, RedHat) already use ext4 driver to handle ext3
> > filesystems for quite some time. There is some ugliness in mm resulting
> > from jbd cleaning buffers in a dirty page without cleaning page dirty
> > bit and also support for buffer bouncing in the block layer when stable
> > pages are required is there only because of jbd. So let's remove the
> > ext3 driver.
> 
> Does this imply that ext4 doesn't do the
> secretly-clean-the-page-via-buffers thing?  If so, how?

The biggest offender which was cleaning pages via buffers was JBD commit
code writing back data=ordered buffers. I have modified JBD2 to do this
via generic_writepages() instead of through buffer heads (which required
locking overhaul in JBD2). So JBD2 doesn't do this for quite a few years.

That being said, JBD2 checkpointing code will still clean pages via buffer
heads so blockdev mapping may still have silently cleaned pages. And in
data=journal mode this can be the case even for other mappings. In these
cases, locking isn't luckily an issue and fixing this is relatively
straightforward. I'm just looking for an elegant way to do this inside JBD2
- I'm hoping for something better than just get page from bh, lock it and
call clear_page_dirty_for_io() and ->writepage(). It works but looks
ugly...

> The comment in shrink_page_list() says the blockdev mapping will do
> this as well, although I can't imagine how - there's no means of
> getting to those buffer_heads except via the page.  So maybe the "even
> if the page is PageDirty()" is no longer true.  It was added by:
> 
> commit 493f4988d640a73337df91f2c63e94c78ecd5e97
> Author: Andrew Morton <akpm@zip.com.au>
> Date:   Mon Jun 17 20:20:53 2002 -0700
> 
>     [PATCH] allow GFP_NOFS allocators to perform swapcache writeout
>     
>     One weakness which was introduced when the buffer LRU went away was
>     that GFP_NOFS allocations became equivalent to GFP_NOIO.  Because all
>     writeback goes via writepage/writepages, which requires entry into the
>     filesystem.
>     
>     However now that swapout no longer calls bmap(), we can honour
>     GFP_NOFS's intent for swapcache pages.  So if the allocation request
>     specifies __GFP_IO and !__GFP_FS, we can wait on swapcache pages and we
>     can perform swapcache writeout.
>     
>     This should strengthen the VM somewhat.
> 
> I wonder what I was thinking.

Well, e.g. sync_mapping_buffers() from fs/buffer.c will write out buffer
heads without cleaning the page. So does the checkpointing code in
JBD/JBD2. So for blockdev mappings, this really happens rather frequently
I'd say.

> Also, what's the status of ext4's data=journal?  It's the hardest ext3
> mode for the rest of the kernel to support and I suspect hardly anyone
> uses it.

As this thread shows, there are people using it (and I occasionally see bug
reports for it as well). It would simplify things if we could get rid of it
but I don't think it's currently an option...

								Honza
-- 
Jan Kara <jack@suse.cz>
SUSE Labs, CR

Re: [PATCH 2/3] fs: Remove ext3 filesystem driver

[Bruce Guenter]

[-- Attachment #1: Type: text/plain, Size: 819 bytes --]

On Wed, Jul 15, 2015 at 01:35:09PM -0400, Austin S Hemmelgarn wrote:
> On 2015-07-15 12:58, Andrew Morton wrote:
> > Also, what's the status of ext4's data=journal?  It's the hardest ext3
> > mode for the rest of the kernel to support and I suspect hardly anyone
> > uses it.
> I use it, as do some other people I know, but only for stuff that pretty 
> much needs to be 100% guaranteed to be consistent after a power failure. 
>   It is _very_ slow, but it does work correctly based on my recent 
> experience.

It is also a benefit for workloads that are heavy on sync writes to
(relatively) small files, such as mail servers. At least, it used to be
considerably faster than data=ordered the last time I benchmarked it.

-- 
Bruce Guenter <bruce@untroubled.org>                http://untroubled.org/

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [PATCH 0/3] Remove ext3 filesystem driver

[Konstantin Khlebnikov]

On 15.07.2015 13:26, Jan Kara wrote:
>    Hello,
>
>    so I have created this patch set which removes ext3 driver (and some related support
> code) from the kernel. See changelog of patch 2/3 for more details. If noone objects,
> I will queue the series in my tree for the next merge window.
>
> Jens, are you OK with me merging patch 3/3 or do you want me to send it to you once
> the first two patches get merged?
>
> Ted, this also touches ext4 driver so that it always handles ext3 filesystems so ack
> from you would be nice.
>
> 									Honza
>

This means some ext3-specific kludges in mm/ could be dropped.
Like page cleanup in try_to_free_buffers() and couple in vmscan.

Re: [PATCH 0/3] Remove ext3 filesystem driver

[Jan Kara]

On Wed 29-07-15 15:20:06, Konstantin Khlebnikov wrote:
> On 15.07.2015 13:26, Jan Kara wrote:
> >   Hello,
> >
> >   so I have created this patch set which removes ext3 driver (and some related support
> >code) from the kernel. See changelog of patch 2/3 for more details. If noone objects,
> >I will queue the series in my tree for the next merge window.
> >
> >Jens, are you OK with me merging patch 3/3 or do you want me to send it to you once
> >the first two patches get merged?
> >
> >Ted, this also touches ext4 driver so that it always handles ext3 filesystems so ack
> >from you would be nice.
> >
> >									Honza
> >
> 
> This means some ext3-specific kludges in mm/ could be dropped.
> Like page cleanup in try_to_free_buffers() and couple in vmscan.

Almost. There are still some cases (like block device mappings or even
standard inode mappings in ext4 data=journal mode) where the page cleanup
hack is needed. See
https://www.mail-archive.com/linux-kernel@vger.kernel.org/msg933319.html

Both issues look fixable although with block device mappings it won't be
trivial.
								Honza
-- 
Jan Kara <jack@suse.com>
SUSE Labs, CR

Re: [PATCH 0/3] Remove ext3 filesystem driver

[Pavel Machek]

On Wed 2015-07-15 12:26:24, Jan Kara wrote:
>   Hello,
> 
>   so I have created this patch set which removes ext3 driver (and some related support
> code) from the kernel. See changelog of patch 2/3 for more details. If noone objects,
> I will queue the series in my tree for the next merge window.
> 
> Jens, are you OK with me merging patch 3/3 or do you want me to send it to you once
> the first two patches get merged?
> 
> Ted, this also touches ext4 driver so that it always handles ext3 filesystems so ack
> from you would be nice.

Nice to see sources getting smaller... but how much bigger do the binaries get
for someone only using ext3?
									Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

Re: [PATCH 0/3] Remove ext3 filesystem driver

[Theodore Ts'o]

On Sun, Sep 27, 2015 at 09:24:21AM +0200, Pavel Machek wrote:
> 
> Nice to see sources getting smaller... but how much bigger do the binaries get
> for someone only using ext3?

The difference in object size between ext3 and ext4 is about 200k.
Most of the efforts to use really tiny kernels are looking at kernels
way smaller than even ext3, so this isn't something I'm terribly
worried about.  (That is, they'll be using ext2, or squashfs, or
jffs2, etc.)

						- Ted