Release of E2fsprogs 1.42.6

Release of E2fsprogs 1.42.6

[Theodore Ts'o]

I've released e2fsprogs 1.42.7 in all of the usual places; it's tagged
in the git trees on git.kernel.org, github, sourceforge, and
code.google.com, and available for ftp at:

ftp://ftp.kernel.org/pub/linux/kernel/people/tytso/e2fsprogs/v1.42.7

and

http://prdownloads.sourceforge.net/e2fsprogs/e2fsprogs-1.42.7.tar.gz


This release includes some significant improvements to resize2fs; in
particular it resolves a number of bugs which could lead to file systems
getting corrupted, especially filesystems with the 64-bit or bigalloc
features enabled.

                                                - Ted


E2fsprogs 1.42.7 (January 21, 2013)
===================================

Add warnings to mke2fs, resize2fs, and tune2fs that the bigalloc and
quota features are still under development.  For more information
please see:
    * https://ext4.wiki.kernel.org/index.php/Bigalloc
    * https://ext4.wiki.kernel.org/index.php/Quota

Add some new options to filefrag from Lustre's patches to e2fsprogs:
    * add -k option to print extents in kB-sized units (like df -k)
    * add -b {blocksize} to print extents in blocksize units
    * add -e option to print extent format, even when FIBMAP is used
    * add -X option to print extents in hexadecimal format

Fix resize2fs so that it can handle off-line resizes of file systems
with the flex_bg feature but without a resize_inode (or if we run out
of reserved gdt blocks).  This also fixes a problem where if the user
creates a filesystem with a restricted number of reserved gdt blocks,
an off-line resize which grows the file system could potentially
result in file system corruption.

Fix a resize2fs bug which could cause it to corrupt bigalloc file
systems when doing an off-line resize.

Further optimize resize2fs so it doesn't use quite as much CPU when
resizing very large file systems. 

Fixed 32-bit overflow bugs which could cause resize2fs to fail and
possibly corrupt the file system while resizing 64-bit file systems.

Fix a bug in mke2fs where parsing "-E resize=NNN" will result in a
mke2fs crash due to a divide-by-zero if the 64bit file system feature
is enabled.

Add better error checking to mke2fs to check for invalid parameters
when creating bigalloc file system.

When creating bigalloc filesystems, the -g option to mke2fs will now
specify the number of clusters per block group.

Add to debugfs the functionality to corrupt a specific file system
block via the "zap_block" command.

Add to debugfs the functionality to print out a hex dump of a block in
the file system via the "block_dump" command.

Add to debugfs the functionality to manipulate the extent tree
directly via the "extent_open" command.

Fixed debugfs's mknod command so that it updates the block group
statistics.

Fix e2fsck so it can detect and fix inconsistencies in the interior
nodes of an inode's extent tree.

Fix a potential memory corruption failure in e2fsck's error path if
the call to ext2fs_open2() fails.

Fix e2fsck if its logging function is enabled in e2fsck.conf, and the
resulting file name for the log file is longer than 100 bytes, that it
properly handles this situation instead of crashing.

E2fsck will now report the amount of memory that it attempted to
allocate when a memory allocation request fails, to make it easier to
track down the problem.

Fix mke2fs's handling of the mmp_update_interval option.  (Addresses
Lustre Bug: LU-1888)

E2image can now include all data blocks in the e2image output file
when the user specifies the -a option.

If e2fsprogs is compiled without --enable-quota, make sure that
tune2fs can not turn on the feature for new-style quota support, since
afterwards, none of the e2fsprogs tools will be willing to touch that
file system.

Optimize e2fsck's so that it uses much less CPU for large file
systems.  This can result in significant speedups, especially on
CPU-constrained systems.  This was primarily done by optimizing
libext2fs's bitmap functions.  (Addresses Google Bug: #7534813)

Fix debugfs's htree command so that all its messages are sent through
the pager.

Fixed debugfs's "dump_file" and "cat" functions work correctly
on file systems whose block size is greater than 8k.

Fix e2freefrag so it works on 64-bit file systems, and so it uses much
less memory.  (Addresses Google Bug: 7269948)

Update the spd_readdir.c file in the contrib directory to include some
additions which were made in 2008 that didn't get folded into the
version which we checked into the e2fsprogs source tree.  These
enhancements include thread safety, support for readdir64_r(), and
safe_getenv() support in case spd_readdir.so is used for setuid binaries.

Update Czech, Dutch, French, German, Polish, Sweedish, and Vietnamese
translations

Add a command to debugfs to create symlinks.

Document the bigalloc feature in the mke2fs man page.  (Addresses
Debian Bug: #669730)


Programmer's Notes
------------------

Fix gcc -Wall nits.

Fix a spelling typo in the libext2fs texinfo documentation.

Change the output from "make check" so that tools such as emacs's "M-x
compile" does not mistake the output as containing a compiler error.

Export two new functions from libext2fs: ext2fs_extent_node_split()
and ext2fs_extent_goto2().

The ext2fs_extents_fix_parents() was fixed so it does not modify the
location of the extent handle if its current location is at an
interior node, rather than a leaf node of an inode's extent tree.

Add a regression test to assure that e2fsck can correctly fix an
inconsistent interior node in an inode's extent tree.

The ext2fs_{mark,unmark,test}_block_bitmap_range2() functions now
correctly support bigalloc file systems which use store block usage
information in units of clusters.

Fixed the help text in the configure script for --enable-quota.

The m68k-specific bitops code has been removed since they were
incorrectly treating bit numbers with the high bit set as signed
integers.  Furthermore, modern compilers do a good enough job
optimizing the generic code there is no point in having the m68k
specific asm statements.

Fixed how we link the test programs so they always use the static
libraries, so that we test using the libraries which we have just
built.

Update config.guess and config.sub to the latest versions from the GNU
project.

Fixed the com_err.texinfo file so that it can produce a valid
postscript/pdf printed output.

Add a regression test which checks debugfs's ability to create
symlinks, named FIFO's, and device nodes.

Add a function ext2fs_symlink(), which creates symlinks to the
libext2fs library.

Add debugging code so we can test old kernel interfaces for online
resize to resize2fs.  This backwards compatibility checking is keyed
off of the RESIZE2FS_KERNEL_VERSION, which is designed to allow us to
test the functionality of the kernel's older resize ioctls without
needing to install an old version of resize2fs, and to also test a
modern resize2fs's ability to work with older kernels without having
to install an older version of the kernel.

Release of E2fsprogs 1.42.6 (Was re: Release of E2fsprogs 1.42.7)

[Theodore Ts'o]

s/1.42.6/1.42.7/

Sorry for the typo in the subject line....

      	      	      	  	  - Ted



On Mon, Jan 21, 2013 at 11:30:53PM -0500, Theodore Ts'o wrote:
> 
> I've released e2fsprogs 1.42.7 in all of the usual places; it's tagged
> in the git trees on git.kernel.org, github, sourceforge, and
> code.google.com, and available for ftp at:
> 
> ftp://ftp.kernel.org/pub/linux/kernel/people/tytso/e2fsprogs/v1.42.7
> 
> and
> 
> http://prdownloads.sourceforge.net/e2fsprogs/e2fsprogs-1.42.7.tar.gz
> 
> 
> This release includes some significant improvements to resize2fs; in
> particular it resolves a number of bugs which could lead to file systems
> getting corrupted, especially filesystems with the 64-bit or bigalloc
> features enabled.
> 
>                                                 - Ted
> 
> 
> E2fsprogs 1.42.7 (January 21, 2013)
> ===================================
> 
> Add warnings to mke2fs, resize2fs, and tune2fs that the bigalloc and
> quota features are still under development.  For more information
> please see:
>     * https://ext4.wiki.kernel.org/index.php/Bigalloc
>     * https://ext4.wiki.kernel.org/index.php/Quota
> 
> Add some new options to filefrag from Lustre's patches to e2fsprogs:
>     * add -k option to print extents in kB-sized units (like df -k)
>     * add -b {blocksize} to print extents in blocksize units
>     * add -e option to print extent format, even when FIBMAP is used
>     * add -X option to print extents in hexadecimal format
> 
> Fix resize2fs so that it can handle off-line resizes of file systems
> with the flex_bg feature but without a resize_inode (or if we run out
> of reserved gdt blocks).  This also fixes a problem where if the user
> creates a filesystem with a restricted number of reserved gdt blocks,
> an off-line resize which grows the file system could potentially
> result in file system corruption.
> 
> Fix a resize2fs bug which could cause it to corrupt bigalloc file
> systems when doing an off-line resize.
> 
> Further optimize resize2fs so it doesn't use quite as much CPU when
> resizing very large file systems. 
> 
> Fixed 32-bit overflow bugs which could cause resize2fs to fail and
> possibly corrupt the file system while resizing 64-bit file systems.
> 
> Fix a bug in mke2fs where parsing "-E resize=NNN" will result in a
> mke2fs crash due to a divide-by-zero if the 64bit file system feature
> is enabled.
> 
> Add better error checking to mke2fs to check for invalid parameters
> when creating bigalloc file system.
> 
> When creating bigalloc filesystems, the -g option to mke2fs will now
> specify the number of clusters per block group.
> 
> Add to debugfs the functionality to corrupt a specific file system
> block via the "zap_block" command.
> 
> Add to debugfs the functionality to print out a hex dump of a block in
> the file system via the "block_dump" command.
> 
> Add to debugfs the functionality to manipulate the extent tree
> directly via the "extent_open" command.
> 
> Fixed debugfs's mknod command so that it updates the block group
> statistics.
> 
> Fix e2fsck so it can detect and fix inconsistencies in the interior
> nodes of an inode's extent tree.
> 
> Fix a potential memory corruption failure in e2fsck's error path if
> the call to ext2fs_open2() fails.
> 
> Fix e2fsck if its logging function is enabled in e2fsck.conf, and the
> resulting file name for the log file is longer than 100 bytes, that it
> properly handles this situation instead of crashing.
> 
> E2fsck will now report the amount of memory that it attempted to
> allocate when a memory allocation request fails, to make it easier to
> track down the problem.
> 
> Fix mke2fs's handling of the mmp_update_interval option.  (Addresses
> Lustre Bug: LU-1888)
> 
> E2image can now include all data blocks in the e2image output file
> when the user specifies the -a option.
> 
> If e2fsprogs is compiled without --enable-quota, make sure that
> tune2fs can not turn on the feature for new-style quota support, since
> afterwards, none of the e2fsprogs tools will be willing to touch that
> file system.
> 
> Optimize e2fsck's so that it uses much less CPU for large file
> systems.  This can result in significant speedups, especially on
> CPU-constrained systems.  This was primarily done by optimizing
> libext2fs's bitmap functions.  (Addresses Google Bug: #7534813)
> 
> Fix debugfs's htree command so that all its messages are sent through
> the pager.
> 
> Fixed debugfs's "dump_file" and "cat" functions work correctly
> on file systems whose block size is greater than 8k.
> 
> Fix e2freefrag so it works on 64-bit file systems, and so it uses much
> less memory.  (Addresses Google Bug: 7269948)
> 
> Update the spd_readdir.c file in the contrib directory to include some
> additions which were made in 2008 that didn't get folded into the
> version which we checked into the e2fsprogs source tree.  These
> enhancements include thread safety, support for readdir64_r(), and
> safe_getenv() support in case spd_readdir.so is used for setuid binaries.
> 
> Update Czech, Dutch, French, German, Polish, Sweedish, and Vietnamese
> translations
> 
> Add a command to debugfs to create symlinks.
> 
> Document the bigalloc feature in the mke2fs man page.  (Addresses
> Debian Bug: #669730)
> 
> 
> Programmer's Notes
> ------------------
> 
> Fix gcc -Wall nits.
> 
> Fix a spelling typo in the libext2fs texinfo documentation.
> 
> Change the output from "make check" so that tools such as emacs's "M-x
> compile" does not mistake the output as containing a compiler error.
> 
> Export two new functions from libext2fs: ext2fs_extent_node_split()
> and ext2fs_extent_goto2().
> 
> The ext2fs_extents_fix_parents() was fixed so it does not modify the
> location of the extent handle if its current location is at an
> interior node, rather than a leaf node of an inode's extent tree.
> 
> Add a regression test to assure that e2fsck can correctly fix an
> inconsistent interior node in an inode's extent tree.
> 
> The ext2fs_{mark,unmark,test}_block_bitmap_range2() functions now
> correctly support bigalloc file systems which use store block usage
> information in units of clusters.
> 
> Fixed the help text in the configure script for --enable-quota.
> 
> The m68k-specific bitops code has been removed since they were
> incorrectly treating bit numbers with the high bit set as signed
> integers.  Furthermore, modern compilers do a good enough job
> optimizing the generic code there is no point in having the m68k
> specific asm statements.
> 
> Fixed how we link the test programs so they always use the static
> libraries, so that we test using the libraries which we have just
> built.
> 
> Update config.guess and config.sub to the latest versions from the GNU
> project.
> 
> Fixed the com_err.texinfo file so that it can produce a valid
> postscript/pdf printed output.
> 
> Add a regression test which checks debugfs's ability to create
> symlinks, named FIFO's, and device nodes.
> 
> Add a function ext2fs_symlink(), which creates symlinks to the
> libext2fs library.
> 
> Add debugging code so we can test old kernel interfaces for online
> resize to resize2fs.  This backwards compatibility checking is keyed
> off of the RESIZE2FS_KERNEL_VERSION, which is designed to allow us to
> test the functionality of the kernel's older resize ioctls without
> needing to install an old version of resize2fs, and to also test a
> modern resize2fs's ability to work with older kernels without having
> to install an older version of the kernel.

Release of e2fsprogs 1.42.6

[Theodore Ts'o]

I've released e2fsprogs 1.42.6 in all of the usual places; it's tagged
in the git trees on git.kernel.org, github, sourceforge, and
code.google.com, and available for ftp at:

ftp://ftp.kernel.org/pub/linux/kernel/people/tytso/e2fsprogs/v1.42.6

and

http://prdownloads.sourceforge.net/e2fsprogs/e2fsprogs-1.42.6.tar.gz

The major improvement in e2fsprogs 1.42.6 is enabling online resizing
for file systems larger than 16TB or which have the 64bit feature
enable.  This requires kernel patches that will be in 3.7.

In addition, I've released the first 1.43 WIP snapshot.   It has all of
the fixes of 1.42.6, plus support for metadata checksums.

                                               - Ted

E2fsprogs 1.43-WIP (September 22, 2012)
=======================================

Add support for the ext4 metadata checksum feature.

Check to make sure file system features which can not be supported by
HURD are not enabled if the file system is created to be
HURD-compatible.


Programmer's Notes
------------------

Reduce the use of libc functions in libext2fs that may not be present
in the boot loader environment, at least for those functions that are
needed by boot loadsers such as yaboot.

Support for the MMP feature can now be disabled at compile time.


E2fsprogs 1.42.6 (September 21, 2012)
=====================================

When mke2fs creates file systems with lazy itable initialization, the
progress updates for writing the inode table happens so quickly that
on a serial console, the time to write the progress updates can be the
bottleneck.  So mke2fs will now only update its progress indicators
once a second.

Resize2fs will skip initializing the inode tables if the kernel
supports lazy_itable_init, which speeds up growing off-line growth of
uninit_bg file systems significantly.  Resize2fs will now also
correctly set the itable_unused field in the block group descriptor to
speed up the first e2fsck after the file system is grown.

Resize2fs has been fixed so that on-line resizing of meta_bg file
systems work correctly.  This is needed to grow file system which are
larger than 16T.

Resize2fs will now correctly handle resizing file systems to 16TB on
32-bit file systems when "16TB" is specified on the command line.

Fix mke2fs so that it will be careful to set the reserved blocks ratio
larger than 50%; this can happen when creating small file systems and
when the last block group is dropped because there are not enough
blocks to support the metadata blocks in the last block group.

Fixed spelling mistake in debugfs's help message.

Fixed a potential seg fault in e2fsck when there is an I/O error while
reading the superblock.

Fixed various Debian Packaging Issues (Addresses Debian Bug #677497)

Updated/fixed various man pages.  (Addresses Sourceforge Bug:
#3559210)

Programmer's Notes
------------------

The configure option --enable-relative-symlink was broken so that it
needed to be --enable-symlink-relative-symlinks.  We will support both
for at least two years, but then the worng configure option will be
removed.

Fixed a regression introduced in 1.42.5 so the link order for
e2fsprogs' libraries will be correct for both static and shared
linking.  (Addresses Sourceforge Bug: #3554345)

Add support for e2fsprogs to be compiled using clang/LLVM.

Fix portability problems on non-Linux systems: avoid compile failures
on systems that don't have malloc.h

Fix f_mmp regression test suite so that debugfs gets killed if the
test is interrupted.

Release of e2fsprogs 1.42.6

[Theodore Ts'o]

I've released e2fsprogs 1.42.6 in all of the usual places; it's tagged
in the git trees on git.kernel.org, github, sourceforge, and
code.google.com, and available for ftp at:

ftp://ftp.kernel.org/pub/linux/kernel/people/tytso/e2fsprogs/v1.42.6

and

http://prdownloads.sourceforge.net/e2fsprogs/e2fsprogs-1.42.6.tar.gz

The major improvement in e2fsprogs 1.42.6 is enabling online resizing
for file systems larger than 16TB or which have the 64bit feature
enable.  This requires kernel patches that will be in 3.7.

In addition, I've released the first 1.43 WIP snapshot.   It has all of
the fixes of 1.42.6, plus support for metadata checksums.

                                               - Ted

E2fsprogs 1.43-WIP (September 22, 2012)
=======================================

Add support for the ext4 metadata checksum feature.

Check to make sure file system features which can not be supported by
HURD are not enabled if the file system is created to be
HURD-compatible.


Programmer's Notes
------------------

Reduce the use of libc functions in libext2fs that may not be present
in the boot loader environment, at least for those functions that are
needed by boot loadsers such as yaboot.

Support for the MMP feature can now be disabled at compile time.


E2fsprogs 1.42.6 (September 21, 2012)
=====================================

When mke2fs creates file systems with lazy itable initialization, the
progress updates for writing the inode table happens so quickly that
on a serial console, the time to write the progress updates can be the
bottleneck.  So mke2fs will now only update its progress indicators
once a second.

Resize2fs will skip initializing the inode tables if the kernel
supports lazy_itable_init, which speeds up growing off-line growth of
uninit_bg file systems significantly.  Resize2fs will now also
correctly set the itable_unused field in the block group descriptor to
speed up the first e2fsck after the file system is grown.

Resize2fs has been fixed so that on-line resizing of meta_bg file
systems work correctly.  This is needed to grow file system which are
larger than 16T.

Resize2fs will now correctly handle resizing file systems to 16TB on
32-bit file systems when "16TB" is specified on the command line.

Fix mke2fs so that it will be careful to set the reserved blocks ratio
larger than 50%; this can happen when creating small file systems and
when the last block group is dropped because there are not enough
blocks to support the metadata blocks in the last block group.

Fixed spelling mistake in debugfs's help message.

Fixed a potential seg fault in e2fsck when there is an I/O error while
reading the superblock.

Fixed various Debian Packaging Issues (Addresses Debian Bug #677497)

Updated/fixed various man pages.  (Addresses Sourceforge Bug:
#3559210)

Programmer's Notes
------------------

The configure option --enable-relative-symlink was broken so that it
needed to be --enable-symlink-relative-symlinks.  We will support both
for at least two years, but then the worng configure option will be
removed.

Fixed a regression introduced in 1.42.5 so the link order for
e2fsprogs' libraries will be correct for both static and shared
linking.  (Addresses Sourceforge Bug: #3554345)

Add support for e2fsprogs to be compiled using clang/LLVM.

Fix portability problems on non-Linux systems: avoid compile failures
on systems that don't have malloc.h

Fix f_mmp regression test suite so that debugfs gets killed if the
test is interrupted.