[GIT PULL] Btrfs updates for 2.6.29-rc

[GIT PULL] Btrfs updates for 2.6.29-rc

[Chris Mason]

Hello everyone,

The current btrfs unstable tree should be ready for 2.6.29-rc:

git://git.kernel.org/pub/scm/linux/kernel/git/mason/btrfs-unstable.git

We've been busy working on cleanups and improvements based on the
feedback for Btrfs.  There are still lots of cleanups left to do,
especially the ones pointed out by Andrew.

Some of them I've been holding on to as starter tasks for people that
are interested in the project.

The current btrfs unstable tree also has a number of fixes hit by
testers and by the Fedora alpha integration.  These include selinux
support, an oops in the compression code, and a few different IO stalls.

There are two big performance related changes in the tree.  I got rid of
the adaptive mutex implementation and switched to a lock that explicitly
switches between a spinlock and a waitqueue.  This is faster and it
allows us to pick off the blocking lock points one by one over time in a
controlled fashion.

The second big performance fix deals with snapshot deletion.  It makes
progress much more effectively and is able to keep up with bigger
workloads.  This is important because every transaction ends up deleting
a snapshot.

There wasn't much code involved in the snapshot deletion improvements,
it is basically just sorting the block pointers before updating their
reference counts, and working on leaves in bulk instead of one by one.

Here's the shortlog:

Chris Ball (1):
      Btrfs: Handle SGID bit when creating inodes

Chris Mason (14):
      Btrfs: Catch missed bios in the async bio submission thread
      Btrfs: async threads should try harder to find work
      Btrfs: sort references by byte number during btrfs_inc_ref
      Btrfs: disable leak debugging checks in extent_io.c
      Btrfs: hash_lock is no longer needed
      Btrfs: Change btree locking to use explicit blocking points
      Btrfs: Make btrfs_drop_snapshot work in larger and more efficient chunks
      Btrfs: Don't try to compress pages past i_size
      Btrfs: Change btrfs_truncate_inode_items to stop when it hits the inode
      Btrfs: change btrfs_del_leaf to drop locks earlier
      Btrfs: fix btrfs_unlock_up_safe to walk the entire path
      Btrfs: Only prep for btree deletion balances when nodes are mostly empty
      Btrfs: don't return congestion in write_cache_pages as often
      Btrfs: Fix memory leak in cache_drop_leaf_ref

Christian Hesse (1):
      Btrfs: make btrfs acls selectable

Jim Owens (1):
      Btrfs: selinux support

Josef Bacik (1):
      Btrfs: join the transaction in __btrfs_setxattr

Re: [GIT PULL] Btrfs updates for 2.6.29-rc

[Linus Torvalds]

On Fri, 6 Feb 2009, Chris Mason wrote:
> 
> Here's the shortlog:

Diffstats too, please.

Even before pulling, I want to know that some pull is supposed to only 
touch files in that persons maintenance area.

And _after_ pulling, I want to be able to verify that what I pulled is 
what the maintainer meant for me to pull.

diffstats help with those.

	git diff -M --stat --summary linus...

or just use "git request-pull" and edit up the end result.

		Linus

Re: [GIT PULL] Btrfs updates for 2.6.29-rc

[Chris Mason]

On Fri, 2009-02-06 at 08:53 -0800, Linus Torvalds wrote:
> 
> On Fri, 6 Feb 2009, Chris Mason wrote:
> > 
> > Here's the shortlog:
> 
> Diffstats too, please.
> 

Whoops, here we go:

fs/btrfs/Kconfig        |   13 ++
 fs/btrfs/async-thread.c |   60 +++++++-
 fs/btrfs/ctree.c        |  259 ++++++++++++++++++++++++++++--
 fs/btrfs/ctree.h        |    5 +-
 fs/btrfs/disk-io.c      |   16 ++-
 fs/btrfs/extent-tree.c  |  397 +++++++++++++++++++++++++++++++++++++++++------
 fs/btrfs/extent_io.c    |   39 ++---
 fs/btrfs/extent_io.h    |   16 ++-
 fs/btrfs/inode.c        |   57 ++++++-
 fs/btrfs/locking.c      |  208 ++++++++++++++++++++++--
 fs/btrfs/locking.h      |    6 +
 fs/btrfs/ref-cache.c    |    1 +
 fs/btrfs/ref-cache.h    |    1 -
 fs/btrfs/tree-defrag.c  |    1 +
 fs/btrfs/tree-log.c     |    4 +
 fs/btrfs/volumes.c      |   11 +-
 fs/btrfs/xattr.c        |   34 ++++-
 fs/btrfs/xattr.h        |    2 +
 18 files changed, 996 insertions(+), 134 deletions(-)

This time around, there are actually lots of comments added.

-chris

Re: [GIT PULL] Btrfs updates for 2.6.29-rc

[Linus Torvalds]

On Fri, 6 Feb 2009, Chris Mason wrote:
>
>  18 files changed, 996 insertions(+), 134 deletions(-)

This is not at all what I get. I get

 27 files changed, 1421 insertions(+), 451 deletions(-)

so there is some confusion there.

		Linus

Re: [GIT PULL] Btrfs updates for 2.6.29-rc

[Chris Mason]

On Fri, 2009-02-06 at 13:59 -0800, Linus Torvalds wrote:
> 
> On Fri, 6 Feb 2009, Chris Mason wrote:
> >
> >  18 files changed, 996 insertions(+), 134 deletions(-)
> 
> This is not at all what I get. I get
> 
>  27 files changed, 1421 insertions(+), 451 deletions(-)
> 
> so there is some confusion there.

Crud, the confusion would be me.  I missed the cleanups and bug fixes
between 2.6.29-rc1 and rc3.   The correct diffstat and short log are
below, and the only non-btrfs change is to the MAINTAINERS file.

MAINTAINERS             |    8
 fs/btrfs/Kconfig        |   13 +
 fs/btrfs/async-thread.c |   61 +++++-
 fs/btrfs/compression.c  |    1
 fs/btrfs/ctree.c        |  276 ++++++++++++++++++++++++++----
 fs/btrfs/ctree.h        |   28 +--
 fs/btrfs/disk-io.c      |  120 +++++++++----
 fs/btrfs/disk-io.h      |    2
 fs/btrfs/extent-tree.c  |  438 ++++++++++++++++++++++++++++++++++++++++--------
 fs/btrfs/extent_io.c    |  132 +++++++++++---
 fs/btrfs/extent_io.h    |   18 +
 fs/btrfs/extent_map.c   |    1
 fs/btrfs/file.c         |    5
 fs/btrfs/inode.c        |   84 +++++++--
 fs/btrfs/ioctl.c        |    1
 fs/btrfs/locking.c      |  208 ++++++++++++++++++++--
 fs/btrfs/locking.h      |    6
 fs/btrfs/ordered-data.c |    4
 fs/btrfs/ref-cache.c    |    1
 fs/btrfs/ref-cache.h    |    1
 fs/btrfs/super.c        |    6
 fs/btrfs/transaction.c  |    4
 fs/btrfs/tree-defrag.c  |    1
 fs/btrfs/tree-log.c     |  354 ++++++++++++++++++--------------------
 fs/btrfs/volumes.c      |   49 ++---
 fs/btrfs/xattr.c        |   48 ++++-
 fs/btrfs/xattr.h        |    2
 27 files changed, 1421 insertions(+), 451 deletions(-)

Chris Ball (1):
      Btrfs: Handle SGID bit when creating inodes

Chris Mason (19):
      Btrfs: stop providing a bmap operation to avoid swapfile corruptions
      Btrfs: do less aggressive btree readahead
      Merge branch 'master' of git://git.kernel.org/.../mason/btrfs-unstable
      Btrfs: fix readdir on 32 bit machines
      Btrfs: Catch missed bios in the async bio submission thread
      Btrfs: async threads should try harder to find work
      Btrfs: sort references by byte number during btrfs_inc_ref
      Btrfs: disable leak debugging checks in extent_io.c
      Btrfs: hash_lock is no longer needed
      Btrfs: Change btree locking to use explicit blocking points
      Btrfs: Make btrfs_drop_snapshot work in larger and more efficient chunks
      Btrfs: Don't try to compress pages past i_size
      Btrfs: Change btrfs_truncate_inode_items to stop when it hits the inode
      Btrfs: change btrfs_del_leaf to drop locks earlier
      Btrfs: fix btrfs_unlock_up_safe to walk the entire path
      Btrfs: Only prep for btree deletion balances when nodes are mostly empty
      Btrfs: don't return congestion in write_cache_pages as often
      Btrfs: Fix memory leak in cache_drop_leaf_ref
      Btrfs: Make sure dir is non-null before doing S_ISGID checks

Christian Hesse (1):
      Btrfs: make btrfs acls selectable

Huang Weiyi (2):
      Btrfs: removed unused #include <version.h>'s
      Btrfs: remove duplicated #include

Jan Engelhardt (1):
      Btrfs: change/remove typedef

Jim Owens (1):
      Btrfs: selinux support

Joe Perches (1):
      Btrfs: MAINTAINERS entry

Josef Bacik (2):
      Btrfs: cleanup xattr code
      Btrfs: join the transaction in __btrfs_setxattr

Qinghuang Feng (3):
      Btrfs: check return value for kthread_run() correctly
      Btrfs: simplify iteration codes
      Btrfs: open_ctree() error handling can oops on fs_info

Roland Dreier (1):
      Btrfs: Remove extra KERN_INFO in the middle of a line

Wang Cong (1):
      Btrfs: cleanup fs/btrfs/super.c::btrfs_control_ioctl()

Yan Zheng (4):
      Btrfs: fix locking issue in btrfs_remove_block_group
      Btrfs: Fix infinite loop in btrfs_extent_post_op
      Btrfs: fix stop searching test in replace_one_extent
      Btrfs: fix tree logs parallel sync

Yehuda Sadeh (1):
      Btrfs: fiemap support

Re: [GIT PULL] Btrfs updates for 2.6.29-rc

[Linus Torvalds]

On Fri, 6 Feb 2009, Chris Mason wrote:
> 
> Crud, the confusion would be me.  I missed the cleanups and bug fixes
> between 2.6.29-rc1 and rc3.   The correct diffstat and short log are
> below, and the only non-btrfs change is to the MAINTAINERS file.

Ok.

I notice that it now asks about ACL's, and says

	If you don't know what Access Control Lists are, say N

which is all fine, but last I saw some (most?) distributions use POSIX 
ACL's for things like desktop access permissions to /dev/audio etc.

I don't particularly have to like it, but it does mean that if you don't 
do POSIX ACL's, and have btrfs as your root filesystem, you'll quite 
likely not get sound when logging in.

I dunno. I haven't actually yet tried to convert my laptop, so I just 
reacted to this thanks to the new question..

		Linus

Re: [GIT PULL] Btrfs updates for 2.6.29-rc

[Chris Mason]

On Sat, 2009-02-07 at 08:37 -0800, Linus Torvalds wrote:
> On Fri, 6 Feb 2009, Chris Mason wrote:
> > 
> > Crud, the confusion would be me.  I missed the cleanups and bug fixes
> > between 2.6.29-rc1 and rc3.   The correct diffstat and short log are
> > below, and the only non-btrfs change is to the MAINTAINERS file.
> 
> Ok.
> 
> I notice that it now asks about ACL's, and says
> 
> 	If you don't know what Access Control Lists are, say N

The text was stolen from ext3, or from someone else that stole it from
ext3....it made sense to me to stay consistent with the others here.
The config option really just provides a way to enable
CONFIG_FS_POSIX_ACL.

The btrfs code only looks for CONFIG_FS_POSIX_ACL, and doesn't have a
special mount -o acl,xattr,security.

-chris

Re: [GIT PULL] Btrfs updates for 2.6.29-rc

[Kay Sievers]

On Sat, Feb 7, 2009 at 17:37, Linus Torvalds
<torvalds@linux-foundation.org> wrote:
> On Fri, 6 Feb 2009, Chris Mason wrote:
>>
>> Crud, the confusion would be me.  I missed the cleanups and bug fixes
>> between 2.6.29-rc1 and rc3.   The correct diffstat and short log are
>> below, and the only non-btrfs change is to the MAINTAINERS file.
>
> Ok.
>
> I notice that it now asks about ACL's, and says
>
>        If you don't know what Access Control Lists are, say N
>
> which is all fine, but last I saw some (most?) distributions use POSIX
> ACL's for things like desktop access permissions to /dev/audio etc.
>
> I don't particularly have to like it, but it does mean that if you don't
> do POSIX ACL's, and have btrfs as your root filesystem, you'll quite
> likely not get sound when logging in.
>
> I dunno. I haven't actually yet tried to convert my laptop, so I just
> reacted to this thanks to the new question..

Systems using ACLs to grant device permissions to logged-in users have
/dev on a tmpfs mount, where the content will not survive a reboot.
This should not be a problem for btrfs, or the rootfs fs type, but
TMPFS_POSIX_ACL. Maybe we want to have a comment there?

Thanks,
Kay

[GIT PULL] Btrfs updates for 2.6.29-rc

[Chris Mason]

[resend, with an address for Linus that didn't come out of an ancient
address book ]

Hello everyone,

We've had good results in testing Josef's initial enospc support for
btrfs.  This mainly deals with better accounting of the data extents
that have been allocated for delalloc, and properly returning enospc for
new data writes that can't possibly succeed.

There is some metadata enospc support as well, but there is more work to
be done in that area.  Because this code changes the time at which some
deep call chains are done, it also cuts down on the btrfs stack usage.

Linus, if you pull the master branch of:

git://git.kernel.org/pub/scm/linux/kernel/git/mason/btrfs-unstable.git

You'll find:

Josef Bacik (2):
      Btrfs: add better -ENOSPC handling
      Btrfs: try committing transaction before returning ENOSPC


 b/fs/btrfs/btrfs_inode.h |    8 +
 b/fs/btrfs/ctree.h       |   40 ++++++--
 b/fs/btrfs/extent-tree.c |   57 ++++++++++--
 b/fs/btrfs/file.c        |   16 ++-
 b/fs/btrfs/inode.c       |   62 +++----------
 b/fs/btrfs/ioctl.c       |    6 -
 fs/btrfs/extent-tree.c   |  215
+++++++++++++++++++++++++++++++++++++++++++----
 7 files changed, 318 insertions(+), 86 deletions(-)

[GIT PULL] Btrfs updates for 2.6.29-rc

[Chris Mason]

Hello everyone,

We've had good results in testing Josef's initial enospc support for
btrfs.  This mainly deals with better accounting of the data extents
that have been allocated for delalloc, and properly returning enospc for
new data writes that can't possibly succeed.

There is some metadata enospc support as well, but there is more work to
be done in that area.  Because this code changes the time at which some
deep call chains are done, it also cuts down on the btrfs stack usage.

Linus, if you pull the master branch of:

git://git.kernel.org/pub/scm/linux/kernel/git/mason/btrfs-unstable.git

You'll find:

Josef Bacik (2):
      Btrfs: add better -ENOSPC handling
      Btrfs: try committing transaction before returning ENOSPC


 b/fs/btrfs/btrfs_inode.h |    8 +
 b/fs/btrfs/ctree.h       |   40 ++++++--
 b/fs/btrfs/extent-tree.c |   57 ++++++++++--
 b/fs/btrfs/file.c        |   16 ++-
 b/fs/btrfs/inode.c       |   62 +++----------
 b/fs/btrfs/ioctl.c       |    6 -
 fs/btrfs/extent-tree.c   |  215
+++++++++++++++++++++++++++++++++++++++++++----
 7 files changed, 318 insertions(+), 86 deletions(-)

[GIT PULL] Btrfs updates for 2.6.29-rc

[Chris Mason]

Hello everyone,

The master branch
git://git.kernel.org/pub/scm/linux/kernel/git/mason/btrfs-unstable.git

Has the following fixes and updates for 2.6.29-rc.  There is also a
branch named experimental that is very very experimental.

I think I pushed things out correctly so that master and experimental
didn't mix, but we've long established my ability to mess that part up.

These are cleanups and fixes that came out of testing and review.  The
lockdep fix is actually the biggest in terms of code, but that is under
CONFIG_DEBUG_LOCK_ALLOC.

SSD mode is changed to work better on ssds that do random writes poorly,
and btrfs_sync_fs is changed to skip the cleaning of old snapshots.
That still happens in the background, and it was making sync look slower
than it really was.

mount -o remount finally looks at the new options sent, so you can send
important things like mount -o compress,ssd for remounting now.

-chris

Chris Mason (4):
      Btrfs: process mount options on mount -o remount,
      Btrfs: use larger metadata clusters in ssd mode
      Btrfs: don't clean old snapshots on sync(1)
      Btrfs: make a lockdep class for the extent buffer locks

Jeff Mahoney (2):
      Btrfs: balance_level checks !child after access
      Btrfs: remove btrfs_init_path

Josef Bacik (1):
      Btrfs: make sure all pending extent operations are complete

Julia Lawall (1):
      Btrfs: fs/btrfs/volumes.c: remove useless kzalloc

Qinghuang Feng (1):
      Btrfs: remove unused code in split_state()

Yan Zheng (2):
      Btrfs: Avoid using __GFP_HIGHMEM with slab allocator
      Btrfs: hold trans_mutex when using btrfs_record_root_in_trans

 b/fs/btrfs/ctree.c       |   45 +++++++++++++++++++-------
 b/fs/btrfs/ctree.h       |   10 +----
 b/fs/btrfs/disk-io.c     |   46 ++++++++++++++++++++++++++
 b/fs/btrfs/disk-io.h     |   10 +++++
 b/fs/btrfs/extent-tree.c |    2 +
 b/fs/btrfs/extent_io.c   |    2 -
 b/fs/btrfs/inode-map.c   |    1 
 b/fs/btrfs/inode.c       |    2 -
 b/fs/btrfs/locking.c     |   11 ------
 b/fs/btrfs/super.c       |    1 
 b/fs/btrfs/transaction.c |    2 +
 b/fs/btrfs/tree-log.c    |    2 +
 b/fs/btrfs/volumes.c     |    2 +
 fs/btrfs/ctree.c         |   13 +------
 fs/btrfs/ctree.h         |    1 
 fs/btrfs/extent-tree.c   |   81 ++++++++++++++++++++++++++++-------------------
 fs/btrfs/inode.c         |    2 -
 fs/btrfs/super.c         |    4 ++
 fs/btrfs/volumes.c       |    4 --
 19 files changed, 157 insertions(+), 84 deletions(-)