* [RFC PATCH] Btrfs: fix memory leak of orphan block rsv
@ 2013-08-19 18:08 Filipe David Borba Manana
2013-08-19 18:31 ` [PATCH v2] " Filipe David Borba Manana
` (4 more replies)
0 siblings, 5 replies; 13+ messages in thread
From: Filipe David Borba Manana @ 2013-08-19 18:08 UTC (permalink / raw)
To: linux-btrfs; +Cc: Filipe David Borba Manana
When adding orphans to an inode's root, we start a transaction for
that root that when ended in several places such as for example
extent-tree.c:btrfs_remove_block_group(), inode.c:btrfs_unlink() and
inode.c:btrfs_evict_node(), doesn't result in a commit, that is,
inode.c:btrfs_orphan_commit_root() doesn't get called (via
transaction.c:commit_fs_roots()).
The respective inode will end up being called by inode.c:btrfs_evict_node()
(via the VFS). So my likely dirty hack, after some debugging, of
freeing the orphan block rsv in btrfs_evict_inode() if its not being
used by other tasks, seems to fix the leak without failing xfstests
so far. But this is unlikely to be a correct solution without causing
issues elsewhere.
This commit also changes calls to btrfs_orphan_del() which grabbed the
root's orphan_block_rsv without acquiring the orphan_lock first (confront
with btrfs_orphan_commit_root(), which acquires the lock before getting
and setting the root's orphan_block_rsv).
This issue is simple to reproduce and observe if kmemleak is enabled.
Two simple ways to reproduce it:
** 1
$ mkfs.btrfs -f /dev/loop0
$ mount /dev/loop0 /mnt/btrfs
$ btrfs balance start /mnt/btrfs
$ umount /mnt/btrfs
** 2
$ mkfs.btrfs -f /dev/loop0
$ mount /dev/loop0 /mnt/btrfs
$ touch /mnt/btrfs/foobar
$ rm -f /mnt/btrfs/foobar
$ umount /mnt/btrfs
After a while, kmemleak reports the leak:
$ cat /sys/kernel/debug/kmemleak
unreferenced object 0xffff880402b13e00 (size 128):
comm "btrfs", pid 19621, jiffies 4341648183 (age 70057.844s)
hex dump (first 32 bytes):
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
00 fc c6 b1 04 88 ff ff 04 00 04 00 ad 4e ad de .............N..
backtrace:
[<ffffffff817275a6>] kmemleak_alloc+0x26/0x50
[<ffffffff8117832b>] kmem_cache_alloc_trace+0xeb/0x1d0
[<ffffffffa04db499>] btrfs_alloc_block_rsv+0x39/0x70 [btrfs]
[<ffffffffa04f8bad>] btrfs_orphan_add+0x13d/0x1b0 [btrfs]
[<ffffffffa04e2b13>] btrfs_remove_block_group+0x143/0x500 [btrfs]
[<ffffffffa0518158>] btrfs_relocate_chunk.isra.63+0x618/0x790 [btrfs]
[<ffffffffa051bc27>] btrfs_balance+0x8f7/0xe90 [btrfs]
[<ffffffffa05240a0>] btrfs_ioctl_balance+0x250/0x550 [btrfs]
[<ffffffffa05269ca>] btrfs_ioctl+0xdfa/0x25f0 [btrfs]
[<ffffffff8119c936>] do_vfs_ioctl+0x96/0x570
[<ffffffff8119cea1>] SyS_ioctl+0x91/0xb0
[<ffffffff81750242>] system_call_fastpath+0x16/0x1b
[<ffffffffffffffff>] 0xffffffffffffffff
This affects btrfs-next, revision be8e3cd00d7293dd177e3f8a4a1645ce09ca3acb
(Btrfs: separate out tests into their own directory).
Signed-off-by: Filipe David Borba Manana <fdmanana@gmail.com>
---
fs/btrfs/ctree.h | 1 +
fs/btrfs/extent-tree.c | 1 +
fs/btrfs/inode.c | 36 +++++++++++++++++++++++++++++-------
3 files changed, 31 insertions(+), 7 deletions(-)
diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h
index 7c93d9f..da3ac13 100644
--- a/fs/btrfs/ctree.h
+++ b/fs/btrfs/ctree.h
@@ -1153,6 +1153,7 @@ struct btrfs_block_rsv {
u64 reserved;
struct btrfs_space_info *space_info;
spinlock_t lock;
+ atomic_t count;
unsigned short full;
unsigned short type;
unsigned short failfast;
diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c
index 32639f2..3ecee8e 100644
--- a/fs/btrfs/extent-tree.c
+++ b/fs/btrfs/extent-tree.c
@@ -4500,6 +4500,7 @@ struct btrfs_block_rsv *btrfs_alloc_block_rsv(struct btrfs_root *root,
btrfs_init_block_rsv(block_rsv, type);
block_rsv->space_info = __find_space_info(fs_info,
BTRFS_BLOCK_GROUP_METADATA);
+ atomic_set(&block_rsv->count, 0);
return block_rsv;
}
diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c
index b67b81a..459ffe2 100644
--- a/fs/btrfs/inode.c
+++ b/fs/btrfs/inode.c
@@ -2870,8 +2870,13 @@ void btrfs_orphan_commit_root(struct btrfs_trans_handle *trans,
return;
}
- block_rsv = root->orphan_block_rsv;
- root->orphan_block_rsv = NULL;
+ if (root->orphan_block_rsv &&
+ atomic_dec_and_test(&root->orphan_block_rsv->count)) {
+ block_rsv = root->orphan_block_rsv;
+ root->orphan_block_rsv = NULL;
+ } else {
+ block_rsv = NULL;
+ }
spin_unlock(&root->orphan_lock);
if (root->orphan_item_inserted &&
@@ -2918,6 +2923,7 @@ int btrfs_orphan_add(struct btrfs_trans_handle *trans, struct inode *inode)
btrfs_free_block_rsv(root, block_rsv);
block_rsv = NULL;
}
+ atomic_inc(&root->orphan_block_rsv->count);
if (!test_and_set_bit(BTRFS_INODE_HAS_ORPHAN_ITEM,
&BTRFS_I(inode)->runtime_flags)) {
@@ -2991,6 +2997,9 @@ static int btrfs_orphan_del(struct btrfs_trans_handle *trans,
int ret = 0;
spin_lock(&root->orphan_lock);
+ if (trans)
+ trans->block_rsv = root->orphan_block_rsv;
+
if (test_and_clear_bit(BTRFS_INODE_HAS_ORPHAN_ITEM,
&BTRFS_I(inode)->runtime_flags))
delete_item = 1;
@@ -4520,12 +4529,10 @@ void btrfs_evict_inode(struct inode *inode)
* Errors here aren't a big deal, it just means we leave orphan items
* in the tree. They will be cleaned up on the next mount.
*/
- if (ret == 0) {
- trans->block_rsv = root->orphan_block_rsv;
+ if (ret == 0)
btrfs_orphan_del(trans, inode);
- } else {
+ else
btrfs_orphan_del(NULL, inode);
- }
trans->block_rsv = &root->fs_info->trans_block_rsv;
if (!(root == root->fs_info->tree_root ||
@@ -4535,6 +4542,22 @@ void btrfs_evict_inode(struct inode *inode)
btrfs_end_transaction(trans, root);
btrfs_btree_balance_dirty(root);
no_delete:
+
+ spin_lock(&root->orphan_lock);
+ if (root->orphan_block_rsv &&
+ atomic_dec_and_test(&root->orphan_block_rsv->count)) {
+ rsv = root->orphan_block_rsv;
+ root->orphan_block_rsv = NULL;
+ } else {
+ rsv = NULL;
+ }
+ spin_unlock(&root->orphan_lock);
+
+ if (rsv) {
+ WARN_ON(rsv->size > 0);
+ btrfs_free_block_rsv(root, rsv);
+ }
+
btrfs_remove_delayed_node(inode);
clear_inode(inode);
return;
@@ -7650,7 +7673,6 @@ static int btrfs_truncate(struct inode *inode)
}
if (ret == 0 && inode->i_nlink > 0) {
- trans->block_rsv = root->orphan_block_rsv;
ret = btrfs_orphan_del(trans, inode);
if (ret)
err = ret;
--
1.7.9.5
^ permalink raw reply related [flat|nested] 13+ messages in thread
* [PATCH v2] Btrfs: fix memory leak of orphan block rsv
2013-08-19 18:08 [RFC PATCH] Btrfs: fix memory leak of orphan block rsv Filipe David Borba Manana
@ 2013-08-19 18:31 ` Filipe David Borba Manana
2013-08-19 20:15 ` [PATCH v3] " Filipe David Borba Manana
` (3 subsequent siblings)
4 siblings, 0 replies; 13+ messages in thread
From: Filipe David Borba Manana @ 2013-08-19 18:31 UTC (permalink / raw)
To: linux-btrfs; +Cc: Filipe David Borba Manana
When adding orphans to an inode's root, we start a transaction for
that root that when ended in several places such as for example
extent-tree.c:btrfs_remove_block_group(), inode.c:btrfs_unlink() and
inode.c:btrfs_evict_node(), doesn't result in a commit, that is,
inode.c:btrfs_orphan_commit_root() doesn't get called (via
transaction.c:commit_fs_roots()).
The respective inode will end up being called by inode.c:btrfs_evict_node()
(via the VFS). So my likely dirty hack, after some debugging, of
freeing the orphan block rsv in btrfs_evict_inode() if it's not being
used by other tasks, seems to fix the leak for the cases listed below and
without failing xfstests.
This commit also changes calls to btrfs_orphan_del() which grabbed the
root's orphan_block_rsv without acquiring the orphan_lock first (confront
with btrfs_orphan_commit_root(), which acquires the lock before getting
and setting the root's orphan_block_rsv).
This issue is simple to reproduce and observe if kmemleak is enabled.
There are at least two simple ways to reproduce it:
** 1
$ mkfs.btrfs -f /dev/loop0
$ mount /dev/loop0 /mnt/btrfs
$ btrfs balance start /mnt/btrfs
$ umount /mnt/btrfs
** 2
$ mkfs.btrfs -f /dev/loop0
$ mount /dev/loop0 /mnt/btrfs
$ touch /mnt/btrfs/foobar
$ rm -f /mnt/btrfs/foobar
$ umount /mnt/btrfs
After a while, kmemleak reports the leak:
$ cat /sys/kernel/debug/kmemleak
unreferenced object 0xffff880402b13e00 (size 128):
comm "btrfs", pid 19621, jiffies 4341648183 (age 70057.844s)
hex dump (first 32 bytes):
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
00 fc c6 b1 04 88 ff ff 04 00 04 00 ad 4e ad de .............N..
backtrace:
[<ffffffff817275a6>] kmemleak_alloc+0x26/0x50
[<ffffffff8117832b>] kmem_cache_alloc_trace+0xeb/0x1d0
[<ffffffffa04db499>] btrfs_alloc_block_rsv+0x39/0x70 [btrfs]
[<ffffffffa04f8bad>] btrfs_orphan_add+0x13d/0x1b0 [btrfs]
[<ffffffffa04e2b13>] btrfs_remove_block_group+0x143/0x500 [btrfs]
[<ffffffffa0518158>] btrfs_relocate_chunk.isra.63+0x618/0x790 [btrfs]
[<ffffffffa051bc27>] btrfs_balance+0x8f7/0xe90 [btrfs]
[<ffffffffa05240a0>] btrfs_ioctl_balance+0x250/0x550 [btrfs]
[<ffffffffa05269ca>] btrfs_ioctl+0xdfa/0x25f0 [btrfs]
[<ffffffff8119c936>] do_vfs_ioctl+0x96/0x570
[<ffffffff8119cea1>] SyS_ioctl+0x91/0xb0
[<ffffffff81750242>] system_call_fastpath+0x16/0x1b
[<ffffffffffffffff>] 0xffffffffffffffff
This affects btrfs-next, revision be8e3cd00d7293dd177e3f8a4a1645ce09ca3acb
(Btrfs: separate out tests into their own directory).
Signed-off-by: Filipe David Borba Manana <fdmanana@gmail.com>
---
V2: removed atomic_t member in struct btrfs_block_rsv, as suggested by
Josef Bacik, and use instead the condition reserved == 0 to decide
when to free the block.
fs/btrfs/inode.c | 35 ++++++++++++++++++++++++++++-------
1 file changed, 28 insertions(+), 7 deletions(-)
diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c
index b67b81a..905de1e2 100644
--- a/fs/btrfs/inode.c
+++ b/fs/btrfs/inode.c
@@ -2870,8 +2870,13 @@ void btrfs_orphan_commit_root(struct btrfs_trans_handle *trans,
return;
}
- block_rsv = root->orphan_block_rsv;
- root->orphan_block_rsv = NULL;
+ if (root->orphan_block_rsv &&
+ root->orphan_block_rsv->reserved == 0) {
+ block_rsv = root->orphan_block_rsv;
+ root->orphan_block_rsv = NULL;
+ } else {
+ block_rsv = NULL;
+ }
spin_unlock(&root->orphan_lock);
if (root->orphan_item_inserted &&
@@ -2991,6 +2996,9 @@ static int btrfs_orphan_del(struct btrfs_trans_handle *trans,
int ret = 0;
spin_lock(&root->orphan_lock);
+ if (trans)
+ trans->block_rsv = root->orphan_block_rsv;
+
if (test_and_clear_bit(BTRFS_INODE_HAS_ORPHAN_ITEM,
&BTRFS_I(inode)->runtime_flags))
delete_item = 1;
@@ -4520,12 +4528,10 @@ void btrfs_evict_inode(struct inode *inode)
* Errors here aren't a big deal, it just means we leave orphan items
* in the tree. They will be cleaned up on the next mount.
*/
- if (ret == 0) {
- trans->block_rsv = root->orphan_block_rsv;
+ if (ret == 0)
btrfs_orphan_del(trans, inode);
- } else {
+ else
btrfs_orphan_del(NULL, inode);
- }
trans->block_rsv = &root->fs_info->trans_block_rsv;
if (!(root == root->fs_info->tree_root ||
@@ -4535,6 +4541,22 @@ void btrfs_evict_inode(struct inode *inode)
btrfs_end_transaction(trans, root);
btrfs_btree_balance_dirty(root);
no_delete:
+
+ spin_lock(&root->orphan_lock);
+ if (root->orphan_block_rsv &&
+ root->orphan_block_rsv->reserved == 0) {
+ rsv = root->orphan_block_rsv;
+ root->orphan_block_rsv = NULL;
+ } else {
+ rsv = NULL;
+ }
+ spin_unlock(&root->orphan_lock);
+
+ if (rsv) {
+ WARN_ON(rsv->size > 0);
+ btrfs_free_block_rsv(root, rsv);
+ }
+
btrfs_remove_delayed_node(inode);
clear_inode(inode);
return;
@@ -7650,7 +7672,6 @@ static int btrfs_truncate(struct inode *inode)
}
if (ret == 0 && inode->i_nlink > 0) {
- trans->block_rsv = root->orphan_block_rsv;
ret = btrfs_orphan_del(trans, inode);
if (ret)
err = ret;
--
1.7.9.5
^ permalink raw reply related [flat|nested] 13+ messages in thread
* [PATCH v3] Btrfs: fix memory leak of orphan block rsv
2013-08-19 18:08 [RFC PATCH] Btrfs: fix memory leak of orphan block rsv Filipe David Borba Manana
2013-08-19 18:31 ` [PATCH v2] " Filipe David Borba Manana
@ 2013-08-19 20:15 ` Filipe David Borba Manana
2013-08-19 20:30 ` [PATCH v4] " Filipe David Borba Manana
` (2 subsequent siblings)
4 siblings, 0 replies; 13+ messages in thread
From: Filipe David Borba Manana @ 2013-08-19 20:15 UTC (permalink / raw)
To: linux-btrfs; +Cc: jbacik, Filipe David Borba Manana
This issue is simple to reproduce and observe if kmemleak is enabled.
Two simple ways to reproduce it:
** 1
$ mkfs.btrfs -f /dev/loop0
$ mount /dev/loop0 /mnt/btrfs
$ btrfs balance start /mnt/btrfs
$ umount /mnt/btrfs
** 2
$ mkfs.btrfs -f /dev/loop0
$ mount /dev/loop0 /mnt/btrfs
$ touch /mnt/btrfs/foobar
$ rm -f /mnt/btrfs/foobar
$ umount /mnt/btrfs
After a while, kmemleak reports the leak:
$ cat /sys/kernel/debug/kmemleak
unreferenced object 0xffff880402b13e00 (size 128):
comm "btrfs", pid 19621, jiffies 4341648183 (age 70057.844s)
hex dump (first 32 bytes):
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
00 fc c6 b1 04 88 ff ff 04 00 04 00 ad 4e ad de .............N..
backtrace:
[<ffffffff817275a6>] kmemleak_alloc+0x26/0x50
[<ffffffff8117832b>] kmem_cache_alloc_trace+0xeb/0x1d0
[<ffffffffa04db499>] btrfs_alloc_block_rsv+0x39/0x70 [btrfs]
[<ffffffffa04f8bad>] btrfs_orphan_add+0x13d/0x1b0 [btrfs]
[<ffffffffa04e2b13>] btrfs_remove_block_group+0x143/0x500 [btrfs]
[<ffffffffa0518158>] btrfs_relocate_chunk.isra.63+0x618/0x790 [btrfs]
[<ffffffffa051bc27>] btrfs_balance+0x8f7/0xe90 [btrfs]
[<ffffffffa05240a0>] btrfs_ioctl_balance+0x250/0x550 [btrfs]
[<ffffffffa05269ca>] btrfs_ioctl+0xdfa/0x25f0 [btrfs]
[<ffffffff8119c936>] do_vfs_ioctl+0x96/0x570
[<ffffffff8119cea1>] SyS_ioctl+0x91/0xb0
[<ffffffff81750242>] system_call_fastpath+0x16/0x1b
[<ffffffffffffffff>] 0xffffffffffffffff
This affects btrfs-next, revision be8e3cd00d7293dd177e3f8a4a1645ce09ca3acb
(Btrfs: separate out tests into their own directory).
Signed-off-by: Filipe David Borba Manana <fdmanana@gmail.com>
---
V2: removed atomic_t member in struct btrfs_block_rsv, as suggested by
Josef Bacik, and use instead the condition reserved == 0 to decide
when to free the block.
V3: simplified patch, just kfree() (and not btrfs_free_block_rsv) the
root's orphan_block_rsv when free'ing the root. Thanks Josef for
the suggestion.
fs/btrfs/disk-io.c | 1 +
1 file changed, 1 insertion(+)
diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c
index 3b12c26..c7d2fd8 100644
--- a/fs/btrfs/disk-io.c
+++ b/fs/btrfs/disk-io.c
@@ -3434,6 +3434,7 @@ static void free_fs_root(struct btrfs_root *root)
free_anon_bdev(root->anon_dev);
free_extent_buffer(root->node);
free_extent_buffer(root->commit_root);
+ kfree(root->orphan_block_rsv);
kfree(root->free_ino_ctl);
kfree(root->free_ino_pinned);
kfree(root->name);
--
1.7.9.5
^ permalink raw reply related [flat|nested] 13+ messages in thread
* [PATCH v4] Btrfs: fix memory leak of orphan block rsv
2013-08-19 18:08 [RFC PATCH] Btrfs: fix memory leak of orphan block rsv Filipe David Borba Manana
2013-08-19 18:31 ` [PATCH v2] " Filipe David Borba Manana
2013-08-19 20:15 ` [PATCH v3] " Filipe David Borba Manana
@ 2013-08-19 20:30 ` Filipe David Borba Manana
2013-08-19 20:42 ` [PATCH v5] " Filipe David Borba Manana
2013-08-19 23:52 ` [PATCH v6] " Filipe David Borba Manana
4 siblings, 0 replies; 13+ messages in thread
From: Filipe David Borba Manana @ 2013-08-19 20:30 UTC (permalink / raw)
To: linux-btrfs; +Cc: jbacik, Filipe David Borba Manana
This issue is simple to reproduce and observe if kmemleak is enabled.
Two simple ways to reproduce it:
** 1
$ mkfs.btrfs -f /dev/loop0
$ mount /dev/loop0 /mnt/btrfs
$ btrfs balance start /mnt/btrfs
$ umount /mnt/btrfs
** 2
$ mkfs.btrfs -f /dev/loop0
$ mount /dev/loop0 /mnt/btrfs
$ touch /mnt/btrfs/foobar
$ rm -f /mnt/btrfs/foobar
$ umount /mnt/btrfs
After a while, kmemleak reports the leak:
$ cat /sys/kernel/debug/kmemleak
unreferenced object 0xffff880402b13e00 (size 128):
comm "btrfs", pid 19621, jiffies 4341648183 (age 70057.844s)
hex dump (first 32 bytes):
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
00 fc c6 b1 04 88 ff ff 04 00 04 00 ad 4e ad de .............N..
backtrace:
[<ffffffff817275a6>] kmemleak_alloc+0x26/0x50
[<ffffffff8117832b>] kmem_cache_alloc_trace+0xeb/0x1d0
[<ffffffffa04db499>] btrfs_alloc_block_rsv+0x39/0x70 [btrfs]
[<ffffffffa04f8bad>] btrfs_orphan_add+0x13d/0x1b0 [btrfs]
[<ffffffffa04e2b13>] btrfs_remove_block_group+0x143/0x500 [btrfs]
[<ffffffffa0518158>] btrfs_relocate_chunk.isra.63+0x618/0x790 [btrfs]
[<ffffffffa051bc27>] btrfs_balance+0x8f7/0xe90 [btrfs]
[<ffffffffa05240a0>] btrfs_ioctl_balance+0x250/0x550 [btrfs]
[<ffffffffa05269ca>] btrfs_ioctl+0xdfa/0x25f0 [btrfs]
[<ffffffff8119c936>] do_vfs_ioctl+0x96/0x570
[<ffffffff8119cea1>] SyS_ioctl+0x91/0xb0
[<ffffffff81750242>] system_call_fastpath+0x16/0x1b
[<ffffffffffffffff>] 0xffffffffffffffff
This affects btrfs-next, revision be8e3cd00d7293dd177e3f8a4a1645ce09ca3acb
(Btrfs: separate out tests into their own directory).
Signed-off-by: Filipe David Borba Manana <fdmanana@gmail.com>
---
V2: removed atomic_t member in struct btrfs_block_rsv, as suggested by
Josef Bacik, and use instead the condition reserved == 0 to decide
when to free the block.
V3: simplified patch, just kfree() (and not btrfs_free_block_rsv) the
root's orphan_block_rsv when free'ing the root. Thanks Josef for
the suggestion.
V4: use btrfs_free_block_rsv() instead of kfree(). The error I was getting
in xfstests when using btrfs_free_block_rsv() was unrelated, Josef just
pointed it to me (separate issue).
fs/btrfs/disk-io.c | 2 ++
1 file changed, 2 insertions(+)
diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c
index 3b12c26..c4b90a0 100644
--- a/fs/btrfs/disk-io.c
+++ b/fs/btrfs/disk-io.c
@@ -3428,6 +3428,8 @@ void btrfs_drop_and_free_fs_root(struct btrfs_fs_info *fs_info,
static void free_fs_root(struct btrfs_root *root)
{
+ btrfs_free_block_rsv(root, root->orphan_block_rsv);
+ root->orphan_block_rsv = NULL;
iput(root->cache_inode);
WARN_ON(!RB_EMPTY_ROOT(&root->inode_tree));
if (root->anon_dev)
--
1.7.9.5
^ permalink raw reply related [flat|nested] 13+ messages in thread
* [PATCH v5] Btrfs: fix memory leak of orphan block rsv
2013-08-19 18:08 [RFC PATCH] Btrfs: fix memory leak of orphan block rsv Filipe David Borba Manana
` (2 preceding siblings ...)
2013-08-19 20:30 ` [PATCH v4] " Filipe David Borba Manana
@ 2013-08-19 20:42 ` Filipe David Borba Manana
2013-08-19 23:52 ` [PATCH v6] " Filipe David Borba Manana
4 siblings, 0 replies; 13+ messages in thread
From: Filipe David Borba Manana @ 2013-08-19 20:42 UTC (permalink / raw)
To: linux-btrfs; +Cc: jbacik, Filipe David Borba Manana
This issue is simple to reproduce and observe if kmemleak is enabled.
Two simple ways to reproduce it:
** 1
$ mkfs.btrfs -f /dev/loop0
$ mount /dev/loop0 /mnt/btrfs
$ btrfs balance start /mnt/btrfs
$ umount /mnt/btrfs
** 2
$ mkfs.btrfs -f /dev/loop0
$ mount /dev/loop0 /mnt/btrfs
$ touch /mnt/btrfs/foobar
$ rm -f /mnt/btrfs/foobar
$ umount /mnt/btrfs
After a while, kmemleak reports the leak:
$ cat /sys/kernel/debug/kmemleak
unreferenced object 0xffff880402b13e00 (size 128):
comm "btrfs", pid 19621, jiffies 4341648183 (age 70057.844s)
hex dump (first 32 bytes):
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
00 fc c6 b1 04 88 ff ff 04 00 04 00 ad 4e ad de .............N..
backtrace:
[<ffffffff817275a6>] kmemleak_alloc+0x26/0x50
[<ffffffff8117832b>] kmem_cache_alloc_trace+0xeb/0x1d0
[<ffffffffa04db499>] btrfs_alloc_block_rsv+0x39/0x70 [btrfs]
[<ffffffffa04f8bad>] btrfs_orphan_add+0x13d/0x1b0 [btrfs]
[<ffffffffa04e2b13>] btrfs_remove_block_group+0x143/0x500 [btrfs]
[<ffffffffa0518158>] btrfs_relocate_chunk.isra.63+0x618/0x790 [btrfs]
[<ffffffffa051bc27>] btrfs_balance+0x8f7/0xe90 [btrfs]
[<ffffffffa05240a0>] btrfs_ioctl_balance+0x250/0x550 [btrfs]
[<ffffffffa05269ca>] btrfs_ioctl+0xdfa/0x25f0 [btrfs]
[<ffffffff8119c936>] do_vfs_ioctl+0x96/0x570
[<ffffffff8119cea1>] SyS_ioctl+0x91/0xb0
[<ffffffff81750242>] system_call_fastpath+0x16/0x1b
[<ffffffffffffffff>] 0xffffffffffffffff
This affects btrfs-next, revision be8e3cd00d7293dd177e3f8a4a1645ce09ca3acb
(Btrfs: separate out tests into their own directory).
Signed-off-by: Filipe David Borba Manana <fdmanana@gmail.com>
---
V2: removed atomic_t member in struct btrfs_block_rsv, as suggested by
Josef Bacik, and use instead the condition reserved == 0 to decide
when to free the block.
V3: simplified patch, just kfree() (and not btrfs_free_block_rsv) the
root's orphan_block_rsv when free'ing the root. Thanks Josef for
the suggestion.
V4: use btrfs_free_block_rsv() instead of kfree(). The error I was getting
in xfstests when using btrfs_free_block_rsv() was unrelated, Josef just
pointed it to me (separate issue).
V5: move the free call below the iput() call, so that btrfs_evict_node()
can process the orphan_block_rsv first to do some needed cleanup before
we free it.
fs/btrfs/disk-io.c | 2 ++
1 file changed, 2 insertions(+)
diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c
index 3b12c26..9e66543 100644
--- a/fs/btrfs/disk-io.c
+++ b/fs/btrfs/disk-io.c
@@ -3430,6 +3430,8 @@ static void free_fs_root(struct btrfs_root *root)
{
iput(root->cache_inode);
WARN_ON(!RB_EMPTY_ROOT(&root->inode_tree));
+ btrfs_free_block_rsv(root, root->orphan_block_rsv);
+ root->orphan_block_rsv = NULL;
if (root->anon_dev)
free_anon_bdev(root->anon_dev);
free_extent_buffer(root->node);
--
1.7.9.5
^ permalink raw reply related [flat|nested] 13+ messages in thread
* [PATCH v6] Btrfs: fix memory leak of orphan block rsv
2013-08-19 18:08 [RFC PATCH] Btrfs: fix memory leak of orphan block rsv Filipe David Borba Manana
` (3 preceding siblings ...)
2013-08-19 20:42 ` [PATCH v5] " Filipe David Borba Manana
@ 2013-08-19 23:52 ` Filipe David Borba Manana
2013-10-23 13:33 ` Alex Lyakas
4 siblings, 1 reply; 13+ messages in thread
From: Filipe David Borba Manana @ 2013-08-19 23:52 UTC (permalink / raw)
To: linux-btrfs; +Cc: jbacik, Filipe David Borba Manana
This issue is simple to reproduce and observe if kmemleak is enabled.
Two simple ways to reproduce it:
** 1
$ mkfs.btrfs -f /dev/loop0
$ mount /dev/loop0 /mnt/btrfs
$ btrfs balance start /mnt/btrfs
$ umount /mnt/btrfs
** 2
$ mkfs.btrfs -f /dev/loop0
$ mount /dev/loop0 /mnt/btrfs
$ touch /mnt/btrfs/foobar
$ rm -f /mnt/btrfs/foobar
$ umount /mnt/btrfs
After a while, kmemleak reports the leak:
$ cat /sys/kernel/debug/kmemleak
unreferenced object 0xffff880402b13e00 (size 128):
comm "btrfs", pid 19621, jiffies 4341648183 (age 70057.844s)
hex dump (first 32 bytes):
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
00 fc c6 b1 04 88 ff ff 04 00 04 00 ad 4e ad de .............N..
backtrace:
[<ffffffff817275a6>] kmemleak_alloc+0x26/0x50
[<ffffffff8117832b>] kmem_cache_alloc_trace+0xeb/0x1d0
[<ffffffffa04db499>] btrfs_alloc_block_rsv+0x39/0x70 [btrfs]
[<ffffffffa04f8bad>] btrfs_orphan_add+0x13d/0x1b0 [btrfs]
[<ffffffffa04e2b13>] btrfs_remove_block_group+0x143/0x500 [btrfs]
[<ffffffffa0518158>] btrfs_relocate_chunk.isra.63+0x618/0x790 [btrfs]
[<ffffffffa051bc27>] btrfs_balance+0x8f7/0xe90 [btrfs]
[<ffffffffa05240a0>] btrfs_ioctl_balance+0x250/0x550 [btrfs]
[<ffffffffa05269ca>] btrfs_ioctl+0xdfa/0x25f0 [btrfs]
[<ffffffff8119c936>] do_vfs_ioctl+0x96/0x570
[<ffffffff8119cea1>] SyS_ioctl+0x91/0xb0
[<ffffffff81750242>] system_call_fastpath+0x16/0x1b
[<ffffffffffffffff>] 0xffffffffffffffff
This affects btrfs-next, revision be8e3cd00d7293dd177e3f8a4a1645ce09ca3acb
(Btrfs: separate out tests into their own directory).
Signed-off-by: Filipe David Borba Manana <fdmanana@gmail.com>
---
V2: removed atomic_t member in struct btrfs_block_rsv, as suggested by
Josef Bacik, and use instead the condition reserved == 0 to decide
when to free the block.
V3: simplified patch, just kfree() (and not btrfs_free_block_rsv) the
root's orphan_block_rsv when free'ing the root. Thanks Josef for
the suggestion.
V4: use btrfs_free_block_rsv() instead of kfree(). The error I was getting
in xfstests when using btrfs_free_block_rsv() was unrelated, Josef just
pointed it to me (separate issue).
V5: move the free call below the iput() call, so that btrfs_evict_node()
can process the orphan_block_rsv first to do some needed cleanup before
we free it.
V6: free the root's orphan_block_rsv in close_ctree() too. After a balance
the orphan_block_rsv of the tree of tree roots was being leaked, because
free_fs_root() is only called for filesystem trees.
fs/btrfs/disk-io.c | 5 +++++
1 file changed, 5 insertions(+)
diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c
index 3b12c26..5d17163 100644
--- a/fs/btrfs/disk-io.c
+++ b/fs/btrfs/disk-io.c
@@ -3430,6 +3430,8 @@ static void free_fs_root(struct btrfs_root *root)
{
iput(root->cache_inode);
WARN_ON(!RB_EMPTY_ROOT(&root->inode_tree));
+ btrfs_free_block_rsv(root, root->orphan_block_rsv);
+ root->orphan_block_rsv = NULL;
if (root->anon_dev)
free_anon_bdev(root->anon_dev);
free_extent_buffer(root->node);
@@ -3582,6 +3584,9 @@ int close_ctree(struct btrfs_root *root)
btrfs_free_stripe_hash_table(fs_info);
+ btrfs_free_block_rsv(root, root->orphan_block_rsv);
+ root->orphan_block_rsv = NULL;
+
return 0;
}
--
1.7.9.5
^ permalink raw reply related [flat|nested] 13+ messages in thread
* Re: [PATCH v6] Btrfs: fix memory leak of orphan block rsv
2013-08-19 23:52 ` [PATCH v6] " Filipe David Borba Manana
@ 2013-10-23 13:33 ` Alex Lyakas
2013-10-23 13:35 ` Filipe David Manana
0 siblings, 1 reply; 13+ messages in thread
From: Alex Lyakas @ 2013-10-23 13:33 UTC (permalink / raw)
To: Filipe David Borba Manana; +Cc: linux-btrfs
Hi Filipe,
On Tue, Aug 20, 2013 at 2:52 AM, Filipe David Borba Manana
<fdmanana@gmail.com> wrote:
>
> This issue is simple to reproduce and observe if kmemleak is enabled.
> Two simple ways to reproduce it:
>
> ** 1
>
> $ mkfs.btrfs -f /dev/loop0
> $ mount /dev/loop0 /mnt/btrfs
> $ btrfs balance start /mnt/btrfs
> $ umount /mnt/btrfs
>
> ** 2
>
> $ mkfs.btrfs -f /dev/loop0
> $ mount /dev/loop0 /mnt/btrfs
> $ touch /mnt/btrfs/foobar
> $ rm -f /mnt/btrfs/foobar
> $ umount /mnt/btrfs
I tried the second repro script on kernel 3.8.13, and kmemleak does
not report a leak (even if I force the kmemleak scan). I did not try
the balance-repro script, though. Am I missing something?
Thanks,
Alex.
>
>
> After a while, kmemleak reports the leak:
>
> $ cat /sys/kernel/debug/kmemleak
> unreferenced object 0xffff880402b13e00 (size 128):
> comm "btrfs", pid 19621, jiffies 4341648183 (age 70057.844s)
> hex dump (first 32 bytes):
> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> 00 fc c6 b1 04 88 ff ff 04 00 04 00 ad 4e ad de .............N..
> backtrace:
> [<ffffffff817275a6>] kmemleak_alloc+0x26/0x50
> [<ffffffff8117832b>] kmem_cache_alloc_trace+0xeb/0x1d0
> [<ffffffffa04db499>] btrfs_alloc_block_rsv+0x39/0x70 [btrfs]
> [<ffffffffa04f8bad>] btrfs_orphan_add+0x13d/0x1b0 [btrfs]
> [<ffffffffa04e2b13>] btrfs_remove_block_group+0x143/0x500 [btrfs]
> [<ffffffffa0518158>] btrfs_relocate_chunk.isra.63+0x618/0x790 [btrfs]
> [<ffffffffa051bc27>] btrfs_balance+0x8f7/0xe90 [btrfs]
> [<ffffffffa05240a0>] btrfs_ioctl_balance+0x250/0x550 [btrfs]
> [<ffffffffa05269ca>] btrfs_ioctl+0xdfa/0x25f0 [btrfs]
> [<ffffffff8119c936>] do_vfs_ioctl+0x96/0x570
> [<ffffffff8119cea1>] SyS_ioctl+0x91/0xb0
> [<ffffffff81750242>] system_call_fastpath+0x16/0x1b
> [<ffffffffffffffff>] 0xffffffffffffffff
>
> This affects btrfs-next, revision be8e3cd00d7293dd177e3f8a4a1645ce09ca3acb
> (Btrfs: separate out tests into their own directory).
>
> Signed-off-by: Filipe David Borba Manana <fdmanana@gmail.com>
> ---
>
> V2: removed atomic_t member in struct btrfs_block_rsv, as suggested by
> Josef Bacik, and use instead the condition reserved == 0 to decide
> when to free the block.
> V3: simplified patch, just kfree() (and not btrfs_free_block_rsv) the
> root's orphan_block_rsv when free'ing the root. Thanks Josef for
> the suggestion.
> V4: use btrfs_free_block_rsv() instead of kfree(). The error I was getting
> in xfstests when using btrfs_free_block_rsv() was unrelated, Josef just
> pointed it to me (separate issue).
> V5: move the free call below the iput() call, so that btrfs_evict_node()
> can process the orphan_block_rsv first to do some needed cleanup before
> we free it.
> V6: free the root's orphan_block_rsv in close_ctree() too. After a balance
> the orphan_block_rsv of the tree of tree roots was being leaked, because
> free_fs_root() is only called for filesystem trees.
>
> fs/btrfs/disk-io.c | 5 +++++
> 1 file changed, 5 insertions(+)
>
> diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c
> index 3b12c26..5d17163 100644
> --- a/fs/btrfs/disk-io.c
> +++ b/fs/btrfs/disk-io.c
> @@ -3430,6 +3430,8 @@ static void free_fs_root(struct btrfs_root *root)
> {
> iput(root->cache_inode);
> WARN_ON(!RB_EMPTY_ROOT(&root->inode_tree));
> + btrfs_free_block_rsv(root, root->orphan_block_rsv);
> + root->orphan_block_rsv = NULL;
> if (root->anon_dev)
> free_anon_bdev(root->anon_dev);
> free_extent_buffer(root->node);
> @@ -3582,6 +3584,9 @@ int close_ctree(struct btrfs_root *root)
>
> btrfs_free_stripe_hash_table(fs_info);
>
> + btrfs_free_block_rsv(root, root->orphan_block_rsv);
> + root->orphan_block_rsv = NULL;
> +
> return 0;
> }
>
> --
> 1.7.9.5
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH v6] Btrfs: fix memory leak of orphan block rsv
2013-10-23 13:33 ` Alex Lyakas
@ 2013-10-23 13:35 ` Filipe David Manana
2013-10-23 14:14 ` Alex Lyakas
0 siblings, 1 reply; 13+ messages in thread
From: Filipe David Manana @ 2013-10-23 13:35 UTC (permalink / raw)
To: Alex Lyakas; +Cc: linux-btrfs
On Wed, Oct 23, 2013 at 2:33 PM, Alex Lyakas
<alex.btrfs@zadarastorage.com> wrote:
> Hi Filipe,
>
>
> On Tue, Aug 20, 2013 at 2:52 AM, Filipe David Borba Manana
> <fdmanana@gmail.com> wrote:
>>
>> This issue is simple to reproduce and observe if kmemleak is enabled.
>> Two simple ways to reproduce it:
>>
>> ** 1
>>
>> $ mkfs.btrfs -f /dev/loop0
>> $ mount /dev/loop0 /mnt/btrfs
>> $ btrfs balance start /mnt/btrfs
>> $ umount /mnt/btrfs
>>
>> ** 2
>>
>> $ mkfs.btrfs -f /dev/loop0
>> $ mount /dev/loop0 /mnt/btrfs
>> $ touch /mnt/btrfs/foobar
>> $ rm -f /mnt/btrfs/foobar
>> $ umount /mnt/btrfs
>
>
> I tried the second repro script on kernel 3.8.13, and kmemleak does
> not report a leak (even if I force the kmemleak scan). I did not try
> the balance-repro script, though. Am I missing something?
Maybe it's not an issue on 3.8.13 and older releases.
This was on btrfs-next from August 19.
thanks for testing
>
> Thanks,
> Alex.
>
>
>>
>>
>> After a while, kmemleak reports the leak:
>>
>> $ cat /sys/kernel/debug/kmemleak
>> unreferenced object 0xffff880402b13e00 (size 128):
>> comm "btrfs", pid 19621, jiffies 4341648183 (age 70057.844s)
>> hex dump (first 32 bytes):
>> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
>> 00 fc c6 b1 04 88 ff ff 04 00 04 00 ad 4e ad de .............N..
>> backtrace:
>> [<ffffffff817275a6>] kmemleak_alloc+0x26/0x50
>> [<ffffffff8117832b>] kmem_cache_alloc_trace+0xeb/0x1d0
>> [<ffffffffa04db499>] btrfs_alloc_block_rsv+0x39/0x70 [btrfs]
>> [<ffffffffa04f8bad>] btrfs_orphan_add+0x13d/0x1b0 [btrfs]
>> [<ffffffffa04e2b13>] btrfs_remove_block_group+0x143/0x500 [btrfs]
>> [<ffffffffa0518158>] btrfs_relocate_chunk.isra.63+0x618/0x790 [btrfs]
>> [<ffffffffa051bc27>] btrfs_balance+0x8f7/0xe90 [btrfs]
>> [<ffffffffa05240a0>] btrfs_ioctl_balance+0x250/0x550 [btrfs]
>> [<ffffffffa05269ca>] btrfs_ioctl+0xdfa/0x25f0 [btrfs]
>> [<ffffffff8119c936>] do_vfs_ioctl+0x96/0x570
>> [<ffffffff8119cea1>] SyS_ioctl+0x91/0xb0
>> [<ffffffff81750242>] system_call_fastpath+0x16/0x1b
>> [<ffffffffffffffff>] 0xffffffffffffffff
>>
>> This affects btrfs-next, revision be8e3cd00d7293dd177e3f8a4a1645ce09ca3acb
>> (Btrfs: separate out tests into their own directory).
>>
>> Signed-off-by: Filipe David Borba Manana <fdmanana@gmail.com>
>> ---
>>
>> V2: removed atomic_t member in struct btrfs_block_rsv, as suggested by
>> Josef Bacik, and use instead the condition reserved == 0 to decide
>> when to free the block.
>> V3: simplified patch, just kfree() (and not btrfs_free_block_rsv) the
>> root's orphan_block_rsv when free'ing the root. Thanks Josef for
>> the suggestion.
>> V4: use btrfs_free_block_rsv() instead of kfree(). The error I was getting
>> in xfstests when using btrfs_free_block_rsv() was unrelated, Josef just
>> pointed it to me (separate issue).
>> V5: move the free call below the iput() call, so that btrfs_evict_node()
>> can process the orphan_block_rsv first to do some needed cleanup before
>> we free it.
>> V6: free the root's orphan_block_rsv in close_ctree() too. After a balance
>> the orphan_block_rsv of the tree of tree roots was being leaked, because
>> free_fs_root() is only called for filesystem trees.
>>
>> fs/btrfs/disk-io.c | 5 +++++
>> 1 file changed, 5 insertions(+)
>>
>> diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c
>> index 3b12c26..5d17163 100644
>> --- a/fs/btrfs/disk-io.c
>> +++ b/fs/btrfs/disk-io.c
>> @@ -3430,6 +3430,8 @@ static void free_fs_root(struct btrfs_root *root)
>> {
>> iput(root->cache_inode);
>> WARN_ON(!RB_EMPTY_ROOT(&root->inode_tree));
>> + btrfs_free_block_rsv(root, root->orphan_block_rsv);
>> + root->orphan_block_rsv = NULL;
>> if (root->anon_dev)
>> free_anon_bdev(root->anon_dev);
>> free_extent_buffer(root->node);
>> @@ -3582,6 +3584,9 @@ int close_ctree(struct btrfs_root *root)
>>
>> btrfs_free_stripe_hash_table(fs_info);
>>
>> + btrfs_free_block_rsv(root, root->orphan_block_rsv);
>> + root->orphan_block_rsv = NULL;
>> +
>> return 0;
>> }
>>
>> --
>> 1.7.9.5
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at http://vger.kernel.org/majordomo-info.html
--
Filipe David Manana,
"Reasonable men adapt themselves to the world.
Unreasonable men adapt the world to themselves.
That's why all progress depends on unreasonable men."
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH v6] Btrfs: fix memory leak of orphan block rsv
2013-10-23 13:35 ` Filipe David Manana
@ 2013-10-23 14:14 ` Alex Lyakas
2013-10-23 14:26 ` Filipe David Manana
0 siblings, 1 reply; 13+ messages in thread
From: Alex Lyakas @ 2013-10-23 14:14 UTC (permalink / raw)
To: Filipe David Borba Manana; +Cc: linux-btrfs
Hello,
On Wed, Oct 23, 2013 at 4:35 PM, Filipe David Manana <fdmanana@gmail.com> wrote:
> On Wed, Oct 23, 2013 at 2:33 PM, Alex Lyakas
> <alex.btrfs@zadarastorage.com> wrote:
>> Hi Filipe,
>>
>>
>> On Tue, Aug 20, 2013 at 2:52 AM, Filipe David Borba Manana
>> <fdmanana@gmail.com> wrote:
>>>
>>> This issue is simple to reproduce and observe if kmemleak is enabled.
>>> Two simple ways to reproduce it:
>>>
>>> ** 1
>>>
>>> $ mkfs.btrfs -f /dev/loop0
>>> $ mount /dev/loop0 /mnt/btrfs
>>> $ btrfs balance start /mnt/btrfs
>>> $ umount /mnt/btrfs
So here it seems that the leak can only happen in case the block-group
has a free-space inode. This is what the orphan item is added for.
Yes, here kmemleak reports.
But: if space_cache option is disabled (and nospace_cache) enabled, it
seems that btrfs still creates the FREE_SPACE inodes, although they
are empty because in cache_save_setup:
inode = lookup_free_space_inode(root, block_group, path);
if (IS_ERR(inode) && PTR_ERR(inode) != -ENOENT) {
ret = PTR_ERR(inode);
btrfs_release_path(path);
goto out;
}
if (IS_ERR(inode)) {
...
ret = create_free_space_inode(root, trans, block_group, path);
and only later it actually sets BTRFS_DC_WRITTEN if space_cache option
is disabled. Amazing!
Although this is a different issue, do you know perhaps why these
empty inodes are needed?
Thanks!
Alex.
>>>
>>> ** 2
>>>
>>> $ mkfs.btrfs -f /dev/loop0
>>> $ mount /dev/loop0 /mnt/btrfs
>>> $ touch /mnt/btrfs/foobar
>>> $ rm -f /mnt/btrfs/foobar
>>> $ umount /mnt/btrfs
>>
>>
>> I tried the second repro script on kernel 3.8.13, and kmemleak does
>> not report a leak (even if I force the kmemleak scan). I did not try
>> the balance-repro script, though. Am I missing something?
>
> Maybe it's not an issue on 3.8.13 and older releases.
> This was on btrfs-next from August 19.
>
> thanks for testing
>
>>
>> Thanks,
>> Alex.
>>
>>
>>>
>>>
>>> After a while, kmemleak reports the leak:
>>>
>>> $ cat /sys/kernel/debug/kmemleak
>>> unreferenced object 0xffff880402b13e00 (size 128):
>>> comm "btrfs", pid 19621, jiffies 4341648183 (age 70057.844s)
>>> hex dump (first 32 bytes):
>>> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
>>> 00 fc c6 b1 04 88 ff ff 04 00 04 00 ad 4e ad de .............N..
>>> backtrace:
>>> [<ffffffff817275a6>] kmemleak_alloc+0x26/0x50
>>> [<ffffffff8117832b>] kmem_cache_alloc_trace+0xeb/0x1d0
>>> [<ffffffffa04db499>] btrfs_alloc_block_rsv+0x39/0x70 [btrfs]
>>> [<ffffffffa04f8bad>] btrfs_orphan_add+0x13d/0x1b0 [btrfs]
>>> [<ffffffffa04e2b13>] btrfs_remove_block_group+0x143/0x500 [btrfs]
>>> [<ffffffffa0518158>] btrfs_relocate_chunk.isra.63+0x618/0x790 [btrfs]
>>> [<ffffffffa051bc27>] btrfs_balance+0x8f7/0xe90 [btrfs]
>>> [<ffffffffa05240a0>] btrfs_ioctl_balance+0x250/0x550 [btrfs]
>>> [<ffffffffa05269ca>] btrfs_ioctl+0xdfa/0x25f0 [btrfs]
>>> [<ffffffff8119c936>] do_vfs_ioctl+0x96/0x570
>>> [<ffffffff8119cea1>] SyS_ioctl+0x91/0xb0
>>> [<ffffffff81750242>] system_call_fastpath+0x16/0x1b
>>> [<ffffffffffffffff>] 0xffffffffffffffff
>>>
>>> This affects btrfs-next, revision be8e3cd00d7293dd177e3f8a4a1645ce09ca3acb
>>> (Btrfs: separate out tests into their own directory).
>>>
>>> Signed-off-by: Filipe David Borba Manana <fdmanana@gmail.com>
>>> ---
>>>
>>> V2: removed atomic_t member in struct btrfs_block_rsv, as suggested by
>>> Josef Bacik, and use instead the condition reserved == 0 to decide
>>> when to free the block.
>>> V3: simplified patch, just kfree() (and not btrfs_free_block_rsv) the
>>> root's orphan_block_rsv when free'ing the root. Thanks Josef for
>>> the suggestion.
>>> V4: use btrfs_free_block_rsv() instead of kfree(). The error I was getting
>>> in xfstests when using btrfs_free_block_rsv() was unrelated, Josef just
>>> pointed it to me (separate issue).
>>> V5: move the free call below the iput() call, so that btrfs_evict_node()
>>> can process the orphan_block_rsv first to do some needed cleanup before
>>> we free it.
>>> V6: free the root's orphan_block_rsv in close_ctree() too. After a balance
>>> the orphan_block_rsv of the tree of tree roots was being leaked, because
>>> free_fs_root() is only called for filesystem trees.
>>>
>>> fs/btrfs/disk-io.c | 5 +++++
>>> 1 file changed, 5 insertions(+)
>>>
>>> diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c
>>> index 3b12c26..5d17163 100644
>>> --- a/fs/btrfs/disk-io.c
>>> +++ b/fs/btrfs/disk-io.c
>>> @@ -3430,6 +3430,8 @@ static void free_fs_root(struct btrfs_root *root)
>>> {
>>> iput(root->cache_inode);
>>> WARN_ON(!RB_EMPTY_ROOT(&root->inode_tree));
>>> + btrfs_free_block_rsv(root, root->orphan_block_rsv);
>>> + root->orphan_block_rsv = NULL;
>>> if (root->anon_dev)
>>> free_anon_bdev(root->anon_dev);
>>> free_extent_buffer(root->node);
>>> @@ -3582,6 +3584,9 @@ int close_ctree(struct btrfs_root *root)
>>>
>>> btrfs_free_stripe_hash_table(fs_info);
>>>
>>> + btrfs_free_block_rsv(root, root->orphan_block_rsv);
>>> + root->orphan_block_rsv = NULL;
>>> +
>>> return 0;
>>> }
>>>
>>> --
>>> 1.7.9.5
>>>
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
>>> the body of a message to majordomo@vger.kernel.org
>>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
>
>
> --
> Filipe David Manana,
>
> "Reasonable men adapt themselves to the world.
> Unreasonable men adapt the world to themselves.
> That's why all progress depends on unreasonable men."
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH v6] Btrfs: fix memory leak of orphan block rsv
2013-10-23 14:14 ` Alex Lyakas
@ 2013-10-23 14:26 ` Filipe David Manana
2013-11-04 12:16 ` Alex Lyakas
0 siblings, 1 reply; 13+ messages in thread
From: Filipe David Manana @ 2013-10-23 14:26 UTC (permalink / raw)
To: Alex Lyakas; +Cc: linux-btrfs
On Wed, Oct 23, 2013 at 3:14 PM, Alex Lyakas
<alex.btrfs@zadarastorage.com> wrote:
> Hello,
>
> On Wed, Oct 23, 2013 at 4:35 PM, Filipe David Manana <fdmanana@gmail.com> wrote:
>> On Wed, Oct 23, 2013 at 2:33 PM, Alex Lyakas
>> <alex.btrfs@zadarastorage.com> wrote:
>>> Hi Filipe,
>>>
>>>
>>> On Tue, Aug 20, 2013 at 2:52 AM, Filipe David Borba Manana
>>> <fdmanana@gmail.com> wrote:
>>>>
>>>> This issue is simple to reproduce and observe if kmemleak is enabled.
>>>> Two simple ways to reproduce it:
>>>>
>>>> ** 1
>>>>
>>>> $ mkfs.btrfs -f /dev/loop0
>>>> $ mount /dev/loop0 /mnt/btrfs
>>>> $ btrfs balance start /mnt/btrfs
>>>> $ umount /mnt/btrfs
>
> So here it seems that the leak can only happen in case the block-group
> has a free-space inode. This is what the orphan item is added for.
> Yes, here kmemleak reports.
> But: if space_cache option is disabled (and nospace_cache) enabled, it
> seems that btrfs still creates the FREE_SPACE inodes, although they
> are empty because in cache_save_setup:
>
> inode = lookup_free_space_inode(root, block_group, path);
> if (IS_ERR(inode) && PTR_ERR(inode) != -ENOENT) {
> ret = PTR_ERR(inode);
> btrfs_release_path(path);
> goto out;
> }
>
> if (IS_ERR(inode)) {
> ...
> ret = create_free_space_inode(root, trans, block_group, path);
>
> and only later it actually sets BTRFS_DC_WRITTEN if space_cache option
> is disabled. Amazing!
> Although this is a different issue, do you know perhaps why these
> empty inodes are needed?
Don't know if they are needed. But you have a point, it seems odd to
create the free space cache inode if mount option nospace_cache was
supplied. Thanks Alex. Testing the following patch:
diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c
index c43ee8a..eb1b7da 100644
--- a/fs/btrfs/extent-tree.c
+++ b/fs/btrfs/extent-tree.c
@@ -3162,6 +3162,9 @@ static int cache_save_setup(struct
btrfs_block_group_cache *block_group,
int retries = 0;
int ret = 0;
+ if (!btrfs_test_opt(root, SPACE_CACHE))
+ return 0;
+
/*
* If this block group is smaller than 100 megs don't bother caching the
* block group.
>
> Thanks!
> Alex.
>
>
>
>>>>
>>>> ** 2
>>>>
>>>> $ mkfs.btrfs -f /dev/loop0
>>>> $ mount /dev/loop0 /mnt/btrfs
>>>> $ touch /mnt/btrfs/foobar
>>>> $ rm -f /mnt/btrfs/foobar
>>>> $ umount /mnt/btrfs
>>>
>>>
>>> I tried the second repro script on kernel 3.8.13, and kmemleak does
>>> not report a leak (even if I force the kmemleak scan). I did not try
>>> the balance-repro script, though. Am I missing something?
>>
>> Maybe it's not an issue on 3.8.13 and older releases.
>> This was on btrfs-next from August 19.
>>
>> thanks for testing
>>
>>>
>>> Thanks,
>>> Alex.
>>>
>>>
>>>>
>>>>
>>>> After a while, kmemleak reports the leak:
>>>>
>>>> $ cat /sys/kernel/debug/kmemleak
>>>> unreferenced object 0xffff880402b13e00 (size 128):
>>>> comm "btrfs", pid 19621, jiffies 4341648183 (age 70057.844s)
>>>> hex dump (first 32 bytes):
>>>> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
>>>> 00 fc c6 b1 04 88 ff ff 04 00 04 00 ad 4e ad de .............N..
>>>> backtrace:
>>>> [<ffffffff817275a6>] kmemleak_alloc+0x26/0x50
>>>> [<ffffffff8117832b>] kmem_cache_alloc_trace+0xeb/0x1d0
>>>> [<ffffffffa04db499>] btrfs_alloc_block_rsv+0x39/0x70 [btrfs]
>>>> [<ffffffffa04f8bad>] btrfs_orphan_add+0x13d/0x1b0 [btrfs]
>>>> [<ffffffffa04e2b13>] btrfs_remove_block_group+0x143/0x500 [btrfs]
>>>> [<ffffffffa0518158>] btrfs_relocate_chunk.isra.63+0x618/0x790 [btrfs]
>>>> [<ffffffffa051bc27>] btrfs_balance+0x8f7/0xe90 [btrfs]
>>>> [<ffffffffa05240a0>] btrfs_ioctl_balance+0x250/0x550 [btrfs]
>>>> [<ffffffffa05269ca>] btrfs_ioctl+0xdfa/0x25f0 [btrfs]
>>>> [<ffffffff8119c936>] do_vfs_ioctl+0x96/0x570
>>>> [<ffffffff8119cea1>] SyS_ioctl+0x91/0xb0
>>>> [<ffffffff81750242>] system_call_fastpath+0x16/0x1b
>>>> [<ffffffffffffffff>] 0xffffffffffffffff
>>>>
>>>> This affects btrfs-next, revision be8e3cd00d7293dd177e3f8a4a1645ce09ca3acb
>>>> (Btrfs: separate out tests into their own directory).
>>>>
>>>> Signed-off-by: Filipe David Borba Manana <fdmanana@gmail.com>
>>>> ---
>>>>
>>>> V2: removed atomic_t member in struct btrfs_block_rsv, as suggested by
>>>> Josef Bacik, and use instead the condition reserved == 0 to decide
>>>> when to free the block.
>>>> V3: simplified patch, just kfree() (and not btrfs_free_block_rsv) the
>>>> root's orphan_block_rsv when free'ing the root. Thanks Josef for
>>>> the suggestion.
>>>> V4: use btrfs_free_block_rsv() instead of kfree(). The error I was getting
>>>> in xfstests when using btrfs_free_block_rsv() was unrelated, Josef just
>>>> pointed it to me (separate issue).
>>>> V5: move the free call below the iput() call, so that btrfs_evict_node()
>>>> can process the orphan_block_rsv first to do some needed cleanup before
>>>> we free it.
>>>> V6: free the root's orphan_block_rsv in close_ctree() too. After a balance
>>>> the orphan_block_rsv of the tree of tree roots was being leaked, because
>>>> free_fs_root() is only called for filesystem trees.
>>>>
>>>> fs/btrfs/disk-io.c | 5 +++++
>>>> 1 file changed, 5 insertions(+)
>>>>
>>>> diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c
>>>> index 3b12c26..5d17163 100644
>>>> --- a/fs/btrfs/disk-io.c
>>>> +++ b/fs/btrfs/disk-io.c
>>>> @@ -3430,6 +3430,8 @@ static void free_fs_root(struct btrfs_root *root)
>>>> {
>>>> iput(root->cache_inode);
>>>> WARN_ON(!RB_EMPTY_ROOT(&root->inode_tree));
>>>> + btrfs_free_block_rsv(root, root->orphan_block_rsv);
>>>> + root->orphan_block_rsv = NULL;
>>>> if (root->anon_dev)
>>>> free_anon_bdev(root->anon_dev);
>>>> free_extent_buffer(root->node);
>>>> @@ -3582,6 +3584,9 @@ int close_ctree(struct btrfs_root *root)
>>>>
>>>> btrfs_free_stripe_hash_table(fs_info);
>>>>
>>>> + btrfs_free_block_rsv(root, root->orphan_block_rsv);
>>>> + root->orphan_block_rsv = NULL;
>>>> +
>>>> return 0;
>>>> }
>>>>
>>>> --
>>>> 1.7.9.5
>>>>
>>>> --
>>>> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
>>>> the body of a message to majordomo@vger.kernel.org
>>>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>>
>>
>>
>> --
>> Filipe David Manana,
>>
>> "Reasonable men adapt themselves to the world.
>> Unreasonable men adapt the world to themselves.
>> That's why all progress depends on unreasonable men."
--
Filipe David Manana,
"Reasonable men adapt themselves to the world.
Unreasonable men adapt the world to themselves.
That's why all progress depends on unreasonable men."
^ permalink raw reply related [flat|nested] 13+ messages in thread
* Re: [PATCH v6] Btrfs: fix memory leak of orphan block rsv
2013-10-23 14:26 ` Filipe David Manana
@ 2013-11-04 12:16 ` Alex Lyakas
2013-11-06 12:19 ` Filipe David Manana
0 siblings, 1 reply; 13+ messages in thread
From: Alex Lyakas @ 2013-11-04 12:16 UTC (permalink / raw)
To: Filipe Manana; +Cc: linux-btrfs
Hi Filipe,
any luck with this patch?:)
Alex.
On Wed, Oct 23, 2013 at 5:26 PM, Filipe David Manana <fdmanana@gmail.com> wrote:
> On Wed, Oct 23, 2013 at 3:14 PM, Alex Lyakas
> <alex.btrfs@zadarastorage.com> wrote:
>> Hello,
>>
>> On Wed, Oct 23, 2013 at 4:35 PM, Filipe David Manana <fdmanana@gmail.com> wrote:
>>> On Wed, Oct 23, 2013 at 2:33 PM, Alex Lyakas
>>> <alex.btrfs@zadarastorage.com> wrote:
>>>> Hi Filipe,
>>>>
>>>>
>>>> On Tue, Aug 20, 2013 at 2:52 AM, Filipe David Borba Manana
>>>> <fdmanana@gmail.com> wrote:
>>>>>
>>>>> This issue is simple to reproduce and observe if kmemleak is enabled.
>>>>> Two simple ways to reproduce it:
>>>>>
>>>>> ** 1
>>>>>
>>>>> $ mkfs.btrfs -f /dev/loop0
>>>>> $ mount /dev/loop0 /mnt/btrfs
>>>>> $ btrfs balance start /mnt/btrfs
>>>>> $ umount /mnt/btrfs
>>
>> So here it seems that the leak can only happen in case the block-group
>> has a free-space inode. This is what the orphan item is added for.
>> Yes, here kmemleak reports.
>> But: if space_cache option is disabled (and nospace_cache) enabled, it
>> seems that btrfs still creates the FREE_SPACE inodes, although they
>> are empty because in cache_save_setup:
>>
>> inode = lookup_free_space_inode(root, block_group, path);
>> if (IS_ERR(inode) && PTR_ERR(inode) != -ENOENT) {
>> ret = PTR_ERR(inode);
>> btrfs_release_path(path);
>> goto out;
>> }
>>
>> if (IS_ERR(inode)) {
>> ...
>> ret = create_free_space_inode(root, trans, block_group, path);
>>
>> and only later it actually sets BTRFS_DC_WRITTEN if space_cache option
>> is disabled. Amazing!
>> Although this is a different issue, do you know perhaps why these
>> empty inodes are needed?
>
> Don't know if they are needed. But you have a point, it seems odd to
> create the free space cache inode if mount option nospace_cache was
> supplied. Thanks Alex. Testing the following patch:
>
> diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c
> index c43ee8a..eb1b7da 100644
> --- a/fs/btrfs/extent-tree.c
> +++ b/fs/btrfs/extent-tree.c
> @@ -3162,6 +3162,9 @@ static int cache_save_setup(struct
> btrfs_block_group_cache *block_group,
> int retries = 0;
> int ret = 0;
>
> + if (!btrfs_test_opt(root, SPACE_CACHE))
> + return 0;
> +
> /*
> * If this block group is smaller than 100 megs don't bother caching the
> * block group.
>
>
>>
>> Thanks!
>> Alex.
>>
>>
>>
>>>>>
>>>>> ** 2
>>>>>
>>>>> $ mkfs.btrfs -f /dev/loop0
>>>>> $ mount /dev/loop0 /mnt/btrfs
>>>>> $ touch /mnt/btrfs/foobar
>>>>> $ rm -f /mnt/btrfs/foobar
>>>>> $ umount /mnt/btrfs
>>>>
>>>>
>>>> I tried the second repro script on kernel 3.8.13, and kmemleak does
>>>> not report a leak (even if I force the kmemleak scan). I did not try
>>>> the balance-repro script, though. Am I missing something?
>>>
>>> Maybe it's not an issue on 3.8.13 and older releases.
>>> This was on btrfs-next from August 19.
>>>
>>> thanks for testing
>>>
>>>>
>>>> Thanks,
>>>> Alex.
>>>>
>>>>
>>>>>
>>>>>
>>>>> After a while, kmemleak reports the leak:
>>>>>
>>>>> $ cat /sys/kernel/debug/kmemleak
>>>>> unreferenced object 0xffff880402b13e00 (size 128):
>>>>> comm "btrfs", pid 19621, jiffies 4341648183 (age 70057.844s)
>>>>> hex dump (first 32 bytes):
>>>>> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
>>>>> 00 fc c6 b1 04 88 ff ff 04 00 04 00 ad 4e ad de .............N..
>>>>> backtrace:
>>>>> [<ffffffff817275a6>] kmemleak_alloc+0x26/0x50
>>>>> [<ffffffff8117832b>] kmem_cache_alloc_trace+0xeb/0x1d0
>>>>> [<ffffffffa04db499>] btrfs_alloc_block_rsv+0x39/0x70 [btrfs]
>>>>> [<ffffffffa04f8bad>] btrfs_orphan_add+0x13d/0x1b0 [btrfs]
>>>>> [<ffffffffa04e2b13>] btrfs_remove_block_group+0x143/0x500 [btrfs]
>>>>> [<ffffffffa0518158>] btrfs_relocate_chunk.isra.63+0x618/0x790 [btrfs]
>>>>> [<ffffffffa051bc27>] btrfs_balance+0x8f7/0xe90 [btrfs]
>>>>> [<ffffffffa05240a0>] btrfs_ioctl_balance+0x250/0x550 [btrfs]
>>>>> [<ffffffffa05269ca>] btrfs_ioctl+0xdfa/0x25f0 [btrfs]
>>>>> [<ffffffff8119c936>] do_vfs_ioctl+0x96/0x570
>>>>> [<ffffffff8119cea1>] SyS_ioctl+0x91/0xb0
>>>>> [<ffffffff81750242>] system_call_fastpath+0x16/0x1b
>>>>> [<ffffffffffffffff>] 0xffffffffffffffff
>>>>>
>>>>> This affects btrfs-next, revision be8e3cd00d7293dd177e3f8a4a1645ce09ca3acb
>>>>> (Btrfs: separate out tests into their own directory).
>>>>>
>>>>> Signed-off-by: Filipe David Borba Manana <fdmanana@gmail.com>
>>>>> ---
>>>>>
>>>>> V2: removed atomic_t member in struct btrfs_block_rsv, as suggested by
>>>>> Josef Bacik, and use instead the condition reserved == 0 to decide
>>>>> when to free the block.
>>>>> V3: simplified patch, just kfree() (and not btrfs_free_block_rsv) the
>>>>> root's orphan_block_rsv when free'ing the root. Thanks Josef for
>>>>> the suggestion.
>>>>> V4: use btrfs_free_block_rsv() instead of kfree(). The error I was getting
>>>>> in xfstests when using btrfs_free_block_rsv() was unrelated, Josef just
>>>>> pointed it to me (separate issue).
>>>>> V5: move the free call below the iput() call, so that btrfs_evict_node()
>>>>> can process the orphan_block_rsv first to do some needed cleanup before
>>>>> we free it.
>>>>> V6: free the root's orphan_block_rsv in close_ctree() too. After a balance
>>>>> the orphan_block_rsv of the tree of tree roots was being leaked, because
>>>>> free_fs_root() is only called for filesystem trees.
>>>>>
>>>>> fs/btrfs/disk-io.c | 5 +++++
>>>>> 1 file changed, 5 insertions(+)
>>>>>
>>>>> diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c
>>>>> index 3b12c26..5d17163 100644
>>>>> --- a/fs/btrfs/disk-io.c
>>>>> +++ b/fs/btrfs/disk-io.c
>>>>> @@ -3430,6 +3430,8 @@ static void free_fs_root(struct btrfs_root *root)
>>>>> {
>>>>> iput(root->cache_inode);
>>>>> WARN_ON(!RB_EMPTY_ROOT(&root->inode_tree));
>>>>> + btrfs_free_block_rsv(root, root->orphan_block_rsv);
>>>>> + root->orphan_block_rsv = NULL;
>>>>> if (root->anon_dev)
>>>>> free_anon_bdev(root->anon_dev);
>>>>> free_extent_buffer(root->node);
>>>>> @@ -3582,6 +3584,9 @@ int close_ctree(struct btrfs_root *root)
>>>>>
>>>>> btrfs_free_stripe_hash_table(fs_info);
>>>>>
>>>>> + btrfs_free_block_rsv(root, root->orphan_block_rsv);
>>>>> + root->orphan_block_rsv = NULL;
>>>>> +
>>>>> return 0;
>>>>> }
>>>>>
>>>>> --
>>>>> 1.7.9.5
>>>>>
>>>>> --
>>>>> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
>>>>> the body of a message to majordomo@vger.kernel.org
>>>>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>>>
>>>
>>>
>>> --
>>> Filipe David Manana,
>>>
>>> "Reasonable men adapt themselves to the world.
>>> Unreasonable men adapt the world to themselves.
>>> That's why all progress depends on unreasonable men."
>
>
>
> --
> Filipe David Manana,
>
> "Reasonable men adapt themselves to the world.
> Unreasonable men adapt the world to themselves.
> That's why all progress depends on unreasonable men."
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH v6] Btrfs: fix memory leak of orphan block rsv
2013-11-04 12:16 ` Alex Lyakas
@ 2013-11-06 12:19 ` Filipe David Manana
2014-06-18 10:33 ` Alex Lyakas
0 siblings, 1 reply; 13+ messages in thread
From: Filipe David Manana @ 2013-11-06 12:19 UTC (permalink / raw)
To: Alex Lyakas; +Cc: linux-btrfs
On Mon, Nov 4, 2013 at 12:16 PM, Alex Lyakas
<alex.btrfs@zadarastorage.com> wrote:
> Hi Filipe,
> any luck with this patch?:)
Hey Alex,
I haven't digged further, but I remember I couldn't reproduce your
issue (with latest btrfs-next of that day) of getting the free space
inodes created even when mount option nospace_cache is given.
What kernel were you using?
>
> Alex.
>
> On Wed, Oct 23, 2013 at 5:26 PM, Filipe David Manana <fdmanana@gmail.com> wrote:
>> On Wed, Oct 23, 2013 at 3:14 PM, Alex Lyakas
>> <alex.btrfs@zadarastorage.com> wrote:
>>> Hello,
>>>
>>> On Wed, Oct 23, 2013 at 4:35 PM, Filipe David Manana <fdmanana@gmail.com> wrote:
>>>> On Wed, Oct 23, 2013 at 2:33 PM, Alex Lyakas
>>>> <alex.btrfs@zadarastorage.com> wrote:
>>>>> Hi Filipe,
>>>>>
>>>>>
>>>>> On Tue, Aug 20, 2013 at 2:52 AM, Filipe David Borba Manana
>>>>> <fdmanana@gmail.com> wrote:
>>>>>>
>>>>>> This issue is simple to reproduce and observe if kmemleak is enabled.
>>>>>> Two simple ways to reproduce it:
>>>>>>
>>>>>> ** 1
>>>>>>
>>>>>> $ mkfs.btrfs -f /dev/loop0
>>>>>> $ mount /dev/loop0 /mnt/btrfs
>>>>>> $ btrfs balance start /mnt/btrfs
>>>>>> $ umount /mnt/btrfs
>>>
>>> So here it seems that the leak can only happen in case the block-group
>>> has a free-space inode. This is what the orphan item is added for.
>>> Yes, here kmemleak reports.
>>> But: if space_cache option is disabled (and nospace_cache) enabled, it
>>> seems that btrfs still creates the FREE_SPACE inodes, although they
>>> are empty because in cache_save_setup:
>>>
>>> inode = lookup_free_space_inode(root, block_group, path);
>>> if (IS_ERR(inode) && PTR_ERR(inode) != -ENOENT) {
>>> ret = PTR_ERR(inode);
>>> btrfs_release_path(path);
>>> goto out;
>>> }
>>>
>>> if (IS_ERR(inode)) {
>>> ...
>>> ret = create_free_space_inode(root, trans, block_group, path);
>>>
>>> and only later it actually sets BTRFS_DC_WRITTEN if space_cache option
>>> is disabled. Amazing!
>>> Although this is a different issue, do you know perhaps why these
>>> empty inodes are needed?
>>
>> Don't know if they are needed. But you have a point, it seems odd to
>> create the free space cache inode if mount option nospace_cache was
>> supplied. Thanks Alex. Testing the following patch:
>>
>> diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c
>> index c43ee8a..eb1b7da 100644
>> --- a/fs/btrfs/extent-tree.c
>> +++ b/fs/btrfs/extent-tree.c
>> @@ -3162,6 +3162,9 @@ static int cache_save_setup(struct
>> btrfs_block_group_cache *block_group,
>> int retries = 0;
>> int ret = 0;
>>
>> + if (!btrfs_test_opt(root, SPACE_CACHE))
>> + return 0;
>> +
>> /*
>> * If this block group is smaller than 100 megs don't bother caching the
>> * block group.
>>
>>
>>>
>>> Thanks!
>>> Alex.
>>>
>>>
>>>
>>>>>>
>>>>>> ** 2
>>>>>>
>>>>>> $ mkfs.btrfs -f /dev/loop0
>>>>>> $ mount /dev/loop0 /mnt/btrfs
>>>>>> $ touch /mnt/btrfs/foobar
>>>>>> $ rm -f /mnt/btrfs/foobar
>>>>>> $ umount /mnt/btrfs
>>>>>
>>>>>
>>>>> I tried the second repro script on kernel 3.8.13, and kmemleak does
>>>>> not report a leak (even if I force the kmemleak scan). I did not try
>>>>> the balance-repro script, though. Am I missing something?
>>>>
>>>> Maybe it's not an issue on 3.8.13 and older releases.
>>>> This was on btrfs-next from August 19.
>>>>
>>>> thanks for testing
>>>>
>>>>>
>>>>> Thanks,
>>>>> Alex.
>>>>>
>>>>>
>>>>>>
>>>>>>
>>>>>> After a while, kmemleak reports the leak:
>>>>>>
>>>>>> $ cat /sys/kernel/debug/kmemleak
>>>>>> unreferenced object 0xffff880402b13e00 (size 128):
>>>>>> comm "btrfs", pid 19621, jiffies 4341648183 (age 70057.844s)
>>>>>> hex dump (first 32 bytes):
>>>>>> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
>>>>>> 00 fc c6 b1 04 88 ff ff 04 00 04 00 ad 4e ad de .............N..
>>>>>> backtrace:
>>>>>> [<ffffffff817275a6>] kmemleak_alloc+0x26/0x50
>>>>>> [<ffffffff8117832b>] kmem_cache_alloc_trace+0xeb/0x1d0
>>>>>> [<ffffffffa04db499>] btrfs_alloc_block_rsv+0x39/0x70 [btrfs]
>>>>>> [<ffffffffa04f8bad>] btrfs_orphan_add+0x13d/0x1b0 [btrfs]
>>>>>> [<ffffffffa04e2b13>] btrfs_remove_block_group+0x143/0x500 [btrfs]
>>>>>> [<ffffffffa0518158>] btrfs_relocate_chunk.isra.63+0x618/0x790 [btrfs]
>>>>>> [<ffffffffa051bc27>] btrfs_balance+0x8f7/0xe90 [btrfs]
>>>>>> [<ffffffffa05240a0>] btrfs_ioctl_balance+0x250/0x550 [btrfs]
>>>>>> [<ffffffffa05269ca>] btrfs_ioctl+0xdfa/0x25f0 [btrfs]
>>>>>> [<ffffffff8119c936>] do_vfs_ioctl+0x96/0x570
>>>>>> [<ffffffff8119cea1>] SyS_ioctl+0x91/0xb0
>>>>>> [<ffffffff81750242>] system_call_fastpath+0x16/0x1b
>>>>>> [<ffffffffffffffff>] 0xffffffffffffffff
>>>>>>
>>>>>> This affects btrfs-next, revision be8e3cd00d7293dd177e3f8a4a1645ce09ca3acb
>>>>>> (Btrfs: separate out tests into their own directory).
>>>>>>
>>>>>> Signed-off-by: Filipe David Borba Manana <fdmanana@gmail.com>
>>>>>> ---
>>>>>>
>>>>>> V2: removed atomic_t member in struct btrfs_block_rsv, as suggested by
>>>>>> Josef Bacik, and use instead the condition reserved == 0 to decide
>>>>>> when to free the block.
>>>>>> V3: simplified patch, just kfree() (and not btrfs_free_block_rsv) the
>>>>>> root's orphan_block_rsv when free'ing the root. Thanks Josef for
>>>>>> the suggestion.
>>>>>> V4: use btrfs_free_block_rsv() instead of kfree(). The error I was getting
>>>>>> in xfstests when using btrfs_free_block_rsv() was unrelated, Josef just
>>>>>> pointed it to me (separate issue).
>>>>>> V5: move the free call below the iput() call, so that btrfs_evict_node()
>>>>>> can process the orphan_block_rsv first to do some needed cleanup before
>>>>>> we free it.
>>>>>> V6: free the root's orphan_block_rsv in close_ctree() too. After a balance
>>>>>> the orphan_block_rsv of the tree of tree roots was being leaked, because
>>>>>> free_fs_root() is only called for filesystem trees.
>>>>>>
>>>>>> fs/btrfs/disk-io.c | 5 +++++
>>>>>> 1 file changed, 5 insertions(+)
>>>>>>
>>>>>> diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c
>>>>>> index 3b12c26..5d17163 100644
>>>>>> --- a/fs/btrfs/disk-io.c
>>>>>> +++ b/fs/btrfs/disk-io.c
>>>>>> @@ -3430,6 +3430,8 @@ static void free_fs_root(struct btrfs_root *root)
>>>>>> {
>>>>>> iput(root->cache_inode);
>>>>>> WARN_ON(!RB_EMPTY_ROOT(&root->inode_tree));
>>>>>> + btrfs_free_block_rsv(root, root->orphan_block_rsv);
>>>>>> + root->orphan_block_rsv = NULL;
>>>>>> if (root->anon_dev)
>>>>>> free_anon_bdev(root->anon_dev);
>>>>>> free_extent_buffer(root->node);
>>>>>> @@ -3582,6 +3584,9 @@ int close_ctree(struct btrfs_root *root)
>>>>>>
>>>>>> btrfs_free_stripe_hash_table(fs_info);
>>>>>>
>>>>>> + btrfs_free_block_rsv(root, root->orphan_block_rsv);
>>>>>> + root->orphan_block_rsv = NULL;
>>>>>> +
>>>>>> return 0;
>>>>>> }
>>>>>>
>>>>>> --
>>>>>> 1.7.9.5
>>>>>>
>>>>>> --
>>>>>> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
>>>>>> the body of a message to majordomo@vger.kernel.org
>>>>>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>>>>
>>>>
>>>>
>>>> --
>>>> Filipe David Manana,
>>>>
>>>> "Reasonable men adapt themselves to the world.
>>>> Unreasonable men adapt the world to themselves.
>>>> That's why all progress depends on unreasonable men."
>>
>>
>>
>> --
>> Filipe David Manana,
>>
>> "Reasonable men adapt themselves to the world.
>> Unreasonable men adapt the world to themselves.
>> That's why all progress depends on unreasonable men."
--
Filipe David Manana,
"Reasonable men adapt themselves to the world.
Unreasonable men adapt the world to themselves.
That's why all progress depends on unreasonable men."
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH v6] Btrfs: fix memory leak of orphan block rsv
2013-11-06 12:19 ` Filipe David Manana
@ 2014-06-18 10:33 ` Alex Lyakas
0 siblings, 0 replies; 13+ messages in thread
From: Alex Lyakas @ 2014-06-18 10:33 UTC (permalink / raw)
To: Filipe Manana; +Cc: linux-btrfs
Hi Filipe,
I finally got to debug this deeper. As it turns out, this happens only
if both "nospace_cache" and "clear_cache" are specified. You need to
unmount and mount again to cause this. After mounting, due to
"clear_cache", all the block-groups are marked as BTRFS_DC_CLEAR, and
then cache_save_setup() is called on them (this function is called
only in case of BTRFS_DC_CLEAR). So cache_save_setup() goes ahead and
creates the free-space inode. But then it realizes that it was mounted
with nospace_cache, so it does not put any content in the inode. But
the inode itself gets created. The patch that fixes this for me:
alex@ubuntu-alex:/mnt/work/alex/linux-stable/source$ git diff -U10
diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c
index d170412..06f876e 100644
--- a/fs/btrfs/extent-tree.c
+++ b/fs/btrfs/extent-tree.c
@@ -2941,20 +2941,26 @@ again:
goto out;
}
if (IS_ERR(inode)) {
BUG_ON(retries);
retries++;
if (block_group->ro)
goto out_free;
+ /* with nospace_cache avoid creating the free-space inode */
+ if (!btrfs_test_opt(root, SPACE_CACHE)) {
+ dcs = BTRFS_DC_WRITTEN;
+ goto out_free;
+ }
+
ret = create_free_space_inode(root, trans, block_group, path);
if (ret)
goto out_free;
goto again;
}
/* We've already setup this transaction, go ahead and exit */
if (block_group->cache_generation == trans->transid &&
i_size_read(inode)) {
dcs = BTRFS_DC_SETUP;
Thanks,
Alex.
On Wed, Nov 6, 2013 at 3:19 PM, Filipe David Manana <fdmanana@gmail.com> wrote:
> On Mon, Nov 4, 2013 at 12:16 PM, Alex Lyakas
> <alex.btrfs@zadarastorage.com> wrote:
>> Hi Filipe,
>> any luck with this patch?:)
>
> Hey Alex,
>
> I haven't digged further, but I remember I couldn't reproduce your
> issue (with latest btrfs-next of that day) of getting the free space
> inodes created even when mount option nospace_cache is given.
>
> What kernel were you using?
>
>>
>> Alex.
>>
>> On Wed, Oct 23, 2013 at 5:26 PM, Filipe David Manana <fdmanana@gmail.com> wrote:
>>> On Wed, Oct 23, 2013 at 3:14 PM, Alex Lyakas
>>> <alex.btrfs@zadarastorage.com> wrote:
>>>> Hello,
>>>>
>>>> On Wed, Oct 23, 2013 at 4:35 PM, Filipe David Manana <fdmanana@gmail.com> wrote:
>>>>> On Wed, Oct 23, 2013 at 2:33 PM, Alex Lyakas
>>>>> <alex.btrfs@zadarastorage.com> wrote:
>>>>>> Hi Filipe,
>>>>>>
>>>>>>
>>>>>> On Tue, Aug 20, 2013 at 2:52 AM, Filipe David Borba Manana
>>>>>> <fdmanana@gmail.com> wrote:
>>>>>>>
>>>>>>> This issue is simple to reproduce and observe if kmemleak is enabled.
>>>>>>> Two simple ways to reproduce it:
>>>>>>>
>>>>>>> ** 1
>>>>>>>
>>>>>>> $ mkfs.btrfs -f /dev/loop0
>>>>>>> $ mount /dev/loop0 /mnt/btrfs
>>>>>>> $ btrfs balance start /mnt/btrfs
>>>>>>> $ umount /mnt/btrfs
>>>>
>>>> So here it seems that the leak can only happen in case the block-group
>>>> has a free-space inode. This is what the orphan item is added for.
>>>> Yes, here kmemleak reports.
>>>> But: if space_cache option is disabled (and nospace_cache) enabled, it
>>>> seems that btrfs still creates the FREE_SPACE inodes, although they
>>>> are empty because in cache_save_setup:
>>>>
>>>> inode = lookup_free_space_inode(root, block_group, path);
>>>> if (IS_ERR(inode) && PTR_ERR(inode) != -ENOENT) {
>>>> ret = PTR_ERR(inode);
>>>> btrfs_release_path(path);
>>>> goto out;
>>>> }
>>>>
>>>> if (IS_ERR(inode)) {
>>>> ...
>>>> ret = create_free_space_inode(root, trans, block_group, path);
>>>>
>>>> and only later it actually sets BTRFS_DC_WRITTEN if space_cache option
>>>> is disabled. Amazing!
>>>> Although this is a different issue, do you know perhaps why these
>>>> empty inodes are needed?
>>>
>>> Don't know if they are needed. But you have a point, it seems odd to
>>> create the free space cache inode if mount option nospace_cache was
>>> supplied. Thanks Alex. Testing the following patch:
>>>
>>> diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c
>>> index c43ee8a..eb1b7da 100644
>>> --- a/fs/btrfs/extent-tree.c
>>> +++ b/fs/btrfs/extent-tree.c
>>> @@ -3162,6 +3162,9 @@ static int cache_save_setup(struct
>>> btrfs_block_group_cache *block_group,
>>> int retries = 0;
>>> int ret = 0;
>>>
>>> + if (!btrfs_test_opt(root, SPACE_CACHE))
>>> + return 0;
>>> +
>>> /*
>>> * If this block group is smaller than 100 megs don't bother caching the
>>> * block group.
>>>
>>>
>>>>
>>>> Thanks!
>>>> Alex.
>>>>
>>>>
>>>>
>>>>>>>
>>>>>>> ** 2
>>>>>>>
>>>>>>> $ mkfs.btrfs -f /dev/loop0
>>>>>>> $ mount /dev/loop0 /mnt/btrfs
>>>>>>> $ touch /mnt/btrfs/foobar
>>>>>>> $ rm -f /mnt/btrfs/foobar
>>>>>>> $ umount /mnt/btrfs
>>>>>>
>>>>>>
>>>>>> I tried the second repro script on kernel 3.8.13, and kmemleak does
>>>>>> not report a leak (even if I force the kmemleak scan). I did not try
>>>>>> the balance-repro script, though. Am I missing something?
>>>>>
>>>>> Maybe it's not an issue on 3.8.13 and older releases.
>>>>> This was on btrfs-next from August 19.
>>>>>
>>>>> thanks for testing
>>>>>
>>>>>>
>>>>>> Thanks,
>>>>>> Alex.
>>>>>>
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> After a while, kmemleak reports the leak:
>>>>>>>
>>>>>>> $ cat /sys/kernel/debug/kmemleak
>>>>>>> unreferenced object 0xffff880402b13e00 (size 128):
>>>>>>> comm "btrfs", pid 19621, jiffies 4341648183 (age 70057.844s)
>>>>>>> hex dump (first 32 bytes):
>>>>>>> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
>>>>>>> 00 fc c6 b1 04 88 ff ff 04 00 04 00 ad 4e ad de .............N..
>>>>>>> backtrace:
>>>>>>> [<ffffffff817275a6>] kmemleak_alloc+0x26/0x50
>>>>>>> [<ffffffff8117832b>] kmem_cache_alloc_trace+0xeb/0x1d0
>>>>>>> [<ffffffffa04db499>] btrfs_alloc_block_rsv+0x39/0x70 [btrfs]
>>>>>>> [<ffffffffa04f8bad>] btrfs_orphan_add+0x13d/0x1b0 [btrfs]
>>>>>>> [<ffffffffa04e2b13>] btrfs_remove_block_group+0x143/0x500 [btrfs]
>>>>>>> [<ffffffffa0518158>] btrfs_relocate_chunk.isra.63+0x618/0x790 [btrfs]
>>>>>>> [<ffffffffa051bc27>] btrfs_balance+0x8f7/0xe90 [btrfs]
>>>>>>> [<ffffffffa05240a0>] btrfs_ioctl_balance+0x250/0x550 [btrfs]
>>>>>>> [<ffffffffa05269ca>] btrfs_ioctl+0xdfa/0x25f0 [btrfs]
>>>>>>> [<ffffffff8119c936>] do_vfs_ioctl+0x96/0x570
>>>>>>> [<ffffffff8119cea1>] SyS_ioctl+0x91/0xb0
>>>>>>> [<ffffffff81750242>] system_call_fastpath+0x16/0x1b
>>>>>>> [<ffffffffffffffff>] 0xffffffffffffffff
>>>>>>>
>>>>>>> This affects btrfs-next, revision be8e3cd00d7293dd177e3f8a4a1645ce09ca3acb
>>>>>>> (Btrfs: separate out tests into their own directory).
>>>>>>>
>>>>>>> Signed-off-by: Filipe David Borba Manana <fdmanana@gmail.com>
>>>>>>> ---
>>>>>>>
>>>>>>> V2: removed atomic_t member in struct btrfs_block_rsv, as suggested by
>>>>>>> Josef Bacik, and use instead the condition reserved == 0 to decide
>>>>>>> when to free the block.
>>>>>>> V3: simplified patch, just kfree() (and not btrfs_free_block_rsv) the
>>>>>>> root's orphan_block_rsv when free'ing the root. Thanks Josef for
>>>>>>> the suggestion.
>>>>>>> V4: use btrfs_free_block_rsv() instead of kfree(). The error I was getting
>>>>>>> in xfstests when using btrfs_free_block_rsv() was unrelated, Josef just
>>>>>>> pointed it to me (separate issue).
>>>>>>> V5: move the free call below the iput() call, so that btrfs_evict_node()
>>>>>>> can process the orphan_block_rsv first to do some needed cleanup before
>>>>>>> we free it.
>>>>>>> V6: free the root's orphan_block_rsv in close_ctree() too. After a balance
>>>>>>> the orphan_block_rsv of the tree of tree roots was being leaked, because
>>>>>>> free_fs_root() is only called for filesystem trees.
>>>>>>>
>>>>>>> fs/btrfs/disk-io.c | 5 +++++
>>>>>>> 1 file changed, 5 insertions(+)
>>>>>>>
>>>>>>> diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c
>>>>>>> index 3b12c26..5d17163 100644
>>>>>>> --- a/fs/btrfs/disk-io.c
>>>>>>> +++ b/fs/btrfs/disk-io.c
>>>>>>> @@ -3430,6 +3430,8 @@ static void free_fs_root(struct btrfs_root *root)
>>>>>>> {
>>>>>>> iput(root->cache_inode);
>>>>>>> WARN_ON(!RB_EMPTY_ROOT(&root->inode_tree));
>>>>>>> + btrfs_free_block_rsv(root, root->orphan_block_rsv);
>>>>>>> + root->orphan_block_rsv = NULL;
>>>>>>> if (root->anon_dev)
>>>>>>> free_anon_bdev(root->anon_dev);
>>>>>>> free_extent_buffer(root->node);
>>>>>>> @@ -3582,6 +3584,9 @@ int close_ctree(struct btrfs_root *root)
>>>>>>>
>>>>>>> btrfs_free_stripe_hash_table(fs_info);
>>>>>>>
>>>>>>> + btrfs_free_block_rsv(root, root->orphan_block_rsv);
>>>>>>> + root->orphan_block_rsv = NULL;
>>>>>>> +
>>>>>>> return 0;
>>>>>>> }
>>>>>>>
>>>>>>> --
>>>>>>> 1.7.9.5
>>>>>>>
>>>>>>> --
>>>>>>> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
>>>>>>> the body of a message to majordomo@vger.kernel.org
>>>>>>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Filipe David Manana,
>>>>>
>>>>> "Reasonable men adapt themselves to the world.
>>>>> Unreasonable men adapt the world to themselves.
>>>>> That's why all progress depends on unreasonable men."
>>>
>>>
>>>
>>> --
>>> Filipe David Manana,
>>>
>>> "Reasonable men adapt themselves to the world.
>>> Unreasonable men adapt the world to themselves.
>>> That's why all progress depends on unreasonable men."
>
>
>
> --
> Filipe David Manana,
>
> "Reasonable men adapt themselves to the world.
> Unreasonable men adapt the world to themselves.
> That's why all progress depends on unreasonable men."
^ permalink raw reply related [flat|nested] 13+ messages in thread
end of thread, other threads:[~2014-06-18 10:33 UTC | newest]
Thread overview: 13+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2013-08-19 18:08 [RFC PATCH] Btrfs: fix memory leak of orphan block rsv Filipe David Borba Manana
2013-08-19 18:31 ` [PATCH v2] " Filipe David Borba Manana
2013-08-19 20:15 ` [PATCH v3] " Filipe David Borba Manana
2013-08-19 20:30 ` [PATCH v4] " Filipe David Borba Manana
2013-08-19 20:42 ` [PATCH v5] " Filipe David Borba Manana
2013-08-19 23:52 ` [PATCH v6] " Filipe David Borba Manana
2013-10-23 13:33 ` Alex Lyakas
2013-10-23 13:35 ` Filipe David Manana
2013-10-23 14:14 ` Alex Lyakas
2013-10-23 14:26 ` Filipe David Manana
2013-11-04 12:16 ` Alex Lyakas
2013-11-06 12:19 ` Filipe David Manana
2014-06-18 10:33 ` Alex Lyakas
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.