* [PATCH v2]Btrfs: pwrite blocked when writing from the mmaped buffer of the same page @ 2010-12-09 9:30 Zhong, Xin 2011-01-27 13:09 ` Johannes Hirte 0 siblings, 1 reply; 43+ messages in thread From: Zhong, Xin @ 2010-12-09 9:30 UTC (permalink / raw) To: linux-btrfs; +Cc: xin.zhong This problem is found in meego testing: http://bugs.meego.com/show_bug.cgi?id=6672 A file in btrfs is mmaped and the mmaped buffer is passed to pwrite to write to the same page of the same file. In btrfs_file_aio_write(), the pages is locked by prepare_pages(). So when btrfs_copy_from_user() is called, page fault happens and the same page needs to be locked again in filemap_fault(). The fix is to move iov_iter_fault_in_readable() before prepage_pages() to make page fault happen before pages are locked. And also disable page fault in critical region in btrfs_copy_from_user(). Reviewed-by: Yan, Zheng<zheng.z.yan@intel.com> Signed-off-by: Zhong, Xin <xin.zhong@intel.com> --- fs/btrfs/file.c | 92 ++++++++++++++++++++++++++++++++++++------------------- 1 files changed, 60 insertions(+), 32 deletions(-) diff --git a/fs/btrfs/file.c b/fs/btrfs/file.c index c1faded..66836d8 100644 --- a/fs/btrfs/file.c +++ b/fs/btrfs/file.c @@ -48,30 +48,34 @@ static noinline int btrfs_copy_from_user(loff_t pos, int num_pages, struct page **prepared_pages, struct iov_iter *i) { - size_t copied; + size_t copied = 0; int pg = 0; int offset = pos & (PAGE_CACHE_SIZE - 1); + int total_copied = 0; while (write_bytes > 0) { size_t count = min_t(size_t, PAGE_CACHE_SIZE - offset, write_bytes); struct page *page = prepared_pages[pg]; -again: - if (unlikely(iov_iter_fault_in_readable(i, count))) - return -EFAULT; - - /* Copy data from userspace to the current page */ - copied = iov_iter_copy_from_user(page, i, offset, count); + /* + * Copy data from userspace to the current page + * + * Disable pagefault to avoid recursive lock since + * the pages are already locked + */ + pagefault_disable(); + copied = iov_iter_copy_from_user_atomic(page, i, offset, count); + pagefault_enable(); /* Flush processor's dcache for this page */ flush_dcache_page(page); iov_iter_advance(i, copied); write_bytes -= copied; + total_copied += copied; + /* Return to btrfs_file_aio_write to fault page */ if (unlikely(copied == 0)) { - count = min_t(size_t, PAGE_CACHE_SIZE - offset, - iov_iter_single_seg_count(i)); - goto again; + break; } if (unlikely(copied < PAGE_CACHE_SIZE - offset)) { @@ -81,7 +85,7 @@ again: offset = 0; } } - return 0; + return total_copied; } /* @@ -854,6 +858,8 @@ static ssize_t btrfs_file_aio_write(struct kiocb *iocb, unsigned long last_index; int will_write; int buffered = 0; + int copied = 0; + int dirty_pages = 0; will_write = ((file->f_flags & O_DSYNC) || IS_SYNC(inode) || (file->f_flags & O_DIRECT)); @@ -970,7 +976,17 @@ static ssize_t btrfs_file_aio_write(struct kiocb *iocb, WARN_ON(num_pages > nrptrs); memset(pages, 0, sizeof(struct page *) * nrptrs); - ret = btrfs_delalloc_reserve_space(inode, write_bytes); + /* + * Fault pages before locking them in prepare_pages + * to avoid recursive lock + */ + if (unlikely(iov_iter_fault_in_readable(&i, write_bytes))) { + ret = -EFAULT; + goto out; + } + + ret = btrfs_delalloc_reserve_space(inode, + num_pages << PAGE_CACHE_SHIFT); if (ret) goto out; @@ -978,37 +994,49 @@ static ssize_t btrfs_file_aio_write(struct kiocb *iocb, pos, first_index, last_index, write_bytes); if (ret) { - btrfs_delalloc_release_space(inode, write_bytes); + btrfs_delalloc_release_space(inode, + num_pages << PAGE_CACHE_SHIFT); goto out; } - ret = btrfs_copy_from_user(pos, num_pages, + copied = btrfs_copy_from_user(pos, num_pages, write_bytes, pages, &i); - if (ret == 0) { + dirty_pages = (copied + PAGE_CACHE_SIZE - 1) >> + PAGE_CACHE_SHIFT; + + if (num_pages > dirty_pages) { + if (copied > 0) + atomic_inc( + &BTRFS_I(inode)->outstanding_extents); + btrfs_delalloc_release_space(inode, + (num_pages - dirty_pages) << + PAGE_CACHE_SHIFT); + } + + if (copied > 0) { dirty_and_release_pages(NULL, root, file, pages, - num_pages, pos, write_bytes); + dirty_pages, pos, copied); } btrfs_drop_pages(pages, num_pages); - if (ret) { - btrfs_delalloc_release_space(inode, write_bytes); - goto out; - } - if (will_write) { - filemap_fdatawrite_range(inode->i_mapping, pos, - pos + write_bytes - 1); - } else { - balance_dirty_pages_ratelimited_nr(inode->i_mapping, - num_pages); - if (num_pages < - (root->leafsize >> PAGE_CACHE_SHIFT) + 1) - btrfs_btree_balance_dirty(root, 1); - btrfs_throttle(root); + if (copied > 0) { + if (will_write) { + filemap_fdatawrite_range(inode->i_mapping, pos, + pos + copied - 1); + } else { + balance_dirty_pages_ratelimited_nr( + inode->i_mapping, + dirty_pages); + if (dirty_pages < + (root->leafsize >> PAGE_CACHE_SHIFT) + 1) + btrfs_btree_balance_dirty(root, 1); + btrfs_throttle(root); + } } - pos += write_bytes; - num_written += write_bytes; + pos += copied; + num_written += copied; cond_resched(); } -- 1.6.2.2 ^ permalink raw reply related [flat|nested] 43+ messages in thread
* Re: [PATCH v2]Btrfs: pwrite blocked when writing from the mmaped buffer of the same page 2010-12-09 9:30 [PATCH v2]Btrfs: pwrite blocked when writing from the mmaped buffer of the same page Zhong, Xin @ 2011-01-27 13:09 ` Johannes Hirte 2011-01-27 22:12 ` Maria Wikström 0 siblings, 1 reply; 43+ messages in thread From: Johannes Hirte @ 2011-01-27 13:09 UTC (permalink / raw) To: Zhong, Xin; +Cc: linux-btrfs On Thursday 09 December 2010 10:30:14 Zhong, Xin wrote: > This problem is found in meego testing: > http://bugs.meego.com/show_bug.cgi?id=6672 > A file in btrfs is mmaped and the mmaped buffer is passed to pwrite to > write to the same page of the same file. In btrfs_file_aio_write(), the > pages is locked by prepare_pages(). So when btrfs_copy_from_user() is > called, page fault happens and the same page needs to be locked again in > filemap_fault(). The fix is to move iov_iter_fault_in_readable() before > prepage_pages() to make page fault happen before pages are locked. And > also disable page fault in critical region in btrfs_copy_from_user(). > > Reviewed-by: Yan, Zheng<zheng.z.yan@intel.com> > Signed-off-by: Zhong, Xin <xin.zhong@intel.com> > --- > fs/btrfs/file.c | 92 > ++++++++++++++++++++++++++++++++++++------------------- 1 files changed, > 60 insertions(+), 32 deletions(-) > > diff --git a/fs/btrfs/file.c b/fs/btrfs/file.c > index c1faded..66836d8 100644 > --- a/fs/btrfs/file.c > +++ b/fs/btrfs/file.c > @@ -48,30 +48,34 @@ static noinline int btrfs_copy_from_user(loff_t pos, > int num_pages, struct page **prepared_pages, > struct iov_iter *i) > { > - size_t copied; > + size_t copied = 0; > int pg = 0; > int offset = pos & (PAGE_CACHE_SIZE - 1); > + int total_copied = 0; > > while (write_bytes > 0) { > size_t count = min_t(size_t, > PAGE_CACHE_SIZE - offset, write_bytes); > struct page *page = prepared_pages[pg]; > -again: > - if (unlikely(iov_iter_fault_in_readable(i, count))) > - return -EFAULT; > - > - /* Copy data from userspace to the current page */ > - copied = iov_iter_copy_from_user(page, i, offset, count); > + /* > + * Copy data from userspace to the current page > + * > + * Disable pagefault to avoid recursive lock since > + * the pages are already locked > + */ > + pagefault_disable(); > + copied = iov_iter_copy_from_user_atomic(page, i, offset, count); > + pagefault_enable(); > > /* Flush processor's dcache for this page */ > flush_dcache_page(page); > iov_iter_advance(i, copied); > write_bytes -= copied; > + total_copied += copied; > > + /* Return to btrfs_file_aio_write to fault page */ > if (unlikely(copied == 0)) { > - count = min_t(size_t, PAGE_CACHE_SIZE - offset, > - iov_iter_single_seg_count(i)); > - goto again; > + break; > } > > if (unlikely(copied < PAGE_CACHE_SIZE - offset)) { > @@ -81,7 +85,7 @@ again: > offset = 0; > } > } > - return 0; > + return total_copied; > } > > /* > @@ -854,6 +858,8 @@ static ssize_t btrfs_file_aio_write(struct kiocb *iocb, > unsigned long last_index; > int will_write; > int buffered = 0; > + int copied = 0; > + int dirty_pages = 0; > > will_write = ((file->f_flags & O_DSYNC) || IS_SYNC(inode) || > (file->f_flags & O_DIRECT)); > @@ -970,7 +976,17 @@ static ssize_t btrfs_file_aio_write(struct kiocb > *iocb, WARN_ON(num_pages > nrptrs); > memset(pages, 0, sizeof(struct page *) * nrptrs); > > - ret = btrfs_delalloc_reserve_space(inode, write_bytes); > + /* > + * Fault pages before locking them in prepare_pages > + * to avoid recursive lock > + */ > + if (unlikely(iov_iter_fault_in_readable(&i, write_bytes))) { > + ret = -EFAULT; > + goto out; > + } > + > + ret = btrfs_delalloc_reserve_space(inode, > + num_pages << PAGE_CACHE_SHIFT); > if (ret) > goto out; > > @@ -978,37 +994,49 @@ static ssize_t btrfs_file_aio_write(struct kiocb > *iocb, pos, first_index, last_index, > write_bytes); > if (ret) { > - btrfs_delalloc_release_space(inode, write_bytes); > + btrfs_delalloc_release_space(inode, > + num_pages << PAGE_CACHE_SHIFT); > goto out; > } > > - ret = btrfs_copy_from_user(pos, num_pages, > + copied = btrfs_copy_from_user(pos, num_pages, > write_bytes, pages, &i); > - if (ret == 0) { > + dirty_pages = (copied + PAGE_CACHE_SIZE - 1) >> > + PAGE_CACHE_SHIFT; > + > + if (num_pages > dirty_pages) { > + if (copied > 0) > + atomic_inc( > + &BTRFS_I(inode)->outstanding_extents); > + btrfs_delalloc_release_space(inode, > + (num_pages - dirty_pages) << > + PAGE_CACHE_SHIFT); > + } > + > + if (copied > 0) { > dirty_and_release_pages(NULL, root, file, pages, > - num_pages, pos, write_bytes); > + dirty_pages, pos, copied); > } > > btrfs_drop_pages(pages, num_pages); > - if (ret) { > - btrfs_delalloc_release_space(inode, write_bytes); > - goto out; > - } > > - if (will_write) { > - filemap_fdatawrite_range(inode->i_mapping, pos, > - pos + write_bytes - 1); > - } else { > - balance_dirty_pages_ratelimited_nr(inode->i_mapping, > - num_pages); > - if (num_pages < > - (root->leafsize >> PAGE_CACHE_SHIFT) + 1) > - btrfs_btree_balance_dirty(root, 1); > - btrfs_throttle(root); > + if (copied > 0) { > + if (will_write) { > + filemap_fdatawrite_range(inode->i_mapping, pos, > + pos + copied - 1); > + } else { > + balance_dirty_pages_ratelimited_nr( > + inode->i_mapping, > + dirty_pages); > + if (dirty_pages < > + (root->leafsize >> PAGE_CACHE_SHIFT) + 1) > + btrfs_btree_balance_dirty(root, 1); > + btrfs_throttle(root); > + } > } > > - pos += write_bytes; > - num_written += write_bytes; > + pos += copied; > + num_written += copied; > > cond_resched(); > } This patch breaks one of my Gentoo boxes. When I try to install/update via emerge, some packages hang. It seems that it's always a "svn info" process that is stuck in kernel eating 100% CPU. I don't know how svn is involved here, but reverting this patch makes the system work again. I'll try to get a simple testcase. regards, Johannes ^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [PATCH v2]Btrfs: pwrite blocked when writing from the mmaped buffer of the same page 2011-01-27 13:09 ` Johannes Hirte @ 2011-01-27 22:12 ` Maria Wikström 2011-01-28 1:26 ` Zhong, Xin 0 siblings, 1 reply; 43+ messages in thread From: Maria Wikström @ 2011-01-27 22:12 UTC (permalink / raw) To: johannes.hirte, xin.zhong; +Cc: linux-btrfs tor 2011-01-27 klockan 14:09 +0100 skrev Johannes Hirte: > On Thursday 09 December 2010 10:30:14 Zhong, Xin wrote: > > This problem is found in meego testing: > > http://bugs.meego.com/show_bug.cgi?id=6672 > > A file in btrfs is mmaped and the mmaped buffer is passed to pwrite to > > write to the same page of the same file. In btrfs_file_aio_write(), the > > pages is locked by prepare_pages(). So when btrfs_copy_from_user() is > > called, page fault happens and the same page needs to be locked again in > > filemap_fault(). The fix is to move iov_iter_fault_in_readable() before > > prepage_pages() to make page fault happen before pages are locked. And > > also disable page fault in critical region in btrfs_copy_from_user(). > > > > Reviewed-by: Yan, Zheng<zheng.z.yan@intel.com> > > Signed-off-by: Zhong, Xin <xin.zhong@intel.com> > > --- > > fs/btrfs/file.c | 92 > > ++++++++++++++++++++++++++++++++++++------------------- 1 files changed, > > 60 insertions(+), 32 deletions(-) > > > > diff --git a/fs/btrfs/file.c b/fs/btrfs/file.c > > index c1faded..66836d8 100644 > > --- a/fs/btrfs/file.c > > +++ b/fs/btrfs/file.c > > @@ -48,30 +48,34 @@ static noinline int btrfs_copy_from_user(loff_t pos, > > int num_pages, struct page **prepared_pages, > > struct iov_iter *i) > > { > > - size_t copied; > > + size_t copied = 0; > > int pg = 0; > > int offset = pos & (PAGE_CACHE_SIZE - 1); > > + int total_copied = 0; > > > > while (write_bytes > 0) { > > size_t count = min_t(size_t, > > PAGE_CACHE_SIZE - offset, write_bytes); > > struct page *page = prepared_pages[pg]; > > -again: > > - if (unlikely(iov_iter_fault_in_readable(i, count))) > > - return -EFAULT; > > - > > - /* Copy data from userspace to the current page */ > > - copied = iov_iter_copy_from_user(page, i, offset, count); > > + /* > > + * Copy data from userspace to the current page > > + * > > + * Disable pagefault to avoid recursive lock since > > + * the pages are already locked > > + */ > > + pagefault_disable(); > > + copied = iov_iter_copy_from_user_atomic(page, i, offset, count); > > + pagefault_enable(); > > > > /* Flush processor's dcache for this page */ > > flush_dcache_page(page); > > iov_iter_advance(i, copied); > > write_bytes -= copied; > > + total_copied += copied; > > > > + /* Return to btrfs_file_aio_write to fault page */ > > if (unlikely(copied == 0)) { > > - count = min_t(size_t, PAGE_CACHE_SIZE - offset, > > - iov_iter_single_seg_count(i)); > > - goto again; > > + break; > > } > > > > if (unlikely(copied < PAGE_CACHE_SIZE - offset)) { > > @@ -81,7 +85,7 @@ again: > > offset = 0; > > } > > } > > - return 0; > > + return total_copied; > > } > > > > /* > > @@ -854,6 +858,8 @@ static ssize_t btrfs_file_aio_write(struct kiocb *iocb, > > unsigned long last_index; > > int will_write; > > int buffered = 0; > > + int copied = 0; > > + int dirty_pages = 0; > > > > will_write = ((file->f_flags & O_DSYNC) || IS_SYNC(inode) || > > (file->f_flags & O_DIRECT)); > > @@ -970,7 +976,17 @@ static ssize_t btrfs_file_aio_write(struct kiocb > > *iocb, WARN_ON(num_pages > nrptrs); > > memset(pages, 0, sizeof(struct page *) * nrptrs); > > > > - ret = btrfs_delalloc_reserve_space(inode, write_bytes); > > + /* > > + * Fault pages before locking them in prepare_pages > > + * to avoid recursive lock > > + */ > > + if (unlikely(iov_iter_fault_in_readable(&i, write_bytes))) { > > + ret = -EFAULT; > > + goto out; > > + } > > + > > + ret = btrfs_delalloc_reserve_space(inode, > > + num_pages << PAGE_CACHE_SHIFT); > > if (ret) > > goto out; > > > > @@ -978,37 +994,49 @@ static ssize_t btrfs_file_aio_write(struct kiocb > > *iocb, pos, first_index, last_index, > > write_bytes); > > if (ret) { > > - btrfs_delalloc_release_space(inode, write_bytes); > > + btrfs_delalloc_release_space(inode, > > + num_pages << PAGE_CACHE_SHIFT); > > goto out; > > } > > > > - ret = btrfs_copy_from_user(pos, num_pages, > > + copied = btrfs_copy_from_user(pos, num_pages, > > write_bytes, pages, &i); > > - if (ret == 0) { > > + dirty_pages = (copied + PAGE_CACHE_SIZE - 1) >> > > + PAGE_CACHE_SHIFT; > > + > > + if (num_pages > dirty_pages) { > > + if (copied > 0) > > + atomic_inc( > > + &BTRFS_I(inode)->outstanding_extents); > > + btrfs_delalloc_release_space(inode, > > + (num_pages - dirty_pages) << > > + PAGE_CACHE_SHIFT); > > + } > > + > > + if (copied > 0) { > > dirty_and_release_pages(NULL, root, file, pages, > > - num_pages, pos, write_bytes); > > + dirty_pages, pos, copied); > > } > > > > btrfs_drop_pages(pages, num_pages); > > - if (ret) { > > - btrfs_delalloc_release_space(inode, write_bytes); > > - goto out; > > - } > > > > - if (will_write) { > > - filemap_fdatawrite_range(inode->i_mapping, pos, > > - pos + write_bytes - 1); > > - } else { > > - balance_dirty_pages_ratelimited_nr(inode->i_mapping, > > - num_pages); > > - if (num_pages < > > - (root->leafsize >> PAGE_CACHE_SHIFT) + 1) > > - btrfs_btree_balance_dirty(root, 1); > > - btrfs_throttle(root); > > + if (copied > 0) { > > + if (will_write) { > > + filemap_fdatawrite_range(inode->i_mapping, pos, > > + pos + copied - 1); > > + } else { > > + balance_dirty_pages_ratelimited_nr( > > + inode->i_mapping, > > + dirty_pages); > > + if (dirty_pages < > > + (root->leafsize >> PAGE_CACHE_SHIFT) + 1) > > + btrfs_btree_balance_dirty(root, 1); > > + btrfs_throttle(root); > > + } > > } > > > > - pos += write_bytes; > > - num_written += write_bytes; > > + pos += copied; > > + num_written += copied; > > > > cond_resched(); > > } > > This patch breaks one of my Gentoo boxes. When I try to install/update via > emerge, some packages hang. It seems that it's always a "svn info" process > that is stuck in kernel eating 100% CPU. I don't know how svn is involved > here, but reverting this patch makes the system work again. I'll try to get a > simple testcase. > > regards, > Johannes The same thing happens here. The only way to kill the process (svn info) is to restart the computer. Running vanilla 2.6.37 without patches. #strace emerge libgcrypt -- cut -- open("/var/tmp/portage/dev-libs/libgcrypt-1.4.6/temp/build.log", O_WRONLY|O_CREAT|O_APPEND|O_LARGEFILE, 0666) = 7 fstat64(7, {st_mode=S_IFREG|0660, st_size=416, ...}) = 0 mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb713a000 fstat64(7, {st_mode=S_IFREG|0660, st_size=416, ...}) = 0 _llseek(7, 416, [416], SEEK_SET) = 0 fstat64(7, {st_mode=S_IFREG|0660, st_size=416, ...}) = 0 stat64("/var/tmp/portage/dev-libs/libgcrypt-1.4.6/temp/build.log", {st_mode=S_IFREG|0660, st_size=416, ...}) = 0 dup(1) = 8 fcntl64(8, F_GETFL) = 0x2 (flags O_RDWR) fstat64(8, {st_mode=S_IFCHR|0620, st_rdev=makedev(136, 3), ...}) = 0 mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7005000 _llseek(8, 0, 0xbff80cfc, SEEK_CUR) = -1 ESPIPE (Illegal seek) fstat64(8, {st_mode=S_IFCHR|0620, st_rdev=makedev(136, 3), ...}) = 0 stat64("/var/tmp/portage/dev-libs/libgcrypt-1.4.6/temp/environment", {st_mode=S_IFREG|0664, st_size=105970, ...}) = 0 clone(child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID| SIGCHLD, child_tidptr=0xb7430728) = 11665 close(6) = 0 gettimeofday({1296162836, 126192}, NULL) = 0 poll([{fd=3, events=POLLIN|POLLERR|POLLHUP|POLLNVAL}, {fd=5, events=POLLIN|POLLERR|POLLHUP|POLLNVAL}], 2, 3000) = 1 ([{fd=5, revents=POLLIN}]) read(5, ">>> Preparing source in /var/tmp"..., 4096) = 91 read(5, 0xb713b000, 4096) = -1 EAGAIN (Resource temporarily unavailable) write(8, ">>> Preparing source in /var/tmp"..., 91>>> Preparing source in /var/tmp/portage/dev-libs/libgcrypt-1.4.6/work/libgcrypt-1.4.6 ... ) = 91 write(7, ">>> Preparing source in /var/tmp"..., 91) = 91 poll([{fd=3, events=POLLIN|POLLERR|POLLHUP|POLLNVAL}, {fd=5, events=POLLIN|POLLERR|POLLHUP|POLLNVAL}], 2, 3000) = 0 (Timeout) poll([{fd=3, events=POLLIN|POLLERR|POLLHUP|POLLNVAL}, {fd=5, events=POLLIN|POLLERR|POLLHUP|POLLNVAL}], 2, 3000) = 0 (Timeout) poll([{fd=3, events=POLLIN|POLLERR|POLLHUP|POLLNVAL}, {fd=5, events=POLLIN|POLLERR|POLLHUP|POLLNVAL}], 2, 3000) = 0 (Timeout) The last line is repeated every 3sec. // Maria ^ permalink raw reply [flat|nested] 43+ messages in thread
* RE: [PATCH v2]Btrfs: pwrite blocked when writing from the mmaped buffer of the same page 2011-01-27 22:12 ` Maria Wikström @ 2011-01-28 1:26 ` Zhong, Xin 2011-01-28 2:54 ` Johannes Hirte 0 siblings, 1 reply; 43+ messages in thread From: Zhong, Xin @ 2011-01-28 1:26 UTC (permalink / raw) To: Maria Wikström, johannes.hirte; +Cc: linux-btrfs UGxlYXNlIHRyeSB0aGUgZml4IGluIGJlbG93IGxpbms6DQpodHRwOi8vd3d3LnNwaW5pY3MubmV0 L2xpc3RzL2xpbnV4LWJ0cmZzL21zZzA4MDUxLmh0bWwNCg0KVGhhbmtzIQ0KDQotLS0tLU9yaWdp bmFsIE1lc3NhZ2UtLS0tLQ0KRnJvbTogbGludXgtYnRyZnMtb3duZXJAdmdlci5rZXJuZWwub3Jn IFttYWlsdG86bGludXgtYnRyZnMtb3duZXJAdmdlci5rZXJuZWwub3JnXSBPbiBCZWhhbGYgT2Yg TWFyaWEgV2lrc3RyP20NClNlbnQ6IEZyaWRheSwgSmFudWFyeSAyOCwgMjAxMSA2OjEyIEFNDQpU bzogam9oYW5uZXMuaGlydGVAZmVtLnR1LWlsbWVuYXUuZGU7IFpob25nLCBYaW4NCkNjOiBsaW51 eC1idHJmc0B2Z2VyLmtlcm5lbC5vcmcNClN1YmplY3Q6IFJlOiBbUEFUQ0ggdjJdQnRyZnM6IHB3 cml0ZSBibG9ja2VkIHdoZW4gd3JpdGluZyBmcm9tIHRoZSBtbWFwZWQgYnVmZmVyIG9mIHRoZSBz YW1lIHBhZ2UNCg0KdG9yIDIwMTEtMDEtMjcga2xvY2thbiAxNDowOSArMDEwMCBza3JldiBKb2hh bm5lcyBIaXJ0ZTogDQo+IE9uIFRodXJzZGF5IDA5IERlY2VtYmVyIDIwMTAgMTA6MzA6MTQgWmhv bmcsIFhpbiB3cm90ZToNCj4gPiBUaGlzIHByb2JsZW0gaXMgZm91bmQgaW4gbWVlZ28gdGVzdGlu ZzoNCj4gPiBodHRwOi8vYnVncy5tZWVnby5jb20vc2hvd19idWcuY2dpP2lkPTY2NzINCj4gPiBB IGZpbGUgaW4gYnRyZnMgaXMgbW1hcGVkIGFuZCB0aGUgbW1hcGVkIGJ1ZmZlciBpcyBwYXNzZWQg dG8gcHdyaXRlIHRvDQo+ID4gd3JpdGUgdG8gdGhlIHNhbWUgcGFnZSBvZiB0aGUgc2FtZSBmaWxl LiBJbiBidHJmc19maWxlX2Fpb193cml0ZSgpLCB0aGUNCj4gPiBwYWdlcyBpcyBsb2NrZWQgYnkg cHJlcGFyZV9wYWdlcygpLiBTbyB3aGVuIGJ0cmZzX2NvcHlfZnJvbV91c2VyKCkgaXMNCj4gPiBj YWxsZWQsIHBhZ2UgZmF1bHQgaGFwcGVucyBhbmQgdGhlIHNhbWUgcGFnZSBuZWVkcyB0byBiZSBs b2NrZWQgYWdhaW4gaW4NCj4gPiBmaWxlbWFwX2ZhdWx0KCkuIFRoZSBmaXggaXMgdG8gbW92ZSBp b3ZfaXRlcl9mYXVsdF9pbl9yZWFkYWJsZSgpIGJlZm9yZQ0KPiA+IHByZXBhZ2VfcGFnZXMoKSB0 byBtYWtlIHBhZ2UgZmF1bHQgaGFwcGVuIGJlZm9yZSBwYWdlcyBhcmUgbG9ja2VkLiBBbmQNCj4g PiBhbHNvIGRpc2FibGUgcGFnZSBmYXVsdCBpbiBjcml0aWNhbCByZWdpb24gaW4gYnRyZnNfY29w eV9mcm9tX3VzZXIoKS4NCj4gPiANCj4gPiBSZXZpZXdlZC1ieTogWWFuLCBaaGVuZzx6aGVuZy56 LnlhbkBpbnRlbC5jb20+DQo+ID4gU2lnbmVkLW9mZi1ieTogWmhvbmcsIFhpbiA8eGluLnpob25n QGludGVsLmNvbT4NCj4gPiAtLS0NCj4gPiAgZnMvYnRyZnMvZmlsZS5jIHwgICA5Mg0KPiA+ICsr KysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKy0tLS0tLS0tLS0tLS0tLS0tLS0gMSBm aWxlcyBjaGFuZ2VkLA0KPiA+IDYwIGluc2VydGlvbnMoKyksIDMyIGRlbGV0aW9ucygtKQ0KPiA+ IA0KPiA+IGRpZmYgLS1naXQgYS9mcy9idHJmcy9maWxlLmMgYi9mcy9idHJmcy9maWxlLmMNCj4g PiBpbmRleCBjMWZhZGVkLi42NjgzNmQ4IDEwMDY0NA0KPiA+IC0tLSBhL2ZzL2J0cmZzL2ZpbGUu Yw0KPiA+ICsrKyBiL2ZzL2J0cmZzL2ZpbGUuYw0KPiA+IEBAIC00OCwzMCArNDgsMzQgQEAgc3Rh dGljIG5vaW5saW5lIGludCBidHJmc19jb3B5X2Zyb21fdXNlcihsb2ZmX3QgcG9zLA0KPiA+IGlu dCBudW1fcGFnZXMsIHN0cnVjdCBwYWdlICoqcHJlcGFyZWRfcGFnZXMsDQo+ID4gIAkJCQkJIHN0 cnVjdCBpb3ZfaXRlciAqaSkNCj4gPiAgew0KPiA+IC0Jc2l6ZV90IGNvcGllZDsNCj4gPiArCXNp emVfdCBjb3BpZWQgPSAwOw0KPiA+ICAJaW50IHBnID0gMDsNCj4gPiAgCWludCBvZmZzZXQgPSBw b3MgJiAoUEFHRV9DQUNIRV9TSVpFIC0gMSk7DQo+ID4gKwlpbnQgdG90YWxfY29waWVkID0gMDsN Cj4gPiANCj4gPiAgCXdoaWxlICh3cml0ZV9ieXRlcyA+IDApIHsNCj4gPiAgCQlzaXplX3QgY291 bnQgPSBtaW5fdChzaXplX3QsDQo+ID4gIAkJCQkgICAgIFBBR0VfQ0FDSEVfU0laRSAtIG9mZnNl dCwgd3JpdGVfYnl0ZXMpOw0KPiA+ICAJCXN0cnVjdCBwYWdlICpwYWdlID0gcHJlcGFyZWRfcGFn ZXNbcGddOw0KPiA+IC1hZ2FpbjoNCj4gPiAtCQlpZiAodW5saWtlbHkoaW92X2l0ZXJfZmF1bHRf aW5fcmVhZGFibGUoaSwgY291bnQpKSkNCj4gPiAtCQkJcmV0dXJuIC1FRkFVTFQ7DQo+ID4gLQ0K PiA+IC0JCS8qIENvcHkgZGF0YSBmcm9tIHVzZXJzcGFjZSB0byB0aGUgY3VycmVudCBwYWdlICov DQo+ID4gLQkJY29waWVkID0gaW92X2l0ZXJfY29weV9mcm9tX3VzZXIocGFnZSwgaSwgb2Zmc2V0 LCBjb3VudCk7DQo+ID4gKwkJLyoNCj4gPiArCQkgKiBDb3B5IGRhdGEgZnJvbSB1c2Vyc3BhY2Ug dG8gdGhlIGN1cnJlbnQgcGFnZQ0KPiA+ICsJCSAqDQo+ID4gKwkJICogRGlzYWJsZSBwYWdlZmF1 bHQgdG8gYXZvaWQgcmVjdXJzaXZlIGxvY2sgc2luY2UNCj4gPiArCQkgKiB0aGUgcGFnZXMgYXJl IGFscmVhZHkgbG9ja2VkDQo+ID4gKwkJICovDQo+ID4gKwkJcGFnZWZhdWx0X2Rpc2FibGUoKTsN Cj4gPiArCQljb3BpZWQgPSBpb3ZfaXRlcl9jb3B5X2Zyb21fdXNlcl9hdG9taWMocGFnZSwgaSwg b2Zmc2V0LCBjb3VudCk7DQo+ID4gKwkJcGFnZWZhdWx0X2VuYWJsZSgpOw0KPiA+IA0KPiA+ICAJ CS8qIEZsdXNoIHByb2Nlc3NvcidzIGRjYWNoZSBmb3IgdGhpcyBwYWdlICovDQo+ID4gIAkJZmx1 c2hfZGNhY2hlX3BhZ2UocGFnZSk7DQo+ID4gIAkJaW92X2l0ZXJfYWR2YW5jZShpLCBjb3BpZWQp Ow0KPiA+ICAJCXdyaXRlX2J5dGVzIC09IGNvcGllZDsNCj4gPiArCQl0b3RhbF9jb3BpZWQgKz0g Y29waWVkOw0KPiA+IA0KPiA+ICsJCS8qIFJldHVybiB0byBidHJmc19maWxlX2Fpb193cml0ZSB0 byBmYXVsdCBwYWdlICovDQo+ID4gIAkJaWYgKHVubGlrZWx5KGNvcGllZCA9PSAwKSkgew0KPiA+ IC0JCQljb3VudCA9IG1pbl90KHNpemVfdCwgUEFHRV9DQUNIRV9TSVpFIC0gb2Zmc2V0LA0KPiA+ IC0JCQkJICAgICAgaW92X2l0ZXJfc2luZ2xlX3NlZ19jb3VudChpKSk7DQo+ID4gLQkJCWdvdG8g YWdhaW47DQo+ID4gKwkJCWJyZWFrOw0KPiA+ICAJCX0NCj4gPiANCj4gPiAgCQlpZiAodW5saWtl bHkoY29waWVkIDwgUEFHRV9DQUNIRV9TSVpFIC0gb2Zmc2V0KSkgew0KPiA+IEBAIC04MSw3ICs4 NSw3IEBAIGFnYWluOg0KPiA+ICAJCQlvZmZzZXQgPSAwOw0KPiA+ICAJCX0NCj4gPiAgCX0NCj4g PiAtCXJldHVybiAwOw0KPiA+ICsJcmV0dXJuIHRvdGFsX2NvcGllZDsNCj4gPiAgfQ0KPiA+IA0K PiA+ICAvKg0KPiA+IEBAIC04NTQsNiArODU4LDggQEAgc3RhdGljIHNzaXplX3QgYnRyZnNfZmls ZV9haW9fd3JpdGUoc3RydWN0IGtpb2NiICppb2NiLA0KPiA+ICAJdW5zaWduZWQgbG9uZyBsYXN0 X2luZGV4Ow0KPiA+ICAJaW50IHdpbGxfd3JpdGU7DQo+ID4gIAlpbnQgYnVmZmVyZWQgPSAwOw0K PiA+ICsJaW50IGNvcGllZCA9IDA7DQo+ID4gKwlpbnQgZGlydHlfcGFnZXMgPSAwOw0KPiA+IA0K PiA+ICAJd2lsbF93cml0ZSA9ICgoZmlsZS0+Zl9mbGFncyAmIE9fRFNZTkMpIHx8IElTX1NZTkMo aW5vZGUpIHx8DQo+ID4gIAkJICAgICAgKGZpbGUtPmZfZmxhZ3MgJiBPX0RJUkVDVCkpOw0KPiA+ IEBAIC05NzAsNyArOTc2LDE3IEBAIHN0YXRpYyBzc2l6ZV90IGJ0cmZzX2ZpbGVfYWlvX3dyaXRl KHN0cnVjdCBraW9jYg0KPiA+ICppb2NiLCBXQVJOX09OKG51bV9wYWdlcyA+IG5ycHRycyk7DQo+ ID4gIAkJbWVtc2V0KHBhZ2VzLCAwLCBzaXplb2Yoc3RydWN0IHBhZ2UgKikgKiBucnB0cnMpOw0K PiA+IA0KPiA+IC0JCXJldCA9IGJ0cmZzX2RlbGFsbG9jX3Jlc2VydmVfc3BhY2UoaW5vZGUsIHdy aXRlX2J5dGVzKTsNCj4gPiArCQkvKg0KPiA+ICsJCSAqIEZhdWx0IHBhZ2VzIGJlZm9yZSBsb2Nr aW5nIHRoZW0gaW4gcHJlcGFyZV9wYWdlcw0KPiA+ICsJCSAqIHRvIGF2b2lkIHJlY3Vyc2l2ZSBs b2NrDQo+ID4gKwkJICovDQo+ID4gKwkJaWYgKHVubGlrZWx5KGlvdl9pdGVyX2ZhdWx0X2luX3Jl YWRhYmxlKCZpLCB3cml0ZV9ieXRlcykpKSB7DQo+ID4gKwkJCXJldCA9IC1FRkFVTFQ7DQo+ID4g KwkJCWdvdG8gb3V0Ow0KPiA+ICsJCX0NCj4gPiArDQo+ID4gKwkJcmV0ID0gYnRyZnNfZGVsYWxs b2NfcmVzZXJ2ZV9zcGFjZShpbm9kZSwNCj4gPiArCQkJCQludW1fcGFnZXMgPDwgUEFHRV9DQUNI RV9TSElGVCk7DQo+ID4gIAkJaWYgKHJldCkNCj4gPiAgCQkJZ290byBvdXQ7DQo+ID4gDQo+ID4g QEAgLTk3OCwzNyArOTk0LDQ5IEBAIHN0YXRpYyBzc2l6ZV90IGJ0cmZzX2ZpbGVfYWlvX3dyaXRl KHN0cnVjdCBraW9jYg0KPiA+ICppb2NiLCBwb3MsIGZpcnN0X2luZGV4LCBsYXN0X2luZGV4LA0K PiA+ICAJCQkJICAgIHdyaXRlX2J5dGVzKTsNCj4gPiAgCQlpZiAocmV0KSB7DQo+ID4gLQkJCWJ0 cmZzX2RlbGFsbG9jX3JlbGVhc2Vfc3BhY2UoaW5vZGUsIHdyaXRlX2J5dGVzKTsNCj4gPiArCQkJ YnRyZnNfZGVsYWxsb2NfcmVsZWFzZV9zcGFjZShpbm9kZSwNCj4gPiArCQkJCQludW1fcGFnZXMg PDwgUEFHRV9DQUNIRV9TSElGVCk7DQo+ID4gIAkJCWdvdG8gb3V0Ow0KPiA+ICAJCX0NCj4gPiAN Cj4gPiAtCQlyZXQgPSBidHJmc19jb3B5X2Zyb21fdXNlcihwb3MsIG51bV9wYWdlcywNCj4gPiAr CQljb3BpZWQgPSBidHJmc19jb3B5X2Zyb21fdXNlcihwb3MsIG51bV9wYWdlcywNCj4gPiAgCQkJ CQkgICB3cml0ZV9ieXRlcywgcGFnZXMsICZpKTsNCj4gPiAtCQlpZiAocmV0ID09IDApIHsNCj4g PiArCQlkaXJ0eV9wYWdlcyA9IChjb3BpZWQgKyBQQUdFX0NBQ0hFX1NJWkUgLSAxKSA+Pg0KPiA+ ICsJCQkJCVBBR0VfQ0FDSEVfU0hJRlQ7DQo+ID4gKw0KPiA+ICsJCWlmIChudW1fcGFnZXMgPiBk aXJ0eV9wYWdlcykgew0KPiA+ICsJCQlpZiAoY29waWVkID4gMCkNCj4gPiArCQkJCWF0b21pY19p bmMoDQo+ID4gKwkJCQkJJkJUUkZTX0koaW5vZGUpLT5vdXRzdGFuZGluZ19leHRlbnRzKTsNCj4g PiArCQkJYnRyZnNfZGVsYWxsb2NfcmVsZWFzZV9zcGFjZShpbm9kZSwNCj4gPiArCQkJCQkobnVt X3BhZ2VzIC0gZGlydHlfcGFnZXMpIDw8DQo+ID4gKwkJCQkJUEFHRV9DQUNIRV9TSElGVCk7DQo+ ID4gKwkJfQ0KPiA+ICsNCj4gPiArCQlpZiAoY29waWVkID4gMCkgew0KPiA+ICAJCQlkaXJ0eV9h bmRfcmVsZWFzZV9wYWdlcyhOVUxMLCByb290LCBmaWxlLCBwYWdlcywNCj4gPiAtCQkJCQkJbnVt X3BhZ2VzLCBwb3MsIHdyaXRlX2J5dGVzKTsNCj4gPiArCQkJCQkJZGlydHlfcGFnZXMsIHBvcywg Y29waWVkKTsNCj4gPiAgCQl9DQo+ID4gDQo+ID4gIAkJYnRyZnNfZHJvcF9wYWdlcyhwYWdlcywg bnVtX3BhZ2VzKTsNCj4gPiAtCQlpZiAocmV0KSB7DQo+ID4gLQkJCWJ0cmZzX2RlbGFsbG9jX3Jl bGVhc2Vfc3BhY2UoaW5vZGUsIHdyaXRlX2J5dGVzKTsNCj4gPiAtCQkJZ290byBvdXQ7DQo+ID4g LQkJfQ0KPiA+IA0KPiA+IC0JCWlmICh3aWxsX3dyaXRlKSB7DQo+ID4gLQkJCWZpbGVtYXBfZmRh dGF3cml0ZV9yYW5nZShpbm9kZS0+aV9tYXBwaW5nLCBwb3MsDQo+ID4gLQkJCQkJCSBwb3MgKyB3 cml0ZV9ieXRlcyAtIDEpOw0KPiA+IC0JCX0gZWxzZSB7DQo+ID4gLQkJCWJhbGFuY2VfZGlydHlf cGFnZXNfcmF0ZWxpbWl0ZWRfbnIoaW5vZGUtPmlfbWFwcGluZywNCj4gPiAtCQkJCQkJCSAgIG51 bV9wYWdlcyk7DQo+ID4gLQkJCWlmIChudW1fcGFnZXMgPA0KPiA+IC0JCQkgICAgKHJvb3QtPmxl YWZzaXplID4+IFBBR0VfQ0FDSEVfU0hJRlQpICsgMSkNCj4gPiAtCQkJCWJ0cmZzX2J0cmVlX2Jh bGFuY2VfZGlydHkocm9vdCwgMSk7DQo+ID4gLQkJCWJ0cmZzX3Rocm90dGxlKHJvb3QpOw0KPiA+ ICsJCWlmIChjb3BpZWQgPiAwKSB7DQo+ID4gKwkJCWlmICh3aWxsX3dyaXRlKSB7DQo+ID4gKwkJ CQlmaWxlbWFwX2ZkYXRhd3JpdGVfcmFuZ2UoaW5vZGUtPmlfbWFwcGluZywgcG9zLA0KPiA+ICsJ CQkJCQkJIHBvcyArIGNvcGllZCAtIDEpOw0KPiA+ICsJCQl9IGVsc2Ugew0KPiA+ICsJCQkJYmFs YW5jZV9kaXJ0eV9wYWdlc19yYXRlbGltaXRlZF9ucigNCj4gPiArCQkJCQkJCWlub2RlLT5pX21h cHBpbmcsDQo+ID4gKwkJCQkJCQlkaXJ0eV9wYWdlcyk7DQo+ID4gKwkJCQlpZiAoZGlydHlfcGFn ZXMgPA0KPiA+ICsJCQkJKHJvb3QtPmxlYWZzaXplID4+IFBBR0VfQ0FDSEVfU0hJRlQpICsgMSkN Cj4gPiArCQkJCQlidHJmc19idHJlZV9iYWxhbmNlX2RpcnR5KHJvb3QsIDEpOw0KPiA+ICsJCQkJ YnRyZnNfdGhyb3R0bGUocm9vdCk7DQo+ID4gKwkJCX0NCj4gPiAgCQl9DQo+ID4gDQo+ID4gLQkJ cG9zICs9IHdyaXRlX2J5dGVzOw0KPiA+IC0JCW51bV93cml0dGVuICs9IHdyaXRlX2J5dGVzOw0K PiA+ICsJCXBvcyArPSBjb3BpZWQ7DQo+ID4gKwkJbnVtX3dyaXR0ZW4gKz0gY29waWVkOw0KPiA+ IA0KPiA+ICAJCWNvbmRfcmVzY2hlZCgpOw0KPiA+ICAJfQ0KPiANCj4gVGhpcyBwYXRjaCBicmVh a3Mgb25lIG9mIG15IEdlbnRvbyBib3hlcy4gV2hlbiBJIHRyeSB0byBpbnN0YWxsL3VwZGF0ZSB2 aWEgDQo+IGVtZXJnZSwgc29tZSBwYWNrYWdlcyBoYW5nLiBJdCBzZWVtcyB0aGF0IGl0J3MgYWx3 YXlzIGEgInN2biBpbmZvIiBwcm9jZXNzIA0KPiB0aGF0IGlzIHN0dWNrIGluIGtlcm5lbCBlYXRp bmcgMTAwJSBDUFUuIEkgZG9uJ3Qga25vdyBob3cgc3ZuIGlzIGludm9sdmVkIA0KPiBoZXJlLCBi dXQgcmV2ZXJ0aW5nIHRoaXMgcGF0Y2ggbWFrZXMgdGhlIHN5c3RlbSB3b3JrIGFnYWluLiBJJ2xs IHRyeSB0byBnZXQgYSANCj4gc2ltcGxlIHRlc3RjYXNlLg0KPiANCj4gcmVnYXJkcywNCj4gICBK b2hhbm5lcw0KDQpUaGUgc2FtZSB0aGluZyBoYXBwZW5zIGhlcmUuIFRoZSBvbmx5IHdheSB0byBr aWxsIHRoZSBwcm9jZXNzIChzdm4gaW5mbykNCmlzIHRvIHJlc3RhcnQgdGhlIGNvbXB1dGVyLiBS dW5uaW5nIHZhbmlsbGEgMi42LjM3IHdpdGhvdXQgcGF0Y2hlcy4NCg0KI3N0cmFjZSBlbWVyZ2Ug bGliZ2NyeXB0DQotLSBjdXQgLS0gDQpvcGVuKCIvdmFyL3RtcC9wb3J0YWdlL2Rldi1saWJzL2xp YmdjcnlwdC0xLjQuNi90ZW1wL2J1aWxkLmxvZyIsDQpPX1dST05MWXxPX0NSRUFUfE9fQVBQRU5E fE9fTEFSR0VGSUxFLCAwNjY2KSA9IDcNCmZzdGF0NjQoNywge3N0X21vZGU9U19JRlJFR3wwNjYw LCBzdF9zaXplPTQxNiwgLi4ufSkgPSAwDQptbWFwMihOVUxMLCA0MDk2LCBQUk9UX1JFQUR8UFJP VF9XUklURSwgTUFQX1BSSVZBVEV8TUFQX0FOT05ZTU9VUywgLTEsDQowKSA9IDB4YjcxM2EwMDAN CmZzdGF0NjQoNywge3N0X21vZGU9U19JRlJFR3wwNjYwLCBzdF9zaXplPTQxNiwgLi4ufSkgPSAw DQpfbGxzZWVrKDcsIDQxNiwgWzQxNl0sIFNFRUtfU0VUKSAgICAgICAgPSAwDQpmc3RhdDY0KDcs IHtzdF9tb2RlPVNfSUZSRUd8MDY2MCwgc3Rfc2l6ZT00MTYsIC4uLn0pID0gMA0Kc3RhdDY0KCIv dmFyL3RtcC9wb3J0YWdlL2Rldi1saWJzL2xpYmdjcnlwdC0xLjQuNi90ZW1wL2J1aWxkLmxvZyIs DQp7c3RfbW9kZT1TX0lGUkVHfDA2NjAsIHN0X3NpemU9NDE2LCAuLi59KSA9IDANCmR1cCgxKSAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICA9IDgNCmZjbnRsNjQoOCwgRl9HRVRGTCkg ICAgICAgICAgICAgICAgICAgICA9IDB4MiAoZmxhZ3MgT19SRFdSKQ0KZnN0YXQ2NCg4LCB7c3Rf bW9kZT1TX0lGQ0hSfDA2MjAsIHN0X3JkZXY9bWFrZWRldigxMzYsIDMpLCAuLi59KSA9IDANCm1t YXAyKE5VTEwsIDQwOTYsIFBST1RfUkVBRHxQUk9UX1dSSVRFLCBNQVBfUFJJVkFURXxNQVBfQU5P TllNT1VTLCAtMSwNCjApID0gMHhiNzAwNTAwMA0KX2xsc2Vlayg4LCAwLCAweGJmZjgwY2ZjLCBT RUVLX0NVUikgICAgID0gLTEgRVNQSVBFIChJbGxlZ2FsIHNlZWspDQpmc3RhdDY0KDgsIHtzdF9t b2RlPVNfSUZDSFJ8MDYyMCwgc3RfcmRldj1tYWtlZGV2KDEzNiwgMyksIC4uLn0pID0gMA0Kc3Rh dDY0KCIvdmFyL3RtcC9wb3J0YWdlL2Rldi1saWJzL2xpYmdjcnlwdC0xLjQuNi90ZW1wL2Vudmly b25tZW50IiwNCntzdF9tb2RlPVNfSUZSRUd8MDY2NCwgc3Rfc2l6ZT0xMDU5NzAsIC4uLn0pID0g MA0KY2xvbmUoY2hpbGRfc3RhY2s9MCwgZmxhZ3M9Q0xPTkVfQ0hJTERfQ0xFQVJUSUR8Q0xPTkVf Q0hJTERfU0VUVElEfA0KU0lHQ0hMRCwgY2hpbGRfdGlkcHRyPTB4Yjc0MzA3MjgpID0gMTE2NjUN CmNsb3NlKDYpICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICA9IDANCmdldHRpbWVvZmRh eSh7MTI5NjE2MjgzNiwgMTI2MTkyfSwgTlVMTCkgPSAwDQpwb2xsKFt7ZmQ9MywgZXZlbnRzPVBP TExJTnxQT0xMRVJSfFBPTExIVVB8UE9MTE5WQUx9LCB7ZmQ9NSwNCmV2ZW50cz1QT0xMSU58UE9M TEVSUnxQT0xMSFVQfFBPTExOVkFMfV0sIDIsIDMwMDApID0gMSAoW3tmZD01LA0KcmV2ZW50cz1Q T0xMSU59XSkNCnJlYWQoNSwgIj4+PiBQcmVwYXJpbmcgc291cmNlIGluIC92YXIvdG1wIi4uLiwg NDA5NikgPSA5MQ0KcmVhZCg1LCAweGI3MTNiMDAwLCA0MDk2KSAgICAgICAgICAgICAgID0gLTEg RUFHQUlOIChSZXNvdXJjZQ0KdGVtcG9yYXJpbHkgdW5hdmFpbGFibGUpDQp3cml0ZSg4LCAiPj4+ IFByZXBhcmluZyBzb3VyY2UgaW4gL3Zhci90bXAiLi4uLCA5MT4+PiBQcmVwYXJpbmcgc291cmNl DQppbiAvdmFyL3RtcC9wb3J0YWdlL2Rldi1saWJzL2xpYmdjcnlwdC0xLjQuNi93b3JrL2xpYmdj cnlwdC0xLjQuNiAuLi4NCikgPSA5MQ0Kd3JpdGUoNywgIj4+PiBQcmVwYXJpbmcgc291cmNlIGlu IC92YXIvdG1wIi4uLiwgOTEpID0gOTENCnBvbGwoW3tmZD0zLCBldmVudHM9UE9MTElOfFBPTExF UlJ8UE9MTEhVUHxQT0xMTlZBTH0sIHtmZD01LA0KZXZlbnRzPVBPTExJTnxQT0xMRVJSfFBPTExI VVB8UE9MTE5WQUx9XSwgMiwgMzAwMCkgPSAwIChUaW1lb3V0KQ0KcG9sbChbe2ZkPTMsIGV2ZW50 cz1QT0xMSU58UE9MTEVSUnxQT0xMSFVQfFBPTExOVkFMfSwge2ZkPTUsDQpldmVudHM9UE9MTElO fFBPTExFUlJ8UE9MTEhVUHxQT0xMTlZBTH1dLCAyLCAzMDAwKSA9IDAgKFRpbWVvdXQpDQpwb2xs KFt7ZmQ9MywgZXZlbnRzPVBPTExJTnxQT0xMRVJSfFBPTExIVVB8UE9MTE5WQUx9LCB7ZmQ9NSwN CmV2ZW50cz1QT0xMSU58UE9MTEVSUnxQT0xMSFVQfFBPTExOVkFMfV0sIDIsIDMwMDApID0gMCAo VGltZW91dCkNCg0KVGhlIGxhc3QgbGluZSBpcyByZXBlYXRlZCBldmVyeSAzc2VjLg0KDQovLyBN YXJpYQ0KDQoNCi0tDQpUbyB1bnN1YnNjcmliZSBmcm9tIHRoaXMgbGlzdDogc2VuZCB0aGUgbGlu ZSAidW5zdWJzY3JpYmUgbGludXgtYnRyZnMiIGluDQp0aGUgYm9keSBvZiBhIG1lc3NhZ2UgdG8g bWFqb3Jkb21vQHZnZXIua2VybmVsLm9yZw0KTW9yZSBtYWpvcmRvbW8gaW5mbyBhdCAgaHR0cDov L3ZnZXIua2VybmVsLm9yZy9tYWpvcmRvbW8taW5mby5odG1sDQo= ^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [PATCH v2]Btrfs: pwrite blocked when writing from the mmaped buffer of the same page 2011-01-28 1:26 ` Zhong, Xin @ 2011-01-28 2:54 ` Johannes Hirte 2011-01-28 3:53 ` Zhong, Xin 2011-01-28 16:47 ` Maria Wikström 0 siblings, 2 replies; 43+ messages in thread From: Johannes Hirte @ 2011-01-28 2:54 UTC (permalink / raw) To: Zhong, Xin; +Cc: Maria Wikström, linux-btrfs On Friday 28 January 2011 02:26:43 Zhong, Xin wrote: > Please try the fix in below link: > http://www.spinics.net/lists/linux-btrfs/msg08051.html > > Thanks! This doesn't fix it for me. At least there is a difference. Whereas the svn process started consuming 100% CPU without any further interaction before, the system just hang now. The svn process starts eating the CPU when I cancel the emerge via ctrl-c. Additional I see a flush-btrfs task now consuming CPU time. regards, Johannes ^ permalink raw reply [flat|nested] 43+ messages in thread
* RE: [PATCH v2]Btrfs: pwrite blocked when writing from the mmaped buffer of the same page 2011-01-28 2:54 ` Johannes Hirte @ 2011-01-28 3:53 ` Zhong, Xin 2011-02-01 23:34 ` Johannes Hirte 2011-01-28 16:47 ` Maria Wikström 1 sibling, 1 reply; 43+ messages in thread From: Zhong, Xin @ 2011-01-28 3:53 UTC (permalink / raw) To: Johannes Hirte; +Cc: Maria Wikström, linux-btrfs Q291bGQgeW91IGRlc2NyaWJlIHRoZSBzdGVwcyB0byByZWNyZWF0ZSBpdD8gDQpJdCB3aWxsIGJl IGEgZ3JlYXQgaGVscCBmb3IgbWUgdG8gbG9vayBmdXJ0aGVyLiBUaGFua3MhDQoNCi0tLS0tT3Jp Z2luYWwgTWVzc2FnZS0tLS0tDQpGcm9tOiBKb2hhbm5lcyBIaXJ0ZSBbbWFpbHRvOmpvaGFubmVz LmhpcnRlQGZlbS50dS1pbG1lbmF1LmRlXSANClNlbnQ6IEZyaWRheSwgSmFudWFyeSAyOCwgMjAx MSAxMDo1NSBBTQ0KVG86IFpob25nLCBYaW4NCkNjOiBNYXJpYSBXaWtzdHLDtm07IGxpbnV4LWJ0 cmZzQHZnZXIua2VybmVsLm9yZw0KU3ViamVjdDogUmU6IFtQQVRDSCB2Ml1CdHJmczogcHdyaXRl IGJsb2NrZWQgd2hlbiB3cml0aW5nIGZyb20gdGhlIG1tYXBlZCBidWZmZXIgb2YgdGhlIHNhbWUg cGFnZQ0KDQpPbiBGcmlkYXkgMjggSmFudWFyeSAyMDExIDAyOjI2OjQzIFpob25nLCBYaW4gd3Jv dGU6DQo+IFBsZWFzZSB0cnkgdGhlIGZpeCBpbiBiZWxvdyBsaW5rOg0KPiBodHRwOi8vd3d3LnNw aW5pY3MubmV0L2xpc3RzL2xpbnV4LWJ0cmZzL21zZzA4MDUxLmh0bWwNCj4gDQo+IFRoYW5rcyEN Cg0KVGhpcyBkb2Vzbid0IGZpeCBpdCBmb3IgbWUuIEF0IGxlYXN0IHRoZXJlIGlzIGEgZGlmZmVy ZW5jZS4gV2hlcmVhcyB0aGUgc3ZuIA0KcHJvY2VzcyBzdGFydGVkIGNvbnN1bWluZyAxMDAlIENQ VSB3aXRob3V0IGFueSBmdXJ0aGVyIGludGVyYWN0aW9uIGJlZm9yZSwgdGhlIA0Kc3lzdGVtIGp1 c3QgaGFuZyBub3cuIFRoZSBzdm4gcHJvY2VzcyBzdGFydHMgZWF0aW5nIHRoZSBDUFUgd2hlbiBJ IGNhbmNlbCB0aGUgDQplbWVyZ2UgdmlhIGN0cmwtYy4gQWRkaXRpb25hbCBJIHNlZSBhIGZsdXNo LWJ0cmZzIHRhc2sgbm93IGNvbnN1bWluZyBDUFUgdGltZS4NCg0KcmVnYXJkcywNCiAgSm9oYW5u ZXMNCg== ^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [PATCH v2]Btrfs: pwrite blocked when writing from the mmaped buffer of the same page 2011-01-28 3:53 ` Zhong, Xin @ 2011-02-01 23:34 ` Johannes Hirte 2011-02-11 4:39 ` Zhong, Xin 0 siblings, 1 reply; 43+ messages in thread From: Johannes Hirte @ 2011-02-01 23:34 UTC (permalink / raw) To: Zhong, Xin; +Cc: Maria Wikström, linux-btrfs On Friday 28 January 2011 04:53:24 Zhong, Xin wrote: > Could you describe the steps to recreate it? > It will be a great help for me to look further. Thanks! It's a little strange. I have to systems with btrfs, both Gentoo-based. One is affected by this bug the other is not. On the affected system it is enough to do a 'emerge dev-libs/libgcrypt' that should normaly compile and install libgcrypt. The emerge command is part of portage, the package management of Gentoo. The strace output looks similar to the one from Maria: open("/home/tmp/portage/dev-libs/libgcrypt-1.4.6/.ipc_in", O_RDONLY| O_NONBLOCK|O_LARGEFILE) = 3 fstat64(3, {st_mode=S_IFIFO|0770, st_size=0, ...}) = 0 ioctl(3, SNDCTL_TMR_TIMEBASE or SNDRV_TIMER_IOCTL_NEXT_DEVICE or TCGETS, 0xbff5f678) = -1 EINVAL (Invalid argument) open("/dev/ptmx", O_RDWR) = 5 ioctl(5, SNDCTL_TMR_TIMEBASE or SNDRV_TIMER_IOCTL_NEXT_DEVICE or TCGETS, {B38400 opost isig icanon echo ...}) = 0 ioctl(5, TIOCGPTN, [2]) = 0 stat64("/dev/pts/2", {st_mode=S_IFCHR|0620, st_rdev=makedev(136, 2), ...}) = 0 getuid32() = 0 ioctl(5, TIOCSPTLCK, [0]) = 0 ioctl(5, SNDCTL_TMR_TIMEBASE or SNDRV_TIMER_IOCTL_NEXT_DEVICE or TCGETS, {B38400 opost isig icanon echo ...}) = 0 ioctl(5, TIOCGPTN, [2]) = 0 stat64("/dev/pts/2", {st_mode=S_IFCHR|0620, st_rdev=makedev(136, 2), ...}) = 0 open("/dev/pts/2", O_RDWR|O_NOCTTY) = 6 ioctl(6, SNDCTL_TMR_TIMEBASE or SNDRV_TIMER_IOCTL_NEXT_DEVICE or TCGETS, {B38400 opost isig icanon echo ...}) = 0 ioctl(6, SNDCTL_TMR_TIMEBASE or SNDRV_TIMER_IOCTL_NEXT_DEVICE or TCGETS, {B38400 opost isig icanon echo ...}) = 0 ioctl(6, SNDCTL_TMR_START or SNDRV_TIMER_IOCTL_TREAD or TCSETS, {B38400 -opost isig icanon echo ...}) = 0 ioctl(1, SNDCTL_TMR_TIMEBASE or SNDRV_TIMER_IOCTL_NEXT_DEVICE or TCGETS, {B38400 opost isig icanon echo ...}) = 0 ioctl(1, SNDCTL_TMR_TIMEBASE or SNDRV_TIMER_IOCTL_NEXT_DEVICE or TCGETS, {B38400 opost isig icanon echo ...}) = 0 ioctl(1, SNDCTL_TMR_TIMEBASE or SNDRV_TIMER_IOCTL_NEXT_DEVICE or TCGETS, {B38400 opost isig icanon echo ...}) = 0 stat64("/root/.terminfo", 0xbff5e790) = -1 ENOENT (No such file or directory) stat64("/etc/terminfo", {st_mode=S_IFDIR|0755, st_size=14, ...}) = 0 access("/etc/terminfo/x/xterm", R_OK) = 0 open("/etc/terminfo/x/xterm", O_RDONLY|O_LARGEFILE) = 7 read(7, "\32\0010\0&\0\17\0\235\1l\5xterm|xterm terminal"..., 4097) = 3258 close(7) = 0 ioctl(1, SNDCTL_TMR_TIMEBASE or SNDRV_TIMER_IOCTL_NEXT_DEVICE or TCGETS, {B38400 opost isig icanon echo ...}) = 0 ioctl(1, SNDCTL_TMR_TIMEBASE or SNDRV_TIMER_IOCTL_NEXT_DEVICE or TCGETS, {B38400 opost isig icanon echo ...}) = 0 ioctl(1, SNDCTL_TMR_TIMEBASE or SNDRV_TIMER_IOCTL_NEXT_DEVICE or TCGETS, {B38400 opost isig icanon echo ...}) = 0 ioctl(1, TIOCGWINSZ, {ws_row=40, ws_col=207, ws_xpixel=0, ws_ypixel=0}) = 0 access("/usr/local/sbin/stty", X_OK) = -1 ENOENT (No such file or directory) access("/usr/local/bin/stty", X_OK) = -1 ENOENT (No such file or directory) access("/usr/sbin/stty", X_OK) = -1 ENOENT (No such file or directory) access("/usr/bin/stty", X_OK) = -1 ENOENT (No such file or directory) access("/sbin/stty", X_OK) = -1 ENOENT (No such file or directory) access("/bin/stty", X_OK) = 0 stat64("/bin/stty", {st_mode=S_IFREG|0755, st_size=58836, ...}) = 0 clone(child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0xb753d728) = 2752 waitpid(2752, [{WIFEXITED(s) && WEXITSTATUS(s) == 0}], 0) = 2752 --- SIGCHLD (Child exited) @ 0 (0) --- fcntl64(5, F_GETFL) = 0x2 (flags O_RDWR) fcntl64(5, F_SETFL, O_RDWR|O_NONBLOCK) = 0 fstat64(5, {st_mode=S_IFCHR|0666, st_rdev=makedev(5, 2), ...}) = 0 ioctl(5, SNDCTL_TMR_TIMEBASE or SNDRV_TIMER_IOCTL_NEXT_DEVICE or TCGETS, {B38400 -opost isig icanon echo ...}) = 0 open("/home/tmp/portage/dev-libs/libgcrypt-1.4.6/temp/build.log", O_WRONLY| O_CREAT|O_APPEND|O_LARGEFILE, 0666) = 7 fstat64(7, {st_mode=S_IFREG|0660, st_size=480, ...}) = 0 _llseek(7, 0, [480], SEEK_END) = 0 ioctl(7, SNDCTL_TMR_TIMEBASE or SNDRV_TIMER_IOCTL_NEXT_DEVICE or TCGETS, 0xbff5fad8) = -1 ENOTTY (Inappropriate ioctl for device) fstat64(7, {st_mode=S_IFREG|0660, st_size=480, ...}) = 0 _llseek(7, 0, [480], SEEK_CUR) = 0 stat64("/home/tmp/portage/dev-libs/libgcrypt-1.4.6/temp/build.log", {st_mode=S_IFREG|0660, st_size=480, ...}) = 0 dup(1) = 8 fstat64(8, {st_mode=S_IFCHR|0620, st_rdev=makedev(136, 0), ...}) = 0 ioctl(8, SNDCTL_TMR_TIMEBASE or SNDRV_TIMER_IOCTL_NEXT_DEVICE or TCGETS, {B38400 opost isig icanon echo ...}) = 0 fstat64(8, {st_mode=S_IFCHR|0620, st_rdev=makedev(136, 0), ...}) = 0 _llseek(8, 0, 0xbff5f820, SEEK_CUR) = -1 ESPIPE (Illegal seek) stat64("/home/tmp/portage/dev-libs/libgcrypt-1.4.6/temp/environment", {st_mode=S_IFREG|0664, st_size=106597, ...}) = 0 clone(child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0xb753d728) = 2753 close(6) = 0 gettimeofday({1296577457, 11117}, NULL) = 0 poll([{fd=3, events=POLLIN|POLLERR|POLLHUP|POLLNVAL}, {fd=5, events=POLLIN| POLLERR|POLLHUP|POLLNVAL}], 2, 3000) = 1 ([{fd=5, revents=POLLIN}]) read(5, ">>> Preparing source in /home/tm"..., 4096) = 92 write(8, ">>> Preparing source in /home/tm"..., 92) = 92 write(7, ">>> Preparing source in /home/tm"..., 92) = 92 poll([{fd=3, events=POLLIN|POLLERR|POLLHUP|POLLNVAL}, {fd=5, events=POLLIN| POLLERR|POLLHUP|POLLNVAL}], 2, 3000) = 0 (Timeout) poll([{fd=3, events=POLLIN|POLLERR|POLLHUP|POLLNVAL}, {fd=5, events=POLLIN| POLLERR|POLLHUP|POLLNVAL}], 2, 3000) = 0 (Timeout) poll([{fd=3, events=POLLIN|POLLERR|POLLHUP|POLLNVAL}, {fd=5, events=POLLIN| POLLERR|POLLHUP|POLLNVAL}], 2, 3000) = 0 (Timeout) poll([{fd=3, events=POLLIN|POLLERR|POLLHUP|POLLNVAL}, {fd=5, events=POLLIN| POLLERR|POLLHUP|POLLNVAL}], 2, 3000) = 0 (Timeout) poll([{fd=3, events=POLLIN|POLLERR|POLLHUP|POLLNVAL}, {fd=5, events=POLLIN| POLLERR|POLLHUP|POLLNVAL}], 2, 3000) = 0 (Timeout) poll([{fd=3, events=POLLIN|POLLERR|POLLHUP|POLLNVAL}, {fd=5, events=POLLIN| POLLERR|POLLHUP|POLLNVAL}], 2, 3000) = 0 (Timeout) poll([{fd=3, events=POLLIN|POLLERR|POLLHUP|POLLNVAL}, {fd=5, events=POLLIN| POLLERR|POLLHUP|POLLNVAL}], 2, 3000) = 0 (Timeout) poll([{fd=3, events=POLLIN|POLLERR|POLLHUP|POLLNVAL}, {fd=5, events=POLLIN| POLLERR|POLLHUP|POLLNVAL}], 2, 3000) = 0 (Timeout) poll([{fd=3, events=POLLIN|POLLERR|POLLHUP|POLLNVAL}, {fd=5, events=POLLIN| POLLERR|POLLHUP|POLLNVAL}], 2, 3000) = 0 (Timeout) poll([{fd=3, events=POLLIN|POLLERR|POLLHUP|POLLNVAL}, {fd=5, events=POLLIN| POLLERR|POLLHUP|POLLNVAL}], 2, 3000) = 0 (Timeout) poll([{fd=3, events=POLLIN|POLLERR|POLLHUP|POLLNVAL}, {fd=5, events=POLLIN| POLLERR|POLLHUP|POLLNVAL}], 2, 3000) = 0 (Timeout) poll([{fd=3, events=POLLIN|POLLERR|POLLHUP|POLLNVAL}, {fd=5, events=POLLIN| POLLERR|POLLHUP|POLLNVAL}], 2, 3000) = 0 (Timeout) poll([{fd=3, events=POLLIN|POLLERR|POLLHUP|POLLNVAL}, {fd=5, events=POLLIN| POLLERR|POLLHUP|POLLNVAL}], 2, 3000) = 0 (Timeout) poll([{fd=3, events=POLLIN|POLLERR|POLLHUP|POLLNVAL}, {fd=5, events=POLLIN| POLLERR|POLLHUP|POLLNVAL}], 2, 3000) = 0 (Timeout) poll([{fd=3, events=POLLIN|POLLERR|POLLHUP|POLLNVAL}, {fd=5, events=POLLIN| POLLERR|POLLHUP|POLLNVAL}], 2, 3000) = 0 (Timeout) poll([{fd=3, events=POLLIN|POLLERR|POLLHUP|POLLNVAL}, {fd=5, events=POLLIN| POLLERR|POLLHUP|POLLNVAL}], 2, 3000 <unfinished ...> ^ permalink raw reply [flat|nested] 43+ messages in thread
* RE: [PATCH v2]Btrfs: pwrite blocked when writing from the mmaped buffer of the same page 2011-02-01 23:34 ` Johannes Hirte @ 2011-02-11 4:39 ` Zhong, Xin 2011-02-18 11:31 ` Maria Wikström 2011-02-22 22:27 ` Johannes Hirte 0 siblings, 2 replies; 43+ messages in thread From: Zhong, Xin @ 2011-02-11 4:39 UTC (permalink / raw) To: Johannes Hirte; +Cc: Maria Wikström, linux-btrfs SGksDQoNCkNvdWxkIHlvdSBwYXN0ZSB0aGUgb3V0cHV0IG9mIHN5c3JxK3QgaGVyZT8gVGhhbmtz IQ0KDQotLS0tLU9yaWdpbmFsIE1lc3NhZ2UtLS0tLQ0KRnJvbTogSm9oYW5uZXMgSGlydGUgW21h aWx0bzpqb2hhbm5lcy5oaXJ0ZUBmZW0udHUtaWxtZW5hdS5kZV0gDQpTZW50OiBXZWRuZXNkYXks IEZlYnJ1YXJ5IDAyLCAyMDExIDc6MzUgQU0NClRvOiBaaG9uZywgWGluDQpDYzogTWFyaWEgV2lr c3Ryw7ZtOyBsaW51eC1idHJmc0B2Z2VyLmtlcm5lbC5vcmcNClN1YmplY3Q6IFJlOiBbUEFUQ0gg djJdQnRyZnM6IHB3cml0ZSBibG9ja2VkIHdoZW4gd3JpdGluZyBmcm9tIHRoZSBtbWFwZWQgYnVm ZmVyIG9mIHRoZSBzYW1lIHBhZ2UNCg0KT24gRnJpZGF5IDI4IEphbnVhcnkgMjAxMSAwNDo1Mzoy NCBaaG9uZywgWGluIHdyb3RlOg0KPiBDb3VsZCB5b3UgZGVzY3JpYmUgdGhlIHN0ZXBzIHRvIHJl Y3JlYXRlIGl0Pw0KPiBJdCB3aWxsIGJlIGEgZ3JlYXQgaGVscCBmb3IgbWUgdG8gbG9vayBmdXJ0 aGVyLiBUaGFua3MhDQoNCkl0J3MgYSBsaXR0bGUgc3RyYW5nZS4gSSBoYXZlIHRvIHN5c3RlbXMg d2l0aCBidHJmcywgYm90aCBHZW50b28tYmFzZWQuIE9uZSBpcyANCmFmZmVjdGVkIGJ5IHRoaXMg YnVnIHRoZSBvdGhlciBpcyBub3QuIE9uIHRoZSBhZmZlY3RlZCBzeXN0ZW0gaXQgaXMgZW5vdWdo IHRvIGRvIA0KYSAnZW1lcmdlIGRldi1saWJzL2xpYmdjcnlwdCcgdGhhdCBzaG91bGQgbm9ybWFs eSBjb21waWxlIGFuZCBpbnN0YWxsIA0KbGliZ2NyeXB0LiBUaGUgZW1lcmdlIGNvbW1hbmQgaXMg cGFydCBvZiBwb3J0YWdlLCB0aGUgcGFja2FnZSBtYW5hZ2VtZW50IG9mIA0KR2VudG9vLiANClRo ZSBzdHJhY2Ugb3V0cHV0IGxvb2tzIHNpbWlsYXIgdG8gdGhlIG9uZSBmcm9tIE1hcmlhOg0KDQpv cGVuKCIvaG9tZS90bXAvcG9ydGFnZS9kZXYtbGlicy9saWJnY3J5cHQtMS40LjYvLmlwY19pbiIs IE9fUkRPTkxZfA0KT19OT05CTE9DS3xPX0xBUkdFRklMRSkgPSAzDQpmc3RhdDY0KDMsIHtzdF9t b2RlPVNfSUZJRk98MDc3MCwgc3Rfc2l6ZT0wLCAuLi59KSA9IDANCmlvY3RsKDMsIFNORENUTF9U TVJfVElNRUJBU0Ugb3IgU05EUlZfVElNRVJfSU9DVExfTkVYVF9ERVZJQ0Ugb3IgVENHRVRTLCAN CjB4YmZmNWY2NzgpID0gLTEgRUlOVkFMIChJbnZhbGlkIGFyZ3VtZW50KQ0Kb3BlbigiL2Rldi9w dG14IiwgT19SRFdSKSAgICAgICAgICAgICAgID0gNQ0KaW9jdGwoNSwgU05EQ1RMX1RNUl9USU1F QkFTRSBvciBTTkRSVl9USU1FUl9JT0NUTF9ORVhUX0RFVklDRSBvciBUQ0dFVFMsIA0Ke0IzODQw MCBvcG9zdCBpc2lnIGljYW5vbiBlY2hvIC4uLn0pID0gMA0KaW9jdGwoNSwgVElPQ0dQVE4sIFsy XSkgICAgICAgICAgICAgICAgID0gMA0Kc3RhdDY0KCIvZGV2L3B0cy8yIiwge3N0X21vZGU9U19J RkNIUnwwNjIwLCBzdF9yZGV2PW1ha2VkZXYoMTM2LCAyKSwgLi4ufSkgPSAwDQpnZXR1aWQzMigp ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgPSAwDQppb2N0bCg1LCBUSU9DU1BUTENLLCBb MF0pICAgICAgICAgICAgICAgPSAwDQppb2N0bCg1LCBTTkRDVExfVE1SX1RJTUVCQVNFIG9yIFNO RFJWX1RJTUVSX0lPQ1RMX05FWFRfREVWSUNFIG9yIFRDR0VUUywgDQp7QjM4NDAwIG9wb3N0IGlz aWcgaWNhbm9uIGVjaG8gLi4ufSkgPSAwDQppb2N0bCg1LCBUSU9DR1BUTiwgWzJdKSAgICAgICAg ICAgICAgICAgPSAwDQpzdGF0NjQoIi9kZXYvcHRzLzIiLCB7c3RfbW9kZT1TX0lGQ0hSfDA2MjAs IHN0X3JkZXY9bWFrZWRldigxMzYsIDIpLCAuLi59KSA9IDANCm9wZW4oIi9kZXYvcHRzLzIiLCBP X1JEV1J8T19OT0NUVFkpICAgICA9IDYNCmlvY3RsKDYsIFNORENUTF9UTVJfVElNRUJBU0Ugb3Ig U05EUlZfVElNRVJfSU9DVExfTkVYVF9ERVZJQ0Ugb3IgVENHRVRTLCANCntCMzg0MDAgb3Bvc3Qg aXNpZyBpY2Fub24gZWNobyAuLi59KSA9IDANCmlvY3RsKDYsIFNORENUTF9UTVJfVElNRUJBU0Ug b3IgU05EUlZfVElNRVJfSU9DVExfTkVYVF9ERVZJQ0Ugb3IgVENHRVRTLCANCntCMzg0MDAgb3Bv c3QgaXNpZyBpY2Fub24gZWNobyAuLi59KSA9IDANCmlvY3RsKDYsIFNORENUTF9UTVJfU1RBUlQg b3IgU05EUlZfVElNRVJfSU9DVExfVFJFQUQgb3IgVENTRVRTLCB7QjM4NDAwIC1vcG9zdCANCmlz aWcgaWNhbm9uIGVjaG8gLi4ufSkgPSAwDQppb2N0bCgxLCBTTkRDVExfVE1SX1RJTUVCQVNFIG9y IFNORFJWX1RJTUVSX0lPQ1RMX05FWFRfREVWSUNFIG9yIFRDR0VUUywgDQp7QjM4NDAwIG9wb3N0 IGlzaWcgaWNhbm9uIGVjaG8gLi4ufSkgPSAwDQppb2N0bCgxLCBTTkRDVExfVE1SX1RJTUVCQVNF IG9yIFNORFJWX1RJTUVSX0lPQ1RMX05FWFRfREVWSUNFIG9yIFRDR0VUUywgDQp7QjM4NDAwIG9w b3N0IGlzaWcgaWNhbm9uIGVjaG8gLi4ufSkgPSAwDQppb2N0bCgxLCBTTkRDVExfVE1SX1RJTUVC QVNFIG9yIFNORFJWX1RJTUVSX0lPQ1RMX05FWFRfREVWSUNFIG9yIFRDR0VUUywgDQp7QjM4NDAw IG9wb3N0IGlzaWcgaWNhbm9uIGVjaG8gLi4ufSkgPSAwDQpzdGF0NjQoIi9yb290Ly50ZXJtaW5m byIsIDB4YmZmNWU3OTApICAgPSAtMSBFTk9FTlQgKE5vIHN1Y2ggZmlsZSBvciBkaXJlY3Rvcnkp DQpzdGF0NjQoIi9ldGMvdGVybWluZm8iLCB7c3RfbW9kZT1TX0lGRElSfDA3NTUsIHN0X3NpemU9 MTQsIC4uLn0pID0gMA0KYWNjZXNzKCIvZXRjL3Rlcm1pbmZvL3gveHRlcm0iLCBSX09LKSAgID0g MA0Kb3BlbigiL2V0Yy90ZXJtaW5mby94L3h0ZXJtIiwgT19SRE9OTFl8T19MQVJHRUZJTEUpID0g Nw0KcmVhZCg3LCAiXDMyXDAwMTBcMCZcMFwxN1wwXDIzNVwxbFw1eHRlcm18eHRlcm0gdGVybWlu YWwiLi4uLCA0MDk3KSA9IDMyNTgNCmNsb3NlKDcpICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICA9IDANCmlvY3RsKDEsIFNORENUTF9UTVJfVElNRUJBU0Ugb3IgU05EUlZfVElNRVJfSU9D VExfTkVYVF9ERVZJQ0Ugb3IgVENHRVRTLCANCntCMzg0MDAgb3Bvc3QgaXNpZyBpY2Fub24gZWNo byAuLi59KSA9IDANCmlvY3RsKDEsIFNORENUTF9UTVJfVElNRUJBU0Ugb3IgU05EUlZfVElNRVJf SU9DVExfTkVYVF9ERVZJQ0Ugb3IgVENHRVRTLCANCntCMzg0MDAgb3Bvc3QgaXNpZyBpY2Fub24g ZWNobyAuLi59KSA9IDANCmlvY3RsKDEsIFNORENUTF9UTVJfVElNRUJBU0Ugb3IgU05EUlZfVElN RVJfSU9DVExfTkVYVF9ERVZJQ0Ugb3IgVENHRVRTLCANCntCMzg0MDAgb3Bvc3QgaXNpZyBpY2Fu b24gZWNobyAuLi59KSA9IDANCmlvY3RsKDEsIFRJT0NHV0lOU1osIHt3c19yb3c9NDAsIHdzX2Nv bD0yMDcsIHdzX3hwaXhlbD0wLCB3c195cGl4ZWw9MH0pID0gMA0KYWNjZXNzKCIvdXNyL2xvY2Fs L3NiaW4vc3R0eSIsIFhfT0spICAgID0gLTEgRU5PRU5UIChObyBzdWNoIGZpbGUgb3IgZGlyZWN0 b3J5KQ0KYWNjZXNzKCIvdXNyL2xvY2FsL2Jpbi9zdHR5IiwgWF9PSykgICAgID0gLTEgRU5PRU5U IChObyBzdWNoIGZpbGUgb3IgZGlyZWN0b3J5KQ0KYWNjZXNzKCIvdXNyL3NiaW4vc3R0eSIsIFhf T0spICAgICAgICAgID0gLTEgRU5PRU5UIChObyBzdWNoIGZpbGUgb3IgZGlyZWN0b3J5KQ0KYWNj ZXNzKCIvdXNyL2Jpbi9zdHR5IiwgWF9PSykgICAgICAgICAgID0gLTEgRU5PRU5UIChObyBzdWNo IGZpbGUgb3IgZGlyZWN0b3J5KQ0KYWNjZXNzKCIvc2Jpbi9zdHR5IiwgWF9PSykgICAgICAgICAg ICAgID0gLTEgRU5PRU5UIChObyBzdWNoIGZpbGUgb3IgZGlyZWN0b3J5KQ0KYWNjZXNzKCIvYmlu L3N0dHkiLCBYX09LKSAgICAgICAgICAgICAgID0gMA0Kc3RhdDY0KCIvYmluL3N0dHkiLCB7c3Rf bW9kZT1TX0lGUkVHfDA3NTUsIHN0X3NpemU9NTg4MzYsIC4uLn0pID0gMA0KY2xvbmUoY2hpbGRf c3RhY2s9MCwgZmxhZ3M9Q0xPTkVfQ0hJTERfQ0xFQVJUSUR8Q0xPTkVfQ0hJTERfU0VUVElEfFNJ R0NITEQsIA0KY2hpbGRfdGlkcHRyPTB4Yjc1M2Q3MjgpID0gMjc1Mg0Kd2FpdHBpZCgyNzUyLCBb e1dJRkVYSVRFRChzKSAmJiBXRVhJVFNUQVRVUyhzKSA9PSAwfV0sIDApID0gMjc1Mg0KLS0tIFNJ R0NITEQgKENoaWxkIGV4aXRlZCkgQCAwICgwKSAtLS0NCmZjbnRsNjQoNSwgRl9HRVRGTCkgICAg ICAgICAgICAgICAgICAgICA9IDB4MiAoZmxhZ3MgT19SRFdSKQ0KZmNudGw2NCg1LCBGX1NFVEZM LCBPX1JEV1J8T19OT05CTE9DSykgID0gMA0KZnN0YXQ2NCg1LCB7c3RfbW9kZT1TX0lGQ0hSfDA2 NjYsIHN0X3JkZXY9bWFrZWRldig1LCAyKSwgLi4ufSkgPSAwDQppb2N0bCg1LCBTTkRDVExfVE1S X1RJTUVCQVNFIG9yIFNORFJWX1RJTUVSX0lPQ1RMX05FWFRfREVWSUNFIG9yIFRDR0VUUywgDQp7 QjM4NDAwIC1vcG9zdCBpc2lnIGljYW5vbiBlY2hvIC4uLn0pID0gMA0Kb3BlbigiL2hvbWUvdG1w L3BvcnRhZ2UvZGV2LWxpYnMvbGliZ2NyeXB0LTEuNC42L3RlbXAvYnVpbGQubG9nIiwgT19XUk9O TFl8DQpPX0NSRUFUfE9fQVBQRU5EfE9fTEFSR0VGSUxFLCAwNjY2KSA9IDcNCmZzdGF0NjQoNywg e3N0X21vZGU9U19JRlJFR3wwNjYwLCBzdF9zaXplPTQ4MCwgLi4ufSkgPSAwDQpfbGxzZWVrKDcs IDAsIFs0ODBdLCBTRUVLX0VORCkgICAgICAgICAgPSAwDQppb2N0bCg3LCBTTkRDVExfVE1SX1RJ TUVCQVNFIG9yIFNORFJWX1RJTUVSX0lPQ1RMX05FWFRfREVWSUNFIG9yIFRDR0VUUywgDQoweGJm ZjVmYWQ4KSA9IC0xIEVOT1RUWSAoSW5hcHByb3ByaWF0ZSBpb2N0bCBmb3IgZGV2aWNlKQ0KZnN0 YXQ2NCg3LCB7c3RfbW9kZT1TX0lGUkVHfDA2NjAsIHN0X3NpemU9NDgwLCAuLi59KSA9IDANCl9s bHNlZWsoNywgMCwgWzQ4MF0sIFNFRUtfQ1VSKSAgICAgICAgICA9IDANCnN0YXQ2NCgiL2hvbWUv dG1wL3BvcnRhZ2UvZGV2LWxpYnMvbGliZ2NyeXB0LTEuNC42L3RlbXAvYnVpbGQubG9nIiwgDQp7 c3RfbW9kZT1TX0lGUkVHfDA2NjAsIHN0X3NpemU9NDgwLCAuLi59KSA9IDANCmR1cCgxKSAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICA9IDgNCmZzdGF0NjQoOCwge3N0X21vZGU9U19J RkNIUnwwNjIwLCBzdF9yZGV2PW1ha2VkZXYoMTM2LCAwKSwgLi4ufSkgPSAwDQppb2N0bCg4LCBT TkRDVExfVE1SX1RJTUVCQVNFIG9yIFNORFJWX1RJTUVSX0lPQ1RMX05FWFRfREVWSUNFIG9yIFRD R0VUUywgDQp7QjM4NDAwIG9wb3N0IGlzaWcgaWNhbm9uIGVjaG8gLi4ufSkgPSAwDQpmc3RhdDY0 KDgsIHtzdF9tb2RlPVNfSUZDSFJ8MDYyMCwgc3RfcmRldj1tYWtlZGV2KDEzNiwgMCksIC4uLn0p ID0gMA0KX2xsc2Vlayg4LCAwLCAweGJmZjVmODIwLCBTRUVLX0NVUikgICAgID0gLTEgRVNQSVBF IChJbGxlZ2FsIHNlZWspDQpzdGF0NjQoIi9ob21lL3RtcC9wb3J0YWdlL2Rldi1saWJzL2xpYmdj cnlwdC0xLjQuNi90ZW1wL2Vudmlyb25tZW50IiwgDQp7c3RfbW9kZT1TX0lGUkVHfDA2NjQsIHN0 X3NpemU9MTA2NTk3LCAuLi59KSA9IDANCmNsb25lKGNoaWxkX3N0YWNrPTAsIGZsYWdzPUNMT05F X0NISUxEX0NMRUFSVElEfENMT05FX0NISUxEX1NFVFRJRHxTSUdDSExELCANCmNoaWxkX3RpZHB0 cj0weGI3NTNkNzI4KSA9IDI3NTMNCmNsb3NlKDYpICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICA9IDANCmdldHRpbWVvZmRheSh7MTI5NjU3NzQ1NywgMTExMTd9LCBOVUxMKSA9IDANCnBv bGwoW3tmZD0zLCBldmVudHM9UE9MTElOfFBPTExFUlJ8UE9MTEhVUHxQT0xMTlZBTH0sIHtmZD01 LCBldmVudHM9UE9MTElOfA0KUE9MTEVSUnxQT0xMSFVQfFBPTExOVkFMfV0sIDIsIDMwMDApID0g MSAoW3tmZD01LCByZXZlbnRzPVBPTExJTn1dKQ0KcmVhZCg1LCAiPj4+IFByZXBhcmluZyBzb3Vy Y2UgaW4gL2hvbWUvdG0iLi4uLCA0MDk2KSA9IDkyDQp3cml0ZSg4LCAiPj4+IFByZXBhcmluZyBz b3VyY2UgaW4gL2hvbWUvdG0iLi4uLCA5MikgPSA5Mg0Kd3JpdGUoNywgIj4+PiBQcmVwYXJpbmcg c291cmNlIGluIC9ob21lL3RtIi4uLiwgOTIpID0gOTINCnBvbGwoW3tmZD0zLCBldmVudHM9UE9M TElOfFBPTExFUlJ8UE9MTEhVUHxQT0xMTlZBTH0sIHtmZD01LCBldmVudHM9UE9MTElOfA0KUE9M TEVSUnxQT0xMSFVQfFBPTExOVkFMfV0sIDIsIDMwMDApID0gMCAoVGltZW91dCkNCnBvbGwoW3tm ZD0zLCBldmVudHM9UE9MTElOfFBPTExFUlJ8UE9MTEhVUHxQT0xMTlZBTH0sIHtmZD01LCBldmVu dHM9UE9MTElOfA0KUE9MTEVSUnxQT0xMSFVQfFBPTExOVkFMfV0sIDIsIDMwMDApID0gMCAoVGlt ZW91dCkNCnBvbGwoW3tmZD0zLCBldmVudHM9UE9MTElOfFBPTExFUlJ8UE9MTEhVUHxQT0xMTlZB TH0sIHtmZD01LCBldmVudHM9UE9MTElOfA0KUE9MTEVSUnxQT0xMSFVQfFBPTExOVkFMfV0sIDIs IDMwMDApID0gMCAoVGltZW91dCkNCnBvbGwoW3tmZD0zLCBldmVudHM9UE9MTElOfFBPTExFUlJ8 UE9MTEhVUHxQT0xMTlZBTH0sIHtmZD01LCBldmVudHM9UE9MTElOfA0KUE9MTEVSUnxQT0xMSFVQ fFBPTExOVkFMfV0sIDIsIDMwMDApID0gMCAoVGltZW91dCkNCnBvbGwoW3tmZD0zLCBldmVudHM9 UE9MTElOfFBPTExFUlJ8UE9MTEhVUHxQT0xMTlZBTH0sIHtmZD01LCBldmVudHM9UE9MTElOfA0K UE9MTEVSUnxQT0xMSFVQfFBPTExOVkFMfV0sIDIsIDMwMDApID0gMCAoVGltZW91dCkNCnBvbGwo W3tmZD0zLCBldmVudHM9UE9MTElOfFBPTExFUlJ8UE9MTEhVUHxQT0xMTlZBTH0sIHtmZD01LCBl dmVudHM9UE9MTElOfA0KUE9MTEVSUnxQT0xMSFVQfFBPTExOVkFMfV0sIDIsIDMwMDApID0gMCAo VGltZW91dCkNCnBvbGwoW3tmZD0zLCBldmVudHM9UE9MTElOfFBPTExFUlJ8UE9MTEhVUHxQT0xM TlZBTH0sIHtmZD01LCBldmVudHM9UE9MTElOfA0KUE9MTEVSUnxQT0xMSFVQfFBPTExOVkFMfV0s IDIsIDMwMDApID0gMCAoVGltZW91dCkNCnBvbGwoW3tmZD0zLCBldmVudHM9UE9MTElOfFBPTExF UlJ8UE9MTEhVUHxQT0xMTlZBTH0sIHtmZD01LCBldmVudHM9UE9MTElOfA0KUE9MTEVSUnxQT0xM SFVQfFBPTExOVkFMfV0sIDIsIDMwMDApID0gMCAoVGltZW91dCkNCnBvbGwoW3tmZD0zLCBldmVu dHM9UE9MTElOfFBPTExFUlJ8UE9MTEhVUHxQT0xMTlZBTH0sIHtmZD01LCBldmVudHM9UE9MTElO fA0KUE9MTEVSUnxQT0xMSFVQfFBPTExOVkFMfV0sIDIsIDMwMDApID0gMCAoVGltZW91dCkNCnBv bGwoW3tmZD0zLCBldmVudHM9UE9MTElOfFBPTExFUlJ8UE9MTEhVUHxQT0xMTlZBTH0sIHtmZD01 LCBldmVudHM9UE9MTElOfA0KUE9MTEVSUnxQT0xMSFVQfFBPTExOVkFMfV0sIDIsIDMwMDApID0g MCAoVGltZW91dCkNCnBvbGwoW3tmZD0zLCBldmVudHM9UE9MTElOfFBPTExFUlJ8UE9MTEhVUHxQ T0xMTlZBTH0sIHtmZD01LCBldmVudHM9UE9MTElOfA0KUE9MTEVSUnxQT0xMSFVQfFBPTExOVkFM fV0sIDIsIDMwMDApID0gMCAoVGltZW91dCkNCnBvbGwoW3tmZD0zLCBldmVudHM9UE9MTElOfFBP TExFUlJ8UE9MTEhVUHxQT0xMTlZBTH0sIHtmZD01LCBldmVudHM9UE9MTElOfA0KUE9MTEVSUnxQ T0xMSFVQfFBPTExOVkFMfV0sIDIsIDMwMDApID0gMCAoVGltZW91dCkNCnBvbGwoW3tmZD0zLCBl dmVudHM9UE9MTElOfFBPTExFUlJ8UE9MTEhVUHxQT0xMTlZBTH0sIHtmZD01LCBldmVudHM9UE9M TElOfA0KUE9MTEVSUnxQT0xMSFVQfFBPTExOVkFMfV0sIDIsIDMwMDApID0gMCAoVGltZW91dCkN CnBvbGwoW3tmZD0zLCBldmVudHM9UE9MTElOfFBPTExFUlJ8UE9MTEhVUHxQT0xMTlZBTH0sIHtm ZD01LCBldmVudHM9UE9MTElOfA0KUE9MTEVSUnxQT0xMSFVQfFBPTExOVkFMfV0sIDIsIDMwMDAp ID0gMCAoVGltZW91dCkNCnBvbGwoW3tmZD0zLCBldmVudHM9UE9MTElOfFBPTExFUlJ8UE9MTEhV UHxQT0xMTlZBTH0sIHtmZD01LCBldmVudHM9UE9MTElOfA0KUE9MTEVSUnxQT0xMSFVQfFBPTExO VkFMfV0sIDIsIDMwMDApID0gMCAoVGltZW91dCkNCnBvbGwoW3tmZD0zLCBldmVudHM9UE9MTElO fFBPTExFUlJ8UE9MTEhVUHxQT0xMTlZBTH0sIHtmZD01LCBldmVudHM9UE9MTElOfA0KUE9MTEVS UnxQT0xMSFVQfFBPTExOVkFMfV0sIDIsIDMwMDAgPHVuZmluaXNoZWQgLi4uPg0KDQo= ^ permalink raw reply [flat|nested] 43+ messages in thread
* RE: [PATCH v2]Btrfs: pwrite blocked when writing from the mmaped buffer of the same page 2011-02-11 4:39 ` Zhong, Xin @ 2011-02-18 11:31 ` Maria Wikström 2011-02-21 1:51 ` Zhong, Xin 2011-02-22 22:27 ` Johannes Hirte 1 sibling, 1 reply; 43+ messages in thread From: Maria Wikström @ 2011-02-18 11:31 UTC (permalink / raw) To: Zhong, Xin; +Cc: Johannes Hirte, linux-btrfs [-- Attachment #1: Type: text/plain, Size: 1348 bytes --] Seems like my reply got eaten by the lists spam filter, so I resend with attachment compressed. Should have thought of that :p fre 2011-02-11 klockan 12:39 +0800 skrev Zhong, Xin: > Hi, > > Could you paste the output of sysrq+t here? Thanks! Yes, it's in the attachment. I tried latest btrfs from git (last commit Mon Feb 14 00:45:29 2011 +0000) but it hang so bad that I couldn't get the output from sysrq+t to hit the disk. So the output is from vanilla 2.6.37 // Maria > -----Original Message----- > From: Johannes Hirte [mailto:johannes.hirte@fem.tu-ilmenau.de] > Sent: Wednesday, February 02, 2011 7:35 AM > To: Zhong, Xin > Cc: Maria Wikström; linux-btrfs@vger.kernel.org > Subject: Re: [PATCH v2]Btrfs: pwrite blocked when writing from the mmaped buffer of the same page > > On Friday 28 January 2011 04:53:24 Zhong, Xin wrote: > > Could you describe the steps to recreate it? > > It will be a great help for me to look further. Thanks! > > It's a little strange. I have to systems with btrfs, both Gentoo-based. One is > affected by this bug the other is not. On the affected system it is enough to do > a 'emerge dev-libs/libgcrypt' that should normaly compile and install > libgcrypt. The emerge command is part of portage, the package management of > Gentoo. > The strace output looks similar to the one from Maria: > [-- Attachment #2: vanilla_2.6.37_sysrq-t.txt.tar.gz --] [-- Type: application/x-compressed-tar, Size: 22248 bytes --] ^ permalink raw reply [flat|nested] 43+ messages in thread
* RE: [PATCH v2]Btrfs: pwrite blocked when writing from the mmaped buffer of the same page 2011-02-18 11:31 ` Maria Wikström @ 2011-02-21 1:51 ` Zhong, Xin 2011-02-24 14:51 ` Maria Wikström 2011-02-24 23:35 ` [PATCH v2]Btrfs: pwrite blocked when writing from the mmaped buffer of the same page Piotr Szymaniak 0 siblings, 2 replies; 43+ messages in thread From: Zhong, Xin @ 2011-02-21 1:51 UTC (permalink / raw) To: Maria Wikström; +Cc: Johannes Hirte, linux-btrfs VGhlIGJhY2t0cmFjZSBpbiB5b3VyIGF0dGFjaG1lbnQgbG9va3MgbGlrZSBhIGtub3duIGJ1ZyBv ZiAyLjYuMzcgd2hpY2ggaGF2ZSBhbHJlYWR5IGJlZW4gZml4ZWQgaW4gMi42LjM4LiBJIGhhdmUg bm8gaWRlYSB3aHkgbGF0ZXN0IGJ0cmZzIHN0aWxsIGhhbmcgaW4geW91ciBlbnZpcm9ubWVudCBp ZiB0aGVyZSdzIG5vIGRlYnVnIGluZm8uLi4NCg0KLS0tLS1PcmlnaW5hbCBNZXNzYWdlLS0tLS0N CkZyb206IE1hcmlhIFdpa3N0csO2bSBbbWFpbHRvOm1hcmlhQHBvbnN0dWRpb3Muc2VdIA0KU2Vu dDogRnJpZGF5LCBGZWJydWFyeSAxOCwgMjAxMSA3OjMyIFBNDQpUbzogWmhvbmcsIFhpbg0KQ2M6 IEpvaGFubmVzIEhpcnRlOyBsaW51eC1idHJmc0B2Z2VyLmtlcm5lbC5vcmcNClN1YmplY3Q6IFJF OiBbUEFUQ0ggdjJdQnRyZnM6IHB3cml0ZSBibG9ja2VkIHdoZW4gd3JpdGluZyBmcm9tIHRoZSBt bWFwZWQgYnVmZmVyIG9mIHRoZSBzYW1lIHBhZ2UNCg0KDQpTZWVtcyBsaWtlIG15IHJlcGx5IGdv dCBlYXRlbiBieSB0aGUgbGlzdHMgc3BhbSBmaWx0ZXIsIHNvIEkgcmVzZW5kIHdpdGggYXR0YWNo bWVudCBjb21wcmVzc2VkLiBTaG91bGQgaGF2ZSB0aG91Z2h0IG9mIHRoYXQgOnANCg0KDQpmcmUg MjAxMS0wMi0xMSBrbG9ja2FuIDEyOjM5ICswODAwIHNrcmV2IFpob25nLCBYaW46DQo+IEhpLA0K PiANCj4gQ291bGQgeW91IHBhc3RlIHRoZSBvdXRwdXQgb2Ygc3lzcnErdCBoZXJlPyBUaGFua3Mh DQoNClllcywgaXQncyBpbiB0aGUgYXR0YWNobWVudC4gSSB0cmllZCBsYXRlc3QgYnRyZnMgZnJv bSBnaXQgKGxhc3QgY29tbWl0IE1vbiBGZWIgMTQgMDA6NDU6MjkgMjAxMSArMDAwMCkgYnV0IGl0 IGhhbmcgc28gYmFkIHRoYXQgSSBjb3VsZG4ndCBnZXQgdGhlIG91dHB1dCBmcm9tIHN5c3JxK3Qg dG8gaGl0IHRoZSBkaXNrLiBTbyB0aGUgb3V0cHV0IGlzIGZyb20gdmFuaWxsYQ0KMi42LjM3IA0K DQovLyBNYXJpYQ0KDQo+IC0tLS0tT3JpZ2luYWwgTWVzc2FnZS0tLS0tDQo+IEZyb206IEpvaGFu bmVzIEhpcnRlIFttYWlsdG86am9oYW5uZXMuaGlydGVAZmVtLnR1LWlsbWVuYXUuZGVdDQo+IFNl bnQ6IFdlZG5lc2RheSwgRmVicnVhcnkgMDIsIDIwMTEgNzozNSBBTQ0KPiBUbzogWmhvbmcsIFhp bg0KPiBDYzogTWFyaWEgV2lrc3Ryw7ZtOyBsaW51eC1idHJmc0B2Z2VyLmtlcm5lbC5vcmcNCj4g U3ViamVjdDogUmU6IFtQQVRDSCB2Ml1CdHJmczogcHdyaXRlIGJsb2NrZWQgd2hlbiB3cml0aW5n IGZyb20gdGhlIA0KPiBtbWFwZWQgYnVmZmVyIG9mIHRoZSBzYW1lIHBhZ2UNCj4gDQo+IE9uIEZy aWRheSAyOCBKYW51YXJ5IDIwMTEgMDQ6NTM6MjQgWmhvbmcsIFhpbiB3cm90ZToNCj4gPiBDb3Vs ZCB5b3UgZGVzY3JpYmUgdGhlIHN0ZXBzIHRvIHJlY3JlYXRlIGl0Pw0KPiA+IEl0IHdpbGwgYmUg YSBncmVhdCBoZWxwIGZvciBtZSB0byBsb29rIGZ1cnRoZXIuIFRoYW5rcyENCj4gDQo+IEl0J3Mg YSBsaXR0bGUgc3RyYW5nZS4gSSBoYXZlIHRvIHN5c3RlbXMgd2l0aCBidHJmcywgYm90aCANCj4g R2VudG9vLWJhc2VkLiBPbmUgaXMgYWZmZWN0ZWQgYnkgdGhpcyBidWcgdGhlIG90aGVyIGlzIG5v dC4gT24gdGhlIA0KPiBhZmZlY3RlZCBzeXN0ZW0gaXQgaXMgZW5vdWdoIHRvIGRvIGEgJ2VtZXJn ZSBkZXYtbGlicy9saWJnY3J5cHQnIHRoYXQgDQo+IHNob3VsZCBub3JtYWx5IGNvbXBpbGUgYW5k IGluc3RhbGwgbGliZ2NyeXB0LiBUaGUgZW1lcmdlIGNvbW1hbmQgaXMgDQo+IHBhcnQgb2YgcG9y dGFnZSwgdGhlIHBhY2thZ2UgbWFuYWdlbWVudCBvZiBHZW50b28uDQo+IFRoZSBzdHJhY2Ugb3V0 cHV0IGxvb2tzIHNpbWlsYXIgdG8gdGhlIG9uZSBmcm9tIE1hcmlhOg0KPiANCg0K ^ permalink raw reply [flat|nested] 43+ messages in thread
* RE: [PATCH v2]Btrfs: pwrite blocked when writing from the mmaped buffer of the same page 2011-02-21 1:51 ` Zhong, Xin @ 2011-02-24 14:51 ` Maria Wikström 2011-02-24 15:55 ` Mitch Harder 2011-02-24 23:35 ` [PATCH v2]Btrfs: pwrite blocked when writing from the mmaped buffer of the same page Piotr Szymaniak 1 sibling, 1 reply; 43+ messages in thread From: Maria Wikström @ 2011-02-24 14:51 UTC (permalink / raw) To: Zhong, Xin; +Cc: Johannes Hirte, linux-btrfs [-- Attachment #1: Type: text/plain, Size: 2414 bytes --] mån 2011-02-21 klockan 09:51 +0800 skrev Zhong, Xin: > The backtrace in your attachment looks like a known bug of 2.6.37 which have already been fixed in 2.6.38. I have no idea why latest btrfs still hang in your environment if there's no debug info... > Haha, yes that's very hard :) 2.6.38-rc6 and btrfs-unstable behaves the same way. I can close the process with ctrl+c and it disappear a few seconds later. There is no CPU usage. Reading works because I can start htop and watch "svn info" disappear, but everything writing to btrfs slows down to a crawl. It takes about 1 minute to log in. So I had to put the logs on an other partition using ext3 to get the output from sysrq+t. // Maria > -----Original Message----- > From: Maria Wikström [mailto:maria@ponstudios.se] > Sent: Friday, February 18, 2011 7:32 PM > To: Zhong, Xin > Cc: Johannes Hirte; linux-btrfs@vger.kernel.org > Subject: RE: [PATCH v2]Btrfs: pwrite blocked when writing from the mmaped buffer of the same page > > > Seems like my reply got eaten by the lists spam filter, so I resend with attachment compressed. Should have thought of that :p > > > fre 2011-02-11 klockan 12:39 +0800 skrev Zhong, Xin: > > Hi, > > > > Could you paste the output of sysrq+t here? Thanks! > > Yes, it's in the attachment. I tried latest btrfs from git (last commit Mon Feb 14 00:45:29 2011 +0000) but it hang so bad that I couldn't get the output from sysrq+t to hit the disk. So the output is from vanilla > 2.6.37 > > // Maria > > > -----Original Message----- > > From: Johannes Hirte [mailto:johannes.hirte@fem.tu-ilmenau.de] > > Sent: Wednesday, February 02, 2011 7:35 AM > > To: Zhong, Xin > > Cc: Maria Wikström; linux-btrfs@vger.kernel.org > > Subject: Re: [PATCH v2]Btrfs: pwrite blocked when writing from the > > mmaped buffer of the same page > > > > On Friday 28 January 2011 04:53:24 Zhong, Xin wrote: > > > Could you describe the steps to recreate it? > > > It will be a great help for me to look further. Thanks! > > > > It's a little strange. I have to systems with btrfs, both > > Gentoo-based. One is affected by this bug the other is not. On the > > affected system it is enough to do a 'emerge dev-libs/libgcrypt' that > > should normaly compile and install libgcrypt. The emerge command is > > part of portage, the package management of Gentoo. > > The strace output looks similar to the one from Maria: > > > [-- Attachment #2: 2.6.38-rc6_sysrq-t.txt.tar.bz2 --] [-- Type: application/x-bzip-compressed-tar, Size: 17699 bytes --] [-- Attachment #3: btrfs-unstable_sysrq-t.txt.tar.bz2 --] [-- Type: application/x-bzip-compressed-tar, Size: 15788 bytes --] ^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [PATCH v2]Btrfs: pwrite blocked when writing from the mmaped buffer of the same page 2011-02-24 14:51 ` Maria Wikström @ 2011-02-24 15:55 ` Mitch Harder 2011-02-24 16:00 ` Chris Mason 0 siblings, 1 reply; 43+ messages in thread From: Mitch Harder @ 2011-02-24 15:55 UTC (permalink / raw) To: Maria Wikström; +Cc: Zhong, Xin, Johannes Hirte, linux-btrfs 2011/2/24 Maria Wikstr=F6m <maria@ponstudios.se>: > m=E5n 2011-02-21 klockan 09:51 +0800 skrev Zhong, Xin: >> The backtrace in your attachment looks like a known bug of 2.6.37 wh= ich have already been fixed in 2.6.38. I have no idea why latest btrfs = still hang in your environment if there's no debug info... >> > > Haha, yes that's very hard :) > > 2.6.38-rc6 and btrfs-unstable behaves the same way. I can close the > process with ctrl+c and it disappear a few seconds later. There is no > CPU usage. Reading works because I can start htop and watch "svn info= " > disappear, but everything writing to btrfs slows down to a crawl. It > takes about 1 minute to log in. So I had to put the logs on an other > partition using ext3 to get the output from sysrq+t. > I believe I've been experiencing this issue also. However, my problem usually results in a "No space left on device" error rather than a lock-up or crash. But I've bisected my issue to this patch, and my "btrfs fi show" and "btrfs fi df" looks similar to others who've posted to this tread with all my space being allocated, but not used. I've been playing around with ftrace to try to get some information on the issue. Since I'm getting a soft error, I can used a command like "echo 1 > tracing_on; emerge -1av openmotif; echo 0 > tracing_on" to end the trace as soon as the build fails. The traces are probably too large for the M/L (~275kb compressed), so I've put them up on my local server in case anybody can find them useful: http://dontpanic.dyndns.org/emerge-openmotif-ftrace.gz I'm still only capturing the tail end of the problem, but maybe someone will find these insightful. Let me know if you want me to increase the ftrace buffer size or insert some trace_printk debugging statements. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" = in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [PATCH v2]Btrfs: pwrite blocked when writing from the mmaped buffer of the same page 2011-02-24 15:55 ` Mitch Harder @ 2011-02-24 16:00 ` Chris Mason 2011-02-24 16:03 ` Mitch Harder 0 siblings, 1 reply; 43+ messages in thread From: Chris Mason @ 2011-02-24 16:00 UTC (permalink / raw) To: Mitch Harder; +Cc: Maria Wikström, Zhong, Xin, Johannes Hirte, linux-btrfs Excerpts from Mitch Harder's message of 2011-02-24 10:55:15 -0500: > 2011/2/24 Maria Wikstr=C3=B6m <maria@ponstudios.se>: > > m=C3=A5n 2011-02-21 klockan 09:51 +0800 skrev Zhong, Xin: > >> The backtrace in your attachment looks like a known bug of 2.6.37 = which have already been fixed in 2.6.38. I have no idea why latest btrf= s still hang in your environment if there's no debug info... > >> > > > > Haha, yes that's very hard :) > > > > 2.6.38-rc6 and btrfs-unstable behaves the same way. I can close the > > process with ctrl+c and it disappear a few seconds later. There is = no > > CPU usage. Reading works because I can start htop and watch "svn in= fo" > > disappear, but everything writing to btrfs slows down to a crawl. I= t > > takes about 1 minute to log in. So I had to put the logs on an othe= r > > partition using ext3 to get the output from sysrq+t. > > >=20 > I believe I've been experiencing this issue also. However, my proble= m > usually results in a "No space left on device" error rather than a > lock-up or crash. But I've bisected my issue to this patch, and my > "btrfs fi show" and "btrfs fi df" looks similar to others who've > posted to this tread with all my space being allocated, but not used. >=20 Sorry, which patch did you bisect the problem down to? > I've been playing around with ftrace to try to get some information o= n > the issue. Since I'm getting a soft error, I can used a command like > "echo 1 > tracing_on; emerge -1av openmotif; echo 0 > tracing_on" to > end the trace as soon as the build fails. >=20 > The traces are probably too large for the M/L (~275kb compressed), so > I've put them up on my local server in case anybody can find them > useful: >=20 > http://dontpanic.dyndns.org/emerge-openmotif-ftrace.gz >=20 > I'm still only capturing the tail end of the problem, but maybe > someone will find these insightful. >=20 > Let me know if you want me to increase the ftrace buffer size or > insert some trace_printk debugging statements. Thanks, I'll take a look. -chris -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" = in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [PATCH v2]Btrfs: pwrite blocked when writing from the mmaped buffer of the same page 2011-02-24 16:00 ` Chris Mason @ 2011-02-24 16:03 ` Mitch Harder 2011-02-24 16:19 ` Chris Mason 0 siblings, 1 reply; 43+ messages in thread From: Mitch Harder @ 2011-02-24 16:03 UTC (permalink / raw) To: Chris Mason; +Cc: Maria Wikström, Zhong, Xin, Johannes Hirte, linux-btrfs On Thu, Feb 24, 2011 at 10:00 AM, Chris Mason <chris.mason@oracle.com> = wrote: > Excerpts from Mitch Harder's message of 2011-02-24 10:55:15 -0500: >> 2011/2/24 Maria Wikstr=F6m <maria@ponstudios.se>: >> > m=E5n 2011-02-21 klockan 09:51 +0800 skrev Zhong, Xin: >> >> The backtrace in your attachment looks like a known bug of 2.6.37= which have already been fixed in 2.6.38. I have no idea why latest btr= fs still hang in your environment if there's no debug info... >> >> >> > >> > Haha, yes that's very hard :) >> > >> > 2.6.38-rc6 and btrfs-unstable behaves the same way. I can close th= e >> > process with ctrl+c and it disappear a few seconds later. There is= no >> > CPU usage. Reading works because I can start htop and watch "svn i= nfo" >> > disappear, but everything writing to btrfs slows down to a crawl. = It >> > takes about 1 minute to log in. So I had to put the logs on an oth= er >> > partition using ext3 to get the output from sysrq+t. >> > >> >> I believe I've been experiencing this issue also. =A0However, my pro= blem >> usually results in a "No space left on device" error rather than a >> lock-up or crash. =A0But I've bisected my issue to this patch, and m= y >> "btrfs fi show" and "btrfs fi df" looks similar to others who've >> posted to this tread with all my space being allocated, but not used= =2E >> > > Sorry, which patch did you bisect the problem down to? > The patch at the head of this thread: Btrfs: pwrite blocked when writing from the mmaped buffer of the same p= age -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" = in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [PATCH v2]Btrfs: pwrite blocked when writing from the mmaped buffer of the same page 2011-02-24 16:03 ` Mitch Harder @ 2011-02-24 16:19 ` Chris Mason 2011-02-24 16:32 ` Mitch Harder 0 siblings, 1 reply; 43+ messages in thread From: Chris Mason @ 2011-02-24 16:19 UTC (permalink / raw) To: Mitch Harder; +Cc: Maria Wikström, Zhong, Xin, Johannes Hirte, linux-btrfs Excerpts from Mitch Harder's message of 2011-02-24 11:03:07 -0500: > On Thu, Feb 24, 2011 at 10:00 AM, Chris Mason <chris.mason@oracle.com= > wrote: > > Excerpts from Mitch Harder's message of 2011-02-24 10:55:15 -0500: > >> 2011/2/24 Maria Wikstr=C3=B6m <maria@ponstudios.se>: > >> > m=C3=A5n 2011-02-21 klockan 09:51 +0800 skrev Zhong, Xin: > >> >> The backtrace in your attachment looks like a known bug of 2.6.= 37 which have already been fixed in 2.6.38. I have no idea why latest b= trfs still hang in your environment if there's no debug info... > >> >> > >> > > >> > Haha, yes that's very hard :) > >> > > >> > 2.6.38-rc6 and btrfs-unstable behaves the same way. I can close = the > >> > process with ctrl+c and it disappear a few seconds later. There = is no > >> > CPU usage. Reading works because I can start htop and watch "svn= info" > >> > disappear, but everything writing to btrfs slows down to a crawl= =2E It > >> > takes about 1 minute to log in. So I had to put the logs on an o= ther > >> > partition using ext3 to get the output from sysrq+t. > >> > > >> > >> I believe I've been experiencing this issue also. =C2=A0However, m= y problem > >> usually results in a "No space left on device" error rather than a > >> lock-up or crash. =C2=A0But I've bisected my issue to this patch, = and my > >> "btrfs fi show" and "btrfs fi df" looks similar to others who've > >> posted to this tread with all my space being allocated, but not us= ed. > >> > > > > Sorry, which patch did you bisect the problem down to? > > >=20 > The patch at the head of this thread: >=20 > Btrfs: pwrite blocked when writing from the mmaped buffer of the same= page Hmmm, that patch shouldn't be changing our performance under delalloc pressure, and it really shouldn't impact early enospc. -chris -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" = in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [PATCH v2]Btrfs: pwrite blocked when writing from the mmaped buffer of the same page 2011-02-24 16:19 ` Chris Mason @ 2011-02-24 16:32 ` Mitch Harder [not found] ` <AANLkTinvyb-bTVVignd1KGojvh-QrYCFmCnwYKBsYC_2@mail.gmail.com> 0 siblings, 1 reply; 43+ messages in thread From: Mitch Harder @ 2011-02-24 16:32 UTC (permalink / raw) To: Chris Mason; +Cc: Maria Wikström, Zhong, Xin, Johannes Hirte, linux-btrfs On Thu, Feb 24, 2011 at 10:19 AM, Chris Mason <chris.mason@oracle.com> = wrote: > Excerpts from Mitch Harder's message of 2011-02-24 11:03:07 -0500: >> On Thu, Feb 24, 2011 at 10:00 AM, Chris Mason <chris.mason@oracle.co= m> wrote: >> > Excerpts from Mitch Harder's message of 2011-02-24 10:55:15 -0500: >> >> 2011/2/24 Maria Wikstr=F6m <maria@ponstudios.se>: >> >> > m=E5n 2011-02-21 klockan 09:51 +0800 skrev Zhong, Xin: >> >> >> The backtrace in your attachment looks like a known bug of 2.6= =2E37 which have already been fixed in 2.6.38. I have no idea why lates= t btrfs still hang in your environment if there's no debug info... >> >> >> >> >> > >> >> > Haha, yes that's very hard :) >> >> > >> >> > 2.6.38-rc6 and btrfs-unstable behaves the same way. I can close= the >> >> > process with ctrl+c and it disappear a few seconds later. There= is no >> >> > CPU usage. Reading works because I can start htop and watch "sv= n info" >> >> > disappear, but everything writing to btrfs slows down to a craw= l. It >> >> > takes about 1 minute to log in. So I had to put the logs on an = other >> >> > partition using ext3 to get the output from sysrq+t. >> >> > >> >> >> >> I believe I've been experiencing this issue also. =A0However, my = problem >> >> usually results in a "No space left on device" error rather than = a >> >> lock-up or crash. =A0But I've bisected my issue to this patch, an= d my >> >> "btrfs fi show" and "btrfs fi df" looks similar to others who've >> >> posted to this tread with all my space being allocated, but not u= sed. >> >> >> > >> > Sorry, which patch did you bisect the problem down to? >> > >> >> The patch at the head of this thread: >> >> Btrfs: pwrite blocked when writing from the mmaped buffer of the sam= e page > > Hmmm, that patch shouldn't be changing our performance under delalloc > pressure, and it really shouldn't impact early enospc. > I've bisected this issue around where this patch went into git, and I've also constructed a testing patch that reverts this patch, and placed it on top of the current Btrfs git sources (I understand this patch addresses a real issue, this was just for testing). It could be that this patch just "uncovers" another problem, but all my tests seem to point to this patch triggering this issue. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" = in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 43+ messages in thread
[parent not found: <AANLkTinvyb-bTVVignd1KGojvh-QrYCFmCnwYKBsYC_2@mail.gmail.com>]
* Re: [PATCH v2]Btrfs: pwrite blocked when writing from the mmaped buffer of the same page [not found] ` <AANLkTinvyb-bTVVignd1KGojvh-QrYCFmCnwYKBsYC_2@mail.gmail.com> @ 2011-02-25 17:11 ` Mitch Harder 2011-02-25 18:43 ` Mitch Harder 0 siblings, 1 reply; 43+ messages in thread From: Mitch Harder @ 2011-02-25 17:11 UTC (permalink / raw) To: Chris Mason; +Cc: Maria Wikström, Zhong, Xin, Johannes Hirte, linux-btrfs On Thu, Feb 24, 2011 at 5:14 PM, Mitch Harder <mitch.harder@sabayonlinux.org> wrote: > On Thu, Feb 24, 2011 at 10:32 AM, Mitch Harder > <mitch.harder@sabayonlinux.org> wrote: >> On Thu, Feb 24, 2011 at 10:19 AM, Chris Mason <chris.mason@oracle.co= m> wrote: >>> Excerpts from Mitch Harder's message of 2011-02-24 11:03:07 -0500: >>>> On Thu, Feb 24, 2011 at 10:00 AM, Chris Mason <chris.mason@oracle.= com> wrote: >>>> > Excerpts from Mitch Harder's message of 2011-02-24 10:55:15 -050= 0: >>>> >> 2011/2/24 Maria Wikstr=F6m <maria@ponstudios.se>: >>>> >> > m=E5n 2011-02-21 klockan 09:51 +0800 skrev Zhong, Xin: >>>> >> >> The backtrace in your attachment looks like a known bug of 2= =2E6.37 which have already been fixed in 2.6.38. I have no idea why lat= est btrfs still hang in your environment if there's no debug info... >>>> >> >> >>>> >> > >>>> >> > Haha, yes that's very hard :) >>>> >> > >>>> >> > 2.6.38-rc6 and btrfs-unstable behaves the same way. I can clo= se the >>>> >> > process with ctrl+c and it disappear a few seconds later. The= re is no >>>> >> > CPU usage. Reading works because I can start htop and watch "= svn info" >>>> >> > disappear, but everything writing to btrfs slows down to a cr= awl. It >>>> >> > takes about 1 minute to log in. So I had to put the logs on a= n other >>>> >> > partition using ext3 to get the output from sysrq+t. >>>> >> > >>>> >> >>>> >> I believe I've been experiencing this issue also. =A0However, m= y problem >>>> >> usually results in a "No space left on device" error rather tha= n a >>>> >> lock-up or crash. =A0But I've bisected my issue to this patch, = and my >>>> >> "btrfs fi show" and "btrfs fi df" looks similar to others who'v= e >>>> >> posted to this tread with all my space being allocated, but not= used. >>>> >> >>>> > >>>> > Sorry, which patch did you bisect the problem down to? >>>> > >>>> >>>> The patch at the head of this thread: >>>> >>>> Btrfs: pwrite blocked when writing from the mmaped buffer of the s= ame page >>> >>> Hmmm, that patch shouldn't be changing our performance under delall= oc >>> pressure, and it really shouldn't impact early enospc. >>> >> >> I've bisected this issue around where this patch went into git, and >> I've also constructed a testing patch that reverts this patch, and >> placed it on top of the current Btrfs git sources (I understand this >> patch addresses a real issue, this was just for testing). >> >> It could be that this patch just "uncovers" another problem, but all >> my tests seem to point to this patch triggering this issue. >> > > I don't belief the previous ftrace I supplied had a large enough scop= e > to capture the issue. > > I've expanded my ftrace buffer, and filtered out everything but btrfs= * > function calls ("# echo btrfs* > > /sys/kernel/debug/tracing/set_ftrace_filter"). > > In this trace, I see btrfs spending a great deal of time in a while > loop (while (iov_iter_count(&i) > 0) {)) in the btrfs_file_aio_write(= ) > function in file.c without exiting the function. > > I'm going to try to inject some debugging trace_printk() statements t= o > find if that portion of code is proceeding normally with my test case= =2E > > I've put my expanded trace up on my local server, but my upload > bandwidth is pretty sad, and it may take a few minutes to transfer > even though it's only a 6MB file. > > http://dontpanic.dyndns.org/trace-openmotif-btrfs-v3.gz > Apologies for only hitting "Reply" instead of "Reply-All" on my last me= ssage. I've inserted additional trace_printk() to the btrfs_file_aio_write() and btrfs_copy_from_user() function in file.c in order to characterize the problem I've been encountering. I can see btrfs getting stuck in a loop in the "while (iov_iter_count(&i) > 0) {}" portion of the btrfs_file_aio_write() function. The loop is more-or-less following this process (from within the "while (iov_iter_count(&i) > 0) {}" loop): (1) Reserve some space with btrfs_delalloc_reserve_space() (2) Prepare the reserved space with prepare_pages() (3) Call btrfs_copy_from_user() to copy to the prepared space. -------------> From btrfs_copy_from_user() (4) ........Try to copy with copied =3D iov_iter_copy_from_user_atomic(= ) (5) ........The above operation results with copied =3D=3D 0. Break and return with a return value of 0 bytes copied. (6) There is no special handling for copied =3D=3D 0 in the "while (iov_iter_count(&i) > 0) {}" loop, so it loops back around, reserves some more space, and tries again. If I look back at how the code was set up before the patch at the head of this thread was applied (Btrfs: pwrite blocked when writing from the mmaped buffer of the same page), the btrfs_copy_from_user() function had some handling for "copied =3D=3D 0" that would change the scope of the amount to write, and loop back to try the write again. I attempted to construct a patch that just reverted the handling for "copied =3D=3D 0" in btrfs_copy_from_user(), however, that just resulte= d in my computer locking up when it reached the point where it was previously beginning to allocate disk space. So, I apologize for not having a patch to address the issue I'm seeing, but I hope I've added some insight. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" = in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [PATCH v2]Btrfs: pwrite blocked when writing from the mmaped buffer of the same page 2011-02-25 17:11 ` Mitch Harder @ 2011-02-25 18:43 ` Mitch Harder 2011-02-25 19:19 ` Chris Mason 2011-02-28 1:46 ` [PATCH] btrfs file write debugging patch Chris Mason 0 siblings, 2 replies; 43+ messages in thread From: Mitch Harder @ 2011-02-25 18:43 UTC (permalink / raw) To: Chris Mason; +Cc: Maria Wikström, Zhong, Xin, Johannes Hirte, linux-btrfs On Fri, Feb 25, 2011 at 11:11 AM, Mitch Harder <mitch.harder@sabayonlinux.org> wrote: > On Thu, Feb 24, 2011 at 5:14 PM, Mitch Harder > <mitch.harder@sabayonlinux.org> wrote: >> On Thu, Feb 24, 2011 at 10:32 AM, Mitch Harder >> <mitch.harder@sabayonlinux.org> wrote: >>> On Thu, Feb 24, 2011 at 10:19 AM, Chris Mason <chris.mason@oracle.c= om> wrote: >>>> Excerpts from Mitch Harder's message of 2011-02-24 11:03:07 -0500: >>>>> On Thu, Feb 24, 2011 at 10:00 AM, Chris Mason <chris.mason@oracle= =2Ecom> wrote: >>>>> > Excerpts from Mitch Harder's message of 2011-02-24 10:55:15 -05= 00: >>>>> >> 2011/2/24 Maria Wikstr=F6m <maria@ponstudios.se>: >>>>> >> > m=E5n 2011-02-21 klockan 09:51 +0800 skrev Zhong, Xin: >>>>> >> >> The backtrace in your attachment looks like a known bug of = 2.6.37 which have already been fixed in 2.6.38. I have no idea why late= st btrfs still hang in your environment if there's no debug info... >>>>> >> >> >>>>> >> > >>>>> >> > Haha, yes that's very hard :) >>>>> >> > >>>>> >> > 2.6.38-rc6 and btrfs-unstable behaves the same way. I can cl= ose the >>>>> >> > process with ctrl+c and it disappear a few seconds later. Th= ere is no >>>>> >> > CPU usage. Reading works because I can start htop and watch = "svn info" >>>>> >> > disappear, but everything writing to btrfs slows down to a c= rawl. It >>>>> >> > takes about 1 minute to log in. So I had to put the logs on = an other >>>>> >> > partition using ext3 to get the output from sysrq+t. >>>>> >> > >>>>> >> >>>>> >> I believe I've been experiencing this issue also. =A0However, = my problem >>>>> >> usually results in a "No space left on device" error rather th= an a >>>>> >> lock-up or crash. =A0But I've bisected my issue to this patch,= and my >>>>> >> "btrfs fi show" and "btrfs fi df" looks similar to others who'= ve >>>>> >> posted to this tread with all my space being allocated, but no= t used. >>>>> >> >>>>> > >>>>> > Sorry, which patch did you bisect the problem down to? >>>>> > >>>>> >>>>> The patch at the head of this thread: >>>>> >>>>> Btrfs: pwrite blocked when writing from the mmaped buffer of the = same page >>>> >>>> Hmmm, that patch shouldn't be changing our performance under delal= loc >>>> pressure, and it really shouldn't impact early enospc. >>>> >>> >>> I've bisected this issue around where this patch went into git, and >>> I've also constructed a testing patch that reverts this patch, and >>> placed it on top of the current Btrfs git sources (I understand thi= s >>> patch addresses a real issue, this was just for testing). >>> >>> It could be that this patch just "uncovers" another problem, but al= l >>> my tests seem to point to this patch triggering this issue. >>> >> >> I don't belief the previous ftrace I supplied had a large enough sco= pe >> to capture the issue. >> >> I've expanded my ftrace buffer, and filtered out everything but btrf= s* >> function calls ("# echo btrfs* > >> /sys/kernel/debug/tracing/set_ftrace_filter"). >> >> In this trace, I see btrfs spending a great deal of time in a while >> loop (while (iov_iter_count(&i) > 0) {)) in the btrfs_file_aio_write= () >> function in file.c without exiting the function. >> >> I'm going to try to inject some debugging trace_printk() statements = to >> find if that portion of code is proceeding normally with my test cas= e. >> >> I've put my expanded trace up on my local server, but my upload >> bandwidth is pretty sad, and it may take a few minutes to transfer >> even though it's only a 6MB file. >> >> http://dontpanic.dyndns.org/trace-openmotif-btrfs-v3.gz >> > > Apologies for only hitting "Reply" instead of "Reply-All" on my last = message. > > I've inserted additional trace_printk() to the btrfs_file_aio_write() > and btrfs_copy_from_user() function in file.c in order to characteriz= e > the problem I've been encountering. > > I can see btrfs getting stuck in a loop in the "while > (iov_iter_count(&i) > 0) {}" portion of the btrfs_file_aio_write() > function. > > The loop is more-or-less following this process (from within the > "while (iov_iter_count(&i) > 0) {}" loop): > > (1) Reserve some space with btrfs_delalloc_reserve_space() > (2) Prepare the reserved space with prepare_pages() > (3) Call btrfs_copy_from_user() to copy to the prepared space. > -------------> From btrfs_copy_from_user() > (4) ........Try to copy with copied =3D iov_iter_copy_from_user_atomi= c() > (5) ........The above operation results with copied =3D=3D 0. Break a= nd > return with a return value of 0 bytes copied. > (6) There is no special handling for copied =3D=3D 0 in the "while > (iov_iter_count(&i) > 0) {}" loop, so it loops back around, reserves > some more space, and tries again. > > If I look back at how the code was set up before the patch at the hea= d > of this thread was applied (Btrfs: pwrite blocked when writing from > the mmaped buffer of the same page), the btrfs_copy_from_user() > function had some handling for "copied =3D=3D 0" that would change th= e > scope of the amount to write, and loop back to try the write again. > > I attempted to construct a patch that just reverted the handling for > "copied =3D=3D 0" in btrfs_copy_from_user(), however, that just resul= ted > in my computer locking up when it reached the point where it was > previously beginning to allocate disk space. > > So, I apologize for not having a patch to address the issue I'm > seeing, but I hope I've added some insight. > Some clarification on my previous message... After looking at my ftrace log more closely, I can see where Btrfs is trying to release the allocated pages. However, the calculation for the number of dirty_pages is equal to 1 when "copied =3D=3D 0". So I'm seeing at least two problems: (1) It keeps looping when "copied =3D=3D 0". (2) One dirty page is not being released on every loop even though "copied =3D=3D 0" (at least this problem keeps it from being an infinit= e loop by eventually exhausting reserveable space on the disk). -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" = in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [PATCH v2]Btrfs: pwrite blocked when writing from the mmaped buffer of the same page 2011-02-25 18:43 ` Mitch Harder @ 2011-02-25 19:19 ` Chris Mason 2011-02-28 1:46 ` [PATCH] btrfs file write debugging patch Chris Mason 1 sibling, 0 replies; 43+ messages in thread From: Chris Mason @ 2011-02-25 19:19 UTC (permalink / raw) To: Mitch Harder; +Cc: Maria Wikström, Zhong, Xin, Johannes Hirte, linux-btrfs Excerpts from Mitch Harder's message of 2011-02-25 13:43:37 -0500: > > The loop is more-or-less following this process (from within the > > "while (iov_iter_count(&i) > 0) {}" loop): > > > > (1) Reserve some space with btrfs_delalloc_reserve_space() > > (2) Prepare the reserved space with prepare_pages() > > (3) Call btrfs_copy_from_user() to copy to the prepared space. > > -------------> From btrfs_copy_from_user() > > (4) ........Try to copy with copied = iov_iter_copy_from_user_atomic() > > (5) ........The above operation results with copied == 0. Break and > > return with a return value of 0 bytes copied. > > (6) There is no special handling for copied == 0 in the "while > > (iov_iter_count(&i) > 0) {}" loop, so it loops back around, reserves > > some more space, and tries again. > > > > If I look back at how the code was set up before the patch at the head > > of this thread was applied (Btrfs: pwrite blocked when writing from > > the mmaped buffer of the same page), the btrfs_copy_from_user() > > function had some handling for "copied == 0" that would change the > > scope of the amount to write, and loop back to try the write again. > > > > I attempted to construct a patch that just reverted the handling for > > "copied == 0" in btrfs_copy_from_user(), however, that just resulted > > in my computer locking up when it reached the point where it was > > previously beginning to allocate disk space. > > > > So, I apologize for not having a patch to address the issue I'm > > seeing, but I hope I've added some insight. > > > > Some clarification on my previous message... > > After looking at my ftrace log more closely, I can see where Btrfs is > trying to release the allocated pages. However, the calculation for > the number of dirty_pages is equal to 1 when "copied == 0". > > So I'm seeing at least two problems: > (1) It keeps looping when "copied == 0". > (2) One dirty page is not being released on every loop even though > "copied == 0" (at least this problem keeps it from being an infinite > loop by eventually exhausting reserveable space on the disk). Very nice, I think you're exactly right. I should be able to reproduce this now, thanks! -chris ^ permalink raw reply [flat|nested] 43+ messages in thread
* [PATCH] btrfs file write debugging patch 2011-02-25 18:43 ` Mitch Harder 2011-02-25 19:19 ` Chris Mason @ 2011-02-28 1:46 ` Chris Mason 2011-02-28 8:56 ` Zhong, Xin 2011-02-28 10:13 ` Johannes Hirte 1 sibling, 2 replies; 43+ messages in thread From: Chris Mason @ 2011-02-28 1:46 UTC (permalink / raw) To: Mitch Harder; +Cc: Maria Wikström, Zhong, Xin, Johannes Hirte, linux-btrfs Excerpts from Mitch Harder's message of 2011-02-25 13:43:37 -0500: > Some clarification on my previous message... > > After looking at my ftrace log more closely, I can see where Btrfs is > trying to release the allocated pages. However, the calculation for > the number of dirty_pages is equal to 1 when "copied == 0". > > So I'm seeing at least two problems: > (1) It keeps looping when "copied == 0". > (2) One dirty page is not being released on every loop even though > "copied == 0" (at least this problem keeps it from being an infinite > loop by eventually exhausting reserveable space on the disk). Hi everyone, There are actually tow bugs here. First the one that Mitch hit, and a second one that still results in bad file_write results with my debugging hunks (the first two hunks below) in place. My patch fixes Mitch's bug by checking for copied == 0 after btrfs_copy_from_user and going the correct delalloc accounting. This one looks solved, but you'll notice the patch is bigger. First, I add some random failures to btrfs_copy_from_user() by failing everyone once and a while. This was much more reliable than trying to use memory pressure than making copy_from_user fail. If copy_from_user fails and we partially update a page, we end up with a page that may go away due to memory pressure. But, btrfs_file_write assumes that only the first and last page may have good data that needs to be read off the disk. This patch ditches that code and puts it into prepare_pages instead. But I'm still having some errors during long stress.sh runs. Ideas are more than welcome, hopefully some other timezones will kick in ideas while I sleep. diff --git a/fs/btrfs/file.c b/fs/btrfs/file.c index 7084140..89a6a26 100644 --- a/fs/btrfs/file.c +++ b/fs/btrfs/file.c @@ -54,6 +54,13 @@ static noinline int btrfs_copy_from_user(loff_t pos, int num_pages, int offset = pos & (PAGE_CACHE_SIZE - 1); int total_copied = 0; + if ((jiffies % 10) == 0) + return 0; + + if ((jiffies % 25) == 0) { + write_bytes /= 2; + } + while (write_bytes > 0) { size_t count = min_t(size_t, PAGE_CACHE_SIZE - offset, write_bytes); @@ -763,6 +770,26 @@ out: } /* + * on error we return an unlocked page and the error value + * on success we return a locked page and 0 + */ +static int prepare_uptodate_page(struct page *page, u64 pos) +{ + int ret = 0; + if ((pos & (PAGE_CACHE_SIZE - 1)) && !PageUptodate(page)) { + ret = btrfs_readpage(NULL, page); + if (ret) + return ret; + lock_page(page); + if (!PageUptodate(page)) { + unlock_page(page); + return -EIO; + } + } + return 0; +} + +/* * this gets pages into the page cache and locks them down, it also properly * waits for data=ordered extents to finish before allowing the pages to be * modified. @@ -777,6 +804,7 @@ static noinline int prepare_pages(struct btrfs_root *root, struct file *file, unsigned long index = pos >> PAGE_CACHE_SHIFT; struct inode *inode = fdentry(file)->d_inode; int err = 0; + int faili = 0; u64 start_pos; u64 last_pos; @@ -794,15 +822,24 @@ again: for (i = 0; i < num_pages; i++) { pages[i] = grab_cache_page(inode->i_mapping, index + i); if (!pages[i]) { - int c; - for (c = i - 1; c >= 0; c--) { - unlock_page(pages[c]); - page_cache_release(pages[c]); - } - return -ENOMEM; + faili = i - 1; + err = -ENOMEM; + goto fail; + } + + if (i == 0) + err = prepare_uptodate_page(pages[i], pos); + else if (i == num_pages - 1) + err = prepare_uptodate_page(pages[i], + pos + write_bytes); + if (err) { + page_cache_release(pages[i]); + faili = i - 1; + goto fail; } wait_on_page_writeback(pages[i]); } + err = 0; if (start_pos < inode->i_size) { struct btrfs_ordered_extent *ordered; lock_extent_bits(&BTRFS_I(inode)->io_tree, @@ -842,6 +879,14 @@ again: WARN_ON(!PageLocked(pages[i])); } return 0; +fail: + while (faili >= 0) { + unlock_page(pages[faili]); + page_cache_release(pages[faili]); + faili--; + } + return err; + } static ssize_t btrfs_file_aio_write(struct kiocb *iocb, @@ -851,7 +896,6 @@ static ssize_t btrfs_file_aio_write(struct kiocb *iocb, struct file *file = iocb->ki_filp; struct inode *inode = fdentry(file)->d_inode; struct btrfs_root *root = BTRFS_I(inode)->root; - struct page *pinned[2]; struct page **pages = NULL; struct iov_iter i; loff_t *ppos = &iocb->ki_pos; @@ -872,9 +916,6 @@ static ssize_t btrfs_file_aio_write(struct kiocb *iocb, will_write = ((file->f_flags & O_DSYNC) || IS_SYNC(inode) || (file->f_flags & O_DIRECT)); - pinned[0] = NULL; - pinned[1] = NULL; - start_pos = pos; vfs_check_frozen(inode->i_sb, SB_FREEZE_WRITE); @@ -962,32 +1003,6 @@ static ssize_t btrfs_file_aio_write(struct kiocb *iocb, first_index = pos >> PAGE_CACHE_SHIFT; last_index = (pos + iov_iter_count(&i)) >> PAGE_CACHE_SHIFT; - /* - * there are lots of better ways to do this, but this code - * makes sure the first and last page in the file range are - * up to date and ready for cow - */ - if ((pos & (PAGE_CACHE_SIZE - 1))) { - pinned[0] = grab_cache_page(inode->i_mapping, first_index); - if (!PageUptodate(pinned[0])) { - ret = btrfs_readpage(NULL, pinned[0]); - BUG_ON(ret); - wait_on_page_locked(pinned[0]); - } else { - unlock_page(pinned[0]); - } - } - if ((pos + iov_iter_count(&i)) & (PAGE_CACHE_SIZE - 1)) { - pinned[1] = grab_cache_page(inode->i_mapping, last_index); - if (!PageUptodate(pinned[1])) { - ret = btrfs_readpage(NULL, pinned[1]); - BUG_ON(ret); - wait_on_page_locked(pinned[1]); - } else { - unlock_page(pinned[1]); - } - } - while (iov_iter_count(&i) > 0) { size_t offset = pos & (PAGE_CACHE_SIZE - 1); size_t write_bytes = min(iov_iter_count(&i), @@ -1024,8 +1039,12 @@ static ssize_t btrfs_file_aio_write(struct kiocb *iocb, copied = btrfs_copy_from_user(pos, num_pages, write_bytes, pages, &i); - dirty_pages = (copied + offset + PAGE_CACHE_SIZE - 1) >> - PAGE_CACHE_SHIFT; + if (copied == 0) + dirty_pages = 0; + else + dirty_pages = (copied + offset + + PAGE_CACHE_SIZE - 1) >> + PAGE_CACHE_SHIFT; if (num_pages > dirty_pages) { if (copied > 0) @@ -1069,10 +1088,6 @@ out: err = ret; kfree(pages); - if (pinned[0]) - page_cache_release(pinned[0]); - if (pinned[1]) - page_cache_release(pinned[1]); *ppos = pos; /* ^ permalink raw reply related [flat|nested] 43+ messages in thread
* RE: [PATCH] btrfs file write debugging patch 2011-02-28 1:46 ` [PATCH] btrfs file write debugging patch Chris Mason @ 2011-02-28 8:56 ` Zhong, Xin 2011-02-28 14:02 ` Chris Mason 2011-02-28 10:13 ` Johannes Hirte 1 sibling, 1 reply; 43+ messages in thread From: Zhong, Xin @ 2011-02-28 8:56 UTC (permalink / raw) To: Chris Mason, Mitch Harder Cc: Maria Wikström, Johannes Hirte, linux-btrfs T25lIHBvc3NpYmxlIGlzc3VlIEkgY2FuIHNlZSBpcyBpbiB0aGUgcmFuZG9tIGZhaWx1cmUgY2Fz ZSAjMiB0aGF0IGNvcHlfZnJvbV91c2VyIG9ubHkgcHJvY2VzcyBoYWxmIG9mIHRoZSBkYXRhLiAN Cg0KRm9yIGV4YW1wbGUsIGlmIGl0IHdyaXRlIGEgNGsgYWxpZ25lZCBwYWdlIGFuZCBjb3B5X2Zy b21fdXNlciBvbmx5IHdyaXRlIDJrLiBUaGVuIGl0IHdpbGwgbm90IGNhbGwgYnRyZnNfZGVsYWxs b2NfcmVsZWFzZV9zcGFjZSBzaW5jZSBudW1fcGFnZXMgYW5kIGRpcnR5X3BhZ2VzIGFyZSBib3Ro IDEuIA0KSW4gdGhlIG5leHQgcm91bmQsIGl0IHdyaXRlIGFub3RoZXIgMmsgYW5kIGJ0cmZzX2Rl bGFsbG9jX3Jlc2VydmVfc3BhY2UgaXMgY2FsbGVkIHR3aWNlIGZvciB0aGUgc2FtZSBwYWdlLiAN Cg0KSXMgaXQgYSBwcm9ibGVtPyBUaGFua3MhDQoNCi0tLS0tT3JpZ2luYWwgTWVzc2FnZS0tLS0t DQpGcm9tOiBDaHJpcyBNYXNvbiBbbWFpbHRvOmNocmlzLm1hc29uQG9yYWNsZS5jb21dIA0KU2Vu dDogTW9uZGF5LCBGZWJydWFyeSAyOCwgMjAxMSA5OjQ2IEFNDQpUbzogTWl0Y2ggSGFyZGVyDQpD YzogTWFyaWEgV2lrc3Ryw7ZtOyBaaG9uZywgWGluOyBKb2hhbm5lcyBIaXJ0ZTsgbGludXgtYnRy ZnNAdmdlci5rZXJuZWwub3JnDQpTdWJqZWN0OiBbUEFUQ0hdIGJ0cmZzIGZpbGUgd3JpdGUgZGVi dWdnaW5nIHBhdGNoDQoNCkV4Y2VycHRzIGZyb20gTWl0Y2ggSGFyZGVyJ3MgbWVzc2FnZSBvZiAy MDExLTAyLTI1IDEzOjQzOjM3IC0wNTAwOg0KPiBTb21lIGNsYXJpZmljYXRpb24gb24gbXkgcHJl dmlvdXMgbWVzc2FnZS4uLg0KPiANCj4gQWZ0ZXIgbG9va2luZyBhdCBteSBmdHJhY2UgbG9nIG1v cmUgY2xvc2VseSwgSSBjYW4gc2VlIHdoZXJlIEJ0cmZzIGlzDQo+IHRyeWluZyB0byByZWxlYXNl IHRoZSBhbGxvY2F0ZWQgcGFnZXMuICBIb3dldmVyLCB0aGUgY2FsY3VsYXRpb24gZm9yDQo+IHRo ZSBudW1iZXIgb2YgZGlydHlfcGFnZXMgaXMgZXF1YWwgdG8gMSB3aGVuICJjb3BpZWQgPT0gMCIu DQo+IA0KPiBTbyBJJ20gc2VlaW5nIGF0IGxlYXN0IHR3byBwcm9ibGVtczoNCj4gKDEpICBJdCBr ZWVwcyBsb29waW5nIHdoZW4gImNvcGllZCA9PSAwIi4NCj4gKDIpICBPbmUgZGlydHkgcGFnZSBp cyBub3QgYmVpbmcgcmVsZWFzZWQgb24gZXZlcnkgbG9vcCBldmVuIHRob3VnaA0KPiAiY29waWVk ID09IDAiIChhdCBsZWFzdCB0aGlzIHByb2JsZW0ga2VlcHMgaXQgZnJvbSBiZWluZyBhbiBpbmZp bml0ZQ0KPiBsb29wIGJ5IGV2ZW50dWFsbHkgZXhoYXVzdGluZyByZXNlcnZlYWJsZSBzcGFjZSBv biB0aGUgZGlzaykuDQoNCkhpIGV2ZXJ5b25lLA0KDQpUaGVyZSBhcmUgYWN0dWFsbHkgdG93IGJ1 Z3MgaGVyZS4gIEZpcnN0IHRoZSBvbmUgdGhhdCBNaXRjaCBoaXQsIGFuZCBhDQpzZWNvbmQgb25l IHRoYXQgc3RpbGwgcmVzdWx0cyBpbiBiYWQgZmlsZV93cml0ZSByZXN1bHRzIHdpdGggbXkNCmRl YnVnZ2luZyBodW5rcyAodGhlIGZpcnN0IHR3byBodW5rcyBiZWxvdykgaW4gcGxhY2UuDQoNCk15 IHBhdGNoIGZpeGVzIE1pdGNoJ3MgYnVnIGJ5IGNoZWNraW5nIGZvciBjb3BpZWQgPT0gMCBhZnRl cg0KYnRyZnNfY29weV9mcm9tX3VzZXIgYW5kIGdvaW5nIHRoZSBjb3JyZWN0IGRlbGFsbG9jIGFj Y291bnRpbmcuICBUaGlzDQpvbmUgbG9va3Mgc29sdmVkLCBidXQgeW91J2xsIG5vdGljZSB0aGUg cGF0Y2ggaXMgYmlnZ2VyLg0KDQpGaXJzdCwgSSBhZGQgc29tZSByYW5kb20gZmFpbHVyZXMgdG8g YnRyZnNfY29weV9mcm9tX3VzZXIoKSBieSBmYWlsaW5nDQpldmVyeW9uZSBvbmNlIGFuZCBhIHdo aWxlLiAgVGhpcyB3YXMgbXVjaCBtb3JlIHJlbGlhYmxlIHRoYW4gdHJ5aW5nIHRvDQp1c2UgbWVt b3J5IHByZXNzdXJlIHRoYW4gbWFraW5nIGNvcHlfZnJvbV91c2VyIGZhaWwuDQoNCklmIGNvcHlf ZnJvbV91c2VyIGZhaWxzIGFuZCB3ZSBwYXJ0aWFsbHkgdXBkYXRlIGEgcGFnZSwgd2UgZW5kIHVw IHdpdGggYQ0KcGFnZSB0aGF0IG1heSBnbyBhd2F5IGR1ZSB0byBtZW1vcnkgcHJlc3N1cmUuICBC dXQsIGJ0cmZzX2ZpbGVfd3JpdGUNCmFzc3VtZXMgdGhhdCBvbmx5IHRoZSBmaXJzdCBhbmQgbGFz dCBwYWdlIG1heSBoYXZlIGdvb2QgZGF0YSB0aGF0IG5lZWRzDQp0byBiZSByZWFkIG9mZiB0aGUg ZGlzay4NCg0KVGhpcyBwYXRjaCBkaXRjaGVzIHRoYXQgY29kZSBhbmQgcHV0cyBpdCBpbnRvIHBy ZXBhcmVfcGFnZXMgaW5zdGVhZC4NCkJ1dCBJJ20gc3RpbGwgaGF2aW5nIHNvbWUgZXJyb3JzIGR1 cmluZyBsb25nIHN0cmVzcy5zaCBydW5zLiAgSWRlYXMgYXJlDQptb3JlIHRoYW4gd2VsY29tZSwg aG9wZWZ1bGx5IHNvbWUgb3RoZXIgdGltZXpvbmVzIHdpbGwga2ljayBpbiBpZGVhcw0Kd2hpbGUg SSBzbGVlcC4NCg0KZGlmZiAtLWdpdCBhL2ZzL2J0cmZzL2ZpbGUuYyBiL2ZzL2J0cmZzL2ZpbGUu Yw0KaW5kZXggNzA4NDE0MC4uODlhNmEyNiAxMDA2NDQNCi0tLSBhL2ZzL2J0cmZzL2ZpbGUuYw0K KysrIGIvZnMvYnRyZnMvZmlsZS5jDQpAQCAtNTQsNiArNTQsMTMgQEAgc3RhdGljIG5vaW5saW5l IGludCBidHJmc19jb3B5X2Zyb21fdXNlcihsb2ZmX3QgcG9zLCBpbnQgbnVtX3BhZ2VzLA0KIAlp bnQgb2Zmc2V0ID0gcG9zICYgKFBBR0VfQ0FDSEVfU0laRSAtIDEpOw0KIAlpbnQgdG90YWxfY29w aWVkID0gMDsNCiANCisJaWYgKChqaWZmaWVzICUgMTApID09IDApDQorCQlyZXR1cm4gMDsNCisN CisJaWYgKChqaWZmaWVzICUgMjUpID09IDApIHsNCisJCXdyaXRlX2J5dGVzIC89IDI7DQorCX0N CisNCiAJd2hpbGUgKHdyaXRlX2J5dGVzID4gMCkgew0KIAkJc2l6ZV90IGNvdW50ID0gbWluX3Qo c2l6ZV90LA0KIAkJCQkgICAgIFBBR0VfQ0FDSEVfU0laRSAtIG9mZnNldCwgd3JpdGVfYnl0ZXMp Ow0KQEAgLTc2Myw2ICs3NzAsMjYgQEAgb3V0Og0KIH0NCiANCiAvKg0KKyAqIG9uIGVycm9yIHdl IHJldHVybiBhbiB1bmxvY2tlZCBwYWdlIGFuZCB0aGUgZXJyb3IgdmFsdWUNCisgKiBvbiBzdWNj ZXNzIHdlIHJldHVybiBhIGxvY2tlZCBwYWdlIGFuZCAwDQorICovDQorc3RhdGljIGludCBwcmVw YXJlX3VwdG9kYXRlX3BhZ2Uoc3RydWN0IHBhZ2UgKnBhZ2UsIHU2NCBwb3MpDQorew0KKwlpbnQg cmV0ID0gMDsNCisJaWYgKChwb3MgJiAoUEFHRV9DQUNIRV9TSVpFIC0gMSkpICYmICFQYWdlVXB0 b2RhdGUocGFnZSkpIHsNCisJCXJldCA9IGJ0cmZzX3JlYWRwYWdlKE5VTEwsIHBhZ2UpOw0KKwkJ aWYgKHJldCkNCisJCQlyZXR1cm4gcmV0Ow0KKwkJbG9ja19wYWdlKHBhZ2UpOw0KKwkJaWYgKCFQ YWdlVXB0b2RhdGUocGFnZSkpIHsNCisJCQl1bmxvY2tfcGFnZShwYWdlKTsNCisJCQlyZXR1cm4g LUVJTzsNCisJCX0NCisJfQ0KKwlyZXR1cm4gMDsNCit9DQorDQorLyoNCiAgKiB0aGlzIGdldHMg cGFnZXMgaW50byB0aGUgcGFnZSBjYWNoZSBhbmQgbG9ja3MgdGhlbSBkb3duLCBpdCBhbHNvIHBy b3Blcmx5DQogICogd2FpdHMgZm9yIGRhdGE9b3JkZXJlZCBleHRlbnRzIHRvIGZpbmlzaCBiZWZv cmUgYWxsb3dpbmcgdGhlIHBhZ2VzIHRvIGJlDQogICogbW9kaWZpZWQuDQpAQCAtNzc3LDYgKzgw NCw3IEBAIHN0YXRpYyBub2lubGluZSBpbnQgcHJlcGFyZV9wYWdlcyhzdHJ1Y3QgYnRyZnNfcm9v dCAqcm9vdCwgc3RydWN0IGZpbGUgKmZpbGUsDQogCXVuc2lnbmVkIGxvbmcgaW5kZXggPSBwb3Mg Pj4gUEFHRV9DQUNIRV9TSElGVDsNCiAJc3RydWN0IGlub2RlICppbm9kZSA9IGZkZW50cnkoZmls ZSktPmRfaW5vZGU7DQogCWludCBlcnIgPSAwOw0KKwlpbnQgZmFpbGkgPSAwOw0KIAl1NjQgc3Rh cnRfcG9zOw0KIAl1NjQgbGFzdF9wb3M7DQogDQpAQCAtNzk0LDE1ICs4MjIsMjQgQEAgYWdhaW46 DQogCWZvciAoaSA9IDA7IGkgPCBudW1fcGFnZXM7IGkrKykgew0KIAkJcGFnZXNbaV0gPSBncmFi X2NhY2hlX3BhZ2UoaW5vZGUtPmlfbWFwcGluZywgaW5kZXggKyBpKTsNCiAJCWlmICghcGFnZXNb aV0pIHsNCi0JCQlpbnQgYzsNCi0JCQlmb3IgKGMgPSBpIC0gMTsgYyA+PSAwOyBjLS0pIHsNCi0J CQkJdW5sb2NrX3BhZ2UocGFnZXNbY10pOw0KLQkJCQlwYWdlX2NhY2hlX3JlbGVhc2UocGFnZXNb Y10pOw0KLQkJCX0NCi0JCQlyZXR1cm4gLUVOT01FTTsNCisJCQlmYWlsaSA9IGkgLSAxOw0KKwkJ CWVyciA9IC1FTk9NRU07DQorCQkJZ290byBmYWlsOw0KKwkJfQ0KKw0KKwkJaWYgKGkgPT0gMCkN CisJCQllcnIgPSBwcmVwYXJlX3VwdG9kYXRlX3BhZ2UocGFnZXNbaV0sIHBvcyk7DQorCQllbHNl IGlmIChpID09IG51bV9wYWdlcyAtIDEpDQorCQkJZXJyID0gcHJlcGFyZV91cHRvZGF0ZV9wYWdl KHBhZ2VzW2ldLA0KKwkJCQkJCSAgICBwb3MgKyB3cml0ZV9ieXRlcyk7DQorCQlpZiAoZXJyKSB7 DQorCQkJcGFnZV9jYWNoZV9yZWxlYXNlKHBhZ2VzW2ldKTsNCisJCQlmYWlsaSA9IGkgLSAxOw0K KwkJCWdvdG8gZmFpbDsNCiAJCX0NCiAJCXdhaXRfb25fcGFnZV93cml0ZWJhY2socGFnZXNbaV0p Ow0KIAl9DQorCWVyciA9IDA7DQogCWlmIChzdGFydF9wb3MgPCBpbm9kZS0+aV9zaXplKSB7DQog CQlzdHJ1Y3QgYnRyZnNfb3JkZXJlZF9leHRlbnQgKm9yZGVyZWQ7DQogCQlsb2NrX2V4dGVudF9i aXRzKCZCVFJGU19JKGlub2RlKS0+aW9fdHJlZSwNCkBAIC04NDIsNiArODc5LDE0IEBAIGFnYWlu Og0KIAkJV0FSTl9PTighUGFnZUxvY2tlZChwYWdlc1tpXSkpOw0KIAl9DQogCXJldHVybiAwOw0K K2ZhaWw6DQorCXdoaWxlIChmYWlsaSA+PSAwKSB7DQorCQl1bmxvY2tfcGFnZShwYWdlc1tmYWls aV0pOw0KKwkJcGFnZV9jYWNoZV9yZWxlYXNlKHBhZ2VzW2ZhaWxpXSk7DQorCQlmYWlsaS0tOw0K Kwl9DQorCXJldHVybiBlcnI7DQorDQogfQ0KIA0KIHN0YXRpYyBzc2l6ZV90IGJ0cmZzX2ZpbGVf YWlvX3dyaXRlKHN0cnVjdCBraW9jYiAqaW9jYiwNCkBAIC04NTEsNyArODk2LDYgQEAgc3RhdGlj IHNzaXplX3QgYnRyZnNfZmlsZV9haW9fd3JpdGUoc3RydWN0IGtpb2NiICppb2NiLA0KIAlzdHJ1 Y3QgZmlsZSAqZmlsZSA9IGlvY2ItPmtpX2ZpbHA7DQogCXN0cnVjdCBpbm9kZSAqaW5vZGUgPSBm ZGVudHJ5KGZpbGUpLT5kX2lub2RlOw0KIAlzdHJ1Y3QgYnRyZnNfcm9vdCAqcm9vdCA9IEJUUkZT X0koaW5vZGUpLT5yb290Ow0KLQlzdHJ1Y3QgcGFnZSAqcGlubmVkWzJdOw0KIAlzdHJ1Y3QgcGFn ZSAqKnBhZ2VzID0gTlVMTDsNCiAJc3RydWN0IGlvdl9pdGVyIGk7DQogCWxvZmZfdCAqcHBvcyA9 ICZpb2NiLT5raV9wb3M7DQpAQCAtODcyLDkgKzkxNiw2IEBAIHN0YXRpYyBzc2l6ZV90IGJ0cmZz X2ZpbGVfYWlvX3dyaXRlKHN0cnVjdCBraW9jYiAqaW9jYiwNCiAJd2lsbF93cml0ZSA9ICgoZmls ZS0+Zl9mbGFncyAmIE9fRFNZTkMpIHx8IElTX1NZTkMoaW5vZGUpIHx8DQogCQkgICAgICAoZmls ZS0+Zl9mbGFncyAmIE9fRElSRUNUKSk7DQogDQotCXBpbm5lZFswXSA9IE5VTEw7DQotCXBpbm5l ZFsxXSA9IE5VTEw7DQotDQogCXN0YXJ0X3BvcyA9IHBvczsNCiANCiAJdmZzX2NoZWNrX2Zyb3pl bihpbm9kZS0+aV9zYiwgU0JfRlJFRVpFX1dSSVRFKTsNCkBAIC05NjIsMzIgKzEwMDMsNiBAQCBz dGF0aWMgc3NpemVfdCBidHJmc19maWxlX2Fpb193cml0ZShzdHJ1Y3Qga2lvY2IgKmlvY2IsDQog CWZpcnN0X2luZGV4ID0gcG9zID4+IFBBR0VfQ0FDSEVfU0hJRlQ7DQogCWxhc3RfaW5kZXggPSAo cG9zICsgaW92X2l0ZXJfY291bnQoJmkpKSA+PiBQQUdFX0NBQ0hFX1NISUZUOw0KIA0KLQkvKg0K LQkgKiB0aGVyZSBhcmUgbG90cyBvZiBiZXR0ZXIgd2F5cyB0byBkbyB0aGlzLCBidXQgdGhpcyBj b2RlDQotCSAqIG1ha2VzIHN1cmUgdGhlIGZpcnN0IGFuZCBsYXN0IHBhZ2UgaW4gdGhlIGZpbGUg cmFuZ2UgYXJlDQotCSAqIHVwIHRvIGRhdGUgYW5kIHJlYWR5IGZvciBjb3cNCi0JICovDQotCWlm ICgocG9zICYgKFBBR0VfQ0FDSEVfU0laRSAtIDEpKSkgew0KLQkJcGlubmVkWzBdID0gZ3JhYl9j YWNoZV9wYWdlKGlub2RlLT5pX21hcHBpbmcsIGZpcnN0X2luZGV4KTsNCi0JCWlmICghUGFnZVVw dG9kYXRlKHBpbm5lZFswXSkpIHsNCi0JCQlyZXQgPSBidHJmc19yZWFkcGFnZShOVUxMLCBwaW5u ZWRbMF0pOw0KLQkJCUJVR19PTihyZXQpOw0KLQkJCXdhaXRfb25fcGFnZV9sb2NrZWQocGlubmVk WzBdKTsNCi0JCX0gZWxzZSB7DQotCQkJdW5sb2NrX3BhZ2UocGlubmVkWzBdKTsNCi0JCX0NCi0J fQ0KLQlpZiAoKHBvcyArIGlvdl9pdGVyX2NvdW50KCZpKSkgJiAoUEFHRV9DQUNIRV9TSVpFIC0g MSkpIHsNCi0JCXBpbm5lZFsxXSA9IGdyYWJfY2FjaGVfcGFnZShpbm9kZS0+aV9tYXBwaW5nLCBs YXN0X2luZGV4KTsNCi0JCWlmICghUGFnZVVwdG9kYXRlKHBpbm5lZFsxXSkpIHsNCi0JCQlyZXQg PSBidHJmc19yZWFkcGFnZShOVUxMLCBwaW5uZWRbMV0pOw0KLQkJCUJVR19PTihyZXQpOw0KLQkJ CXdhaXRfb25fcGFnZV9sb2NrZWQocGlubmVkWzFdKTsNCi0JCX0gZWxzZSB7DQotCQkJdW5sb2Nr X3BhZ2UocGlubmVkWzFdKTsNCi0JCX0NCi0JfQ0KLQ0KIAl3aGlsZSAoaW92X2l0ZXJfY291bnQo JmkpID4gMCkgew0KIAkJc2l6ZV90IG9mZnNldCA9IHBvcyAmIChQQUdFX0NBQ0hFX1NJWkUgLSAx KTsNCiAJCXNpemVfdCB3cml0ZV9ieXRlcyA9IG1pbihpb3ZfaXRlcl9jb3VudCgmaSksDQpAQCAt MTAyNCw4ICsxMDM5LDEyIEBAIHN0YXRpYyBzc2l6ZV90IGJ0cmZzX2ZpbGVfYWlvX3dyaXRlKHN0 cnVjdCBraW9jYiAqaW9jYiwNCiANCiAJCWNvcGllZCA9IGJ0cmZzX2NvcHlfZnJvbV91c2VyKHBv cywgbnVtX3BhZ2VzLA0KIAkJCQkJICAgd3JpdGVfYnl0ZXMsIHBhZ2VzLCAmaSk7DQotCQlkaXJ0 eV9wYWdlcyA9IChjb3BpZWQgKyBvZmZzZXQgKyBQQUdFX0NBQ0hFX1NJWkUgLSAxKSA+Pg0KLQkJ CQlQQUdFX0NBQ0hFX1NISUZUOw0KKwkJaWYgKGNvcGllZCA9PSAwKQ0KKwkJCWRpcnR5X3BhZ2Vz ID0gMDsNCisJCWVsc2UNCisJCQlkaXJ0eV9wYWdlcyA9IChjb3BpZWQgKyBvZmZzZXQgKw0KKwkJ CQkgICAgICAgUEFHRV9DQUNIRV9TSVpFIC0gMSkgPj4NCisJCQkJICAgICAgIFBBR0VfQ0FDSEVf U0hJRlQ7DQogDQogCQlpZiAobnVtX3BhZ2VzID4gZGlydHlfcGFnZXMpIHsNCiAJCQlpZiAoY29w aWVkID4gMCkNCkBAIC0xMDY5LDEwICsxMDg4LDYgQEAgb3V0Og0KIAkJZXJyID0gcmV0Ow0KIA0K IAlrZnJlZShwYWdlcyk7DQotCWlmIChwaW5uZWRbMF0pDQotCQlwYWdlX2NhY2hlX3JlbGVhc2Uo cGlubmVkWzBdKTsNCi0JaWYgKHBpbm5lZFsxXSkNCi0JCXBhZ2VfY2FjaGVfcmVsZWFzZShwaW5u ZWRbMV0pOw0KIAkqcHBvcyA9IHBvczsNCiANCiAJLyoNCg== ^ permalink raw reply [flat|nested] 43+ messages in thread
* RE: [PATCH] btrfs file write debugging patch 2011-02-28 8:56 ` Zhong, Xin @ 2011-02-28 14:02 ` Chris Mason 0 siblings, 0 replies; 43+ messages in thread From: Chris Mason @ 2011-02-28 14:02 UTC (permalink / raw) To: Zhong, Xin; +Cc: Mitch Harder, Maria Wikström, Johannes Hirte, linux-btrfs Excerpts from Zhong, Xin's message of 2011-02-28 03:56:40 -0500: > One possible issue I can see is in the random failure case #2 that co= py_from_user only process half of the data.=20 >=20 > For example, if it write a 4k aligned page and copy_from_user only wr= ite 2k. Then it will not call btrfs_delalloc_release_space since num_pa= ges and dirty_pages are both 1.=20 > In the next round, it write another 2k and btrfs_delalloc_reserve_spa= ce is called twice for the same page.=20 >=20 > Is it a problem? Thanks! It should be the correct answer. The result of the short copy_from_use= r should be exactly the same as two write calls where one does 2K and the other does another 2K. Either way, it shouldn't result in incorrect bytes in the file, which i= s still happening for me with the debugging hunks in place. -chris >=20 > -----Original Message----- > From: Chris Mason [mailto:chris.mason@oracle.com]=20 > Sent: Monday, February 28, 2011 9:46 AM > To: Mitch Harder > Cc: Maria Wikstr=C3=B6m; Zhong, Xin; Johannes Hirte; linux-btrfs@vger= =2Ekernel.org > Subject: [PATCH] btrfs file write debugging patch >=20 > Excerpts from Mitch Harder's message of 2011-02-25 13:43:37 -0500: > > Some clarification on my previous message... > >=20 > > After looking at my ftrace log more closely, I can see where Btrfs = is > > trying to release the allocated pages. However, the calculation fo= r > > the number of dirty_pages is equal to 1 when "copied =3D=3D 0". > >=20 > > So I'm seeing at least two problems: > > (1) It keeps looping when "copied =3D=3D 0". > > (2) One dirty page is not being released on every loop even though > > "copied =3D=3D 0" (at least this problem keeps it from being an inf= inite > > loop by eventually exhausting reserveable space on the disk). >=20 > Hi everyone, >=20 > There are actually tow bugs here. First the one that Mitch hit, and = a > second one that still results in bad file_write results with my > debugging hunks (the first two hunks below) in place. >=20 > My patch fixes Mitch's bug by checking for copied =3D=3D 0 after > btrfs_copy_from_user and going the correct delalloc accounting. This > one looks solved, but you'll notice the patch is bigger. >=20 > First, I add some random failures to btrfs_copy_from_user() by failin= g > everyone once and a while. This was much more reliable than trying t= o > use memory pressure than making copy_from_user fail. >=20 > If copy_from_user fails and we partially update a page, we end up wit= h a > page that may go away due to memory pressure. But, btrfs_file_write > assumes that only the first and last page may have good data that nee= ds > to be read off the disk. >=20 > This patch ditches that code and puts it into prepare_pages instead. > But I'm still having some errors during long stress.sh runs. Ideas a= re > more than welcome, hopefully some other timezones will kick in ideas > while I sleep. >=20 > diff --git a/fs/btrfs/file.c b/fs/btrfs/file.c > index 7084140..89a6a26 100644 > --- a/fs/btrfs/file.c > +++ b/fs/btrfs/file.c > @@ -54,6 +54,13 @@ static noinline int btrfs_copy_from_user(loff_t po= s, int num_pages, > int offset =3D pos & (PAGE_CACHE_SIZE - 1); > int total_copied =3D 0; > =20 > + if ((jiffies % 10) =3D=3D 0) > + return 0; > + > + if ((jiffies % 25) =3D=3D 0) { > + write_bytes /=3D 2; > + } > + > while (write_bytes > 0) { > size_t count =3D min_t(size_t, > PAGE_CACHE_SIZE - offset, write_bytes); > @@ -763,6 +770,26 @@ out: > } > =20 > /* > + * on error we return an unlocked page and the error value > + * on success we return a locked page and 0 > + */ > +static int prepare_uptodate_page(struct page *page, u64 pos) > +{ > + int ret =3D 0; > + if ((pos & (PAGE_CACHE_SIZE - 1)) && !PageUptodate(page)) { > + ret =3D btrfs_readpage(NULL, page); > + if (ret) > + return ret; > + lock_page(page); > + if (!PageUptodate(page)) { > + unlock_page(page); > + return -EIO; > + } > + } > + return 0; > +} > + > +/* > * this gets pages into the page cache and locks them down, it also = properly > * waits for data=3Dordered extents to finish before allowing the pa= ges to be > * modified. > @@ -777,6 +804,7 @@ static noinline int prepare_pages(struct btrfs_ro= ot *root, struct file *file, > unsigned long index =3D pos >> PAGE_CACHE_SHIFT; > struct inode *inode =3D fdentry(file)->d_inode; > int err =3D 0; > + int faili =3D 0; > u64 start_pos; > u64 last_pos; > =20 > @@ -794,15 +822,24 @@ again: > for (i =3D 0; i < num_pages; i++) { > pages[i] =3D grab_cache_page(inode->i_mapping, index + i); > if (!pages[i]) { > - int c; > - for (c =3D i - 1; c >=3D 0; c--) { > - unlock_page(pages[c]); > - page_cache_release(pages[c]); > - } > - return -ENOMEM; > + faili =3D i - 1; > + err =3D -ENOMEM; > + goto fail; > + } > + > + if (i =3D=3D 0) > + err =3D prepare_uptodate_page(pages[i], pos); > + else if (i =3D=3D num_pages - 1) > + err =3D prepare_uptodate_page(pages[i], > + pos + write_bytes); > + if (err) { > + page_cache_release(pages[i]); > + faili =3D i - 1; > + goto fail; > } > wait_on_page_writeback(pages[i]); > } > + err =3D 0; > if (start_pos < inode->i_size) { > struct btrfs_ordered_extent *ordered; > lock_extent_bits(&BTRFS_I(inode)->io_tree, > @@ -842,6 +879,14 @@ again: > WARN_ON(!PageLocked(pages[i])); > } > return 0; > +fail: > + while (faili >=3D 0) { > + unlock_page(pages[faili]); > + page_cache_release(pages[faili]); > + faili--; > + } > + return err; > + > } > =20 > static ssize_t btrfs_file_aio_write(struct kiocb *iocb, > @@ -851,7 +896,6 @@ static ssize_t btrfs_file_aio_write(struct kiocb = *iocb, > struct file *file =3D iocb->ki_filp; > struct inode *inode =3D fdentry(file)->d_inode; > struct btrfs_root *root =3D BTRFS_I(inode)->root; > - struct page *pinned[2]; > struct page **pages =3D NULL; > struct iov_iter i; > loff_t *ppos =3D &iocb->ki_pos; > @@ -872,9 +916,6 @@ static ssize_t btrfs_file_aio_write(struct kiocb = *iocb, > will_write =3D ((file->f_flags & O_DSYNC) || IS_SYNC(inode) || > (file->f_flags & O_DIRECT)); > =20 > - pinned[0] =3D NULL; > - pinned[1] =3D NULL; > - > start_pos =3D pos; > =20 > vfs_check_frozen(inode->i_sb, SB_FREEZE_WRITE); > @@ -962,32 +1003,6 @@ static ssize_t btrfs_file_aio_write(struct kioc= b *iocb, > first_index =3D pos >> PAGE_CACHE_SHIFT; > last_index =3D (pos + iov_iter_count(&i)) >> PAGE_CACHE_SHIFT; > =20 > - /* > - * there are lots of better ways to do this, but this code > - * makes sure the first and last page in the file range are > - * up to date and ready for cow > - */ > - if ((pos & (PAGE_CACHE_SIZE - 1))) { > - pinned[0] =3D grab_cache_page(inode->i_mapping, first_index)= ; > - if (!PageUptodate(pinned[0])) { > - ret =3D btrfs_readpage(NULL, pinned[0]); > - BUG_ON(ret); > - wait_on_page_locked(pinned[0]); > - } else { > - unlock_page(pinned[0]); > - } > - } > - if ((pos + iov_iter_count(&i)) & (PAGE_CACHE_SIZE - 1)) { > - pinned[1] =3D grab_cache_page(inode->i_mapping, last_index); > - if (!PageUptodate(pinned[1])) { > - ret =3D btrfs_readpage(NULL, pinned[1]); > - BUG_ON(ret); > - wait_on_page_locked(pinned[1]); > - } else { > - unlock_page(pinned[1]); > - } > - } > - > while (iov_iter_count(&i) > 0) { > size_t offset =3D pos & (PAGE_CACHE_SIZE - 1); > size_t write_bytes =3D min(iov_iter_count(&i), > @@ -1024,8 +1039,12 @@ static ssize_t btrfs_file_aio_write(struct kio= cb *iocb, > =20 > copied =3D btrfs_copy_from_user(pos, num_pages, > write_bytes, pages, &i); > - dirty_pages =3D (copied + offset + PAGE_CACHE_SIZE - 1) >> > - PAGE_CACHE_SHIFT; > + if (copied =3D=3D 0) > + dirty_pages =3D 0; > + else > + dirty_pages =3D (copied + offset + > + PAGE_CACHE_SIZE - 1) >> > + PAGE_CACHE_SHIFT; > =20 > if (num_pages > dirty_pages) { > if (copied > 0) > @@ -1069,10 +1088,6 @@ out: > err =3D ret; > =20 > kfree(pages); > - if (pinned[0]) > - page_cache_release(pinned[0]); > - if (pinned[1]) > - page_cache_release(pinned[1]); > *ppos =3D pos; > =20 > /* -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" = in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [PATCH] btrfs file write debugging patch 2011-02-28 1:46 ` [PATCH] btrfs file write debugging patch Chris Mason 2011-02-28 8:56 ` Zhong, Xin @ 2011-02-28 10:13 ` Johannes Hirte 2011-02-28 14:00 ` Chris Mason 2011-02-28 16:10 ` Josef Bacik 1 sibling, 2 replies; 43+ messages in thread From: Johannes Hirte @ 2011-02-28 10:13 UTC (permalink / raw) To: Chris Mason; +Cc: Mitch Harder, Maria Wikström, Zhong, Xin, linux-btrfs On Monday 28 February 2011 02:46:05 Chris Mason wrote: > Excerpts from Mitch Harder's message of 2011-02-25 13:43:37 -0500: > > Some clarification on my previous message... > > > > After looking at my ftrace log more closely, I can see where Btrfs is > > trying to release the allocated pages. However, the calculation for > > the number of dirty_pages is equal to 1 when "copied == 0". > > > > So I'm seeing at least two problems: > > (1) It keeps looping when "copied == 0". > > (2) One dirty page is not being released on every loop even though > > "copied == 0" (at least this problem keeps it from being an infinite > > loop by eventually exhausting reserveable space on the disk). > > Hi everyone, > > There are actually tow bugs here. First the one that Mitch hit, and a > second one that still results in bad file_write results with my > debugging hunks (the first two hunks below) in place. > > My patch fixes Mitch's bug by checking for copied == 0 after > btrfs_copy_from_user and going the correct delalloc accounting. This > one looks solved, but you'll notice the patch is bigger. > > First, I add some random failures to btrfs_copy_from_user() by failing > everyone once and a while. This was much more reliable than trying to > use memory pressure than making copy_from_user fail. > > If copy_from_user fails and we partially update a page, we end up with a > page that may go away due to memory pressure. But, btrfs_file_write > assumes that only the first and last page may have good data that needs > to be read off the disk. > > This patch ditches that code and puts it into prepare_pages instead. > But I'm still having some errors during long stress.sh runs. Ideas are > more than welcome, hopefully some other timezones will kick in ideas > while I sleep. At least it doesn't fix the emerge-problem for me. The behavior is now the same as with 2.6.38-rc3. It needs a 'emerge --oneshot dev-libs/libgcrypt' with no further interaction to get the emerge-process hang with a svn-process consuming 100% CPU. I can cancel the emerge-process with ctrl-c but the spawned svn-process stays and it needs a reboot to get rid of it. ^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [PATCH] btrfs file write debugging patch 2011-02-28 10:13 ` Johannes Hirte @ 2011-02-28 14:00 ` Chris Mason 2011-02-28 16:10 ` Josef Bacik 1 sibling, 0 replies; 43+ messages in thread From: Chris Mason @ 2011-02-28 14:00 UTC (permalink / raw) To: Johannes Hirte; +Cc: Mitch Harder, Maria Wikström, Zhong, Xin, linux-btrfs Excerpts from Johannes Hirte's message of 2011-02-28 05:13:59 -0500: > On Monday 28 February 2011 02:46:05 Chris Mason wrote: > > Excerpts from Mitch Harder's message of 2011-02-25 13:43:37 -0500: > > > Some clarification on my previous message... > > > > > > After looking at my ftrace log more closely, I can see where Btrfs is > > > trying to release the allocated pages. However, the calculation for > > > the number of dirty_pages is equal to 1 when "copied == 0". > > > > > > So I'm seeing at least two problems: > > > (1) It keeps looping when "copied == 0". > > > (2) One dirty page is not being released on every loop even though > > > "copied == 0" (at least this problem keeps it from being an infinite > > > loop by eventually exhausting reserveable space on the disk). > > > > Hi everyone, > > > > There are actually tow bugs here. First the one that Mitch hit, and a > > second one that still results in bad file_write results with my > > debugging hunks (the first two hunks below) in place. > > > > My patch fixes Mitch's bug by checking for copied == 0 after > > btrfs_copy_from_user and going the correct delalloc accounting. This > > one looks solved, but you'll notice the patch is bigger. > > > > First, I add some random failures to btrfs_copy_from_user() by failing > > everyone once and a while. This was much more reliable than trying to > > use memory pressure than making copy_from_user fail. > > > > If copy_from_user fails and we partially update a page, we end up with a > > page that may go away due to memory pressure. But, btrfs_file_write > > assumes that only the first and last page may have good data that needs > > to be read off the disk. > > > > This patch ditches that code and puts it into prepare_pages instead. > > But I'm still having some errors during long stress.sh runs. Ideas are > > more than welcome, hopefully some other timezones will kick in ideas > > while I sleep. > > At least it doesn't fix the emerge-problem for me. The behavior is now the same > as with 2.6.38-rc3. It needs a 'emerge --oneshot dev-libs/libgcrypt' with no > further interaction to get the emerge-process hang with a svn-process > consuming 100% CPU. I can cancel the emerge-process with ctrl-c but the > spawned svn-process stays and it needs a reboot to get rid of it. I think your problem really is more enospc related. Still working on that as well. But please don't try the patch without removing the debugging hunk at the top (anything that mentions jiffies). -chris ^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [PATCH] btrfs file write debugging patch 2011-02-28 10:13 ` Johannes Hirte 2011-02-28 14:00 ` Chris Mason @ 2011-02-28 16:10 ` Josef Bacik 2011-02-28 16:45 ` Maria Wikström 1 sibling, 1 reply; 43+ messages in thread From: Josef Bacik @ 2011-02-28 16:10 UTC (permalink / raw) To: Johannes Hirte Cc: Chris Mason, Mitch Harder, Maria Wikström, Zhong, Xin, linux-btrfs On Mon, Feb 28, 2011 at 11:13:59AM +0100, Johannes Hirte wrote: > On Monday 28 February 2011 02:46:05 Chris Mason wrote: > > Excerpts from Mitch Harder's message of 2011-02-25 13:43:37 -0500: > > > Some clarification on my previous message... > > > > > > After looking at my ftrace log more closely, I can see where Btrfs is > > > trying to release the allocated pages. However, the calculation for > > > the number of dirty_pages is equal to 1 when "copied == 0". > > > > > > So I'm seeing at least two problems: > > > (1) It keeps looping when "copied == 0". > > > (2) One dirty page is not being released on every loop even though > > > "copied == 0" (at least this problem keeps it from being an infinite > > > loop by eventually exhausting reserveable space on the disk). > > > > Hi everyone, > > > > There are actually tow bugs here. First the one that Mitch hit, and a > > second one that still results in bad file_write results with my > > debugging hunks (the first two hunks below) in place. > > > > My patch fixes Mitch's bug by checking for copied == 0 after > > btrfs_copy_from_user and going the correct delalloc accounting. This > > one looks solved, but you'll notice the patch is bigger. > > > > First, I add some random failures to btrfs_copy_from_user() by failing > > everyone once and a while. This was much more reliable than trying to > > use memory pressure than making copy_from_user fail. > > > > If copy_from_user fails and we partially update a page, we end up with a > > page that may go away due to memory pressure. But, btrfs_file_write > > assumes that only the first and last page may have good data that needs > > to be read off the disk. > > > > This patch ditches that code and puts it into prepare_pages instead. > > But I'm still having some errors during long stress.sh runs. Ideas are > > more than welcome, hopefully some other timezones will kick in ideas > > while I sleep. > > At least it doesn't fix the emerge-problem for me. The behavior is now the same > as with 2.6.38-rc3. It needs a 'emerge --oneshot dev-libs/libgcrypt' with no > further interaction to get the emerge-process hang with a svn-process > consuming 100% CPU. I can cancel the emerge-process with ctrl-c but the > spawned svn-process stays and it needs a reboot to get rid of it. Can you cat /proc/$pid/wchan a few times so we can get an idea of where it's looping? Thanks, Josef ^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [PATCH] btrfs file write debugging patch 2011-02-28 16:10 ` Josef Bacik @ 2011-02-28 16:45 ` Maria Wikström 2011-02-28 17:47 ` Mitch Harder 0 siblings, 1 reply; 43+ messages in thread From: Maria Wikström @ 2011-02-28 16:45 UTC (permalink / raw) To: Josef Bacik Cc: Johannes Hirte, Chris Mason, Mitch Harder, Zhong, Xin, linux-btrfs m=C3=A5n 2011-02-28 klockan 11:10 -0500 skrev Josef Bacik: > On Mon, Feb 28, 2011 at 11:13:59AM +0100, Johannes Hirte wrote: > > On Monday 28 February 2011 02:46:05 Chris Mason wrote: > > > Excerpts from Mitch Harder's message of 2011-02-25 13:43:37 -0500= : > > > > Some clarification on my previous message... > > > >=20 > > > > After looking at my ftrace log more closely, I can see where Bt= rfs is > > > > trying to release the allocated pages. However, the calculatio= n for > > > > the number of dirty_pages is equal to 1 when "copied =3D=3D 0". > > > >=20 > > > > So I'm seeing at least two problems: > > > > (1) It keeps looping when "copied =3D=3D 0". > > > > (2) One dirty page is not being released on every loop even th= ough > > > > "copied =3D=3D 0" (at least this problem keeps it from being an= infinite > > > > loop by eventually exhausting reserveable space on the disk). > > >=20 > > > Hi everyone, > > >=20 > > > There are actually tow bugs here. First the one that Mitch hit, = and a > > > second one that still results in bad file_write results with my > > > debugging hunks (the first two hunks below) in place. > > >=20 > > > My patch fixes Mitch's bug by checking for copied =3D=3D 0 after > > > btrfs_copy_from_user and going the correct delalloc accounting. = This > > > one looks solved, but you'll notice the patch is bigger. > > >=20 > > > First, I add some random failures to btrfs_copy_from_user() by fa= iling > > > everyone once and a while. This was much more reliable than tryi= ng to > > > use memory pressure than making copy_from_user fail. > > >=20 > > > If copy_from_user fails and we partially update a page, we end up= with a > > > page that may go away due to memory pressure. But, btrfs_file_wr= ite > > > assumes that only the first and last page may have good data that= needs > > > to be read off the disk. > > >=20 > > > This patch ditches that code and puts it into prepare_pages inste= ad. > > > But I'm still having some errors during long stress.sh runs. Ide= as are > > > more than welcome, hopefully some other timezones will kick in id= eas > > > while I sleep. > >=20 > > At least it doesn't fix the emerge-problem for me. The behavior is = now the same=20 > > as with 2.6.38-rc3. It needs a 'emerge --oneshot dev-libs/libgcrypt= ' with no=20 > > further interaction to get the emerge-process hang with a svn-proce= ss=20 > > consuming 100% CPU. I can cancel the emerge-process with ctrl-c but= the=20 > > spawned svn-process stays and it needs a reboot to get rid of it.=20 >=20 > Can you cat /proc/$pid/wchan a few times so we can get an idea of whe= re it's > looping? Thanks, >=20 > Josef It behaves the same way here with btrfs-unstable. The output of "cat /proc/$pid/wchan" is 0.=20 // Maria > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs= " in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html >=20 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" = in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [PATCH] btrfs file write debugging patch 2011-02-28 16:45 ` Maria Wikström @ 2011-02-28 17:47 ` Mitch Harder 2011-02-28 20:20 ` Mitch Harder 0 siblings, 1 reply; 43+ messages in thread From: Mitch Harder @ 2011-02-28 17:47 UTC (permalink / raw) To: Maria Wikström Cc: Josef Bacik, Johannes Hirte, Chris Mason, Zhong, Xin, linux-btrfs [-- Attachment #1: Type: text/plain, Size: 3729 bytes --] 2011/2/28 Maria Wikström <maria@ponstudios.se>: > mån 2011-02-28 klockan 11:10 -0500 skrev Josef Bacik: >> On Mon, Feb 28, 2011 at 11:13:59AM +0100, Johannes Hirte wrote: >> > On Monday 28 February 2011 02:46:05 Chris Mason wrote: >> > > Excerpts from Mitch Harder's message of 2011-02-25 13:43:37 -0500: >> > > > Some clarification on my previous message... >> > > > >> > > > After looking at my ftrace log more closely, I can see where Btrfs is >> > > > trying to release the allocated pages. However, the calculation for >> > > > the number of dirty_pages is equal to 1 when "copied == 0". >> > > > >> > > > So I'm seeing at least two problems: >> > > > (1) It keeps looping when "copied == 0". >> > > > (2) One dirty page is not being released on every loop even though >> > > > "copied == 0" (at least this problem keeps it from being an infinite >> > > > loop by eventually exhausting reserveable space on the disk). >> > > >> > > Hi everyone, >> > > >> > > There are actually tow bugs here. First the one that Mitch hit, and a >> > > second one that still results in bad file_write results with my >> > > debugging hunks (the first two hunks below) in place. >> > > >> > > My patch fixes Mitch's bug by checking for copied == 0 after >> > > btrfs_copy_from_user and going the correct delalloc accounting. This >> > > one looks solved, but you'll notice the patch is bigger. >> > > >> > > First, I add some random failures to btrfs_copy_from_user() by failing >> > > everyone once and a while. This was much more reliable than trying to >> > > use memory pressure than making copy_from_user fail. >> > > >> > > If copy_from_user fails and we partially update a page, we end up with a >> > > page that may go away due to memory pressure. But, btrfs_file_write >> > > assumes that only the first and last page may have good data that needs >> > > to be read off the disk. >> > > >> > > This patch ditches that code and puts it into prepare_pages instead. >> > > But I'm still having some errors during long stress.sh runs. Ideas are >> > > more than welcome, hopefully some other timezones will kick in ideas >> > > while I sleep. >> > >> > At least it doesn't fix the emerge-problem for me. The behavior is now the same >> > as with 2.6.38-rc3. It needs a 'emerge --oneshot dev-libs/libgcrypt' with no >> > further interaction to get the emerge-process hang with a svn-process >> > consuming 100% CPU. I can cancel the emerge-process with ctrl-c but the >> > spawned svn-process stays and it needs a reboot to get rid of it. >> >> Can you cat /proc/$pid/wchan a few times so we can get an idea of where it's >> looping? Thanks, >> >> Josef > > It behaves the same way here with btrfs-unstable. > The output of "cat /proc/$pid/wchan" is 0. > > // Maria > >> -- >> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in >> the body of a message to majordomo@vger.kernel.org >> More majordomo info at http://vger.kernel.org/majordomo-info.html >> > > > I've applied the patch at the head of this thread (with the jiffies debugging commented out) and I'm attaching a ftrace using the function_graph tracer when I'm stuck in the loop. I've just snipped out a couple of the loops (the full trace file is quite large, and mostly repititious). I'm going to try to modify file.c with some trace_printk debugging to show the values of several of the relevant variables at various stages. I'm going to try to exit the loop after 256 tries with an EFAULT so I can stop the tracing at that point and capture a trace of the entry into the problem (the ftrace ring buffer fills up too fast for me to capture the entry point). [-- Attachment #2: ftrace-btrfs-file-write-debugging.gz --] [-- Type: application/x-gzip, Size: 2590 bytes --] ^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [PATCH] btrfs file write debugging patch 2011-02-28 17:47 ` Mitch Harder @ 2011-02-28 20:20 ` Mitch Harder 2011-03-01 5:09 ` Mitch Harder ` (2 more replies) 0 siblings, 3 replies; 43+ messages in thread From: Mitch Harder @ 2011-02-28 20:20 UTC (permalink / raw) To: Maria Wikström Cc: Josef Bacik, Johannes Hirte, Chris Mason, Zhong, Xin, linux-btrfs 2011/2/28 Mitch Harder <mitch.harder@sabayonlinux.org>: > 2011/2/28 Maria Wikstr=F6m <maria@ponstudios.se>: >> m=E5n 2011-02-28 klockan 11:10 -0500 skrev Josef Bacik: >>> On Mon, Feb 28, 2011 at 11:13:59AM +0100, Johannes Hirte wrote: >>> > On Monday 28 February 2011 02:46:05 Chris Mason wrote: >>> > > Excerpts from Mitch Harder's message of 2011-02-25 13:43:37 -05= 00: >>> > > > Some clarification on my previous message... >>> > > > >>> > > > After looking at my ftrace log more closely, I can see where = Btrfs is >>> > > > trying to release the allocated pages. =A0However, the calcul= ation for >>> > > > the number of dirty_pages is equal to 1 when "copied =3D=3D 0= ". >>> > > > >>> > > > So I'm seeing at least two problems: >>> > > > (1) =A0It keeps looping when "copied =3D=3D 0". >>> > > > (2) =A0One dirty page is not being released on every loop eve= n though >>> > > > "copied =3D=3D 0" (at least this problem keeps it from being = an infinite >>> > > > loop by eventually exhausting reserveable space on the disk). >>> > > >>> > > Hi everyone, >>> > > >>> > > There are actually tow bugs here. =A0First the one that Mitch h= it, and a >>> > > second one that still results in bad file_write results with my >>> > > debugging hunks (the first two hunks below) in place. >>> > > >>> > > My patch fixes Mitch's bug by checking for copied =3D=3D 0 afte= r >>> > > btrfs_copy_from_user and going the correct delalloc accounting.= =A0This >>> > > one looks solved, but you'll notice the patch is bigger. >>> > > >>> > > First, I add some random failures to btrfs_copy_from_user() by = failing >>> > > everyone once and a while. =A0This was much more reliable than = trying to >>> > > use memory pressure than making copy_from_user fail. >>> > > >>> > > If copy_from_user fails and we partially update a page, we end = up with a >>> > > page that may go away due to memory pressure. =A0But, btrfs_fil= e_write >>> > > assumes that only the first and last page may have good data th= at needs >>> > > to be read off the disk. >>> > > >>> > > This patch ditches that code and puts it into prepare_pages ins= tead. >>> > > But I'm still having some errors during long stress.sh runs. =A0= Ideas are >>> > > more than welcome, hopefully some other timezones will kick in = ideas >>> > > while I sleep. >>> > >>> > At least it doesn't fix the emerge-problem for me. The behavior i= s now the same >>> > as with 2.6.38-rc3. It needs a 'emerge --oneshot dev-libs/libgcry= pt' with no >>> > further interaction to get the emerge-process hang with a svn-pro= cess >>> > consuming 100% CPU. I can cancel the emerge-process with ctrl-c b= ut the >>> > spawned svn-process stays and it needs a reboot to get rid of it. >>> >>> Can you cat /proc/$pid/wchan a few times so we can get an idea of w= here it's >>> looping? =A0Thanks, >>> >>> Josef >> >> It behaves the same way here with btrfs-unstable. >> The output of "cat /proc/$pid/wchan" is 0. >> >> // Maria >> >>> -- >>> To unsubscribe from this list: send the line "unsubscribe linux-btr= fs" in >>> the body of a message to majordomo@vger.kernel.org >>> More majordomo info at =A0http://vger.kernel.org/majordomo-info.htm= l >>> >> >> >> > > I've applied the patch at the head of this thread (with the jiffies > debugging commented out) and I'm attaching a ftrace using the > function_graph tracer when I'm stuck in the loop. =A0I've just snippe= d > out a couple of the loops (the full trace file is quite large, and > mostly repititious). > > I'm going to try to modify file.c with some trace_printk debugging to > show the values of several of the relevant variables at various > stages. > > I'm going to try to exit the loop after 256 tries with an EFAULT so I > can stop the tracing at that point and capture a trace of the entry > into the problem (the ftrace ring buffer fills up too fast for me to > capture the entry point). > As promised, I'm put together a modified file.c with many trace_printk debugging statements to augment the ftrace. The trace is ~128K compressed (about 31,600 lines or 2.6MB uncompressed), so I'm putting it up on my local server instead of attaching. Let me know if it would be more appropriate to send to the list as an attachment. http://dontpanic.dyndns.org/ftrace-btrfs-file-write-debug-v2.gz I preface all my trace_printk comments with "TPK:" to make skipping through the trace easier. The trace contains the trace of about 3 or 4 successful passes through the btrfs_file_aio_write() function to show what a successful trace looks like. The pass through the btrfs_file_aio_write() that breaks begins on line = 1088. I let it loop through the while (iov_iter_count(&i) > 0) {} loop for 256 times when copied=3D=3D0 (otherwise it would loop infinitely). The= n exit out and stop the trace. =46or reference, here's a diff file for the debugging statements I've added to file.c: Let me know if you would like me to re-run this trial with different debugging lines. fs/btrfs/file.c /usr/src/linux/fs/btrfs/file.c --- fs/btrfs/file.c 2011-02-28 10:13:45.000000000 -0600 +++ /usr/src/linux/fs/btrfs/file.c 2011-02-28 13:07:11.000000000 -0600 @@ -53,12 +53,14 @@ int offset =3D pos & (PAGE_CACHE_SIZE - 1); int total_copied =3D 0; + /*************************** if ((jiffies % 10) =3D=3D 0) return 0; if ((jiffies % 25) =3D=3D 0) { write_bytes /=3D 2; } + **************************/ while (write_bytes > 0) { size_t count =3D min_t(size_t, @@ -82,10 +84,13 @@ /* Return to btrfs_file_aio_write to fault page */ if (unlikely(copied =3D=3D 0)) { + trace_printk("TPK: unlikely copied =3D=3D 0 in btrfs_copy_from_user (total_copied=3D%i)\n", + total_copied); break; } if (unlikely(copied < PAGE_CACHE_SIZE - offset)) { + trace_printk("TPK: unlikely copied < PAGE_CACHE_SIZE - offset in btrfs_copy_from_user\n"); offset +=3D copied; } else { pg++; @@ -910,8 +915,13 @@ int will_write; int buffered =3D 0; int copied =3D 0; + int copied_problem =3D 0; + int copied_loop_count =3D 0; + int stop_ftrace =3D 0; int dirty_pages =3D 0; + trace_printk("TPK: Entering btrfs_file_aio_write()\n"); + will_write =3D ((file->f_flags & O_DSYNC) || IS_SYNC(inode) || (file->f_flags & O_DIRECT)); @@ -953,6 +963,7 @@ BTRFS_I(inode)->sequence++; if (unlikely(file->f_flags & O_DIRECT)) { + trace_printk("TPK: transferred to unlikely(file->f_flags \& O_DIRECT= )\n"); num_written =3D generic_file_direct_write(iocb, iov, &nr_segs, pos, ppos, count, ocount); @@ -984,6 +995,8 @@ */ buffered =3D 1; pos +=3D num_written; + trace_printk("TPK: end unlikely(file->f_flags \& O_DIRECT) with num_written=3D%i\n", + num_written); } iov_iter_init(&i, iov, nr_segs, count, num_written); @@ -1003,6 +1016,8 @@ last_index =3D (pos + iov_iter_count(&i)) >> PAGE_CACHE_SHIFT; while (iov_iter_count(&i) > 0) { + trace_printk("TPK: start section while (iov_iter_count() > 0)\n"); + size_t offset =3D pos & (PAGE_CACHE_SIZE - 1); size_t write_bytes =3D min(iov_iter_count(&i), nrptrs * (size_t)PAGE_CACHE_SIZE - @@ -1010,6 +1025,9 @@ size_t num_pages =3D (write_bytes + offset + PAGE_CACHE_SIZE - 1) >> PAGE_CACHE_SHIFT; + trace_printk("TPK: iov_iter_count()=3D%i || nr_segs=3D%lu || nrptrs=3D= %i || offset=3D%lu || write_bytes=3D%lu || num_pages=3D%lu\n", + iov_iter_count(&i), nr_segs, nrptrs, offset, write_bytes, num_pa= ges); + WARN_ON(num_pages > nrptrs); memset(pages, 0, sizeof(struct page *) * nrptrs); @@ -1022,6 +1040,19 @@ goto out; } + if (unlikely(copied_problem)) { + copied_loop_count++; + trace_printk("TPK: copied problem(%i)\n", + copied_loop_count); + /* Give up if we've already tried 256 times */ + if (copied_loop_count > 256) { + ret =3D -EFAULT; + stop_ftrace =3D 1; + trace_printk("TPK: copied loop count exceeded, returning EFAULT...= =2E\n"); + goto out; + } + } + ret =3D btrfs_delalloc_reserve_space(inode, num_pages << PAGE_CACHE_SHIFT); if (ret) @@ -1045,6 +1076,10 @@ PAGE_CACHE_SIZE - 1) >> PAGE_CACHE_SHIFT; + if (copied =3D=3D 0) { + copied_problem =3D 1; + } + if (num_pages > dirty_pages) { if (copied > 0) atomic_inc( @@ -1080,11 +1115,19 @@ num_written +=3D copied; cond_resched(); + trace_printk("TPK: end section while (iov_iter_count() > 0)\n"); + trace_printk(" copied=3D%i || dirty_pages=3D%i || num_written=3D%i |= | ending iov_iter_count=3D%i\n", + copied, dirty_pages, num_written, iov_iter_count(&i) ); } out: + trace_printk("TPK: Reached: out:\n"); + mutex_unlock(&inode->i_mutex); - if (ret) + if (ret) { err =3D ret; + trace_printk("TPK: ret,err had value %i when out: was reached (num_written: %i)\n", + ret, num_written); + } kfree(pages); *ppos =3D pos; @@ -1140,6 +1183,19 @@ } done: current->backing_dev_info =3D NULL; + if (ret) { + trace_printk("TPK: btrfs_file_aio_write exiting with non-zero ret value (%i)\n", ret); + trace_printk("TPK: Returning num_written (%i) ? num_written (%i) : err (%i) =3D %i\n", + num_written, num_written, err, num_written ? num_written : err); + } else { + trace_printk("TPK: btrfs_file_aio_write exiting normally with (%i)", + num_written ? num_written : err); + } +=09 + if (unlikely(stop_ftrace)) { + tracing_off(); + } + return num_written ? num_written : err; } -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" = in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [PATCH] btrfs file write debugging patch 2011-02-28 20:20 ` Mitch Harder @ 2011-03-01 5:09 ` Mitch Harder 2011-03-01 10:14 ` Zhong, Xin 2011-03-01 21:56 ` Piotr Szymaniak 2 siblings, 0 replies; 43+ messages in thread From: Mitch Harder @ 2011-03-01 5:09 UTC (permalink / raw) To: Maria Wikström Cc: Josef Bacik, Johannes Hirte, Chris Mason, Zhong, Xin, linux-btrfs 2011/2/28 Mitch Harder <mitch.harder@sabayonlinux.org>: > 2011/2/28 Mitch Harder <mitch.harder@sabayonlinux.org>: >> 2011/2/28 Maria Wikstr=F6m <maria@ponstudios.se>: >>> m=E5n 2011-02-28 klockan 11:10 -0500 skrev Josef Bacik: >>>> On Mon, Feb 28, 2011 at 11:13:59AM +0100, Johannes Hirte wrote: >>>> > On Monday 28 February 2011 02:46:05 Chris Mason wrote: >>>> > > Excerpts from Mitch Harder's message of 2011-02-25 13:43:37 -0= 500: >>>> > > > Some clarification on my previous message... >>>> > > > >>>> > > > After looking at my ftrace log more closely, I can see where= Btrfs is >>>> > > > trying to release the allocated pages. =A0However, the calcu= lation for >>>> > > > the number of dirty_pages is equal to 1 when "copied =3D=3D = 0". >>>> > > > >>>> > > > So I'm seeing at least two problems: >>>> > > > (1) =A0It keeps looping when "copied =3D=3D 0". >>>> > > > (2) =A0One dirty page is not being released on every loop ev= en though >>>> > > > "copied =3D=3D 0" (at least this problem keeps it from being= an infinite >>>> > > > loop by eventually exhausting reserveable space on the disk)= =2E >>>> > > >>>> > > Hi everyone, >>>> > > >>>> > > There are actually tow bugs here. =A0First the one that Mitch = hit, and a >>>> > > second one that still results in bad file_write results with m= y >>>> > > debugging hunks (the first two hunks below) in place. >>>> > > >>>> > > My patch fixes Mitch's bug by checking for copied =3D=3D 0 aft= er >>>> > > btrfs_copy_from_user and going the correct delalloc accounting= =2E =A0This >>>> > > one looks solved, but you'll notice the patch is bigger. >>>> > > >>>> > > First, I add some random failures to btrfs_copy_from_user() by= failing >>>> > > everyone once and a while. =A0This was much more reliable than= trying to >>>> > > use memory pressure than making copy_from_user fail. >>>> > > >>>> > > If copy_from_user fails and we partially update a page, we end= up with a >>>> > > page that may go away due to memory pressure. =A0But, btrfs_fi= le_write >>>> > > assumes that only the first and last page may have good data t= hat needs >>>> > > to be read off the disk. >>>> > > >>>> > > This patch ditches that code and puts it into prepare_pages in= stead. >>>> > > But I'm still having some errors during long stress.sh runs. =A0= Ideas are >>>> > > more than welcome, hopefully some other timezones will kick in= ideas >>>> > > while I sleep. >>>> > >>>> > At least it doesn't fix the emerge-problem for me. The behavior = is now the same >>>> > as with 2.6.38-rc3. It needs a 'emerge --oneshot dev-libs/libgcr= ypt' with no >>>> > further interaction to get the emerge-process hang with a svn-pr= ocess >>>> > consuming 100% CPU. I can cancel the emerge-process with ctrl-c = but the >>>> > spawned svn-process stays and it needs a reboot to get rid of it= =2E >>>> >>>> Can you cat /proc/$pid/wchan a few times so we can get an idea of = where it's >>>> looping? =A0Thanks, >>>> >>>> Josef >>> >>> It behaves the same way here with btrfs-unstable. >>> The output of "cat /proc/$pid/wchan" is 0. >>> >>> // Maria >>> >>>> -- >>>> To unsubscribe from this list: send the line "unsubscribe linux-bt= rfs" in >>>> the body of a message to majordomo@vger.kernel.org >>>> More majordomo info at =A0http://vger.kernel.org/majordomo-info.ht= ml >>>> >>> >>> >>> >> >> I've applied the patch at the head of this thread (with the jiffies >> debugging commented out) and I'm attaching a ftrace using the >> function_graph tracer when I'm stuck in the loop. =A0I've just snipp= ed >> out a couple of the loops (the full trace file is quite large, and >> mostly repititious). >> >> I'm going to try to modify file.c with some trace_printk debugging t= o >> show the values of several of the relevant variables at various >> stages. >> >> I'm going to try to exit the loop after 256 tries with an EFAULT so = I >> can stop the tracing at that point and capture a trace of the entry >> into the problem (the ftrace ring buffer fills up too fast for me to >> capture the entry point). >> > > As promised, I'm put together a modified file.c with many trace_print= k > debugging statements to augment the ftrace. > > The trace is ~128K compressed (about 31,600 lines or 2.6MB > uncompressed), so I'm putting it up on my local server instead of > attaching. =A0Let me know if it would be more appropriate to send to = the > list as an attachment. > > http://dontpanic.dyndns.org/ftrace-btrfs-file-write-debug-v2.gz > > I preface all my trace_printk comments with "TPK:" to make skipping > through the trace easier. > > The trace contains the trace of about 3 or 4 successful passes throug= h > the btrfs_file_aio_write() function to show what a successful trace > looks like. > > The pass through the btrfs_file_aio_write() that breaks begins on lin= e 1088. > > I let it loop through the while (iov_iter_count(&i) > 0) {} loop for > 256 times when copied=3D=3D0 (otherwise it would loop infinitely). =A0= Then > exit out and stop the trace. > > For reference, here's a diff file for the debugging statements I've > added to file.c: > > Let me know if you would like me to re-run this trial with different > debugging lines. > > =A0fs/btrfs/file.c /usr/src/linux/fs/btrfs/file.c > --- fs/btrfs/file.c =A0 =A0 2011-02-28 10:13:45.000000000 -0600 > +++ /usr/src/linux/fs/btrfs/file.c =A0 =A0 =A02011-02-28 13:07:11.000= 000000 -0600 > @@ -53,12 +53,14 @@ > =A0 =A0 =A0 =A0int offset =3D pos & (PAGE_CACHE_SIZE - 1); > =A0 =A0 =A0 =A0int total_copied =3D 0; > > + =A0 =A0 =A0 /*************************** > =A0 =A0 =A0 =A0if ((jiffies % 10) =3D=3D 0) > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0return 0; > > =A0 =A0 =A0 =A0if ((jiffies % 25) =3D=3D 0) { > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0write_bytes /=3D 2; > =A0 =A0 =A0 =A0} > + =A0 =A0 =A0 **************************/ > > =A0 =A0 =A0 =A0while (write_bytes > 0) { > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0size_t count =3D min_t(size_t, > @@ -82,10 +84,13 @@ > > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0/* Return to btrfs_file_aio_write to f= ault page */ > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0if (unlikely(copied =3D=3D 0)) { > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 trace_printk("TPK: unli= kely copied =3D=3D 0 in btrfs_copy_from_user > (total_copied=3D%i)\n", > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0= =A0total_copied); > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0break; > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0} > > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0if (unlikely(copied < PAGE_CACHE_SIZE = - offset)) { > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 trace_printk("TPK: unli= kely copied < PAGE_CACHE_SIZE - offset in > btrfs_copy_from_user\n"); > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0offset +=3D copied; > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0} else { > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0pg++; > @@ -910,8 +915,13 @@ > =A0 =A0 =A0 =A0int will_write; > =A0 =A0 =A0 =A0int buffered =3D 0; > =A0 =A0 =A0 =A0int copied =3D 0; > + =A0 =A0 =A0 int copied_problem =3D 0; > + =A0 =A0 =A0 int copied_loop_count =3D 0; > + =A0 =A0 =A0 int stop_ftrace =3D 0; > =A0 =A0 =A0 =A0int dirty_pages =3D 0; > > + =A0 =A0 =A0 trace_printk("TPK: Entering btrfs_file_aio_write()\n"); > + > =A0 =A0 =A0 =A0will_write =3D ((file->f_flags & O_DSYNC) || IS_SYNC(i= node) || > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0(file->f_flags & O_DIRECT)= ); > > @@ -953,6 +963,7 @@ > =A0 =A0 =A0 =A0BTRFS_I(inode)->sequence++; > > =A0 =A0 =A0 =A0if (unlikely(file->f_flags & O_DIRECT)) { > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 trace_printk("TPK: transferred to unlik= ely(file->f_flags \& O_DIRECT)\n"); > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0num_written =3D generic_file_direct_wr= ite(iocb, iov, &nr_segs, > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0= =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0pos, ppos, count, > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0= =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0ocount); > @@ -984,6 +995,8 @@ > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 */ > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0buffered =3D 1; > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0pos +=3D num_written; > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 trace_printk("TPK: end unlikely(file->f= _flags \& O_DIRECT) with > num_written=3D%i\n", > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0num_written)= ; > =A0 =A0 =A0 =A0} > > =A0 =A0 =A0 =A0iov_iter_init(&i, iov, nr_segs, count, num_written); > @@ -1003,6 +1016,8 @@ > =A0 =A0 =A0 =A0last_index =3D (pos + iov_iter_count(&i)) >> PAGE_CACH= E_SHIFT; > > =A0 =A0 =A0 =A0while (iov_iter_count(&i) > 0) { > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 trace_printk("TPK: start section while = (iov_iter_count() > 0)\n"); > + > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0size_t offset =3D pos & (PAGE_CACHE_SI= ZE - 1); > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0size_t write_bytes =3D min(iov_iter_co= unt(&i), > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0= =A0 =A0 nrptrs * (size_t)PAGE_CACHE_SIZE - > @@ -1010,6 +1025,9 @@ > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0size_t num_pages =3D (write_bytes + of= fset + > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0= PAGE_CACHE_SIZE - 1) >> PAGE_CACHE_SHIFT; > > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 trace_printk("TPK: iov_iter_count()=3D%= i || nr_segs=3D%lu || nrptrs=3D%i > || offset=3D%lu || write_bytes=3D%lu || num_pages=3D%lu\n", > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0= =A0 =A0 =A0 =A0 =A0 =A0 iov_iter_count(&i), nr_segs, nrptrs, offset, w= rite_bytes, num_pages); > + > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0WARN_ON(num_pages > nrptrs); > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0memset(pages, 0, sizeof(struct page *)= * nrptrs); > > @@ -1022,6 +1040,19 @@ > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0goto out; > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0} > > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 if (unlikely(copied_problem)) { > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 copied_loop_count++; > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 trace_printk("TPK: copi= ed problem(%i)\n", > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0= =A0copied_loop_count); > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 /* Give up if we've alr= eady tried 256 times */ > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 if (copied_loop_count >= 256) { > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 ret =3D= -EFAULT; > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 stop_ft= race =3D 1; > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 trace_p= rintk("TPK: copied loop count exceeded, returning EFAULT....\n"); > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 goto ou= t; > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 } > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 } > + > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0ret =3D btrfs_delalloc_reserve_space(i= node, > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0= =A0 =A0num_pages << PAGE_CACHE_SHIFT); > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0if (ret) > @@ -1045,6 +1076,10 @@ > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0= =A0 PAGE_CACHE_SIZE - 1) >> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0= =A0 PAGE_CACHE_SHIFT; > > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 if (copied =3D=3D 0) { > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 copied_problem =3D 1; > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 } > + > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0if (num_pages > dirty_pages) { > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0if (copied > 0) > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0atomic= _inc( > @@ -1080,11 +1115,19 @@ > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0num_written +=3D copied; > > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0cond_resched(); > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 trace_printk("TPK: end section while (i= ov_iter_count() > 0)\n"); > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 trace_printk(" copied=3D%i || dirty_pag= es=3D%i || num_written=3D%i || > ending iov_iter_count=3D%i\n", > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0= =A0 =A0 =A0 =A0 =A0 =A0 copied, dirty_pages, num_written, iov_iter_cou= nt(&i) ); > =A0 =A0 =A0 =A0} > =A0out: > + =A0 =A0 =A0 trace_printk("TPK: Reached: out:\n"); > + > =A0 =A0 =A0 =A0mutex_unlock(&inode->i_mutex); > - =A0 =A0 =A0 if (ret) > + =A0 =A0 =A0 if (ret) { > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0err =3D ret; > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 trace_printk("TPK: ret,err had value %i= when out: was reached > (num_written: %i)\n", > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0ret, num_wri= tten); > + =A0 =A0 =A0 } > > =A0 =A0 =A0 =A0kfree(pages); > =A0 =A0 =A0 =A0*ppos =3D pos; > @@ -1140,6 +1183,19 @@ > =A0 =A0 =A0 =A0} > =A0done: > =A0 =A0 =A0 =A0current->backing_dev_info =3D NULL; > + =A0 =A0 =A0 if (ret) { > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 trace_printk("TPK: btrfs_file_aio_write= exiting with non-zero ret > value (%i)\n", ret); > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 trace_printk("TPK: Returning num_writte= n (%i) ? num_written (%i) : > err (%i) =3D %i\n", > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 num_wri= tten, num_written, err, num_written ? num_written : err); > + =A0 =A0 =A0 } else { > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 trace_printk("TPK: btrfs_file_aio_write= exiting normally with (%i)", > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0num_written = ? num_written : err); > + =A0 =A0 =A0 } > + > + =A0 =A0 =A0 if (unlikely(stop_ftrace)) { > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 tracing_off(); > + =A0 =A0 =A0 } > + > =A0 =A0 =A0 =A0return num_written ? num_written : err; > =A0} > I'm developing a hypothesis that something is going wrong when Btrfs is trying to lock multiple pages. Page faults are being encountered at the same spot, over and over (BTW, I ran a memcheck to rule that possibility out). If I scan through my traces, it looks like most calls to write are submitted 1 block (4096 bytes) at a time, at the most (also many are < 4096 bytes). The portion that is causing a problem is trying to write 12288 bytes (3k). For some reason, the first 24 bytes are written, and the remainder of the loop is spent on the 12264 that are remaining. But page faults are encountered on each loop, and no more bytes are copied. I modified file.c to cut back on the scope of the attempted write to smaller chunks. The following patch "fixes" my problem, and this build completes without error. I'm not submitting this patch as a solution. It's clearly papering over a deeper issue. However, I think it gives insight into the problem. I wrote the patch to allow for sequentially smaller blocks if failures continue. But the block cleared once I limited the scope to 4096 bytes. It never needed to try smaller data segments. As hoped, limiting the scope of the write allowed the 12264 bytes to be written in the next three subsequent loops after lowering the scope of the write. It was interesting to note that cutting the scope to 4096 didn't guarantee that you were limiting the write to one block. There was usually some overlap, and 2 dirty block needed to be written. But I'm still suspicious that there is a mismatch somewhere when trying to lock multiple blocks. Here's the debugging patch I constructed which actually allowed the build to complete without error. This was applied (for testing purposes) on top of the previous debugging patch. As noted earlier, it never went lower than a 4096 byte window for write_bytes. --- file.c.file-write-patch-v1 2011-02-28 13:06:37.000000000 -0600 +++ file.c 2011-02-28 19:23:21.000000000 -0600 @@ -908,6 +908,7 @@ ssize_t err =3D 0; size_t count; size_t ocount; + size_t write_bytes =3D 0; int ret =3D 0; int nrptrs; unsigned long first_index; @@ -963,7 +964,7 @@ BTRFS_I(inode)->sequence++; if (unlikely(file->f_flags & O_DIRECT)) { - trace_printk("TPK: transferred to unlikely(file->f_flags \& O_DIRECT= )\n"); + trace_printk("TPK: transferred to unlikely(file->f_flags & O_DIRECT)= \n"); num_written =3D generic_file_direct_write(iocb, iov, &nr_segs, pos, ppos, count, ocount); @@ -995,7 +996,7 @@ */ buffered =3D 1; pos +=3D num_written; - trace_printk("TPK: end unlikely(file->f_flags \& O_DIRECT) with num_written=3D%i\n", + trace_printk("TPK: end unlikely(file->f_flags & O_DIRECT) with num_written=3D%i\n", num_written); } @@ -1019,9 +1020,20 @@ trace_printk("TPK: start section while (iov_iter_count() > 0)\n"); size_t offset =3D pos & (PAGE_CACHE_SIZE - 1); - size_t write_bytes =3D min(iov_iter_count(&i), - nrptrs * (size_t)PAGE_CACHE_SIZE - - offset); + write_bytes =3D min(iov_iter_count(&i), + nrptrs * (size_t)PAGE_CACHE_SIZE - + offset); + if (unlikely(copied_problem)) { + if (copied_loop_count > 128) { + write_bytes =3D min(write_bytes, 4096); + } else if (copied_loop_count > 160) { + write_bytes =3D min(write_bytes, 2048); + } else if (copied_loop_count > 192) { + write_bytes =3D min(write_bytes, 1024); + } else if (copied_loop_count > 224) { + write_bytes =3D min(write_bytes, 512); + } + } size_t num_pages =3D (write_bytes + offset + PAGE_CACHE_SIZE - 1) >> PAGE_CACHE_SHIFT; -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" = in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 43+ messages in thread
* RE: [PATCH] btrfs file write debugging patch 2011-02-28 20:20 ` Mitch Harder 2011-03-01 5:09 ` Mitch Harder @ 2011-03-01 10:14 ` Zhong, Xin 2011-03-01 11:56 ` Zhong, Xin 2011-03-01 14:51 ` Mitch Harder 2011-03-01 21:56 ` Piotr Szymaniak 2 siblings, 2 replies; 43+ messages in thread From: Zhong, Xin @ 2011-03-01 10:14 UTC (permalink / raw) To: Mitch Harder, Maria Wikström Cc: Josef Bacik, Johannes Hirte, Chris Mason, linux-btrfs Is your system running out of memory or is there any other thread like = flush-btrfs competing for the same page? I can only see one process in your ftrace log. You may need to trace al= l btrfs.ko function calls instead of a single process. Thanks! -----Original Message----- =46rom: linux-btrfs-owner@vger.kernel.org [mailto:linux-btrfs-owner@vge= r.kernel.org] On Behalf Of Mitch Harder Sent: Tuesday, March 01, 2011 4:20 AM To: Maria Wikstr=F6m Cc: Josef Bacik; Johannes Hirte; Chris Mason; Zhong, Xin; linux-btrfs@v= ger.kernel.org Subject: Re: [PATCH] btrfs file write debugging patch 2011/2/28 Mitch Harder <mitch.harder@sabayonlinux.org>: > 2011/2/28 Maria Wikstr=F6m <maria@ponstudios.se>: >> m=E5n 2011-02-28 klockan 11:10 -0500 skrev Josef Bacik: >>> On Mon, Feb 28, 2011 at 11:13:59AM +0100, Johannes Hirte wrote: >>> > On Monday 28 February 2011 02:46:05 Chris Mason wrote: >>> > > Excerpts from Mitch Harder's message of 2011-02-25 13:43:37 -05= 00: >>> > > > Some clarification on my previous message... >>> > > > >>> > > > After looking at my ftrace log more closely, I can see where = Btrfs is >>> > > > trying to release the allocated pages. =A0However, the calcul= ation for >>> > > > the number of dirty_pages is equal to 1 when "copied =3D=3D 0= ". >>> > > > >>> > > > So I'm seeing at least two problems: >>> > > > (1) =A0It keeps looping when "copied =3D=3D 0". >>> > > > (2) =A0One dirty page is not being released on every loop eve= n though >>> > > > "copied =3D=3D 0" (at least this problem keeps it from being = an infinite >>> > > > loop by eventually exhausting reserveable space on the disk). >>> > > >>> > > Hi everyone, >>> > > >>> > > There are actually tow bugs here. =A0First the one that Mitch h= it, and a >>> > > second one that still results in bad file_write results with my >>> > > debugging hunks (the first two hunks below) in place. >>> > > >>> > > My patch fixes Mitch's bug by checking for copied =3D=3D 0 afte= r >>> > > btrfs_copy_from_user and going the correct delalloc accounting.= =A0This >>> > > one looks solved, but you'll notice the patch is bigger. >>> > > >>> > > First, I add some random failures to btrfs_copy_from_user() by = failing >>> > > everyone once and a while. =A0This was much more reliable than = trying to >>> > > use memory pressure than making copy_from_user fail. >>> > > >>> > > If copy_from_user fails and we partially update a page, we end = up with a >>> > > page that may go away due to memory pressure. =A0But, btrfs_fil= e_write >>> > > assumes that only the first and last page may have good data th= at needs >>> > > to be read off the disk. >>> > > >>> > > This patch ditches that code and puts it into prepare_pages ins= tead. >>> > > But I'm still having some errors during long stress.sh runs. =A0= Ideas are >>> > > more than welcome, hopefully some other timezones will kick in = ideas >>> > > while I sleep. >>> > >>> > At least it doesn't fix the emerge-problem for me. The behavior i= s now the same >>> > as with 2.6.38-rc3. It needs a 'emerge --oneshot dev-libs/libgcry= pt' with no >>> > further interaction to get the emerge-process hang with a svn-pro= cess >>> > consuming 100% CPU. I can cancel the emerge-process with ctrl-c b= ut the >>> > spawned svn-process stays and it needs a reboot to get rid of it. >>> >>> Can you cat /proc/$pid/wchan a few times so we can get an idea of w= here it's >>> looping? =A0Thanks, >>> >>> Josef >> >> It behaves the same way here with btrfs-unstable. >> The output of "cat /proc/$pid/wchan" is 0. >> >> // Maria >> >>> -- >>> To unsubscribe from this list: send the line "unsubscribe linux-btr= fs" in >>> the body of a message to majordomo@vger.kernel.org >>> More majordomo info at =A0http://vger.kernel.org/majordomo-info.htm= l >>> >> >> >> > > I've applied the patch at the head of this thread (with the jiffies > debugging commented out) and I'm attaching a ftrace using the > function_graph tracer when I'm stuck in the loop. =A0I've just snippe= d > out a couple of the loops (the full trace file is quite large, and > mostly repititious). > > I'm going to try to modify file.c with some trace_printk debugging to > show the values of several of the relevant variables at various > stages. > > I'm going to try to exit the loop after 256 tries with an EFAULT so I > can stop the tracing at that point and capture a trace of the entry > into the problem (the ftrace ring buffer fills up too fast for me to > capture the entry point). > As promised, I'm put together a modified file.c with many trace_printk debugging statements to augment the ftrace. The trace is ~128K compressed (about 31,600 lines or 2.6MB uncompressed), so I'm putting it up on my local server instead of attaching. Let me know if it would be more appropriate to send to the list as an attachment. http://dontpanic.dyndns.org/ftrace-btrfs-file-write-debug-v2.gz I preface all my trace_printk comments with "TPK:" to make skipping through the trace easier. The trace contains the trace of about 3 or 4 successful passes through the btrfs_file_aio_write() function to show what a successful trace looks like. The pass through the btrfs_file_aio_write() that breaks begins on line = 1088. I let it loop through the while (iov_iter_count(&i) > 0) {} loop for 256 times when copied=3D=3D0 (otherwise it would loop infinitely). The= n exit out and stop the trace. =46or reference, here's a diff file for the debugging statements I've added to file.c: Let me know if you would like me to re-run this trial with different debugging lines. fs/btrfs/file.c /usr/src/linux/fs/btrfs/file.c --- fs/btrfs/file.c 2011-02-28 10:13:45.000000000 -0600 +++ /usr/src/linux/fs/btrfs/file.c 2011-02-28 13:07:11.000000000 -0600 @@ -53,12 +53,14 @@ int offset =3D pos & (PAGE_CACHE_SIZE - 1); int total_copied =3D 0; + /*************************** if ((jiffies % 10) =3D=3D 0) return 0; if ((jiffies % 25) =3D=3D 0) { write_bytes /=3D 2; } + **************************/ while (write_bytes > 0) { size_t count =3D min_t(size_t, @@ -82,10 +84,13 @@ /* Return to btrfs_file_aio_write to fault page */ if (unlikely(copied =3D=3D 0)) { + trace_printk("TPK: unlikely copied =3D=3D 0 in btrfs_copy_from_user (total_copied=3D%i)\n", + total_copied); break; } if (unlikely(copied < PAGE_CACHE_SIZE - offset)) { + trace_printk("TPK: unlikely copied < PAGE_CACHE_SIZE - offset in btrfs_copy_from_user\n"); offset +=3D copied; } else { pg++; @@ -910,8 +915,13 @@ int will_write; int buffered =3D 0; int copied =3D 0; + int copied_problem =3D 0; + int copied_loop_count =3D 0; + int stop_ftrace =3D 0; int dirty_pages =3D 0; + trace_printk("TPK: Entering btrfs_file_aio_write()\n"); + will_write =3D ((file->f_flags & O_DSYNC) || IS_SYNC(inode) || (file->f_flags & O_DIRECT)); @@ -953,6 +963,7 @@ BTRFS_I(inode)->sequence++; if (unlikely(file->f_flags & O_DIRECT)) { + trace_printk("TPK: transferred to unlikely(file->f_flags \& O_DIRECT= )\n"); num_written =3D generic_file_direct_write(iocb, iov, &nr_segs, pos, ppos, count, ocount); @@ -984,6 +995,8 @@ */ buffered =3D 1; pos +=3D num_written; + trace_printk("TPK: end unlikely(file->f_flags \& O_DIRECT) with num_written=3D%i\n", + num_written); } iov_iter_init(&i, iov, nr_segs, count, num_written); @@ -1003,6 +1016,8 @@ last_index =3D (pos + iov_iter_count(&i)) >> PAGE_CACHE_SHIFT; while (iov_iter_count(&i) > 0) { + trace_printk("TPK: start section while (iov_iter_count() > 0)\n"); + size_t offset =3D pos & (PAGE_CACHE_SIZE - 1); size_t write_bytes =3D min(iov_iter_count(&i), nrptrs * (size_t)PAGE_CACHE_SIZE - @@ -1010,6 +1025,9 @@ size_t num_pages =3D (write_bytes + offset + PAGE_CACHE_SIZE - 1) >> PAGE_CACHE_SHIFT; + trace_printk("TPK: iov_iter_count()=3D%i || nr_segs=3D%lu || nrptrs=3D= %i || offset=3D%lu || write_bytes=3D%lu || num_pages=3D%lu\n", + iov_iter_count(&i), nr_segs, nrptrs, offset, write_bytes, num_pa= ges); + WARN_ON(num_pages > nrptrs); memset(pages, 0, sizeof(struct page *) * nrptrs); @@ -1022,6 +1040,19 @@ goto out; } + if (unlikely(copied_problem)) { + copied_loop_count++; + trace_printk("TPK: copied problem(%i)\n", + copied_loop_count); + /* Give up if we've already tried 256 times */ + if (copied_loop_count > 256) { + ret =3D -EFAULT; + stop_ftrace =3D 1; + trace_printk("TPK: copied loop count exceeded, returning EFAULT...= =2E\n"); + goto out; + } + } + ret =3D btrfs_delalloc_reserve_space(inode, num_pages << PAGE_CACHE_SHIFT); if (ret) @@ -1045,6 +1076,10 @@ PAGE_CACHE_SIZE - 1) >> PAGE_CACHE_SHIFT; + if (copied =3D=3D 0) { + copied_problem =3D 1; + } + if (num_pages > dirty_pages) { if (copied > 0) atomic_inc( @@ -1080,11 +1115,19 @@ num_written +=3D copied; cond_resched(); + trace_printk("TPK: end section while (iov_iter_count() > 0)\n"); + trace_printk(" copied=3D%i || dirty_pages=3D%i || num_written=3D%i |= | ending iov_iter_count=3D%i\n", + copied, dirty_pages, num_written, iov_iter_count(&i) ); } out: + trace_printk("TPK: Reached: out:\n"); + mutex_unlock(&inode->i_mutex); - if (ret) + if (ret) { err =3D ret; + trace_printk("TPK: ret,err had value %i when out: was reached (num_written: %i)\n", + ret, num_written); + } kfree(pages); *ppos =3D pos; @@ -1140,6 +1183,19 @@ } done: current->backing_dev_info =3D NULL; + if (ret) { + trace_printk("TPK: btrfs_file_aio_write exiting with non-zero ret value (%i)\n", ret); + trace_printk("TPK: Returning num_written (%i) ? num_written (%i) : err (%i) =3D %i\n", + num_written, num_written, err, num_written ? num_written : err); + } else { + trace_printk("TPK: btrfs_file_aio_write exiting normally with (%i)", + num_written ? num_written : err); + } +=09 + if (unlikely(stop_ftrace)) { + tracing_off(); + } + return num_written ? num_written : err; } -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" = in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" = in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 43+ messages in thread
* RE: [PATCH] btrfs file write debugging patch 2011-03-01 10:14 ` Zhong, Xin @ 2011-03-01 11:56 ` Zhong, Xin 2011-03-01 14:54 ` Mitch Harder 2011-03-01 14:51 ` Mitch Harder 1 sibling, 1 reply; 43+ messages in thread From: Zhong, Xin @ 2011-03-01 11:56 UTC (permalink / raw) To: Zhong, Xin, Mitch Harder, Maria Wikström Cc: Josef Bacik, Johannes Hirte, Chris Mason, linux-btrfs Hi Mitch, I suspect there's a lock contention between flush-btrfs (lock_dellalloc= _pages) and btrfs_file_aio_write. However I can not recreate it locally= =2E Could you please try below patch? Thanks! diff --git a/fs/btrfs/file.c b/fs/btrfs/file.c index 65338a1..b9d0929 1= 00644 --- a/fs/btrfs/file.c +++ b/fs/btrfs/file.c @@ -1007,17 +1007,16 @@ static ssize_t btrfs_file_aio_write(struct kioc= b *iocb, goto out; } =20 - ret =3D btrfs_delalloc_reserve_space(inode, - num_pages << PAGE_CACHE_SHIFT); - if (ret) - goto out; - ret =3D prepare_pages(root, file, pages, num_pages, pos, first_index, last_index, write_bytes); - if (ret) { - btrfs_delalloc_release_space(inode, + if (ret) + goto out; + =09 + ret =3D btrfs_delalloc_reserve_space(inode, num_pages << PAGE_CACHE_SHIFT); + if (ret) { + btrfs_drop_pages(pages, num_pages); goto out; } -----Original Message----- =46rom: linux-btrfs-owner@vger.kernel.org [mailto:linux-btrfs-owner@vge= r.kernel.org] On Behalf Of Zhong, Xin Sent: Tuesday, March 01, 2011 6:15 PM To: Mitch Harder; Maria Wikstr=F6m Cc: Josef Bacik; Johannes Hirte; Chris Mason; linux-btrfs@vger.kernel.o= rg Subject: RE: [PATCH] btrfs file write debugging patch Is your system running out of memory or is there any other thread like = flush-btrfs competing for the same page? I can only see one process in your ftrace log. You may need to trace al= l btrfs.ko function calls instead of a single process. Thanks! -----Original Message----- =46rom: linux-btrfs-owner@vger.kernel.org [mailto:linux-btrfs-owner@vge= r.kernel.org] On Behalf Of Mitch Harder Sent: Tuesday, March 01, 2011 4:20 AM To: Maria Wikstr=F6m Cc: Josef Bacik; Johannes Hirte; Chris Mason; Zhong, Xin; linux-btrfs@v= ger.kernel.org Subject: Re: [PATCH] btrfs file write debugging patch 2011/2/28 Mitch Harder <mitch.harder@sabayonlinux.org>: > 2011/2/28 Maria Wikstr=F6m <maria@ponstudios.se>: >> m=E5n 2011-02-28 klockan 11:10 -0500 skrev Josef Bacik: >>> On Mon, Feb 28, 2011 at 11:13:59AM +0100, Johannes Hirte wrote: >>> > On Monday 28 February 2011 02:46:05 Chris Mason wrote: >>> > > Excerpts from Mitch Harder's message of 2011-02-25 13:43:37 -05= 00: >>> > > > Some clarification on my previous message... >>> > > > >>> > > > After looking at my ftrace log more closely, I can see where = Btrfs is >>> > > > trying to release the allocated pages. =A0However, the calcul= ation for >>> > > > the number of dirty_pages is equal to 1 when "copied =3D=3D 0= ". >>> > > > >>> > > > So I'm seeing at least two problems: >>> > > > (1) =A0It keeps looping when "copied =3D=3D 0". >>> > > > (2) =A0One dirty page is not being released on every loop eve= n though >>> > > > "copied =3D=3D 0" (at least this problem keeps it from being = an infinite >>> > > > loop by eventually exhausting reserveable space on the disk). >>> > > >>> > > Hi everyone, >>> > > >>> > > There are actually tow bugs here. =A0First the one that Mitch h= it, and a >>> > > second one that still results in bad file_write results with my >>> > > debugging hunks (the first two hunks below) in place. >>> > > >>> > > My patch fixes Mitch's bug by checking for copied =3D=3D 0 afte= r >>> > > btrfs_copy_from_user and going the correct delalloc accounting.= =A0This >>> > > one looks solved, but you'll notice the patch is bigger. >>> > > >>> > > First, I add some random failures to btrfs_copy_from_user() by = failing >>> > > everyone once and a while. =A0This was much more reliable than = trying to >>> > > use memory pressure than making copy_from_user fail. >>> > > >>> > > If copy_from_user fails and we partially update a page, we end = up with a >>> > > page that may go away due to memory pressure. =A0But, btrfs_fil= e_write >>> > > assumes that only the first and last page may have good data th= at needs >>> > > to be read off the disk. >>> > > >>> > > This patch ditches that code and puts it into prepare_pages ins= tead. >>> > > But I'm still having some errors during long stress.sh runs. =A0= Ideas are >>> > > more than welcome, hopefully some other timezones will kick in = ideas >>> > > while I sleep. >>> > >>> > At least it doesn't fix the emerge-problem for me. The behavior i= s now the same >>> > as with 2.6.38-rc3. It needs a 'emerge --oneshot dev-libs/libgcry= pt' with no >>> > further interaction to get the emerge-process hang with a svn-pro= cess >>> > consuming 100% CPU. I can cancel the emerge-process with ctrl-c b= ut the >>> > spawned svn-process stays and it needs a reboot to get rid of it. >>> >>> Can you cat /proc/$pid/wchan a few times so we can get an idea of w= here it's >>> looping? =A0Thanks, >>> >>> Josef >> >> It behaves the same way here with btrfs-unstable. >> The output of "cat /proc/$pid/wchan" is 0. >> >> // Maria >> >>> -- >>> To unsubscribe from this list: send the line "unsubscribe linux-btr= fs" in >>> the body of a message to majordomo@vger.kernel.org >>> More majordomo info at =A0http://vger.kernel.org/majordomo-info.htm= l >>> >> >> >> > > I've applied the patch at the head of this thread (with the jiffies > debugging commented out) and I'm attaching a ftrace using the > function_graph tracer when I'm stuck in the loop. =A0I've just snippe= d > out a couple of the loops (the full trace file is quite large, and > mostly repititious). > > I'm going to try to modify file.c with some trace_printk debugging to > show the values of several of the relevant variables at various > stages. > > I'm going to try to exit the loop after 256 tries with an EFAULT so I > can stop the tracing at that point and capture a trace of the entry > into the problem (the ftrace ring buffer fills up too fast for me to > capture the entry point). > As promised, I'm put together a modified file.c with many trace_printk debugging statements to augment the ftrace. The trace is ~128K compressed (about 31,600 lines or 2.6MB uncompressed), so I'm putting it up on my local server instead of attaching. Let me know if it would be more appropriate to send to the list as an attachment. http://dontpanic.dyndns.org/ftrace-btrfs-file-write-debug-v2.gz I preface all my trace_printk comments with "TPK:" to make skipping through the trace easier. The trace contains the trace of about 3 or 4 successful passes through the btrfs_file_aio_write() function to show what a successful trace looks like. The pass through the btrfs_file_aio_write() that breaks begins on line = 1088. I let it loop through the while (iov_iter_count(&i) > 0) {} loop for 256 times when copied=3D=3D0 (otherwise it would loop infinitely). The= n exit out and stop the trace. =46or reference, here's a diff file for the debugging statements I've added to file.c: Let me know if you would like me to re-run this trial with different debugging lines. fs/btrfs/file.c /usr/src/linux/fs/btrfs/file.c --- fs/btrfs/file.c 2011-02-28 10:13:45.000000000 -0600 +++ /usr/src/linux/fs/btrfs/file.c 2011-02-28 13:07:11.000000000 -0600 @@ -53,12 +53,14 @@ int offset =3D pos & (PAGE_CACHE_SIZE - 1); int total_copied =3D 0; + /*************************** if ((jiffies % 10) =3D=3D 0) return 0; if ((jiffies % 25) =3D=3D 0) { write_bytes /=3D 2; } + **************************/ while (write_bytes > 0) { size_t count =3D min_t(size_t, @@ -82,10 +84,13 @@ /* Return to btrfs_file_aio_write to fault page */ if (unlikely(copied =3D=3D 0)) { + trace_printk("TPK: unlikely copied =3D=3D 0 in btrfs_copy_from_user (total_copied=3D%i)\n", + total_copied); break; } if (unlikely(copied < PAGE_CACHE_SIZE - offset)) { + trace_printk("TPK: unlikely copied < PAGE_CACHE_SIZE - offset in btrfs_copy_from_user\n"); offset +=3D copied; } else { pg++; @@ -910,8 +915,13 @@ int will_write; int buffered =3D 0; int copied =3D 0; + int copied_problem =3D 0; + int copied_loop_count =3D 0; + int stop_ftrace =3D 0; int dirty_pages =3D 0; + trace_printk("TPK: Entering btrfs_file_aio_write()\n"); + will_write =3D ((file->f_flags & O_DSYNC) || IS_SYNC(inode) || (file->f_flags & O_DIRECT)); @@ -953,6 +963,7 @@ BTRFS_I(inode)->sequence++; if (unlikely(file->f_flags & O_DIRECT)) { + trace_printk("TPK: transferred to unlikely(file->f_flags \& O_DIRECT= )\n"); num_written =3D generic_file_direct_write(iocb, iov, &nr_segs, pos, ppos, count, ocount); @@ -984,6 +995,8 @@ */ buffered =3D 1; pos +=3D num_written; + trace_printk("TPK: end unlikely(file->f_flags \& O_DIRECT) with num_written=3D%i\n", + num_written); } iov_iter_init(&i, iov, nr_segs, count, num_written); @@ -1003,6 +1016,8 @@ last_index =3D (pos + iov_iter_count(&i)) >> PAGE_CACHE_SHIFT; while (iov_iter_count(&i) > 0) { + trace_printk("TPK: start section while (iov_iter_count() > 0)\n"); + size_t offset =3D pos & (PAGE_CACHE_SIZE - 1); size_t write_bytes =3D min(iov_iter_count(&i), nrptrs * (size_t)PAGE_CACHE_SIZE - @@ -1010,6 +1025,9 @@ size_t num_pages =3D (write_bytes + offset + PAGE_CACHE_SIZE - 1) >> PAGE_CACHE_SHIFT; + trace_printk("TPK: iov_iter_count()=3D%i || nr_segs=3D%lu || nrptrs=3D= %i || offset=3D%lu || write_bytes=3D%lu || num_pages=3D%lu\n", + iov_iter_count(&i), nr_segs, nrptrs, offset, write_bytes, num_pa= ges); + WARN_ON(num_pages > nrptrs); memset(pages, 0, sizeof(struct page *) * nrptrs); @@ -1022,6 +1040,19 @@ goto out; } + if (unlikely(copied_problem)) { + copied_loop_count++; + trace_printk("TPK: copied problem(%i)\n", + copied_loop_count); + /* Give up if we've already tried 256 times */ + if (copied_loop_count > 256) { + ret =3D -EFAULT; + stop_ftrace =3D 1; + trace_printk("TPK: copied loop count exceeded, returning EFAULT...= =2E\n"); + goto out; + } + } + ret =3D btrfs_delalloc_reserve_space(inode, num_pages << PAGE_CACHE_SHIFT); if (ret) @@ -1045,6 +1076,10 @@ PAGE_CACHE_SIZE - 1) >> PAGE_CACHE_SHIFT; + if (copied =3D=3D 0) { + copied_problem =3D 1; + } + if (num_pages > dirty_pages) { if (copied > 0) atomic_inc( @@ -1080,11 +1115,19 @@ num_written +=3D copied; cond_resched(); + trace_printk("TPK: end section while (iov_iter_count() > 0)\n"); + trace_printk(" copied=3D%i || dirty_pages=3D%i || num_written=3D%i |= | ending iov_iter_count=3D%i\n", + copied, dirty_pages, num_written, iov_iter_count(&i) ); } out: + trace_printk("TPK: Reached: out:\n"); + mutex_unlock(&inode->i_mutex); - if (ret) + if (ret) { err =3D ret; + trace_printk("TPK: ret,err had value %i when out: was reached (num_written: %i)\n", + ret, num_written); + } kfree(pages); *ppos =3D pos; @@ -1140,6 +1183,19 @@ } done: current->backing_dev_info =3D NULL; + if (ret) { + trace_printk("TPK: btrfs_file_aio_write exiting with non-zero ret value (%i)\n", ret); + trace_printk("TPK: Returning num_written (%i) ? num_written (%i) : err (%i) =3D %i\n", + num_written, num_written, err, num_written ? num_written : err); + } else { + trace_printk("TPK: btrfs_file_aio_write exiting normally with (%i)", + num_written ? num_written : err); + } +=09 + if (unlikely(stop_ftrace)) { + tracing_off(); + } + return num_written ? num_written : err; } -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" = in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" = in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" = in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [PATCH] btrfs file write debugging patch 2011-03-01 11:56 ` Zhong, Xin @ 2011-03-01 14:54 ` Mitch Harder 0 siblings, 0 replies; 43+ messages in thread From: Mitch Harder @ 2011-03-01 14:54 UTC (permalink / raw) To: Zhong, Xin Cc: Maria Wikström, Josef Bacik, Johannes Hirte, Chris Mason, linux-btrfs On Tue, Mar 1, 2011 at 5:56 AM, Zhong, Xin <xin.zhong@intel.com> wrote: > Hi Mitch, > > I suspect there's a lock contention between flush-btrfs (lock_dellall= oc_pages) and btrfs_file_aio_write. However I can not recreate it local= ly. Could you please try below patch? Thanks! > > diff --git a/fs/btrfs/file.c b/fs/btrfs/file.c index 65338a1..b9d0929= 100644 > --- a/fs/btrfs/file.c > +++ b/fs/btrfs/file.c > @@ -1007,17 +1007,16 @@ static ssize_t btrfs_file_aio_write(struct ki= ocb *iocb, > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0goto out; > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0} > > - =A0 =A0 =A0 =A0 =A0 =A0 =A0 ret =3D btrfs_delalloc_reserve_space(in= ode, > - =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0= =A0 =A0 num_pages << PAGE_CACHE_SHIFT); > - =A0 =A0 =A0 =A0 =A0 =A0 =A0 if (ret) > - =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 goto out; > - > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0ret =3D prepare_pages(root, file, page= s, num_pages, > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0= pos, first_index, last_index, > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0= write_bytes); > - =A0 =A0 =A0 =A0 =A0 =A0 =A0 if (ret) { > - =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 btrfs_delalloc_release_= space(inode, > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 if (ret) > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 goto out; > + > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 ret =3D btrfs_delalloc_reserve_space(in= ode, > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0= =A0 =A0num_pages << PAGE_CACHE_SHIFT); > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 if (ret) { > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 btrfs_drop_pages(pages,= num_pages); > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0goto out; > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0} > > Thanks. I've tested this patch, but the build is still failing at the same point as before. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" = in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [PATCH] btrfs file write debugging patch 2011-03-01 10:14 ` Zhong, Xin 2011-03-01 11:56 ` Zhong, Xin @ 2011-03-01 14:51 ` Mitch Harder 1 sibling, 0 replies; 43+ messages in thread From: Mitch Harder @ 2011-03-01 14:51 UTC (permalink / raw) To: Zhong, Xin Cc: Maria Wikström, Josef Bacik, Johannes Hirte, Chris Mason, linux-btrfs On Tue, Mar 1, 2011 at 4:14 AM, Zhong, Xin <xin.zhong@intel.com> wrote: > Is your system running out of memory or is there any other thread like flush-btrfs competing for the same page? > There's no sign of memory pressure. Although I only have 1 GB in this box, I'm still show ~1/2 GB RAM free during this build. There's no swap space allocated, and nothing in dmesg that indicates there's a transient spike of RAM pressure. > I can only see one process in your ftrace log. You may need to trace all btrfs.ko function calls instead of a single process. Thanks! > That ftrace.log was run with ftrace defaults for a function trace. It should collect calls from the whole system. For the sake of consistency, I am intentionally trying to insure that very few other things are going on at the same time as this build. And I'm building with "-j1" so things will happen the same way each time. Also, I supplied just the tail end of the trace log. The full log shows a few of the other build processes leading up to the problem, but the ftrace ring buffer fills up surprisingly fast. Even with a 50MB ring buffer for ftrace, I usually collect less than 1 second of information when something busy like a build is going on. Let me know if you'd like to see the full log. It's bigger, but I can find someplace to put it. But I'm pretty sure that wmldbcreate is the only thing that is going on when the breakage occurs. Otherwise I wouldn't get such consistent breakage in the same spot every time. ^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [PATCH] btrfs file write debugging patch 2011-02-28 20:20 ` Mitch Harder 2011-03-01 5:09 ` Mitch Harder 2011-03-01 10:14 ` Zhong, Xin @ 2011-03-01 21:56 ` Piotr Szymaniak 2 siblings, 0 replies; 43+ messages in thread From: Piotr Szymaniak @ 2011-03-01 21:56 UTC (permalink / raw) To: Mitch Harder Cc: Josef Bacik, Johannes Hirte, Chris Mason, Zhong, Xin, linux-btrfs [-- Attachment #1: Type: text/plain, Size: 836 bytes --] On Mon, Feb 28, 2011 at 02:20:22PM -0600, Mitch Harder wrote: > As promised, I'm put together a modified file.c with many trace_printk > debugging statements to augment the ftrace. > *snip* Just my few cents. I've applied the patch from Chris Mason (Sun, 27 Feb 2011 20:46:05 -0500) and this one from Mitch (Mon, 28 Feb 2011 14:20:22 -0600) on top of vanilla 2.6.38-rc6 and it seems that it resolves my issues with hanging `svn info' during libgcrypt emerge. Piotr Szymaniak. -- - (...) Nie wyobrazam sobie, co ta gora miesa moglaby ci dac, czego ja nie moglbym ofiarowac. Oczywiscie poza piecdziesiecioma funtami rozrosnietych miesni. - Moze mnie wlasnie pociagaja rozrosniete miesnie. (...) W koncu wielu mezczyzn pociaga rozrosnieta tkanka tluszczowa piersi. -- Graham Masterton, "The Wells of Hell" [-- Attachment #2: Type: application/pgp-signature, Size: 198 bytes --] ^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [PATCH v2]Btrfs: pwrite blocked when writing from the mmaped buffer of the same page 2011-02-21 1:51 ` Zhong, Xin 2011-02-24 14:51 ` Maria Wikström @ 2011-02-24 23:35 ` Piotr Szymaniak 1 sibling, 0 replies; 43+ messages in thread From: Piotr Szymaniak @ 2011-02-24 23:35 UTC (permalink / raw) To: Zhong, Xin; +Cc: Maria Wikström, Johannes Hirte, linux-btrfs [-- Attachment #1.1: Type: text/plain, Size: 2887 bytes --] On Mon, Feb 21, 2011 at 09:51:11AM +0800, Zhong, Xin wrote: > The backtrace in your attachment looks like a known bug of 2.6.37 which have already been fixed in 2.6.38. I have no idea why latest btrfs still hang in your environment if there's no debug info... Hi list. I'm watching this list for a while because it seems, that I'm also affected by this bug. In the archives I found someone with Gentoo system with freezing `svn info' (thats why I joined). Well, seems that I have same issue here. Attached latest _rc kernel sysrq+t output (first part when the `svn info' freezed on libgcrypt, and second after ctrl+c that emerge). Seems that my backtrace is small compared to Marias. I'm missing some features in kernel to get larger backtraces? Piotr Szymaniak. > -----Original Message----- > From: Maria Wikström [mailto:maria@ponstudios.se] > Sent: Friday, February 18, 2011 7:32 PM > To: Zhong, Xin > Cc: Johannes Hirte; linux-btrfs@vger.kernel.org > Subject: RE: [PATCH v2]Btrfs: pwrite blocked when writing from the mmaped buffer of the same page > > > Seems like my reply got eaten by the lists spam filter, so I resend with attachment compressed. Should have thought of that :p > > > fre 2011-02-11 klockan 12:39 +0800 skrev Zhong, Xin: > > Hi, > > > > Could you paste the output of sysrq+t here? Thanks! > > Yes, it's in the attachment. I tried latest btrfs from git (last commit Mon Feb 14 00:45:29 2011 +0000) but it hang so bad that I couldn't get the output from sysrq+t to hit the disk. So the output is from vanilla > 2.6.37 > > // Maria > > > -----Original Message----- > > From: Johannes Hirte [mailto:johannes.hirte@fem.tu-ilmenau.de] > > Sent: Wednesday, February 02, 2011 7:35 AM > > To: Zhong, Xin > > Cc: Maria Wikström; linux-btrfs@vger.kernel.org > > Subject: Re: [PATCH v2]Btrfs: pwrite blocked when writing from the > > mmaped buffer of the same page > > > > On Friday 28 January 2011 04:53:24 Zhong, Xin wrote: > > > Could you describe the steps to recreate it? > > > It will be a great help for me to look further. Thanks! > > > > It's a little strange. I have to systems with btrfs, both > > Gentoo-based. One is affected by this bug the other is not. On the > > affected system it is enough to do a 'emerge dev-libs/libgcrypt' that > > should normaly compile and install libgcrypt. The emerge command is > > part of portage, the package management of Gentoo. > > The strace output looks similar to the one from Maria: > > > > N?????r??y????b?X??ǧv?^?){.n?+????{?n?߲)????w*\x1fjg???\x1e?????ݢj/???z?ޖ??2?ޙ????&?)ߡ?a??\x7f??\x1e?G???h?\x0f?j:+v???w??٥ -- Druzyna futbolowa tutaj jest do niczego, ale ucze sie troche pilki noznej. Trener mowi, ze pilka nozna to futbol dla inteligentnych, a futbol to futbol dla kretynow. -- Stephen King, "The Dead Zone" [-- Attachment #1.2: vanilla-2.6.38-rc6_sysrq-t.bz2 --] [-- Type: application/x-bzip2, Size: 4837 bytes --] [-- Attachment #2: Type: application/pgp-signature, Size: 198 bytes --] ^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [PATCH v2]Btrfs: pwrite blocked when writing from the mmaped buffer of the same page 2011-02-11 4:39 ` Zhong, Xin 2011-02-18 11:31 ` Maria Wikström @ 2011-02-22 22:27 ` Johannes Hirte 2011-02-23 7:27 ` Zhong, Xin 1 sibling, 1 reply; 43+ messages in thread From: Johannes Hirte @ 2011-02-22 22:27 UTC (permalink / raw) To: Zhong, Xin; +Cc: Maria Wikström, linux-btrfs [-- Attachment #1: Type: Text/Plain, Size: 610 bytes --] On Friday 11 February 2011 05:39:39 Zhong, Xin wrote: > Hi, > > Could you paste the output of sysrq+t here? Thanks! Sorry for the long delay. I've attached the dmesg output of sysrq-t from 2.6.38-rc3 and rc4 as the behavior differs between both versions. With rc3 it don't need any interaction. I start 'emerge sys-devel/libgcrypt', the tasks hangs and top shows a svn process eating 100% CPU. With rc4 the system hangs too but the CPU load stays low. When I try to cancel the emerge process (ctrl c) the CPU load goes up and top shows the svn process together with a flush- btrfs task, eating the CPU. [-- Attachment #2: sysrq-t-38-rc.tar.bz2 --] [-- Type: application/x-bzip-compressed-tar, Size: 45354 bytes --] ^ permalink raw reply [flat|nested] 43+ messages in thread
* RE: [PATCH v2]Btrfs: pwrite blocked when writing from the mmaped buffer of the same page 2011-02-22 22:27 ` Johannes Hirte @ 2011-02-23 7:27 ` Zhong, Xin 2011-02-23 21:56 ` Chris Mason 0 siblings, 1 reply; 43+ messages in thread From: Zhong, Xin @ 2011-02-23 7:27 UTC (permalink / raw) To: Johannes Hirte; +Cc: Maria Wikström, linux-btrfs SW4gdGhlIGRtZXNnIG9mIHJjNCwgSSBjYW4gc2VlIHN2biBoYW5nIGluIHNocmlua19kZWxsYWxs b2MgYW5kIHRoZXJlJ3MgdHdvIGZsdXNoLWJ0cmZzIHRocmVhZHMgaGFuZyB0aGVyZSB0b28uDQoN Ckpvc2VmLCBpdCBzZWVtcyB5b3UgYXJlIHRoZSBleHBlcnQgaW4gdGhpcyBhcmVhLiBDb3VsZCB5 b3UgdGFrZSBhIHF1aWNrIGxvb2s/IFRoYW5rcyENCg0KLS0tLS1PcmlnaW5hbCBNZXNzYWdlLS0t LS0NCkZyb206IEpvaGFubmVzIEhpcnRlIFttYWlsdG86am9oYW5uZXMuaGlydGVAZmVtLnR1LWls bWVuYXUuZGVdIA0KU2VudDogV2VkbmVzZGF5LCBGZWJydWFyeSAyMywgMjAxMSA2OjI3IEFNDQpU bzogWmhvbmcsIFhpbg0KQ2M6IE1hcmlhIFdpa3N0csO2bTsgbGludXgtYnRyZnNAdmdlci5rZXJu ZWwub3JnDQpTdWJqZWN0OiBSZTogW1BBVENIIHYyXUJ0cmZzOiBwd3JpdGUgYmxvY2tlZCB3aGVu IHdyaXRpbmcgZnJvbSB0aGUgbW1hcGVkIGJ1ZmZlciBvZiB0aGUgc2FtZSBwYWdlDQoNCk9uIEZy aWRheSAxMSBGZWJydWFyeSAyMDExIDA1OjM5OjM5IFpob25nLCBYaW4gd3JvdGU6DQo+IEhpLA0K PiANCj4gQ291bGQgeW91IHBhc3RlIHRoZSBvdXRwdXQgb2Ygc3lzcnErdCBoZXJlPyBUaGFua3Mh DQoNClNvcnJ5IGZvciB0aGUgbG9uZyBkZWxheS4gSSd2ZSBhdHRhY2hlZCB0aGUgZG1lc2cgb3V0 cHV0IG9mIHN5c3JxLXQgZnJvbQ0KMi42LjM4LXJjMyBhbmQgcmM0IGFzIHRoZSBiZWhhdmlvciBk aWZmZXJzIGJldHdlZW4gYm90aCB2ZXJzaW9ucy4gV2l0aCByYzMgaXQgZG9uJ3QgbmVlZCBhbnkg aW50ZXJhY3Rpb24uIEkgc3RhcnQgJ2VtZXJnZSBzeXMtZGV2ZWwvbGliZ2NyeXB0JywgdGhlIHRh c2tzIGhhbmdzIGFuZCB0b3Agc2hvd3MgYSBzdm4gcHJvY2VzcyBlYXRpbmcgMTAwJSBDUFUuIFdp dGggcmM0IHRoZSBzeXN0ZW0gaGFuZ3MgdG9vIGJ1dCB0aGUgQ1BVIGxvYWQgc3RheXMgbG93LiBX aGVuIEkgdHJ5IHRvIGNhbmNlbCB0aGUgZW1lcmdlIHByb2Nlc3MgKGN0cmwNCmMpIHRoZSBDUFUg bG9hZCBnb2VzIHVwIGFuZCB0b3Agc2hvd3MgdGhlIHN2biBwcm9jZXNzIHRvZ2V0aGVyIHdpdGgg YSBmbHVzaC0gYnRyZnMgdGFzaywgZWF0aW5nIHRoZSBDUFUuDQo= ^ permalink raw reply [flat|nested] 43+ messages in thread
* RE: [PATCH v2]Btrfs: pwrite blocked when writing from the mmaped buffer of the same page 2011-02-23 7:27 ` Zhong, Xin @ 2011-02-23 21:56 ` Chris Mason 2011-02-23 23:02 ` Johannes Hirte 0 siblings, 1 reply; 43+ messages in thread From: Chris Mason @ 2011-02-23 21:56 UTC (permalink / raw) To: Zhong, Xin; +Cc: Johannes Hirte, Maria Wikström, linux-btrfs Excerpts from Zhong, Xin's message of 2011-02-23 02:27:05 -0500: > In the dmesg of rc4, I can see svn hang in shrink_dellalloc and there's two flush-btrfs threads hang there too. > > Josef, it seems you are the expert in this area. Could you take a quick look? Thanks! Ok, it does look like the fluhs-btrfs threads are busy trying to flush things. Could you please do a btrfs-show and a btrfs fi df /xxx (where xxx is your mount point) and send the results here? -chris > > -----Original Message----- > From: Johannes Hirte [mailto:johannes.hirte@fem.tu-ilmenau.de] > Sent: Wednesday, February 23, 2011 6:27 AM > To: Zhong, Xin > Cc: Maria Wikstr\xc3\xb6m; linux-btrfs@vger.kernel.org > Subject: Re: [PATCH v2]Btrfs: pwrite blocked when writing from the mmaped buffer of the same page > > On Friday 11 February 2011 05:39:39 Zhong, Xin wrote: > > Hi, > > > > Could you paste the output of sysrq+t here? Thanks! > > Sorry for the long delay. I've attached the dmesg output of sysrq-t from > 2.6.38-rc3 and rc4 as the behavior differs between both versions. With rc3 it don't need any interaction. I start 'emerge sys-devel/libgcrypt', the tasks hangs and top shows a svn process eating 100% CPU. With rc4 the system hangs too but the CPU load stays low. When I try to cancel the emerge process (ctrl > c) the CPU load goes up and top shows the svn process together with a flush- btrfs task, eating the CPU. > \xff\xf4\xe8\xba{.n\xc7+\x89\xb7\x9f\xae\x89\xad\x86+%\x8a\xcb\xff\xb1\xe9\xdd\xb6\x17\xa5\x8aw\xff\xba{.n\xc7+\x89\xb7\xa5\x8a{\xb1\xfd\xbbk~\xcf\xe2\x9e\xd8^n\x87r\xa1\xf6\xa6z\xcb\x1a\x81\xebh\x99\xa8\xe8\xad\xda&\xa3\xfb\xe0z\xbf\xe4z\xb9\xde\x97\xfa+\x80\xca+zf\xa3\xa2\xb7h\x9a\x88\xa7~\x86\xad\x86\xdbi\xff\xff\xef\x81\xea\xff\x91\xea\xe7z_\xe8\xae\x0f\xe6j:+v\x89\xa8\xfe)\xdf\xa3\xf8m ^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [PATCH v2]Btrfs: pwrite blocked when writing from the mmaped buffer of the same page 2011-02-23 21:56 ` Chris Mason @ 2011-02-23 23:02 ` Johannes Hirte 2011-02-24 15:23 ` Chris Mason 0 siblings, 1 reply; 43+ messages in thread From: Johannes Hirte @ 2011-02-23 23:02 UTC (permalink / raw) To: Chris Mason; +Cc: Zhong, Xin, Maria Wikström, linux-btrfs On Wednesday 23 February 2011 22:56:27 Chris Mason wrote: > Excerpts from Zhong, Xin's message of 2011-02-23 02:27:05 -0500: > > In the dmesg of rc4, I can see svn hang in shrink_dellalloc and there's > > two flush-btrfs threads hang there too. > > > > Josef, it seems you are the expert in this area. Could you take a quick > > look? Thanks! > > Ok, it does look like the fluhs-btrfs threads are busy trying to flush > things. > > Could you please do a btrfs-show and a btrfs fi df /xxx (where xxx is > your mount point) and send the results here? > > -chris failed to read /dev/sr0 Label: none uuid: 00eab15f-c4cf-4403-a529-9bc11fa50167 Total devices 1 FS bytes used 47.72GB devid 1 size 65.69GB used 65.69GB path /dev/sda2 Label: none uuid: c6f4e6e6-c4ba-4394-9e9c-bbc3d0b32793 Total devices 1 FS bytes used 9.48GB devid 1 size 20.01GB used 20.01GB path /dev/sda1 Btrfs v0.19-35-g1b444cd-dirty and btrfs fi df on / Data: total=15.49GB, used=8.35GB System, DUP: total=8.00MB, used=12.00KB System: total=4.00MB, used=0.00 Metadata, DUP: total=2.25GB, used=1.13GB /home Data: total=63.42GB, used=47.47GB System: total=4.00MB, used=16.00KB Metadata: total=2.27GB, used=251.34MB The bug is reproducable on both filesystems. regards, Johannes ^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [PATCH v2]Btrfs: pwrite blocked when writing from the mmaped buffer of the same page 2011-02-23 23:02 ` Johannes Hirte @ 2011-02-24 15:23 ` Chris Mason 0 siblings, 0 replies; 43+ messages in thread From: Chris Mason @ 2011-02-24 15:23 UTC (permalink / raw) To: Johannes Hirte; +Cc: Zhong, Xin, Maria Wikström, linux-btrfs Excerpts from Johannes Hirte's message of 2011-02-23 18:02:44 -0500: > On Wednesday 23 February 2011 22:56:27 Chris Mason wrote: > > Excerpts from Zhong, Xin's message of 2011-02-23 02:27:05 -0500: > > > In the dmesg of rc4, I can see svn hang in shrink_dellalloc and there's > > > two flush-btrfs threads hang there too. > > > > > > Josef, it seems you are the expert in this area. Could you take a quick > > > look? Thanks! > > > > Ok, it does look like the fluhs-btrfs threads are busy trying to flush > > things. > > > > Could you please do a btrfs-show and a btrfs fi df /xxx (where xxx is > > your mount point) and send the results here? > > > > -chris > > failed to read /dev/sr0 > Label: none uuid: 00eab15f-c4cf-4403-a529-9bc11fa50167 > Total devices 1 FS bytes used 47.72GB > devid 1 size 65.69GB used 65.69GB path /dev/sda2 > > Label: none uuid: c6f4e6e6-c4ba-4394-9e9c-bbc3d0b32793 > Total devices 1 FS bytes used 9.48GB > devid 1 size 20.01GB used 20.01GB path /dev/sda1 > > Btrfs v0.19-35-g1b444cd-dirty > > and btrfs fi df on > > / > > Data: total=15.49GB, used=8.35GB > System, DUP: total=8.00MB, used=12.00KB > System: total=4.00MB, used=0.00 > Metadata, DUP: total=2.25GB, used=1.13GB > > /home > > Data: total=63.42GB, used=47.47GB > System: total=4.00MB, used=16.00KB > Metadata: total=2.27GB, used=251.34MB > > The bug is reproducable on both filesystems. Ok, you've got a good amount of metadata space free, but it is frantically trying to make room for the delayed allocation. Let me see if I can recreate this setup here. -chris ^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [PATCH v2]Btrfs: pwrite blocked when writing from the mmaped buffer of the same page 2011-01-28 2:54 ` Johannes Hirte 2011-01-28 3:53 ` Zhong, Xin @ 2011-01-28 16:47 ` Maria Wikström 2011-01-28 18:27 ` Rui Miguel Silva 1 sibling, 1 reply; 43+ messages in thread From: Maria Wikström @ 2011-01-28 16:47 UTC (permalink / raw) To: xin.zhong; +Cc: linux-btrfs, johannes.hirte fre 2011-01-28 klockan 03:54 +0100 skrev Johannes Hirte: > On Friday 28 January 2011 02:26:43 Zhong, Xin wrote: > > Please try the fix in below link: > > http://www.spinics.net/lists/linux-btrfs/msg08051.html > > > > Thanks! > > This doesn't fix it for me. At least there is a difference. Whereas the svn > process started consuming 100% CPU without any further interaction before, the > system just hang now. The svn process starts eating the CPU when I cancel the > emerge via ctrl-c. Additional I see a flush-btrfs task now consuming CPU time. > > regards, > Johannes > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > The patch makes the process exit cleanly but complains that there is no space left. It should be a few GB but it is only a few MB!? I delete 5GB and controls that the space is usable. Try again but this time it almost hangs the system, I can blink the leds and move the mouse but noting more. I boot 2.6.36.1 and use a snapshot as root, "emerge libgcrypt" compiles and installs fine. Boots back into 2.6.37 and try again and the system hangs again. // Maria ^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [PATCH v2]Btrfs: pwrite blocked when writing from the mmaped buffer of the same page 2011-01-28 16:47 ` Maria Wikström @ 2011-01-28 18:27 ` Rui Miguel Silva 2011-01-29 15:38 ` Maria Wikström 0 siblings, 1 reply; 43+ messages in thread From: Rui Miguel Silva @ 2011-01-28 18:27 UTC (permalink / raw) To: Maria Wikström; +Cc: xin.zhong, linux-btrfs, johannes.hirte On Friday, January 28, 2011 04:47:21 pm Maria Wikstr=C3=B6m wrote: > fre 2011-01-28 klockan 03:54 +0100 skrev Johannes Hirte: > > On Friday 28 January 2011 02:26:43 Zhong, Xin wrote: > > > Please try the fix in below link: > > > http://www.spinics.net/lists/linux-btrfs/msg08051.html > > >=20 > > > Thanks! > >=20 > > This doesn't fix it for me. At least there is a difference. Whereas= the > > svn process started consuming 100% CPU without any further interact= ion > > before, the system just hang now. The svn process starts eating the= CPU > > when I cancel the emerge via ctrl-c. Additional I see a flush-btrfs= task > > now consuming CPU time. > >=20 > > regards, > >=20 > > Johannes > >=20 > > -- > > To unsubscribe from this list: send the line "unsubscribe linux-btr= fs" in > > the body of a message to majordomo@vger.kernel.org > > More majordomo info at http://vger.kernel.org/majordomo-info.html >=20 > The patch makes the process exit cleanly but complains that there is = no > space left. It should be a few GB but it is only a few MB!? I delete = 5GB > and controls that the space is usable. Try again but this time it alm= ost > hangs the system, I can blink the leds and move the mouse but noting > more. > I boot 2.6.36.1 and use a snapshot as root, "emerge libgcrypt" compil= es > and installs fine. Boots back into 2.6.37 and try again and the syste= m > hangs again. >=20 > // Maria >=20 Hi Maria, I had something similar with vanilla 2.6.37 just for running: cscope -R -b -q allways hang complety the process, only solution reboot. I just applied the btrfs patches that arrived in the main tree between= the=20 2.6.37 and 2.6.38-rc1 and they seem to fix the issue. I did not have the time to try to found which patch fixed the problem. Hope this could help you. Cheers, //Rui -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" = in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [PATCH v2]Btrfs: pwrite blocked when writing from the mmaped buffer of the same page 2011-01-28 18:27 ` Rui Miguel Silva @ 2011-01-29 15:38 ` Maria Wikström 0 siblings, 0 replies; 43+ messages in thread From: Maria Wikström @ 2011-01-29 15:38 UTC (permalink / raw) To: Rui Miguel Silva; +Cc: xin.zhong, linux-btrfs, johannes.hirte fre 2011-01-28 klockan 18:27 +0000 skrev Rui Miguel Silva:=20 > On Friday, January 28, 2011 04:47:21 pm Maria Wikstr=C3=B6m wrote: > > fre 2011-01-28 klockan 03:54 +0100 skrev Johannes Hirte: > > > On Friday 28 January 2011 02:26:43 Zhong, Xin wrote: > > > > Please try the fix in below link: > > > > http://www.spinics.net/lists/linux-btrfs/msg08051.html > > > >=20 > > > > Thanks! > > >=20 > > > This doesn't fix it for me. At least there is a difference. Where= as the > > > svn process started consuming 100% CPU without any further intera= ction > > > before, the system just hang now. The svn process starts eating t= he CPU > > > when I cancel the emerge via ctrl-c. Additional I see a flush-btr= fs task > > > now consuming CPU time. > > >=20 > > > regards, > > >=20 > > > Johannes > > >=20 > > > -- > > > To unsubscribe from this list: send the line "unsubscribe linux-b= trfs" in > > > the body of a message to majordomo@vger.kernel.org > > > More majordomo info at http://vger.kernel.org/majordomo-info.htm= l > >=20 > > The patch makes the process exit cleanly but complains that there i= s no > > space left. It should be a few GB but it is only a few MB!? I delet= e 5GB > > and controls that the space is usable. Try again but this time it a= lmost > > hangs the system, I can blink the leds and move the mouse but notin= g > > more. > > I boot 2.6.36.1 and use a snapshot as root, "emerge libgcrypt" comp= iles > > and installs fine. Boots back into 2.6.37 and try again and the sys= tem > > hangs again. > >=20 > > // Maria > >=20 >=20 > Hi Maria, > I had something similar with vanilla 2.6.37 just for running: > cscope -R -b -q >=20 > allways hang complety the process, only solution reboot. > I just applied the btrfs patches that arrived in the main tree betwe= en the=20 > 2.6.37 and 2.6.38-rc1 and they seem to fix the issue. >=20 > I did not have the time to try to found which patch fixed the problem= =2E >=20 > Hope this could help you. >=20 > Cheers, > //Rui I have tried with both vanilla 2.6.38-rc2 and btrfs-unstable from git, both having the problem.=20 Btrfs-unstable with the patch above behaves differently. The process ca= n be killed with Ctrl + C but then the terminal and gnome-pty-helper eats cpu instead. They can be killed with SysRq + i together with everything else. // Maria -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" = in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 43+ messages in thread
end of thread, other threads:[~2011-03-01 21:56 UTC | newest]
Thread overview: 43+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2010-12-09 9:30 [PATCH v2]Btrfs: pwrite blocked when writing from the mmaped buffer of the same page Zhong, Xin
2011-01-27 13:09 ` Johannes Hirte
2011-01-27 22:12 ` Maria Wikström
2011-01-28 1:26 ` Zhong, Xin
2011-01-28 2:54 ` Johannes Hirte
2011-01-28 3:53 ` Zhong, Xin
2011-02-01 23:34 ` Johannes Hirte
2011-02-11 4:39 ` Zhong, Xin
2011-02-18 11:31 ` Maria Wikström
2011-02-21 1:51 ` Zhong, Xin
2011-02-24 14:51 ` Maria Wikström
2011-02-24 15:55 ` Mitch Harder
2011-02-24 16:00 ` Chris Mason
2011-02-24 16:03 ` Mitch Harder
2011-02-24 16:19 ` Chris Mason
2011-02-24 16:32 ` Mitch Harder
[not found] ` <AANLkTinvyb-bTVVignd1KGojvh-QrYCFmCnwYKBsYC_2@mail.gmail.com>
2011-02-25 17:11 ` Mitch Harder
2011-02-25 18:43 ` Mitch Harder
2011-02-25 19:19 ` Chris Mason
2011-02-28 1:46 ` [PATCH] btrfs file write debugging patch Chris Mason
2011-02-28 8:56 ` Zhong, Xin
2011-02-28 14:02 ` Chris Mason
2011-02-28 10:13 ` Johannes Hirte
2011-02-28 14:00 ` Chris Mason
2011-02-28 16:10 ` Josef Bacik
2011-02-28 16:45 ` Maria Wikström
2011-02-28 17:47 ` Mitch Harder
2011-02-28 20:20 ` Mitch Harder
2011-03-01 5:09 ` Mitch Harder
2011-03-01 10:14 ` Zhong, Xin
2011-03-01 11:56 ` Zhong, Xin
2011-03-01 14:54 ` Mitch Harder
2011-03-01 14:51 ` Mitch Harder
2011-03-01 21:56 ` Piotr Szymaniak
2011-02-24 23:35 ` [PATCH v2]Btrfs: pwrite blocked when writing from the mmaped buffer of the same page Piotr Szymaniak
2011-02-22 22:27 ` Johannes Hirte
2011-02-23 7:27 ` Zhong, Xin
2011-02-23 21:56 ` Chris Mason
2011-02-23 23:02 ` Johannes Hirte
2011-02-24 15:23 ` Chris Mason
2011-01-28 16:47 ` Maria Wikström
2011-01-28 18:27 ` Rui Miguel Silva
2011-01-29 15:38 ` Maria Wikström
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.