* [PULL] Btrfs for 4.13, part 1
@ 2017-06-23 15:16 David Sterba
2017-06-24 10:05 ` Wang Shilong
` (3 more replies)
0 siblings, 4 replies; 13+ messages in thread
From: David Sterba @ 2017-06-23 15:16 UTC (permalink / raw)
To: clm; +Cc: David Sterba, linux-btrfs
Hi,
this is the main batch for 4.13. There are some user visible changes, see
below. The core updates improve error handling (mostly related to bios), with
the usual incremental work on the GFP_NOFS (mis)use removal. All patches have
been in for-next for an extensive amount of time.
Thre will be followups but I want push the series (111 patches) forward. There
are also some updates to adjacent subsystems (writeback and blocklayer), so I
want to give some stable point for merging in the upcoming weeks.
User visible changes:
* quota override tunable
* improved compression thresholds
* statx support
* obsoleted mount option alloc_start
Core updates:
* bio-related updates
- faster bio cloning
- no allocation failures
- preallocated flush bios
* more kvzalloc use, memalloc_nofs protections, GFP_NOFS updates
* prep work for btree_inode removal
* dir-item validation
* qgoup fixes and updates
* cleanups
- removed unused struct members, unused code, refactoring
- argument refactoring (fs_info/root, caller -> callee sink)
- SEARCH_TREE docs
----------------------------------------------------------------
The following changes since commit 41f1830f5a7af77cf5c86359aba3cbd706687e52:
Linux 4.12-rc6 (2017-06-19 22:19:37 +0800)
are available in the git repository at:
git://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux.git for-4.13-part1
for you to fetch changes up to f3f000297be88b1b75fde5027d660a8d8a44de14:
btrfs: qgroup: Fix qgroup reserved space underflow by only freeing reserved ranges (2017-06-21 20:56:14 +0200)
----------------------------------------------------------------
Anand Jain (7):
btrfs: cleanup unused qgroup trace event
btrfs: add framework to handle device flush error as a volume
btrfs: btrfs_decompress_bio() could accept compressed_bio instead
btrfs: reduce arguments for decompress_bio ops
btrfs: write_dev_flush does not return ENOMEM anymore
btrfs: remove redundant null bdev counting during flush submission
btrfs: wait part of the write_dev_flush() can be separated out
Arnd Bergmann (1):
Btrfs: work around maybe-uninitialized warning
Daichou (1):
Btrfs: remove obsolete FIXMEs in qgroup ioctls
Dan Carpenter (1):
Btrfs: remove an unused variable
David Sterba (39):
btrfs: remove unused member err from reada_extent
btrfs: remove unused member list from async_submit_bio
btrfs: remove unused members dir_path from recorded_ref
btrfs: remove unused member list from btrfs_end_io_wq
btrfs: fix bool type in btrfs_page_exists_in_range
btrfs: scrub: inline helper scrub_setup_wr_ctx
btrfs: scrub: inline helper scrub_free_wr_ctx
btrfs: scrub: simplify cleanup of wr_ctx in scrub_free_ctx
btrfs: scrub: use fs_info::sectorsize and drop it from scrub context
btrfs: scrub: embed scrub_wr_ctx into scrub context
btrfs: use generic slab for for btrfs_transaction
btrfs: replace opencoded kvzalloc with the helper
btrfs: send: use kvmalloc in iterate_dir_item
btrfs: scrub: add memalloc_nofs protection around init_ipath
btrfs: use GFP_KERNEL in init_ipath
btrfs: adjust includes after vmalloc removal
btrfs: add memalloc_nofs protections around alloc_workspace callback
btrfs: switch kmallocs to GFP_KERNEL in lzo/zlib alloc_workspace
btrfs: switch to kvmalloc and GFP_KERNEL in lzo/zlib alloc_workspace
btrfs: bioset allocations will never fail, adapt our helpers
btrfs: btrfs_bio_alloc never fails, skip error handling
btrfs: btrfs_bio_clone never fails, skip error handling
btrfs: btrfs_io_bio_alloc never fails, skip error handling
btrfs: sink gfp parameter to btrfs_bio_clone
btrfs: remove redundant parameters from btrfs_bio_alloc
btrfs: opencode trivial compressed_bio_alloc, simplify error handling
btrfs: pass bytes to btrfs_bio_alloc
btrfs: document mandatory order of bio in btrfs_io_bio
btrfs: add helper to initialize the non-bio part of btrfs_io_bio
btrfs: sink gfp parameter to btrfs_io_bio_alloc
btrfs: use GFP_KERNEL in btrfs_calc_avail_data_space
btrfs: use GFP_KERNEL in btrfs_init_dev_replace_tgtdev
btrfs: cleanup duplicate return value in insert_inline_extent
btrfs: move fs_info::fs_frozen to the flags
btrfs: obsolete and remove mount option alloc_start
btrfs: preallocate device flush bio
btrfs: account as waiting for IO, while waiting fot the flush bio completion
btrfs: move dev stats accounting out of wait_dev_flush
btrfs: fix validation of XATTR_ITEM dir items
Fabian Frederick (1):
btrfs: kmap() can't fail
Filipe Manana (3):
Btrfs: fix invalid extent maps due to hole punching
Btrfs: send, fix invalid path after renaming and linking file
Btrfs: incremental send, fix invalid path for unlink commands
Guoqing Jiang (1):
btrfs: simplify code with bio_io_error
Hans van Kranenburg (1):
Btrfs: btrfs_ioctl_search_key documentation
Jeff Layton (1):
btrfs: btrfs_wait_tree_block_writeback can be void return
Jeff Mahoney (3):
btrfs: cleanup root usage by btrfs_get_alloc_profile
btrfs: remove root usage from can_overcommit
btrfs: add cond_resched to btrfs_qgroup_trace_leaf_items
Josef Bacik (3):
Btrfs: replace tree->mapping with tree->private_data
btrfs: remove inode argument from repair_io_failure
Btrfs: don't pass the inode through clean_io_failure
Liu Bo (12):
Btrfs: use bio_clone_fast to clone our bio
Btrfs: new helper btrfs_bio_clone_partial
Btrfs: use bio_clone_bioset_partial to simplify DIO submit
Btrfs: change how we iterate bios in endio
Btrfs: record error if one block has failed to retry
Btrfs: check-integrity use bvec_iter
Btrfs: unify naming of btrfs_io_bio
Btrfs: hardcode GFP_NOFS for btrfs_bio_clone_partial
Btrfs: skip commit transaction if we don't have enough pinned bytes
Btrfs: let btrfs_print_leaf print more about block group
Btrfs: tolerate errors if we have retried successfully
Btrfs: skip checksum verification if IO error occurs
Nikolay Borisov (8):
btrfs: Convert fs_info->free_chunk_space to atomic64_t
btrfs: Separate space_info create/update
btrfs: Refactor update_space_info
btrfs: rename btrfs_leaf_data to BTRFS_LEAF_DATA_OFFSET
btrfs: remove __BTRFS_LEAF_DATA_SIZE
btrfs: Use btrfs_space_info_used instead of opencoding it
btrfs: Manually implement device_total_bytes getter/setter
btrfs: Round down values which are written for total_bytes_size
Omar Sandoval (7):
Btrfs: use memalloc_nofs and kvzalloc() for free space tree bitmaps
Btrfs: make add_pinned_bytes() take an s64 num_bytes instead of u64
Btrfs: make BUG_ON() in add_pinned_bytes() an ASSERT()
Btrfs: update total_bytes_pinned when pinning down extents
Btrfs: always account pinned bytes when dropping a tree block ref
Btrfs: return old and new total ref mods when adding delayed refs
Btrfs: rework delayed ref total_bytes_pinned accounting
Qu Wenruo (6):
btrfs: qgroup: Add quick exit for non-fs extents
btrfs: qgroup: Cleanup btrfs_qgroup_prepare_account_extents function
btrfs: qgroup: Return actually freed bytes for qgroup release or free data
btrfs: qgroup: Fix qgroup reserved space underflow caused by buffered write and quotas being enabled
btrfs: qgroup: Introduce extent changeset for qgroup reserve functions
btrfs: qgroup: Fix qgroup reserved space underflow by only freeing reserved ranges
Sahil Kang (1):
btrfs: Remove unnecessary branching in free-space-tree.c
Sargun Dhillon (2):
btrfs: add quota override flag to enable quota override for CAP_SYS_RESOURCE
btrfs: Add quota_override knob into sysfs
Su Yue (9):
btrfs: Introduce btrfs_is_name_len_valid to avoid reading beyond boundary
btrfs: Check name_len with boundary in verify dir_item
btrfs: Check name_len on add_inode_ref call path
btrfs: Verify dir_item in replay_xattr_deletes
btrfs: Check name_len in btrfs_check_ref_name_override
btrfs: Check name_len before read in iterate_dir_item
btrfs: Check name_len before reading btrfs_get_name
btrfs: Check name_len before in btrfs_del_root_ref
btrfs: Verify dir_item in iterate_object_props
Timofey Titovets (3):
Btrfs: lzo: fix typo in error message after failed deflate
Btrfs: lzo: compressed data size must be less then input size
Btrfs: compression must free at least one sector size
Yonghong Song (1):
Btrfs: add statx support
fs/btrfs/backref.c | 10 +-
fs/btrfs/check-integrity.c | 53 ++---
fs/btrfs/compression.c | 94 ++------
fs/btrfs/compression.h | 44 +++-
fs/btrfs/ctree.c | 42 ++--
fs/btrfs/ctree.h | 84 ++++---
fs/btrfs/delayed-ref.c | 29 ++-
fs/btrfs/delayed-ref.h | 6 +-
fs/btrfs/dir-item.c | 94 +++++++-
fs/btrfs/disk-io.c | 179 +++++++--------
fs/btrfs/disk-io.h | 8 +-
fs/btrfs/export.c | 5 +
fs/btrfs/extent-tree.c | 481 +++++++++++++++++++++------------------
fs/btrfs/extent_io.c | 217 ++++++++----------
fs/btrfs/extent_io.h | 82 +++++--
fs/btrfs/file-item.c | 31 ++-
fs/btrfs/file.c | 46 ++--
fs/btrfs/free-space-tree.c | 38 ++--
fs/btrfs/inode-map.c | 4 +-
fs/btrfs/inode.c | 449 ++++++++++++++++++++----------------
fs/btrfs/ioctl.c | 16 +-
fs/btrfs/lzo.c | 33 +--
fs/btrfs/print-tree.c | 7 +-
fs/btrfs/props.c | 7 +
fs/btrfs/qgroup.c | 223 +++++++++++++-----
fs/btrfs/qgroup.h | 9 +-
fs/btrfs/raid56.c | 16 +-
fs/btrfs/reada.c | 1 -
fs/btrfs/relocation.c | 15 +-
fs/btrfs/root-tree.c | 7 +
fs/btrfs/scrub.c | 209 +++++++----------
fs/btrfs/send.c | 112 ++++++---
fs/btrfs/super.c | 72 +-----
fs/btrfs/sysfs.c | 41 ++++
fs/btrfs/tests/extent-io-tests.c | 2 +-
fs/btrfs/transaction.c | 23 +-
fs/btrfs/tree-log.c | 44 +++-
fs/btrfs/volumes.c | 74 +++---
fs/btrfs/volumes.h | 7 +
fs/btrfs/xattr.c | 2 +-
fs/btrfs/zlib.c | 20 +-
include/trace/events/btrfs.h | 36 ---
include/uapi/linux/btrfs.h | 63 +++--
43 files changed, 1682 insertions(+), 1353 deletions(-)
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PULL] Btrfs for 4.13, part 1
2017-06-23 15:16 [PULL] Btrfs for 4.13, part 1 David Sterba
@ 2017-06-24 10:05 ` Wang Shilong
2017-06-25 21:34 ` Jeff Mahoney
2017-06-28 14:28 ` David Sterba
2017-06-26 13:40 ` [PULL] Btrfs for 4.13, part 1 (update 1) David Sterba
` (2 subsequent siblings)
3 siblings, 2 replies; 13+ messages in thread
From: Wang Shilong @ 2017-06-24 10:05 UTC (permalink / raw)
To: David Sterba; +Cc: Chris Mason, linux-btrfs
Sorry for bikeshedding.
On Fri, Jun 23, 2017 at 11:16 PM, David Sterba <dsterba@suse.com> wrote:
> Hi,
>
> this is the main batch for 4.13. There are some user visible changes, see
> below. The core updates improve error handling (mostly related to bios), with
> the usual incremental work on the GFP_NOFS (mis)use removal. All patches have
>
> Fabian Frederick (1):
> btrfs: kmap() can't fail
>
<..SNIP..>
>
> Sargun Dhillon (2):
> btrfs: add quota override flag to enable quota override for CAP_SYS_RESOURCE
> btrfs: Add quota_override knob into sysfs
>
> Su Yue (9):
> btrfs: Introduce btrfs_is_name_len_valid to avoid reading beyond boundary
> btrfs: Check name_len with boundary in verify dir_item
> btrfs: Check name_len on add_inode_ref call path
> btrfs: Verify dir_item in replay_xattr_deletes
> btrfs: Check name_len in btrfs_check_ref_name_override
> btrfs: Check name_len before read in iterate_dir_item
> btrfs: Check name_len before reading btrfs_get_name
> btrfs: Check name_len before in btrfs_del_root_ref
> btrfs: Verify dir_item in iterate_object_props
Hmm..add those check might be expensive for metadata operations,
especially in hot path, i could see similar behavior in Ext4 for ext4
dentry check.
Could we run some metadata tests to confirm？it makes sense to
add check but with min affects to performace.
Thanks,
Shilong
> Timofey Titovets (3):
> Btrfs: lzo: fix typo in error message after failed deflate
> Btrfs: lzo: compressed data size must be less then input size
> Btrfs: compression must free at least one sector size
>
> Yonghong Song (1):
> Btrfs: add statx support
>
> fs/btrfs/backref.c | 10 +-
> fs/btrfs/check-integrity.c | 53 ++---
> fs/btrfs/compression.c | 94 ++------
> fs/btrfs/compression.h | 44 +++-
> fs/btrfs/ctree.c | 42 ++--
> fs/btrfs/ctree.h | 84 ++++---
> fs/btrfs/delayed-ref.c | 29 ++-
> fs/btrfs/delayed-ref.h | 6 +-
> fs/btrfs/dir-item.c | 94 +++++++-
> fs/btrfs/disk-io.c | 179 +++++++--------
> fs/btrfs/disk-io.h | 8 +-
> fs/btrfs/export.c | 5 +
> fs/btrfs/extent-tree.c | 481 +++++++++++++++++++++------------------
> fs/btrfs/extent_io.c | 217 ++++++++----------
> fs/btrfs/extent_io.h | 82 +++++--
> fs/btrfs/file-item.c | 31 ++-
> fs/btrfs/file.c | 46 ++--
> fs/btrfs/free-space-tree.c | 38 ++--
> fs/btrfs/inode-map.c | 4 +-
> fs/btrfs/inode.c | 449 ++++++++++++++++++++----------------
> fs/btrfs/ioctl.c | 16 +-
> fs/btrfs/lzo.c | 33 +--
> fs/btrfs/print-tree.c | 7 +-
> fs/btrfs/props.c | 7 +
> fs/btrfs/qgroup.c | 223 +++++++++++++-----
> fs/btrfs/qgroup.h | 9 +-
> fs/btrfs/raid56.c | 16 +-
> fs/btrfs/reada.c | 1 -
> fs/btrfs/relocation.c | 15 +-
> fs/btrfs/root-tree.c | 7 +
> fs/btrfs/scrub.c | 209 +++++++----------
> fs/btrfs/send.c | 112 ++++++---
> fs/btrfs/super.c | 72 +-----
> fs/btrfs/sysfs.c | 41 ++++
> fs/btrfs/tests/extent-io-tests.c | 2 +-
> fs/btrfs/transaction.c | 23 +-
> fs/btrfs/tree-log.c | 44 +++-
> fs/btrfs/volumes.c | 74 +++---
> fs/btrfs/volumes.h | 7 +
> fs/btrfs/xattr.c | 2 +-
> fs/btrfs/zlib.c | 20 +-
> include/trace/events/btrfs.h | 36 ---
> include/uapi/linux/btrfs.h | 63 +++--
> 43 files changed, 1682 insertions(+), 1353 deletions(-)
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PULL] Btrfs for 4.13, part 1
2017-06-24 10:05 ` Wang Shilong
@ 2017-06-25 21:34 ` Jeff Mahoney
2017-06-26 0:53 ` Qu Wenruo
2017-06-28 14:28 ` David Sterba
1 sibling, 1 reply; 13+ messages in thread
From: Jeff Mahoney @ 2017-06-25 21:34 UTC (permalink / raw)
To: Wang Shilong, David Sterba; +Cc: Chris Mason, linux-btrfs
[-- Attachment #1.1: Type: text/plain, Size: 4900 bytes --]
On 6/24/17 6:05 AM, Wang Shilong wrote:
> Sorry for bikeshedding.
>
> On Fri, Jun 23, 2017 at 11:16 PM, David Sterba <dsterba@suse.com> wrote:
>> Hi,
>>
>> this is the main batch for 4.13. There are some user visible changes, see
>> below. The core updates improve error handling (mostly related to bios), with
>> the usual incremental work on the GFP_NOFS (mis)use removal. All patches have
>>
>> Fabian Frederick (1):
>> btrfs: kmap() can't fail
>>
> <..SNIP..>
>>
>> Sargun Dhillon (2):
>> btrfs: add quota override flag to enable quota override for CAP_SYS_RESOURCE
>> btrfs: Add quota_override knob into sysfs
>>
>> Su Yue (9):
>> btrfs: Introduce btrfs_is_name_len_valid to avoid reading beyond boundary
>> btrfs: Check name_len with boundary in verify dir_item
>> btrfs: Check name_len on add_inode_ref call path
>> btrfs: Verify dir_item in replay_xattr_deletes
>> btrfs: Check name_len in btrfs_check_ref_name_override
>> btrfs: Check name_len before read in iterate_dir_item
>> btrfs: Check name_len before reading btrfs_get_name
>> btrfs: Check name_len before in btrfs_del_root_ref
>> btrfs: Verify dir_item in iterate_object_props
>
> Hmm..add those check might be expensive for metadata operations,
> especially in hot path, i could see similar behavior in Ext4 for ext4
> dentry check.
>
> Could we run some metadata tests to confirm？it makes sense to
> add check but with min affects to performace.
Agreed. XFS does this at read/write and results in invalid entries
never even making it to the consumer. That's the approach I had
partially written up. This approach has the advantage of being more
fine-grained so we don't end up dropping e.g. an entire node's worth of
entries but is more expensive to maintain and at runtime.
-Jeff
>> Timofey Titovets (3):
>> Btrfs: lzo: fix typo in error message after failed deflate
>> Btrfs: lzo: compressed data size must be less then input size
>> Btrfs: compression must free at least one sector size
>>
>> Yonghong Song (1):
>> Btrfs: add statx support
>>
>> fs/btrfs/backref.c | 10 +-
>> fs/btrfs/check-integrity.c | 53 ++---
>> fs/btrfs/compression.c | 94 ++------
>> fs/btrfs/compression.h | 44 +++-
>> fs/btrfs/ctree.c | 42 ++--
>> fs/btrfs/ctree.h | 84 ++++---
>> fs/btrfs/delayed-ref.c | 29 ++-
>> fs/btrfs/delayed-ref.h | 6 +-
>> fs/btrfs/dir-item.c | 94 +++++++-
>> fs/btrfs/disk-io.c | 179 +++++++--------
>> fs/btrfs/disk-io.h | 8 +-
>> fs/btrfs/export.c | 5 +
>> fs/btrfs/extent-tree.c | 481 +++++++++++++++++++++------------------
>> fs/btrfs/extent_io.c | 217 ++++++++----------
>> fs/btrfs/extent_io.h | 82 +++++--
>> fs/btrfs/file-item.c | 31 ++-
>> fs/btrfs/file.c | 46 ++--
>> fs/btrfs/free-space-tree.c | 38 ++--
>> fs/btrfs/inode-map.c | 4 +-
>> fs/btrfs/inode.c | 449 ++++++++++++++++++++----------------
>> fs/btrfs/ioctl.c | 16 +-
>> fs/btrfs/lzo.c | 33 +--
>> fs/btrfs/print-tree.c | 7 +-
>> fs/btrfs/props.c | 7 +
>> fs/btrfs/qgroup.c | 223 +++++++++++++-----
>> fs/btrfs/qgroup.h | 9 +-
>> fs/btrfs/raid56.c | 16 +-
>> fs/btrfs/reada.c | 1 -
>> fs/btrfs/relocation.c | 15 +-
>> fs/btrfs/root-tree.c | 7 +
>> fs/btrfs/scrub.c | 209 +++++++----------
>> fs/btrfs/send.c | 112 ++++++---
>> fs/btrfs/super.c | 72 +-----
>> fs/btrfs/sysfs.c | 41 ++++
>> fs/btrfs/tests/extent-io-tests.c | 2 +-
>> fs/btrfs/transaction.c | 23 +-
>> fs/btrfs/tree-log.c | 44 +++-
>> fs/btrfs/volumes.c | 74 +++---
>> fs/btrfs/volumes.h | 7 +
>> fs/btrfs/xattr.c | 2 +-
>> fs/btrfs/zlib.c | 20 +-
>> include/trace/events/btrfs.h | 36 ---
>> include/uapi/linux/btrfs.h | 63 +++--
>> 43 files changed, 1682 insertions(+), 1353 deletions(-)
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at http://vger.kernel.org/majordomo-info.html
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
--
Jeff Mahoney
SUSE Labs
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PULL] Btrfs for 4.13, part 1
2017-06-25 21:34 ` Jeff Mahoney
@ 2017-06-26 0:53 ` Qu Wenruo
2017-06-26 1:46 ` Jeff Mahoney
2017-06-28 14:38 ` David Sterba
0 siblings, 2 replies; 13+ messages in thread
From: Qu Wenruo @ 2017-06-26 0:53 UTC (permalink / raw)
To: Jeff Mahoney, Wang Shilong, David Sterba; +Cc: Chris Mason, linux-btrfs
At 06/26/2017 05:34 AM, Jeff Mahoney wrote:
> On 6/24/17 6:05 AM, Wang Shilong wrote:
>> Sorry for bikeshedding.
>>
>> On Fri, Jun 23, 2017 at 11:16 PM, David Sterba <dsterba@suse.com> wrote:
>>> Hi,
>>>
>>> this is the main batch for 4.13. There are some user visible changes, see
>>> below. The core updates improve error handling (mostly related to bios), with
>>> the usual incremental work on the GFP_NOFS (mis)use removal. All patches have
>>>
>>> Fabian Frederick (1):
>>> btrfs: kmap() can't fail
>>>
>> <..SNIP..>
>>>
>>> Sargun Dhillon (2):
>>> btrfs: add quota override flag to enable quota override for CAP_SYS_RESOURCE
>>> btrfs: Add quota_override knob into sysfs
>>>
>>> Su Yue (9):
>>> btrfs: Introduce btrfs_is_name_len_valid to avoid reading beyond boundary
>>> btrfs: Check name_len with boundary in verify dir_item
>>> btrfs: Check name_len on add_inode_ref call path
>>> btrfs: Verify dir_item in replay_xattr_deletes
>>> btrfs: Check name_len in btrfs_check_ref_name_override
>>> btrfs: Check name_len before read in iterate_dir_item
>>> btrfs: Check name_len before reading btrfs_get_name
>>> btrfs: Check name_len before in btrfs_del_root_ref
>>> btrfs: Verify dir_item in iterate_object_props
>>
>> Hmm..add those check might be expensive for metadata operations,
>> especially in hot path, i could see similar behavior in Ext4 for ext4
>> dentry check.
>>
>> Could we run some metadata tests to confirm？it makes sense to
>> add check but with min affects to performace.
IIRC there is a backtrace in one of the patches.
If needed, we can also upload the fuzzed image.
Although we don't have good enough test suite which accepts fuzzed image
for kernel.
>
> Agreed. XFS does this at read/write and results in invalid entries
> never even making it to the consumer. That's the approach I had
> partially written up. This approach has the advantage of being more
> fine-grained so we don't end up dropping e.g. an entire node's worth of
> entries but is more expensive to maintain and at runtime.
We already have tons of runtime check at chunk/super block read time.
This new check is not as comprehensive as what we did in
chunk/superblock read time, but only to ensure that item with variable
length doesn't cross its boundary.
So performance wise it should not be a problem.
Although it's expensive to maintain, as for each structure with variable
length, we need to call verification function every time.
But we can also extract the check to leaf reading time, this should
reduce the effort to maintain it and make it easier to expand (or even
make it optional if it really affects performance).
Thanks,
Qu
>
> -Jeff
>
>>> Timofey Titovets (3):
>>> Btrfs: lzo: fix typo in error message after failed deflate
>>> Btrfs: lzo: compressed data size must be less then input size
>>> Btrfs: compression must free at least one sector size
>>>
>>> Yonghong Song (1):
>>> Btrfs: add statx support
>>>
>>> fs/btrfs/backref.c | 10 +-
>>> fs/btrfs/check-integrity.c | 53 ++---
>>> fs/btrfs/compression.c | 94 ++------
>>> fs/btrfs/compression.h | 44 +++-
>>> fs/btrfs/ctree.c | 42 ++--
>>> fs/btrfs/ctree.h | 84 ++++---
>>> fs/btrfs/delayed-ref.c | 29 ++-
>>> fs/btrfs/delayed-ref.h | 6 +-
>>> fs/btrfs/dir-item.c | 94 +++++++-
>>> fs/btrfs/disk-io.c | 179 +++++++--------
>>> fs/btrfs/disk-io.h | 8 +-
>>> fs/btrfs/export.c | 5 +
>>> fs/btrfs/extent-tree.c | 481 +++++++++++++++++++++------------------
>>> fs/btrfs/extent_io.c | 217 ++++++++----------
>>> fs/btrfs/extent_io.h | 82 +++++--
>>> fs/btrfs/file-item.c | 31 ++-
>>> fs/btrfs/file.c | 46 ++--
>>> fs/btrfs/free-space-tree.c | 38 ++--
>>> fs/btrfs/inode-map.c | 4 +-
>>> fs/btrfs/inode.c | 449 ++++++++++++++++++++----------------
>>> fs/btrfs/ioctl.c | 16 +-
>>> fs/btrfs/lzo.c | 33 +--
>>> fs/btrfs/print-tree.c | 7 +-
>>> fs/btrfs/props.c | 7 +
>>> fs/btrfs/qgroup.c | 223 +++++++++++++-----
>>> fs/btrfs/qgroup.h | 9 +-
>>> fs/btrfs/raid56.c | 16 +-
>>> fs/btrfs/reada.c | 1 -
>>> fs/btrfs/relocation.c | 15 +-
>>> fs/btrfs/root-tree.c | 7 +
>>> fs/btrfs/scrub.c | 209 +++++++----------
>>> fs/btrfs/send.c | 112 ++++++---
>>> fs/btrfs/super.c | 72 +-----
>>> fs/btrfs/sysfs.c | 41 ++++
>>> fs/btrfs/tests/extent-io-tests.c | 2 +-
>>> fs/btrfs/transaction.c | 23 +-
>>> fs/btrfs/tree-log.c | 44 +++-
>>> fs/btrfs/volumes.c | 74 +++---
>>> fs/btrfs/volumes.h | 7 +
>>> fs/btrfs/xattr.c | 2 +-
>>> fs/btrfs/zlib.c | 20 +-
>>> include/trace/events/btrfs.h | 36 ---
>>> include/uapi/linux/btrfs.h | 63 +++--
>>> 43 files changed, 1682 insertions(+), 1353 deletions(-)
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
>>> the body of a message to majordomo@vger.kernel.org
>>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>>
>
>
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PULL] Btrfs for 4.13, part 1
2017-06-26 0:53 ` Qu Wenruo
@ 2017-06-26 1:46 ` Jeff Mahoney
2017-06-28 14:38 ` David Sterba
1 sibling, 0 replies; 13+ messages in thread
From: Jeff Mahoney @ 2017-06-26 1:46 UTC (permalink / raw)
To: Qu Wenruo, Wang Shilong, David Sterba; +Cc: Chris Mason, linux-btrfs
[-- Attachment #1.1: Type: text/plain, Size: 6569 bytes --]
On 6/25/17 8:53 PM, Qu Wenruo wrote:
>
>
> At 06/26/2017 05:34 AM, Jeff Mahoney wrote:
>> On 6/24/17 6:05 AM, Wang Shilong wrote:
>>> Sorry for bikeshedding.
>>>
>>> On Fri, Jun 23, 2017 at 11:16 PM, David Sterba <dsterba@suse.com> wrote:
>>>> Hi,
>>>>
>>>> this is the main batch for 4.13. There are some user visible
>>>> changes, see
>>>> below. The core updates improve error handling (mostly related to
>>>> bios), with
>>>> the usual incremental work on the GFP_NOFS (mis)use removal. All
>>>> patches have
>>>>
>>>> Fabian Frederick (1):
>>>> btrfs: kmap() can't fail
>>>>
>>> <..SNIP..>
>>>>
>>>> Sargun Dhillon (2):
>>>> btrfs: add quota override flag to enable quota override for
>>>> CAP_SYS_RESOURCE
>>>> btrfs: Add quota_override knob into sysfs
>>>>
>>>> Su Yue (9):
>>>> btrfs: Introduce btrfs_is_name_len_valid to avoid reading
>>>> beyond boundary
>>>> btrfs: Check name_len with boundary in verify dir_item
>>>> btrfs: Check name_len on add_inode_ref call path
>>>> btrfs: Verify dir_item in replay_xattr_deletes
>>>> btrfs: Check name_len in btrfs_check_ref_name_override
>>>> btrfs: Check name_len before read in iterate_dir_item
>>>> btrfs: Check name_len before reading btrfs_get_name
>>>> btrfs: Check name_len before in btrfs_del_root_ref
>>>> btrfs: Verify dir_item in iterate_object_props
>>>
>>> Hmm..add those check might be expensive for metadata operations,
>>> especially in hot path, i could see similar behavior in Ext4 for ext4
>>> dentry check.
>>>
>>> Could we run some metadata tests to confirm？it makes sense to
>>> add check but with min affects to performace.
>
> IIRC there is a backtrace in one of the patches.
> If needed, we can also upload the fuzzed image.
> Although we don't have good enough test suite which accepts fuzzed image
> for kernel.
>
>>
>> Agreed. XFS does this at read/write and results in invalid entries
>> never even making it to the consumer. That's the approach I had
>> partially written up. This approach has the advantage of being more
>> fine-grained so we don't end up dropping e.g. an entire node's worth of
>> entries but is more expensive to maintain and at runtime.
>
> We already have tons of runtime check at chunk/super block read time.
>
> This new check is not as comprehensive as what we did in
> chunk/superblock read time, but only to ensure that item with variable
> length doesn't cross its boundary.
> So performance wise it should not be a problem.
>
> Although it's expensive to maintain, as for each structure with variable
> length, we need to call verification function every time.
>
> But we can also extract the check to leaf reading time, this should
> reduce the effort to maintain it and make it easier to expand (or even
> make it optional if it really affects performance).
Well, ideally, we'd do complete checks for any type that we have enough
information to check at read or write time. If a check is important
enough to be done at a consumer site, it's important enough to be done
at read/write. Then we can make the consumer code much simpler and
catch some forms of corruption before it hits the disk.
-Jeff
> Thanks,
> Qu
>
>>
>> -Jeff
>>
>>>> Timofey Titovets (3):
>>>> Btrfs: lzo: fix typo in error message after failed deflate
>>>> Btrfs: lzo: compressed data size must be less then input size
>>>> Btrfs: compression must free at least one sector size
>>>>
>>>> Yonghong Song (1):
>>>> Btrfs: add statx support
>>>>
>>>> fs/btrfs/backref.c | 10 +-
>>>> fs/btrfs/check-integrity.c | 53 ++---
>>>> fs/btrfs/compression.c | 94 ++------
>>>> fs/btrfs/compression.h | 44 +++-
>>>> fs/btrfs/ctree.c | 42 ++--
>>>> fs/btrfs/ctree.h | 84 ++++---
>>>> fs/btrfs/delayed-ref.c | 29 ++-
>>>> fs/btrfs/delayed-ref.h | 6 +-
>>>> fs/btrfs/dir-item.c | 94 +++++++-
>>>> fs/btrfs/disk-io.c | 179 +++++++--------
>>>> fs/btrfs/disk-io.h | 8 +-
>>>> fs/btrfs/export.c | 5 +
>>>> fs/btrfs/extent-tree.c | 481
>>>> +++++++++++++++++++++------------------
>>>> fs/btrfs/extent_io.c | 217 ++++++++----------
>>>> fs/btrfs/extent_io.h | 82 +++++--
>>>> fs/btrfs/file-item.c | 31 ++-
>>>> fs/btrfs/file.c | 46 ++--
>>>> fs/btrfs/free-space-tree.c | 38 ++--
>>>> fs/btrfs/inode-map.c | 4 +-
>>>> fs/btrfs/inode.c | 449
>>>> ++++++++++++++++++++----------------
>>>> fs/btrfs/ioctl.c | 16 +-
>>>> fs/btrfs/lzo.c | 33 +--
>>>> fs/btrfs/print-tree.c | 7 +-
>>>> fs/btrfs/props.c | 7 +
>>>> fs/btrfs/qgroup.c | 223 +++++++++++++-----
>>>> fs/btrfs/qgroup.h | 9 +-
>>>> fs/btrfs/raid56.c | 16 +-
>>>> fs/btrfs/reada.c | 1 -
>>>> fs/btrfs/relocation.c | 15 +-
>>>> fs/btrfs/root-tree.c | 7 +
>>>> fs/btrfs/scrub.c | 209 +++++++----------
>>>> fs/btrfs/send.c | 112 ++++++---
>>>> fs/btrfs/super.c | 72 +-----
>>>> fs/btrfs/sysfs.c | 41 ++++
>>>> fs/btrfs/tests/extent-io-tests.c | 2 +-
>>>> fs/btrfs/transaction.c | 23 +-
>>>> fs/btrfs/tree-log.c | 44 +++-
>>>> fs/btrfs/volumes.c | 74 +++---
>>>> fs/btrfs/volumes.h | 7 +
>>>> fs/btrfs/xattr.c | 2 +-
>>>> fs/btrfs/zlib.c | 20 +-
>>>> include/trace/events/btrfs.h | 36 ---
>>>> include/uapi/linux/btrfs.h | 63 +++--
>>>> 43 files changed, 1682 insertions(+), 1353 deletions(-)
>>>> --
>>>> To unsubscribe from this list: send the line "unsubscribe
>>>> linux-btrfs" in
>>>> the body of a message to majordomo@vger.kernel.org
>>>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe
>>> linux-btrfs" in
>>> the body of a message to majordomo@vger.kernel.org
>>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>>>
>>
>>
>
>
>
--
Jeff Mahoney
SUSE Labs
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PULL] Btrfs for 4.13, part 1 (update 1)
2017-06-23 15:16 [PULL] Btrfs for 4.13, part 1 David Sterba
2017-06-24 10:05 ` Wang Shilong
@ 2017-06-26 13:40 ` David Sterba
2017-06-26 16:58 ` [PULL] Btrfs for 4.13, part 1 Chris Mason
2017-06-28 6:49 ` Nikolay Borisov
3 siblings, 0 replies; 13+ messages in thread
From: David Sterba @ 2017-06-26 13:40 UTC (permalink / raw)
To: David Sterba; +Cc: clm, linux-btrfs
On Fri, Jun 23, 2017 at 05:16:46PM +0200, David Sterba wrote:
Two more patches added to the branch
Chris Mason (1):
btrfs: fix integer overflow in calc_reclaim_items_nr
David Sterba (1):
btrfs: scrub: fix target device intialization while setting up scrub context
Updated branch and tag:
----------------------------------------------------------------
The following changes since commit 41f1830f5a7af77cf5c86359aba3cbd706687e52:
Linux 4.12-rc6 (2017-06-19 22:19:37 +0800)
are available in the git repository at:
git://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux.git for-4.13-part1
for you to fetch changes up to 8399f53f0c7450ab050b1b0ffee4e2c1ddd2a3e0:
btrfs: fix integer overflow in calc_reclaim_items_nr (2017-06-26 15:33:42 +0200)
----------------------------------------------------------------
Previous:
> ----------------------------------------------------------------
> The following changes since commit 41f1830f5a7af77cf5c86359aba3cbd706687e52:
>
> Linux 4.12-rc6 (2017-06-19 22:19:37 +0800)
>
> are available in the git repository at:
>
> git://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux.git for-4.13-part1
>
> for you to fetch changes up to f3f000297be88b1b75fde5027d660a8d8a44de14:
>
> btrfs: qgroup: Fix qgroup reserved space underflow by only freeing reserved ranges (2017-06-21 20:56:14 +0200)
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PULL] Btrfs for 4.13, part 1
2017-06-23 15:16 [PULL] Btrfs for 4.13, part 1 David Sterba
2017-06-24 10:05 ` Wang Shilong
2017-06-26 13:40 ` [PULL] Btrfs for 4.13, part 1 (update 1) David Sterba
@ 2017-06-26 16:58 ` Chris Mason
2017-06-27 11:00 ` Filipe Manana
2017-06-28 6:49 ` Nikolay Borisov
3 siblings, 1 reply; 13+ messages in thread
From: Chris Mason @ 2017-06-26 16:58 UTC (permalink / raw)
To: David Sterba; +Cc: linux-btrfs
On 06/23/2017 11:16 AM, David Sterba wrote:
> Hi,
>
> this is the main batch for 4.13. There are some user visible changes, see
> below. The core updates improve error handling (mostly related to bios), with
> the usual incremental work on the GFP_NOFS (mis)use removal. All patches have
> been in for-next for an extensive amount of time.
>
> Thre will be followups but I want push the series (111 patches) forward. There
> are also some updates to adjacent subsystems (writeback and blocklayer), so I
> want to give some stable point for merging in the upcoming weeks.
Thanks Dave, I ran this (along with the updates we added) through a long
stress and the usual xfstests.
For everyone else on the list, since I'm heading off to vacation until
~July 9th, Dave is sending this off to Linus once the merge window starts.
-chris
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PULL] Btrfs for 4.13, part 1
2017-06-26 16:58 ` [PULL] Btrfs for 4.13, part 1 Chris Mason
@ 2017-06-27 11:00 ` Filipe Manana
2017-06-28 15:39 ` David Sterba
0 siblings, 1 reply; 13+ messages in thread
From: Filipe Manana @ 2017-06-27 11:00 UTC (permalink / raw)
To: Chris Mason; +Cc: David Sterba, linux-btrfs
On Mon, Jun 26, 2017 at 5:58 PM, Chris Mason <clm@fb.com> wrote:
> On 06/23/2017 11:16 AM, David Sterba wrote:
>>
>> Hi,
>>
>> this is the main batch for 4.13. There are some user visible changes, see
>> below. The core updates improve error handling (mostly related to bios),
>> with
>> the usual incremental work on the GFP_NOFS (mis)use removal. All patches
>> have
>> been in for-next for an extensive amount of time.
>>
>> Thre will be followups but I want push the series (111 patches) forward.
>> There
>> are also some updates to adjacent subsystems (writeback and blocklayer),
>> so I
>> want to give some stable point for merging in the upcoming weeks.
>
>
> Thanks Dave, I ran this (along with the updates we added) through a long
> stress and the usual xfstests.
Have any of you run xfstests without failures?
I'm finding it hard to believe, as some of these changes cause
problems all the time, and at least one of them deterministically
("btrfs: fix validation of XATTR_ITEM dir items"), as I've just noted
in the respective thread for that patch.
I haven't traced which change is causing a BUG_ON in the raid56 code,
due to lack of time, but running btrfs tests from 060 to 074 triggers
it very easily, for example:
[ 6086.912567] run fstests btrfs/064 at 2017-06-26 21:10:55
[ 6089.821472] BTRFS: device fsid 26f67507-c01d-4101-8435-2299cddab311
devid 1 transid 5 /dev/sdc
[ 6089.913286] BTRFS: device fsid 26f67507-c01d-4101-8435-2299cddab311
devid 2 transid 5 /dev/sdd
[ 6089.966903] BTRFS: device fsid 26f67507-c01d-4101-8435-2299cddab311
devid 3 transid 5 /dev/sde
[ 6090.065205] BTRFS: device fsid 26f67507-c01d-4101-8435-2299cddab311
devid 4 transid 5 /dev/sdf
[ 6090.873020] BTRFS info (device sdf): turning on discard
[ 6090.874452] BTRFS info (device sdf): disk space caching is enabled
[ 6090.893146] BTRFS info (device sdf): has skinny extents
[ 6091.110544] BTRFS info (device sdf): creating UUID tree
[ 6093.459619] BTRFS info (device sdf): dev_replace from /dev/sdd
(devid 2) to /dev/sdg started
[ 6099.046000] BTRFS info (device sdf): dev_replace from /dev/sdd
(devid 2) to /dev/sdg finished
[ 6101.221892] BTRFS info (device sdf): dev_replace from /dev/sde
(devid 3) to /dev/sdd started
[ 6104.224275] BTRFS info (device sdf): dev_replace from /dev/sde
(devid 3) to /dev/sdd finished
[ 6104.610774] BTRFS info (device sdf): dev_replace from /dev/sdf
(devid 4) to /dev/sde started
[ 6105.621356] BTRFS info (device sdf): dev_replace from /dev/sdf
(devid 4) to /dev/sde finished
[ 6111.693426] BTRFS info (device sdf): relocating block group
3242196992 flags data
[ 6111.731322] BTRFS info (device sdf): found 1 extents
[ 6111.762907] BTRFS info (device sdf): found 1 extents
[ 6111.785554] BTRFS info (device sdf): relocating block group
2168455168 flags metadata
[ 6111.823810] BTRFS info (device sdf): relocating block group
1094713344 flags data
[ 6111.974995] BTRFS info (device sdf): found 209 extents
[ 6112.471755] BTRFS info (device sdf): found 209 extents
[ 6112.485584] BTRFS info (device sdf): relocating block group
20971520 flags metadata
[ 6112.527924] BTRFS info (device sdf): found 2 extents
[ 6112.535532] BTRFS info (device sdf): relocating block group
12582912 flags data
[ 6112.585868] BTRFS info (device sdf): found 92 extents
[ 6112.838224] BTRFS info (device sdf): found 92 extents
[ 6112.847285] BTRFS info (device sdf): relocating block group 4194304
flags metadata
[ 6112.963176] BTRFS info (device sdf): found 116 extents
[ 6112.986034] BTRFS info (device sdf): relocating block group 0 flags system
[ 6113.013839] BTRFS info (device sdf): found 1 extents
[ 6114.496170] blk_update_request: I/O error, dev fd0, sector 0
[ 6114.497343] floppy: error -5 while reading block 0
[ 6114.700151] blk_update_request: I/O error, dev fd0, sector 0
[ 6114.701047] floppy: error -5 while reading block 0
[ 6114.975828] BTRFS: device fsid 7ea53fb0-9bc6-48e7-90ed-3d75bf00c180
devid 3 transid 5 /dev/sde
[ 6115.047448] BTRFS: device fsid 7ea53fb0-9bc6-48e7-90ed-3d75bf00c180
devid 1 transid 5 /dev/sdc
[ 6115.080295] BTRFS: device fsid 7ea53fb0-9bc6-48e7-90ed-3d75bf00c180
devid 2 transid 5 /dev/sdd
[ 6115.225944] BTRFS: device fsid 7ea53fb0-9bc6-48e7-90ed-3d75bf00c180
devid 4 transid 5 /dev/sdf
[ 6115.525122] BTRFS info (device sdf): turning on discard
[ 6115.525122] BTRFS info (device sdf): disk space caching is enabled
[ 6115.541244] BTRFS info (device sdf): has skinny extents
[ 6115.653520] BTRFS info (device sdf): creating UUID tree
[ 6115.744243] systemd-journald[618]: Data hash table of
/run/log/journal/bba104ad3dc06c1bd2dc3d7c546a5b43/system.journal has a
fill level at 75.0 (26843 of 35790 items, 16777216 file size, 625
bytes per hash table item), suggesting rotation.
[ 6115.813742] systemd-journald[618]:
/run/log/journal/bba104ad3dc06c1bd2dc3d7c546a5b43/system.journal:
Journal header limits reached or header out-of-date, rotating.
[ 6115.820095] systemd-journald[618]: Rotating...
[ 6115.841292] systemd-journald[618]: Reserving 35790 entries in hash table.
[ 6115.854029] systemd-journald[618]: Vacuuming...
[ 6115.855641] systemd-journald[618]: Vacuuming done, freed 0B of
archived journals on disk.
[ 6116.428897] blk_update_request: I/O error, dev fd0, sector 0
[ 6116.432844] floppy: error -5 while reading block 0
[ 6121.422099] BTRFS info (device sdf): dev_replace from /dev/sdd
(devid 2) to /dev/sdg started
[ 6131.042683] BTRFS info (device sdf): dev_replace from /dev/sdd
(devid 2) to /dev/sdg finished
[ 6136.197456] BTRFS info (device sdf): relocating block group
9768534016 flags data|raid0
[ 6136.338019] BTRFS info (device sdf): found 4 extents
[ 6136.444110] BTRFS info (device sdf): found 4 extents
[ 6136.599240] BTRFS info (device sdf): relocating block group
8694792192 flags metadata|raid0
[ 6136.766634] BTRFS info (device sdf): found 43 extents
[ 6136.854219] BTRFS info (device sdf): relocating block group
8627683328 flags system|raid0
[ 6136.956588] BTRFS info (device sdf): relocating block group
4332716032 flags data|raid0
[ 6143.223627] BTRFS info (device sdf): found 268 extents
[ 6143.225358] systemd-journald[618]: Sent WATCHDOG=1 notification.
[ 6143.657919] BTRFS info (device sdf): found 268 extents
[ 6143.744224] BTRFS info (device sdf): relocating block group
37748736 flags metadata|raid0
[ 6143.867552] BTRFS info (device sdf): found 102 extents
[ 6143.924525] BTRFS info (device sdf): relocating block group
20971520 flags system|raid0
[ 6143.983050] BTRFS info (device sdf): found 1 extents
[ 6145.424161] blk_update_request: I/O error, dev fd0, sector 0
[ 6145.425595] floppy: error -5 while reading block 0
[ 6145.628137] blk_update_request: I/O error, dev fd0, sector 0
[ 6145.628960] floppy: error -5 while reading block 0
[ 6147.044939] BTRFS: device fsid 94157252-d25f-4c1c-9d26-8b7729d7bc5a
devid 1 transid 5 /dev/sdc
[ 6147.049400] BTRFS: device fsid 94157252-d25f-4c1c-9d26-8b7729d7bc5a
devid 2 transid 5 /dev/sdd
[ 6147.086296] BTRFS: device fsid 94157252-d25f-4c1c-9d26-8b7729d7bc5a
devid 3 transid 5 /dev/sde
[ 6147.130074] BTRFS: device fsid 94157252-d25f-4c1c-9d26-8b7729d7bc5a
devid 4 transid 5 /dev/sdf
[ 6147.853052] BTRFS info (device sdf): turning on discard
[ 6147.854682] BTRFS info (device sdf): disk space caching is enabled
[ 6147.873477] BTRFS info (device sdf): has skinny extents
[ 6148.285495] BTRFS info (device sdf): creating UUID tree
[ 6149.101071] blk_update_request: I/O error, dev fd0, sector 0
[ 6149.104974] floppy: error -5 while reading block 0
[ 6151.577159] BTRFS info (device sdf): dev_replace from /dev/sdd
(devid 2) to /dev/sdg started
[ 6241.821804] systemd-journald[618]: Sent WATCHDOG=1 notification.
[ 6306.238550] BTRFS info (device sdf): dev_replace from /dev/sdd
(devid 2) to /dev/sdg finished
[ 6320.201287] systemd-journald[618]: Sent WATCHDOG=1 notification.
[ 6334.496060] blk_update_request: I/O error, dev fd0, sector 0
[ 6334.496849] floppy: error -5 while reading block 0
[ 6368.344060] blk_update_request: I/O error, dev fd0, sector 0
[ 6368.344850] floppy: error -5 while reading block 0
[ 6405.872088] BTRFS: device fsid 3b456cae-f946-4422-8b69-3e562660d3ca
devid 1 transid 5 /dev/sdc
[ 6407.056284] BTRFS: device fsid 3b456cae-f946-4422-8b69-3e562660d3ca
devid 2 transid 5 /dev/sdd
[ 6409.036854] BTRFS: device fsid 3b456cae-f946-4422-8b69-3e562660d3ca
devid 3 transid 5 /dev/sde
[ 6409.061008] systemd-journald[618]: Sent WATCHDOG=1 notification.
[ 6410.988164] BTRFS: device fsid 3b456cae-f946-4422-8b69-3e562660d3ca
devid 4 transid 5 /dev/sdf
[ 6414.503001] BTRFS info (device sdf): turning on discard
[ 6414.542595] BTRFS info (device sdf): disk space caching is enabled
[ 6414.570512] BTRFS info (device sdf): has skinny extents
[ 6415.488940] BTRFS info (device sdf): creating UUID tree
[ 6420.048032] blk_update_request: I/O error, dev fd0, sector 0
[ 6420.051396] floppy: error -5 while reading block 0
[ 6466.781846] BTRFS info (device sdf): relocating block group
1103101952 flags data|raid1
[ 6499.163148] systemd-journald[618]: Sent WATCHDOG=1 notification.
[ 6500.988823] BTRFS info (device sdf): found 163 extents
[ 6520.228862] BTRFS info (device sdf): found 155 extents
[ 6526.150033] BTRFS info (device sdf): relocating block group
29360128 flags metadata|raid1
[ 6543.788137] BTRFS info (device sdf): found 94 extents
[ 6548.960587] BTRFS info (device sdf): relocating block group
20971520 flags system|raid1
[ 6554.786983] BTRFS info (device sdf): found 1 extents
[ 6577.584717] blk_update_request: I/O error, dev fd0, sector 0
[ 6577.588657] floppy: error -5 while reading block 0
[ 6601.128050] blk_update_request: I/O error, dev fd0, sector 0
[ 6601.128840] floppy: error -5 while reading block 0
[ 6601.140652] systemd-journald[618]: Sent WATCHDOG=1 notification.
[ 6625.542882] BTRFS: device fsid e3828911-137a-48d1-9be3-c3b7ec16f953
devid 1 transid 5 /dev/sdc
[ 6625.581187] BTRFS: device fsid e3828911-137a-48d1-9be3-c3b7ec16f953
devid 2 transid 5 /dev/sdd
[ 6625.601238] BTRFS: device fsid e3828911-137a-48d1-9be3-c3b7ec16f953
devid 3 transid 5 /dev/sde
[ 6625.723378] BTRFS: device fsid e3828911-137a-48d1-9be3-c3b7ec16f953
devid 4 transid 5 /dev/sdf
[ 6626.304067] BTRFS info (device sdf): turning on discard
[ 6626.304830] BTRFS info (device sdf): disk space caching is enabled
[ 6626.305520] BTRFS info (device sdf): has skinny extents
[ 6626.471590] BTRFS info (device sdf): creating UUID tree
[ 6628.368615] BTRFS info (device sdf): dev_replace from /dev/sdd
(devid 2) to /dev/sdg started
[ 6641.236372] BTRFS info (device sdf): dev_replace from /dev/sdd
(devid 2) to /dev/sdg finished
[ 6647.060891] BTRFS info (device sdf): relocating block group
5473566720 flags data|raid10
[ 6647.164149] BTRFS info (device sdf): found 2 extents
[ 6647.259690] BTRFS info (device sdf): found 2 extents
[ 6647.330143] BTRFS info (device sdf): relocating block group
4399824896 flags metadata|raid10
[ 6647.458031] BTRFS info (device sdf): found 34 extents
[ 6647.523545] BTRFS info (device sdf): relocating block group
4332716032 flags system|raid10
[ 6647.603603] BTRFS info (device sdf): relocating block group
2185232384 flags data|raid10
[ 6651.461293] BTRFS info (device sdf): found 245 extents
[ 6651.803813] BTRFS info (device sdf): found 245 extents
[ 6651.872335] BTRFS info (device sdf): relocating block group
37748736 flags metadata|raid10
[ 6652.067394] BTRFS info (device sdf): found 102 extents
[ 6652.124856] BTRFS info (device sdf): relocating block group
20971520 flags system|raid10
[ 6652.199854] BTRFS info (device sdf): found 1 extents
[ 6661.700121] blk_update_request: I/O error, dev fd0, sector 0
[ 6661.701526] floppy: error -5 while reading block 0
[ 6662.052252] blk_update_request: I/O error, dev fd0, sector 0
[ 6662.053296] floppy: error -5 while reading block 0
[ 6663.040606] BTRFS: device fsid fac7c5d7-9979-4280-9ced-6adf1baf673d
devid 1 transid 5 /dev/sdc
[ 6663.224627] BTRFS: device fsid fac7c5d7-9979-4280-9ced-6adf1baf673d
devid 2 transid 5 /dev/sdd
[ 6663.266865] BTRFS: device fsid fac7c5d7-9979-4280-9ced-6adf1baf673d
devid 3 transid 5 /dev/sde
[ 6663.280780] BTRFS: device fsid fac7c5d7-9979-4280-9ced-6adf1baf673d
devid 4 transid 5 /dev/sdf
[ 6663.784636] BTRFS info (device sdf): turning on discard
[ 6663.811720] BTRFS info (device sdf): disk space caching is enabled
[ 6663.824680] BTRFS info (device sdf): has skinny extents
[ 6664.005123] BTRFS info (device sdf): creating UUID tree
[ 6664.254147] btrfs_print_data_csum_error: 11 callbacks suppressed
[ 6664.292699] BTRFS warning (device sdf): csum failed root 5 ino 284
off 294912 csum 0x98f94189 expected csum 0xf2364862 mirror 1
[ 6664.344601] BTRFS warning (device sdf): csum failed root 5 ino 284
off 299008 csum 0x98f94189 expected csum 0xf2364862 mirror 1
[ 6664.376676] BTRFS warning (device sdf): csum failed root 5 ino 284
off 303104 csum 0x98f94189 expected csum 0xf2364862 mirror 1
[ 6664.391328] BTRFS warning (device sdf): csum failed root 5 ino 284
off 294912 csum 0x98f94189 expected csum 0xf2364862 mirror 1
[ 6664.401801] BTRFS warning (device sdf): csum failed root 5 ino 284
off 299008 csum 0x98f94189 expected csum 0xf2364862 mirror 1
[ 6664.440441] BTRFS warning (device sdf): csum failed root 5 ino 284
off 303104 csum 0x98f94189 expected csum 0xf2364862 mirror 1
[ 6664.540695] BTRFS warning (device sdf): csum failed root 5 ino 284
off 294912 csum 0x98f94189 expected csum 0xf2364862 mirror 2
[ 6664.542010] BTRFS warning (device sdf): csum failed root 5 ino 284
off 299008 csum 0x98f94189 expected csum 0xf2364862 mirror 2
[ 6664.543258] BTRFS warning (device sdf): csum failed root 5 ino 284
off 303104 csum 0x98f94189 expected csum 0xf2364862 mirror 2
[ 6664.672878] BTRFS warning (device sdf): csum failed root 5 ino 284
off 303104 csum 0x98f94189 expected csum 0xf2364862 mirror 1
[ 6665.232628] blk_update_request: I/O error, dev fd0, sector 0
[ 6665.233526] floppy: error -5 while reading block 0
[ 6669.256609] btrfs_print_data_csum_error: 297 callbacks suppressed
[ 6669.291821] BTRFS warning (device sdf): csum failed root 5 ino 300
off 684032 csum 0x98f94189 expected csum 0x42186b7f mirror 1
[ 6669.319412] BTRFS warning (device sdf): csum failed root 5 ino 300
off 688128 csum 0x98f94189 expected csum 0x42186b7f mirror 1
[ 6669.332644] BTRFS warning (device sdf): csum failed root 5 ino 298
off 892928 csum 0x98f94189 expected csum 0x28d76294 mirror 2
[ 6669.356673] BTRFS warning (device sdf): csum failed root 5 ino 298
off 897024 csum 0x98f94189 expected csum 0x28d76294 mirror 2
[ 6669.376583] BTRFS warning (device sdf): csum failed root 5 ino 300
off 692224 csum 0x98f94189 expected csum 0x42186b7f mirror 1
[ 6669.447063] BTRFS warning (device sdf): csum failed root 5 ino 298
off 901120 csum 0x98f94189 expected csum 0x28d76294 mirror 2
[ 6669.500703] BTRFS warning (device sdf): csum failed root 5 ino 298
off 905216 csum 0x98f94189 expected csum 0x28d76294 mirror 2
[ 6669.505183] BTRFS warning (device sdf): csum failed root 5 ino 298
off 929792 csum 0x98f94189 expected csum 0x28d76294 mirror 2
[ 6669.505238] BTRFS warning (device sdf): csum failed root 5 ino 281
off 901120 csum 0x98f94189 expected csum 0x27a85bb4 mirror 2
[ 6669.508819] BTRFS warning (device sdf): csum failed root 5 ino 300
off 696320 csum 0x98f94189 expected csum 0x42186b7f mirror 1
[ 6669.920580] BTRFS info (device sdf): dev_replace from /dev/sdd
(devid 2) to /dev/sdg started
[ 6674.297803] btrfs_print_data_csum_error: 138 callbacks suppressed
[ 6674.317501] BTRFS warning (device sdf): csum failed root 5 ino 402
off 831488 csum 0x98f94189 expected csum 0x4d67525f mirror 2
[ 6674.348626] BTRFS warning (device sdf): csum failed root 5 ino 311
off 200704 csum 0x98f94189 expected csum 0x4d67525f mirror 1
[ 6674.380668] BTRFS warning (device sdf): csum failed root 5 ino 311
off 204800 csum 0x98f94189 expected csum 0x4d67525f mirror 1
[ 6674.389074] BTRFS warning (device sdf): csum failed root 5 ino 388
off 192512 csum 0x98f94189 expected csum 0xfd497142 mirror 1
[ 6674.407222] BTRFS warning (device sdf): csum failed root 5 ino 402
off 729088 csum 0x42186b7f expected csum 0x4d67525f mirror 2
[ 6674.416111] BTRFS warning (device sdf): csum failed root 5 ino 388
off 147456 csum 0x98f94189 expected csum 0xfd497142 mirror 2
[ 6674.416156] ------------[ cut here ]------------
[ 6674.416157] kernel BUG at fs/btrfs/raid56.c:1897!
[ 6674.416159] invalid opcode: 0000 [#1] PREEMPT SMP
[ 6674.416160] Modules linked in: dm_flakey dm_mod dax ppdev tpm_tis
parport_pc tpm_tis_core evdev tpm psmouse sg i2c_piix4 pcspkr parport
i2c_core serio_raw button sunrpc loop autofs4 ext4 crc16 jbd2 mbcache
btrfs raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor
async_tx xor raid6_pq libcrc32c crc32c_generic raid1 raid0 multipath
linear md_mod sr_mod cdrom ata_generic sd_mod virtio_scsi ata_piix
virtio_pci virtio_ring libata scsi_mod e1000 virtio floppy
[ 6674.416184] CPU: 3 PID: 19236 Comm: kworker/u32:10 Not tainted
4.12.0-rc6-btrfs-next-44+ #1
[ 6674.416185] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996),
BIOS rel-1.9.1-0-gb3ef39f-prebuilt.qemu-project.org 04/01/2014
[ 6674.416210] Workqueue: btrfs-endio btrfs_endio_helper [btrfs]
[ 6674.416211] task: ffff880147f6c740 task.stack: ffffc90001fb8000
[ 6674.416229] RIP: 0010:__raid_recover_end_io+0x1ac/0x370 [btrfs]
[ 6674.416230] RSP: 0018:ffffc90001fbbb90 EFLAGS: 00010217
[ 6674.416231] RAX: ffff8801ff4b4f00 RBX: 0000000000000002 RCX: 0000000000000001
[ 6674.416232] RDX: ffff880099b045d8 RSI: ffffffff81a5f6e0 RDI: 0000000000000004
[ 6674.416232] RBP: ffffc90001fbbbc8 R08: 0000000000000001 R09: 0000000000000001
[ 6674.416233] R10: ffffc90001fbbac8 R11: 0000000000001000 R12: 0000000000000002
[ 6674.416234] R13: ffff880099b045c0 R14: 0000000000000004 R15: ffff88012bff2000
[ 6674.416235] FS: 0000000000000000(0000) GS:ffff88023f2c0000(0000)
knlGS:0000000000000000
[ 6674.416235] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 6674.416236] CR2: 00007f28cf282000 CR3: 00000001000c6000 CR4: 00000000000006e0
[ 6674.416239] Call Trace:
[ 6674.416259] __raid56_parity_recover+0xfc/0x16e [btrfs]
[ 6674.416276] raid56_parity_recover+0x157/0x16b [btrfs]
[ 6674.416293] btrfs_map_bio+0xe0/0x259 [btrfs]
[ 6674.416310] btrfs_submit_bio_hook+0xbf/0x147 [btrfs]
[ 6674.416327] end_bio_extent_readpage+0x27b/0x4a0 [btrfs]
[ 6674.416331] bio_endio+0x17d/0x1b3
[ 6674.416346] end_workqueue_fn+0x3c/0x3f [btrfs]
[ 6674.416362] btrfs_scrubparity_helper+0x1aa/0x3b8 [btrfs]
[ 6674.416379] btrfs_endio_helper+0xe/0x10 [btrfs]
[ 6674.416381] process_one_work+0x276/0x4b6
[ 6674.416384] worker_thread+0x1ac/0x266
[ 6674.416386] ? rescuer_thread+0x278/0x278
[ 6674.416387] kthread+0x106/0x10e
[ 6674.416389] ? __list_del_entry+0x22/0x22
[ 6674.416391] ret_from_fork+0x27/0x40
[ 6674.416395] Code: 44 89 e2 be 00 10 00 00 ff 15 b0 ab ef ff eb 72
4d 89 e8 89 d9 44 89 e2 be 00 10 00 00 ff 15 a3 ab ef ff eb 5d 41 83
fc ff 74 02 <0f> 0b 49 63 97 0c 01 00 00 48 63 c3 b9 00 04 00 00 49 8d
44 c5
[ 6674.416432] RIP: __raid_recover_end_io+0x1ac/0x370 [btrfs] RSP:
ffffc90001fbbb90
[ 6674.416434] ---[ end trace 74d56ebe7489dd6a ]---
Scrub also finds lots of errors in those tests, causing them to fail.
btrfs/048 also fails due to this new dir item length validation function....
And so on...
>
> For everyone else on the list, since I'm heading off to vacation until ~July
> 9th, Dave is sending this off to Linus once the merge window starts.
>
> -chris
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
--
Filipe David Manana,
“Whether you think you can, or you think you can't — you're right.”
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PULL] Btrfs for 4.13, part 1
2017-06-23 15:16 [PULL] Btrfs for 4.13, part 1 David Sterba
` (2 preceding siblings ...)
2017-06-26 16:58 ` [PULL] Btrfs for 4.13, part 1 Chris Mason
@ 2017-06-28 6:49 ` Nikolay Borisov
2017-06-30 7:44 ` David Sterba
3 siblings, 1 reply; 13+ messages in thread
From: Nikolay Borisov @ 2017-06-28 6:49 UTC (permalink / raw)
To: David Sterba, clm; +Cc: linux-btrfs
On 23.06.2017 18:16, David Sterba wrote:
> Hi,
>
> this is the main batch for 4.13. There are some user visible changes, see
> below. The core updates improve error handling (mostly related to bios), with
> the usual incremental work on the GFP_NOFS (mis)use removal. All patches have
> been in for-next for an extensive amount of time.
>
> Thre will be followups but I want push the series (111 patches) forward. There
> are also some updates to adjacent subsystems (writeback and blocklayer), so I
> want to give some stable point for merging in the upcoming weeks.
>
> User visible changes:
> * quota override tunable
> * improved compression thresholds
> * statx support
> * obsoleted mount option alloc_start
>
> Core updates:
> * bio-related updates
> - faster bio cloning
> - no allocation failures
> - preallocated flush bios
> * more kvzalloc use, memalloc_nofs protections, GFP_NOFS updates
> * prep work for btree_inode removal
> * dir-item validation
> * qgoup fixes and updates
> * cleanups
> - removed unused struct members, unused code, refactoring
> - argument refactoring (fs_info/root, caller -> callee sink)
> - SEARCH_TREE docs
>
> ----------------------------------------------------------------
> The following changes since commit 41f1830f5a7af77cf5c86359aba3cbd706687e52:
>
> Linux 4.12-rc6 (2017-06-19 22:19:37 +0800)
>
> are available in the git repository at:
>
> git://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux.git for-4.13-part1
>
> for you to fetch changes up to f3f000297be88b1b75fde5027d660a8d8a44de14:
>
> btrfs: qgroup: Fix qgroup reserved space underflow by only freeing reserved ranges (2017-06-21 20:56:14 +0200)
>
Dave I run bisection of all failures between -rc6 and your
pull request and here are the results:
btrfs/048 - e79a33270d05 ("btrfs: Check name_len with boundary in verify dir_item")
btrfs/053 - b780e22768d3 ("btrfs: fix validation of XATTR_ITEM dir items")
generic/273 - 28785f70ef88 ("Btrfs: skip commit transaction if we don't have enough pinned bytes")
generic/337 - e79a33270d05 ("btrfs: Check name_len with boundary in verify dir_item")
shared/002 - e79a33270d05 ("btrfs: Check name_len with boundary in verify dir_item")
So those are tests which are failing as a direct of
the listed patches.
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PULL] Btrfs for 4.13, part 1
2017-06-24 10:05 ` Wang Shilong
2017-06-25 21:34 ` Jeff Mahoney
@ 2017-06-28 14:28 ` David Sterba
1 sibling, 0 replies; 13+ messages in thread
From: David Sterba @ 2017-06-28 14:28 UTC (permalink / raw)
To: Wang Shilong; +Cc: David Sterba, Chris Mason, linux-btrfs
On Sat, Jun 24, 2017 at 06:05:03PM +0800, Wang Shilong wrote:
> Sorry for bikeshedding.
No problem, I won't read the rest of the mail then. (IOW, don't write
such things if you want people to read your mail.)
> On Fri, Jun 23, 2017 at 11:16 PM, David Sterba <dsterba@suse.com> wrote:
> > Hi,
> >
> > this is the main batch for 4.13. There are some user visible changes, see
> > below. The core updates improve error handling (mostly related to bios), with
> > the usual incremental work on the GFP_NOFS (mis)use removal. All patches have
> >
> > Fabian Frederick (1):
> > btrfs: kmap() can't fail
> >
> <..SNIP..>
> >
> > Sargun Dhillon (2):
> > btrfs: add quota override flag to enable quota override for CAP_SYS_RESOURCE
> > btrfs: Add quota_override knob into sysfs
> >
> > Su Yue (9):
> > btrfs: Introduce btrfs_is_name_len_valid to avoid reading beyond boundary
> > btrfs: Check name_len with boundary in verify dir_item
> > btrfs: Check name_len on add_inode_ref call path
> > btrfs: Verify dir_item in replay_xattr_deletes
> > btrfs: Check name_len in btrfs_check_ref_name_override
> > btrfs: Check name_len before read in iterate_dir_item
> > btrfs: Check name_len before reading btrfs_get_name
> > btrfs: Check name_len before in btrfs_del_root_ref
> > btrfs: Verify dir_item in iterate_object_props
>
> Hmm..add those check might be expensive for metadata operations,
> especially in hot path, i could see similar behavior in Ext4 for ext4
> dentry check.
The checks are really simple, comparing a few numbers, and performed in
places that do not seem to be performance critical. The validation is
supposed to happen after the blocks are read from the disk (or using the
interface that does that, the page might be actually cached).
> Could we run some metadata tests to confirm？it makes sense to
> add check but with min affects to performace.
We can run such tests, I expect that the effect won't be measurable.
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PULL] Btrfs for 4.13, part 1
2017-06-26 0:53 ` Qu Wenruo
2017-06-26 1:46 ` Jeff Mahoney
@ 2017-06-28 14:38 ` David Sterba
1 sibling, 0 replies; 13+ messages in thread
From: David Sterba @ 2017-06-28 14:38 UTC (permalink / raw)
To: Qu Wenruo
Cc: Jeff Mahoney, Wang Shilong, David Sterba, Chris Mason, linux-btrfs
On Mon, Jun 26, 2017 at 08:53:14AM +0800, Qu Wenruo wrote:
> At 06/26/2017 05:34 AM, Jeff Mahoney wrote:
> > On 6/24/17 6:05 AM, Wang Shilong wrote:
> >> Sorry for bikeshedding.
> >>
> >> On Fri, Jun 23, 2017 at 11:16 PM, David Sterba <dsterba@suse.com> wrote:
> >>> Hi,
> >>>
> >>> this is the main batch for 4.13. There are some user visible changes, see
> >>> below. The core updates improve error handling (mostly related to bios), with
> >>> the usual incremental work on the GFP_NOFS (mis)use removal. All patches have
> >>>
> >>> Fabian Frederick (1):
> >>> btrfs: kmap() can't fail
> >>>
> >> <..SNIP..>
> >>>
> >>> Sargun Dhillon (2):
> >>> btrfs: add quota override flag to enable quota override for CAP_SYS_RESOURCE
> >>> btrfs: Add quota_override knob into sysfs
> >>>
> >>> Su Yue (9):
> >>> btrfs: Introduce btrfs_is_name_len_valid to avoid reading beyond boundary
> >>> btrfs: Check name_len with boundary in verify dir_item
> >>> btrfs: Check name_len on add_inode_ref call path
> >>> btrfs: Verify dir_item in replay_xattr_deletes
> >>> btrfs: Check name_len in btrfs_check_ref_name_override
> >>> btrfs: Check name_len before read in iterate_dir_item
> >>> btrfs: Check name_len before reading btrfs_get_name
> >>> btrfs: Check name_len before in btrfs_del_root_ref
> >>> btrfs: Verify dir_item in iterate_object_props
> >>
> >> Hmm..add those check might be expensive for metadata operations,
> >> especially in hot path, i could see similar behavior in Ext4 for ext4
> >> dentry check.
> >>
> >> Could we run some metadata tests to confirm？it makes sense to
> >> add check but with min affects to performace.
>
> IIRC there is a backtrace in one of the patches.
> If needed, we can also upload the fuzzed image.
> Although we don't have good enough test suite which accepts fuzzed image
> for kernel.
As we have the fuzzed images in progs, I started prototyping a way to
export just the testsuite from the git repository, to a tarball. The
test would then iterate over the stored images and attempt to mount
them. So any images you'd like mount-tested, also submit to btrfs-progs.
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PULL] Btrfs for 4.13, part 1
2017-06-27 11:00 ` Filipe Manana
@ 2017-06-28 15:39 ` David Sterba
0 siblings, 0 replies; 13+ messages in thread
From: David Sterba @ 2017-06-28 15:39 UTC (permalink / raw)
To: Filipe Manana; +Cc: Chris Mason, David Sterba, linux-btrfs
On Tue, Jun 27, 2017 at 12:00:20PM +0100, Filipe Manana wrote:
> On Mon, Jun 26, 2017 at 5:58 PM, Chris Mason <clm@fb.com> wrote:
> > On 06/23/2017 11:16 AM, David Sterba wrote:
> >> this is the main batch for 4.13. There are some user visible changes, see
> >> below. The core updates improve error handling (mostly related to bios),
> >> with
> >> the usual incremental work on the GFP_NOFS (mis)use removal. All patches
> >> have
> >> been in for-next for an extensive amount of time.
> >>
> >> Thre will be followups but I want push the series (111 patches) forward.
> >> There
> >> are also some updates to adjacent subsystems (writeback and blocklayer),
> >> so I
> >> want to give some stable point for merging in the upcoming weeks.
> >
> > Thanks Dave, I ran this (along with the updates we added) through a long
> > stress and the usual xfstests.
>
> Have any of you run xfstests without failures?
Without any failures, no, with acceptable failures at this point in
development cycle, yes. The dir-item validation failures are fixable and
will come next. I haven't seen the crashes in raid56, thanks for the
report.
> I'm finding it hard to believe, as some of these changes cause
> problems all the time, and at least one of them deterministically
> ("btrfs: fix validation of XATTR_ITEM dir items"), as I've just noted
> in the respective thread for that patch.
Yeah, that was an incomplete fix for a bug that I had reported but did
not receive a fix in time to send the pull request. I want the dir-item
validation patches merged now, we have enough time to fix the failures.
The merge window is due in the next weeks, major part of the patches
be ready earlier, so I'm deciding what's applicable even it's not
perfect. Known and fixable failures are just fine right now, theres
whole rc1-rc5 period.
You might disagree with this approach. The quality dropped, somebody did
not verify the code, run the tests or check the results. Fill in the
names. I'm not developers' QA, I put patches together and push further
upstream. Reviews cost a lot of time, driving multiple patchsets forward
is exhausting. So I do not test each and every commit or address all
failures immediatelly, I count on the time where this can happen. And
also on the rest of the developer crew.
(Note for casual readers of this mail: I want to set the expectations
right, there's nothing personal.)
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PULL] Btrfs for 4.13, part 1
2017-06-28 6:49 ` Nikolay Borisov
@ 2017-06-30 7:44 ` David Sterba
0 siblings, 0 replies; 13+ messages in thread
From: David Sterba @ 2017-06-30 7:44 UTC (permalink / raw)
To: Nikolay Borisov; +Cc: David Sterba, clm, linux-btrfs
On Wed, Jun 28, 2017 at 09:49:49AM +0300, Nikolay Borisov wrote:
> Dave I run bisection of all failures between -rc6 and your
> pull request and here are the results:
>
> btrfs/048 - e79a33270d05 ("btrfs: Check name_len with boundary in verify dir_item")
> btrfs/053 - b780e22768d3 ("btrfs: fix validation of XATTR_ITEM dir items")
> generic/273 - 28785f70ef88 ("Btrfs: skip commit transaction if we don't have enough pinned bytes")
> generic/337 - e79a33270d05 ("btrfs: Check name_len with boundary in verify dir_item")
> shared/002 - e79a33270d05 ("btrfs: Check name_len with boundary in verify dir_item")
>
> So those are tests which are failing as a direct of
> the listed patches.
Thanks for the list. I'm not sure the bisection helps here, the bug is
in the validation function, other patches call it in various places. But
I've used the list of tests for verifying my fix I sent yesterday.
Overnight run of all fstests still found one instance that fails, test
generic/066 so I'll look at it today.
^ permalink raw reply [flat|nested] 13+ messages in thread
end of thread, other threads:[~2017-06-30 7:45 UTC | newest]
Thread overview: 13+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-06-23 15:16 [PULL] Btrfs for 4.13, part 1 David Sterba
2017-06-24 10:05 ` Wang Shilong
2017-06-25 21:34 ` Jeff Mahoney
2017-06-26 0:53 ` Qu Wenruo
2017-06-26 1:46 ` Jeff Mahoney
2017-06-28 14:38 ` David Sterba
2017-06-28 14:28 ` David Sterba
2017-06-26 13:40 ` [PULL] Btrfs for 4.13, part 1 (update 1) David Sterba
2017-06-26 16:58 ` [PULL] Btrfs for 4.13, part 1 Chris Mason
2017-06-27 11:00 ` Filipe Manana
2017-06-28 15:39 ` David Sterba
2017-06-28 6:49 ` Nikolay Borisov
2017-06-30 7:44 ` David Sterba
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.