All of lore.kernel.org
 help / color / mirror / Atom feed
From: Qu Wenruo <quwenruo.btrfs@gmx.com>
To: Christoph Hellwig <hch@lst.de>, fstests@vger.kernel.org
Cc: linux-btrfs@vger.kernel.org
Subject: Re: [PATCH 1/9] btrfs: add a helpers for read repair testing
Date: Tue, 24 May 2022 21:18:53 +0800	[thread overview]
Message-ID: <2f850377-0adf-cdf3-9fc6-02136b9a5282@gmx.com> (raw)
In-Reply-To: <20220524071838.715013-2-hch@lst.de>



On 2022/5/24 15:18, Christoph Hellwig wrote:
> Add a few helpers to consolidate code for btrfs read repair testing:
>
>   - _btrfs_get_first_logical() gets the btrfs logical address for the
>     first extent in a file
>   - _btrfs_get_device_path and _btrfs_get_physical use the
>     btrfs-map-logical tool to find the device path and physical address
>     for btrfs logical address for a specific mirror
>   - _btrfs_direct_read_on_mirror and _btrfs_buffered_read_on_mirror
>     read the data from a specific mirror
>
> These will be used to consolidate the read repair tests and avoid
> duplication for new tests.
>
> Signed-off-by: Christoph Hellwig <hch@lst.de>

Reviewed-by: Qu Wenruo <wqu@suse.com>

Some some small nitpicks inlined below.
> ---
>   common/btrfs  | 75 +++++++++++++++++++++++++++++++++++++++++++++++++++
>   common/config |  1 +
>   2 files changed, 76 insertions(+)
>
> diff --git a/common/btrfs b/common/btrfs
> index ac597ca4..129a83f7 100644
> --- a/common/btrfs
> +++ b/common/btrfs
> @@ -505,3 +505,78 @@ _btrfs_metadump()
>   	$BTRFS_IMAGE_PROG "$device" "$dumpfile"
>   	[ -n "$DUMP_COMPRESSOR" ] && $DUMP_COMPRESSOR -f "$dumpfile" &> /dev/null
>   }
> +
> +# Return the btrfs logical address for the first block in a file
> +_btrfs_get_first_logical()
> +{
> +	local file=$1
> +	_require_command "$FILEFRAG_PROG" filefrag
> +
> +	${FILEFRAG_PROG} -v $SCRATCH_MNT/foobar >> $seqres.full
> +	${FILEFRAG_PROG} -v $file | _filter_filefrag | cut -d '#' -f 1
> +}
> +
> +# Find the device path for a btrfs logical offset
> +_btrfs_get_device_path()
> +{
> +	local logical=$1
> +	local stripe=$2
> +
> +	_require_command "$BTRFS_MAP_LOGICAL_PROG" btrfs-map-logical
> +
> +	$BTRFS_MAP_LOGICAL_PROG -l $logical $SCRATCH_DEV | \
> +		$AWK_PROG "(\$1 ~ /mirror/ && \$2 ~ /$stripe/) { print \$8 }"
> +}
> +
> +
> +# Find the device physical sector for a btrfs logical offset
> +_btrfs_get_physical()
> +{
> +	local logical=$1
> +	local stripe=$2
> +
> +	_require_command "$BTRFS_MAP_LOGICAL_PROG" btrfs-map-logical
> +
> +	$BTRFS_MAP_LOGICAL_PROG -l $logical $SCRATCH_DEV >> $seqres.full 2>&1
> +	$BTRFS_MAP_LOGICAL_PROG -l $logical $SCRATCH_DEV | \
> +		$AWK_PROG "(\$1 ~ /mirror/ && \$2 ~ /$stripe/) { print \$6 }"

Since the map logical output is saved into seqres, maybe it's better to
add "-b <sectorsize>" parameters.

The default bytes is the nodesize (which is super werid), thus under
corner cases, like at the stripe boundary, it can return multiple result
groups and cause some confusion.

Although we will need another helper to grab the sectorsize of a mounted
btrfs though.

Thanks,
Qu
> +}
> +
> +# Read from a specific stripe to test read recovery that corrupted a specific
> +# stripe.  Btrfs uses the PID to select the mirror, so keep reading until the
> +# xfs_io process that performed the read was executed with a PID that ends up
> +# on the intended mirror.
> +_btrfs_direct_read_on_mirror()
> +{
> +	local mirror=$1
> +	local nr_mirrors=$2
> +	local file=$3
> +	local offset=$4
> +	local size=$5
> +
> +	while [[ -z $( (( BASHPID % nr_mirrors == mirror )) &&
> +		exec $XFS_IO_PROG -d \
> +			-c "pread -b $size $offset $size" $file) ]]; do
> +		:
> +	done
> +}
> +
> +# Read from a specific stripe to test read recovery that corrupted a specific
> +# stripe.  Btrfs uses the PID to select the mirror, so keep reading until the
> +# xfs_io process that performed the read was executed with a PID that ends up
> +# on the intended mirror.
> +_btrfs_buffered_read_on_mirror()
> +{
> +	local mirror=$1
> +	local nr_mirrors=$2
> +	local file=$3
> +	local offset=$4
> +	local size=$5
> +
> +	echo 3 > /proc/sys/vm/drop_caches
> +	while [[ -z $( (( BASHPID % nr_mirrors == mirror )) &&
> +		exec $XFS_IO_PROG \
> +			-c "pread -b $size $offset $size" $file) ]]; do
> +		:
> +	done
> +}
> diff --git a/common/config b/common/config
> index c6428f90..df20afc1 100644
> --- a/common/config
> +++ b/common/config
> @@ -228,6 +228,7 @@ export E2IMAGE_PROG="$(type -P e2image)"
>   export BLKZONE_PROG="$(type -P blkzone)"
>   export GZIP_PROG="$(type -P gzip)"
>   export BTRFS_IMAGE_PROG="$(type -P btrfs-image)"
> +export BTRFS_MAP_LOGICAL_PROG=$(type -P btrfs-map-logical)
>
>   # use 'udevadm settle' or 'udevsettle' to wait for lv to be settled.
>   # newer systems have udevadm command but older systems like RHEL5 don't.

  reply	other threads:[~2022-05-24 13:19 UTC|newest]

Thread overview: 23+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-05-24  7:18 two more btrfs read repair tests v2 Christoph Hellwig
2022-05-24  7:18 ` [PATCH 1/9] btrfs: add a helpers for read repair testing Christoph Hellwig
2022-05-24 13:18   ` Qu Wenruo [this message]
2022-05-27 11:22   ` Nikolay Borisov
2022-05-27 13:33     ` Christoph Hellwig
2022-05-24  7:18 ` [PATCH 2/9] btrfs/140: use common read repair helpers Christoph Hellwig
2022-05-24 13:21   ` Qu Wenruo
2022-05-24  7:18 ` [PATCH 3/9] btrfs/141: " Christoph Hellwig
2022-05-24 13:21   ` Qu Wenruo
2022-05-24  7:18 ` [PATCH 4/9] btrfs/142: " Christoph Hellwig
2022-05-24 13:21   ` Qu Wenruo
2022-05-24  7:18 ` [PATCH 5/9] btrfs/143: " Christoph Hellwig
2022-05-24 13:21   ` Qu Wenruo
2022-05-24  7:18 ` [PATCH 6/9] btrfs/157: use _btrfs_get_first_logical Christoph Hellwig
2022-05-24 13:21   ` Qu Wenruo
2022-05-24  7:18 ` [PATCH 7/9] btrfs/215: " Christoph Hellwig
2022-05-24 13:21   ` Qu Wenruo
2022-05-24  7:18 ` [PATCH 8/9] btrfs: test repair with sectors corrupted in multiple mirrors Christoph Hellwig
2022-05-24 13:28   ` Qu Wenruo
2022-06-02 11:30   ` Filipe Manana
2022-06-02 12:07     ` Christoph Hellwig
2022-05-24  7:18 ` [PATCH 9/9] btrfs: test repair with corrupted sectors interleaved over " Christoph Hellwig
2022-05-24 13:30   ` Qu Wenruo

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=2f850377-0adf-cdf3-9fc6-02136b9a5282@gmx.com \
    --to=quwenruo.btrfs@gmx.com \
    --cc=fstests@vger.kernel.org \
    --cc=hch@lst.de \
    --cc=linux-btrfs@vger.kernel.org \
    --subject='Re: [PATCH 1/9] btrfs: add a helpers for read repair testing' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.