From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.3 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS, USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 16671CA9EAF for ; Mon, 21 Oct 2019 17:17:56 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id D4A1820882 for ; Mon, 21 Oct 2019 17:17:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726289AbfJURRz (ORCPT ); Mon, 21 Oct 2019 13:17:55 -0400 Received: from sandeen.net ([63.231.237.45]:43856 "EHLO sandeen.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728375AbfJURRz (ORCPT ); Mon, 21 Oct 2019 13:17:55 -0400 Received: from Liberator-6.local (liberator [10.0.0.4]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by sandeen.net (Postfix) with ESMTPSA id BFD1AEC3; Mon, 21 Oct 2019 12:17:08 -0500 (CDT) Subject: Re: [PATCH 06/11] xfs_scrub: reduce fsmap activity for media errors To: "Darrick J. Wong" Cc: linux-xfs@vger.kernel.org References: <156944736739.300131.5717633994765951730.stgit@magnolia> <156944740409.300131.11774965759850122284.stgit@magnolia> From: Eric Sandeen Autocrypt: addr=sandeen@sandeen.net; prefer-encrypt=mutual; keydata= mQINBE6x99QBEADMR+yNFBc1Y5avoUhzI/sdR9ANwznsNpiCtZlaO4pIWvqQJCjBzp96cpCs nQZV32nqJBYnDpBDITBqTa/EF+IrHx8gKq8TaSBLHUq2ju2gJJLfBoL7V3807PQcI18YzkF+ WL05ODFQ2cemDhx5uLghHEeOxuGj+1AI+kh/FCzMedHc6k87Yu2ZuaWF+Gh1W2ix6hikRJmQ vj5BEeAx7xKkyBhzdbNIbbjV/iGi9b26B/dNcyd5w2My2gxMtxaiP7q5b6GM2rsQklHP8FtW ZiYO7jsg/qIppR1C6Zr5jK1GQlMUIclYFeBbKggJ9mSwXJH7MIftilGQ8KDvNuV5AbkronGC sEEHj2khs7GfVv4pmUUHf1MRIvV0x3WJkpmhuZaYg8AdJlyGKgp+TQ7B+wCjNTdVqMI1vDk2 BS6Rg851ay7AypbCPx2w4d8jIkQEgNjACHVDU89PNKAjScK1aTnW+HNUqg9BliCvuX5g4z2j gJBs57loTWAGe2Ve3cMy3VoQ40Wt3yKK0Eno8jfgzgb48wyycINZgnseMRhxc2c8hd51tftK LKhPj4c7uqjnBjrgOVaVBupGUmvLiePlnW56zJZ51BR5igWnILeOJ1ZIcf7KsaHyE6B1mG+X dmYtjDhjf3NAcoBWJuj8euxMB6TcQN2MrSXy5wSKaw40evooGwARAQABtCVFcmljIFIuIFNh bmRlZW4gPHNhbmRlZW5Ac2FuZGVlbi5uZXQ+iQI7BBMBAgAlAhsDBgsJCAcDAgYVCAIJCgsE FgIDAQIeAQIXgAUCUzMzbAIZAQAKCRAgrhaS4T3e4Fr7D/wO+fenqVvHjq21SCjDCrt8HdVj aJ28B1SqSU2toxyg5I160GllAxEHpLFGdbFAhQfBtnmlY9eMjwmJb0sCIrkrB6XNPSPA/B2B UPISh0z2odJv35/euJF71qIFgWzp2czJHkHWwVZaZpMWWNvsLIroXoR+uA9c2V1hQFVAJZyk EE4xzfm1+oVtjIC12B9tTCuS00pY3AUy21yzNowT6SSk7HAzmtG/PJ/uSB5wEkwldB6jVs2A sjOg1wMwVvh/JHilsQg4HSmDfObmZj1d0RWlMWcUE7csRnCE0ZWBMp/ttTn+oosioGa09HAS 9jAnauznmYg43oQ5Akd8iQRxz5I58F/+JsdKvWiyrPDfYZtFS+UIgWD7x+mHBZ53Qjazszox gjwO9ehZpwUQxBm4I0lPDAKw3HJA+GwwiubTSlq5PS3P7QoCjaV8llH1bNFZMz2o8wPANiDx 5FHgpRVgwLHakoCU1Gc+LXHXBzDXt7Cj02WYHdFzMm2hXaslRdhNGowLo1SXZFXa41KGTlNe 4di53y9CK5ynV0z+YUa+5LR6RdHrHtgywdKnjeWdqhoVpsWIeORtwWGX8evNOiKJ7j0RsHha WrePTubr5nuYTDsQqgc2r4aBIOpeSRR2brlT/UE3wGgy9LY78L4EwPR0MzzecfE1Ws60iSqw Pu3vhb7h3bkCDQROsffUARAA0DrUifTrXQzqxO8aiQOC5p9Tz25Np/Tfpv1rofOwL8VPBMvJ X4P5l1V2yd70MZRUVgjmCydEyxLJ6G2YyHO2IZTEajUY0Up+b3ErOpLpZwhvgWatjifpj6bB SKuDXeThqFdkphF5kAmgfVAIkan5SxWK3+S0V2F/oxstIViBhMhDwI6XsRlnVBoLLYcEilxA 2FlRUS7MOZGmRJkRtdGD5koVZSM6xVZQSmfEBaYQ/WJBGJQdPy94nnlAVn3lH3+N7pXvNUuC GV+t4YUt3tLcRuIpYBCOWlc7bpgeCps5Xa0dIZgJ8Louu6OBJ5vVXjPxTlkFdT0S0/uerCG5 1u8p6sGRLnUeAUGkQfIUqGUjW2rHaXgWNvzOV6i3tf9YaiXKl3avFaNW1kKBs0T5M1cnlWZU Utl6k04lz5OjoNY9J/bGyV3DSlkblXRMK87iLYQSrcV6cFz9PRl4vW1LGff3xRQHngeN5fPx ze8X5NE3hb+SSwyMSEqJxhVTXJVfQWWW0dQxP7HNwqmOWYF/6m+1gK/Y2gY3jAQnsWTru4RV TZGnKwEPmOCpSUvsTRXsVHgsWJ70qd0yOSjWuiv4b8vmD3+QFgyvCBxPMdP3xsxN5etheLMO gRwWpLn6yNFq/xtgs+ECgG+gR78yXQyA7iCs5tFs2OrMqV5juSMGmn0kxJUAEQEAAYkCHwQY AQIACQUCTrH31AIbDAAKCRAgrhaS4T3e4BKwD/0ZOOmUNOZCSOLAMjZx3mtYtjYgfUNKi0ki YPveGoRWTqbis8UitPtNrG4XxgzLOijSdOEzQwkdOIp/QnZhGNssMejCnsluK0GQd+RkFVWN mcQT78hBeGcnEMAXZKq7bkIKzvc06GFmkMbX/gAl6DiNGv0UNAX+5FYh+ucCJZSyAp3sA+9/ LKjxnTedX0aygXA6rkpX0Y0FvN/9dfm47+LGq7WAqBOyYTU3E6/+Z72bZoG/cG7ANLxcPool LOrU43oqFnD8QwcN56y4VfFj3/jDF2MX3xu4v2OjglVjMEYHTCxP3mpxesGHuqOit/FR+mF0 MP9JGfj6x+bj/9JMBtCW1bY/aPeMdPGTJvXjGtOVYblGZrSjXRn5++Uuy36CvkcrjuziSDG+ JEexGxczWwN4mrOQWhMT5Jyb+18CO+CWxJfHaYXiLEW7dI1AynL4jjn4W0MSiXpWDUw+fsBO Pk6ah10C4+R1Jc7dyUsKksMfvvhRX1hTIXhth85H16706bneTayZBhlZ/hK18uqTX+s0onG/ m1F3vYvdlE4p2ts1mmixMF7KajN9/E5RQtiSArvKTbfsB6Two4MthIuLuf+M0mI4gPl9SPlf fWCYVPhaU9o83y1KFbD/+lh1pjP7bEu/YudBvz7F2Myjh4/9GUAijrCTNeDTDAgvIJDjXuLX pA== Message-ID: Date: Mon, 21 Oct 2019 12:17:52 -0500 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:68.0) Gecko/20100101 Thunderbird/68.1.2 MIME-Version: 1.0 In-Reply-To: <156944740409.300131.11774965759850122284.stgit@magnolia> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit Sender: linux-xfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org On 9/25/19 4:36 PM, Darrick J. Wong wrote: > From: Darrick J. Wong > > Right now we rather foolishly query the fsmap data for every single > media error that we find. This is a silly waste of time since we > have yet to combine adjacent bad blocks into bad extents, so move the > rmap query until after we've constructed the bad block bitmap data. > > Signed-off-by: Darrick J. Wong > --- > libfrog/bitmap.c | 10 +--- > scrub/phase6.c | 148 ++++++++++++++++++++++++++++++++++++++---------------- > 2 files changed, 108 insertions(+), 50 deletions(-) > > > diff --git a/libfrog/bitmap.c b/libfrog/bitmap.c > index 6a88ef48..5daa1081 100644 > --- a/libfrog/bitmap.c > +++ b/libfrog/bitmap.c > @@ -314,7 +314,6 @@ bitmap_clear( > } > #endif > > -#ifdef DEBUG > /* Iterate the set regions of this bitmap. */ > int > bitmap_iterate( > @@ -324,20 +323,19 @@ bitmap_iterate( > { > struct avl64node *node; > struct bitmap_node *ext; > - int error = 0; > + int ret; > > pthread_mutex_lock(&bmap->bt_lock); > avl_for_each(bmap->bt_tree, node) { > ext = container_of(node, struct bitmap_node, btn_node); > - error = fn(ext->btn_start, ext->btn_length, arg); > - if (error) > + ret = fn(ext->btn_start, ext->btn_length, arg); > + if (ret) > break; > } > pthread_mutex_unlock(&bmap->bt_lock); > > - return error; > + return ret; > } > -#endif > > /* Iterate the set regions of part of this bitmap. */ > int > diff --git a/scrub/phase6.c b/scrub/phase6.c > index ec821373..378ea0fb 100644 > --- a/scrub/phase6.c > +++ b/scrub/phase6.c > @@ -341,27 +341,9 @@ xfs_report_verify_dirent( > return moveon; > } > > -/* Given bad extent lists for the data & rtdev, find bad files. */ > -static bool > -xfs_report_verify_errors( > - struct scrub_ctx *ctx, > - struct media_verify_state *vs) > -{ > - bool moveon; > - > - /* Scan the directory tree to get file paths. */ > - moveon = scan_fs_tree(ctx, xfs_report_verify_dir, > - xfs_report_verify_dirent, vs); > - if (!moveon) > - return false; > - > - /* Scan for unlinked files. */ > - return xfs_scan_all_inodes(ctx, xfs_report_verify_inode, vs); > -} > - > /* Report an IO error resulting from read-verify based off getfsmap. */ > static bool > -xfs_check_rmap_error_report( > +ioerr_fsmap_report( > struct scrub_ctx *ctx, > const char *descr, > struct fsmap *map, > @@ -409,12 +391,31 @@ xfs_check_rmap_error_report( > return true; > } > > +static struct bitmap * > +bitmap_for_disk( > + struct scrub_ctx *ctx, > + struct disk *disk, > + struct media_verify_state *vs) > +{ > + dev_t dev = xfs_disk_to_dev(ctx, disk); > + > + /* > + * If we don't have parent pointers, save the bad extent for > + * later rescanning. > + */ This comment doesn't make sense here, does it? > + if (dev == ctx->fsinfo.fs_datadev) > + return vs->d_bad; > + else if (dev == ctx->fsinfo.fs_rtdev) > + return vs->r_bad; > + return NULL; > +} > + > /* > * Remember a read error for later, and see if rmap will tell us about the > * owner ahead of time. > */ > static void > -xfs_check_rmap_ioerr( > +remember_ioerr( > struct scrub_ctx *ctx, > struct disk *disk, > uint64_t start, > @@ -422,32 +423,39 @@ xfs_check_rmap_ioerr( > int error, > void *arg) > { > - struct fsmap keys[2]; > - char descr[DESCR_BUFSZ]; > struct media_verify_state *vs = arg; > struct bitmap *tree; > - dev_t dev; > int ret; > > - dev = xfs_disk_to_dev(ctx, disk); > + tree = bitmap_for_disk(ctx, disk, vs); > + if (!tree) > + return; > > - /* > - * If we don't have parent pointers, save the bad extent for > - * later rescanning. > - */ > - if (dev == ctx->fsinfo.fs_datadev) > - tree = vs->d_bad; > - else if (dev == ctx->fsinfo.fs_rtdev) > - tree = vs->r_bad; > - else > - tree = NULL; > - if (tree) { > - ret = bitmap_set(tree, start, length); > - if (ret) > - str_liberror(ctx, ret, _("setting bad block bitmap")); > - } Maybe that comment should be here? > + ret = bitmap_set(tree, start, length); > + if (ret) > + str_liberror(ctx, ret, _("setting bad block bitmap")); > +} > + > +struct walk_ioerr { > + struct scrub_ctx *ctx; > + struct disk *disk; > +}; comment here would be great. Is this walking an ioerror? Reporting an ioerror from a walk? (or maybe also/instead a comment on walk_ioerrs()) also whee, functions and structures w/ the same name :D > +static int > +walk_ioerr( > + uint64_t start, > + uint64_t length, > + void *arg) > +{ > + struct walk_ioerr *wioerr = arg; > + struct fsmap keys[2]; > + char descr[DESCR_BUFSZ]; > + dev_t dev; > + > + dev = xfs_disk_to_dev(wioerr->ctx, wioerr->disk); > > - snprintf(descr, DESCR_BUFSZ, _("dev %d:%d ioerr @ %"PRIu64":%"PRIu64" "), > + snprintf(descr, DESCR_BUFSZ, > +_("dev %d:%d ioerr @ %"PRIu64":%"PRIu64" "), > major(dev), minor(dev), start, length); > > /* Go figure out which blocks are bad from the fsmap. */ > @@ -459,8 +467,60 @@ xfs_check_rmap_ioerr( > (keys + 1)->fmr_owner = ULLONG_MAX; > (keys + 1)->fmr_offset = ULLONG_MAX; > (keys + 1)->fmr_flags = UINT_MAX; > - xfs_iterate_fsmap(ctx, descr, keys, xfs_check_rmap_error_report, > + xfs_iterate_fsmap(wioerr->ctx, descr, keys, ioerr_fsmap_report, > &start); > + return 0; > +} > + > +static int > +walk_ioerrs( > + struct scrub_ctx *ctx, > + struct disk *disk, > + struct media_verify_state *vs) > +{ > + struct walk_ioerr wioerr = { > + .ctx = ctx, > + .disk = disk, > + };> + struct bitmap *tree; > + > + if (!disk) > + return 0; > + tree = bitmap_for_disk(ctx, disk, vs); > + if (!tree) > + return 0; > + return bitmap_iterate(tree, walk_ioerr, &wioerr); > +} > + > +/* Given bad extent lists for the data & rtdev, find bad files. */ maybe "find and report bad files" just to tie it in w/ the "report" in the function name? is this only for media errors? Maybev xfs_report_media_errors? There are so many things going on in scrub (and apparently such a limited namespace for functions :D) that I find myself wishing for a little more context when I read something in isolation... > +static bool > +xfs_report_verify_errors( > + struct scrub_ctx *ctx, > + struct media_verify_state *vs) > +{ > + bool moveon; > + int ret; > + > + ret = walk_ioerrs(ctx, ctx->datadev, vs); > + if (ret) { > + str_liberror(ctx, ret, _("walking datadev io errors")); > + return false; > + } > + > + ret = walk_ioerrs(ctx, ctx->rtdev, vs); > + if (ret) { > + str_liberror(ctx, ret, _("walking rtdev io errors")); > + return false; > + } > + > + /* Scan the directory tree to get file paths. */ > + moveon = scan_fs_tree(ctx, xfs_report_verify_dir, > + xfs_report_verify_dirent, vs); > + if (!moveon) > + return false; > + > + /* Scan for unlinked files. */ > + return xfs_scan_all_inodes(ctx, xfs_report_verify_inode, vs); > } > > /* Schedule a read-verify of a (data block) extent. */ > @@ -571,7 +631,7 @@ xfs_scan_blocks( > } > > ret = read_verify_pool_alloc(ctx, ctx->datadev, > - ctx->mnt.fsgeom.blocksize, xfs_check_rmap_ioerr, > + ctx->mnt.fsgeom.blocksize, remember_ioerr, > scrub_nproc(ctx), &vs.rvp_data); > if (ret) { > str_liberror(ctx, ret, _("creating datadev media verifier")); > @@ -579,7 +639,7 @@ xfs_scan_blocks( > } > if (ctx->logdev) { > ret = read_verify_pool_alloc(ctx, ctx->logdev, > - ctx->mnt.fsgeom.blocksize, xfs_check_rmap_ioerr, > + ctx->mnt.fsgeom.blocksize, remember_ioerr, > scrub_nproc(ctx), &vs.rvp_log); > if (ret) { > str_liberror(ctx, ret, > @@ -589,7 +649,7 @@ xfs_scan_blocks( > } > if (ctx->rtdev) { > ret = read_verify_pool_alloc(ctx, ctx->rtdev, > - ctx->mnt.fsgeom.blocksize, xfs_check_rmap_ioerr, > + ctx->mnt.fsgeom.blocksize, remember_ioerr, > scrub_nproc(ctx), &vs.rvp_realtime); > if (ret) { > str_liberror(ctx, ret, >