From: Lukas Czerner <lczerner@redhat.com>
To: Theodore Ts'o <tytso@mit.edu>
Cc: Ext4 Developers List <linux-ext4@vger.kernel.org>,
darrick.wong@oracle.com
Subject: Re: [PATCH 8/9] e2scrub_all: refactor device probe loop
Date: Thu, 21 Mar 2019 11:27:42 +0100 [thread overview]
Message-ID: <20190321102742.k2oos4epoj6fyjao@work> (raw)
In-Reply-To: <20190321020218.5154-8-tytso@mit.edu>
On Wed, Mar 20, 2019 at 10:02:17PM -0400, Theodore Ts'o wrote:
> From: "Darrick J. Wong" <darrick.wong@oracle.com>
>
> Paul Menzel reported that the e2scrub_all reaper service that runs at
> startup takes a long time to run, and Ted Ts'o pointed out that we could
> do a lot less work by using lvs as the outer loop in the ext4 filesystem
> probe function so that we only have to lsblk the lvm devices containing
> ext4 filesystems.
>
> Therefore, refactor the loops to put lvs first, which should boost speed
> a bit.
>
> [ Made some of the further optimizations suggested by Lukas Czerner. -- TYT ]
>
> Reported-by: Paul Menzel <pmenzel@molgen.mpg.de>
> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
> ---
> scrub/e2scrub_all.in | 34 +++++++++++++++++++++++-----------
> 1 file changed, 23 insertions(+), 11 deletions(-)
>
> diff --git a/scrub/e2scrub_all.in b/scrub/e2scrub_all.in
> index 4cb90a0de..cad232987 100644
> --- a/scrub/e2scrub_all.in
> +++ b/scrub/e2scrub_all.in
> @@ -22,6 +22,7 @@ PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
>
> scrub_all=0
> snap_size_mb=256
> +reap=0
> conffile="@root_sysconfdir@/e2scrub.conf"
>
> test -f "${conffile}" && . "${conffile}"
> @@ -65,7 +66,7 @@ exitcode() {
> while getopts "nrAV" opt; do
> case "${opt}" in
> "n") DBG="echo Would execute: " ;;
> - "r") scrub_args="${scrub_args} -r";;
> + "r") scrub_args="${scrub_args} -r"; reap=1;;
> "A") scrub_all=1;;
> "V") print_version; exitcode 0;;
> *) print_help; exitcode 2;;
> @@ -88,9 +89,12 @@ if ! type lvcreate >& /dev/null ; then
> fi
>
> # Find scrub targets, make sure we only do this once.
> -ls_scrub_targets() {
> - lsblk -o NAME,FSTYPE,MOUNTPOINT -p -P -n | while read vars; do
> +ls_scan_targets() {
> + lvs --name-prefixes -o vg_name,lv_path \
> + -S lv_active=active,lv_role=public --noheadings | \
> + while read vars; do
> eval "${vars}"
> + eval "$(lsblk -o FSTYPE,MOUNTPOINT -p -P -n "${LVM2_LV_PATH}")"
>
> # Skip non-ext[234]
> case "${FSTYPE}" in
> @@ -103,12 +107,6 @@ ls_scrub_targets() {
> continue;
> fi
>
> - # Skip non-lvm devices and lvm snapshots
> - lvm_vars="$(lvs --nameprefixes -o vg_name,lv_name,lv_role --noheadings "${NAME}" 2> /dev/null)"
> - test $? -ne 0 && continue
> - eval "${lvm_vars}"
> - echo "${LVM2_LV_ROLE}" | grep -q "snapshot" && continue
This is wrong.
lvs --name-prefixes -o vg_name,lv_path \
-S lv_active=active,lv_role=public --noheadings | \
lv_role=public does not exclude snapshots and so it will fail later in
e2scrub when you try to create a snapshot of a thicksnapshot which is
not supported.
Snapshot of a thinspanshot is allowed though, so we might want to
include those. Not sure if it's wise to do it by default, but regardless
it's probably something for a separate change.
Also I am not sure what's the rush, but it seems you've ignored my other
suggestions.
-Lukas
> -
> free_space="$(vgs -o vg_free --units m --noheadings --no-suffix "${LVM2_VG_NAME}" 2> /dev/null | sed -e 's/\..*//')"
> test "${snap_size_mb}" -gt "${free_space}" && continue
>
> @@ -120,6 +118,20 @@ ls_scrub_targets() {
> done | sort | uniq
> }
>
> +# Find leftover scrub snapshots
> +ls_reap_targets() {
> + lvs -o lv_path -S lv_role=snapshot -S lv_name=~\(e2scrub$\) --noheadings
> +}
> +
> +# Figure out what we're targeting
> +ls_targets() {
> + if [ "${reap}" -eq 1 ]; then
> + ls_reap_targets
> + else
> + ls_scan_targets
> + fi
> +}
> +
> # systemd doesn't know to do path escaping on the instance variable we pass
> # to the e2scrub service, which breaks things if there is a dash in the path
> # name. Therefore, do the path escaping ourselves if needed.
> @@ -140,10 +152,10 @@ escape_path_for_systemd() {
>
> # Scrub any mounted fs on lvm by creating a snapshot and fscking that.
> stdin="$(realpath /dev/stdin)"
> -ls_scrub_targets | while read tgt; do
> +ls_targets | while read tgt; do
> # If we're not reaping and systemd is present, try invoking the
> # systemd service.
> - if [ -z "${scrub_args}" ] && type systemctl > /dev/null 2>&1; then
> + if [ "${reap}" -ne 1 ] && type systemctl > /dev/null 2>&1; then
> tgt_esc="$(escape_path_for_systemd "${tgt}")"
> ${DBG} systemctl start "e2scrub@${tgt_esc}" 2> /dev/null < "${stdin}"
> res=$?
> --
> 2.19.1
>
next prev parent reply other threads:[~2019-03-21 10:27 UTC|newest]
Thread overview: 39+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-03-21 2:02 [PATCH 1/9] e2scrub: check to make sure lvm2 is installed Theodore Ts'o
2019-03-21 2:02 ` [PATCH 2/9] debian: drop lvm2 from the recommends line Theodore Ts'o
2019-03-21 3:57 ` Darrick J. Wong
2019-03-21 2:02 ` [PATCH 3/9] Fix "make install-strip" Theodore Ts'o
2019-03-21 2:02 ` [PATCH 4/9] e2scrub: fix up "make install-strip" support Theodore Ts'o
2019-03-21 2:02 ` [PATCH 5/9] e2fscrub: add the -n option which shows what commands e2scrub would execute Theodore Ts'o
2019-03-21 3:59 ` Darrick J. Wong
2019-03-21 10:57 ` Lukas Czerner
2019-03-21 14:32 ` Theodore Ts'o
2019-03-21 2:02 ` [PATCH 6/9] e2scrub_all: add the -n option which shows what e2scrub_all would do Theodore Ts'o
2019-03-21 4:01 ` Darrick J. Wong
2019-03-21 2:02 ` [PATCH 7/9] e2scrub_all: make sure there's enough free space for a snapshot Theodore Ts'o
2019-03-21 4:02 ` Darrick J. Wong
2019-03-21 11:18 ` Lukas Czerner
2019-03-21 14:26 ` Theodore Ts'o
2019-03-21 2:02 ` [PATCH 8/9] e2scrub_all: refactor device probe loop Theodore Ts'o
2019-03-21 4:05 ` Darrick J. Wong
2019-03-21 10:27 ` Lukas Czerner [this message]
2019-03-21 14:31 ` Theodore Ts'o
2019-03-21 15:57 ` Lukas Czerner
2019-03-21 18:24 ` Theodore Ts'o
2019-03-21 20:17 ` Lukas Czerner
2019-03-21 20:48 ` Theodore Ts'o
2019-03-21 21:14 ` Lukas Czerner
2019-03-21 22:04 ` Theodore Ts'o
2019-03-21 22:08 ` Theodore Ts'o
2019-03-22 9:38 ` Lukas Czerner
2019-03-21 20:09 ` Andreas Dilger
2019-03-21 17:48 ` Theodore Ts'o
2019-03-21 19:49 ` Lukas Czerner
2019-03-21 20:23 ` Theodore Ts'o
2019-03-21 16:10 ` Lukas Czerner
2019-03-21 2:02 ` [PATCH 9/9] e2scrub,e2scrub_all: print a (more understandable) error if not run as root Theodore Ts'o
2019-03-21 4:04 ` Darrick J. Wong
2019-03-21 11:36 ` Lukas Czerner
2019-03-21 14:40 ` Theodore Ts'o
2019-03-21 3:55 ` [PATCH 1/9] e2scrub: check to make sure lvm2 is installed Darrick J. Wong
2019-03-21 20:25 [PATCH -v2 0/9] e2fsprogs: e2scrub cleanups Theodore Ts'o
2019-03-21 20:25 ` [PATCH 8/9] e2scrub_all: refactor device probe loop Theodore Ts'o
2019-03-21 20:55 ` Lukas Czerner
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20190321102742.k2oos4epoj6fyjao@work \
--to=lczerner@redhat.com \
--cc=darrick.wong@oracle.com \
--cc=linux-ext4@vger.kernel.org \
--cc=tytso@mit.edu \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).