linux-ext4.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [PATCH -v2 0/9] e2fsprogs: e2scrub cleanups
@ 2019-03-21 20:25 Theodore Ts'o
  2019-03-21 20:25 ` [PATCH 1/9] e2scrub: check to make sure lvm2 is installed Theodore Ts'o
                   ` (8 more replies)
  0 siblings, 9 replies; 12+ messages in thread
From: Theodore Ts'o @ 2019-03-21 20:25 UTC (permalink / raw)
  To: Ext4 Developers List; +Cc: darrick.wong, lczerner, Theodore Ts'o

The primary changes are in the device probe loop refactor for
e2scrub_all.

There were also some documentation changes for the -n option.

Darrick J. Wong (1):
  e2scrub_all: refactor device probe loop

Theodore Ts'o (8):
  e2scrub: check to make sure lvm2 is installed
  debian: drop lvm2 from the recommends line
  Fix "make install-strip"
  e2scrub: fix up "make install-strip" support
  e2scrub: add the -n option which shows what commands e2scrub would
    execute
  e2scrub_all: add the -n option which shows what e2scrub_all would do
  e2scrub_all: make sure there's enough free space for a snapshot
  e2scrub,e2scrub_all: print a (more understandable) error if not run as
    root

-- 
2.19.1


^ permalink raw reply	[flat|nested] 12+ messages in thread

* [PATCH 1/9] e2scrub: check to make sure lvm2 is installed
  2019-03-21 20:25 [PATCH -v2 0/9] e2fsprogs: e2scrub cleanups Theodore Ts'o
@ 2019-03-21 20:25 ` Theodore Ts'o
  2019-03-21 20:25 ` [PATCH 2/9] debian: drop lvm2 from the recommends line Theodore Ts'o
                   ` (7 subsequent siblings)
  8 siblings, 0 replies; 12+ messages in thread
From: Theodore Ts'o @ 2019-03-21 20:25 UTC (permalink / raw)
  To: Ext4 Developers List; +Cc: darrick.wong, lczerner, Theodore Ts'o

Not all systems will have the lvm2 package installed, so check for
that.  Pretty much all systems should have util-linux installed, but
check for that as well.

Of course, if lvm2 is installed we shouldn't find any LVM devices ---
but eventually the Demon Murphy will find a way to make it happen. :-)

Also, set the PATH so we don't have to worry about the script failing
due to /sbin not being in the path.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
---
 scrub/e2scrub.in     | 12 ++++++++++++
 scrub/e2scrub_all.in | 16 ++++++++++++++++
 2 files changed, 28 insertions(+)

diff --git a/scrub/e2scrub.in b/scrub/e2scrub.in
index e1965db4e..51a909373 100644
--- a/scrub/e2scrub.in
+++ b/scrub/e2scrub.in
@@ -23,6 +23,8 @@
 # check filesystems in VGs that have at least 256MB (or so) of
 # free space.
 
+PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
+
 snap_size_mb=256
 fstrim=0
 reap=0
@@ -82,6 +84,16 @@ if [ -z "${arg}" ]; then
 	exitcode 1
 fi
 
+if ! type lsblk >& /dev/null ; then
+    echo "e2scrub: can't find lsblk --- is util-linux installed?"
+    exitcode 1
+fi
+
+if ! type lvcreate >& /dev/null ; then
+    echo "e2scrub: can't find lvcreate --- is lvm2 installed?"
+    exitcode 1
+fi
+
 # Find the device for a given mountpoint
 dev_from_mount() {
 	local mountpt="$(realpath "$1")"
diff --git a/scrub/e2scrub_all.in b/scrub/e2scrub_all.in
index 23d122d25..d725a7f2e 100644
--- a/scrub/e2scrub_all.in
+++ b/scrub/e2scrub_all.in
@@ -18,6 +18,8 @@
 #  along with this program; if not, write the Free Software Foundation,
 #  Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301, USA.
 
+PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
+
 scrub_all=0
 conffile="@root_sysconfdir@/e2scrub.conf"
 
@@ -68,6 +70,20 @@ while getopts "ArV" opt; do
 done
 shift "$((OPTIND - 1))"
 
+# If some prerequisite packages are not installed, exit with a code
+# indicating success to avoid spamming the sysadmin with fail messages
+# when e2scrub_all is run out of cron or a systemd timer.
+
+if ! type lsblk >& /dev/null ; then
+    echo "e2scrub_all: can't find lsblk --- is util-linux installed?"
+    exitcode 0
+fi
+
+if ! type lvcreate >& /dev/null ; then
+    echo "e2scrub_all: can't find lvcreate --- is lvm2 installed?"
+    exitcode 0
+fi
+
 # Find scrub targets, make sure we only do this once.
 ls_scrub_targets() {
 	lsblk -o NAME,FSTYPE,MOUNTPOINT -p -P -n | while read vars; do
-- 
2.19.1


^ permalink raw reply related	[flat|nested] 12+ messages in thread

* [PATCH 2/9] debian: drop lvm2 from the recommends line
  2019-03-21 20:25 [PATCH -v2 0/9] e2fsprogs: e2scrub cleanups Theodore Ts'o
  2019-03-21 20:25 ` [PATCH 1/9] e2scrub: check to make sure lvm2 is installed Theodore Ts'o
@ 2019-03-21 20:25 ` Theodore Ts'o
  2019-03-21 20:25 ` [PATCH 3/9] Fix "make install-strip" Theodore Ts'o
                   ` (6 subsequent siblings)
  8 siblings, 0 replies; 12+ messages in thread
From: Theodore Ts'o @ 2019-03-21 20:25 UTC (permalink / raw)
  To: Ext4 Developers List; +Cc: darrick.wong, lczerner, Theodore Ts'o

If the user doesn't intend to use lvm2, and it's not installed,
installing e2fsprogs shouldn't drag it (and all of its dependencies)
into the system.

Addresses-Debian-Bug: 924275

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
---
 debian/control | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/debian/control b/debian/control
index 1bb8adb15..e454687f9 100644
--- a/debian/control
+++ b/debian/control
@@ -191,7 +191,7 @@ XB-Important: yes
 Pre-Depends: ${shlibs:Depends}, ${misc:Depends}, libblkid1, libuuid1
 Multi-Arch: foreign
 Suggests: gpart, parted, fuse2fs, e2fsck-static
-Recommends: e2fsprogs-l10n, lvm2
+Recommends: e2fsprogs-l10n
 Architecture: any
 Description: ext2/ext3/ext4 file system utilities
  The ext2, ext3 and ext4 file systems are successors of the original ext
-- 
2.19.1


^ permalink raw reply related	[flat|nested] 12+ messages in thread

* [PATCH 3/9] Fix "make install-strip"
  2019-03-21 20:25 [PATCH -v2 0/9] e2fsprogs: e2scrub cleanups Theodore Ts'o
  2019-03-21 20:25 ` [PATCH 1/9] e2scrub: check to make sure lvm2 is installed Theodore Ts'o
  2019-03-21 20:25 ` [PATCH 2/9] debian: drop lvm2 from the recommends line Theodore Ts'o
@ 2019-03-21 20:25 ` Theodore Ts'o
  2019-03-21 20:25 ` [PATCH 4/9] e2scrub: fix up "make install-strip" support Theodore Ts'o
                   ` (5 subsequent siblings)
  8 siblings, 0 replies; 12+ messages in thread
From: Theodore Ts'o @ 2019-03-21 20:25 UTC (permalink / raw)
  To: Ext4 Developers List; +Cc: darrick.wong, lczerner, Theodore Ts'o

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
---
 lib/Makefile.bsd-lib     | 2 +-
 lib/Makefile.darwin-lib  | 2 +-
 lib/Makefile.elf-lib     | 2 +-
 lib/Makefile.library     | 2 ++
 lib/Makefile.solaris-lib | 2 +-
 5 files changed, 6 insertions(+), 4 deletions(-)

diff --git a/lib/Makefile.bsd-lib b/lib/Makefile.bsd-lib
index db0947f9a..2792ba3df 100644
--- a/lib/Makefile.bsd-lib
+++ b/lib/Makefile.bsd-lib
@@ -36,7 +36,7 @@ install-shlibs install:: $(BSD_LIB)
 
 install-strip: install
 
-install-shlibs-strip: install-shlibs
+install-shlibs-strip:: install-shlibs
 
 uninstall-shlibs uninstall::
 	$(RM) -f $(DESTDIR)$(BSDLIB_INSTALL_DIR)/$(BSD_LIB)
diff --git a/lib/Makefile.darwin-lib b/lib/Makefile.darwin-lib
index c94a5e721..95cdd4b49 100644
--- a/lib/Makefile.darwin-lib
+++ b/lib/Makefile.darwin-lib
@@ -39,7 +39,7 @@ install-shlibs install:: $(BSD_LIB)
 
 install-strip: install
 
-install-shlibs-strip: install-shlibs
+install-shlibs-strip:: install-shlibs
 
 uninstall-shlibs uninstall::
 	$(RM) -f $(DESTDIR)$(BSDLIB_INSTALL_DIR)/$(BSD_LIB)
diff --git a/lib/Makefile.elf-lib b/lib/Makefile.elf-lib
index bd7b2b3c4..f850f3ddb 100644
--- a/lib/Makefile.elf-lib
+++ b/lib/Makefile.elf-lib
@@ -58,7 +58,7 @@ install-strip: install
 	$(Q) $(STRIP) --strip-unneeded --remove-section=.comment \
 		--remove-section=.note $(DESTDIR)$(ELF_INSTALL_DIR)/$(ELF_LIB)
 
-install-shlibs-strip: install-shlibs
+install-shlibs-strip:: install-shlibs
 	$(E) "	STRIP-LIB $(ELF_INSTALL_DIR)/$(ELF_LIB)"
 	$(Q) $(STRIP) --strip-unneeded --remove-section=.comment \
 		--remove-section=.note $(DESTDIR)$(ELF_INSTALL_DIR)/$(ELF_LIB)
diff --git a/lib/Makefile.library b/lib/Makefile.library
index 1b86b0235..f78467aa7 100644
--- a/lib/Makefile.library
+++ b/lib/Makefile.library
@@ -1,5 +1,7 @@
 all:: subdirs $(LIBRARY).a
 
+install-shlibs-strip::
+
 install-shlibs::
 
 uninstall-shlibs::
diff --git a/lib/Makefile.solaris-lib b/lib/Makefile.solaris-lib
index 304df7d40..1e636368f 100644
--- a/lib/Makefile.solaris-lib
+++ b/lib/Makefile.solaris-lib
@@ -50,7 +50,7 @@ install-shlibs install:: $(ELF_LIB) installdirs-elf-lib
 install-strip: install
 	$(STRIP) -x $(DESTDIR)$(ELF_INSTALL_DIR)/$(ELF_LIB)
 
-install-shlibs-strip: install-shlibs
+install-shlibs-strip:: install-shlibs
 	$(STRIP) -x $(DESTDIR)$(ELF_INSTALL_DIR)/$(ELF_LIB)
 
 uninstall-shlibs uninstall::
-- 
2.19.1


^ permalink raw reply related	[flat|nested] 12+ messages in thread

* [PATCH 4/9] e2scrub: fix up "make install-strip" support
  2019-03-21 20:25 [PATCH -v2 0/9] e2fsprogs: e2scrub cleanups Theodore Ts'o
                   ` (2 preceding siblings ...)
  2019-03-21 20:25 ` [PATCH 3/9] Fix "make install-strip" Theodore Ts'o
@ 2019-03-21 20:25 ` Theodore Ts'o
  2019-03-21 20:25 ` [PATCH 5/9] e2scrub: add the -n option which shows what commands e2scrub would execute Theodore Ts'o
                   ` (4 subsequent siblings)
  8 siblings, 0 replies; 12+ messages in thread
From: Theodore Ts'o @ 2019-03-21 20:25 UTC (permalink / raw)
  To: Ext4 Developers List; +Cc: darrick.wong, lczerner, Theodore Ts'o

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
---
 scrub/Makefile.in | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/scrub/Makefile.in b/scrub/Makefile.in
index 2f6d63014..f1e917fcf 100644
--- a/scrub/Makefile.in
+++ b/scrub/Makefile.in
@@ -129,6 +129,8 @@ install-systemd: $(SERVICE_FILES)
 		$(INSTALL_DATA) $$i $(DESTDIR)$(SYSTEMD_SYSTEM_UNIT_DIR)/$$i; \
 	done
 
+install-strip: install
+
 install: $(PROGS) $(MANPAGES) $(FMANPAGES) installdirs $(INSTALL_TGT)
 	$(Q) for i in $(PROGS); do \
 		$(ES) "	INSTALL $(root_sbindir)/$$i"; \
-- 
2.19.1


^ permalink raw reply related	[flat|nested] 12+ messages in thread

* [PATCH 5/9] e2scrub: add the -n option which shows what commands e2scrub would execute
  2019-03-21 20:25 [PATCH -v2 0/9] e2fsprogs: e2scrub cleanups Theodore Ts'o
                   ` (3 preceding siblings ...)
  2019-03-21 20:25 ` [PATCH 4/9] e2scrub: fix up "make install-strip" support Theodore Ts'o
@ 2019-03-21 20:25 ` Theodore Ts'o
  2019-03-21 20:25 ` [PATCH 6/9] e2scrub_all: add the -n option which shows what e2scrub_all would do Theodore Ts'o
                   ` (3 subsequent siblings)
  8 siblings, 0 replies; 12+ messages in thread
From: Theodore Ts'o @ 2019-03-21 20:25 UTC (permalink / raw)
  To: Ext4 Developers List; +Cc: darrick.wong, lczerner, Theodore Ts'o

Also fix the copyright symbol so it is properly formatted when
processed into postscript or pdf.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
---
 scrub/e2scrub.8.in | 11 ++++++++++-
 scrub/e2scrub.in   |  6 ++++--
 2 files changed, 14 insertions(+), 3 deletions(-)

diff --git a/scrub/e2scrub.8.in b/scrub/e2scrub.8.in
index ff03523e3..d0c8b2528 100644
--- a/scrub/e2scrub.8.in
+++ b/scrub/e2scrub.8.in
@@ -38,6 +38,15 @@ If the filesystem is not repaired,
 will be run before the next mount.
 .SH OPTIONS
 .TP
+\fB-n\fR
+Print what commands
+.B e2scrub
+would execute to check the file system.  (Note: the commands will not
+actually be executed; however, since
+.B e2scrub
+needs to run some commands to query the system to determine what
+commands would be executed, it still needs to be run as root.)
+.TP
 \fB-r\fR
 Remove the e2scrub snapshot and exit without checking anything.
 .TP
@@ -57,4 +66,4 @@ The exit codes are the same as in
 .SH AUTHOR
 Darrick J. Wong <darrick.wong@oracle.com>
 .SH COPYRIGHT
-Copyright ©2018 Oracle.  License is GPLv2+. <http://www.gnu.org/licenses/gpl-2.0.html>
+Copyright \[co]2018 Oracle.  License is GPLv2+. <http://www.gnu.org/licenses/gpl-2.0.html>
diff --git a/scrub/e2scrub.in b/scrub/e2scrub.in
index 51a909373..301574968 100644
--- a/scrub/e2scrub.in
+++ b/scrub/e2scrub.in
@@ -37,6 +37,7 @@ print_help() {
 	echo "Usage: $0 [OPTIONS] mountpoint | device"
 	echo
 	echo "mountpoint must be on a LVM-managed block device"
+	echo "-n: Show what commands e2scrub would execute."
 	echo "-r: Remove e2scrub snapshot and exit, do not check anything."
 	echo "-t: Run fstrim if successful."
 	echo "-V: Print version information and exit."
@@ -68,8 +69,9 @@ exitcode() {
 	exit "${ret}"
 }
 
-while getopts "rtV" opt; do
-	case "${opt}" in
+while getopts "nrtV" opt; do
+    case "${opt}" in
+	"n") DBG="echo Would execute: " ;;
 	"r") reap=1;;
 	"t") fstrim=1;;
 	"V") print_version; exitcode 0;;
-- 
2.19.1


^ permalink raw reply related	[flat|nested] 12+ messages in thread

* [PATCH 6/9] e2scrub_all: add the -n option which shows what e2scrub_all would do
  2019-03-21 20:25 [PATCH -v2 0/9] e2fsprogs: e2scrub cleanups Theodore Ts'o
                   ` (4 preceding siblings ...)
  2019-03-21 20:25 ` [PATCH 5/9] e2scrub: add the -n option which shows what commands e2scrub would execute Theodore Ts'o
@ 2019-03-21 20:25 ` Theodore Ts'o
  2019-03-21 20:25 ` [PATCH 7/9] e2scrub_all: make sure there's enough free space for a snapshot Theodore Ts'o
                   ` (2 subsequent siblings)
  8 siblings, 0 replies; 12+ messages in thread
From: Theodore Ts'o @ 2019-03-21 20:25 UTC (permalink / raw)
  To: Ext4 Developers List; +Cc: darrick.wong, lczerner, Theodore Ts'o

Also fix the copyright symbol so it is properly formatted when
processed into postscript or pdf.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
---
 scrub/e2scrub_all.8.in | 17 ++++++++++++++---
 scrub/e2scrub_all.in   |  8 +++++---
 2 files changed, 19 insertions(+), 6 deletions(-)

diff --git a/scrub/e2scrub_all.8.in b/scrub/e2scrub_all.8.in
index ba3b8735e..e2cd5e44a 100644
--- a/scrub/e2scrub_all.8.in
+++ b/scrub/e2scrub_all.8.in
@@ -20,12 +20,23 @@ See the
 manual page for more information about how the checking is performed.
 .SH OPTIONS
 .TP
-\fB-A\fR
-Scrub all ext[234] filesystems even if they are not mounted.
+\fB-n\fR
+Print what commands
+.B e2scrub_all
+would execute to initiate the e2scrub operations.
+(Note: these commands will not actually be executed; however, since
+.B e2scrub_all
+needs to run some additional, privileged commands to query the
+system to determine which
+.B e2scrub
+commands would be executed, it still needs to be run as root.)
 .TP
 \fB-r\fR
 Remove e2scrub snapshots but do not check anything.
 .TP
+\fB-A\fR
+Scrub all ext[234] filesystems even if they are not mounted.
+.TP
 \fB-V\fR
 Print version information and exit.
 .SH SEE ALSO
@@ -33,4 +44,4 @@ Print version information and exit.
 .SH AUTHOR
 Darrick J. Wong <darrick.wong@oracle.com>
 .SH COPYRIGHT
-Copyright ©2018 Oracle.  License is GPLv2+. <http://www.gnu.org/licenses/gpl-2.0.html>
+Copyright \[co]2018 Oracle.  License is GPLv2+. <http://www.gnu.org/licenses/gpl-2.0.html>
diff --git a/scrub/e2scrub_all.in b/scrub/e2scrub_all.in
index d725a7f2e..8bc868aa0 100644
--- a/scrub/e2scrub_all.in
+++ b/scrub/e2scrub_all.in
@@ -29,8 +29,9 @@ scrub_args=""
 
 print_help() {
 	echo "Usage: $0 [OPTIONS]"
-	echo " -A: Scrub all ext[234] filesystems even if not mounted."
+	echo " -n: Show what commands e2scrub_all would execute."
 	echo " -r: Remove e2scrub snapshots."
+	echo " -A: Scrub all ext[234] filesystems even if not mounted."
 	echo " -V: Print version information and exit."
 }
 
@@ -60,10 +61,11 @@ exitcode() {
 	exit "${ret}"
 }
 
-while getopts "ArV" opt; do
+while getopts "nrAV" opt; do
 	case "${opt}" in
-	"A") scrub_all=1;;
+	"n") DBG="echo Would execute: " ;;
 	"r") scrub_args="${scrub_args} -r";;
+	"A") scrub_all=1;;
 	"V") print_version; exitcode 0;;
 	*) print_help; exitcode 2;;
 	esac
-- 
2.19.1


^ permalink raw reply related	[flat|nested] 12+ messages in thread

* [PATCH 7/9] e2scrub_all: make sure there's enough free space for a snapshot
  2019-03-21 20:25 [PATCH -v2 0/9] e2fsprogs: e2scrub cleanups Theodore Ts'o
                   ` (5 preceding siblings ...)
  2019-03-21 20:25 ` [PATCH 6/9] e2scrub_all: add the -n option which shows what e2scrub_all would do Theodore Ts'o
@ 2019-03-21 20:25 ` Theodore Ts'o
  2019-03-21 20:25 ` [PATCH 8/9] e2scrub_all: refactor device probe loop Theodore Ts'o
  2019-03-21 20:25 ` [PATCH 9/9] e2scrub,e2scrub_all: print a (more understandable) error if not run as root Theodore Ts'o
  8 siblings, 0 replies; 12+ messages in thread
From: Theodore Ts'o @ 2019-03-21 20:25 UTC (permalink / raw)
  To: Ext4 Developers List; +Cc: darrick.wong, lczerner, Theodore Ts'o

If there isn't, skip the volume so we don't spam the system
administrator with error messages.  It's quite commkon that there is
is zero free space in the volume group.

Addresses-Debian-Bug: #924301

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
---
 scrub/e2scrub_all.in | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/scrub/e2scrub_all.in b/scrub/e2scrub_all.in
index 8bc868aa0..4cb90a0de 100644
--- a/scrub/e2scrub_all.in
+++ b/scrub/e2scrub_all.in
@@ -21,6 +21,7 @@
 PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
 
 scrub_all=0
+snap_size_mb=256
 conffile="@root_sysconfdir@/e2scrub.conf"
 
 test -f "${conffile}" && . "${conffile}"
@@ -108,6 +109,9 @@ ls_scrub_targets() {
 		eval "${lvm_vars}"
 		echo "${LVM2_LV_ROLE}" | grep -q "snapshot" && continue
 
+		free_space="$(vgs -o vg_free --units m --noheadings --no-suffix "${LVM2_VG_NAME}" 2> /dev/null | sed -e 's/\..*//')"
+		test "${snap_size_mb}" -gt "${free_space}" && continue
+
 		if [ -n "${MOUNTPOINT}" ]; then
 			echo "${MOUNTPOINT}"
 		else
-- 
2.19.1


^ permalink raw reply related	[flat|nested] 12+ messages in thread

* [PATCH 8/9] e2scrub_all: refactor device probe loop
  2019-03-21 20:25 [PATCH -v2 0/9] e2fsprogs: e2scrub cleanups Theodore Ts'o
                   ` (6 preceding siblings ...)
  2019-03-21 20:25 ` [PATCH 7/9] e2scrub_all: make sure there's enough free space for a snapshot Theodore Ts'o
@ 2019-03-21 20:25 ` Theodore Ts'o
  2019-03-21 20:55   ` Lukas Czerner
  2019-03-21 20:25 ` [PATCH 9/9] e2scrub,e2scrub_all: print a (more understandable) error if not run as root Theodore Ts'o
  8 siblings, 1 reply; 12+ messages in thread
From: Theodore Ts'o @ 2019-03-21 20:25 UTC (permalink / raw)
  To: Ext4 Developers List; +Cc: darrick.wong, lczerner, Theodore Ts'o

From: "Darrick J. Wong" <darrick.wong@oracle.com>

Paul Menzel reported that the e2scrub_all reaper service that runs at
startup takes a long time to run, and Ted Ts'o pointed out that we could
do a lot less work by using lvs as the outer loop in the ext4 filesystem
probe function so that we only have to lsblk the lvm devices containing
ext4 filesystems.

Therefore, refactor the loops to put lvs first, which should boost speed
a bit.

[ Made some of the further optimizations suggested by Lukas Czerner.  -- TYT ]

Reported-by: Paul Menzel <pmenzel@molgen.mpg.de>
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
---
 scrub/e2scrub_all.in | 49 ++++++++++++++++++++++----------------------
 1 file changed, 24 insertions(+), 25 deletions(-)

diff --git a/scrub/e2scrub_all.in b/scrub/e2scrub_all.in
index 4cb90a0de..81340b76f 100644
--- a/scrub/e2scrub_all.in
+++ b/scrub/e2scrub_all.in
@@ -22,6 +22,7 @@ PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
 
 scrub_all=0
 snap_size_mb=256
+reap=0
 conffile="@root_sysconfdir@/e2scrub.conf"
 
 test -f "${conffile}" && . "${conffile}"
@@ -65,7 +66,7 @@ exitcode() {
 while getopts "nrAV" opt; do
 	case "${opt}" in
 	"n") DBG="echo Would execute: " ;;
-	"r") scrub_args="${scrub_args} -r";;
+	"r") scrub_args="${scrub_args} -r"; reap=1;;
 	"A") scrub_all=1;;
 	"V") print_version; exitcode 0;;
 	*) print_help; exitcode 2;;
@@ -88,36 +89,34 @@ if ! type lvcreate >& /dev/null ; then
 fi
 
 # Find scrub targets, make sure we only do this once.
-ls_scrub_targets() {
-	lsblk -o NAME,FSTYPE,MOUNTPOINT -p -P -n | while read vars; do
-		eval "${vars}"
-
+ls_scan_targets() {
+	for NAME in $(lvs -o lv_path --noheadings \
+		   -S "lv_active=active,lv_role=public,lv_role!=snapshot,vg_free>${snap_size_mb}") ; do
 		# Skip non-ext[234]
-		case "${FSTYPE}" in
+		case "$(blkid -o value -s TYPE ${NAME})" in
 		ext[234])	;;
 		*)		continue;;
 		esac
 
-		# Skip unmounted filesystems unless -A
-		if [ "${scrub_all}" -eq 0 ] && [ -z "${MOUNTPOINT}" ]; then
-			continue;
+		if [ "${scrub_all}" -eq 1 ] ||
+		   [ -n "$(lsblk -o MOUNTPOINT --noheadings ${NAME})" ]; then
+		    echo ${NAME}
 		fi
+	done | sort | uniq
+}
 
-		# Skip non-lvm devices and lvm snapshots
-		lvm_vars="$(lvs --nameprefixes -o vg_name,lv_name,lv_role --noheadings "${NAME}" 2> /dev/null)"
-		test $? -ne 0 && continue
-		eval "${lvm_vars}"
-		echo "${LVM2_LV_ROLE}" | grep -q "snapshot" && continue
-
-		free_space="$(vgs -o vg_free --units m --noheadings --no-suffix "${LVM2_VG_NAME}" 2> /dev/null | sed -e 's/\..*//')"
-		test "${snap_size_mb}" -gt "${free_space}" && continue
+# Find leftover scrub snapshots
+ls_reap_targets() {
+	lvs -o lv_path -S lv_role=snapshot -S lv_name=~\(e2scrub$\) --noheadings
+}
 
-		if [ -n "${MOUNTPOINT}" ]; then
-			echo "${MOUNTPOINT}"
-		else
-			echo "${NAME}"
-		fi
-	done | sort | uniq
+# Figure out what we're targeting
+ls_targets() {
+	if [ "${reap}" -eq 1 ]; then
+		ls_reap_targets
+	else
+		ls_scan_targets
+	fi
 }
 
 # systemd doesn't know to do path escaping on the instance variable we pass
@@ -140,10 +139,10 @@ escape_path_for_systemd() {
 
 # Scrub any mounted fs on lvm by creating a snapshot and fscking that.
 stdin="$(realpath /dev/stdin)"
-ls_scrub_targets | while read tgt; do
+ls_targets | while read tgt; do
 	# If we're not reaping and systemd is present, try invoking the
 	# systemd service.
-	if [ -z "${scrub_args}" ] && type systemctl > /dev/null 2>&1; then
+	if [ "${reap}" -ne 1 ] && type systemctl > /dev/null 2>&1; then
 		tgt_esc="$(escape_path_for_systemd "${tgt}")"
 		${DBG} systemctl start "e2scrub@${tgt_esc}" 2> /dev/null < "${stdin}"
 		res=$?
-- 
2.19.1


^ permalink raw reply related	[flat|nested] 12+ messages in thread

* [PATCH 9/9] e2scrub,e2scrub_all: print a (more understandable) error if not run as root
  2019-03-21 20:25 [PATCH -v2 0/9] e2fsprogs: e2scrub cleanups Theodore Ts'o
                   ` (7 preceding siblings ...)
  2019-03-21 20:25 ` [PATCH 8/9] e2scrub_all: refactor device probe loop Theodore Ts'o
@ 2019-03-21 20:25 ` Theodore Ts'o
  8 siblings, 0 replies; 12+ messages in thread
From: Theodore Ts'o @ 2019-03-21 20:25 UTC (permalink / raw)
  To: Ext4 Developers List; +Cc: darrick.wong, lczerner, Theodore Ts'o

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
---
 scrub/e2scrub.in     | 5 +++++
 scrub/e2scrub_all.in | 5 +++++
 2 files changed, 10 insertions(+)

diff --git a/scrub/e2scrub.in b/scrub/e2scrub.in
index 301574968..666d6485a 100644
--- a/scrub/e2scrub.in
+++ b/scrub/e2scrub.in
@@ -25,6 +25,11 @@
 
 PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
 
+if (( $EUID != 0 )); then
+    echo "e2scrub must be run as root"
+    exit 1
+fi
+
 snap_size_mb=256
 fstrim=0
 reap=0
diff --git a/scrub/e2scrub_all.in b/scrub/e2scrub_all.in
index 81340b76f..3eca32e74 100644
--- a/scrub/e2scrub_all.in
+++ b/scrub/e2scrub_all.in
@@ -20,6 +20,11 @@
 
 PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
 
+if (( $EUID != 0 )); then
+    echo "e2scrub_all must be run as root"
+    exit 1
+fi
+
 scrub_all=0
 snap_size_mb=256
 reap=0
-- 
2.19.1


^ permalink raw reply related	[flat|nested] 12+ messages in thread

* Re: [PATCH 8/9] e2scrub_all: refactor device probe loop
  2019-03-21 20:25 ` [PATCH 8/9] e2scrub_all: refactor device probe loop Theodore Ts'o
@ 2019-03-21 20:55   ` Lukas Czerner
  0 siblings, 0 replies; 12+ messages in thread
From: Lukas Czerner @ 2019-03-21 20:55 UTC (permalink / raw)
  To: Theodore Ts'o; +Cc: Ext4 Developers List, darrick.wong

On Thu, Mar 21, 2019 at 04:25:12PM -0400, Theodore Ts'o wrote:
> From: "Darrick J. Wong" <darrick.wong@oracle.com>
> 
> Paul Menzel reported that the e2scrub_all reaper service that runs at
> startup takes a long time to run, and Ted Ts'o pointed out that we could
> do a lot less work by using lvs as the outer loop in the ext4 filesystem
> probe function so that we only have to lsblk the lvm devices containing
> ext4 filesystems.
> 
> Therefore, refactor the loops to put lvs first, which should boost speed
> a bit.
> 
> [ Made some of the further optimizations suggested by Lukas Czerner.  -- TYT ]
> 
> Reported-by: Paul Menzel <pmenzel@molgen.mpg.de>
> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
> ---
>  scrub/e2scrub_all.in | 49 ++++++++++++++++++++++----------------------
>  1 file changed, 24 insertions(+), 25 deletions(-)
> 
> diff --git a/scrub/e2scrub_all.in b/scrub/e2scrub_all.in
> index 4cb90a0de..81340b76f 100644
> --- a/scrub/e2scrub_all.in
> +++ b/scrub/e2scrub_all.in
> @@ -22,6 +22,7 @@ PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
>  
>  scrub_all=0
>  snap_size_mb=256
> +reap=0
>  conffile="@root_sysconfdir@/e2scrub.conf"
>  
>  test -f "${conffile}" && . "${conffile}"
> @@ -65,7 +66,7 @@ exitcode() {
>  while getopts "nrAV" opt; do
>  	case "${opt}" in
>  	"n") DBG="echo Would execute: " ;;
> -	"r") scrub_args="${scrub_args} -r";;
> +	"r") scrub_args="${scrub_args} -r"; reap=1;;
>  	"A") scrub_all=1;;
>  	"V") print_version; exitcode 0;;
>  	*) print_help; exitcode 2;;
> @@ -88,36 +89,34 @@ if ! type lvcreate >& /dev/null ; then
>  fi
>  
>  # Find scrub targets, make sure we only do this once.
> -ls_scrub_targets() {
> -	lsblk -o NAME,FSTYPE,MOUNTPOINT -p -P -n | while read vars; do
> -		eval "${vars}"
> -
> +ls_scan_targets() {
> +	for NAME in $(lvs -o lv_path --noheadings \
> +		   -S "lv_active=active,lv_role=public,lv_role!=snapshot,vg_free>${snap_size_mb}") ; do
>  		# Skip non-ext[234]
> -		case "${FSTYPE}" in
> +		case "$(blkid -o value -s TYPE ${NAME})" in

as I said in previous email for some reason blkid makes it terribly slow
for me.

	lsblk -o FSTYPE -n ${NAME}

lsblk makes is almost 9 times faster on my system. Not really sure what's
going on with blkid in my setup.

>  		ext[234])	;;
>  		*)		continue;;
>  		esac
>  
> -		# Skip unmounted filesystems unless -A
> -		if [ "${scrub_all}" -eq 0 ] && [ -z "${MOUNTPOINT}" ]; then
> -			continue;
> +		if [ "${scrub_all}" -eq 1 ] ||
> +		   [ -n "$(lsblk -o MOUNTPOINT --noheadings ${NAME})" ]; then
> +		    echo ${NAME}
>  		fi
> +	done | sort | uniq

It's already sorted and unique from lvs.

-Lukas

> +}
>  
> -		# Skip non-lvm devices and lvm snapshots
> -		lvm_vars="$(lvs --nameprefixes -o vg_name,lv_name,lv_role --noheadings "${NAME}" 2> /dev/null)"
> -		test $? -ne 0 && continue
> -		eval "${lvm_vars}"
> -		echo "${LVM2_LV_ROLE}" | grep -q "snapshot" && continue
> -
> -		free_space="$(vgs -o vg_free --units m --noheadings --no-suffix "${LVM2_VG_NAME}" 2> /dev/null | sed -e 's/\..*//')"
> -		test "${snap_size_mb}" -gt "${free_space}" && continue
> +# Find leftover scrub snapshots
> +ls_reap_targets() {
> +	lvs -o lv_path -S lv_role=snapshot -S lv_name=~\(e2scrub$\) --noheadings
> +}
>  
> -		if [ -n "${MOUNTPOINT}" ]; then
> -			echo "${MOUNTPOINT}"
> -		else
> -			echo "${NAME}"
> -		fi
> -	done | sort | uniq
> +# Figure out what we're targeting
> +ls_targets() {
> +	if [ "${reap}" -eq 1 ]; then
> +		ls_reap_targets
> +	else
> +		ls_scan_targets
> +	fi
>  }
>  
>  # systemd doesn't know to do path escaping on the instance variable we pass
> @@ -140,10 +139,10 @@ escape_path_for_systemd() {
>  
>  # Scrub any mounted fs on lvm by creating a snapshot and fscking that.
>  stdin="$(realpath /dev/stdin)"
> -ls_scrub_targets | while read tgt; do
> +ls_targets | while read tgt; do
>  	# If we're not reaping and systemd is present, try invoking the
>  	# systemd service.
> -	if [ -z "${scrub_args}" ] && type systemctl > /dev/null 2>&1; then
> +	if [ "${reap}" -ne 1 ] && type systemctl > /dev/null 2>&1; then
>  		tgt_esc="$(escape_path_for_systemd "${tgt}")"
>  		${DBG} systemctl start "e2scrub@${tgt_esc}" 2> /dev/null < "${stdin}"
>  		res=$?
> -- 
> 2.19.1
> 

^ permalink raw reply	[flat|nested] 12+ messages in thread

* [PATCH 3/9] Fix "make install-strip"
  2019-03-21  2:02 [PATCH 1/9] e2scrub: check to make sure lvm2 is installed Theodore Ts'o
@ 2019-03-21  2:02 ` Theodore Ts'o
  0 siblings, 0 replies; 12+ messages in thread
From: Theodore Ts'o @ 2019-03-21  2:02 UTC (permalink / raw)
  To: Ext4 Developers List; +Cc: darrick.wong, Theodore Ts'o

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
---
 lib/Makefile.bsd-lib     | 2 +-
 lib/Makefile.darwin-lib  | 2 +-
 lib/Makefile.elf-lib     | 2 +-
 lib/Makefile.library     | 2 ++
 lib/Makefile.solaris-lib | 2 +-
 5 files changed, 6 insertions(+), 4 deletions(-)

diff --git a/lib/Makefile.bsd-lib b/lib/Makefile.bsd-lib
index db0947f9a..2792ba3df 100644
--- a/lib/Makefile.bsd-lib
+++ b/lib/Makefile.bsd-lib
@@ -36,7 +36,7 @@ install-shlibs install:: $(BSD_LIB)
 
 install-strip: install
 
-install-shlibs-strip: install-shlibs
+install-shlibs-strip:: install-shlibs
 
 uninstall-shlibs uninstall::
 	$(RM) -f $(DESTDIR)$(BSDLIB_INSTALL_DIR)/$(BSD_LIB)
diff --git a/lib/Makefile.darwin-lib b/lib/Makefile.darwin-lib
index c94a5e721..95cdd4b49 100644
--- a/lib/Makefile.darwin-lib
+++ b/lib/Makefile.darwin-lib
@@ -39,7 +39,7 @@ install-shlibs install:: $(BSD_LIB)
 
 install-strip: install
 
-install-shlibs-strip: install-shlibs
+install-shlibs-strip:: install-shlibs
 
 uninstall-shlibs uninstall::
 	$(RM) -f $(DESTDIR)$(BSDLIB_INSTALL_DIR)/$(BSD_LIB)
diff --git a/lib/Makefile.elf-lib b/lib/Makefile.elf-lib
index bd7b2b3c4..f850f3ddb 100644
--- a/lib/Makefile.elf-lib
+++ b/lib/Makefile.elf-lib
@@ -58,7 +58,7 @@ install-strip: install
 	$(Q) $(STRIP) --strip-unneeded --remove-section=.comment \
 		--remove-section=.note $(DESTDIR)$(ELF_INSTALL_DIR)/$(ELF_LIB)
 
-install-shlibs-strip: install-shlibs
+install-shlibs-strip:: install-shlibs
 	$(E) "	STRIP-LIB $(ELF_INSTALL_DIR)/$(ELF_LIB)"
 	$(Q) $(STRIP) --strip-unneeded --remove-section=.comment \
 		--remove-section=.note $(DESTDIR)$(ELF_INSTALL_DIR)/$(ELF_LIB)
diff --git a/lib/Makefile.library b/lib/Makefile.library
index 1b86b0235..f78467aa7 100644
--- a/lib/Makefile.library
+++ b/lib/Makefile.library
@@ -1,5 +1,7 @@
 all:: subdirs $(LIBRARY).a
 
+install-shlibs-strip::
+
 install-shlibs::
 
 uninstall-shlibs::
diff --git a/lib/Makefile.solaris-lib b/lib/Makefile.solaris-lib
index 304df7d40..1e636368f 100644
--- a/lib/Makefile.solaris-lib
+++ b/lib/Makefile.solaris-lib
@@ -50,7 +50,7 @@ install-shlibs install:: $(ELF_LIB) installdirs-elf-lib
 install-strip: install
 	$(STRIP) -x $(DESTDIR)$(ELF_INSTALL_DIR)/$(ELF_LIB)
 
-install-shlibs-strip: install-shlibs
+install-shlibs-strip:: install-shlibs
 	$(STRIP) -x $(DESTDIR)$(ELF_INSTALL_DIR)/$(ELF_LIB)
 
 uninstall-shlibs uninstall::
-- 
2.19.1


^ permalink raw reply related	[flat|nested] 12+ messages in thread

end of thread, other threads:[~2019-03-21 20:55 UTC | newest]

Thread overview: 12+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-03-21 20:25 [PATCH -v2 0/9] e2fsprogs: e2scrub cleanups Theodore Ts'o
2019-03-21 20:25 ` [PATCH 1/9] e2scrub: check to make sure lvm2 is installed Theodore Ts'o
2019-03-21 20:25 ` [PATCH 2/9] debian: drop lvm2 from the recommends line Theodore Ts'o
2019-03-21 20:25 ` [PATCH 3/9] Fix "make install-strip" Theodore Ts'o
2019-03-21 20:25 ` [PATCH 4/9] e2scrub: fix up "make install-strip" support Theodore Ts'o
2019-03-21 20:25 ` [PATCH 5/9] e2scrub: add the -n option which shows what commands e2scrub would execute Theodore Ts'o
2019-03-21 20:25 ` [PATCH 6/9] e2scrub_all: add the -n option which shows what e2scrub_all would do Theodore Ts'o
2019-03-21 20:25 ` [PATCH 7/9] e2scrub_all: make sure there's enough free space for a snapshot Theodore Ts'o
2019-03-21 20:25 ` [PATCH 8/9] e2scrub_all: refactor device probe loop Theodore Ts'o
2019-03-21 20:55   ` Lukas Czerner
2019-03-21 20:25 ` [PATCH 9/9] e2scrub,e2scrub_all: print a (more understandable) error if not run as root Theodore Ts'o
  -- strict thread matches above, loose matches on Subject: below --
2019-03-21  2:02 [PATCH 1/9] e2scrub: check to make sure lvm2 is installed Theodore Ts'o
2019-03-21  2:02 ` [PATCH 3/9] Fix "make install-strip" Theodore Ts'o

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).