All of lore.kernel.org
 help / color / mirror / Atom feed
From: Alexander Patrakov <patrakov@gmail.com>
To: fstests@vger.kernel.org
Cc: Alexander Patrakov <patrakov@gmail.com>
Subject: [PATCH v2] _require_sparse_files: add a safeguard against media wearout
Date: Mon, 18 Dec 2023 05:00:53 +0800	[thread overview]
Message-ID: <20231217210053.9180-1-patrakov@gmail.com> (raw)
In-Reply-To: <CAN_LGv2UUOZvxN6w83tLudPZ5O9VPN0aJVDJay4rJDsvtcn29Q@mail.gmail.com>

_require_sparse_files is implemented as a list of filesystems known not to
support sparse files, and therefore it misses some cases.

However, if sparse files do not work as expected during a test, the risk
is that the test will write out to the disk all the zeros that would
normally be unwritten. This amounts to at least 4 TB for the generic/129
test, and therefore there is a significant media wearout concern here.

Adding more filesystems to the list of exclusions would not scale and
would not work anyway because CIFS backed by SAMBA is safe, while CIFS
backed by Windows Server 2022 is not.

In other words, Windows reserves the right to sometimes (!) ignore our
intent to create a sparse file.

More discussion: https://lore.kernel.org/fstests/20231206184759.GA3964019@frogsfrogsfrogs/T/#t

Mitigate this risk by doing a small-scale test that reliably triggers
Windows misbehavior and checking if the resulting file ends up being
not sufficiently sparse.

Signed-off-by: Alexander Patrakov <patrakov@gmail.com>
---
 common/rc | 23 +++++++++++++++++++++++
 1 file changed, 23 insertions(+)

diff --git a/common/rc b/common/rc
index cc92fe06..5d27602a 100644
--- a/common/rc
+++ b/common/rc
@@ -2871,6 +2871,12 @@ _require_fs_space()
 # Check if the filesystem supports sparse files.
 #
 # Unfortunately there is no better way to do this than a manual black list.
+# However, letting tests expand all the holes and write terabytes of zeros to
+# the media is also not acceptable due to wearout concerns.
+#
+# Note: even though CIFS supports sparse files, this test will mark it as
+# failing the requirement if we can coax the server into allocating and writing
+# the ranges where holes are expected. This happens with Windows servers.
 #
 _require_sparse_files()
 {
@@ -2881,6 +2887,23 @@ _require_sparse_files()
     *)
         ;;
     esac
+
+    local testfile="$TEST_DIR/$$.sparsefiletest"
+    rm -f "$testfile"
+
+    # A small-scale version of looptest - known to trigger Microsoft SMB server
+    # into the decision to write zeros to the disk. Also creates a non-sparse file
+    # on vfat.
+    # See also the discussion at https://lore.kernel.org/fstests/20231206184759.GA3964019@frogsfrogsfrogs/T/#t
+    $XFS_IO_PROG -f \
+	-c 'truncate 0' -c 'pwrite -b 102400 -S 0x61 102400 102400' \
+	-c 'truncate 0' -c 'pwrite -b 102400 -S 0x61 204800 102400' \
+	-c 'truncate 0' -c 'pwrite -b 102400 -S 0x61 307200 102400' \
+	-c 'truncate 0' -c 'pwrite -b 102400 -S 0x61 409600 102400' "$testfile" >/dev/null
+    resulting_file_size_kb=$( du -sk "$testfile" | cut -f 1 )
+    rm -f "$testfile"
+    [ $resulting_file_size_kb -ge 300 ] && \
+	_notrun "Sparse files do not work as expected, skipping test due to media wearout concerns"
 }
 
 _require_debugfs()
-- 
2.43.0


  reply	other threads:[~2023-12-17 21:01 UTC|newest]

Thread overview: 8+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-12-17 15:43 Excessive media wearout caused by generic/129 Alexander E. Patrakov
2023-12-17 17:24 ` [PATCH] generic/129: add a safeguard against media wearout Alexander Patrakov
2023-12-17 18:45 ` Excessive media wearout caused by generic/129 Alexander E. Patrakov
2023-12-17 21:00   ` Alexander Patrakov [this message]
2023-12-18 17:45     ` [PATCH v2] _require_sparse_files: add a safeguard against media wearout Darrick J. Wong
2023-12-18 18:27       ` Alexander E. Patrakov
2023-12-18 20:57         ` [PATCH v3] _require_sparse_files: rewrite as a direct test instead of a black list Alexander Patrakov
2023-12-18 22:45           ` Darrick J. Wong

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20231217210053.9180-1-patrakov@gmail.com \
    --to=patrakov@gmail.com \
    --cc=fstests@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.