fstests.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Brian Foster <bfoster@redhat.com>
To: fstests@vger.kernel.org
Cc: linux-xfs@vger.kernel.org
Subject: [PATCH] xfs/502: scale file count based on AG count to avoid thrashing
Date: Fri,  9 Apr 2021 15:08:34 -0400	[thread overview]
Message-ID: <20210409190834.1026968-1-bfoster@redhat.com> (raw)

xfs/502 currently creates a default of 30k unlinked files per CPU.
While this completes in a reasonable amount of time on systems with
lesser numbers of CPUs, this scales poorly on high CPU count systems
that are otherwise testing smaller default filesystems. For example,
on an 80xcpu box and a 15GB (4 AG) XFS filesystem, xfs/502 requires
3 hours to complete. The same test on a 4xcpu vm (or the 80xcpu
hardware with an 80AG filesystem instead of the default of 4AGs)
completes in a little over 5 minutes. This is a rather severe
thrashing breakdown that doesn't add much value to the test
coverage.

Address this problem by scaling the file count to the AG count of
the filesystem rather than the CPU count of the test system. Since
the AG count is likely to be less than the CPU count, bump the
default scaling factor a bit from 30k per CPU to 50k per AG. From
there, larger counts can still be exercised via the global load
factor configuration.

Signed-off-by: Brian Foster <bfoster@redhat.com>
---
 tests/xfs/502 | 13 ++++++++++---
 1 file changed, 10 insertions(+), 3 deletions(-)

diff --git a/tests/xfs/502 b/tests/xfs/502
index 337ae07e..202bfbc6 100755
--- a/tests/xfs/502
+++ b/tests/xfs/502
@@ -28,6 +28,7 @@ _cleanup()
 # get standard environment, filters and checks
 . ./common/rc
 . ./common/inject
+. ./common/filter
 
 # real QA test starts here
 _supported_fs xfs
@@ -36,15 +37,21 @@ _require_scratch
 _require_test_program "t_open_tmpfiles"
 
 rm -f $seqres.full
-_scratch_mkfs >> $seqres.full 2>&1
+
+_scratch_mkfs | _filter_mkfs 2> $tmp.mkfs > /dev/null
+cat $tmp.mkfs >> $seqres.full
+. $tmp.mkfs
+
 _scratch_mount
 
 # Load up all the CPUs, two threads per CPU.
 nr_cpus=$(( $(getconf _NPROCESSORS_ONLN) * 2 ))
 
-# Set ULIMIT_NOFILE to min(file-max / $nr_cpus / 2, 30000 files per cpu per LOAD_FACTOR)
+# Set ULIMIT_NOFILE to min(file-max / $nr_cpus / 2, 50000 files per AG per LOAD_FACTOR)
 # so that this test doesn't take forever or OOM the box
-max_files=$((30000 * LOAD_FACTOR))
+max_files=$((50000 * agcount * LOAD_FACTOR))
+max_files=$((max_files / $nr_cpus))
+
 max_allowable_files=$(( $(cat /proc/sys/fs/file-max) / $nr_cpus / 2 ))
 test $max_allowable_files -gt 0 && test $max_files -gt $max_allowable_files && \
 	max_files=$max_allowable_files
-- 
2.26.3


                 reply	other threads:[~2021-04-09 19:08 UTC|newest]

Thread overview: [no followups] expand[flat|nested]  mbox.gz  Atom feed

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20210409190834.1026968-1-bfoster@redhat.com \
    --to=bfoster@redhat.com \
    --cc=fstests@vger.kernel.org \
    --cc=linux-xfs@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).