All of lore.kernel.org
 help / color / mirror / Atom feed
From: Dave Chinner <david@fromorbit.com>
To: Theodore Ts'o <tytso@mit.edu>
Cc: fstests@vger.kernel.org, hughd@google.com, Junho Ryu <jayr@google.com>
Subject: Re: [PATCH 02/12] common: _scratch_mkfs_sized() for tmpfs
Date: Thu, 11 Feb 2016 09:37:29 +1100	[thread overview]
Message-ID: <20160210223729.GY19486@dastard> (raw)
In-Reply-To: <20160210155837.GD26922@thunk.org>

On Wed, Feb 10, 2016 at 10:58:37AM -0500, Theodore Ts'o wrote:
> On Wed, Feb 10, 2016 at 05:00:58PM +1100, Dave Chinner wrote:
> > > +    tmpfs)
> > > +	export MOUNT_OPTIONS="-o size=$fssize $TMPFS_MOUNT_OPTIONS"
> > 
> > So what happens when the test asks for 10GB of device space, and you
> > only have 4GB of RAM?
> 
> The largest amount of storage space requested by shared/generic tests
> via _scratch_mkfs_sized is 2G, and that's from generic/27[345].

Sure, but that doesn't doesn't invalidate my question.

I run xfstests on machines with 1GB RAM, so those tests you list
above are going to have problems with tmpfs, yes?

> We currently could run into problems today if the storage device only
> had 3G of space, and some test, such as f2fs/001, called
> _scratch_mkfs_sized requesting 4G of space.  We currently aren't
> checking error returns from the mkfs command; we just assume that it
> will succeed.

That would be a test bug.  We have a general rule of thumb that
test/scratch devices are typically a couple of GB in size, so if the
test needs less space than that it doesn't need to check. If the
test needs more than 1-2GB of space, then it needs to call
_require_fs_space to abort the test if the device is too small.
Hence we end up with tests being not run with:

generic/312      [not run] this test requires $SCRATCH_DEV has 5368709120B space

This needs to work for tmpfs, too, so that we don't run tests that
will overcommit RAM on machines with limited memory.

> Because of the someone interesting behavior of tmpfs assuming inode
> space is free, we already have problems where even with 8G of memory,
> generic/269 and generic/273 will result in the test getting OOM
> killed, and if you use less than 4G, there are a more tests that end
> up getting OOM killed as a result.

As I said, this is a tmpfs bug that needs fixing, not working around
in xfstests. If tmpfs is limited to 512MB or 1GB of space, then
having it be able to consume multiple times it's requested size in
memory is simply wrong.

> So I think it would be fair to simply document that if you are
> testing tmpfs, you need a VM or or a memory container with at
> least 8G of memory. If we start limiting the number of inodes in a
> tmpfs mount, we could probably bring that requirement down to 4G,
> which is certainly fair.  We do document minimum requirements for
> the size of the TEST_DEV and SCRATCH_DEV, don't we? 

No, not that I know of.

> (Minimum
> requirements certainly do exist in practice, in any case.)

See above.

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

  reply	other threads:[~2016-02-10 22:37 UTC|newest]

Thread overview: 44+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2016-02-10  1:49 [PATCH 00/12] xfstests: fix up various tmpfs failures Theodore Ts'o
2016-02-10  1:49 ` [PATCH 01/12] check: avoid error messages of tests/$FS does not exist Theodore Ts'o
2016-02-10  5:45   ` Dave Chinner
2016-02-10 15:34     ` Theodore Ts'o
2016-02-10 16:28       ` Christoph Hellwig
2016-02-10 23:17         ` Theodore Ts'o
2016-02-10 22:19       ` Dave Chinner
2016-02-10 23:32         ` Theodore Ts'o
2016-02-10  1:49 ` [PATCH 02/12] common: _scratch_mkfs_sized() for tmpfs Theodore Ts'o
2016-02-10  6:00   ` Dave Chinner
2016-02-10 15:58     ` Theodore Ts'o
2016-02-10 22:37       ` Dave Chinner [this message]
2016-02-10  1:49 ` [PATCH 03/12] generic: use mount point instead of device name Theodore Ts'o
2016-02-10  1:49 ` [PATCH 04/12] generic: add _require_odirect to three more tests Theodore Ts'o
2016-02-10  9:15   ` Eryu Guan
2016-02-10 16:11     ` Theodore Ts'o
2016-02-10 22:51       ` Dave Chinner
2016-02-10 23:21         ` Theodore Ts'o
2016-02-10  1:49 ` [PATCH 05/12] xfstests: do not unmount tmpfs during remount Theodore Ts'o
2016-02-10  6:07   ` Dave Chinner
2016-02-10 16:07     ` Theodore Ts'o
2016-02-10 18:04       ` Theodore Ts'o
2016-02-10 23:07       ` Dave Chinner
2016-02-10 23:28         ` Theodore Ts'o
2016-02-11  3:07           ` Dave Chinner
2016-02-11 15:25             ` Theodore Ts'o
2016-02-11 17:36               ` Darrick J. Wong
2016-02-10  1:49 ` [PATCH 06/12] generic: do not unmount before calling _check_scratch_fs() Theodore Ts'o
2016-02-10  1:49 ` [PATCH 07/12] generic: require fiemap for generic/009 Theodore Ts'o
2016-02-10  1:49 ` [PATCH 08/12] xfstests: fix generic/312 on tmpfs, ignore /proc/partitions Theodore Ts'o
2016-02-10  5:54   ` Dave Chinner
2016-02-10 23:39     ` Theodore Ts'o
2016-02-11  2:53       ` Dave Chinner
2016-02-10  1:49 ` [PATCH 09/12] xfstests: generic/079 requires chattr, not xattrs Theodore Ts'o
2016-02-10  9:09   ` Eryu Guan
2016-02-10 16:09     ` Theodore Ts'o
2016-02-10  1:49 ` [PATCH 10/12] generic: disable generic/027 for tmpfs Theodore Ts'o
2016-02-10  5:58   ` Dave Chinner
2016-02-10 15:48     ` Theodore Ts'o
2016-02-10 23:13       ` Dave Chinner
2016-02-10  1:50 ` [PATCH 11/12] xfstests: add executable permission to tests Theodore Ts'o
2016-02-10  9:07   ` Eryu Guan
2016-02-10  1:50 ` [PATCH 12/12] xfstests: increase tmpfs memory size Theodore Ts'o
2016-02-10  2:10 ` [PATCH 00/12] xfstests: fix up various tmpfs failures Theodore Ts'o

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20160210223729.GY19486@dastard \
    --to=david@fromorbit.com \
    --cc=fstests@vger.kernel.org \
    --cc=hughd@google.com \
    --cc=jayr@google.com \
    --cc=tytso@mit.edu \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.