All of lore.kernel.org
 help / color / mirror / Atom feed
From: "Darrick J. Wong" <darrick.wong@oracle.com>
To: darrick.wong@oracle.com
Cc: linux-xfs@vger.kernel.org
Subject: [PATCH 16/16] xfs: abort scrubs if the oom killer fires
Date: Tue, 28 Nov 2017 17:27:02 -0800	[thread overview]
Message-ID: <151191882279.8553.4374638698461727073.stgit@magnolia> (raw)
In-Reply-To: <151191872395.8553.15627872818207535470.stgit@magnolia>

From: Darrick J. Wong <darrick.wong@oracle.com>

On a filesystem with a large amount of metadata, we can run the system
nearly out of memory while we process metadata.  If the OOM killer fires
anywhere in the system, ask the running scrub processes to abort with
ENOMEM and try again later.

(This will become more of a problem with online repair where we will
have to hold all of a reconstructed data structure in memory.)

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
 fs/xfs/scrub/common.h |    5 +++++
 fs/xfs/scrub/scrub.c  |   27 +++++++++++++++++++++++++++
 fs/xfs/scrub/scrub.h  |    4 ++++
 3 files changed, 36 insertions(+)


diff --git a/fs/xfs/scrub/common.h b/fs/xfs/scrub/common.h
index 6372456..16fa0b7 100644
--- a/fs/xfs/scrub/common.h
+++ b/fs/xfs/scrub/common.h
@@ -30,6 +30,11 @@ xfs_scrub_should_terminate(
 	struct xfs_scrub_context	*sc,
 	int				*error)
 {
+	if (sc->is_oom) {
+		if (*error == 0)
+			*error = -ENOMEM;
+		return true;
+	}
 	if (fatal_signal_pending(current)) {
 		if (*error == 0)
 			*error = -EAGAIN;
diff --git a/fs/xfs/scrub/scrub.c b/fs/xfs/scrub/scrub.c
index c4ad1b7..7edb26a 100644
--- a/fs/xfs/scrub/scrub.c
+++ b/fs/xfs/scrub/scrub.c
@@ -49,6 +49,8 @@
 #include "scrub/scrub.h"
 #include "scrub/btree.h"
 
+#include <linux/oom.h>
+
 /*
  * Online Scrub and Repair
  *
@@ -156,6 +158,9 @@ xfs_scrub_teardown(
 	struct xfs_inode		*ip_in,
 	int				error)
 {
+	if (sc->oom_notify.notifier_call)
+		unregister_oom_notifier(&sc->oom_notify);
+
 	xfs_scrub_ag_free(sc, &sc->sa);
 	if (sc->tp) {
 		xfs_trans_cancel(sc->tp);
@@ -295,6 +300,21 @@ xfs_scrub_experimental_warning(
 "EXPERIMENTAL online scrub feature in use. Use at your own risk!");
 }
 
+/* Uhoh, an OOM kill happened, try to kill any running scrubs. */
+static int
+xfs_scrub_oom_kill(
+	struct notifier_block		*notifier,
+	unsigned long			action,
+	void				*data)
+{
+	struct xfs_scrub_context	*sc;
+
+	sc = container_of(notifier, struct xfs_scrub_context, oom_notify);
+	sc->is_oom = true;
+
+	return NOTIFY_DONE;
+}
+
 /* Dispatch metadata scrubbing. */
 int
 xfs_scrub_metadata(
@@ -366,6 +386,13 @@ xfs_scrub_metadata(
 	sc.ops = ops;
 	sc.try_harder = try_harder;
 	sc.sa.agno = NULLAGNUMBER;
+	sc.oom_notify.notifier_call = xfs_scrub_oom_kill;
+	sc.oom_notify.priority = 1; /* call us first */
+	error = register_oom_notifier(&sc.oom_notify);
+	if (error) {
+		sc.oom_notify.notifier_call = NULL;
+		goto out_teardown;
+	}
 	error = sc.ops->setup(&sc, ip);
 	if (error)
 		goto out_teardown;
diff --git a/fs/xfs/scrub/scrub.h b/fs/xfs/scrub/scrub.h
index a1cd43d..610b88d6 100644
--- a/fs/xfs/scrub/scrub.h
+++ b/fs/xfs/scrub/scrub.h
@@ -63,6 +63,10 @@ struct xfs_scrub_context {
 	uint				ilock_flags;
 	bool				try_harder;
 
+	/* Kill scrub/repair if we OOM. */
+	struct notifier_block		oom_notify;
+	bool				is_oom;
+
 	/* State tracking for single-AG operations. */
 	struct xfs_scrub_ag		sa;
 };


      parent reply	other threads:[~2017-11-29  1:27 UTC|newest]

Thread overview: 17+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2017-11-29  1:25 [PATCH v10 00/16] xfs: online scrub xref support Darrick J. Wong
2017-11-29  1:25 ` [PATCH 01/16] xfs: add scrub cross-referencing helpers for the free space btrees Darrick J. Wong
2017-11-29  1:25 ` [PATCH 02/16] xfs: add scrub cross-referencing helpers for the inode btrees Darrick J. Wong
2017-11-29  1:25 ` [PATCH 03/16] xfs: add scrub cross-referencing helpers for the rmap btrees Darrick J. Wong
2017-11-29  1:25 ` [PATCH 04/16] xfs: add scrub cross-referencing helpers for the refcount btrees Darrick J. Wong
2017-11-29  1:25 ` [PATCH 05/16] xfs: set up scrub cross-referencing helpers Darrick J. Wong
2017-11-29  1:26 ` [PATCH 06/16] xfs: check btree block ownership with bnobt/rmapbt when scrubbing btree Darrick J. Wong
2017-11-29  1:26 ` [PATCH 07/16] xfs: introduce scrubber cross-referencing stubs Darrick J. Wong
2017-11-29  1:26 ` [PATCH 08/16] xfs: scrub should cross-reference with the bnobt Darrick J. Wong
2017-11-29  1:26 ` [PATCH 09/16] xfs: cross-reference bnobt records with cntbt Darrick J. Wong
2017-11-29  1:26 ` [PATCH 10/16] xfs: cross-reference inode btrees during scrub Darrick J. Wong
2017-11-29  1:26 ` [PATCH 11/16] xfs: cross-reference reverse-mapping btree Darrick J. Wong
2017-11-29  1:26 ` [PATCH 12/16] xfs: cross-reference the rmapbt data with the refcountbt Darrick J. Wong
2017-11-29  1:26 ` [PATCH 13/16] xfs: cross-reference refcount btree during scrub Darrick J. Wong
2017-11-29  1:26 ` [PATCH 14/16] xfs: scrub should cross-reference the realtime bitmap Darrick J. Wong
2017-11-29  1:26 ` [PATCH 15/16] xfs: cross-reference the block mappings when possible Darrick J. Wong
2017-11-29  1:27 ` Darrick J. Wong [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=151191882279.8553.4374638698461727073.stgit@magnolia \
    --to=darrick.wong@oracle.com \
    --cc=linux-xfs@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.