From: Philipp Reisner <philipp.reisner@linbit.com>
To: Jens Axboe <axboe@fb.com>, linux-kernel@vger.kernel.org
Cc: drbd-dev@lists.linbit.com
Subject: [PATCH 17/30] drbd: don't forget error completion when "unsuspending" IO
Date: Mon, 13 Jun 2016 16:09:05 +0200 [thread overview]
Message-ID: <1465826958-19398-18-git-send-email-philipp.reisner@linbit.com> (raw)
In-Reply-To: <1465826958-19398-1-git-send-email-philipp.reisner@linbit.com>
From: Lars Ellenberg <lars.ellenberg@linbit.com>
Possibly sequence of events:
SyncTarget is made Primary, then loses replication link
(only path to good data on SyncSource).
Behavior is then controlled by the on-no-data-accessible policy,
which defaults to OND_IO_ERROR (may be set to OND_SUSPEND_IO).
If OND_IO_ERROR is in fact the current policy, we clear the susp_fen
(IO suspended due to fencing policy) flag, do NOT set the susp_nod
(IO suspended due to no data) flag.
But we forgot to call the IO error completion for all pending,
suspended, requests.
While at it, also add a race check for a theoretically possible
race with a new handshake (network hickup), we may be able to
re-send requests, and can avoid passing IO errors up the stack.
Signed-off-by: Philipp Reisner <philipp.reisner@linbit.com>
Signed-off-by: Lars Ellenberg <lars.ellenberg@linbit.com>
---
drivers/block/drbd/drbd_nl.c | 48 +++++++++++++++++++++++++++++---------------
1 file changed, 32 insertions(+), 16 deletions(-)
diff --git a/drivers/block/drbd/drbd_nl.c b/drivers/block/drbd/drbd_nl.c
index 4a4eb80..e5fdcc6 100644
--- a/drivers/block/drbd/drbd_nl.c
+++ b/drivers/block/drbd/drbd_nl.c
@@ -442,19 +442,17 @@ static enum drbd_fencing_p highest_fencing_policy(struct drbd_connection *connec
}
rcu_read_unlock();
- if (fp == FP_NOT_AVAIL) {
- /* IO Suspending works on the whole resource.
- Do it only for one device. */
- vnr = 0;
- peer_device = idr_get_next(&connection->peer_devices, &vnr);
- drbd_change_state(peer_device->device, CS_VERBOSE | CS_HARD, NS(susp_fen, 0));
- }
-
return fp;
}
+static bool resource_is_supended(struct drbd_resource *resource)
+{
+ return resource->susp || resource->susp_fen || resource->susp_nod;
+}
+
bool conn_try_outdate_peer(struct drbd_connection *connection)
{
+ struct drbd_resource * const resource = connection->resource;
unsigned int connect_cnt;
union drbd_state mask = { };
union drbd_state val = { };
@@ -462,21 +460,41 @@ bool conn_try_outdate_peer(struct drbd_connection *connection)
char *ex_to_string;
int r;
- spin_lock_irq(&connection->resource->req_lock);
+ spin_lock_irq(&resource->req_lock);
if (connection->cstate >= C_WF_REPORT_PARAMS) {
drbd_err(connection, "Expected cstate < C_WF_REPORT_PARAMS\n");
- spin_unlock_irq(&connection->resource->req_lock);
+ spin_unlock_irq(&resource->req_lock);
return false;
}
connect_cnt = connection->connect_cnt;
- spin_unlock_irq(&connection->resource->req_lock);
+ spin_unlock_irq(&resource->req_lock);
fp = highest_fencing_policy(connection);
switch (fp) {
case FP_NOT_AVAIL:
drbd_warn(connection, "Not fencing peer, I'm not even Consistent myself.\n");
- goto out;
+ spin_lock_irq(&resource->req_lock);
+ if (connection->cstate < C_WF_REPORT_PARAMS) {
+ _conn_request_state(connection,
+ (union drbd_state) { { .susp_fen = 1 } },
+ (union drbd_state) { { .susp_fen = 0 } },
+ CS_VERBOSE | CS_HARD | CS_DC_SUSP);
+ /* We are no longer suspended due to the fencing policy.
+ * We may still be suspended due to the on-no-data-accessible policy.
+ * If that was OND_IO_ERROR, fail pending requests. */
+ if (!resource_is_supended(resource))
+ _tl_restart(connection, CONNECTION_LOST_WHILE_PENDING);
+ }
+ /* Else: in case we raced with a connection handshake,
+ * let the handshake figure out if we maybe can RESEND,
+ * and do not resume/fail pending requests here.
+ * Worst case is we stay suspended for now, which may be
+ * resolved by either re-establishing the replication link, or
+ * the next link failure, or eventually the administrator. */
+ spin_unlock_irq(&resource->req_lock);
+ return false;
+
case FP_DONT_CARE:
return true;
default: ;
@@ -529,13 +547,11 @@ bool conn_try_outdate_peer(struct drbd_connection *connection)
drbd_info(connection, "fence-peer helper returned %d (%s)\n",
(r>>8) & 0xff, ex_to_string);
- out:
-
/* Not using
conn_request_state(connection, mask, val, CS_VERBOSE);
here, because we might were able to re-establish the connection in the
meantime. */
- spin_lock_irq(&connection->resource->req_lock);
+ spin_lock_irq(&resource->req_lock);
if (connection->cstate < C_WF_REPORT_PARAMS && !test_bit(STATE_SENT, &connection->flags)) {
if (connection->connect_cnt != connect_cnt)
/* In case the connection was established and droped
@@ -544,7 +560,7 @@ bool conn_try_outdate_peer(struct drbd_connection *connection)
else
_conn_request_state(connection, mask, val, CS_VERBOSE);
}
- spin_unlock_irq(&connection->resource->req_lock);
+ spin_unlock_irq(&resource->req_lock);
return conn_highest_pdsk(connection) <= D_OUTDATED;
}
--
2.7.4
next prev parent reply other threads:[~2016-06-13 14:19 UTC|newest]
Thread overview: 64+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-06-13 14:08 [PATCH 00/30] DRBD updates Philipp Reisner
2016-06-13 14:08 ` [PATCH 01/30] drbd: bitmap bulk IO: do not always suspend IO Philipp Reisner
2016-06-13 14:08 ` [PATCH 02/30] drbd: change bitmap write-out when leaving resync states Philipp Reisner
2016-06-13 14:08 ` [PATCH 03/30] drbd: Kill code duplication Philipp Reisner
2016-06-13 14:08 ` [PATCH 04/30] drbd: Implement handling of thinly provisioned storage on resync target nodes Philipp Reisner
2016-06-13 14:08 ` [PATCH 05/30] drbd: Introduce new disk config option rs-discard-granularity Philipp Reisner
2016-06-13 14:08 ` [PATCH 06/30] drbd: Create the protocol feature THIN_RESYNC Philipp Reisner
2016-06-13 14:08 ` [PATCH 07/30] drbd: adjust assert in w_bitmap_io to account for BM_LOCKED_CHANGE_ALLOWED Philipp Reisner
2016-06-13 14:08 ` [PATCH 08/30] drbd: fix regression: protocol A sometimes synchronous, C sometimes double-latency Philipp Reisner
2016-06-13 14:08 ` [PATCH 09/30] drbd: fix for truncated minor number in callback command line Philipp Reisner
2016-06-13 14:08 ` [PATCH 10/30] drbd: allow parallel flushes for multi-volume resources Philipp Reisner
2016-06-13 14:08 ` [PATCH 11/30] drbd: when receiving P_TRIM, zero-out partial unaligned chunks Philipp Reisner
2016-06-13 14:09 ` [PATCH 12/30] drbd: possibly disable discard support, if backend has discard_zeroes_data=0 Philipp Reisner
2016-06-13 14:09 ` [PATCH 13/30] drbd: zero-out partial unaligned discards on local backend Philipp Reisner
2016-06-13 14:09 ` [PATCH 14/30] drbd: allow larger max_discard_sectors Philipp Reisner
2016-06-13 14:09 ` [PATCH 15/30] drbd: finish resync on sync source only by notification from sync target Philipp Reisner
2016-06-13 14:09 ` [PATCH 16/30] drbd: introduce unfence-peer handler Philipp Reisner
2016-06-13 14:09 ` Philipp Reisner [this message]
2016-06-13 14:09 ` [PATCH 18/30] drbd: if there is no good data accessible, writes should be IO errors Philipp Reisner
2016-06-13 14:09 ` [PATCH 19/30] drbd: only restart frozen disk io when D_UP_TO_DATE Philipp Reisner
2016-06-13 14:09 ` [PATCH 20/30] drbd: discard_zeroes_if_aligned allows "thin" resync for discard_zeroes_data=0 Philipp Reisner
2016-06-13 14:09 ` [PATCH 21/30] drbd: report sizes if rejecting too small peer disk Philipp Reisner
2016-06-13 14:09 ` [PATCH 22/30] drbd: introduce WRITE_SAME support Philipp Reisner
2016-06-13 14:09 ` [PATCH 23/30] drbd: sync_handshake: handle identical uuids with current (frozen) Primary Philipp Reisner
2016-06-13 14:09 ` [PATCH 24/30] drbd: disallow promotion during resync handshake, avoid deadlock and hard reset Philipp Reisner
2016-06-13 14:09 ` [PATCH 25/30] drbd: bump current uuid when resuming IO with diskless peer Philipp Reisner
2016-06-13 14:09 ` [PATCH 26/30] drbd: code cleanups without semantic changes Philipp Reisner
2016-06-13 14:09 ` [PATCH 27/30] drbd: get rid of empty statement in is_valid_state Philipp Reisner
2016-06-13 14:09 ` [PATCH 28/30] drbd: finally report ms, not jiffies, in log message Philipp Reisner
2016-06-13 14:09 ` [PATCH 29/30] drbd: al_write_transaction: skip re-scanning of bitmap page pointer array Philipp Reisner
2016-06-13 14:09 ` [PATCH 30/30] drbd: correctly handle failed crypto_alloc_hash Philipp Reisner
2016-06-13 15:11 ` [PATCH 00/30] DRBD updates Jens Axboe
2016-06-13 22:26 ` Philipp Reisner
2016-06-13 22:26 ` [PATCH 01/30] drbd: bitmap bulk IO: do not always suspend IO Philipp Reisner
2016-06-13 22:26 ` [PATCH 02/30] drbd: change bitmap write-out when leaving resync states Philipp Reisner
2016-06-13 22:26 ` [PATCH 03/30] drbd: Kill code duplication Philipp Reisner
2016-06-13 22:26 ` [PATCH 04/30] drbd: Implement handling of thinly provisioned storage on resync target nodes Philipp Reisner
2016-06-13 22:26 ` [PATCH 05/30] drbd: Introduce new disk config option rs-discard-granularity Philipp Reisner
2016-06-13 22:26 ` [PATCH 06/30] drbd: Create the protocol feature THIN_RESYNC Philipp Reisner
2016-06-13 22:26 ` [PATCH 07/30] drbd: adjust assert in w_bitmap_io to account for BM_LOCKED_CHANGE_ALLOWED Philipp Reisner
2016-06-13 22:26 ` [PATCH 08/30] drbd: fix regression: protocol A sometimes synchronous, C sometimes double-latency Philipp Reisner
2016-06-13 22:26 ` [PATCH 09/30] drbd: fix for truncated minor number in callback command line Philipp Reisner
2016-06-13 22:26 ` [PATCH 10/30] drbd: allow parallel flushes for multi-volume resources Philipp Reisner
2016-06-13 22:26 ` [PATCH 11/30] drbd: when receiving P_TRIM, zero-out partial unaligned chunks Philipp Reisner
2016-06-13 22:26 ` [PATCH 12/30] drbd: possibly disable discard support, if backend has discard_zeroes_data=0 Philipp Reisner
2016-06-13 22:26 ` [PATCH 13/30] drbd: zero-out partial unaligned discards on local backend Philipp Reisner
2016-06-13 22:26 ` [PATCH 14/30] drbd: allow larger max_discard_sectors Philipp Reisner
2016-06-13 22:26 ` [PATCH 15/30] drbd: finish resync on sync source only by notification from sync target Philipp Reisner
2016-06-13 22:26 ` [PATCH 16/30] drbd: introduce unfence-peer handler Philipp Reisner
2016-06-13 22:26 ` [PATCH 17/30] drbd: don't forget error completion when "unsuspending" IO Philipp Reisner
2016-06-13 22:26 ` [PATCH 18/30] drbd: if there is no good data accessible, writes should be IO errors Philipp Reisner
2016-06-13 22:26 ` [PATCH 19/30] drbd: only restart frozen disk io when D_UP_TO_DATE Philipp Reisner
2016-06-13 22:26 ` [PATCH 20/30] drbd: discard_zeroes_if_aligned allows "thin" resync for discard_zeroes_data=0 Philipp Reisner
2016-06-13 22:26 ` [PATCH 21/30] drbd: report sizes if rejecting too small peer disk Philipp Reisner
2016-06-13 22:26 ` [PATCH 22/30] drbd: introduce WRITE_SAME support Philipp Reisner
2016-06-13 22:26 ` [PATCH 23/30] drbd: sync_handshake: handle identical uuids with current (frozen) Primary Philipp Reisner
2016-06-13 22:26 ` [PATCH 24/30] drbd: disallow promotion during resync handshake, avoid deadlock and hard reset Philipp Reisner
2016-06-13 22:26 ` [PATCH 25/30] drbd: bump current uuid when resuming IO with diskless peer Philipp Reisner
2016-06-13 22:26 ` [PATCH 26/30] drbd: code cleanups without semantic changes Philipp Reisner
2016-06-13 22:26 ` [PATCH 27/30] drbd: get rid of empty statement in is_valid_state Philipp Reisner
2016-06-13 22:26 ` [PATCH 28/30] drbd: finally report ms, not jiffies, in log message Philipp Reisner
2016-06-13 22:26 ` [PATCH 29/30] drbd: al_write_transaction: skip re-scanning of bitmap page pointer array Philipp Reisner
2016-06-13 22:26 ` [PATCH 30/30] drbd: correctly handle failed crypto_alloc_hash Philipp Reisner
-- strict thread matches above, loose matches on Subject: below --
2016-04-25 12:07 [PATCH 00/30] DBRD updates Philipp Reisner
2016-04-25 12:07 ` [PATCH 17/30] drbd: don't forget error completion when "unsuspending" IO Philipp Reisner
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1465826958-19398-18-git-send-email-philipp.reisner@linbit.com \
--to=philipp.reisner@linbit.com \
--cc=axboe@fb.com \
--cc=drbd-dev@lists.linbit.com \
--cc=linux-kernel@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).