From: Sage Weil <sage@newdream.net>
To: ceph-devel@vger.kernel.org
Cc: Sage Weil <sage@newdream.net>
Subject: [PATCH 07/23] libceph: don't time out osd requests that haven't been received
Date: Fri, 22 Jul 2011 17:02:46 -0700 [thread overview]
Message-ID: <1311379382-9218-8-git-send-email-sage@newdream.net> (raw)
In-Reply-To: <1311379382-9218-1-git-send-email-sage@newdream.net>
Keep track of when an outgoing message is ACKed (i.e., the server fully
received it and, presumably, queued it for processing). Time out OSD
requests only if it's been too long since they've been received.
This prevents timeouts and connection thrashing when the OSDs are simply
busy and are throttling the requests they read off the network.
Signed-off-by: Sage Weil <sage@newdream.net>
---
include/linux/ceph/messenger.h | 1 +
net/ceph/messenger.c | 12 +++++-------
net/ceph/osd_client.c | 6 ++++++
3 files changed, 12 insertions(+), 7 deletions(-)
diff --git a/include/linux/ceph/messenger.h b/include/linux/ceph/messenger.h
index 31d91a6..d7adf15 100644
--- a/include/linux/ceph/messenger.h
+++ b/include/linux/ceph/messenger.h
@@ -94,6 +94,7 @@ struct ceph_msg {
bool more_to_follow;
bool needs_out_seq;
int front_max;
+ unsigned long ack_stamp; /* tx: when we were acked */
struct ceph_msgpool *pool;
};
diff --git a/net/ceph/messenger.c b/net/ceph/messenger.c
index 78b55f4..c340e2e 100644
--- a/net/ceph/messenger.c
+++ b/net/ceph/messenger.c
@@ -486,13 +486,10 @@ static void prepare_write_message(struct ceph_connection *con)
m = list_first_entry(&con->out_queue,
struct ceph_msg, list_head);
con->out_msg = m;
- if (test_bit(LOSSYTX, &con->state)) {
- list_del_init(&m->list_head);
- } else {
- /* put message on sent list */
- ceph_msg_get(m);
- list_move_tail(&m->list_head, &con->out_sent);
- }
+
+ /* put message on sent list */
+ ceph_msg_get(m);
+ list_move_tail(&m->list_head, &con->out_sent);
/*
* only assign outgoing seq # if we haven't sent this message
@@ -1399,6 +1396,7 @@ static void process_ack(struct ceph_connection *con)
break;
dout("got ack for seq %llu type %d at %p\n", seq,
le16_to_cpu(m->hdr.type), m);
+ m->ack_stamp = jiffies;
ceph_msg_remove(m);
}
prepare_read_tag(con);
diff --git a/net/ceph/osd_client.c b/net/ceph/osd_client.c
index 7330c27..ce310ee 100644
--- a/net/ceph/osd_client.c
+++ b/net/ceph/osd_client.c
@@ -1085,9 +1085,15 @@ static void handle_timeout(struct work_struct *work)
req = list_entry(osdc->req_lru.next, struct ceph_osd_request,
r_req_lru_item);
+ /* hasn't been long enough since we sent it? */
if (time_before(jiffies, req->r_stamp + timeout))
break;
+ /* hasn't been long enough since it was acked? */
+ if (req->r_request->ack_stamp == 0 ||
+ time_before(jiffies, req->r_request->ack_stamp + timeout))
+ break;
+
BUG_ON(req == last_req && req->r_stamp == last_stamp);
last_req = req;
last_stamp = req->r_stamp;
--
1.7.0
next prev parent reply other threads:[~2011-07-23 0:05 UTC|newest]
Thread overview: 24+ messages / expand[flat|nested] mbox.gz Atom feed top
2011-07-23 0:02 [PATCH 00/23] Ceph queue for v3.0 Sage Weil
2011-07-23 0:02 ` [PATCH 01/23] ceph: add flags field to file_info Sage Weil
2011-07-23 0:02 ` [PATCH 02/23] ceph: add F_SYNC file flag to force sync (non-O_DIRECT) io Sage Weil
2011-07-23 0:02 ` [PATCH 03/23] ceph: use flag bit for at_end readdir flag Sage Weil
2011-07-23 0:02 ` [PATCH 04/23] ceph: fix snap writeback when racing with writes Sage Weil
2011-07-23 0:02 ` [PATCH 05/23] ceph: only queue capsnap if caps are dirty Sage Weil
2011-07-23 0:02 ` [PATCH 06/23] ceph: report f_bfree based on kb_avail rather than diffing Sage Weil
2011-07-23 0:02 ` Sage Weil [this message]
2011-07-23 0:02 ` [PATCH 08/23] ceph: avoid carrying Fw cap during write into page cache Sage Weil
2011-07-23 0:02 ` [PATCH 09/23] ceph: fix bad parent_inode calc in ceph_lookup_open Sage Weil
2011-07-23 0:02 ` [PATCH 10/23] ceph: only link open operations to directory unsafe list if O_CREAT|O_TRUNC Sage Weil
2011-07-23 0:02 ` [PATCH 11/23] ceph: fix ceph_lookup_open intent usage Sage Weil
2011-07-23 0:02 ` [PATCH 12/23] ceph: ignore lease mask Sage Weil
2011-07-23 0:02 ` [PATCH 13/23] rbd: cancel watch request when releasing the device Sage Weil
2011-07-23 0:02 ` [PATCH 14/23] ceph: set up readahead size when rsize is not passed Sage Weil
2011-07-23 0:02 ` [PATCH 15/23] rbd: set blk_queue request sizes to object size Sage Weil
2011-07-23 0:02 ` [PATCH 16/23] ceph: set dir complete frag after adding capability Sage Weil
2011-07-23 0:02 ` [PATCH 17/23] ceph: handle racing calls to ceph_init_dentry Sage Weil
2011-07-23 0:02 ` [PATCH 18/23] ceph: protect access to d_parent Sage Weil
2011-07-23 0:02 ` [PATCH 19/23] ceph: protect d_parent access in ceph_d_revalidate Sage Weil
2011-07-23 0:02 ` [PATCH 20/23] ceph: avoid d_parent in ceph_dentry_hash; fix ceph_encode_fh() hashing bug Sage Weil
2011-07-23 0:03 ` [PATCH 21/23] ceph: document locking for ceph_set_dentry_offset Sage Weil
2011-07-23 0:03 ` [PATCH 22/23] ceph: explicitly reference rename old_dentry parent dir in request Sage Weil
2011-07-23 0:03 ` [PATCH 23/23] ceph: document unlocked d_parent accesses Sage Weil
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1311379382-9218-8-git-send-email-sage@newdream.net \
--to=sage@newdream.net \
--cc=ceph-devel@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.